HTTPS mocking reaction in Go

I am trying to write tests for a package that makes requests to a web service. I am encountering problems, possibly due to my lack of understanding of TLS.

My test currently looks something like this:

func TestSimple() { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(200) fmt.Fprintf(w, `{ "fake" : "json data here" }`) })) transport := &http.Transport{ Proxy: func(req *http.Request) (*url.URL, error) { return url.Parse(server.URL) }, } // Client is the type in my package that makes requests client := Client{ c: http.Client{Transport: transport}, } client.DoRequest() // ... } 

There is a package variable in my package (I would like it to be a constant ..) for the base address of the web service for the request. This is the https address. The test server created above is a simple HTTP, without TLS.

By default, my test crashes with the error "tls: the first record is not like a TLS handshake."

To make this work, my tests change the package variable to a simple http address instead of https before making a request.

Is there any way around this? Can I make the package variable permanent (https) and set up a http.Transport that "downgrades" to unencrypted HTTP, or uses httptest.NewTLSServer() instead?

(When I try to use NewTLSServer() , I get the error message "http: TLS handshake" from 127.0.0.1:45678: tls: oversized record received with length 20037 ")

+6
source share
2 answers

Most of the behavior in net/http can be mocked, expanded, or modified. Although http.Client is a specific type that implements the semantics of the HTTP client, all of its fields are exported and can be configured.

The Client.Transport field, in particular, can be replaced to force the client to do anything from using user protocols (for example, ftp: // or file: //) to connect directly to local handlers (without generating HTTP protocol bytes or sending anything over the network).

Client functions, such as http.Get , use the exported package variable http.DefaultClient (which you can change), so the code that uses these convenience functions, for example, should not be changed to calling methods in the client’s user variable. Note that while it would be unreasonable to modify global behavior in a public library, it is very useful to do this in applications and tests (including library tests).

http://play.golang.org/p/afljO086iB contains a custom http.RoundTripper that rewrites the request URL so that it is redirected to the locally hosted httptest.Server , and another example that directly passes the http.Handler request, and also a custom http.ResponseWriter implementation to create http.Response . The second approach is not as diligent as the first (it does not fill in as many fields in the answer), but it is more efficient and should be compatible enough to work with most processors and callers.

The above code is also given below:

 package main import ( "fmt" "io" "log" "net/http" "net/http/httptest" "net/url" "os" "path" "strings" ) func Handler(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "hello %s\n", path.Base(r.URL.Path)) } func main() { s := httptest.NewServer(http.HandlerFunc(Handler)) u, err := url.Parse(s.URL) if err != nil { log.Fatalln("failed to parse httptest.Server URL:", err) } http.DefaultClient.Transport = RewriteTransport{URL: u} resp, err := http.Get("https://google.com/path-one") if err != nil { log.Fatalln("failed to send first request:", err) } fmt.Println("[First Response]") resp.Write(os.Stdout) fmt.Print("\n", strings.Repeat("-", 80), "\n\n") http.DefaultClient.Transport = HandlerTransport{http.HandlerFunc(Handler)} resp, err = http.Get("https://google.com/path-two") if err != nil { log.Fatalln("failed to send second request:", err) } fmt.Println("[Second Response]") resp.Write(os.Stdout) } // RewriteTransport is an http.RoundTripper that rewrites requests // using the provided URL Scheme and Host, and its Path as a prefix. // The Opaque field is untouched. // If Transport is nil, http.DefaultTransport is used type RewriteTransport struct { Transport http.RoundTripper URL *url.URL } func (t RewriteTransport) RoundTrip(req *http.Request) (*http.Response, error) { // note that url.URL.ResolveReference doesn't work here // since tu is an absolute url req.URL.Scheme = t.URL.Scheme req.URL.Host = t.URL.Host req.URL.Path = path.Join(t.URL.Path, req.URL.Path) rt := t.Transport if rt == nil { rt = http.DefaultTransport } return rt.RoundTrip(req) } type HandlerTransport struct{ h http.Handler } func (t HandlerTransport) RoundTrip(req *http.Request) (*http.Response, error) { r, w := io.Pipe() resp := &http.Response{ Proto: "HTTP/1.1", ProtoMajor: 1, ProtoMinor: 1, Header: make(http.Header), Body: r, Request: req, } ready := make(chan struct{}) prw := &pipeResponseWriter{r, w, resp, ready} go func() { defer w.Close() thServeHTTP(prw, req) }() <-ready return resp, nil } type pipeResponseWriter struct { r *io.PipeReader w *io.PipeWriter resp *http.Response ready chan<- struct{} } func (w *pipeResponseWriter) Header() http.Header { return w.resp.Header } func (w *pipeResponseWriter) Write(p []byte) (int, error) { if w.ready != nil { w.WriteHeader(http.StatusOK) } return wwWrite(p) } func (w *pipeResponseWriter) WriteHeader(status int) { if w.ready == nil { // already called return } w.resp.StatusCode = status w.resp.Status = fmt.Sprintf("%d %s", status, http.StatusText(status)) close(w.ready) w.ready = nil } 
+13
source

The reason you get the http: TLS handshake error from 127.0.0.1:45678: tls: oversized record received with length 20037 is because https requires a domain name (not an IP address). Domain names are assigned SSL certificates.

Start the httptest server in TLS mode using your own certificates

 cert, err := tls.LoadX509KeyPair("cert.pem", "key.pem") if err != nil { log.Panic("bad server certs: ", err) } certs := []tls.Certificate{cert} server = httptest.NewUnstartedServer(router) server.TLS = &tls.Config{Certificates: certs} server.StartTLS() serverPort = ":" + strings.Split(server.URL, ":")[2] // it always https://127.0.0.1:<port> server.URL = "https://sub.domain.com" + serverPort 

To provide a valid SSL certificate for connectivity, follow these steps:

  • Do not supply certificate and key
  • Supply of self-signed certificate and key
  • Providing a real valid certificate and key

No certificate

If you do not provide your own certificate, the default example.com certificate is loaded by default.

Self-Signed Cert

To create a test certificate, you can use the built-in self-signed certificate generator in $GOROOT/src/crypto/tls/generate_cert.go --host "*.domain.name"

You will receive x509: certificate signed by unknown authority warnings because they are self-signed, so you need your client to skip these warnings by adding the following to the http.Transport field:

  TLSClientConfig: &tls.Config{InsecureSkipVerify: true} 

Valid Valid Certificate

Finally, if you intend to use a real certificate, keep a valid cert and a key where they can be downloaded.


The key point here is to use server.URL = https://sub.domain.com to provide your own domain.

+1
source

Source: https://habr.com/ru/post/980811/


All Articles