GAE Golang - urlfetch timeout?

Question

I'm having problems with urlfetch timeouts in Google App Engine in Go. Perhaps the application does not want to use a longer timeout than about 5 seconds (it ignores a longer timeout and expires after its own time).

My code is:

var TimeoutDuration time.Duration = time.Second*30 func Call(c appengine.Context, address string, allowInvalidServerCertificate bool, method string, id interface{}, params []interface{})(map[string]interface{}, error){ data, err := json.Marshal(map[string]interface{}{ "method": method, "id": id, "params": params, }) if err != nil { return nil, err } req, err:=http.NewRequest("POST", address, strings.NewReader(string(data))) if err!=nil{ return nil, err } tr := &urlfetch.Transport{Context: c, Deadline: TimeoutDuration, AllowInvalidServerCertificate: allowInvalidServerCertificate} resp, err:=tr.RoundTrip(req) if err != nil { return nil, err } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { return nil, err } result := make(map[string]interface{}) err = json.Unmarshal(body, &result) if err != nil { return nil, err } return result, nil } 

No matter what I try to set the TimeoutDuration , the application expires after 5 seconds. How to prevent this? Did I make a mistake in my code?

Answer 1

You need to pass such a length of time (otherwise, it will default to a timeout of 5 seconds):

 tr := &urlfetch.Transport{Context: c, Deadline: time.Duration(30) * time.Second} 

Update January 2, 2016:

With the new GAE golang packages ( google.golang.org/appengine/* ) this has changed. urlfetch no longer gets the duration in transport time.

Now you must set the timeout through the new context package. For example, so you would set 1 minute:

 func someFunc(ctx context.Context) { ctx_with_deadline, _ := context.WithTimeout(ctx, 1*time.Minute) client := &http.Client{ Transport: &oauth2.Transport{ Base: &urlfetch.Transport{Context: ctx_with_deadline}, }, } 

Answer 2

Try the code below:

 // createClient is urlfetch.Client with Deadline func createClient(context appengine.Context, t time.Duration) *http.Client { return &http.Client{ Transport: &urlfetch.Transport{ Context: context, Deadline: t, }, } } 

Here's how to use it.

 // urlfetch client := createClient(c, time.Second*60) 

Courtesy @gosharplite

Answer 3

Looking at the source code of Go appengine:

• http://code.google.com/p/appengine-go/source/browse/appengine/urlfetch/urlfetch.go

and the generated protobuffer code:

• http://code.google.com/p/appengine-go/source/browse/appengine_internal/urlfetch/urlfetch_service.pb.go

It seems that there should be no problems with the duration itself.

My assumption is that the whole application is in appengine timeouts after 5 seconds.

Answer 4

Now this has changed with recent updates in the library. Now the duration of the timeout / delay should be transferred by the context, urlfetch.transport no longer has Deadline fields in it. context.WithTimeout or context.WithDeadline is the way to use it, here is the link https://godoc.org/golang.org/x/net/context#WithTimeout

Answer 5

it worked for me:

 ctx_with_deadline, _ := context.WithTimeout(ctx, 15*time.Second) client := urlfetch.Client(ctx_with_deadline)