AppEngine datastore timeout error reached before reaching 30 second limit

Question

Using AppEngine Go, I constantly see this error when reading about 40k objects from tables:

API 5 error (datastore_v3: TIMEOUT): data store timeout or data is temporarily unavailable.

This occurs within 400 ms of the request.

Is there anything else that could cause this error, besides pushing the limit on the wall clock?

Thanks!

Answer 1

It turns out that each individual API call has a timeout of 5 s. If you need more time, you can wrap your context like this:

ctx := appengine.Timeout(appengine.NewContext(req), 30*time.Second) 

Answer 2

The above answer is deprecated, for newer versions of Go see README here:

appengine.Timeout has been deleted. Use context.WithTimeout instead.

https://github.com/golang/appengine#2-update-code-using-deprecated-removed-or-modified-apis

Making it easier for everyone, here is an example we use:

 timeout := 300 * time.Second c, cFunc := context.WithTimeout(c, timeout) 

The important part is calling the cFunc() method after your context is no longer required to avoid context leaks.

Enjoy.