404 when saving to GCS with the client library on the local host dev_appserver

Question

I have a GAE application for python that uses GCS to store files. It works great in production, but doesn’t work completely when running locally on dev_appserver.

Any help achieving all of this will be appreciated.

Thanks.

CODE:

BUCKET = '/mybucket' filename = BUCKET + '/foo.jpg' gcs_file = gcs.open(filename, 'w', content_type=content_type) gcs_file.write(file) gcs_file.close() 

ERROR:

 INFO:root:default: "POST /_ah/gcs/mybucket/foo.jpg HTTP/1.1" 404 52 ERROR 2013-11-12 07:45:05,905 webapp2.py:1552] Expect status [201] from Google Storage. But got status 404. Path: u'/mybucket/foo.jpg'. Request headers: {'x-goog-resumable': 'start', 'x-goog-api-version': '2', 'content-type': u'image/jpeg', 'accept-encoding': 'gzip, *'}. Response headers: {'date': 'Tue, 12 Nov 2013 07:45:05 GMT', 'expires': 'Fri, 01 Jan 1990 00:00:00 GMT', 'content-type': 'text/plain; charset=UTF-8', 'content-length': '52', 'server': 'Development/2.0', 'cache-control': 'no-cache'}. Extra info: None. Traceback (most recent call last): File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__ rv = self.handle_exception(request, response, e) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__ rv = self.router.dispatch(request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher INFO:root:default: "POST /admin/projects HTTP/1.1" 500 3587 return route.handler_adapter(request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__ return handler.dispatch() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch return self.handle_exception(e, self.app.debug) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch return method(*args, **kwargs) File "/Users/me/myapp/admin.py", line 194, in post photo = self.create_image_entity(img_object) File "/Users/me/myapp/admin.py", line 234, in create_image_entity self.write_file_to_gcs(filename=filename, file=img_data, content_type=img_object['mimetype']) File "/Users/me/myapp/admin.py", line 276, in write_file_to_gcs gcs_file = gcs.open(filename, 'w', content_type=content_type) File "/Users/me/myapp/external/cloudstorage/cloudstorage_api.py", line 74, in open return storage_api.StreamingBuffer(api, filename, content_type, options) File "/Users/me/myapp/external/cloudstorage/storage_api.py", line 597, in __init__ errors.check_status(status, [201], path, headers, resp_headers) File "/Users/me/myapp/external/cloudstorage/errors.py", line 108, in check_status raise NotFoundError(msg) NotFoundError: Expect status [201] from Google Storage. But got status 404. Path: u'/mybucket/foo.jpg'. Request headers: {'x-goog-resumable': 'start', 'x-goog-api-version': '2', 'content-type': u'image/jpeg', 'accept-encoding': 'gzip, *'}. Response headers: {'date': 'Tue, 12 Nov 2013 07:45:05 GMT', 'expires': 'Fri, 01 Jan 1990 00:00:00 GMT', 'content-type': 'text/plain; charset=UTF-8', 'content-length': '52', 'server': 'Development/2.0', 'cache-control': 'no-cache'}. Extra info: None. 

Answer 1

I have the same problem. I am using google_appengine_1.8.7 (from zip) and appengine-gcs-client-python-r127.zip.

The endpoint "_ah / gcs" is not processed in dev_appserver.

Actually I think this is due to this change ( https://code.google.com/p/appengine-gcs-client/source/detail?r=125 ) and it seems like GAE 1.8.7. does not match this change.

I am using this appengine-gcs-client-python-r65.zip and now it works great!

Answer 2

I think the problem is that writing to a gcs file requires authorization, when u is executed at the production stage, then you are using an application that is allowed to write to gcs at this stage (as described in the documentation).

API calls in BigQuery and Cloud Storage require authorization. We will use the authorization method for the App Engine service account in this code. To configure the App Engine application to make calls to these APIs:

 Copy the App Engine service account name that we noted above (in the form of [e-mail] ). Visit your API console Team tab, and add the service account name to the project teammate with Can edit permissions. 

while working on the local server you do not have this permission.