How to create a Google Cloud Storage revolving URL with the Google Client Library for Java in App Engine?

Question

I found the following note that describes exactly what I would like to do:

Note If your users only upload resources (recording) to an access-controlled bucket, you can use the Google Cloud Storage renewable download functionality and avoid signing URLs or having to use a Google account.In a renewable download scenario, your (server) code authenticates and initiates a download in Google Cloud Storage without actually downloading any data. The initialization request returns a load identifier, which can then be used in a client request to load data. A client request does not need to be signed because the boot identifier essentially acts as an authentication token. If you choose this path, be sure to pass the download identifier over HTTPS.

https://cloud.google.com/storage/docs/access-control#Signed-URLs

However, I cannot figure out how to do this with the Google Cloud Storage Library for Java.

https://developers.google.com/resources/api-libraries/documentation/storage/v1/java/latest/

I cannot find a link to renewable files or get the URL for a file anywhere in this API. How can i do this?

Answer 1

This library does not display the URLs that it creates for its caller, which means that you cannot use it for this. If you want to use either signed URLs or the trick mentioned above, you will need to implement it manually.

I would advise going with a solution to a signed URL over a solution where the server initiates a resumed download, if possible. This is a more flexible and easy way to get right, and there are some odd edge cases with the last method you might come across.

Someone wrote a quick example of signing URLs from App Engine some time ago in another question: Storage cloud and secure download strategy on the application engine. GCS acl or blobstore

Answer 2

You can create a URL yourself. Here is an example:

OkHttpClient client = new OkHttpClient(); AppIdentityService appIdentityService = credential.getAppIdentityService(); Collection<String> scopes = credential.getScopes(); String accessToken = appIdentityService.getAccessToken(scopes).getAccessToken(); Request request = new Request.Builder() .url("https://www.googleapis.com/upload/storage/v1/b/" + bucket + "/o?name=" + fileName + "&uploadType=resumable") .post(RequestBody.create(MediaType.parse(mimeType), new byte[0])) .addHeader("X-Upload-Content-Type", mimeType) .addHeader("X-Upload-Content-Length", "" + length) .addHeader("Origin", "http://localhost:8080") .addHeader("Origin", "*") .addHeader("authorization", "Bearer "+accessToken) .build(); Response response = client.newCall(request).execute(); return response.header("location"); 

Answer 3

It took some digging, but I came up with the following that does the right thing. Some official documentation on how to do this would be nice, especially because the endpoint for actually starting a renewable download is different from what the documents call. What comes from using the gsutil tool to sign requests and then to develop what is being done. The insufficiently documented additional information is that the code that POST for this URL to get the renewable session URL must include the header "x-goog-resumable: start" to start the download. From there, everything is the same as the documents for performing renewable downloads in GCS.

 import base64 import datetime import time import urllib from google.appengine.api import app_identity SIGNED_URL_EXPIRATION = datetime.timedelta(days=7) def SignResumableUploadUrl(gcs_resource_path): """Generates a signed resumable upload URL. Note that documentation on this ability is sketchy. The canonical source is derived from running the gsutil program to generate a RESUMABLE URL with the "-m RESUMABLE" argument. Run "gsutil help signurl" for info and the following for an example: gsutil -m RESUMABLE -d 10m keyfile gs://bucket/file/name Note that this generates a URL different from the standard mechanism for deriving a resumable start URL and the initiator needs to add the header: x-goog-resumable:start Args: gcs_resource_path: The path of the GCS resource, including bucket name. Returns: A full signed URL. """ method = "POST" expiration = datetime.datetime.utcnow() + SIGNED_URL_EXPIRATION expiration = int(time.mktime(expiration.timetuple())) signature_string = "\n".join([ method, "", # content md5 "", # content type str(expiration), "x-goog-resumable:start", gcs_resource_path ]) _, signature_bytes = app_identity.sign_blob(signature_string) signature = base64.b64encode(signature_bytes) query_params = { "GoogleAccessId": app_identity.get_service_account_name(), "Expires": str(expiration), "Signature": signature, } return "{endpoint}{resource}?{querystring}".format( endpoint="https://storage.googleapis.com", resource=gcs_resource_path, querystring=urllib.urlencode(query_params))