How to read large files from Google Drive in gae blobstore

Question

This is the code I'm currently using to read files from Google Drive in the blobstore of the Google engine.

private BlobKey getBlobKey(File f, DriveObject driveObject) throws IOException, MalformedURLException { Drive service = ((GoogleDrive) driveObject).getService(); byte[] buffer = new byte[(int) f.getFileSize().intValue()]; GenericUrl url = new GenericUrl(f.getDownloadUrl()); HttpResponse response = service.getRequestFactory() .buildGetRequest(url).execute(); InputStream is = response.getContent(); FileService fileService = FileServiceFactory.getFileService(); AppEngineFile file = null; boolean lock = true; try { file = fileService.createNewBlobFile("application/zip"); FileWriteChannel writeChannel = fileService.openWriteChannel( file, lock); int len; while ((len = is.read(buffer)) >= 0) { ByteBuffer bb = ByteBuffer.wrap(buffer, 0, len); writeChannel.write(bb); } writeChannel.closeFinally(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } BlobKey bk = fileService.getBlobKey(file); return bk; } 

This is great for small files, but an error appears if the file is larger than ~ 25 MB. Is there a better way to do what I'm trying to do?

This is the error that I see

 Uncaught exception from servlet com.google.appengine.api.urlfetch.ResponseTooLargeException: The response from url https://doc-0o-bs-docs.googleusercontent.com/docs/securesc/97o44sjeam7f23b1flie9m3qqidhrago/97q4akk8bmpub24itqhclo1vakjusu84/1368259200000/14350098165097390874/14350098165097390874/0ByS2XCxCEzq-NUVuajAwWGVoR1U?h=16653014193614665626&e=download&gd=true was too large. at com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:132) at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:43) at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:417) at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:296) at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getResponseCode(URLFetchServiceStreamHandler.java:149) at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37) at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:95) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:980) 

I must add that

 Drive service = ((GoogleDrive) driveObject).getService(); 

returns

 new Drive.Builder(httpTransport, jsonFactory, cred) .build() 

Answer 1

You can load a file from disk in chunks. Use the Range header in your download request as described in the Google Drive Documentation .

 Range: bytes=500-999 

Answer 2

This FilsService and FileWriteChannel api are deprecated in favor of the GCS App Engine client.