Google Appengine JAVA - Zip many images saved in Blobstore

Question

I developed a simple media library where you can select a set of images and upload them. When a client requests a download, the servlet receives blob keys to use to create the zip file, and then Task is run for the procedure

The task of iterating through the received blob and zip keys of the image into the archive. When the task is completed, an email will be sent to the user with a link to download.

Here is my problem:

FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock); OutputStream blobOutputStream = Channels.newOutputStream(writeChannel); ZipOutputStream zip = new ZipOutputStream(blobOutputStream); 

One channel can only process this number of bytes.

BlobstoreService.MAX_BLOB_FETCH_SIZE

Because of this, I have to open and close the channel every 1 mb of data that I need to write (the same problem for reading, but for reading I used this code and it works). or the write () method throws an exception

Opening and closing a channel using regular outputStream does not present a problem, for example, this code

But to process the zip file I also need to manage

 ZipOutputStream zip = new ZipOutputStream(blobOutputStream); ZipEntry zipEntry = new ZipEntry(image_file_name); zipOut.putNextEntry(zipEntry); // while the image has bytes to write zipOut.write(bytesToWrite); 

After I wrote 1MB of data in ZipEntry, I need to close the channel and reopen it.

So, here is the problem: where I open a new channel, I can’t access the previous zipEntry that I wrote, and then I can not continue to write the next 1 MB image that I am processing.

And after opening a new channel, if I try to write a zipEntry object (without reinitializing), I get a ClosedChannel exception

Here is the SAMPLE code I wrote, I know it doesn't work, but explains what I'm trying to do.

My question is: how (if possible, from the course) can I create a zip file that writes 1 MB in time?

I also have access to other approaches, I need to zip some images into one mailbox and save it in blobstore, if you have other ideas to do this, please tell me

Answer 1

You must create your own stream that can control the channels. When the droplet size limit is reached, your stream closes the current channel and opens a new one.

Example for local files:

 public class ZipChannels { public static void main(String[] args) throws IOException { File dirToZip = new File("target\\dependency"); //create zip-files ChannelOutput out = new ChannelOutput(); ZipOutputStream zip = new ZipOutputStream(out); int b = 0; for(File file: dirToZip.listFiles()) { ZipEntry zipEntry = new ZipEntry(file.getName()); zip.putNextEntry(zipEntry); BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file)); while((b = bis.read()) != -1) { zip.write(b); } bis.close(); zip.closeEntry(); } zip.close(); //merge all into one file for check it BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream("package_all.zip")); for (int i = 0; i < out.getChannelCount(); i++) { BufferedInputStream bis = new BufferedInputStream(new FileInputStream("package_" + i + ".zip")); while((b = bis.read()) != -1) { bos.write(b); } bis.close(); } bos.close(); } public static class ChannelOutput extends OutputStream { private OutputStream channel; private int count = 0; final private int MAX = 1000000; @Override public void write(int b) throws IOException { if(count++ % MAX == 0) { openNewChannel(); } channel.write(b); } protected void openNewChannel() throws IOException { if(channel != null) { channel.close(); } channel = new BufferedOutputStream(new FileOutputStream("package_" + (count / MAX) + ".zip")); } public int getChannelCount() { return count / MAX + 1; } @Override public void close() throws IOException { channel.close(); } @Override public void flush() throws IOException { channel.flush(); } } } 

If you have any questions, please feel free to ask.