How to periodically flush ZipOutputStream in java

Question

I am trying to archive a list of files in zip format and then upload it for a user on the fly ...

I have a memory problem while loading a zip of 1gb size

Please help me how can I solve this problem without increasing the jvm heap size. I would like to periodically flush the stream.

I NOW TO REPEAT PERIODICALLY, BUT THIS DOES NOT WORK FOR ME.

Please find my code below:

try{ ServletOutputStream out = response.getOutputStream(); ZipOutputStream zip = new ZipOutputStream(out); response.setContentType("application/octet-stream"); response.addHeader("Content-Disposition", "attachment; filename=\"ResultFiles.zip\""); //adding multiple files to zip ZipUtility.addFileToZip("c:\\a", "print1.txt", zip); ZipUtility.addFileToZip("c:\\a", "print2.txt", zip); ZipUtility.addFileToZip("c:\\a", "print3.txt", zip); ZipUtility.addFileToZip("c:\\a", "print4.txt", zip); zip.flush(); zip.close(); out.close(); } catch (ZipException ex) { System.out.println("zip exception"); } catch (Exception ex) { System.out.println("exception"); ex.printStackTrace(); } 

---

 public class ZipUtility { static public void addFileToZip(String path, String srcFile, ZipOutputStream zip) throws Exception { File file = new File(path + "\\" + srcFile); boolean exists = file.exists(); if (exists) { long fileSize = file.length(); int buffersize = (int) fileSize; byte[] buf = new byte[buffersize]; int len; FileInputStream fin = new FileInputStream(path + "\\" + srcFile); zip.putNextEntry(new ZipEntry(srcFile)); int bytesread = 0, bytesBuffered = 0; while ((bytesread = fin.read(buf)) > -1) { zip.write(buf, 0, bytesread); bytesBuffered += bytesread; if (bytesBuffered > 1024 * 1024) { //flush after 1mb bytesBuffered = 0; zip.flush(); } } zip.closeEntry(); zip.flush(); fin.close(); } } } } 

Answer 1

You want to use encoded encoding to send a file that would otherwise work with the servlet container and find out the size of the data you are trying to send before sending, so that it can set the Content-Length header. Since you are compressing files, you do not know the size of the data being sent. Chunked-Encoding allows you to send response fragments into smaller pieces. Do not set the length of the stream content. You can try using curl or something to see HTTP headers in response to receiving from the server. If it is not broken into pieces, you need to understand this. You will want to learn how to force a servlet container to send an encoded encoding. You may need to add this to the response header so that the servlet container sends it to chunked.

 response.setHeader("Transfer-Encoding", "chunked"); 

Another option is to compress the file into a temporary file using File.createTemp (), and then send its contents. If you are compressing a temporary file first, you can find out how large the file is and set the content length for the servlet.

Answer 2

I think you are digging in the wrong direction. Try replacing the servlet output stream with a file stream and see if there is still a problem. I suspect your web container is trying to collect all the servlet output to calculate the length of the content before sending the http headers.

Answer 3

Another thing ... you do your job inside a catch try block. This leaves a chance that the stream will remain open in your files if you have an exception, and also DOES NOT allow the stream to flush to disk.

Always make sure your closure is in the finally block (at least until you get Java 7 with a try-with-resources block)

 //build the byte buffer for transferring the data from the file //to the zip. final int BUFFER = 2048; byte [] data = new byte[BUFFER]; File zipFile= new File("C\:\\myZip.zip"); BufferedInputStream in = null; ZipOutputStream zipOut = null; try { //create the out stream to send the file to and zip it. //we want it buffered as that is more efficient. FileOutputStream destination = new FileOutputStream(zipFile); zipOut = new ZipOutputStream(new BufferedOutputStream(destination)); zipOut.setMethod(ZipOutputStream.DEFLATED); //create the input stream (buffered) to read in the file so we //can write it to the zip. in = new BufferedInputStream(new FileInputStream(fileToZip), BUFFER); //now "add" the file to the zip (in object speak only). ZipEntry zipEntry = new ZipEntry(fileName); zipOut.putNextEntry(zipEntry); //now actually read from the file and write the file to the zip. int count; while((count = in.read(data, 0, BUFFER)) != -1) { zipOut.write(data, 0, count); } } catch (FileNotFoundException e) { throw e; } catch (IOException e) { throw e; } finally { //whether we succeed or not, close the streams. if(in != null) { try { in.close(); } catch (IOException e) { //note and do nothing. e.printStackTrace(); } } if(zipOut != null) { try { zipOut.close(); } catch (IOException e) { //note and do nothing. e.printStackTrace(); } } } 

Now, if you need to loop, you can just go around the part where you need to add more files. Perhaps go to the file array and loop into it. This code worked for me, zipping the file up.

Answer 4

Do not change buf size based on file size, use fixed size buffer.