Zip files that are present in one FTP location and copy them to another FTP location directly

Question

I want to create a zip file with files located in one ftp location and copy this zip file to another ftp location without saving locally.

I can handle this for small file sizes. It works well for small files of 1 mb in size, etc.

But if the file size is large, like 100 MB, 200 MB, 300 MB, then its error output is like,

java.io.FileNotFoundException: STOR myfile.zip : 550 The process cannot access the file because it is being used by another process. at sun.net.ftp.FtpClient.readReply(FtpClient.java:251) at sun.net.ftp.FtpClient.issueCommand(FtpClient.java:208) at sun.net.ftp.FtpClient.openDataConnection(FtpClient.java:398) at sun.net.ftp.FtpClient.put(FtpClient.java:609) 

My code

  URLConnection urlConnection=null; ZipOutputStream zipOutputStream=null; InputStream inputStream = null; byte[] buf; int ByteRead,ByteWritten=0; ***Destination where file will be zipped*** URL url = new URL("ftp://" + ftpuser+ ":" + ftppass + "@"+ ftppass + "/" + fileNameToStore + ";type=i"); urlConnection=url.openConnection(); OutputStream outputStream = urlConnection.getOutputStream(); zipOutputStream = new ZipOutputStream(outputStream); buf = new byte[size]; for (int i=0; i<li.size(); i++) { try { ***Souce from where file will be read*** URL u= new URL((String)li.get(i)); // this li has values http://xyz.com/folder /myPDF.pdf URLConnection uCon = u.openConnection(); inputStream = uCon.getInputStream(); zipOutputStream.putNextEntry(new ZipEntry((String)li.get(i).substring((int)li.get(i).lastIndexOf("/")+1).trim())); while ((ByteRead = inputStream .read(buf)) != -1) { zipOutputStream.write(buf, 0, ByteRead); ByteWritten += ByteRead; } zipOutputStream.closeEntry(); } catch(Exception e) { e.printStackTrace(); } } if (inputStream != null) { try { inputStream .close(); } catch (Exception e) { e.printStackTrace(); } } if (zipOutputStream != null) { try { zipOutputStream.close(); } catch (Exception e){ e.printStackTrace(); } } 

Can someone let me know how I can avoid this error and process large files

Answer 1

This is not related to file sizes; as the error says, you cannot replace the file because some other process is blocking it.

The reason you see this more often with large files is because they take longer, so the likelihood of simultaneous access is higher.

Thus, the only solution is to make sure that no one is using the file when trying to transfer it. Good luck with that.

Possible other solutions:

• Do not use Windows on the server.
• Transfer the file under a temporary name and rename it when it is complete. Thus, other processes will not see incomplete files. Always a good thing.
• Use rsync instead of reinventing the wheel again.

Answer 2

On the same day, before we had network security, there were FTP servers that allowed third-party transfers. You can use special site commands and directly send the file to another FTP server. These times are long gone. Sigh.

Well, maybe not so long ago. Some FTP servers support a proxy command. Discussed here: http://www.math.iitb.ac.in/resources/manuals/Unix_Unleashed/Vol_1/ch27.htm