The same file gets again and again in spring -ftp, but with different names

Question

I have a spring input channel defined as

<file:inbound-channel-adapter prevent-duplicates="false" id="inpChannel" directory="file:/Users/abhisheksingh/req" auto-startup="true"> <int:poller id="poller" fixed-delay="1000" /> </file:inbound-channel-adapter> <int:service-activator input-channel="inpChannel" ref="inpHandler" /> 

Example file name as TEST.SQQ. SQQ is the file format that the client uses to host files in ftp. However, I see that the same file is uploaded again using the ftp spring adapter with different file names. So the first time is TEST.SQQ. Then next time it's TEST.SQQ-20170204.PQQ, and then next time it's TEST.SQQ-20170204.PQQ.20170304.PQQ. It continues. I have a filter at my end that checks the name of an already processed file. But since the name of the file that was polled is different each time, all these files are selected for processing.

This is my ftp adapter -

 <int-ftp:inbound-channel-adapter id="sqqFtpInbound" channel="ftpChannel" session-factory="sqqFtpClientFactory" auto-create-local-directory="true" delete-remote-files="false" local-filter="acceptAllFileListFilter" local-directory="file:/Users/abhisheksingh/ddrive/everge_ws/sqqReq" auto-startup="true" > <int:poller id="poller" fixed-delay="1000" /> </int-ftp:inbound-channel-adapter> 

Here is my ftp server image -

Here is my local directory image -

I do not understand why the same file gets again and again. I would be grateful for your help!

This is the file list filter code.

 public class TestFileListFilter<F> extends AbstractFileListFilter<F> { private static final Logger log = LoggerFactory.getLogger(EvergeFileListFilter.class); @Override protected boolean accept(F file) { File f = (File) file; if(f.getAbsolutePath().contains(".PQQ")) { String newDir = "/Users/abhisheksingh/ddrive/sample/pqqReq/"; String archiveLocation = "/Users/abhisheksingh/ddrive/sample/pqqArchive/"; String fullName = archiveLocation + f.getName(); log.info("Check if the file has already been processed " + fullName); File fl = new File(fullName); final File dir = new File(archiveLocation); for (final File child : dir.listFiles()) { String archiveName = FilenameUtils.getBaseName(child.getName()); String inputName = FilenameUtils.getBaseName(fl.getName()); log.info("Archive file name is " + archiveName); log.info("Input file name is " + inputName); if(inputName.contains(archiveName)) { log.info("The file is already processed "+inputName); } } if(fl.exists()) { log.error("PQQ file has already been processed."); removeFile(f); return false; }else{ log.info("PQQ File received " + f.getAbsolutePath()); } moveFile(f, newDir); return true; } } 

Answer 1

I think your custom local-filter has some vulnerabilities to rely on a non-existent fact to wait for unique files from remote storage.

You must provide this ability because it is not enabled by default.

To this end, consider adding a filter option to <int-ftp:inbound-channel-adapter> as a reference to AcceptOnceFileListFilter or FtpPersistentAcceptOnceFileListFilter .

We have JIRA on this.

Please confirm that this is definitely a problem for you, and we can review the priority for this ticket and fix it soon.