Ruby FTP Separating Files from Folders

Question

I am trying to bypass FTP and pull out all files recursively.

So far I have been trying to pull out a directory using

ftp.list.each do |entry| if entry.split(/\s+/)[0][0, 1] == "d" out[:dirs] << entry.split.last unless black_dirs.include? entry.split.last else out[:files] << entry.split.last unless black_files.include? entry.split.last end 

But it turns out, if you split the list to the last space, file names and directories with spaces will be erroneous. Need a little help with the logic here.

Answer 1

You can also use regex. I put it together. Please check if it works for you, and also I don’t know what your directory will look different. You must use Ruby 1.9 btw.

 reg = /^(?<type>.{1})(?<mode>\S+)\s+(?<number>\d+)\s+(?<owner>\S+)\s+(?<group>\S+)\s+(?<size>\d+)\s+(?<mod_time>.{12})\s+(?<path>.+)$/ match = entry.match(reg) 

You can access the elements by name, then

match[:type] contains 'd' if it is a directory, space if it is a file.

All other elements are. The most important thing is match[:path] .

Answer 2

You can avoid recursion if you list all files at once

files = ftp.nlst('**/*.*')

Directories are not included in the list, but the full ftp path is still available in the name.

EDIT

I assume that each file name contains point and directory names. Thanks for mentioning @Niklas B.

Answer 3

There are a huge number of FTP servers.

We have clients that use some obscure proprietary Windows-based servers and the list of files they return is completely different from the Linux versions.

So, what did I do for each file / directory entry, I'm trying to change the directory to it, and if that doesn't work, consider it a file :)

The following bullet proof method:

 # Checks if the give file_name is actually a file. def is_ftp_file?(ftp, file_name) ftp.chdir(file_name) ftp.chdir('..') false rescue true end file_names = ftp.nlst.select {|fname| is_ftp_file?(ftp, fname)} 

It works like a charm, but note: if there are many files in the FTP directory, this method takes some time to go through all of them. p>

Answer 4

Assuming the FTP server returns Unix-like file lists, the following code works. At least for me.

 regex = /^d[r|w|x|-]+\s+[0-9]\s+\S+\s+\S+\s+\d+\s+\w+\s+\d+\s+[\d|:]+\s(.+)/ ftp.ls.each do |line| if dir = line.match(regex) puts dir[1] end end 

dir[1] contains the directory name (given that the checked string actually represents the directory).

Answer 5

As @Alex noted, using templates in file names for this is hardly reliable. Directories MAY have periods in their names (for example, .ssh), and lists may be different on different servers.

His method works, but, as he himself points out, it takes too much time. I prefer to use the .size method from Net :: FTP. It returns the file size or throws an error if the file is a directory.

 def item_is_file? (item) ftp = Net::FTP.new(host, username, password) begin if ftp.size(item).is_a? Numeric true end rescue Net::FTPPermError return false end end