How to use Java JSch library to read deleted files line by line?

Question

I'm trying to use Java to read a file line by line, which is very simple (there are several solutions for stackoverflow.com for this), but the caveat here is that the file is on a remote server and it is not possible to get a local copy (this is a massive collection of millions of Amazon reviews in a single .txt file).

JSch comes with two example classes that copy files to and from remote hosts, namely ScpTo and ScpFrom. I am interested in reading a file from a remote host line by line; ScpFrom will try to copy all this into a local file, which will take some age.

Here is a link to ScpFrom: http://www.jcraft.com/jsch/examples/ScpFrom.java.html

I would try to download a piece of code and then modify it to read the deleted file line by line rather than writing to a local file, but most of the code is Greek to me as soon as the author declares an array of bytes and starts reading bytes from the remote file. I admit that this is something that I almost do not understand; BufferedReader provides a much higher level of interface. Essentially, I want to do this: How to read a large text file line by line using Java?

with the exception of using BufferReader, which can also read deleted files line by line, if the host name and user credentials (password, etc.) are specified, i.e. RemoteBufferReader?

This is the test code I wrote; How can I read in the remote file line by line using AO?

public class test2 { static String user = "myusername"; static String host = " user@remotehost "; static String password = "mypasswd"; static String rfile = "/path/to/remote/file/on/remote/host"; public static void main(String[] args) throws FileNotFoundException, IOException, JSchException { JSch jsch=new JSch(); Session session=jsch.getSession(user, host, 22); session.setPassword(password); session.connect(); // exec 'scp -f rfile' remotely String command="scp -f "+rfile; Channel channel=session.openChannel("exec"); ((ChannelExec)channel).setCommand(command); // get I/O streams for remote scp OutputStream out=channel.getOutputStream(); channel.connect() //no idea what to do next } } 

Answer 1

To manage files via ssh, you are better off using sftp than scp or pure ssh. Jsch has built-in sftp support. After you open a session, do this to open the sftp channel:

 ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp"); 

Once you have opened the sftp channel, there are methods for reading the remote file, which allows you to access the contents of the file as an InputStream . You can convert this to Reader if you need to read line by line:

 InputStream stream = sftp.get("/some/file"); try { BufferedReader br = new BufferedReader(new InputStreamReader(stream)); // read from br } finally { stream.close(); } 

Answer 2

The JSch library is a powerful library that can be used to read a file from an SFTP server. Below is the tested code for reading a file from an SFTP location by line

  JSch jsch = new JSch(); Session session = null; try { session = jsch.getSession("user", "127.0.0.1", 22); session.setConfig("StrictHostKeyChecking", "no"); session.setPassword("password"); session.connect(); Channel channel = session.openChannel("sftp"); channel.connect(); ChannelSftp sftpChannel = (ChannelSftp) channel; InputStream stream = sftpChannel.get("/usr/home/testfile.txt"); try { BufferedReader br = new BufferedReader(new InputStreamReader(stream)); String line; while ((line = br.readLine()) != null) { System.out.println(line); } } catch (IOException io) { System.out.println("Exception occurred during reading file from SFTP server due to " + io.getMessage()); io.getMessage(); } catch (Exception e) { System.out.println("Exception occurred during reading file from SFTP server due to " + e.getMessage()); e.getMessage(); } sftpChannel.exit(); session.disconnect(); } catch (JSchException e) { e.printStackTrace(); } catch (SftpException e) { e.printStackTrace(); } 

Please refer to the blog for the entire program.