SSH.NET Upload Multiple files throw an exception asynchronously

Question

I am creating an application that will

• processes the CSV file,
• create a JObject for each entry in the CSV file and save the JSON as a txt file and finally
• upload all these JSON files to the SFTP server

Having examined the free library for the third point, I decided to use SSH.NET .

I created the following class to perform a load operation asynchronously.

 public class JsonFtpClient : IJsonFtpClient { private string _sfptServerIp, _sfptUsername, _sfptPassword; public JsonFtpClient(string sfptServerIp, string sfptUsername, string sfptPassword) { _sfptServerIp = sfptServerIp; _sfptUsername = sfptUsername; _sfptPassword = sfptPassword; } public Task<string> UploadDocumentAsync(string sourceFilePath, string destinationFilePath) { return Task.Run(() => { using (var client = new SftpClient(_sfptServerIp, _sfptUsername, _sfptPassword)) { client.Connect(); using (Stream stream = File.OpenRead(sourceFilePath)) { client.UploadFile(stream, destinationFilePath); } client.Disconnect(); } return (destinationFilePath); }); } } 

The UploadDocumentAsync method returns a TPL Task so that I can call it to asynchronously add multiple files.

I call this UploadDocumentAsync method from the following method, which is in another class:

 private async Task<int> ProcessJsonObjects(List<JObject> jsons) { var uploadTasks = new List<Task>(); foreach (JObject jsonObj in jsons) { var fileName = string.Format("{0}{1}", Guid.NewGuid(), ".txt"); //save the file to a temp location FileHelper.SaveTextIntoFile(AppSettings.ProcessedJsonMainFolder, fileName, jsonObj.ToString()); //call the FTP client class and store the Task in a collection var uploadTask = _ftpClient.UploadDocumentAsync( Path.Combine(AppSettings.ProcessedJsonMainFolder, fileName), string.Format("/Files/{0}", fileName)); uploadTasks.Add(uploadTask); } //wait for all files to be uploaded await Task.WhenAll(uploadTasks); return jsons.Count(); } 

Although the CSV file leads to thousands of JSON entries, I want to load them in at least 50 packages. This ProcessJsonObjects always gets a list of 50 JObject at the time that I want to load the SFTP server asynchronously. But I get the following error in the client.Connect(); UploadDocumentAsync method:

 Session operation has timed out 

Reducing the batch size to 2 works great, but sometimes leads to the following error:

 Client not connected. 

I need to be able to upload many files at once. Or tell me if IIS or SFTP server is needed for this type of operation and what it is.

What am I doing wrong? Your help is greatly appreciated.

Answer 1

The template is completely hacked.

Yours does not wait in the right places, and you do not join in the right places that should be before sending in order to reduce latency.

An unrelated error is because the connection ended before you connected.

Doesn’t work on large ones because you don’t send anything for too long.