"java.net.SocketException: Socket is closing" when reading an S3 file

Question

I am trying to read a csv text file from S3 and then send each of its lines to a distributed queue to process them.

When I try to read this, I get a "java.net.SocketException: Socket is closed" exception at different points in the file being read (in different versions). This is the code:

AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials(MyClass.class.getResourceAsStream("myCredentials.properties"))); String bucketName = "myBucket"; String key = "myFile"; S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); InputStream in = object.getObjectContent(); BufferedReader readerS3 = new BufferedReader(new InputStreamReader(in, Charset.forName(fileInfo.getEncoding()))); try { String line = null; while ((line = readerS3.readLine()) != null) { // Sending the line to a distributed queue } } catch (IOException e) { e.printStackTrace(); }finally { try { in.close(); } catch (IOException e) { e.printStackTrace(); } } 

Any idea on how to solve this problem?

UPDATE:

This exception occurs the second time I run the method, if I stop the entire program and run it again, then the first time the method is run, it works fine.

Answer 1

Maybe you should close readerS3 in yours, finally, instead of "in". That is, close an external object that can be covered by wrapped children.

If you close "in" first, then InputStreamReader and BufferedReader are still open, and if they try to do something with the object that they are wrapped, it will already be closed.

Answer 2

As jsn suggested in the comments on the question, the problem is that you need to configure AmazonS3 using ClientConfiguration :

 ClientConfiguration config = new ClientConfiguration(); config.setSocketTimeout(0); AmazonS3 s3 = new AmazonS3Client(/* credentials */, config); 

Answer 3

Thanks, @jsn, your suggestion was my problem.

I have a method that only returns an InputStream, so the AmazonS3 object receives the garbage collection and forces it to close the InputStream.

I made a reference to the AmazonS3 object and fixed my problem.

Answer 4

Closing the input stream or output stream of a socket or any shell of a stream / reader / writer around them closes the socket (and, therefore, the output or input stream, respectively).

Answer 5

no need to repeat s3 initialization.

on your onCreate, make a call to initialize s3Object and s3Client.

then in your asintet just use the call

this way, your s3Client will keep the same data and never close the socket connection with s3 while doing the read. Be smart and learn.

 S3Client s3Client; S3Object s3Object; onCreate() { s3Client = new AmazonS3Client(new BasicSessionCredentials(Constants.ACCESS_KEY_ID, Constants.SECRET_KEY, Constants.TOKEN)); object = new S3Object(); } doinbackground() { object = s3Client.getObject(new GetObjectRequest(Constants.getBucket(), id +".png")); } 

Answer 6

I had the same problem and this question helped me solve the problem: S3 Java client fails a lot with "Premature body end with content length delimiters" or "java.net.SocketException Socket closed"

Basically, I created a new S3Client object for each file, but at some point this object was collecting garbage. So instead, I converted my class to use Singleton:

 private static AmazonS3 s3Client; static { s3Client = new AmazonS3Client(new BasicAWSCredentials(AWSKey, AWSSecretKey)); } public AmazonS3 getService() { return s3Client; }