I have been struggling with the TransferManager problem for several weeks and I hope to get some feedback,
I have a server application waiting to send jobs. Some of these tasks require downloading to S3. The task contains all the necessary information to download, including access keys.
In my actual download code, I need to create a new S3Client and TransferManager object every time I need to do the download, because I do not know the access keys ahead of time.
After the system processes approximately 1,200 downloads, I received an error message indicating that the JVM would not be able to receive more native threads. I attached the profiler to the application and noticed that the TransferManager object was not cleaned up properly, there were "s3-transfer-manager-worker-1" threads that were idle.
I tried adding a call to TransferManager.shutdownNow() after the download was complete. This cleared the threads. However, I started to get a RejectedExecutionException whenever a new TransferManager was created and an attempt was made to load.
TransferManager contains UploadMonitor and UploadMonitor has a static ScheduledExecutorService . TransferManager.shutdownNow() calls the static method UploadMonitor.shutdownNow() , which calls shutdownNow() in the executing service. This makes it so that I can no longer use any TransferManager objects, even if I try to create a new one.
How can I use multiple transfer objects without thread expiration? This seems like a mistake.
source share