I use S3.Client.upload_fileobj()with a stream BytesIOas input to upload a file to S3 from a stream. My function should not return until the download is complete, so I need to wait.
From the documentation, there is no obvious way to wait for the translation to complete, but there are some hints that might work:
- Use the callback arg argument to wait until progress reaches 100%. In Javascript, this would be trivial with callbacks or promises, but in Python I'm not sure.
- Use an object that checks if the object exists. But he does this by interrogating every 5 seconds and seems very inefficient. Also, I'm not sure if it will wait for the object to complete.
S3.Waiter - The class is using a method , but I doubt it does what I want.
S3.MultipartUpload.complete() - Cycle , which checks if the object is fully loaded, and if not, sleeps a bit. But how to check if an object is complete?
I searched Google, but no one seems to be asking the same question. In addition, most of the results regarding their related problems use a different API (I think it upload_fileobj()is fairly new).
EDIT
If you find out about , which also accepts a file-like object and locks until the server responds. But will this work in conjunction with threads? I'm not sure how Python multithreading works here. The stream comes initially from , receives the channel through and then needs to be loaded back to S3. Both loading and the subprocess are performed by parallel threads / processes, as I can tell. S3.Client.put_objectS3.Client.download_fileobj()subprocess.Popen()
source
share