Amazon S3 - Your proposed download is less than the minimum size

Question

I am having problems when I want to upload an image to my amazon s3 bucket.

I am trying to download a 238 kb jpg image. I put try / catch in my code to check what the error is. I always get this error:

Your proposed load is less than the minimum size

I also tried this with images with 1 MB and 2 MB, with the same error ....

Here is my code:

<?php // Include the SDK using the Composer autoloader require 'AWSSDKforPHP/aws.phar'; use Aws\S3\S3Client; use Aws\Common\Enum\Size; $bucket = 'mybucketname'; $keyname = 'images'; $filename = 'thelinktomyimage'; // Instantiate the S3 client with your AWS credentials and desired AWS region $client = S3Client::factory(array( 'key' => 'key', 'secret' => 'secretkey', )); // Create a new multipart upload and get the upload ID. $response = $client->createMultipartUpload(array( 'Bucket' => $bucket, 'Key' => $keyname )); $uploadId = $response['UploadId']; // 3. Upload the file in parts. $file = fopen($filename, 'r'); $parts = array(); $partNumber = 1; while (!feof($file)) { $result = $client->uploadPart(array( 'Bucket' => $bucket, 'Key' => $keyname, 'UploadId' => $uploadId, 'PartNumber' => $partNumber, 'Body' => fread($file, 5 * 1024 * 1024), )); $parts[] = array( 'PartNumber' => $partNumber++, 'ETag' => $result['ETag'], ); } // Complete multipart upload. try{ $result = $client->completeMultipartUpload(array( 'Bucket' => $bucket, 'Key' => $keyname, 'UploadId' => $uploadId, 'Parts' => $parts, )); $url = $result['Location']; fclose($file); } catch(Exception $e){ var_dump($e->getMessage()); } 

(I changed the bucket, keys and image link.)
Has anyone had this before? Searching the web did not help me much.
Also, a search to change the minimum download size did not provide much help.

UPDATE :
When I tried this with a local image (changed the file name), it will work! How can I do this work with an image that is online? Now I save it in my temporary file and then load it from there. But is there no way to save it directly without saving it locally?

Answer 1

The minimum size for multi-part download is 5 MB (1). You probably want to use "normal" downloads rather than multi-part downloads.

(1) http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html

Answer 2

This error occurs when one of the parts is less than 5 MB in size and is not the last part (the last part can be of any size). fread() can return a line shorter than the specified size, so you need to keep calling fread() until you have at least 5 MB of data (or you have reached the end of the file) before downloading this part.

So your third step:

 // 3. Upload the file in parts. $file = fopen($filename, 'r'); $parts = array(); $partNumber = 1; while (!feof($file)) { // Get at least 5MB or reach end-of-file $data = ''; $minSize = 5 * 1024 * 1024; while (!feof($file) && strlen($data) < $minSize) { $data .= fread($file, $minSize - strlen($data)); } $result = $client->uploadPart(array( 'Bucket' => $bucket, 'Key' => $keyname, 'UploadId' => $uploadId, 'PartNumber' => $partNumber, 'Body' => $data, // <= send our buffered part )); $parts[] = array( 'PartNumber' => $partNumber++, 'ETag' => $result['ETag'], ); }