Is there a way to use an AWS signed URL to do multi-page loading?

Question

I am writing a client for a service that provides a signed download URL. This works great for small downloads, but is not suitable for large downloads, which can be useful when using multi-page downloads.

In authorization documents, it is proposed to use the provided signature and key to access the key in both the URL and the authorization header. I am trying to use the header approach to start a multi-page download, but I am being denied access. When I use the query string approach, I get the method invalid (POST in this case).

I use boto to create a url. For instance:

import boto

c = boto.connect_s3()
bucket = c.get_bucket('my-bucket')
key = boto.s3.key.Key(bucket, 'my-big-file.gz')
signed_url = key.generate_url(60 * 60, 'POST')  # expires in an hour

Then, trying to start a multi-page download using a signed URL, I do the following:

import requests

url = signed_url + '&uploads'
resp = requests.post(url)

This returns a method that is not allowed.

Is this strategy possible? Is there a better way to provide limited credentials for a specific resource to allow large multi-page downloads?

Update

I managed to find a slightly more specific error that makes me think that this is impossible. Unfortunately, I get a 403 message that the signature does not match the request.

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>SignatureDoesNotMatch</Code>
  <Message>The request signature we calculated does not match the signature you
  provided. Check your key and signing method.</Message>
  <StringToSignBytes>.../StringToSignBytes>
  <RequestId>...</RequestId>
  <HostId>...</HostId>
  <SignatureProvided>...</SignatureProvided>
  <StringToSign>POST


  1402941975
  /my-sandbox/test-mp-upload.txt?uploads</StringToSign>
  <AWSAccessKeyId>...</AWSAccessKeyId>
</Error>

This makes me think that I will not be able to use the signed URL because the signature will not match.

UPDATE

, URL- . , , . , URL- , URL-, , . , ( ) , URL- .

, , , / . , boto, .

Answer 1

, . .

Answer 2

, , :

import json
from uuid import uuid4

import boto3


def get_upload_credentials_for(bucket, key, username):
    arn = 'arn:aws:s3:::%s/%s' % (bucket, key)
    policy = {"Version": "2012-10-17",
              "Statement": [{
                  "Sid": "Stmt1",
                  "Effect": "Allow",
                  "Action": ["s3:PutObject"],
                  "Resource": [arn],
              }]}
    client = boto3.client('sts')
    response = client.get_federation_token(
        Name=username, Policy=json.dumps(policy))
    return response['Credentials']


def client_from_credentials(service, credentials):
    return boto3.client(
        service,
        aws_access_key_id=credentials['AccessKeyId'],
        aws_secret_access_key=credentials['SecretAccessKey'],
        aws_session_token=credentials['SessionToken'],
    )


def example():
    bucket = 'mybucket'
    filename = '/path/to/file'

    key = uuid4().hex
    print(key)

    prefix = 'tmp_upload_'
    username = prefix + key[:32 - len(prefix)]
    print(username)
    assert len(username) <= 32  # required by the AWS API

    credentials = get_upload_credentials_for(bucket, key, username)
    client = client_from_credentials('s3', credentials)
    client.upload_file(filename, bucket, key)
    client.upload_file(filename, bucket, key + 'bob')  # fails


example()