0

A client wants to allow file upload capability on their website for any files up to 1GB in size.

When an HTTP POST request is sent by the site visitor does the server know the file size [content-length] sent at the start of the request or does it have to get the whole file to get the REAL file size?

The server will be apache or nginx on a linux OS with https enabled. Processing language will be php 7.

The files will be directly uploaded to an amazon s3 bucket as requested by the client.

Lid
  • 15
  • 5

3 Answers3

1

Yes, the server knows the size of the request body, via the Content-Length request header.

womble
  • 97,049
  • Can Content-Length be faked by http clients like curl? – Lid May 06 '18 at 01:18
  • @Lid Like everything it receives the server can have no guarantee for this header as well as others that they are correct. – Patrick Mevzek May 06 '18 at 03:05
  • 1
    However, if the content-length header is incorrect, it'll break the transfer, because the server will (should) read the number of octets specified in the header. – womble May 06 '18 at 03:28
1

As partially mentioned by @duskwuff if you're allowing users to upload files directly to S3 then you can restrict the size of the uploaded object via S3 Bucket Policies.

S3 Bucket Policies

A good description of an S3 Bucket policy can be found here:

S3 bucket policies specify what actions are allowed or denied for which principals on the bucket that the bucket policy is attached to (e.g. allow user Alice to PUT but not DELETE objects in the bucket).

There's a good collection of sample bucket policies here. Taking one of these and modifying it, a policy that would allow anyone to upload a file to an S3 bucket, but restrict the file size to 1GB is:

{
  "Version": "2012-10-17",
  "Id": "S3PolicyId1",
  "Statement": [
    {
      "Sid": "AllowPublicUpload",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::examplebucket/*",
      "Condition": {
        ['content-length-range', 1, 1073741824] //min and max
      } 
    } 
  ]
}

This uses the content-length-range condition which is described here as:

The minimum and maximum allowable size for the uploaded content.

This condition supports content-length-range condition match type.

Other Considerations

If you want to limit uploads to specific users, or force users to put uploads in a home folder (i.e. a folder with a name unique to them), then you should check-out Amazon Cognito. With Cognito you'd use IAM policies containing variables that represent a Cognito User Id etc.

The last thing to remember is that an s3:PutObject can overwrite files, so keep this in mind when deciding where to allow users to upload to.

Alex Hague
  • 361
0

The files will be directly uploaded to an amazon s3 bucket as requested by the client.

If that's what you're doing, you can use pre-signed URLs to allow the browser to upload the object directly to S3. The upload doesn't need to pass through your web server at all.

The signature used can include a policy to restrict the size of the uploaded object.

  • so after the user clicks on upload in their browser to post the file/object would go directly to amazon? – Lid May 06 '18 at 13:44
  • @Lid Correct. And once the upload is complete, the user is redirected to a URL that you specify. –  May 06 '18 at 20:00