Skip to content

S3 Performance

Baseline performance

  • Amazon S3 automatically scales to high request rates, latency 100-200ms
  • Your application can achieve at least 3,500 PUT/COPY/POST/DELETE and 5,5,500 GET/HEAD requests per second per prefix in a bucket.
  • There are no limites to the number of prefixes in a bucket.
  • Example (object => prefix)

    • bucket/folder1/sub1/file => /folder1/sub1
    • bucket/folder1/sub2/file => /folder1/sub2
    • bucket/1/file => /1/
    • bucket/2/file => /2/
  • If you spread reads across all four prefixes evenly, you can achieve 22,000 requests per second for GET and HEAD.

KMS Limitation

  • If you use SSE-KMS, you may be impacted by the KMS limits
  • When you upload, it calls the GenerateDataKey KMS API
  • When you download, it calls the Decrypt KMS API
  • Count towards the KMS quote per second (5500, 10000, 30000 req/s based on region)
  • You can request a quote increase using Service Quotas Console

S3 Performance

  • Multi-Part upload:

    • Recommended for files > 100MB, must use for files > 5GB
    • Can help parallelize uploads (speed up transfers)
  • S3 Transfer Acceleration

    • Increase transfer speed by transferring file to an AWS Edge Location which will forward the data to the S3 bucket in the target region
    • Compatible with multi-part upload

S3 Performance - S3 Byte-Range Fetches

  • Parallelize GETs by requesting specific byte ranges
  • Better resilience in case of failures
  • Can be used to speed up downloads
  • Can be used to retrieve only partial data (for example the head of a file)