Skip to content
Gallery
6. AWS Simple Storage Service (S3)
AWS S3
Storage Classes
Access and Access Policies in Amazon S3
Multipart Upload
Transfer Acceleration
Pre-signed URLs
S3 Versioning
Object Lifecycle Management
S3 Encryption
Event Notifications
S3 Replication
S3 Access Points
Misc
More
Share
Explore
Overview
Purpose: Object storage service to store and retrieve any amount of data from anywhere on the Internet.
Attributes: Durable, highly available, and infinitely scalable data storage at low costs.
Architecture
Distributed Architecture: Objects are redundantly stored across multiple devices and facilities (AZs) in an S3 region.
Key-Based Object Store:
Keys can be any string and can mimic hierarchical attributes.
Use S3 Object Tagging for organizing data across buckets/prefixes.
⁠
image.png
⁠
⁠
⁠
image.png
⁠
⁠
Interfaces
REST Web Services Interface: Works with any Internet-development toolkit.
Access Methods: Recommended to use SDKs and APIs (console uses APIs).
Storage Specifications
Object Size: 0 bytes to 5 TB.
Single PUT Limit: 5 GB.
Multipart Upload: For objects larger than 100 MB.
Data Consistency
Atomic Updates: Ensures complete retrieval of new or old data, never partial or corrupt.
Read After Write Consistency: For PUTS of new objects.
Eventual Consistency: For overwrite PUTS and DELETES.
Access and Security
Unlimited Storage: No limits on the amount of data stored.
Time-Limited Access: Provide temporary access to objects.
Requester Pays: Function shifts the cost to the requester, removing anonymous access.
Event Notifications
Triggers: Alerts for specific actions.
Destinations:
SNS Topics
SQS Queue
Lambda Functions
Configuration: Must configure SNS/SQS/Lambda before setting up S3 notifications.
Cost: No extra charges from S3, but costs apply for SNS, SQS, and Lambda.
Data Structure
Components:
Key: Name of the object.
Value: Data.
Version ID: Versioning information.
Metadata: Additional data about the object.
Access Control Lists: Permissions.
Performance
Request Rates:
At least 3,500 PUT/POST/DELETE requests per second per prefix.
At least 5,500 GET requests per second per prefix.
Scalability: No limits on the number of prefixes in a bucket.
Optimization: Use CloudFront edge locations for read-intensive requests to offload from S3.
Additional Points
Successful Writes: Indicated by HTTP 200 code.
Unlimited Storage Availability: No upper limits to storage capacity.
Organizing Data: Use keys and tags for organization across all buckets.
⁠

Additional capabilities offered by Amazon S3 include:

Capabilities
Column 1
Column 2
1
Additional S3 Capability
How it works
2
Transfer Acceleration
Speeds up data uploads using CloudFront in reverse
3
Requester Pays
The requester rather than the bucket owner pays for requests and data transfer
4
Tags
Assign tags to objects to use in hosting, billing, security etc.
5
Events
Trigger notifications to SNS, SQS, or Lambda when certain events happen in your bucket
6
Static Web Hosting
Simple and massively scalable static web hosting
7
BitTorrent
Use the BitTorrent protocol to retrieve any publicly available object by automatically generating a .torrent file.
8
Storage Class Analysis
Analyzes storage access patterns to help you decide when to transition the right data to the right storage class.
9
Storage Lens
Delivers organization-wide visibility into object storage usage, activity trends, and makes actionable recommendations to improve cost-efficiency and apply data protection best practices.
10
S3 Object Lambda
Add your own code to S3 GET requests to modify and process data as it is returned to an application.
There are no rows in this table
⁠

Amazon S3 Use Cases

1. Backup and Storage
Purpose: Provide data backup and storage services.
Attributes: High durability, persistent storage, non-volatile (retains data when powered off).
2. Application Hosting
Purpose: Deploy, install, and manage web applications.
Attributes: Scalable and highly available infrastructure.
3. Media Hosting
Purpose: Host video, photo, or music uploads and downloads.
Attributes: Redundant, scalable, and highly available infrastructure.
4. Software Delivery
Purpose: Host software applications for customer downloads.
Attributes: Secure and reliable distribution platform.
5. Static Website Hosting
Purpose: Run a static website directly from an S3 bucket.
Attributes: Easy to configure, cost-effective, and reliable.

Key Characteristics of S3
Persistent, Highly Durable Data Store:
Non-volatile storage system.
Retains data even when powered off.
Contrast:
Transient Data Stores: Lose data when powered off.
Ephemeral Data Stores: Temporary storage that is deleted after use or when powered off.
⁠
Description of persistent, transient, and ephemeral data stores and which AWS service to use
Column 1
Column 2
Column 3
1
Storage Type
Description
Examples
2
Persistent data store
Data is durable and sticks around after reboots, restarts, or power cycles
S3, Glacier, EBS, EFS
3
Transient Data Store
Data is just temporarily stored and passed along to another process or persistent store
SQS, SNS
4
Ephemeral Data Store
Data is lost when the system is stopped
EC2 Instance Store, Memcached (Elasticache)
There are no rows in this table
⁠
⁠

Object Tags

S3 object tags are key-value pairs applied to S3 objects which can be created, updated, or deleted at any time during the lifetime of the object.
Allow you to create Identity and Access Management (IAM) policies, setup S3 Lifecycle policies, and customize storage metrics.
Up to ten tags can be added to each S3 object and you can use either the AWS Management Console, the REST API, the AWS CLI, or the AWS SDKs to add object tags.
⁠
⁠
⁠

Amazon S3 CloudWatch Metrics

You can use the AWS Management Console to enable the generation of 1-minute CloudWatch request metrics for your S3 bucket or configure filters for the metrics using a prefix or object tag.
Alternatively, you can call the S3 PUT Bucket Metrics API to enable and configure publication of S3 storage metrics.
CloudWatch Request Metrics will be available in CloudWatch within 15 minutes after they are enabled.
CloudWatch Storage Metrics are enabled by default for all buckets and reported once per day.
The S3 metrics that can be monitored include:
S3 requests.
Bucket storage.
Bucket size.
All requests.
HTTP 4XX/5XX errors.
⁠

Monitoring and Reporting

Amazon
 CloudWatch
⁠
metrics for Amazon S3 can help you understand and improve the performance of applications that use Amazon S3. There are several ways that you can use CloudWatch with Amazon S3.
Daily storage metrics for buckets ‐ Monitor bucket storage using CloudWatch, which collects and processes storage data from Amazon S3 into readable, daily metrics. These storage metrics for Amazon S3 are reported once per day and are provided to all customers at no additional cost.
Request metrics ‐ Monitor Amazon S3 requests to quickly identify and act on operational issues. The metrics are available at 1-minute intervals after some latency to process. These CloudWatch metrics are billed at the same rate as the Amazon CloudWatch custom metrics.
Replication metrics ‐ Monitor the total number of S3 API operations that are pending replication, the total size of objects pending replication, and the maximum replication time to the destination Region. Only replication rules that have S3 Replication Time Control (S3 RTC) enabled will publish replication metrics.
⁠

Logging and Auditing

You can record the actions that are taken by users, roles, or AWS services on Amazon S3 resources and maintain log records for auditing and compliance purposes.
To do this, you can use
Amazon S3 server access logging
⁠
,
AWS CloudTrail logs
⁠
, or a combination of both.
AWS recommend that you use AWS CloudTrail for
 logging bucket and object-level actions
⁠
for your Amazon S3 resources.
This information can be used for auditing.
You must not set the bucket being logged to be the destination for the logs as this creates a logging loop and the bucket will grow exponentially.

S3 Server Access Logging

Purpose: Logs requests made to an S3 bucket for detailed access records.
Scope: Specific to S3 bucket operations.
Content: Logs information about the requester, bucket name, request time, request action, response status, and error code.
Location: Logs are stored in a specified S3 bucket.
Use Case: Primarily used for tracking access to S3 buckets, analyzing traffic patterns, and troubleshooting issues.

AWS CloudTrail Logs

Purpose: Logs API calls across AWS services for auditing and monitoring.
Scope: Broad, covering all supported AWS services, not just S3.
Content: Records details of API calls such as the identity of the caller, time of the call, source IP address, request parameters, and response elements.
Location: Logs are stored in an S3 bucket, and optionally, can be sent to CloudWatch Logs.
Use Case: Used for auditing, compliance, and security monitoring across all AWS resources, providing a comprehensive view of API activity.
⁠
⁠
image.png
⁠
⁠
Additional capabilities offered by Amazon S3 include:
Capabilities
Amazon S3 Use Cases
Description of persistent, transient, and ephemeral data stores and which AWS service to use
Object Tags
Amazon S3 CloudWatch Metrics
Monitoring and Reporting
Logging and Auditing
S3 Server Access Logging
AWS CloudTrail Logs
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.