Storage

In EBS, data is stored in blocks. Collection of blocks are presented as a Volume.

EBS is mountable as a file system as well as bootable for an OS.

EBS is a network drive attached to an instance.

Only one EBS volume can be attached per instance and the volume is usually bound within 1 AZ. An exception is the multi-attach volumes which can be used if the application is smart enough to have only one instance actively writing.

Instances communicate with EBS via a network.

EBS volumes can be detached from one instance and attached to another.

EBS volumes are redundant within an AZ.

EBS Snapshots are used to transfer data of EBS from one AZ to another. The snapshot images are stored in S3.

EBS volumes and EBS snapshots are charged per GB per Month.

Instance store provides temporary block level storage. It is located physically on disks attached to the host computer that your EC2 instance runs on.

Amazon Elastic File System (Amazon EFS) provides serverless, fully elastic file storage so that you can share file data without provisioning or managing storage capacity and performance.

It supports NFSv4.

EFS does not work with Windows based EC2 instances.

EFS can be shared across multiple EC2 instances.

EFS is VPC specific.

EFS is made available by a mount target. Mount target is deployed in a subnet, it is just an IP address.

As EFS is a file system storage, you cannot mount an Operating System on it.

A fully managed service that provides a high performance file storage. It takes care of file servers and volumes, replication of data, patching the file server, addressing hardware issues and backups.

S3 is a service that provides object based storage. It is a global service, however a bucket is created within a region.

S3 cannot be used for file-based or block-based storage.

Objects created in S3 have a key, value and version/metadata.

S3 has a flat file structure, the folders are an illusion.

S3 is designed for high availability.

S3 bucket names must be unique globally across all AWS accounts.

S3 can handle unlimited number of objects.

The maximum size of an S3 bucket is 5 TB.

100 buckets are supported by default. However, this number can be increased to 1000 by requesting a service limit increase.

Multipart upload can break an object into parts before uploading to S3. Multipart upload is recommended once the size goes over 100 MB.

S3 Transfer acceleration uses the Amazon Edge locations for faster uploads and downloads.

S3 Byte Range fetches can fetch a part of a file.

S3 Select and Glacier Select can filter data on server side using sql.

Only empty S3 buckets can be deleted.

S3 Batch Operator is used for bulk operations like encrypting all unencrypted objects. S3 Inventory can be used to filter the object list

Header to select the storage class is x-amz-storage-class

Lifecycle rules can be created to automatically move objects or even to delete objects.

S3 Analytics can be used for Storage Class Analysis.

S3 Versioning is used to maintain a history of objects and helps to recover deleted objects. Versioning is enabled at a bucket level, not at object level. Once versioning is enabled, it cannot be disabled, it can only be suspended. You are charged for all versions of your object.

MFA delete provides extra security to make any change in the versioning state of your bucket. It can be enabled only through CLI.

3 states in versioning are not versioned, versioning enabled, versioning suspended.

S3 Replication is used to create copies of objects. For replication, it is necessary that versioning is enabled in all regions participating in the replication. It is asynchronous.

There are two modes of replication:

Once replication is enabled, only the new objects are replicated. For existing objects there is an option to enable it through S3 Batch Replication.

By default, delete markers are not replicated, but can be enabled. Only delete markers are replicated, not the actual deletes.

S3 ACLs, IAM and Resource Policies

By default, all buckets are locked out. Only the user who created the bucket and the root user has access to the bucket.

IAM Policy determines what a user can do. With IAM policies, you can only grant users within your own AWS account permission to access your Amazon S3 resources.

Resource Policy determines what can be done on a resource.

Both IAM Policy and Resource Policy should pass so that a User has access to a bucket

Access Control Lists (ACLs) give read or write access on buckets or objects to groups of users. With ACLs, you can only grant other AWS accounts (not specific users) access to your Amazon S3 resources.

Requester Pays is an option to bill the requester for the network cost.

Event Notifications are used for auditing information to SQS, SNS and Lambda functions. Another option is to use EventBridge.

S3 Encryption can be enabled for both Client side and Server side.
Server side encryption has the following options:

Glacier Vault Lock and S3 Object Lock are used to prevent deletion of objects.

S3 Object Lambda can be used to change the object just before it is retrieved.

S3 Sharding is a feature that stores data in shards across multiple nodes. S3 manages the sharding and reordering of large files that are uploaded in parts.

Storage Gateway is a hybrid cloud storage service. It allows your on-premise applications to integrate and use the cloud storage services.
It can be used as an extension for your on-premise storage needs.
It can also assist migrations into the cloud.
Other use-cases include backups and disaster recovery.
The Storage Gateway is either a physical machine or a virtual machine that is provisioned on your on-premise environment.

Use cases:

The Storage Gateway has to be installed on the corporate data center. If you don’t have the hardware on-premise, you can order it from Amazon via the Storage Gateway Hardware Appliance.