Block and File Storage
- Block storage operates at a lower level - the raw storage device level and manages data as a set of numbered, fixed size blocks. Block storage is accessed directly or via a SAN using protocols such as iSCSI and Fiberchannel. Directly attached or network attached, block storage is very closely associated to the server and operating system that is using the storage.
- File storage operates at upper level - at the OS level which manages data as a named hierarchy of files and folders. This type is storage is accessed directly or over a NAS using protocols such as CIFS and NFS. This type of storage is also closely associated with the server and OS that is using the storage.
- Amazon S3 storage is different from file or block storage as it is a cloud object storage. It is not associated to the server or OS that is using it. It functions as an independent storage.Instead of managing data as files or blocks using iSCSI, Fiber channel or NFS, it manages objects using API built on standard http verbs.
Amazon S3 is -
- An object based strorage (to store pics, files, documents - OS cannot be installed on this type of storage).
- Size of files that can be uploaded - 0 bytes to 5TB
- Storage capacity of an S3 bucket is - Unlimited
- It is a reliable storage as data is spread replicated across multiple devices and facilities
- Files are stored in "Buckets" which are like folders on the cloud (web folders). Buckets are flat and have to file hierarchy. Buckets cannot be nested (i.e bucket within a bucket is not allowed).
- Nomenclature for an S3 bucket - https://"bucketname".s3.amazon.aws.com/"name of object". It is best practice to use bucket names that contain your domain name and conform to rules of DNS names. This ensures that your bucket names are your own, can be used in all regions and can host static websites.
- Even though the namespace for an Amazon S3 bucket is global, each Amazon S3 bucket is created in a specific region that you choose. You can create buckets in a region closer to your users to reduce latency, to fulfill data localization and compliance issues or you can create them away from your primary DC facilities for DR and business continuity needs. Data in an Amazon S3 is stored within the region unless you explicitly do cross region replication.
- HTTP code 200 is received after successful upload of an object to S3
- Built for 99.99 availability
- Amazon guarantees 99.9999999% (11 nines) of durability.
- Provides tiered storage
- Provides lifecycle management, versioning, encryption
- It is a simple key value store which is object based. An object has the following attributes -
  1. Name
  2. Value
  3. Version ID
  4. Metadata - Is of two types -
    1. System metadata - created and used by AWS. will include date last modified, object size, md5 digest, http content type (see pic below)
    2. User metadata - is optional and can be specified by the user only at the time when the object is created. e.g - tags. It must begin with "x-amz-meta-"
  5. Sub-resources namely
    - ACLs
    - Torrents
Nearly every AWS service uses S3 directly or indirectly. For example, Amazon S3 serves as a durable target storage for Amazon Kinesis and Amazon elastic mapReduce (Amazon EMR), it can be used as storage for Elastic block storage (EBS) and Amazon RDS snapshots or can be used as data staging and loading mechanism for Amazon RedShift and Amazon DynamoDB. Some common use cases can be -
- Back up and archiving on premise and cloud data
- Content, media and software storage and distribution
- Big data analytics
- Static website hosting
- Cloud native mobile and internet application hosting
- Disaster recovery
Data can be secured in an S3 using/ Ways to control access to S3 buckets -
- ACLs (to secure/restrict access to individual objects or complete buckets) - not used very widely these days
- Bucket policies (to secure/restrict access to entire buckets) - recommended. Bucket policies include an explicit reference to the IAM principal in the policy. This principal may be associated with a different AWS account so Amazon S3 bucket policies allow you to assign cross-account access to Amazon S3 resources. Using a bucket policy, you can specify -
  - Who can access the bucket
  - from which subnet (CIDR block) or IP address
  - during what time of the day
Difference between a normal folder and an S3 bucket is that an S3 bucket has a unique namespace that can be accessed by a URL and resolved by a DNS (so the namespace should be globally unique i.e bucket names are global)
Objects inside a bucket DO NOT inherit the tags for the bucket.
Data consistency model for S3
- Read after write consistency for PUTS of new objects
- Eventual consistency for overwrite PUTS and DELETES (i.e can take some time to propogate). In all cases updates to a single key are atomic, i.e you will either get new data or old data, but never an inconsistent mix of data.
Amazon S3 API supports the following operations -
- Create/Delete a bucket
- Write an object
- Read an object
- delete an object
- List keys in a bucket
Native interface for Amazon S3 is the REST (Representational State Transfer) API that supports the operations mentioned above.
Prefixes and Delimiters - While amazon S3 uses a flat structure in a bucket, it supports the use of prefix and delimiter parameters while listing key names. This feature helps you organize, browse and retrieve objects within a bucket hierarchically. / and \ are used as delimiters. eg. logs/2016/January/Server1.log.

Used with IAM or S3 bucket policies, prefixes and delimiters also allow you to create equivalent of department subdirectories or user home directories within a single bucket, restricting or sharing access to these subdirectories as needed.

	Erstellt von Gaurav Sehgal vor fast 8 Jahre

Notiz am AWS Certified Solutions Architect Associate notes, erstellt von Gaurav Sehgal am 16/11/2018.