Databases

Databases

1. RDS

   • Amazon provisions an EC2 & EBS volume type and size. Support for Read Replicas and Multi AZ. Security through IAM, Security groups, KMS, SSL in transit
     • Backup / snapshot / Point in time r
     • Monitoring cloudwatch
     • UC: Store relational datasets, perform SQL queires, Transaction inserts, update , delete is available
     • O: Small downtime when failover/maintence.
       • S: AWS responsible OS, You responsible KMS, Security Groups, IAM policies, Authorizing DB, using SSL.
         • R: Multi AZ feature, failover in case of failures
           • P: Depends on ec2 instance type, Volume type, ability to Read Replicas, Storage auto-scaling & manual scaling of instances
             • C: Pay per hour based on provisioned EC2 & EBS

2. Athena

   • Fuly serverless database with SQL capabilities. Used to query data in s3. Pay per query. Output results back to s3. Secured through IAM
     • O: Serverless
       • S: IAM + S3 Security
         • R: Managed service , uses Presto engine, Highly available
           • P: Queries scale based on data size
             • C: Pay per query / Per TB of data scanned, serverless
   • UC: One time SQL queries, serverless queries on s3, log analytics
     • RS: Faster queries / joins / aggregations thanks to indexes

3. ElastiCache

   • Managed Redis / Memcashed. In-memory data store, sub millisecond latency, Provision EC2 instance type, Support for Clustering and Multi AZ, Read Replicas . Managed and scheduled maintenance. Cloud watch monitoring.
     • Security through IAM, Security Groups, Redis Auth , KMS
       • Backup / Snapshot / Point in time restore feature
     • UC: Key Value store, Frequent reads, less writes, cache results for DB queries, store session data for websites, cannot use SQL.
     • o: Same as RDS
       • S: AWS/OS , YOU/ KMS,Security Group, IAM policies, Users, SSL
         • R: Clustering, Multi AZ
           • P: Sub-millisecond performance, in memory, read replicas for sharding, very popular cache option
             • C: Pay per hour based on EC2 and storage use

4. DynamoDB

   • Managed NoSQL database. Serverlerss, provisioned capacity, auto scaling , on demand capacity. Can replace Elasticache as a key/value store. Highly available, Multi AZ by default, Read and Writes are decoupled, DAX for read cache
     • Reads can be eventually consistent or strongly consistent. Security authentication and autorization is done through IAM. DynamoDB streams to integrate with AWS Lambda. Backup/Restore feature, Global Table feature.
       • Monitoring through Cloud Watch
         • Can only query on primary key, sort key, indexes
       • UC: Serverless applications development, distributed serverless cache, doesn't have SQL query language available, has transactions capability from nov 2018
     • O: No operations needed, autoscaling capability, serverless
       • S:Full security through IAM policies, KMS encryption, SSL in flight
         • R: Multi AZ, Backups
           • P: Single digit millisecond performance, DAX for caching reads, performance doesn't degrade if your application scales
             • C: Pay per provisioned capacity and storage usage, pay per exact usage with autoscaling

5. S3

   • S3 Key / Value store for objects. Great for big objects because more latency. Serverless, scales infinitely, max object size is 5TB. Strong consistency. Tiers: S3 Standard, S3 IA, S3 One Zone IA, Glacier for backups
     • Versioning, Encryption, Cross Region Replication
       • Security: IAM, Bucket policies, ACL
         • Encryption: SS3-3, SS3-KMS, SSE-C, client side encryption, SSL Transit
     • UC: Static files, key value store for big files, website hosting
     • O: No operations needed
       • S: IAM, Bucket Policies, ACL, Encryption, SSL
         • R: 99.999999999% durability, Multi AZ, CRR
           • P: Scales thousand read/write per second, transfer acceleration, multi part big files
             • C: Pay per storage usage, network cost, requests number

6. Aurora

   • Data held in 6 replicas, across 3 AZ. Auto healing capability. Multi AZ, Auto Scaling Read Replicas, Read Replicas can be Global. Aurora database can be Global for DR/Latency.
     • Autoscaling storage from 10gb to 128 TB
       • UC: Same as RDS, less maintenance, more flexibility + performance
     • Same security monitoring and maintenance as RDS
     • Aurora serverless - Unpredictable/ Intermittent
     • O: Less operations, auto scaling storage
       • S: AWS responsible for OS, You responsible for KMS,Security Groups, IAM policies, Authorizing users DB,Using SSL
         • R: Multi AZ, Highly available, More than RDS, Aurora Serverless, Multi-Master option
           • P: 5x Perfromance due to architectural optimizations. 15 Read replicas
             • C: Pay per hour based on EC2 * Storage use

7. Redshift

   • Based on PostgreSQL, not used for OLTP. 10 x better for performance than other data warehouses, scale to PBs of data. Columnar storage of data. Massively Parallel Query Execution. Pay as you go based on the instances provisioned. Has a SQL interface for perfomring the queires.
     • OLAP- Online analytical processing
       • Redshift = Analytics / BI / Data Warehouse
     • Columnar storage of data. Massively parallel Query Execution. Pay as you go based on the instances provisioned. Has a SQL interface for performing the queries. BI tools such as AWS Quicksight or Tableau integrate with it
       • Data is loaded from s3, DynamoDB, DMS. From 1 node to 128 nodes. Leader node for query planning, results aggregation. Compute node: for performing the queries, send results to leader. Redshift Spectrum: perform queries directly against s3. Backup & Restore Security VPC / IAM / KMS, monitoring. Redshift Enhanced VPC Routing: Copy / Unload
         • Redshift has no "Multi-AZ". Snapshots are point-in-time backups of a cluster; stored internally in s3. Snapshots incremental. Automated: Every 8 hours, 5gb, schedule. Manual: Snapshot is retained until you delete it
           • Can configure Amazon Redshift to copy snapshots of a cluster to another AWS Region
         • Redshift Spectrum - Query data that is already in S3 without loading it. Must have a Redshift cluster available to start the query. The query is then submitted to thousands of Redshift Spectrum nodes.
           • O: Like RDS
             • S: IAM, VPC, KMS, SSL
               • R: Auto healing features, cross-region snapshot copy
                 • P: 10 x performance vs other data warehousing, compression
                   • Cost: Pay per node provisioned 1/10th of the cost vs other warehouses

8. Glue

   • Managed extract, transform, and load. Useful to prepare and transform data for analytics. Fully serverless service. Glue Data Catalog: catalog of datasets. Metadata. Information about your data.

9. Neptune

   • Fully managed graph database. High relationship data. Social networking: Users friends with Users, replied to comment on post. Knowledge graphs (Wiki) . Highly available across 3 AZ, with up to 15 read replicas
     • O: Similar to RDS
       • S: IAM, VPC, KMS, SSL + IAM Authentication
         • R: Multi-AZ, clustering
           • R: Performance: best suited for graphs, clustering to improve performance
             • C: Pay per node provisioned
     • Graphs :)

10. Elastic Search

    • Search any field, even partially matches
      • Common to use ElasticSearch as a complement to another database. ElasticSearch also has some usage for Big data applications. You can provision a cluster of instances. Built-in integrations: Amazon Kinesis Data Firehose, AWS IoT, and Amazon CloudWatch Logs for data ingestion.
        • Security through Cognito * IAM , KMS encryption
          • O: Similar to RDS
            • S: Cognito, IAM, VPC , K,S , SSL
              • R: Multi AZ, Clustering
                • P: Based on ElasticSearch project
                  • C: Pay per node provisioned
        • Elastic Search = Search / Indexing