Architecting for the Cloud - Design Principles

Page 1

Scalability

Growth should introduce economies of scale, and cost should follow the same dimension that generates business value out of that system. While cloud computing provides virtually unlimited on-demand capacity, your design needs to be able to take advantage of those resources seamlessly

Scale Up/Vertical

Increasing the specifications of an individual resource

Increasing the RAM or CPU to a machine

This way of scaling can eventually each a limit and is not always a cost efficient or highly available approach

Very easy to implement and can be sufficient for many use cases in the short term

Scale Out/Horizontal

Increasing the number of resources

Great way to leverage the elasticity of the cloud

Stateless Applications

A stateless application is an application that does not need knowledge of previous interactions and does not store session information

Given the same input provides the same output to any end user

Distribute Load to Multiple Nodes

Push Model

Using Elastic Load Balancer to distribute a workload

Using Route53 to implement a DNS round robin

This approach doesn't always work well with the elasticity of the cloud because caching DNS resolvers are outside of Route53s control

Pull Model

Tasks that need to be performed or data that needs to be processed can be stored as messages in a queue using SQS. Then multiple compute resources can pull and consume those tasks and process them in a distributed fashion

Useful for asynchronous, event-driven workloads

Stateless Components

​​​​​​​Store as much as you can server side in S3 or a database if that data is required

Only store a unique session identifier in an http cookie

Stateful Components

​​​​​​​Databases are stateful

Real-time Multiplayer gaming is an example of a stateful component

Implement Session Affinity

​​​​​​​Binding a user to a specific resource for the duration of a session

Using sticky sessions feature of an Application Load Balancer

Using client side load balancing is another option

Can also accomplish this using DNS or a discovery API

Distributed Processing - Parallelize where you can

​​​​​​​Dividing a task and its data into many small fragments of work at can be executed in parallel across many compute resources

Implement Distributed Processing

​​​Offline batch jobs: AWS Batch, AWS Glue

Hadoop workloads: Amazon EMR

Real-time processing of streaming data: Amazon Kineses partitions many shards to be consumed by EC2 fleets or Lambda

Page 2

Disposable Resources Instead of Fixed Servers

Fixed resources lead to many bad practices such as manually logging into servers to configure software or fix issues, hardcoding IP addresses, and running test or processing jobs sequentially. Fixed, long running servers also lead to configuration drift. Changes and software patches over time can lead to untested and heterogeneous configurations across different environments.

Think of servers and other components as temporary and disposable resources in the cloud. You can launch as many as you need and use them for only as long as you need while only paying for what you use.

Instantiating Compute Resources: create automated and repeatable processes with the following

Bootstrapping

Scripts that run commands

Good for items that change often or differ between various environments

Used for EC2 Instances or RDS Instances

Golden Images

Good for items that do not change often or that introduce external dependencies

Faster start times and removes dependencies to configuration services or 3rd party repositories

Important in auto-scaled environments

Create an AMI (Amazon Machine Image)

Use case: launching a new test environment you can pre-populate the database by instantiating it from a specific RDS snapshot instead of using lengthy SQL scripts

EC2, RDS, Elastic Block Store (EBS) volumes

Containers

Elastic Beanstalk, Elastic Container Service (ECS), AWS Fargate

Hybrid

Elastic Beanstalk follows the hybrid model

It has preconfigured run time environments initiated from its own AMI but allows you to run bootstrap actions

Infrastructure as Code

CloudFormation

Page 3

Automation

Improve both your system’s stability and the efficiency of your organization. Consider introducing one or more of these types of automation into your application architecture to ensure more resiliency, scalability, and performance.

Serverless Management and Deployment

The operational focus is on the automation of the deployment pipeline.

AWS manages the underlying services, scale, and availability.

CodePipeline, CodeBuild, CodeDeploy support automated deployments

Infrastructure Management and Deployment

Elastic Beanstalk

EC2 Auto Recovery

AWS Systems Manager

Auto Scaling

Alarms and Events

CloudWatch Alarms

CloudWatch Events

AWS Lambda Scheduled Events

AWS WAF Security Automations

Services Not Servers

Increase developer productivity and operational efficiency.

Managed Services

Using a wide variety of AWS services can lessen the load by removing the need to think about and handle capacity, configuration, replication, etc.

Serverless Architecture

Serverless architectures can reduce the operational complexity of running applications.

Stop managing infrastructure and reduce cost

Page 4

Loose Coupling

Systems should be designed in a way that reduces interdependencies—a change or a failure in one component should not cascade to other components. Loose coupling is a crucial element if you want to take advantage of the elasticity of cloud computing where new resources can be launched or terminated at any point in time.

Well Defined Interfaces

Allow the various components to interact with each other only through specific, technology-agnostic interfaces, such as RESTful APIs

This granular design pattern is commonly referred to as a microservices architecture

Amazon API Gateway

Service Discovery

Applications that are deployed as a set of smaller services depend on the ability of those services to interact with each other. Because each of those services can be running across multiple compute resources, there needs to be a way for each service to be addressed.

Implement Service Discovery

Use ELB with EC2. Each load balancer gets its own hostname you can consume a service through a stable endpoint

Combine with DNS and private Route53 zones to be able to abstract the particular load balancers endpoint

Route53 supports auto-naming which lets you automatically create DNS records based on configs

Asynchronous Integration

One component generates events and another consumes those events

This model is suitable for any interaction that does not need an immediate response and where an acknowledgement that a request has been registered will suffice

Tight Coupling

controller A ----call B from A ----> controller B ----call C from B ----> controller C

Loose Coupling

Queue A ----> Controller A ----> Queue B -----> Controller B ----> Queue C ----> Controller C

This approach decouples the two components and introduces additional resiliency.

If a process that is reading messages from the queue fails, messages can still be added to the queue and processed when the system recovers.

It also allows you to protect a less scalable back-end service from front-end spikes

Distributed Systems Best Practices

Graceful Failure in Practice

A request that fails can be retried with an exponential backoff and Jitter strategy, or it can be stored in a queue for later processing.

For front-end interfaces, it might be possible to provide alternative or cached content instead of failing completely when, for example, your database server becomes unavailable

Page 5

Databases - Relational

Normalize data into well defined tabular structures known as tables, which consist of rows and columns. They provide a powerful query language, flexible indexing capabilities, strong integrity controls, and the ability to combine data from multiple tables in a fast and efficient manner.

Scalability

Read Replicas

Separate database instances that are replicated asynchronously.

Subject to replication lag and might be missing some of the latest transactions

For read-heavy applications

Read replicas cannot write - only read

Data Partitioning or Sharding

Data is split across multiple database schemas each running in its own autonomous primary DB instance

Workloads that need to scale their write capacity beyond the constraints of a single DB instance

Sharding introduces complexity - the application’s data access layer needs to have awareness of how data is split so that it can direct queries to the right instance. \

Schema changes must be performed across multiple database schemas

High Availability

Use Amazon RDS Multi-AZ deployments for production databases

Creates a synchronously replicated standby instance in a different availability zone

In case of failure RDS automatically performs a failover to the replicated standby instance

Resilient applications can be designed for Graceful Failure by offering reduced functionality, such as read-only mode by using read replicas.

Anti-Patterns

If your application primarily indexes and queries data with no need for joins or complex transactions consider a NoSQL database instead

Especially if you expect a write throughput beyond the constraints of a single instance

If you have large binary files (audio, video, and image), it will be more efficient to store the actual files in Amazon S3 and only hold the metadata for the files in your database

AWS Services

Amazon RDS - Relational Database Service

Amazon Aurora

Amazons RDS

Designed to deliver much higher throughput

Multi-master capability to enable reads and writes to be scaled across Availability Zones

Supports cross-Region replication

Page 6

Databases - NoSQL

NoSQL databases trade some of the query and transaction capabilities of relational databases for a more flexible data model that seamlessly scales horizontally. NoSQL databases use a variety of data models, including graphs, key-value pairs, and JSON documents, and are widely recognized for ease of development, scalable performance, high availability, and resilience.

Scalability

NoSQL database engines will typically perform data partitioning and replication to scale both the reads and the writes in a horizontal fashion. They do this transparently, and don’t need the data partitioning logic implemented in the data access layer of your application.

High Availability

Use replicas to provide fault tolerance

Anti-Patterns

If your schema cannot be denormalized, and your application requires joins or complex

transactions use RDS

If you have large binary files (audio, video, and image), it will be more efficient to store the actual files in Amazon S3 and only hold the metadata for the files in your database

AWS Services

Amazon DynamoDB

Fast and flexible NoSQL database service for applications that need consistent single-digit, millisecond latency at any scale

Supports document and key-value store models

Manages table partitioning automatically, adding new partitions as your table grows in size or as read-provisioned and write-provisioned capacity changes

Synchronously replicates data across three facilities in an AWS Region

Supports global tables to provide a fully managed, multi-Region, multi-master database that provides fast, local, read and write performance for massively scaled global applications

Amazon DynamoDB Accelerator (DAX)

A managed, highly available, in-memory cache for DynamoDB to leverage significant performance improvements

Page 7

Databases - Data Warehouse

A data warehouse is a specialized type of relational database, which is optimized for analysis and reporting of large amounts of data. It can be used to combine transactional data from disparate sources (such as user behavior in a web application, data from your finance and billing system, or customer relationship management or CRM) to make them available for analysis and decision-making

Scalability

Massive Parallel Processing (MPP)

Columnar data storage

Targeted data compression encoding schemes

High Availability

Deploy production workloads in multi-node clusters, so that data that is written to a node is automatically replicated to other nodes within the cluster.

Anti-Patterns

Don't use a Data Warehouse for:

High concurrency workload that generally involves reading and writing all of the columns for a small number of records at a time

Workloads doing Online Transaction Processing (OLTP) functions

AWS Services

Amazon Redshift

Managed Data Warehouse service designed to operate at less than a 10th of the cost of traditional solutions

Particularly suited to analytic and reporting workloads against very large data sets

Continuously monitors the health of the cluster and automatically re-replicates data from failed drives and replaces nodes as necessary

Amazon Redshift Spectrum

Enables Amazon Redshift SQL queries against exabytes of data in Amazon S3

Page 8

Databases - Search

Search enables datasets to be queried that are not precisely structured. For this reason, applications that require sophisticated search functionality will typically outgrow the capabilities of relational or NoSQL databases. A search service can be used to index and search both structured and free text format and can support functionality that is not available in other databases, such as customizable result ranking, faceting for filtering, synonyms, and stemming.

AWS Services

Amazon CloudSearch

Managed service that requires littler configuration and will scale automatically

Amazon ElasticSearch Service (ES)

Offers an API and gives more control over configuration details

Often used as an analytics engine for log analytics, real-time application monitoring, and click stream analytics

Scalability

Both search services use data partitioning and replication to scale horizontally

CloudSearch does it automatically

High Availability

Both include features to store data redundantly across Availability Zones

Databases - Graph Databases

A graph database uses graph structures for queries. A graph is defined as consisting of edges (relationships), which directly relate to nodes (data entities) in the store. The relationships enable data in the store to be linked together directly, which allows for the fast retrieval of complex hierarchical structures in relational systems.

Use for

Social Networking

Recommendation Engines

Fraud Detection

AWS Services

Amazon Neptune

Purpose built, high-performance graph database optimized for processing graph queries

Highly Available

Read replicas, point-in-time recovery, continuous backup to S3, and replication across Availability Zones

Secure with support for encryption at rest and in transit

Page 9

Managing Increasing Volumes of Data

Data Lake Architecture

A data lake is an architectural approach that allows you to store massive amounts of data in a central location so that it's readily available to be categorized, processed, analyzed, and consumed by diverse groups within your organization

Data is stored as-is and does not need to be converted into predefined schema

No need to know what to ask about your data before storing it

Can create data lakes in S3 and use Athena to query S3 data

Optimize for Cost

When you move your existing architectures into the cloud, you can reduce capital expenses and drive savings as a result of the AWS economies of scale.

Right Sizing

Select the right EC2 instance type for the workload

Select the correct storage options

Reduce costs with continuous monitoring a tagging

Continuously evaluate your solution over time given new services being released and application usage evolves.

Utilize the available AWS services designed to identify cost-savings

Elasticity

Implement Auto-Scaling to scale up and down as needed

Automate turning off non-prod

Use AWS Lambda when possible

Take Advantage of the Variety of Purchasing Options

​​​​​​​Reserved EC2 Instances

Get a discount by paying up front for reserved instances

You should not commit to Reserved Instance purchases before you have sufficiently benchmarked your application in production.

After you have purchased reserved capacity, you can use the Reserved Instance utilization reports to make sure you are still making the most of your reserved capacity.

Spot EC2 Instances

Use spare Amazon EC2 computing capacity.

Since Spot instances are often available at a discount compared to On-Demand pricing, you can significantly reduce the cost of running your applications.

Your Spot instance runs whenever capacity is available and the maximum price per hour for your request exceeds the Spot price

Great for workloads that can tolerate interruption

Page 10

Removing Single Points of Failure

Introduce Redundancy

Have multiple resources for the same task

Standby Mode

When a resource fails, functionality is recovered on a secondary resource with the failover process

The failover typically requires some time before it completes, and during this period the resource remains unavailable

Often used for stateful components such as relational databases

Active Mode

Requests are distributed to multiple redundant compute resources.

When one of them fails, the rest can simply absorb a larger share of the workload.

Compared to standby redundancy, active redundancy can achieve better usage and affect a smaller population when there is a failure.

Detect Failure

Build as much automation as possible in both detecting and reacting to failure

Can use ELB or Route53 configure health checks and route traffic to healthy endpoints

Can use Auto Scaling, EC2 auto-recovery, or AWS Elastic Beanstalk to automatically replace unhealthy nodes

Collect enough logs to understand normal system behavior and then set up alarms for manual intervention or automated response

Design Good Health Checks

In a typical three-tier application, you configure health checks on ELB

Design your health checks with the objective of reliably assessing the health of the back-end nodes

Assess whether the web server can return an HTTP 200 response for some simple request to ensure the server process is working instead of just detecting if the instance is healthy

Might not be good to configure a deep health check, which is a test that depends on other layers of your application to be successful, because of false positives

For example, if your health check also assesses whether the instance can connect to a back-end database, you risk marking all of your web servers as unhealthy when that database node becomes shortly unavailable

A layered approach is often best

Use Route53 to check if database is down and route to static error page until DB is back up

Durable Data Storage

Data Replication

Redundant copies of data

Helps horizontally scale read capacity and increases data durability and availability

Create Backups

Use Versioning

Align each technology you use and their behavior during failover or backup/restore scenarios to your recovery point objective (RPO) and your recovery time objective (RTO).

How much data you expect to lose and how quickly you need to resume operations

For example, the Redis engine for Amazon ElastiCache supports replication with automatic failover, but the Redis engine’s replication is asynchronous. During a failover, it is highly likely that some recent transactions will be lost.

Amazon RDS, with its Multi-AZ feature, is designed to provide synchronous replication to keep data on the standby node up-to-date with the primary.

Synchronous Replication

Only acknowledges a transaction after it has been durably stored in both the primary location and its replicas.

Ideal for protecting the integrity of data from the event of a failure of the primary node.

Scale read capacity for queries that require the most up-to-date data (strong consistency).

Drawback is that the primary node is coupled to the replicas

Can compromise performance and availability especially in topologies that run across unreliable or high-latency network connections

Not recommended to maintain many synchronous replicas

Asynchronous Replication

Decouples the primary node from its replicas at the expense of introducing replication lag

Changes on the primary node are not immediately reflected on its replicas.

Used to horizontally scale the system’s read capacity for queries that can tolerate that replication lag.

Used to increase data durability when some loss of recent transactions can be tolerated during a failover.

For example, you can maintain an asynchronous replica of a database in a separate AWS Region as a disaster recovery solution

Quorum-based Replication

Combines synchronous and asynchronous replication to overcome the challenges of large-scale distributed database systems.

Automated Multi-Data Center Resilience

Have data centers far enough away from each other to protect against catastrophic failure

Use multiple Regions and Availability Zones

Implement Active Redundancy so that the failure is automated and transparent to the users

Fault Isolation and Traditional Horizontal Scaling

If a particular request happens to trigger a bug that causes the system to fail over, then the caller may trigger a cascading failure by repeatedly trying the same request against all instances.

Shuffle Sharding

Group instances into shards

Example: You have 8 instances for your service. You create 4 shards of two instances each (two so there is redundancy within shards). Distribute each customer to a specific shard.

Reduce the impact on customers.

Some customers will still be affected so make the client fault-tolerant by having it try every endpoint in a set of sharded resources until one succeeds

Page 11

Caching

A technique that stores previously calculated data for future use. This technique is used to improve application performance and increase the cost efficiency of an implementation. It can be applied at multiple layers

Application Data Caching

Cache results of I/O intensive database queries, or the outcome of computationally intensive processing.

Improves latency for end users and reduces load on back-end systems

Amazon ElastiCache

A web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud

Supports two open-source, in-memory caching engines: Memcached and Redis.

Amazon DynamoDB Accelerator (DAX)

A fully managed, highly available, in-memory cache for DynamoDB that delivers performance improvements from milliseconds to microseconds, for high throughput.

DAX adds in-memory acceleration to your DynamoDB tables without requiring you to manage cache invalidation, data population, or cluster management.

Edge Caching

​​​​​​​Edge caching allows content to be served by infrastructure that is closer to viewers

Lowers latency and gives the high, sustained data transfer rates necessary to deliver large popular objects to end users at scale.

Use Amazon CloudFront - A CDN with multiple edge locations

Connection optimizations are also applied to avoid internet bottlenecks and fully use available bandwidth between the edge location and the viewer

Same performance benefits for both uploading and downloading

Security

Use AWS Features for Defense in Depth

Build a VPC topology that isolates parts of the infrastructure through the use of subnets, security groups, and routing controls.

AWS WAF can help protect your web applications from SQL injection and other vulnerabilities

Use IAM to define a granular set of policies and assign them to users, groups, and AWS resources for access control

Share Security Responsibility with AWS

AWS is responsible for the security OF the underlying cloud infrastructure and you are responsible for securing the workloads you deploy IN the cloud

When using Amazon RDS and Amazon ElastiCache - security patches are applied automatically to your configuration settings

Reduce Privileged Access

Use CloudWatch to create logs to troubleshoot issues

Use AWS Systems Manager to view and automate actions on groups of resources

Use Roles instead of storing credentials in configs or on servers

Can use Federated access tokens

Security as Code

Golden Environment templates used by CloudFormation

AWS CloudFormation templates can be imported as products into AWS Service Catalog.

This allows you to centrally manage your resources to support consistent governance, security, and compliance requirements, while enabling your users to quickly deploy only the approved IT services they need.

Real-Time Auditing

Implement continuous monitoring and automation of controls to minimize exposure to security risks

AWS Config, Amazon Inspector, and AWS Trusted Advisor continually monitor for compliance or vulnerabilities, giving you a clear overview of which IT resources are in compliance, and which are not.

Implement extensive logging for applications with CloudWatch logs and AWS CloudTrail

Use AWS Lambda, Amazon EMR, Amazon ES, Amazon Athena, or third-party tools from AWS Marketplace to scan log data to detect events such as unused permissions, privileged account overuse, key usage, anomalous logins, policy violations, and system abuse