Skip to content

Latest commit

 

History

History
328 lines (270 loc) · 21.7 KB

overview.md

File metadata and controls

328 lines (270 loc) · 21.7 KB

Table of Contents

Amazon EBS

  • Amazon EBS provides three volume types to best meet the needs of your workloads:

    • General Purpose (SSD)
      • General Purpose (SSD) volumes are suitable for a broad range of workloads, including small to medium-sized databases, development and test environments, and boot volumes.
    • Provisioned IOPS (SSD)
      • These volumes offer storage with consistent and low-latency performance and are designed for I/O intensive applications such as large relational or NoSQL databases.
    • Magnetic
      • for workloads where data are accessed infrequently, and applications where the lowest storage cost is important.

  • Here is a list of important information about EBS Volumes:

    • When you create an EBS volume in an Availability Zone, it is automatically replicated within that zone to prevent data loss due to a failure of any single hardware component.

    • An EBS volume can only be attached to one EC2 instance at a time.

    • After you create a volume, you can attach it to any EC2 instance in the same Availability Zone

    • An EBS volume is off-instance storage that can persist independently from the life of an instance. You can specify not to terminate the EBS volume when you terminate the EC2 instance during instance creation.

    • EBS volumes support live configuration changes while in production which means that you can modify the volume type, volume size, and IOPS capacity without service interruptions.

    • Amazon EBS encryption uses 256-bit Advanced Encryption Standard algorithms (AES-256)

    • EBS Volumes offer 99.999% SLA. This

Cloudwatch

  • Monitoring tool for your AWS resources and applications.
  • Display metrics and create alarms that watch the metrics and send notifications or automatically make changes to the resources you are monitoring when a threshold is breached.

AWS Identity and Access Management

  • You should always associate IAM role to EC2 instances not IAM user for the purpose of accessing other AWS services

  • IAM roles are designed so that your application can securely make API requests from your instances, without requiring you to manage the security credentials that the application use.

    • Instead of creating and distributing your AWS credentials, you can delegate permission to make API requests using IAM roles

  • AWS Organization is a service that allows you to manage multiple AWS accounts easily.

  • AWS IAM Identity Center can be integrated with your corporate directory service for centralized authentication.

    • This means you can sign in to multiple AWS accounts with just one set of credentials.
    • This integration helps to streamline the authentication process and makes it easier for companies to switch between accounts.

  • SCP you can also configure a service control policy (SCP) to manage your AWS accounts.

    • SCPs help you enforce policies across your organization and control the services and features accessible to your other account.
    • prevents unauthorized access

  • Security Token Service (STS) is the service that you can use to create and provide trusted users with temporary security credentials that can control access to your AWS resources.

    • Temporary security credentials work almost identically to the long-term access key credentials that your IAM users can use.

  • AWS Control Tower provides a single location to easily set up your new well-architected multi-account environment and govern your AWS workloads with rules for security,operations, and internal compliance.

    • You can automate the setup of your AWS environment with best-practices blueprints for multi-account structure, identity, access management, and account provisioning workflow.
    • offers "guardrails" for ongoing governance of your AWS environment.

  • You can use an IAM role to specify permissions for users whose identity is federated from your organization or a third-party identity provider (IdP).

    • Federating users with SAML 2.0
      • If your organization already uses an identity provider software package that supports SAML 2.0 (Security Assertion Markup Language 2.0), you can create trust between your organization as an identity provider (IdP) and AWS as the service provider.
      • You can then use SAML to provide your users with federated single-sign on (SSO) to the AWS Management Console or federated access to call AWS API operations.
      • For example: if your company uses Microsoft Active Directory and Active Directory Federation Services, then you can federate using SAML 2.0
    • Federating users by creating a custom identity broker application

      • If your identity store is not compatible with SAML 2.0, then you can build a custom identity broker application to perform a similar function.

      • The broker application authenticates users, requests temporary credentials for users from AWS, and then provides them to the user to access AWS resources.

      • The application verifies that employees are signed into the existing corporate network's identity and authentication system, which might use LDAP, Active Directory, or another system. The identity broker application then obtains temporary security credentials for the employees.

      • To get temporary security credentials, the identity broker application calls either AssumeRole or GetFederationToken to obtain temporary security credentials, depending on how you want to manage the policies for users and when the temporary credentials should expire.

      • The call returns temporary security credentials consisting of an AWS access key ID, a secret access key, and a session token. The identity broker application makes these temporary security credentials available to the internal company application.

      • This scenario has the following attributes:

        • The identity broker application has permissions to access IAM's token service (STS) API to create temporary security credentials.

        • The identity broker application is able to verify that employees are authenticated within the existing authentication system.

        • Users are able to get a temporary URL that gives them access to the AWS Management Console (which is referred to as single sign-on).

RDS

  • Supports Aurora, MySQL, MariaDB, PostgreSQL, Oracle, Microsoft SQL Server.
  • DB Instance
    • For production OLTP use cases, use Multi-AZ deployments for enhanced fault tolerance with Provisioned IOPS storage for fast and predictable performance.
      • You can use PIOPS storage with Read Replicas for MySQL, MariaDB or PostgreSQL.
    • Magnetic
      • Doesn’t allow you to scale storage when using the SQL Server database engine.
        • Doesn’t support elastic volumes.
        • Limited to a maximum size of 3 TiB.
        • Limited to a maximum of 1,000 IOPS.
  • RDS automatically performs a failover in the event of any of the following:
    1. Loss of availability in primary Availability Zone.
    2. Loss of network connectivity to primary.
    3. Compute unit failure on primary.
    4. Storage failure on primary.

Athena

  • An interactive query service that makes it easy to analyze data directly in Amazon S3 and other data sources using SQL.
  • Serverless
  • Has a built-in query editor.
  • highly available and durable
  • integrates with Amazon QuickSight for easy data visualization.
  • retains query history for 45 days.
  • You pay only for the queries that you run. You are charged based on the amount of data scanned by each query.
  • There are 2 types of cost controls:
    • Per-query limit
      • specifies a threshold for the total amount of data scanned per query.
      • Any query running in a workgroup is canceled once it exceeds the specified limit.
      • Only one per-query limit can be created
    • Per-workgroup limit
      • this limits the total amount of data scanned by all queries running within a specific time frame.

Kinesis

  • A fully managed AWS service that you can use to stream live video from devices to the AWS Cloud, or build applications for real-time video processing or batch-oriented video analytics.
  • Amazon Kinesis Data Streams enables real-time processing of streaming big data. It provides ordering of records, as well as the ability to read and/or replay records in the same order to multiple Amazon Kinesis Applications
  • A Kinesis data stream is a set of shards that has a sequence of data records , and each data record has a sequence number that is assigned by Kinesis Data Streams.
    • Kinesis can also easily handle the high volume of messages being sent to the service.
    • durable
    • no missing of messages

DynamoDB

  • How to choose the right partition key ?

    • What is partition key ?
      • DynamoDB supports 2 types of primary keys
        • Partition key: A simple primary key, composed of one attribute known as the partition key.
        • Partition key and Sort key: Referred to as Composite Primary Key, this type of key is composed of two attributes. 1st one is partition key and 2nd one is sort key
    • Why do I need a partition key?
      • DynamoDb stores data as groups of attributes, - Items
      • Items are similar to rows or records in other database systems.
      • DynamoDB stores and retrieves each item based on the primary key value which must be unique
      • DynamoDb uses the partition key's value as an input to an internal hash function. The output from the hash function determines the partition in which the item is stored. Each item's location is determined by the hash value of its partition key.
    • DynamoDB automatically supports access patterns using the throughput you have provisioned, or upto your account limits in the on-demand mode
    • Regardless of the capacity mode you choose if your access pattern exceeds 3000 RCU and 1000 WCU for a single partition key value, your requests might be throttled with a ProvisionedThroughputExceededException error
    • Recommended for Partition keys :
      • Use high-cardinality attributes. These are attributes that have distinct values for each item, like emailid, employee_no, customerid, sessionid, orderid
      • Use composite attributes Try to combine more tha one attribute to form a unique key, if that meets your access pattern
      • Cache the popular items when there is high volume of read traffic using DAX (DynamoDB Accelerator)
      • DAX is fully managed, in-memory cache for DynamoDB that doesn't require developers to manage cache invalidation, data population or cluster management.
      • DAX also is compatible with DynamoDB API calls, so developer can incorporate it more easily into existing applications

Storage Gateway

  • Connects an on-premise software appliance with cloud-based storage to provide seamless integration with data security features between your on-premises IT environment and the AWS storage infrastructure.

  • You can use the service to store data in the AWS cloud for scalable and cost-effective storage that helps maintain

  • It stores files as native S3 objects, archives virtual tapes in Amazon Glacier and stores EBS snapshots generated by the Volume Gateway with Amazon EBS.

Elastic Load Balancer

  • Distributes incoming application or network traffic across multiple targets, such as EC2 instances containers (ECS), Lambda functions and IP addresses in multiple Availability zones

Security Group

  • A security group acts as a virtual firewall for your instance to control inbound and outbound traffic.

Route 53

  • A highly available and scalable Domain Name System (DNS) web service used for domain registration, DNS routing and health checking

AWS Transit Gateway

  • A networking service that uses a hub and spoke model to connect the on-premises data centers and Amazon VPCs to a Single Gateway.
  • With this service, customers only have to create and manage a single connection from the central gateway into each on-premises data center
  • Features:
    • Inter-region peering
      • allows customers to route traffic
      • easy and cost-effective way
    • Multicast
      • allows customers to have fine-grain control on who can consume and produce multicast traffic
    • Automated provisioning
      • customers can automatically identify the Site-to-site VPN connections and on-premises resources with which they are associated using AWS Transit Gateway

Amazon EMR

  • EMR (Elastic MapReduce)

  • A managed cluster that simplifies running big data frameworks like Apache Hadoop and Apache Spark on AWS to process and analyze vast amounts of data.

  • You can process data for analytics purposes and business intelligence workloads using EMR together with Apache Hive and Apache Pig

  • You can use EMR to move large amounts of data in and out of other AWS data stores and databases like S3 and DynamoDB

  • Purchasing options:

    • On-Demand:reliable, predictable, won't be terminated
    • Reserved (min 1 year): cost savings (EMR will automatically use if available)

Auto Scaling

  • Configure automatic scaling for the AWS resources quickly through a scaling plan that uses Dynamic Scaling and Predictive scaling .
  • Useful for :
    • Cyclical traffic such as high use of resources during regular business hours and low use of resources
    • On and Off traffic such as batch processing, testing and periodic analysis
    • Variable traffic patterns, such as software for marketing growth with periods of spiky growth
  • Dynamic Scaling
    • To add and remove capacity for resources to maintain resource at target value
  • Predictive Scaling
    • To forecast the future load demands by analyzing your historical records for a metric
    • Allows schedule scaling by adding or removing capacity and controls maximum capacity
    • Only available for EC2 scaling groups
  • In Auto Scaling, the following statements are correct regarding the cooldown period:
    • It ensures that the Auto Scaling group does not launch or terminate additional EC2 instances before the previous scaling activity takes effect.
    • Its default value is 300 seconds.
    • It is a configurable setting for your Auto Scaling group.

S3

  • Server-side encryption (SSE) is about data encryption at rest-that is, Amazon S3 encrypts your data at the object level as it writes it to disks in its data centers and decrypts it for you when you access it.

    • You have three mutually exclusive options depending on how you choose to manage the encryption keys:

      1.Amazon S3-Managed Keys (SSE-S3)

      1. AWS KMS-Managed Keys (SSE-KMS)

      2. Customer-Provided Keys (SSE-C)

    • S3-Managed Encryption Keys (SSE-S3)

      • Amazon S3 will encrypt each object with a unique key and as an additional safeguard, it encrypts the key itself with a master key that it rotates regularly.

    • SSE-AES S3 handles the key, uses AES-256 algorithm

      • one of the strongest block ciphers available, 256-bit Advanced Encryption Standard (AES-256), to encrypt your data.
  • Client-side Encryption using
    1. AWS KMS-managed customer master key
    2. client-side master key
  • Cross-Account Access You can provide another AWS account access to an object that is stored in an S3 bucket.

    • These are the methods on how to grant cross-account access to objects that are stored in your own Amazon S3 bucket:

      • Resource-based policies and IAM policies
      • Resource-based Access Control List (ACL) and IAM policies

    • Cross-account IAM roles for programmatic and console access to S3 bucket objects

    • Supports failover controls for S3 Multi-Region access points.

  • Requester Pays Buckets
    • Bucket owners pay for all of the Amazon S3 storage and data transfer costs associated with their bucket.

CloudFront

Secrets Manager

  • Helps to manage, retrieve and rotate database credentials, application credentials, OAuth tokens, API keys and other secrets throughout their lifecycles
  • Helps to improve security posture , because you no longer need hard-coded credentials in application source code.
    • Storing the credentials in Secrets Manager helps avoid possible compromise by anyone who can inspect the application or the components.
    • Replace hard-coded credentials with a runtime call to the Secrets Manager service to retrieve credentials with a runtime call to the Secrets Manager service to retrieve credentials dynamically when you need them.

Textract

  • A fully managed document analysis service for detecting and extracting information from scanned documents
  • Return extracted data as key-value pairs (e.g. Name: John Doe)
  • Supports virtually any type of documents
  • Pricing
    • Pay for what you use
    • Charges vary for Detect Document Text API and Analyze Document API with the later being more expensive

RPO and RTO

  • RTO (Recovery Time Object)
    • measures how quickly the application should be available after an outage
  • RPO (Recovery Point Object)
    • refers to how much data loss can the application can tolerate

- Data loss is measured from most recent backup to the point of disaster. Downtime is measured from the point of disaster until fully recovered and available for service.

EC2

Network Firewall

  • AWS Network Firewall supports domain name stateful network traffic inspection
  • Can create allow lists and deny lists with domain names that the stateful rules engine looks for in network traffic
  • AWS Network Firewall is a stateful, managed network firewall and intrusion detection and prevention service for your virtual private cloud (VPC) that you created in Amazon Virtual Private Cloud (Amazon VPC).
    • With Network Firewall, you can filter traffic at the perimeter of your VPC.
    • This includes filtering traffic going to and coming from an internet gateway, NAT gateway, or over VPN or AWS Direct Connect.
  • Network Firewall uses the open source intrusion prevention system (IPS), Suricata, for stateful inspection. Network Firewall supports Suricata compatible rules.

Security

  • The security pillar includes the ability to protect data, systems, and assets to take advantage of cloud technologies to improve security

  • Zero Trust security is a model where application components or microservices are considered discrete from each other and no component or microservice trusts any other.

    Design Principles

    1. Implement a strong identity foundation

    2. Enable traceability

    3. Apply security at all layers:

      • Apply a defense in depth approach with multiple security controls

      • Implementing security to multiple layers (for example, edge of network, VPC, load balancing, every instance and compute service, operating system, application, and code).

    4. Automate security best practices:

    5. Protect data in transit and at rest:

    6. Keep people away from data:

    7. Prepare for security events: