AWS Secrets Manager vs AWS KMS (Key Management Service)
Both AWS Secrets Manager and AWS Key Management Service (KMS) are used for security and encryption, but they serve different purposes.
Feature | AWS Secrets Manager | AWS KMS (Key Management Service) |
---|---|---|
Purpose | Manages and rotates sensitive secrets (passwords, API keys, database credentials, etc.) | Manages and controls cryptographic keys for encryption and decryption |
Use Case | Storing, retrieving, and auto-rotating secrets securely | Encrypting and decrypting data, signing messages, managing encryption keys |
Encryption | Secrets are encrypted using KMS keys | Provides encryption keys for services like S3, EBS, RDS, etc. |
Automatic Rotation | Supports automatic rotation of secrets (database credentials, API keys) | No automatic rotation for keys (must be rotated manually or via policies) |
Integration | Used with RDS, Redshift, Lambda, API Gateway, etc. | Used with S3, EBS, Lambda, CloudTrail, DynamoDB, etc. |
Cost Model | Charged per stored secret + API calls | Charged per API call + key usage fees |
Access Control | IAM policies + fine-grained resource-based permissions | IAM policies + grants and key policies |
Retrieval | Uses AWS SDK, CLI, or Secrets Manager API to fetch secrets | Used for encryption/decryption API calls |
When to Use Which?
- Use AWS Secrets Manager if you need to securely store and automatically rotate secrets like database passwords, API keys, or OAuth tokens.
- Use AWS KMS if you need to encrypt/decrypt data at rest or control encryption keys for EBS, S3, or RDS.
AWS AppFlow is a fully managed integration service that allows you to securely transfer data between AWS services and SaaS applications like Salesforce, SAP, Slack, and ServiceNow—without writing custom code. It helps automate data ingestion, synchronization, and transformation, making it easier to manage workflows across different platforms.
Key Features of AWS AppFlow
1. No-Code Data Integration
- Connects AWS services (like S3, Redshift, and EventBridge) with SaaS applications (Salesforce, Google Analytics, Zendesk, etc.).
- Supports bidirectional data flow between AWS and third-party services.
2. Secure Data Transfer
- Uses AWS PrivateLink to avoid exposing data to the public internet.
- Supports encryption using AWS KMS.
3. Automatic Data Transformation
- Supports filtering, validation, mapping, and aggregation.
- Applies transformations like merging fields, masking sensitive data, and filtering records.
4. Event-Driven & Scheduled Flows
- Triggers data flows on-demand, on a schedule, or based on events.
- Example: Automatically sync new Salesforce records to an S3 bucket.
5. Scalable & Cost-Effective
- Pay-as-you-go pricing with no upfront costs.
- Scales based on data volume and frequency.
Amazon S3 Glacier Select
S3 Glacier Select allows you to query and retrieve specific data from Amazon S3 Glacier without restoring the entire archive. It supports SQL-based filtering, reducing data retrieval time and cost.
1. How S3 Glacier Select Works
- Instead of retrieving the full archive, you can run SQL queries on CSV, JSON, or Apache Parquet files stored in Glacier.
- Returns only the required data, optimizing cost and retrieval performance.
- Works with Standard, Bulk, and Expedited retrieval tiers.
2. Use Cases
- Extracting specific records from large compressed log files.
- Running partial data analysis without full archive retrieval.
- Reducing storage costs by avoiding unnecessary data transfers.
AWS Systems Manager (SSM) is a centralized management service that helps automate, monitor, and secure AWS and on-premises infrastructure. It simplifies operations, troubleshooting, patching, and security management.
1. Key Features of AWS Systems Manager
a) Session Manager (Secure Remote Access)
- Provides secure shell (SSH) and PowerShell access to EC2 instances without opening ports.
- No need for bastion hosts or VPNs.
- Supports IAM-based authentication and logging to CloudWatch/S3.
b) SSM Run Command (Automate Tasks Remotely)
- Runs commands across multiple EC2 instances at once (e.g., install updates, restart services).
- Works on Windows, Linux, and hybrid environments.
c) Patch Manager (Automated Patching)
- Automatically applies security patches to EC2, on-premises servers, and VMs.
- Supports scheduled patching with maintenance windows.
d) Parameter Store (Secure Configuration Management)
- Stores secrets, API keys, database credentials, and environment variables.
- Supports AWS KMS encryption for sensitive data.
e) Inventory Manager (Track EC2 Configurations)
- Collects software, OS patches, network configurations, and file details.
- Helps with compliance auditing and troubleshooting.
f) State Manager (Ensure System Compliance)
- Enforces desired system configurations using SSM documents (e.g., ensure an app is always running).
- Helps auto-remediate configuration drift.
g) Automation (Workflow Automation & Self-Healing)
- Automates complex operational tasks like instance refresh, backup, and remediation.
- Helps in auto-remediation and operational workflows.
2. AWS Systems Manager Use Cases
Use Case | Feature |
---|---|
Secure EC2 Access (No SSH Keys) | Session Manager |
Automate Patching | Patch Manager |
Run Commands Remotely | Run Command |
Store Secrets & Configs | Parameter Store |
Monitor Inventory & Compliance | Inventory Manager |
Automate Infrastructure Tasks | Automation |
The Visibility Timeout is a key setting in Amazon Simple Queue Service (SQS) that defines the amount of time a message remains hidden from other consumers after it has been received by a consumer. This prevents multiple consumers from processing the same message simultaneously.
1. How Visibility Timeout Works
- Message Received: When a consumer (e.g., EC2 instance, Lambda function) receives a message from the SQS queue, the message becomes invisible to other consumers for the duration of the Visibility Timeout period.
- Processing Time: During this timeout, the message is being processed, and no other consumer can pick it up until the timeout expires.
- Message Deletion or Visibility Reset:
- If processed successfully, the consumer deletes the message from the queue.
- If processing fails or times out, the message becomes visible again after the timeout period ends, allowing another consumer to attempt processing.
2. Visibility Timeout Settings
- Default Timeout: The default visibility timeout for a new SQS queue is 30 seconds.
- Maximum Timeout: The maximum allowed visibility timeout is 12 hours.
- Adjustable Timeout: You can configure the visibility timeout for each queue, and you can also override the default for individual message retrieval operations.
Amazon DynamoDB Point-in-Time Recovery (PITR)
Amazon DynamoDB Point-in-Time Recovery (PITR) allows you to restore a table to a specific point in time within the past 35 days. It provides continuous backup to protect against accidental deletions, data corruption, or unintended write operations.
1. Key Features of PITR
- Continuous Backup: Automatically backs up data every second.
- Restore Any Point in Time: Can restore to any second within the last 35 days.
- No Performance Impact: Does not affect table performance or availability.
- Full Table Restoration: Restores all table attributes, including global secondary indexes (GSIs) and local secondary indexes (LSIs).
- IAM Control: Can be enabled or disabled via IAM permissions.
- Deleted Tables Can Be Restored: Even if a table is deleted, it can be restored within the 35-day window.
Amazon Aurora Database Cloning is a feature that allows you to quickly create an exact copy of an existing Aurora database cluster. The cloned database is created with a low overhead and doesn’t require a full backup or restore process, making it ideal for scenarios like testing, development, and database troubleshooting.
Key Features of Aurora Database Cloning:
- Fast and Cost-Efficient:
- Aurora cloning is fast and uses minimal storage during the initial creation. The clone starts as a copy-on-write snapshot of the original database, which means that storage is only used for changes made to the cloned database.
- No need to wait for a full backup or restore process to complete. You can start working with the clone almost immediately.
- Independent Operations:
- Once created, the clone is fully independent of the source database. You can perform read/write operations on the clone without affecting the original database.
- Changes to the cloned database don’t impact the source, and vice versa.
- Low Overhead:
- Since the clone is based on a copy-on-write mechanism, it starts with shared storage and only uses additional storage for the changes made to the cloned database.
- Same Performance:
- The performance of the clone is identical to that of the source database, as it uses the same underlying Aurora architecture.
- Useful for Testing & Development:
- You can create a clone of the production database to test new features, run queries, or replicate specific use cases without affecting production workloads.
- It’s also useful for development teams to work with realistic data while ensuring that the production database remains untouched.
- Supports Aurora MySQL & Aurora PostgreSQL:
- Aurora database cloning is available for both Aurora MySQL and Aurora PostgreSQL databases.
Summary: How Key Rotation Works in Different S3 Encryption Methods
Encryption Type | Key Managed By | Key Rotation Method |
---|---|---|
SSE-S3 (S3-managed) | AWS S3 | Automatic every year |
SSE-KMS (AWS KMS) | AWS KMS | Automatic (if enabled) or Manual |
SSE-C (Customer-provided) | Customer | Manual (Re-encrypt objects) |