A practical, production-inspired cloud security log aggregation and real-time threat detection system built on AWS.
Fox Cloud Sentinel automatically provisions hardened EC2 instances in private subnets, forwards authentication and web access logs to Amazon CloudWatch Logs, analyzes events with an AWS Lambda detection engine, correlates suspicious activity, and sends near-real-time security alerts through Amazon SNS.
Portfolio project: designed to demonstrate practical cloud infrastructure security, centralized logging, event-driven detection, least-privilege IAM, and AWS automation.
flowchart TD
Internet((Internet)) --> ALB
subgraph Public["Public Subnets · AZ-a / AZ-b"]
ALB["Application Load Balancer<br/>Security Group: ALB"]
end
subgraph Private["Private Subnets · AZ-a / AZ-b"]
EC2["EC2 Auto Scaling Group<br/>Nginx + CloudWatch Agent<br/>Security Group: EC2"]
end
ALB -->|"HTTP/HTTPS only"| EC2
EC2 -->|"Authentication + Nginx logs"| CW["Amazon CloudWatch Logs"]
subgraph Detection["Detection Pipeline"]
CW --> SF["CloudWatch Logs<br/>Subscription Filter"]
SF --> Lambda["AWS Lambda<br/>Fox Cloud Sentinel<br/>Threat Detector"]
Lambda --> DynamoDB["DynamoDB<br/>Short-lived Detection State"]
Lambda --> SNS["Amazon SNS<br/>Security Alerts"]
end
SNS --> Notify["Email / Slack / PagerDuty"]
SSM["AWS Systems Manager<br/>Session Manager"] -.->|"Administrative access<br/>No public SSH"| EC2
style Internet fill:#eceff1,stroke:#37474f
style ALB fill:#e3f2fd,stroke:#1565c0
style EC2 fill:#e1f5fe,stroke:#01579b
style CW fill:#f3e5f5,stroke:#6a1b9a
style Lambda fill:#fff3e0,stroke:#e65100
style DynamoDB fill:#fff8e1,stroke:#ff8f00
style SNS fill:#e8f5e9,stroke:#2e7d32
style SSM fill:#fce4ec,stroke:#ad1457
style Notify fill:#f5f5f5,stroke:#424242
Architecture flow: Internet traffic enters only through the Application Load Balancer. EC2 instances run without public IP addresses in private subnets and accept application traffic only from the ALB. The CloudWatch Agent forwards authentication and Nginx logs to CloudWatch Logs, where subscription filters send events to the Lambda threat detector. Lambda performs signature detection and short-lived event correlation before publishing security alerts through SNS. Administrative access is performed through Systems Manager Session Manager rather than public SSH.
Fox Cloud Sentinel follows a defense-in-depth approach:
Internet
│
▼
┌─────────────────────┐
│ Application Load │
│ Balancer │
└──────────┬──────────┘
│
│ HTTP/HTTPS
▼
┌─────────────────────┐
│ Private EC2 │
│ Instances │
│ │
│ Nginx │
│ CloudWatch Agent │
└──────────┬──────────┘
│
│ Logs
▼
┌─────────────────────┐
│ CloudWatch Logs │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Lambda Detection │
│ Engine │
└───────┬───────┬─────┘
│ │
▼ ▼
DynamoDB SNS
Correlation Alerts
AWS IAM roles are used instead of static access keys.
The EC2 instance role provides the permissions required for:
The Lambda execution role is restricted to:
Fox Cloud Sentinel uses the Amazon CloudWatch Agent to continuously forward local logs from the EC2 instances.
For Amazon Linux 2023, the default authentication/security log used by this project is:
/var/log/secure
Nginx logs are collected from:
/var/log/nginx/access.log
/var/log/nginx/error.log
The pipeline is:
EC2
│
│ CloudWatch Agent
▼
CloudWatch Logs
│
│ Subscription Filter
▼
Lambda Threat Detector
CloudWatch Logs becomes the centralized logging layer rather than relying exclusively on logs remaining on individual instances.
The system is designed to improve centralized visibility and reduce dependence on local logs. Local log files still exist temporarily on the EC2 instances and should not be considered inherently tamper-proof.
The Lambda detection engine processes CloudWatch Logs subscription events in near real time.
It performs:
| Rule | Severity | Description |
|---|---|---|
BRUTE_FORCE_SSH |
HIGH | Multiple failed SSH authentication attempts from the same source |
SQL_INJECTION |
HIGH | Common SQL injection signatures |
PATH_TRAVERSAL |
HIGH | Attempts to traverse directories or access sensitive files |
WEB_SCANNER |
MEDIUM | Common automated security scanner signatures |
A single failed authentication event does not immediately generate an alert.
Instead, Fox Cloud Sentinel can correlate events from the same source:
Source: 203.0.113.10
Failed attempt #1
Failed attempt #2
Failed attempt #3
Failed attempt #4
Failed attempt #5
│
▼
BRUTE_FORCE_SSH
Severity: HIGH
│
▼
SNS
The correlation state is maintained temporarily in DynamoDB.
The default configuration is:
Threshold: 5 attempts
Window: 60 seconds
Cooldown: 300 seconds
These values can be configured through Terraform variables.
Detected threats are published to an Amazon SNS topic.
Example alert structure:
{
"rule": "BRUTE_FORCE_SSH",
"severity": "HIGH",
"source_ip": "203.0.113.10",
"log_group": "/fox-cloud-sentinel/auth",
"log_stream": "i-0123456789abcdef0",
"attempt_count": 5,
"message": "Failed password for invalid user sentinel..."
}
SNS can distribute alerts to subscribed endpoints such as:
Slack integration can be added through an appropriate webhook or AWS integration layer.
EC2 instances are configured automatically during launch.
The bootstrap process:
This allows new instances created by the Auto Scaling Group to become operational without manual configuration.
Instead of periodically polling logs:
CloudWatch Logs
│
▼
Subscription Filter
│
▼
Lambda
│
▼
Detection
│
▼
SNS
This provides near-real-time processing while keeping the detection engine separate from the application instances.
DynamoDB is used to maintain lightweight detection state.
This enables rules such as:
N failed authentication attempts
from the same source
within T seconds
without requiring a permanent database of all log events.
The compute layer is managed by an EC2 Auto Scaling Group.
New instances automatically receive:
The ALB automatically distributes traffic across healthy instances.
fox-cloud-sentinel/
│
├── infrastructure/
│ └── terraform/
│ ├── versions.tf
│ ├── variables.tf
│ ├── main.tf
│ ├── iam.tf
│ ├── lambda.tf
│ ├── outputs.tf
│ └── terraform.tfvars.example
│
├── lambda/
│ └── threat_detector.py
│
├── user-data/
│ └── bootstrap.sh
│
├── docs/
│ └── architecture.md
│
├── .gitignore
├── LICENSE
└── README.md
| Component | Technology |
|---|---|
| Cloud | AWS |
| Infrastructure as Code | Terraform |
| Compute | Amazon EC2 |
| Load Balancing | Application Load Balancer |
| Networking | Amazon VPC |
| Logging | Amazon CloudWatch Logs |
| Log Agent | Amazon CloudWatch Agent |
| Detection | AWS Lambda / Python |
| Correlation | Amazon DynamoDB |
| Alerting | Amazon SNS |
| Administration | AWS Systems Manager |
| Web Server | Nginx |
| OS | Amazon Linux 2023 |
You need:
>= 1.6Verify your AWS credentials:
aws sts get-caller-identity
Verify Terraform:
terraform version
Enter the Terraform directory:
cd infrastructure/terraform
Create your variables file:
cp terraform.tfvars.example terraform.tfvars
Edit:
aws_region = "ap-south-1"
project_name = "fox-cloud-sentinel"
instance_type = "t3.micro"
instance_count = 2
log_retention_days = 14
alert_email = "your-email@example.com"
brute_force_threshold = 5
brute_force_window = 60
alert_cooldown = 300
Do not commit terraform.tfvars because it may contain environment-specific configuration.
Initialize Terraform:
terraform init
Review the infrastructure:
terraform plan
Deploy:
terraform apply
Terraform will create the VPC, subnets, route tables, NAT gateway, security groups, ALB, Auto Scaling Group, IAM roles, CloudWatch log groups, Lambda function, DynamoDB table, and SNS topic.
After deployment:
terraform output alb_dns_name
Open the returned address in a browser.
You should see the Fox Cloud Sentinel Nginx test page.
If an email address was provided:
alert_email = "your-email@example.com"
AWS SNS will send a subscription confirmation message.
The subscription must be confirmed before email alerts can be delivered.
Only perform security testing against infrastructure you own or are explicitly authorized to test.
Use Systems Manager Session Manager to access a test instance, then generate controlled authentication events:
for i in {1..5}; do
logger -t sshd "Failed password for invalid user sentinel from 203.0.113.10 port 4242 ssh2"
done
The detector should eventually produce:
Rule: BRUTE_FORCE_SSH
Severity: HIGH
Source: 203.0.113.10
A controlled request such as:
/?id=1%20UNION%20SELECT%201,2
should produce a corresponding Nginx access log event.
The detector normalizes URL-encoded input before applying its signatures.
Expected result:
Rule: SQL_INJECTION
Severity: HIGH
A request containing a traversal sequence such as:
/../../etc/passwd
can be used to verify the PATH_TRAVERSAL detection rule.
Again, perform this only against your own test deployment.
List CloudWatch log groups:
aws logs describe-log-groups \
--log-group-name-prefix "/fox-cloud-sentinel"
Tail authentication logs:
aws logs tail /fox-cloud-sentinel/auth --follow
Tail Nginx access logs:
aws logs tail /fox-cloud-sentinel/nginx-access --follow
Tail Lambda logs:
aws logs tail /aws/lambda/fox-cloud-sentinel-detector --follow
No public SSH access is required.
Use Systems Manager Session Manager:
aws ssm start-session --target INSTANCE_ID
The EC2 instances receive the required Systems Manager IAM policy through their instance profile.
This is preferable to opening:
TCP/22 → 0.0.0.0/0
on the Internet-facing security boundary.
This project demonstrates several important security principles:
Instances remain in private subnets.
The EC2 security group permits HTTP traffic only from the ALB security group.
Conceptually:
Internet
│
▼
ALB SG
│
▼
EC2 SG
rather than:
Internet
│
▼
EC2 SG
The Terraform launch template requires EC2 Instance Metadata Service v2 tokens.
No AWS access keys are embedded in:
Security-relevant logs are forwarded to CloudWatch Logs so that investigation does not depend solely on individual EC2 disks.
DynamoDB records expire through TTL, preventing the detection-state table from becoming an uncontrolled permanent event store.
Fox Cloud Sentinel is a production-inspired portfolio project, not a replacement for a commercial SIEM, IDS, WAF, or managed security platform.
Current limitations include:
Planned improvements include:
This project uses AWS services that can incur charges.
In particular, pay attention to:
Destroy the test environment when it is no longer required:
terraform destroy
Always verify current AWS pricing for your selected region and configuration before deploying.
From the Terraform directory:
terraform destroy
Confirm the resources Terraform proposes to remove.
After destruction, verify that no unrelated resources remain in the AWS account.
Additional architecture and security details are available in:
docs/architecture.md
Fox Cloud Sentinel was designed to demonstrate practical understanding of:
Abhrankan Chakrabarti
Fox Hackerz
Distributed under the MIT License.
See LICENSE for the complete license text.