End-to-End AWS Infrastructure for TradeFlow Analytics

At AMJ Cloud Technologies, we architected a robust AWS 3-tier infrastructure for TradeFlow Analytics, a supply chain analytics startup. Using Terraform, we delivered a scalable, secure platform for real-time insights, enabling clients to optimize global trade flows and reduce costs by 15% on average.

Situation

TradeFlow Analytics needed a scalable and secure cloud infrastructure to support its platform, which processes large datasets for real-time supply chain insights. Manual infrastructure setups led to inconsistent environments, high operational costs, and scalability challenges. The startup required a solution ensuring high availability, secure data handling, and cost efficiency.

Task

Our goal was to design and deploy a 3-tier AWS infrastructure using Terraform, including:

• A VPC with Web, Application, and Database layers across multiple AZs.
• An Application Load Balancer (ALB) for traffic distribution.
• Secure EC2 instances and a Multi-AZ RDS MySQL database.
• A Bastion Host and NAT Gateway for secure access and connectivity.
• Adherence to AWS best practices for security, scalability, and cost optimization.
• Completion within five months.

Action

To meet TradeFlow Analytics’ requirements, we implemented the following actions:

1. Terraform for Infrastructure as Code (IaC):

   • Developed modular Terraform scripts for consistent, repeatable deployments.
   • Executed commands: terraform init, terraform validate, terraform plan, terraform apply, terraform destroy.

2. VPC with 3-Tier Architecture:

   • Created a VPC with CIDR 10.0.0.0/16 across us-east-1a and us-east-1b.

   • Defined public subnets (10.0.1.0/24, 10.0.2.0/24) for ALB and Bastion Host, private subnets (10.0.3.0/24, 10.0.4.0/24 for Application, 10.0.5.0/24, 10.0.6.0/24 for Database).

     resource "aws_vpc" "main" {
       cidr_block = "10.0.0.0/16"
       enable_dns_support   = true
       enable_dns_hostnames = true
       tags = {
         Name = "tradeflow-vpc"
       }
     }
     
     resource "aws_subnet" "public_1" {
       vpc_id            = aws_vpc.main.id
       cidr_block        = "10.0.1.0/24"
       availability_zone = "us-east-1a"
       tags = {
         Name = "public-subnet-1"
       }
     }
     
     resource "aws_subnet" "private_app_1" {
       vpc_id            = aws_vpc.main.id
       cidr_block        = "10.0.3.0/24"
       availability_zone = "us-east-1a"
       tags = {
         Name = "private-app-subnet-1"
       }
     }
     
     resource "aws_subnet" "private_db_1" {
       vpc_id            = aws_vpc.main.id
       cidr_block        = "10.0.5.0/24"
       availability_zone = "us-east-1a"
       tags = {
         Name = "private-db-subnet-1"
       }
     }
     

3. Granular Security Groups:

   • Configured security groups for ALB (HTTP/443 from internet), EC2 (from ALB and Bastion Host), RDS (from EC2 on port 3306), and Bastion Host (SSH from admin IPs).

     resource "aws_security_group" "alb" {
       vpc_id = aws_vpc.main.id
       ingress {
         from_port   = 443
         to_port     = 443
         protocol    = "tcp"
         cidr_blocks = ["0.0.0.0/0"]
       }
       egress {
         from_port   = 0
         to_port     = 0
         protocol    = "-1"
         cidr_blocks = ["0.0.0.0/0"]
       }
       tags = {
         Name = "alb-sg"
       }
     }
     
     resource "aws_security_group" "rds" {
       vpc_id = aws_vpc.main.id
       ingress {
         from_port       = 3306
         to_port         = 3306
         protocol        = "tcp"
         security_groups = [aws_security_group.app.id]
       }
       tags = {
         Name = "rds-sg"
       }
     }
     
     resource "aws_security_group" "bastion" {
       vpc_id = aws_vpc.main.id
       ingress {
         from_port   = 22
         to_port     = 22
         protocol    = "tcp"
         cidr_blocks = ["<admin-ip>/32"]
       }
       tags = {
         Name = "bastion-sg"
       }
     }
     

4. Application Load Balancer and Target Groups:

   • Deployed an ALB in public subnets with Auto Scaling Groups for EC2 instances.

     resource "aws_lb" "main" {
       name               = "tradeflow-alb"
       internal           = false
       load_balancer_type = "application"
       subnets            = [aws_subnet.public_1.id, aws_subnet.public_2.id]
       security_groups    = [aws_security_group.alb.id]
       tags = {
         Name = "tradeflow-alb"
       }
     }
     
     resource "aws_autoscaling_group" "app" {
       vpc_zone_identifier = [aws_subnet.private_app_1.id, aws_subnet.private_app_2.id]
       desired_capacity    = 2
       min_size            = 2
       max_size            = 4
       launch_template {
         id      = aws_launch_template.app.id
         version = "$Latest"
       }
       target_group_arns = [aws_lb_target_group.main.arn]
       tags = {
         Name = "app-asg"
       }
     }
     
     resource "aws_launch_template" "app" {
       name_prefix   = "tradeflow-app-"
       image_id      = "ami-12345678"
       instance_type = "t3.medium"
       user_data     = base64encode(<<-EOF
                       #!/bin/bash
                       yum install -y python3
                       EOF
       )
       security_group_ids = [aws_security_group.app.id]
     }
     

5. EC2 Instances for Application Hosting:

   • Launched EC2 instances in private subnets with Amazon Linux 2 and user data scripts.

     resource "aws_security_group" "app" {
       vpc_id = aws_vpc.main.id
       ingress {
         from_port       = 80
         to_port         = 80
         protocol        = "tcp"
         security_groups = [aws_security_group.alb.id]
       }
       egress {
         from_port   = 0
         to_port     = 0
         protocol    = "-1"
         cidr_blocks = ["0.0.0.0/0"]
       }
       tags = {
         Name = "app-sg"
       }
     }
     

6. RDS MySQL in Multi-AZ Configuration:

   • Deployed a Multi-AZ RDS MySQL instance in private subnets with automated backups.

     resource "aws_db_instance" "main" {
       identifier          = "tradeflow-db"
       engine             = "mysql"
       instance_class     = "db.t3.medium"
       allocated_storage  = 20
       multi_az           = true
       db_subnet_group_name = aws_db_subnet_group.main.name
       vpc_security_group_ids = [aws_security_group.rds.id]
       backup_retention_period = 7
       tags = {
         Name = "tradeflow-db"
       }
     }
     
     resource "aws_db_subnet_group" "main" {
       name       = "tradeflow-db"
       subnet_ids = [aws_subnet.private_db_1.id, aws_subnet.private_db_2.id]
       tags = {
         Name = "tradeflow-db-subnet-group"
       }
     }
     

7. Bastion Host and NAT Gateway:

   • Configured a Bastion Host in a public subnet and a NAT Gateway for private subnet connectivity.

     resource "aws_instance" "bastion" {
       ami           = "ami-12345678"
       instance_type = "t3.micro"
       subnet_id     = aws_subnet.public_1.id
       vpc_security_group_ids = [aws_security_group.bastion.id]
       tags = {
         Name = "tradeflow-bastion"
       }
     }
     
     resource "aws_eip" "bastion" {
       instance = aws_instance.bastion.id
       vpc      = true
     }
     
     resource "aws_nat_gateway" "nat" {
       allocation_id = aws_eip.nat.id
       subnet_id     = aws_subnet.public_1.id
       tags = {
         Name = "tradeflow-nat"
       }
     }
     
     resource "aws_eip" "nat" {
       vpc = true
     }
     

8. Route Tables for Network Traffic:

   • Configured route tables for public and private subnets.

     resource "aws_internet_gateway" "igw" {
       vpc_id = aws_vpc.main.id
       tags = {
         Name = "tradeflow-igw"
       }
     }
     
     resource "aws_route_table" "public" {
       vpc_id = aws_vpc.main.id
       route {
         cidr_block = "0.0.0.0/0"
         gateway_id = aws_internet_gateway.igw.id
       }
       tags = {
         Name = "public-route-table"
       }
     }
     
     resource "aws_route_table" "private" {
       vpc_id = aws_vpc.main.id
       route {
         cidr_block     = "0.0.0.0/0"
         nat_gateway_id = aws_nat_gateway.nat.id
       }
       tags = {
         Name = "private-route-table"
       }
     }
     
     resource "aws_route_table_association" "public_1" {
       subnet_id      = aws_subnet.public_1.id
       route_table_id = aws_route_table.public.id
     }
     
     resource "aws_route_table_association" "private_app_1" {
       subnet_id      = aws_subnet.private_app_1.id
       route_table_id = aws_route_table.private.id
     }
     

9. Monitoring and Logging:

   • Enabled CloudWatch for EC2, ALB, and RDS metrics, with logs and alarms.

10. CI/CD Integration:

    • Integrated Terraform with AWS CodePipeline and CodeBuild for automated updates.

Result

The 3-tier infrastructure delivered transformative outcomes for TradeFlow Analytics:

• 300% Traffic Increase: ALB and Auto Scaling handled surges seamlessly.
• 25% Cost Reduction: IaC and scaling optimized resource usage.
• 99.99% Uptime: Multi-AZ ensured enterprise-grade reliability.
• 20-Minute Provisioning: Terraform reduced setup time from 3 days.
• 99.9% Security Improvement: Eliminated unauthorized access attempts.
• 15% Client Cost Savings: Enabled real-time analytics for supply chain optimization.

This project underscores AMJ Cloud Technologies’ expertise in delivering scalable, secure, and cost-effective cloud solutions for analytics-driven startups.

Technologies Used

• AWS VPC: Isolated network environment.
• AWS EC2: Hosted application logic.
• AWS RDS: Managed MySQL database.
• AWS Application Load Balancer: Distributed traffic.
• AWS Auto Scaling: Scaled dynamically.
• AWS Bastion Host: Secured remote access.
• AWS NAT Gateway: Enabled private subnet connectivity.
• AWS Internet Gateway: Facilitated public access.
• AWS Security Groups: Enforced access controls.
• AWS CloudWatch: Monitored performance and logs.
• AWS CodePipeline: Automated deployments.
• AWS CodeBuild: Built infrastructure updates.
• Terraform: Automated infrastructure provisioning.

Key Takeaways

This project highlights the power of Terraform and 3-tier architectures in supply chain analytics, delivering scalability, security, and efficiency. AMJ Cloud Technologies continues to empower startups with innovative cloud solutions that drive operational excellence and business growth.

Architectural Diagram