Skip to content
@NTI-Django-React-Project

Gig Router DevOps Project

NTI Project Team

AWS Terraform Kubernetes ArgoCD Jenkins Docker


A comprehensive DevOps practice project demonstrating the full lifecycle of building, securing, deploying, and operating a cloud-native application on AWS EKS — from infrastructure provisioning with Terraform, through multi-stage CI/CD pipelines, to GitOps-driven Kubernetes deployments with observability, logging, and intelligent automation.


📑 Table of Contents


🏗️ Architecture Overview

%%{init: {'theme': 'dark', 'themeVariables': {'fontFamily': 'monospace'}}}%%
flowchart TB
    subgraph DEV["🧑‍💻 Developer Workflow"]
        direction LR
        CODE[Push Code] --> GH[GitHub]
    end

    subgraph INFRA["☁️ AWS Infrastructure · Terraform"]
        direction TB
        VPC["🌐 VPC · Subnets · SGs"]
        EKS["⎈ EKS Cluster"]
        RDS["🗄️ RDS PostgreSQL"]
        ECR["📦 ECR Registry"]
        SM["🔑 Secrets Manager"]
        IAM["👤 IAM · IRSA"]
        VPC --> EKS
        VPC --> RDS
        IAM --> EKS
    end

    subgraph CI["🔄 Jenkins CI/CD"]
        direction TB
        TEST["✅ Unit Tests\n+ Coverage"]
        SONAR["🔍 SonarQube\nQuality Gate"]
        OWASP["🛡️ OWASP\nDep-Check"]
        KANIKO["🐳 Kaniko\nBuild"]
        TRIVY["🔒 Trivy\nImage Scan"]
        NEXUS["📚 Nexus\nArtifacts"]
        TEST --> SONAR --> OWASP --> KANIKO --> TRIVY
        KANIKO --> NEXUS
    end

    subgraph CD["🚀 GitOps · Argo CD"]
        direction TB
        ARGO["Argo CD\nApp-of-Apps"]
        SYNC["Auto Sync\n+ Self-Heal"]
        ARGO --> SYNC
    end

    subgraph K8S["⎈ Kubernetes Workloads"]
        direction TB
        FE["⚛️ Frontend\nReact · Nginx"]
        BE["🐍 Backend\nDjango · Gunicorn"]
        REDIS["⚡ Redis\nCache"]
        MC["💾 Memcached"]
        FE -->|"/api proxy"| BE
        BE --> REDIS
    end

    subgraph OBS["📊 Observability"]
        direction LR
        PROM["Prometheus"]
        GRAF["Grafana"]
        EFK_S["EFK Stack"]
        PROM --> GRAF
    end

    subgraph NOTIFY["🤖 Automation"]
        N8N["n8n Workflows\nSmart Alerts"]
    end

    GH -->|Webhook| CI
    TRIVY -->|Push Image| ECR
    CI -->|Notify| N8N
    GH -->|Sync| CD
    SYNC --> K8S
    K8S --> OBS
    BE -->|SSL| RDS
    K8S ---|Secrets via IRSA| SM
    EKS --- K8S

    style DEV fill:#0d1117,stroke:#58a6ff,stroke-width:2px,color:#c9d1d9
    style INFRA fill:#0d1117,stroke:#f0883e,stroke-width:2px,color:#c9d1d9
    style CI fill:#0d1117,stroke:#d24939,stroke-width:2px,color:#c9d1d9
    style CD fill:#0d1117,stroke:#ef7b4d,stroke-width:2px,color:#c9d1d9
    style K8S fill:#0d1117,stroke:#326ce5,stroke-width:2px,color:#c9d1d9
    style OBS fill:#0d1117,stroke:#e6a817,stroke-width:2px,color:#c9d1d9
    style NOTIFY fill:#0d1117,stroke:#ff6d5a,stroke-width:2px,color:#c9d1d9
Loading

📦 Repository Map

Repository Purpose Key Technologies
Terraform-Infra AWS infrastructure as code — EKS, RDS, VPC, IAM, ECR, Secrets Manager Terraform AWS
App-Back-End Django REST backend with full Jenkins CI/CD pipeline Django Jenkins
App-Front-End React frontend served via Nginx with Jenkins CI/CD React Nginx
k8s-manifests Kubernetes manifests — Argo CD apps, deployments, observability, logging K8s Argo
n8n Workflow automation — smart CI/CD notifications via webhooks n8n

☁️ Infrastructure Layer — Terraform

Fully modular IaC provisioning all AWS resources needed to run a production Kubernetes platform.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    subgraph TF_BACKEND["🗃️ Remote State"]
        S3["S3 Bucket"]
        DDB["DynamoDB Lock"]
    end

    subgraph MODULES["📦 Reusable Modules"]
        direction TB
        M_VPC["security-group\nsecurity-group-rule"]
        M_EKS["eks\neks-network\neks-namespace\neks-irsa"]
        M_IAM["iam-role · iam-policy\niam-group · iam-user"]
        M_DATA["rds\necr\nsecret-manager"]
        M_COMPUTE["ec2"]
    end

    subgraph STACKS["🏗️ Infrastructure Stacks"]
        EKS_STACK["EKS Infra\nmain.tf · 28 KB"]
        JENKINS_STACK["Stand-Alone\nJenkins Server"]
    end

    TF_BACKEND --> STACKS
    MODULES --> STACKS

    style TF_BACKEND fill:#0d1117,stroke:#844fba,stroke-width:2px,color:#c9d1d9
    style MODULES fill:#0d1117,stroke:#844fba,stroke-width:2px,color:#c9d1d9
    style STACKS fill:#0d1117,stroke:#844fba,stroke-width:2px,color:#c9d1d9
Loading

🧱 Module Inventory

CategoryModulesWhat They Provision
🌐 Networking security-group · security-group-rule VPC security groups with fine-grained ingress/egress rules
⎈ Kubernetes eks · eks-network · eks-namespace · eks-irsa EKS cluster, networking add-ons, namespace isolation, IAM Roles for Service Accounts
👤 Identity iam-role · iam-policy · iam-group · iam-user · iam-role-policy-attachment · iam-group-policy-attachment · iam-user-group-membership Complete IAM hierarchy — roles, policies, groups, users, and least-privilege bindings
🗄️ Data rds · ecr · secret-manager PostgreSQL RDS instance, ECR container repos, and AWS Secrets Manager entries
🖥️ Compute ec2 Stand-alone Jenkins server on EC2

📐 Infrastructure Stacks

Stack Description
eks infra/ The main production stack — provisions VPC, subnets, EKS cluster with managed node groups, RDS PostgreSQL, ECR repositories, Secrets Manager, IAM roles/IRSA, and all networking
stand-alone-jenkins-server/ Dedicated EC2 instance pre-configured as a Jenkins CI server with Docker, AWS CLI, and security tooling
terraform-backend/ S3 bucket + DynamoDB table for remote state management and state locking

🔄 CI/CD Pipelines — Jenkins

Declarative, security-first pipelines that test, scan, build, and push container images — with zero-trust Docker builds via Kaniko.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    subgraph BACKEND_PIPE["🐍 Backend Pipeline — 14 Stages"]
        direction LR
        B1["📥 Checkout"] --> B2["🗄️ Start Test DB"] --> B3["🐍 Setup Python"] --> B4["🔌 Validate DB"] --> B5["📋 Migrations"] --> B6["🏗️ Build Django"] --> B7["✅ Tests + Coverage"]
        B7 --> B8["🔍 SonarQube"] --> B9["🚦 Quality Gate"] --> B10["🛡️ OWASP"] --> B11["📚 Nexus Upload"] --> B12["🐳 Kaniko Build"] --> B13["🔒 Trivy Scan"] --> B14["📤 Push to ECR"]
    end

    style BACKEND_PIPE fill:#0d1117,stroke:#d24939,stroke-width:2px,color:#c9d1d9
Loading
%%{init: {'theme': 'dark'}}%%
flowchart LR
    subgraph FRONTEND_PIPE["⚛️ Frontend Pipeline — 10 Stages"]
        direction LR
        F1["📥 Checkout"] --> F2["📦 npm install"] --> F3["🏗️ npm run build"] --> F4["🔍 SonarQube"] --> F5["🚦 Quality Gate"]
        F5 --> F6["🛡️ OWASP"] --> F7["📚 Nexus Upload"] --> F8["🐳 Kaniko Build"] --> F9["🔒 Trivy Image"] --> F10["🔍 Trivy FS"]
    end

    style FRONTEND_PIPE fill:#0d1117,stroke:#61dafb,stroke-width:2px,color:#c9d1d9
Loading

🔑 Pipeline Highlights

🐍 Backend Pipeline

  • Real PostgreSQL 15 container spun up for integration tests
  • pytest with coverage reports (HTML, XML, terminal)
  • Multi-stage Docker build → non-root appuser
  • Images pushed to AWS ECR (eu-north-1)
  • Tags: {BUILD_NUMBER}-{SHORT_COMMIT} + latest

⚛️ Frontend Pipeline

  • React production build via Node.js 18
  • Multi-stage Docker: node:20-alpine → nginx:alpine
  • Nginx reverse-proxy config for /api → backend
  • Build artifacts archived to Nexus Raw repository
  • Images pushed to Docker Hub

🛡️ Security Gates (Both Pipelines)

Gate Tool Purpose
Code Quality SonarQube + Quality Gate Static analysis, code smells, coverage enforcement — pipeline aborts on failure
Dependency Scan OWASP Dependency-Check + NVD CVE scanning for all project dependencies
Image Scan Trivy HIGH/CRITICAL vulnerability scanning on final container images
Rootless Builds Kaniko In-cluster, daemon-less Docker builds — no privileged containers needed

🚀 GitOps & Deployment — Argo CD

App-of-Apps pattern with sync waves for ordered, self-healing deployments.

Argo CD watches the k8s-manifests repository and automatically reconciles the cluster state. Every component is declared as an Argo CD Application resource, organized by sync waves to respect dependency ordering.

%%{init: {'theme': 'dark'}}%%
flowchart TB
    PLATFORM["🎯 Platform App\n(App-of-Apps)"]

    subgraph WAVE0["🌊 Wave 0 — Foundation"]
        ALB["AWS Load Balancer\nController"]
    end

    subgraph WAVE1["🌊 Wave 1 — CRDs"]
        CRDS["Prometheus CRDs"]
    end

    subgraph WAVE2["🌊 Wave 2 — Operators"]
        ESO["External Secrets\nOperator"]
    end

    subgraph WAVE3["🌊 Wave 3 — Platform Services"]
        MON["Monitoring\nPrometheus + Grafana"]
        MEM["Memcached"]
    end

    subgraph WAVE4["🌊 Wave 4 — Applications"]
        BK["Backend\nDjango + Redis"]
        LOG["Logging\nEFK Stack"]
    end

    subgraph WAVE5["🌊 Wave 5 — Frontend"]
        FT["Frontend\nReact + Nginx"]
    end

    PLATFORM --> WAVE0 --> WAVE1 --> WAVE2 --> WAVE3 --> WAVE4 --> WAVE5

    style PLATFORM fill:#ef7b4d,stroke:#333,stroke-width:2px,color:white
    style WAVE0 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
    style WAVE1 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
    style WAVE2 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
    style WAVE3 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
    style WAVE4 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
    style WAVE5 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
Loading

⎈ Kubernetes Resources Deployed

Backend Namespace

  • Deployment — 2 replicas, Gunicorn workers
  • Service — ClusterIP on port 8000
  • Ingress — ALB with SSL termination (/api)
  • ConfigMap — RDS host, DB name, Redis URL
  • SecretStore — AWS Secrets Manager via IRSA
  • ExternalSecret — Auto-syncs RDS password
  • Redis — In-cluster cache (Deployment + Service)
  • Job — Database migration runner

Frontend Namespace

  • Deployment — 2 replicas, Nginx serving React build
  • Service — ClusterIP on port 80
  • Ingress — ALB with SSL termination (/)

Shared Infrastructure

  • AWS ALB Controller — Kubernetes Ingress → AWS ALB
  • External Secrets Operator — K8s ↔ AWS Secrets Manager
  • Shared ALB group (shared-alb) for cost optimization
  • SSL via ACM certificate

🔐 Secrets Management Flow

%%{init: {'theme': 'dark'}}%%
flowchart LR
    A["AWS Secrets Manager\n🔑 RDS Password"] -->|IRSA Auth| B["External Secrets\nOperator"]
    B -->|Creates/Syncs| C["K8s Secret\nrds-db-secret"]
    C -->|Mounted as env| D["Backend Pod\n🐍 Django"]
    
    E["ConfigMap\nrds-config"] -->|"DB_HOST, DB_NAME\nDB_USER, REDIS_URL"| D

    style A fill:#f0883e,stroke:#333,color:white
    style B fill:#326ce5,stroke:#333,color:white
    style C fill:#238636,stroke:#333,color:white
    style D fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
    style E fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
Loading

📊 Observability Stack

Full-stack monitoring with Prometheus metrics collection and Grafana visualization, deployed via Helm through Argo CD.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    subgraph METRICS["📈 Metrics Collection"]
        NE["Node Exporter\nHost Metrics"]
        KSM["Kube State Metrics\nK8s Objects"]
        KUB["Kubelet\nPod Metrics"]
    end

    PROM["🔴 Prometheus\n1 Replica · 5Gi cap"]

    GRAF["📊 Grafana\nPersistent · 5Gi\n🌐 grafana.yassinabuelsheikh.store"]

    NE --> PROM
    KSM --> PROM
    KUB --> PROM
    PROM --> GRAF

    style METRICS fill:#0d1117,stroke:#e6a817,stroke-width:2px,color:#c9d1d9
    style PROM fill:#e6522c,stroke:#333,stroke-width:2px,color:white
    style GRAF fill:#f46800,stroke:#333,stroke-width:2px,color:white
Loading
Component Configuration
Prometheus 1 replica, 3h retention, 200MB storage cap, resource-bounded (200m–2000m CPU, 1–5Gi RAM)
Grafana Persistent storage (5Gi), admin creds from K8s Secret, exposed via ALB Ingress with SSL
Kube State Metrics Enabled with resource limits, tracking all K8s object states
Node Exporter DaemonSet on all nodes, lightweight host-level metrics

📝 Centralized Logging — EFK

Elasticsearch + Fluent Bit + Kibana for aggregating, searching, and visualizing logs from all pods across the cluster.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    PODS["🐳 All Pods\n/var/log/pods"] -->|Tail Logs| FB["📤 Fluent Bit\nDaemonSet\nv2.2.2"]
    FB -->|Index: fluent-bit| ES["🔍 Elasticsearch\nSingle-node\nv8.11.1"]
    ES --> KB["📊 Kibana\nDashboards\nv8.11.1"]

    style PODS fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
    style FB fill:#49bda5,stroke:#333,color:white
    style ES fill:#fed10a,stroke:#333,color:black
    style KB fill:#e8478b,stroke:#333,color:white
Loading
Component Details
Fluent Bit DaemonSet tailing /var/log/pods/*/*/*.log, forwarding to Elasticsearch on port 9200
Elasticsearch Single-node deployment, 2Gi memory, security disabled for internal use, readiness probe on /_cluster/health
Kibana Connected to Elasticsearch, readiness probe on /api/status, resource limits: 512Mi–1Gi RAM

🤖 Intelligent Automation — n8n

Smart CI/CD notification workflows that bridge Jenkins pipelines with the engineering team through rich, classified alerts.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    J["🔄 Jenkins\nPipeline"] -->|"POST /webhook/jenkins-notify\nor /webhook/essam"| W["📡 n8n\nWebhook"]
    W --> P["⚙️ Data Processor\nClassify Failure Type"]
    
    P -->|"SUCCESS"| S_OK["✅ Success Report"]
    P -->|"BUILD_FAILED"| S_FAIL["❌ Build Report"]
    P -->|"SECURITY_ISSUE"| S_SEC["🛡️ Security Report\nTrivy + OWASP"]

    S_OK --> EMAIL["📧 Gmail\nRich HTML Email"]
    S_FAIL --> EMAIL
    S_SEC --> EMAIL

    style J fill:#d24939,stroke:#333,color:white
    style W fill:#ff6d5a,stroke:#333,color:white
    style P fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
    style EMAIL fill:#ea4335,stroke:#333,color:white
Loading

📨 Notification Features

Feature Description
Dynamic Headers Color-coded — 🟢 green for success, 🔴 red for failure
Failure Classification Distinguishes between BUILD_FAILED and SECURITY_ISSUE
Security Insights Highlights Trivy critical CVEs and OWASP dependency vulnerabilities
Code Quality State Includes SonarQube issue summary when available
Build Metadata Job name, build number, image tag, and direct link to Jenkins build

🔐 Security Practices

%%{init: {'theme': 'dark'}}%%
mindmap
  root((🔐 Security))
    🐳 Container Security
      Multi-stage Docker builds
      Non-root containers (appuser)
      Kaniko daemon-less builds
      Trivy image scanning
    🔍 Code Security
      SonarQube static analysis
      Quality Gate enforcement
      OWASP Dependency-Check
    ☁️ Cloud Security
      IAM least privilege
      IRSA for pod-level access
      Secrets Manager integration
      SSL/TLS via ACM
    ⎈ Kubernetes Security
      Namespace isolation
      NetworkPolicy for Memcached
      External Secrets Operator
      Service account bindings
Loading

🌐 Traffic Flow & Network Topology

%%{init: {'theme': 'dark'}}%%
flowchart TB
    USER["👤 User\nhttps://yassinabuelsheikh.store"] -->|HTTPS :443| ALB["⚖️ AWS ALB\nShared ALB Group\nSSL Termination"]
    
    ALB -->|"/ → frontend:80"| FE["⚛️ Frontend\nNginx · React"]
    ALB -->|"/api → backend:8000"| BE["🐍 Backend\nDjango · Gunicorn"]
    ALB -->|"grafana.* → grafana:3000"| GRAF["📊 Grafana"]

    FE -->|"Internal /api proxy"| BE
    BE -->|"SSL :5432"| RDS["🗄️ RDS PostgreSQL\ngig-route.*.rds.amazonaws.com"]
    BE -->|":6379"| REDIS["⚡ Redis"]

    style USER fill:#58a6ff,stroke:#333,color:white
    style ALB fill:#f0883e,stroke:#333,color:white
    style FE fill:#61dafb,stroke:#333,color:black
    style BE fill:#092e20,stroke:#333,color:white
    style RDS fill:#3b48cc,stroke:#333,color:white
    style REDIS fill:#dc382d,stroke:#333,color:white
    style GRAF fill:#f46800,stroke:#333,color:white
Loading

🧰 Technology Stack

LayerTechnologies
Application Django React Nginx Gunicorn PostgreSQL Redis
Infrastructure Terraform EKS RDS ECR
CI/CD Jenkins ArgoCD Kaniko Nexus
Security SonarQube Trivy OWASP
Observability Prometheus Grafana ES Fluent Bit Kibana
Automation n8n

👥 Team & Purpose

🎓 This organization serves as our DevOps engineering practice lab — a hands-on environment where we design, build, and operate production-grade infrastructure and CI/CD systems using real-world tools and best practices.

🎯 What We Practice

☁️
IaC
Terraform modules,
remote state, AWS

🔄
CI/CD
Jenkins declarative
pipelines, security gates

⎈
Kubernetes
EKS, namespaces,
IRSA, ingress

🚀
GitOps
Argo CD, app-of-apps,
sync waves

📊
Observability
Prometheus, Grafana,
EFK stack


📄 License

This project is maintained for educational purposes as part of the NTI DevOps training program.


👥 Our Team & Contributions

Meet the DevOps Engineers behind this project

Abdelaziz Ak
Abdelaziz Ak

Infrastructure & Terraform

Commits
Yassin Abu El-Sheikh
Yassin Abu El-Sheikh

Kubernetes & EKS

Commits
Mohamed Essam
Mohamed Essam

CI Pipelines

Commits
Name 4
Mazen Mostafa

CD Pipelines

Commits
Abdulrahman Mahmoud
Abdulrahman Mahmoud

EFK Monitoring

Commits


🤝 Collaboration
Working together
Learning together
Growing together

Built with ❤️ by the NTI DevOps Gig Router Project Team · DevOps Engineering Practice

Popular repositories Loading

  1. App-Back-End App-Back-End Public

    Python 1

  2. App-Front-End App-Front-End Public

    JavaScript 1

  3. Terraform-Infra Terraform-Infra Public

    HCL 1

  4. k8s-manifests k8s-manifests Public

    1

  5. .github .github Public

  6. n8n n8n Public

Repositories

Showing 6 of 6 repositories

Top languages

Loading…

Most used topics

Loading…