A comprehensive DevOps practice project demonstrating the full lifecycle of building, securing, deploying, and operating a cloud-native application on AWS EKS — from infrastructure provisioning with Terraform, through multi-stage CI/CD pipelines, to GitOps-driven Kubernetes deployments with observability, logging, and intelligent automation.
%%{init: {'theme': 'dark', 'themeVariables': {'fontFamily': 'monospace'}}}%%
flowchart TB
subgraph DEV["🧑💻 Developer Workflow"]
direction LR
CODE[Push Code] --> GH[GitHub]
end
subgraph INFRA["☁️ AWS Infrastructure · Terraform"]
direction TB
VPC["🌐 VPC · Subnets · SGs"]
EKS["⎈ EKS Cluster"]
RDS["🗄️ RDS PostgreSQL"]
ECR["📦 ECR Registry"]
SM["🔑 Secrets Manager"]
IAM["👤 IAM · IRSA"]
VPC --> EKS
VPC --> RDS
IAM --> EKS
end
subgraph CI["🔄 Jenkins CI/CD"]
direction TB
TEST["✅ Unit Tests\n+ Coverage"]
SONAR["🔍 SonarQube\nQuality Gate"]
OWASP["🛡️ OWASP\nDep-Check"]
KANIKO["🐳 Kaniko\nBuild"]
TRIVY["🔒 Trivy\nImage Scan"]
NEXUS["📚 Nexus\nArtifacts"]
TEST --> SONAR --> OWASP --> KANIKO --> TRIVY
KANIKO --> NEXUS
end
subgraph CD["🚀 GitOps · Argo CD"]
direction TB
ARGO["Argo CD\nApp-of-Apps"]
SYNC["Auto Sync\n+ Self-Heal"]
ARGO --> SYNC
end
subgraph K8S["⎈ Kubernetes Workloads"]
direction TB
FE["⚛️ Frontend\nReact · Nginx"]
BE["🐍 Backend\nDjango · Gunicorn"]
REDIS["⚡ Redis\nCache"]
MC["💾 Memcached"]
FE -->|"/api proxy"| BE
BE --> REDIS
end
subgraph OBS["📊 Observability"]
direction LR
PROM["Prometheus"]
GRAF["Grafana"]
EFK_S["EFK Stack"]
PROM --> GRAF
end
subgraph NOTIFY["🤖 Automation"]
N8N["n8n Workflows\nSmart Alerts"]
end
GH -->|Webhook| CI
TRIVY -->|Push Image| ECR
CI -->|Notify| N8N
GH -->|Sync| CD
SYNC --> K8S
K8S --> OBS
BE -->|SSL| RDS
K8S ---|Secrets via IRSA| SM
EKS --- K8S
style DEV fill:#0d1117,stroke:#58a6ff,stroke-width:2px,color:#c9d1d9
style INFRA fill:#0d1117,stroke:#f0883e,stroke-width:2px,color:#c9d1d9
style CI fill:#0d1117,stroke:#d24939,stroke-width:2px,color:#c9d1d9
style CD fill:#0d1117,stroke:#ef7b4d,stroke-width:2px,color:#c9d1d9
style K8S fill:#0d1117,stroke:#326ce5,stroke-width:2px,color:#c9d1d9
style OBS fill:#0d1117,stroke:#e6a817,stroke-width:2px,color:#c9d1d9
style NOTIFY fill:#0d1117,stroke:#ff6d5a,stroke-width:2px,color:#c9d1d9
| Repository | Purpose | Key Technologies |
|---|---|---|
| Terraform-Infra | AWS infrastructure as code — EKS, RDS, VPC, IAM, ECR, Secrets Manager | |
| App-Back-End | Django REST backend with full Jenkins CI/CD pipeline | |
| App-Front-End | React frontend served via Nginx with Jenkins CI/CD | |
| k8s-manifests | Kubernetes manifests — Argo CD apps, deployments, observability, logging | |
| n8n | Workflow automation — smart CI/CD notifications via webhooks |
Fully modular IaC provisioning all AWS resources needed to run a production Kubernetes platform.
%%{init: {'theme': 'dark'}}%%
flowchart LR
subgraph TF_BACKEND["🗃️ Remote State"]
S3["S3 Bucket"]
DDB["DynamoDB Lock"]
end
subgraph MODULES["📦 Reusable Modules"]
direction TB
M_VPC["security-group\nsecurity-group-rule"]
M_EKS["eks\neks-network\neks-namespace\neks-irsa"]
M_IAM["iam-role · iam-policy\niam-group · iam-user"]
M_DATA["rds\necr\nsecret-manager"]
M_COMPUTE["ec2"]
end
subgraph STACKS["🏗️ Infrastructure Stacks"]
EKS_STACK["EKS Infra\nmain.tf · 28 KB"]
JENKINS_STACK["Stand-Alone\nJenkins Server"]
end
TF_BACKEND --> STACKS
MODULES --> STACKS
style TF_BACKEND fill:#0d1117,stroke:#844fba,stroke-width:2px,color:#c9d1d9
style MODULES fill:#0d1117,stroke:#844fba,stroke-width:2px,color:#c9d1d9
style STACKS fill:#0d1117,stroke:#844fba,stroke-width:2px,color:#c9d1d9
| Category | Modules | What They Provision |
|---|---|---|
| 🌐 Networking | security-group · security-group-rule |
VPC security groups with fine-grained ingress/egress rules |
| ⎈ Kubernetes | eks · eks-network · eks-namespace · eks-irsa |
EKS cluster, networking add-ons, namespace isolation, IAM Roles for Service Accounts |
| 👤 Identity | iam-role · iam-policy · iam-group · iam-user · iam-role-policy-attachment · iam-group-policy-attachment · iam-user-group-membership |
Complete IAM hierarchy — roles, policies, groups, users, and least-privilege bindings |
| 🗄️ Data | rds · ecr · secret-manager |
PostgreSQL RDS instance, ECR container repos, and AWS Secrets Manager entries |
| 🖥️ Compute | ec2 |
Stand-alone Jenkins server on EC2 |
| Stack | Description |
|---|---|
eks infra/ |
The main production stack — provisions VPC, subnets, EKS cluster with managed node groups, RDS PostgreSQL, ECR repositories, Secrets Manager, IAM roles/IRSA, and all networking |
stand-alone-jenkins-server/ |
Dedicated EC2 instance pre-configured as a Jenkins CI server with Docker, AWS CLI, and security tooling |
terraform-backend/ |
S3 bucket + DynamoDB table for remote state management and state locking |
Declarative, security-first pipelines that test, scan, build, and push container images — with zero-trust Docker builds via Kaniko.
%%{init: {'theme': 'dark'}}%%
flowchart LR
subgraph BACKEND_PIPE["🐍 Backend Pipeline — 14 Stages"]
direction LR
B1["📥 Checkout"] --> B2["🗄️ Start Test DB"] --> B3["🐍 Setup Python"] --> B4["🔌 Validate DB"] --> B5["📋 Migrations"] --> B6["🏗️ Build Django"] --> B7["✅ Tests + Coverage"]
B7 --> B8["🔍 SonarQube"] --> B9["🚦 Quality Gate"] --> B10["🛡️ OWASP"] --> B11["📚 Nexus Upload"] --> B12["🐳 Kaniko Build"] --> B13["🔒 Trivy Scan"] --> B14["📤 Push to ECR"]
end
style BACKEND_PIPE fill:#0d1117,stroke:#d24939,stroke-width:2px,color:#c9d1d9
%%{init: {'theme': 'dark'}}%%
flowchart LR
subgraph FRONTEND_PIPE["⚛️ Frontend Pipeline — 10 Stages"]
direction LR
F1["📥 Checkout"] --> F2["📦 npm install"] --> F3["🏗️ npm run build"] --> F4["🔍 SonarQube"] --> F5["🚦 Quality Gate"]
F5 --> F6["🛡️ OWASP"] --> F7["📚 Nexus Upload"] --> F8["🐳 Kaniko Build"] --> F9["🔒 Trivy Image"] --> F10["🔍 Trivy FS"]
end
style FRONTEND_PIPE fill:#0d1117,stroke:#61dafb,stroke-width:2px,color:#c9d1d9
|
🐍 Backend Pipeline
|
⚛️ Frontend Pipeline
|
| Gate | Tool | Purpose |
|---|---|---|
| Code Quality | SonarQube + Quality Gate | Static analysis, code smells, coverage enforcement — pipeline aborts on failure |
| Dependency Scan | OWASP Dependency-Check + NVD | CVE scanning for all project dependencies |
| Image Scan | Trivy | HIGH/CRITICAL vulnerability scanning on final container images |
| Rootless Builds | Kaniko | In-cluster, daemon-less Docker builds — no privileged containers needed |
App-of-Apps pattern with sync waves for ordered, self-healing deployments.
Argo CD watches the k8s-manifests repository and automatically reconciles the cluster state. Every component is declared as an Argo CD Application resource, organized by sync waves to respect dependency ordering.
%%{init: {'theme': 'dark'}}%%
flowchart TB
PLATFORM["🎯 Platform App\n(App-of-Apps)"]
subgraph WAVE0["🌊 Wave 0 — Foundation"]
ALB["AWS Load Balancer\nController"]
end
subgraph WAVE1["🌊 Wave 1 — CRDs"]
CRDS["Prometheus CRDs"]
end
subgraph WAVE2["🌊 Wave 2 — Operators"]
ESO["External Secrets\nOperator"]
end
subgraph WAVE3["🌊 Wave 3 — Platform Services"]
MON["Monitoring\nPrometheus + Grafana"]
MEM["Memcached"]
end
subgraph WAVE4["🌊 Wave 4 — Applications"]
BK["Backend\nDjango + Redis"]
LOG["Logging\nEFK Stack"]
end
subgraph WAVE5["🌊 Wave 5 — Frontend"]
FT["Frontend\nReact + Nginx"]
end
PLATFORM --> WAVE0 --> WAVE1 --> WAVE2 --> WAVE3 --> WAVE4 --> WAVE5
style PLATFORM fill:#ef7b4d,stroke:#333,stroke-width:2px,color:white
style WAVE0 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
style WAVE1 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
style WAVE2 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
style WAVE3 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
style WAVE4 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
style WAVE5 fill:#0d1117,stroke:#58a6ff,stroke-width:1px,color:#c9d1d9
|
Backend Namespace
|
Frontend Namespace
Shared Infrastructure
|
%%{init: {'theme': 'dark'}}%%
flowchart LR
A["AWS Secrets Manager\n🔑 RDS Password"] -->|IRSA Auth| B["External Secrets\nOperator"]
B -->|Creates/Syncs| C["K8s Secret\nrds-db-secret"]
C -->|Mounted as env| D["Backend Pod\n🐍 Django"]
E["ConfigMap\nrds-config"] -->|"DB_HOST, DB_NAME\nDB_USER, REDIS_URL"| D
style A fill:#f0883e,stroke:#333,color:white
style B fill:#326ce5,stroke:#333,color:white
style C fill:#238636,stroke:#333,color:white
style D fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
style E fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
Full-stack monitoring with Prometheus metrics collection and Grafana visualization, deployed via Helm through Argo CD.
%%{init: {'theme': 'dark'}}%%
flowchart LR
subgraph METRICS["📈 Metrics Collection"]
NE["Node Exporter\nHost Metrics"]
KSM["Kube State Metrics\nK8s Objects"]
KUB["Kubelet\nPod Metrics"]
end
PROM["🔴 Prometheus\n1 Replica · 5Gi cap"]
GRAF["📊 Grafana\nPersistent · 5Gi\n🌐 grafana.yassinabuelsheikh.store"]
NE --> PROM
KSM --> PROM
KUB --> PROM
PROM --> GRAF
style METRICS fill:#0d1117,stroke:#e6a817,stroke-width:2px,color:#c9d1d9
style PROM fill:#e6522c,stroke:#333,stroke-width:2px,color:white
style GRAF fill:#f46800,stroke:#333,stroke-width:2px,color:white
| Component | Configuration |
|---|---|
| Prometheus | 1 replica, 3h retention, 200MB storage cap, resource-bounded (200m–2000m CPU, 1–5Gi RAM) |
| Grafana | Persistent storage (5Gi), admin creds from K8s Secret, exposed via ALB Ingress with SSL |
| Kube State Metrics | Enabled with resource limits, tracking all K8s object states |
| Node Exporter | DaemonSet on all nodes, lightweight host-level metrics |
Elasticsearch + Fluent Bit + Kibana for aggregating, searching, and visualizing logs from all pods across the cluster.
%%{init: {'theme': 'dark'}}%%
flowchart LR
PODS["🐳 All Pods\n/var/log/pods"] -->|Tail Logs| FB["📤 Fluent Bit\nDaemonSet\nv2.2.2"]
FB -->|Index: fluent-bit| ES["🔍 Elasticsearch\nSingle-node\nv8.11.1"]
ES --> KB["📊 Kibana\nDashboards\nv8.11.1"]
style PODS fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
style FB fill:#49bda5,stroke:#333,color:white
style ES fill:#fed10a,stroke:#333,color:black
style KB fill:#e8478b,stroke:#333,color:white
| Component | Details |
|---|---|
| Fluent Bit | DaemonSet tailing /var/log/pods/*/*/*.log, forwarding to Elasticsearch on port 9200 |
| Elasticsearch | Single-node deployment, 2Gi memory, security disabled for internal use, readiness probe on /_cluster/health |
| Kibana | Connected to Elasticsearch, readiness probe on /api/status, resource limits: 512Mi–1Gi RAM |
Smart CI/CD notification workflows that bridge Jenkins pipelines with the engineering team through rich, classified alerts.
%%{init: {'theme': 'dark'}}%%
flowchart LR
J["🔄 Jenkins\nPipeline"] -->|"POST /webhook/jenkins-notify\nor /webhook/essam"| W["📡 n8n\nWebhook"]
W --> P["⚙️ Data Processor\nClassify Failure Type"]
P -->|"SUCCESS"| S_OK["✅ Success Report"]
P -->|"BUILD_FAILED"| S_FAIL["❌ Build Report"]
P -->|"SECURITY_ISSUE"| S_SEC["🛡️ Security Report\nTrivy + OWASP"]
S_OK --> EMAIL["📧 Gmail\nRich HTML Email"]
S_FAIL --> EMAIL
S_SEC --> EMAIL
style J fill:#d24939,stroke:#333,color:white
style W fill:#ff6d5a,stroke:#333,color:white
style P fill:#0d1117,stroke:#58a6ff,color:#c9d1d9
style EMAIL fill:#ea4335,stroke:#333,color:white
| Feature | Description |
|---|---|
| Dynamic Headers | Color-coded — 🟢 green for success, 🔴 red for failure |
| Failure Classification | Distinguishes between BUILD_FAILED and SECURITY_ISSUE |
| Security Insights | Highlights Trivy critical CVEs and OWASP dependency vulnerabilities |
| Code Quality State | Includes SonarQube issue summary when available |
| Build Metadata | Job name, build number, image tag, and direct link to Jenkins build |
%%{init: {'theme': 'dark'}}%%
mindmap
root((🔐 Security))
🐳 Container Security
Multi-stage Docker builds
Non-root containers (appuser)
Kaniko daemon-less builds
Trivy image scanning
🔍 Code Security
SonarQube static analysis
Quality Gate enforcement
OWASP Dependency-Check
☁️ Cloud Security
IAM least privilege
IRSA for pod-level access
Secrets Manager integration
SSL/TLS via ACM
⎈ Kubernetes Security
Namespace isolation
NetworkPolicy for Memcached
External Secrets Operator
Service account bindings
%%{init: {'theme': 'dark'}}%%
flowchart TB
USER["👤 User\nhttps://yassinabuelsheikh.store"] -->|HTTPS :443| ALB["⚖️ AWS ALB\nShared ALB Group\nSSL Termination"]
ALB -->|"/ → frontend:80"| FE["⚛️ Frontend\nNginx · React"]
ALB -->|"/api → backend:8000"| BE["🐍 Backend\nDjango · Gunicorn"]
ALB -->|"grafana.* → grafana:3000"| GRAF["📊 Grafana"]
FE -->|"Internal /api proxy"| BE
BE -->|"SSL :5432"| RDS["🗄️ RDS PostgreSQL\ngig-route.*.rds.amazonaws.com"]
BE -->|":6379"| REDIS["⚡ Redis"]
style USER fill:#58a6ff,stroke:#333,color:white
style ALB fill:#f0883e,stroke:#333,color:white
style FE fill:#61dafb,stroke:#333,color:black
style BE fill:#092e20,stroke:#333,color:white
style RDS fill:#3b48cc,stroke:#333,color:white
style REDIS fill:#dc382d,stroke:#333,color:white
style GRAF fill:#f46800,stroke:#333,color:white
🎓 This organization serves as our DevOps engineering practice lab — a hands-on environment where we design, build, and operate production-grade infrastructure and CI/CD systems using real-world tools and best practices.
|
☁️ |
🔄 |
⎈ |
🚀 |
📊 |
This project is maintained for educational purposes as part of the NTI DevOps training program.
Meet the DevOps Engineers behind this project
Abdelaziz Ak Infrastructure & Terraform |
Yassin Abu El-Sheikh Kubernetes & EKS |
Mohamed Essam CI Pipelines |
Mazen Mostafa CD Pipelines |
Abdulrahman Mahmoud EFK Monitoring |
🤝 Collaboration Working together Learning together Growing together |