Skip to content

Add lld - #178

Merged
jcbashdown merged 4 commits into
mainfrom
add-lld
Sep 28, 2026
Merged

jcbashdown merged 4 commits into
mainfrom
add-lld

Conversation

@jcbashdown

Copy link
Copy Markdown
Contributor

No description provided.

Fix all tests to update expectations from fixture update - no hard
coding
@jcbashdown
jcbashdown requested a review from zuhdil September 23, 2026 09:54
Comment thread docs/lld/IGH-Pipeline-Portal-LLD.md Outdated
| Document | Low-Level Design — technical description |
| System | IGH Pipeline Portal |
| Companion document | `IGH-Pipeline-Portal-HLD.docx` (functional description) |
| Repositories | `igh-dashboard`, `igh-data-sync`, `igh-data-transform`, `igh-airflow`, `igh-dashboard-content` |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be necessary to add a link to each repository because without it it might be difficult for readers to find them.

Comment thread docs/lld/IGH-Pipeline-Portal-LLD.md Outdated
file is written. The chain is wired with Airflow 3 **Assets**: a task that finishes with an
asset in its `outlets` emits an event, and a DAG with that asset in its `schedule` starts.
`igh-airflow/dags/igh_assets.py` declares one asset per database. Neither
`TriggerDagRunOperator` nor `ExternalTaskSensor` is used. Declaring the chain this way rather

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioning things that aren't used doesn't add to the reader's understanding; it is simply contextual noise.

## 3. Container Diagram

![Containers](diagrams/out/02-container.svg)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to explain the Airflow infrastructure. It's a third-party tool that we use as-is, and it can obscure our important components. I think Airflow can be described as a black box -much like Traefik, Redis, and Celery- that orchestrates our three DAGs.

Comment thread docs/lld/IGH-Pipeline-Portal-LLD.md Outdated
docker compose stop && docker compose up -d
```

Test deploys on every merge to `main`. Production deploys when a GitHub Release is published.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More precisely, on every push to main, PR merge is a form of push.

@jcbashdown
jcbashdown merged commit 9979c6b into main Sep 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants