Skip to content

Refactor/composable features - #305

Merged
igorbenav merged 207 commits into
mainfrom
refactor/composable-features
Oct 25, 2026
Merged

igorbenav merged 207 commits into
mainfrom
refactor/composable-features

Conversation

@igorbenav

@igorbenav igorbenav commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Composable features

The boilerplate said "delete the parts you don't want", which was true of directories and false of imports. modules/rate_limit imported the tier models, the API throttle read tier_id off the user row, the authorization checks imported the role tables, create_application named every router by hand, the seeders ran whether or not their tables existed, and the test fixtures assumed every account had a tier. Delete one directory and four others broke, so nobody deleted anything, and the parts nobody deletes are the parts nobody reads.

This branch makes nine features removable, each of them leaving with its routes, models, settings, admin views, seeders, fixtures and tests: accounts, rbac, tiers, tier_limits, ratelimit, api_keys, admin, cache and taskiq. The cross-feature imports are gone, replaced by a composition root. The auth stack moves to crudauth 0.8.2, which takes back the session routes, the session store and the lockout counters that had been rewritten here. Roles and API keys now share the permission registry as their only vocabulary, so a scoped credential is weaker than its owner rather than equal to it.

Three review rounds went over the result and found a privilege escalation this branch had introduced, two races that corrupted state instead of failing, a family of 500s that should have been 422 since the beginning, and recovery routes whose timing told a caller which addresses had accounts. All of that is fixed here, each fix in its own commit, with the failure reproduced first and the new guard mutated afterwards to prove its test fails without it. The full gate runs before anything lands: lint, types, the suite forwards and in reversed collection order, the CLI, and the removal drill. A security pass closes the branch: the two reports open against the project were audited line by line, and most of what they describe was already answered by the work above, so what was left, the admin panel's defaults, is fixed here, and every dependency carrying a published advisory is at a patched release.


A feature is a set of paths and a few contributions

The composition root is a rule about direction: a feature contributes to extension points it does not own, and nothing imports it back. The shapes it contributes through live in infrastructure/composition.py and are deliberately thin, a RouterMount, a Lifecycle, a PermissionSource, a RateLimitResolver, a TierDeleteGuard and TierDeleteRelease, a ReadinessCheck in one of two groups, model and schema extensions for the user row, and an installer callable. Each is a function type or a frozen dataclass, because the moment a contribution needs a base class, the feature on the other side has to import it.

What a project selected is listed in src/wiring/: app.py, settings.py, hooks.py, models.py, admin.py, email.py, cache.py, transports.py, tasks.py and tables.py, with backend/scripts/seeders.py and backend/tests/wiring.py doing the same for seeders and fixtures. Those files are the only place that knows the whole set, since a feature owns its settings class, its error-to-status mapping, its fixtures and its permission declarations. Composable Features is the guide.

Some code belongs to two features at once: an API key scoped to an RBAC permission, a tier column on an admin form, a test that needs both. None of that is a contribution from one feature to another, it is code that makes no sense without both, so it lives in a path that leaves with either side. The manifest calls those combination_paths, and the two tests and one admin assertion that moved into them were each found by a preset failing rather than by someone reading.

Why the wiring is generated and then compared

Hand-written wiring and a generator that claims to reproduce it will drift, on the day somebody edits one of them. So tools/removal_drill.py holds the generator, and tests/unit/test_removal_drill.py regenerates every wiring module for the features this repository carries and compares the result against the committed file by AST, ignoring docstrings, comments and formatting. A contribution added by hand and not to the generator fails there, which has already caught the two drifting apart.

The same comparison decides where a reader puts code of their own. The Logfire recipe in Observability wires its installer in the composition root, src/interfaces/main.py, rather than in src/wiring/app.py, because an earlier version of that recipe edited the generated file and left anyone who followed the guide with a failing test.

The drill is the test of the architecture

python tools/removal_drill.py copies the repository to a scratch directory once per preset, deletes the paths of every unselected feature, regenerates the wiring for what is left, and then runs six checks over it: the app imports, every module imports, ruff check src tests, mypy src scripts migrations tests, pytest tests, and an Alembic baseline applied to an empty database. Nine presets run as a CI matrix, from core-only with no features at all up to everything.

The baseline is in there because it is the only check that makes a removal visible. A feature's tables leave with it, and the schema the baseline creates is compared against what the models declare, so dropping the key_permissions table showed up as one table and five indexes fewer in exactly the place it should have. A preset carrying fewer features skips the tests whose subject it no longer has, which is the drill's own build tests for absent features and a throttle comparison that needs two throttled routes to compare budgets. On the full tree nothing is skipped.

Why a copy of the repository and not a setting

A feature flag would prove the code runs with a feature switched off, which is a weaker claim than the one being made. The drill deletes paths instead, so a stray import of a removed module is an ImportError in the copy rather than a branch nobody took. That costs minutes per preset, and it is the version of the check that means something.

Auth: giving the work back to the library

The pin moves from crudauth[all]>=0.7.1,<0.8.0 to >=0.8.2,<0.9.0, and what the library now does better is deleted here: the session routes written in this repository, the logout path, the session store and the lockout counters. crudauth's session-management router replaces them and brings a device list and per-device sign-out with it. Setting SESSION_BACKEND=database and RATE_LIMITER_BACKEND=database keeps sessions and lockout counters in the project's own database, so several workers share them without a Redis.

Two of the choices there read as omissions. The first is set-password, which this project does not mount: it would let anyone holding a session put a password on a provider-only account, and that is a takeover whenever a session is borrowed. Such an account sets its first password through the reset link, which proves the address first.

The second is the sign-out. crudauth 0.8 looks a session up under an HMAC of its id keyed with SECRET_KEY, so sessions written by 0.7 sit under the raw id and are no longer found, and everyone signs in again after the deploy. A SECRET_KEY rotation now does the same thing, which the admin-panel guide states plainly instead of recommending a rotation as the way to clear panel sessions.

The bump was priced before the pin changed: every symbol and method this project touches was diffed between the installed 0.7.5 and the 0.8.2 source, and the suite was then run against 0.8.2 by PYTHONPATH shadowing. Two tests failed, both of them the re-login revocation above. The test written for it was run against 0.7.5 in a scratch target afterwards, to watch it fail there, since a test that passes on both versions proves nothing about either.

One vocabulary for authorisation

Permissions are a registry that any module declares into without importing rbac, and GET /api/v1/permissions lists what the project has while /permissions/me lists what the caller holds. Roles and API key scopes carry the same registry names, and there is no second vocabulary for either. The one that used to exist was a table of key_permissions rows keyed by words like conversations, credits, user_profile and *, describing access that no route could check.

The role API runs on one rule: nobody hands out what they do not hold. can_delegate_permissions and can_assign_role compare against the caller's effective permissions, resolved once per request and narrowed to the credential's scope, and deleting a role or emptying it is held to the same rule for every live holder, because taking a permission away from an account the caller cannot reach is the same escalation as giving one.

A key is never a superuser, and the flag is cancelled where the principal is built: the transport returns a principal without it and records the owner's flag in the credential's metadata, where only the permission narrowing reads it. The first version cancelled the flag at each gate, and review found two gates it had missed, role/delegation.py and user/service.py, the first of which let a superuser's key scoped to role.assign assign any role to any account. Cancelling it once means no consumer has to remember. The same reasoning covers the owner's own resources: a key scoped for reading cannot rename the account behind it, because ownership on its own is not a licence.

What an API key is, and what it may do

A key is secrets.token_urlsafe(48), which is 384 bits of server-generated randomness, so there is nothing to guess and a work factor buys nothing. It costs something, though. The old scrypt storage made every key request pay a derivation, and since key_prefix appears in listings, anybody holding a prefix could make the server pay it on junk requests. Storage is sha256$<digest> now, found by one indexed lookup, while keys stored the old way still authenticate by prefix and the old verification and are rewritten on first use. A script names the ones that never moved and revokes them on request. A scanner may flag SHA-256 here as weak password hashing, and for this column that reading is wrong, which the guide explains.

Key management belongs to a session. Minting, reading, rescoping and revoking all refuse a key, so a key cannot mint another, widen itself, or revoke the one an operator would use to stop it. Minting also needs a verified address, because an unverified address may belong to somebody else, and that is what lets an attacker register under a victim's address, mint a key, and keep API access after the victim claims the account through a provider. The test holding all of this reads the key routes off the app rather than from a list, so a route added later is covered without anyone remembering.

Usage limits are counted at authentication, per key id, on the limiter behind the login lockout, which means they cover every route the key can reach whether or not that route is throttled. Windows are counted shortest first, so a request the minute window refuses keeps its hour and day budgets, and counting fails open, because a limiter nobody can reach should let the request through rather than take the API down with it. A key held at its limit records one refused request per window instead of one per request, so a client hammering a full window costs the limiter and nothing else.

Every request a key authenticates writes one usage row as a background task after the response, carrying the path, the method, the status the client actually got, the duration, the address and the user agent. Rows are kept for KEY_USAGE_RETENTION_DAYS, pruned by a scheduled task where there is a worker and by scripts/prune_key_usage.py where there is not.

What a route answers

Much of the review was not about features at all, but about answers. Input that used to reach Postgres and come back a 500 answers 422 now: page numbers and ids past the Integer range, expiries that cannot be expressed in UTC, a NUL character anywhere in a body or a query, text no column can hold on the routes crudauth owns, and a rate limit's path, which is validated as the route template it has to match. A signup that loses a race for an address or a username answers 409, and the index that refused the row decides which of the two it was, rather than a pre-check. A key whose stored row no schema can read answers 401 instead of a 500 on every request it makes.

Recovery routes answer the same way and in the same time. They send only where the address has an account, and the sender used to be awaited in the request, so one path published to the broker while the other did nothing, and a caller could time the difference to learn who is registered. Against a broker that was down the gap was the whole BROKER_CONNECT_SECONDS, with the SMTP fallback on top of it. The message now goes to a task of its own and the shutdown drains what was handed over, so both paths answer in the time it takes to make a task.

That handover started inside the taskiq feature, which left a project without a worker carrying the very oracle the fix was for. It is core now, in infrastructure/handover.py, the sender such a project is wired to is wrapped in HandingOverSender, and a lifecycle drains it at shutdown. The two drain budgets differ by what each can legitimately take, one delivery to a mail server here, the queue leg on top of it with a worker.

A signup answers the same way whether the address was free or taken, creating nothing in the second case and telling the account that holds the address that somebody tried to sign up with it. A message waiting in the queue is sealed, so a verification or reset link cannot be read out of Redis or RabbitMQ, or out of a backup of either. And a confirmed email change ends every session the account had, since the address is what recovers an account.

Writes that race

Two findings in the last round were worse than the 500s that led to them.

POST /roles/{id}/users/{user} looked for an assignment and then inserted it, so two at once answered 201 and 500. PUT /roles/{id}/permissions was quieter and worse: each replacement deletes what the role carries and inserts its own, neither saw the other, and the role came out carrying the permissions of both requests, one set of them from a caller who had asked for the other. That is not a failed request, it is a role nobody configured.

Every write path takes the role's row first with SELECT … FOR UPDATE, so writes to one role queue, and that turned out not to be enough on its own. The checks that decide whether a caller may reach an account read what that account holds, and a superuser handing it a stronger role landed between the check and the write, leaving the caller having changed the roles of an account stronger than themselves. So the account's row is taken as well, with FOR NO KEY UPDATE, which blocks another grant write, a delete and a promotion while leaving alone the FOR KEY SHARE an unrelated foreign key takes, so an API key or a usage row for that account never waits on a role change. Where a write changes every holder's grants the holders are taken FOR SHARE, which lets two of those run at once while an assignment to one of them waits, and a role's row is always taken before an account's, so two orders never meet.

Rate limits had the same shape without the lock. Nothing stopped a second row for one tier_id and path, and the limit a caller got was whichever row the lookup read first, so two rows saying 10 and 1000 made the throttle a toss-up that could land differently after a restart. A partial unique index over (tier_id, path), covering the rows a soft delete has not taken out, refuses the second one; PATCH answers 409 when the new path is one the tier already limits; and a rule for a tier deleted mid-write answers 404 for the tier instead of a conflict with another rule, because the constraint that refused the row is read off the driver's own error.

What the tests for those had to look like

A race whose every outcome is legal cannot be caught by asserting a status, since two concurrent assignments answering 201 and 201 is the same pair whether the writes queued or not. These tests assert on waiting instead: the competing write starts as a task, and the request holding the lock checks that it has not finished. The writes that must not wait are awaited under a timeout, so a write that is genuinely blocked fails the test rather than hanging the suite. asyncio.gather on its own never reproduced any of this, because lockstep scheduling misses the window, so the interleavings are forced with a barrier around the real per-request permission lookup.

Operations

Readiness answers for what a request needs. GET /health/ready is 503 while the database, its schema, the lockout limiter or the session store is unreachable, and stays 200 for the cache and the task broker, which no request waits on. Probes run together, each under its own two-second timeout, deduplicated per server, and the report is reused for a couple of seconds so that a flood of probes cannot take the connection pool from real requests. Two checks that reach one host as different users are told apart, so a cache pointed at the limiter's host with the wrong password is reported down rather than ready by mirroring the limiter's answer.

The admin panel sits inside the app's security rather than beside it, and is off until a project asks for it: ADMIN_ENABLED unset leaves /admin unmounted, and so does either credential being empty, since such a panel could only ever serve a login form. Its login failures used to be counted where the public login's lockout could not see them, and per address, so one password could be tried from one address after another; they feed the shared lockout against admin-panel:<address> now, and every refusal is recorded. The panel refuses cross-origin writes, ends a session on a POST only, and binds a session to the credentials that made it, with the fingerprint it binds by derived under a work factor once, as the panel is built, rather than per request. Its CSV export writes formula-like cells as text, and the suite fails any exporting view that doesn't.

Tracing ships as a recipe rather than as a feature. The project emits structured logs, a support id per failed request, readiness per dependency and a usage row per key request, and Observability shows how to put Logfire or a plain OTLP collector in front of that material without exporting a session cookie or a bound parameter. It also says what a project that wants observability to be removable would have to add to the manifest instead.

How each change was proved

The loop is the same every time: reproduce the failure as a test that fails for the stated reason, make the smallest change that answers it, then mutate the new guard and run its test again to watch it fail without the guard.

A mutation that leaves the suite green means the guard is untested, and three times it meant the test was asserting something else: an API key refusal's message that could never fail, a role path's account id the sweep had never covered, and a lockout read under the identifier as typed rather than as the limiter keys it. Two of my own tests were rewritten after their mutations survived, one that asserted a script appeared somewhere on a documentation page rather than in each tree that draws it, and one that proved a delivery was not waited on by hanging when it was. Two mutations are left green on purpose and the commits say so: a bound's number can be changed in the one place every schema, column and test reads it from, and a lock stricter than it needs to be still passes.

The documentation is held to the code by tests rather than by review. Every python block's project imports have to resolve, the documented baseline command has to name the database it runs against, no documented block may bring a stack up from backend/, both script trees have to draw every runnable script, and the tracing recipe has to wire itself outside the generated wiring.


Verification

The architecture

  • All nine presets pass all six checks, core-only through everything
  • Each generated wiring module matches the committed one by AST, for the features this repository carries
  • A feature's tables leave with it: the baseline's tables and indexes are compared against the models, per preset
  • src/interfaces/main.py survives a regeneration unchanged, which is what the tracing recipe relies on

Authorisation

  • A key scoped to nothing cannot rename its owner's account; one scoped user.update can
  • A superuser's scoped key cannot assign a role its scope does not name
  • Deleting or emptying a role is refused while an account stronger than the caller holds it, and allowed once that account is soft-deleted
  • A role carrying a permission the registry no longer knows can still be renamed, emptied and deleted
  • Every /api-keys/ route refuses a key and answers a session, read off the app rather than from a list

Races

  • Two assignments of one role answer 201 and 201, and two replacements leave the role carrying one set of permissions rather than both
  • A superuser's stronger assignment to the same account waits for the write in flight instead of landing between its check and its insert
  • Two role writes sharing a holder run at once, and an unrelated row whose foreign key names that account never waits
  • A second rate-limit rule for one tier and path is refused, and a soft-deleted rule leaves the path free to configure again

Answers

  • A recovery request for a registered address and one for an unknown address answer identically, with and without a worker, and neither waits for the mail server
  • A signup for a taken address answers 201 with nothing created and notifies the owner; a taken username still answers 409
  • An emailed link's destination is bounded in the body and in the OAuth query, and an oversize one starts no flow
  • An account deleted through the panel while an assignment is being written waits for it, and the assignment answers 201

Operations

  • Readiness is 503 for an unreachable database, schema, limiter or session store, and 200 for a cache or broker that is down
  • A cache pointed at the limiter's host with the wrong password is reported down, not ready
  • The panel's login failures lock an operator out of the panel and the API alike
  • pre-commit run --all-files passes

A fresh clone

  • A clone, uv sync, bp deploy generate local and docker compose up bring up the API, the worker, the scheduler, Postgres and Redis, with the schema built and the admin seeded
  • Signing in, signing up, minting a key, spending it against the routes its scope names, and reading the usage rows back all work against that stack
  • A recovery request for a registered address and one for an unknown address answer within a few milliseconds of each other, with the mail sealed in Mailpit and the link spent once
  • 429 carries Retry-After, a production environment refuses the development defaults, and the generated baseline carries uq_rate_limits_tier_path with its WHERE, the varchar(2048) and both token_version columns

The run found five things the suite could not see, each fixed in this branch. The documented Docker quickstart had no generate step and ran compose from backend/, where the compose file no longer is. .env.example shipped an ADMIN_PASSWORD the password policy refuses, so the first seed half-succeeded and said little about why. Alembic inside the container found no configuration, because the stages work from /app/src and alembic.ini sits at /app. Every container command opened with a warning about an environment file a container never has. And bp env validate audited for production while its findings named the environment the configuration was read in. The pages a reader follows are swept by a test now, as the samples and the script trees already were.

A second run, on a fresh clone of the upgraded branch, found one more. The generated compose mapped the database credentials through ${POSTGRES_PASSWORD:-postgres} in the postgres service's environment block, which compose fills from the shell rather than from the env_file beside it, and which overrides that file: a password set in backend/.env was replaced inside the container by the literal postgres, so the database initialised with the well-known default and the API could not connect with the password it had been given. All three modes carried it. The block is gone, since env_file already supplies the names the image reads, and the readiness probe resolves the user inside the container now. Both are pinned by a test over the rendered compose for each mode.


Dependencies

One pin moved for the work itself: crudauth[all]>=0.8.2,<0.9.0, up from >=0.7.1,<0.8.0.

The rest is security. Every dependency with a published advisory is at a patched release in the lock, which is what the image installs from: aiohttp, anyio, certifi, Mako, multidict, pydantic-settings, PyJWT, python-multipart, sqladmin, starlette and urllib3. Two floors this project declares moved with them, sqladmin>=0.28.0 and pydantic-settings>=2.14.2, and anyio>=4.15.1 and starlette>=1.3.1 are declared for the first time, because the task groups, middleware, requests and responses the code imports come from those two rather than through FastAPI. sqladmin takes a request in export_data as of 0.28, which the CSV export mixin accepts and passes on, and which is where the floor is set: on 0.27 that call would raise TypeError.

Breaking changes

The changelog carries every one of them with its migration step. These are the ones a caller or a client notices.

  • An API key's scope is a list of registry permission names. usage_limits may name only the three windows the limiter counts, key_permissions is gone, and src/modules/api_keys/__init__.py exports nothing.
  • Minting a key answers 403 until the account's address is verified.
  • POST /api/v1/users/ answers 201 with a fixed message whether the address was free or taken, and no longer returns the created profile. A client reading response.json()["id"] has to read the account back after signing in.
  • POST /auth/refresh-csrf is now POST /auth/csrf/refresh, and logout-all answers {"detail", "revoked"} where it answered {"message", "terminated_count"}.
  • PATCH /users/{username} no longer takes email. An account moves its address through the confirmed flow, which ends every session it had.
  • Upgrading signs every user out once, and so does a SECRET_KEY rotation from now on.
  • Rate-limit rows naming a concrete path stop matching. Rewrite them as the route template the router declares.
  • items_per_page above 100 answers 422, as do out-of-range ids, pages and expiries, and text no column can hold.
  • A profile image URL and a redirect_to are held to 2048 characters, and a usage row keeps the first 512 of a user agent.
  • An empty CORS_ORIGINS allows nothing, an unknown SESSION_BACKEND is refused at startup, and CREATE_TABLES_ON_STARTUP is refused in production.
  • The admin panel is off until a project asks for it. ADMIN_ENABLED defaults to false, and a panel whose ADMIN_USERNAME or ADMIN_PASSWORD is empty is not mounted, since it could only ever serve a login form.
  • The caller-resolving dependency aliases moved to infrastructure/auth/deps.py.

What a deployment has to do, in order

  1. Regenerate the compose file for your mode, bp deploy generate local|prod|nginx, before anything else runs through it. The one you have takes the database credentials from the shell rather than from backend/.env, so a stack that worked was running Postgres on the default password; if the volume was initialised with it, recreate the volume and restore, or change the password inside the database to the one backend/.env carries.
  2. Run scripts/cleanup_api_key_json.py and rescope the keys it reports.
  3. Find the rows the two new indexes will refuse and fix them: two live rate-limit rules for one tier and path, and any profile_image_url longer than 2048. Migrations has the query for each, next to the one for addresses that differ only in case.
  4. Generate the baseline revision and apply it with the compose migrate profile. It adds user.token_version and api_keys.owner_token_version, both defaulting to 0, the (tier_id, path) index and the length on profile_image_url, and it drops key_permissions.
  5. Rewrite rate-limit rows as route templates, and set CORS_ORIGINS explicitly.
  6. Decide KEY_USAGE_RETENTION_DAYS and schedule the prune: the taskiq task where there is a worker, scripts/prune_key_usage.py from cron where there is not.
  7. Expect one round of sign-ins, and tell accounts holding keys to verify their address before minting another.
  8. Set ADMIN_ENABLED=true, with ADMIN_USERNAME and ADMIN_PASSWORD, if you were reaching /admin without setting it. Unset, it is off now, and the panel is not mounted without both credentials.

…pose file, and seed with a password the policy accepts
@igorbenav
igorbenav merged commit b2c2900 into main Oct 25, 2026
26 checks passed
@igorbenav
igorbenav deleted the refactor/composable-features branch October 25, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants