Skip to content

feat(lint): accept model file paths in the lint command - #6053

Open
tripleaceme wants to merge 2 commits into
SQLMesh:mainfrom
tripleaceme:lint-model-paths
Open

feat(lint): accept model file paths in the lint command#6053
tripleaceme wants to merge 2 commits into
SQLMesh:mainfrom
tripleaceme:lint-model-paths

Conversation

@tripleaceme

Copy link
Copy Markdown
Contributor

Description

Closes #6021.

sqlmesh lint could only select models by name with --model, so path-based tooling such as pre-commit — which passes the names of the changed files — could not drive it without a wrapper that mapped paths back to model names.

This adds a positional PATHS argument to sqlmesh lint, mirroring the shape of sqlmesh format:

sqlmesh lint --local --use-project-index models/a.sql models/b.py

Behaviour:

  • Each path is resolved to the model(s) defined in that file. SQL and Python model files both work.
  • Paths can be combined with --model. A model selected both ways is linted once.
  • No paths and no --model still lints every model, unchanged.
  • A path that defines no model (a typo, a non-model file) is rejected with a clear error instead of silently falling back to linting the whole project:
    Error: No models were found at the following path(s): models/typo.sql
    
  • --local and --use-project-index keep working when the selection comes from paths.

Implementation note

--use-project-index needs the paths before models are parsed, so that only the selected files and their upstream dependencies are loaded. The persistent index can only be read from inside Loader.load() (_model_index_id() depends on the macro/signal/audit mtimes that the load tracks), so rather than resolving paths up front, the selected paths are plumbed through Context.loadLoader.load_load_models alongside the existing model_fqns. SqlMeshLoader._selected_model_paths then seeds its selection from both the requested FQNs and the requested paths, and the existing upstream-closure and stale-index fallbacks apply unchanged.

The final model set is always resolved from the loaded models via Model._path, which is what produces the "no models at this path" error and keeps the multi-project case (one Loader per --paths) correct.

Test Plan

New tests:

  • tests/core/test_context.py::test_lint_models_by_path — a path selects only the models in that file; unrelated violations are not reported; Python model files work; relative paths resolve against the cwd; paths and --model combine and de-duplicate; an unknown path raises; error-level violations still raise.
  • tests/core/test_context.py::test_lint_models_by_path_with_project_index — asserts _load_sql_models is called with selected_paths == {a.sql, b.sql} for a path-selected model with one upstream, and that an unknown path is rejected before any models are loaded.
  • tests/core/test_context.py::test_lint_models_by_path_without_index_falls_back_to_full_load — a missing index falls back to a full load.
  • tests/cli/test_cli.py::test_lint_paths, test_lint_relative_path, test_lint_unknown_path — CLI coverage for single/multiple paths, paths plus --model, relative paths, --local/--use-project-index, and both unknown-path cases.
  • tests/cli/test_cli.py::test_lint_model_scopes_validation_with_multiple_projects — extended to cover selecting the same model by path in a multi-project context.

Also verified by hand on a fresh sqlmesh init duckdb project with rules: "ALL" and an added SELECT * model: linting a single file reports only that model, linting with no arguments still reports all three, an unknown path and an audit file both error out, and --use-project-index produces the same result on the run that builds the index and on the run that reads it.

make fast-test passes (2619 passed). The 5 pre-existing failures in tests/utils/test_git_client.py and test_expand_git_selection_integration reproduce identically on an unmodified checkout in my environment and are unrelated to this change.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

`sqlmesh lint` could only select models by name with `--model`, so
path-based tooling such as pre-commit — which passes the names of the
changed files — could not drive it without a wrapper that mapped paths
back to model names.

Add a positional `PATHS` argument to `sqlmesh lint`, mirroring the shape
of `sqlmesh format`. Each path is resolved to the model(s) defined in
that file, and paths can be combined with `--model`; a model selected
both ways is linted once. Linting with no selection still lints every
model.

Model file paths are plumbed into the load path alongside `model_fqns`,
so `--use-project-index` scopes a path-based selection the same way it
scopes `--model`: only the selected models and their transitive upstream
dependencies are loaded, resolved and validated. A path that defines no
models is rejected with a clear error rather than silently falling back
to linting the whole project.

Closes SQLMesh#6021

Signed-off-by: Adegbite Ayoade <tripleaceme@gmail.com>
@mday-io mday-io self-assigned this Sep 11, 2026
@mday-io
mday-io self-requested a review September 11, 2026 12:41
@mday-io

mday-io commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Two things worth a look before merge:

1. Perf: the indexed path lookup does a filesystem call per model, not per selection

In SqlMeshLoader._selected_model_paths, matching by --model name is a pure in-memory dict lookup (fast, no I/O, which is the whole point of --use-project-index). But matching by path does this:

selected.update(
    fqn for fqn, path in model_to_path.items() if path.resolve() in model_paths
)

path.resolve() runs once per model in the whole project's index, not once per path the user actually asked for. On a big project (thousands of models - exactly who uses --use-project-index), that's thousands of stat/symlink-resolution syscalls just to find the one or two files someone selected. It'll still be way cheaper than a full parse, so nobody will notice on small-to-medium projects, but it does undercut the "index avoids touching unrelated files" guarantee for large ones.

Suggest normalizing the index side the same cheap way the paths are already built (config_path / relative_path, lexical join, no symlink resolution), rather than calling .resolve() per entry. Only the handful of user-supplied paths need the expensive resolve, not every entry in the index.

2. Docs: the added description text doesn't match sqlmesh lint --help

docs/reference/cli.md mirrors real --help output for every command. This PR adds "Models can be selected by name with --model, by model file path, or by both." to the lint description, but that sentence isn't actually in the CLI's docstring, so running sqlmesh lint --help doesn't print it. Either add that sentence to the lint command's docstring in cli/main.py so it's real, or drop it from the docs page so the page stays an accurate mirror.

Matching a selected path walked every entry in the index and called
path.resolve() on each one, so selecting a single file cost one filesystem
call per model in the project — the opposite of what --use-project-index is
for. Resolve the project root once and key the index by that instead, turning
the match into a dict lookup per path the user actually asked for.

A model file that is itself a symlink is no longer covered by joining onto the
resolved root, so the previous resolve-based scan is kept as a fallback for
paths left unmatched.

Also move the path-selection sentence into the lint docstring so that it is
real --help output, and mirror it in the CLI reference rather than documenting
text the command never prints.

Signed-off-by: Adegbite Ayoade <tripleaceme@gmail.com>
@tripleaceme

tripleaceme commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, both good catches — fixed in 27057ae.

1. Path lookup

You're right, and it was worse than a normalisation mismatch: the index side was already being built lexically (self.config_path / relative_path), so the .resolve() existed purely to make the two sides comparable at match time, and paid a syscall per indexed model to do it.

Now the project root is resolved once and the index is keyed by resolved_config_path / relative_path, which turns the match into a dict lookup per path the user actually passed rather than a scan over every model.

One thing worth flagging, since it wasn't only a perf change: the old .resolve() also followed symlinks on individual model files, not just the root. Joining onto a resolved root doesn't, so a model file that is itself a symlink would have silently stopped matching. I kept the old resolve-based scan as a fallback for paths still unmatched after the lookup — the common case never reaches it, and the symlink case keeps working. Happy to drop the fallback if you'd rather not support that.

Added two tests: one asserting no .sql path is resolved while matching off the index (it fails on the previous implementation), and one covering the symlinked model file.

2. Docs

Agreed the page should stay an accurate mirror. It's now a second paragraph on the lint docstring, so sqlmesh lint --help actually prints it, and I mirrored the real output on the page. The summary line stays a single terse sentence, matching format and the rest of the commands.

I left the rest of that block alone, though while I was in there I noticed the page's --local text is longer than what the command prints and --model TEXT is really --models, --model TEXT. Both predate this PR, so I've not touched them — happy to fix separately if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sqlmesh lint should accept model file paths

2 participants