Skip to content

0.33.4: cap sqlglot <30.18 so scans work on fresh installs, clear lockfile CVEs - #118

Merged
bassim-matar merged 2 commits into
mainfrom
wip-20261007-174134
Oct 7, 2026
Merged

bassim-matar merged 2 commits into
mainfrom
wip-20261007-174134

Conversation

@bassim-matar

@bassim-matar bassim-matar commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

  • fix (user-facing): on a fresh install (Python ≥ 3.10), Excel/ODS, Iceberg and partitioned-Parquet scans fail with ParserException: syntax error at end of input. Cap sqlglot<30.18 until ibis ships the upstream fix. Bumps to 0.33.4 so the fix reaches PyPI.
  • deps (lockfile): clear the advisories failing the CI audit (fsspec, multidict) and bump pyjwt. This supersedes Dependabot's #117.
  • tests (Windows CI): build SQLite URLs portably. The windows-3.13 leg broke with Python 3.13.16; this is test-only, with no product change.

Root cause 1: ibis 12 + sqlglot ≥ 30.18

ibis 12.0.0 drops the view behind an in-memory create_table by passing a raw string to sge.Drop. sqlglot 30.18 (2026-09-03) stopped coercing that string, so the SQL becomes a nameless DROP VIEW IF EXISTS and DuckDB rejects it. ibis 12.0.0 puts no upper bound on sqlglot, so any fresh resolution gets ibis 12.0.0 + sqlglot 30.21.

Why CI only failed on 3.10-lowest: that job's make test (uv run) re-resolves to the newest versions after uv sync --resolution lowest, so it is the only leg that saw ibis 12. The 3.14 leg uses the lock (ibis 11), and 3.9 cannot install ibis 12.

Root cause 2: Windows leg and Python 3.13.16

Database tests built sqlite:////{tmp_path}, the POSIX-only absolute form, which gives the path \C:\Users\… on Windows.

  • Before: this only worked because ntpath.realpath rebuilt paths with ntpath.join, whose drive semantics silently dropped the current drive (D:\ + C:\… → C:\…).
  • Now: CPython 3.13.16 concatenates instead, so Path.resolve() returns D:\C:\Users\… and stat() fails with WinError 123. The WinError 32 teardown errors follow from the database left open.

Tests now use sqlite:///{path.as_posix()}, like tests/test_incremental_database.py. This is also the documented form and the URL add_database builds itself. The URL is unchanged on POSIX.

Changes

  • pyproject.toml: add "sqlglot<30.18", with a comment saying when to drop it. Version 0.33.4.
  • CHANGELOG.md: 0.33.4 entry.
  • uv.lock: targeted upgrades only:
    • fsspec 2026.2.0 → 2026.9.0 (CVE-2026-104851). s3fs, gcsfs and adlfs pin fsspec exactly, so they move with it.
    • multidict 6.7.1 → 6.9.1 (CVE-2026-104874)
    • pyjwt 2.13.0 → 2.15.1
  • tests/database/test_{sqlite,db_introspect,geopackage}.py: portable SQLite URLs (37 lines).

Verification (local, replaying CI commands)

Job Result
audit No known vulnerabilities (was: fsspec, multidict)
lint / typecheck ruff clean, pyright 0 errors
3.14 highest (lock) 2002 passed, coverage 100%
3.10 lowest (same uv sequence as CI, uv 0.12.23) 2002 passed (was 21 failed, 53 errors)
3.9 lowest 1982 passed
windows 3.13 cause identified from the CI log; parsing sqlite:///C:/… simulated → absolute C:/… path. This PR's CI is the end-to-end check

Notes

  • pandas 3, out of scope: with a fully fresh resolution, test_read_statistical_mixed_types fails because it asserts dtype == object and pandas 3 infers str. The test is too strict; the package behaves correctly. No current CI leg resolves pandas 3.
  • Python 3.9: fsspec stays at 2025.10.0, the last 3.9-compatible release. Same situation as the aiohttp gate; the CI audit does not evaluate the 3.9 branch.

🤖 Generated with Claude Code

bassim-matar and others added 2 commits October 7, 2026 17:55
…kfile CVEs

ibis 12.0.0 drops the view backing an in-memory create_table with a raw
string passed to sge.Drop; sqlglot 30.18 stopped coercing it, so the
statement renders as a nameless `DROP VIEW IF EXISTS` and DuckDB raises
ParserException. ibis 12.0.0 puts no upper bound on sqlglot, so any fresh
install on Python >= 3.10 resolves ibis 12.0.0 + sqlglot 30.21 and breaks
Excel/ODS, Iceberg and partitioned-Parquet scans. Fixed upstream in
ibis-project/ibis#12104 but not released yet (ibis-project/ibis#12124), so
cap sqlglot below 30.18 (the pin ibis maintainers proposed themselves)
until an ibis release ships the fix. CI only caught it on the 3.10-lowest
leg, whose `uv run` re-resolves to the newest versions; the 3.14 leg uses
the lock (ibis 11) and never saw it.

Refresh the lockfile to clear the advisories failing the CI audit:
fsspec 2026.2.0 -> 2026.9.0 (CVE-2026-104851; s3fs/gcsfs/adlfs pin fsspec
exactly, so they move with it), multidict 6.7.1 -> 6.9.1 (CVE-2026-104874),
and pyjwt 2.13.0 -> 2.15.1, superseding Dependabot's #117.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…3.13.16+

Database tests built `sqlite:////{tmp_path}`, the POSIX-only absolute form.
On Windows it yields the path `\C:\Users\...`, which only worked because
ntpath.realpath used to rebuild paths with ntpath.join, whose drive
semantics silently dropped the current drive (`D:\` + `C:\...` -> `C:\...`).
CPython 3.13.16 joins by plain concatenation instead, so Path.resolve()
now returns `D:\C:\Users\...` and stat() fails with WinError 123 (the
WinError 32 teardown errors follow from the database left open).

Use `sqlite:///{path.as_posix()}` like the other tests, matching the
documented form and the URL add_database builds itself. The URL is
unchanged on POSIX, and on Windows it becomes `sqlite:///C:/...`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bassim-matar
bassim-matar merged commit 01bbf8a into main Oct 7, 2026
8 checks passed
@bassim-matar
bassim-matar deleted the wip-20261007-174134 branch October 7, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant