fix(expressions): accept a bare true / false as an AND / OR / NOT operand - #3917
fix(expressions): accept a bare true / false as an AND / OR / NOT operand#3917jackylee-ch wants to merge 3 commits into
Conversation
…rand
The BooleanLiteral to AlwaysTrue/AlwaysFalse conversion was attached only to
the whole expression, so a bare boolean reached And/Or/Not as a
BooleanLiteral and failed pydantic validation:
parse("(false) or foo = 1") -> EqualTo(foo, 1)
parse("false or foo = 1") -> ValidationError: 1 validation error for Or
Seeding a filter with `true` and appending clauses is a common way to build
a row_filter, so `row_filter="true and status = 'x'"` raised instead of
scanning. Give `predicate` its own copy of the boolean element that folds to
AlwaysTrue/AlwaysFalse; `literal` and `literal_set` keep the raw
BooleanLiteral, so `foo = true` and `foo in (true, false)` are unchanged.
Co-Authored-By: Claude Code <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
The change is narrowly scoped, aligns with existing AlwaysTrue/AlwaysFalse absorption logic, and is covered by targeted new tests.
Pull request overview
This PR updates the expressions parser so that a bare true/false token used as an operand to and/or/not is folded into AlwaysTrue/AlwaysFalse during parsing, avoiding pydantic validation failures while preserving boolean literals inside comparisons and IN lists.
Changes:
- Introduce a dedicated boolean grammar element for operand positions that folds to
AlwaysTrue/AlwaysFalse. - Adjust the parser’s
predicaterule to use the folded-boolean element instead of the literal-boolean element. - Add tests covering boolean operands in
and/or/not, and pin behavior forfoo = true/foo in (true, false).
File summaries
| File | Description |
|---|---|
pyiceberg/expressions/parser.py |
Adds a separate boolean operand parse element and uses it in predicate so And/Or/Not receive valid BooleanExpression operands. |
tests/expressions/test_parser.py |
Adds regression tests to ensure bare boolean operands parse and boolean literals in comparisons/sets remain unchanged. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Fokko
left a comment
There was a problem hiding this comment.
I'm okay widening this, thanks @jackylee-ch. I left one comment which I think we might want to clean up
The extra `boolean.copy()` is not needed. `handle_always_expression` already did this fold; it was only attached to the whole expression, which is why `(true) and foo = 1` worked and `true and foo = 1` did not -- infix_notation returns the same Forward that a parenthesized sub-expression recurses through, so the fold ran only when an operand happened to take that path. Attach it to `predicate` instead so every operand folds. `literal` and `literal_set` consume the boolean further in, so `foo = true` and `foo in (true, false)` keep the raw `BooleanLiteral`. The top-level attachment is now unreachable and is removed. Tests are unchanged. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Thanks for adding this @jackylee-ch |
|
@Fokko, the integration failure occurs while building the Hive image: apt-get update rejects an expired Debian security repository Release file, before the integration tests run. This appears unrelated to the changes in this PR. Could you please rerun the failed jobs, or let me know if any action is needed on my side? |
Rationale for this change
The
BooleanLiteraltoAlwaysTrue/AlwaysFalseconversion is attached to the whole expression, so a bare boolean used as an operand reachesAnd/Or/Notas aBooleanLiteraland fails pydantic validation. The two spellings of the same filter disagree:Seeding a filter with
trueand appending clauses is a common way to build one, sorow_filter="true and status = 'x'"raises instead of scanning.The parenthesized form works by accident:
infix_notationreturns the sameForwardthat a parenthesized sub-expression recurses through, so the fold runs only when an operand happens to take that path. Attaching it topredicateinstead folds every operand.literalandliteral_setconsume the boolean further in, sofoo = trueandfoo in (true, false)keep the rawBooleanLiteral. The top-level attachment is then unreachable and is removed.Are these changes tested?
Yes,
test_boolean_as_operandcoversand/or/notin both positions, andtest_boolean_as_literal_is_unchangedpinsfoo = trueandfoo in (true, false). The seven operand cases fail without the change; the literal case passes either way.Are there any user-facing changes?
Filters using a bare
true/falseas an operand parse instead of raising.