fix(validation): report empty names/refs and survive nested input

[OGsiji]

Summary

Fixes #408. validation/validate.py silently skipped validation or crashed with a raw traceback on several inputs that the JSON Schema itself accepts: name, from, and to are strings with no minLength, so "" is schema-valid and reaches the semantic checks.

Bugs fixed

1. Empty-string names bypassed uniqueness checks. validate_unique_names collected names with a truthiness filter (if d.get("name")), so duplicate "" dataset/field/metric/relationship names were dropped before duplicate detection. Now filtered with is not None.
2. Empty-string relationship endpoints were never flagged. validate_references used if from_ds / if to_ds, so a relationship with from: "" or to: "" silently passed instead of being reported as an unknown dataset. Now is not None.
3. Deeply nested SQL crashed the validator. validate_sql_expression only caught ParseError/TokenError; pathologically nested expressions raised an uncaught RecursionError. Now caught and reported as a [SQL] diagnostic.
4. Deeply nested YAML crashed with a traceback. main() only caught yaml.YAMLError; deeply nested flow collections raise RecursionError during composition. Now caught, exits cleanly.

Tests

Added regression tests in both suites:

• validation/tests/test_validate.py (pytest): empty-name duplicates, missing-name skip, empty from/to reported, missing-endpoint skip, deeply nested SQL diagnostic.
• validation/test_validate.py (unittest): deeply nested YAML exits cleanly without a traceback, empty endpoint reported end-to-end.

All Validation CI steps pass locally on Python 3.11 and 3.12:

• uv run validation/test_validate.py: 41 passed
• pytest validation/tests/: 55 passed
• uv run validation/validate.py examples/tpcds_semantic_model.yaml: PASSED

[kayemkim]

Thanks for picking this up the day after the issue went in, and welcome. Good call adding the cases to both suites, since validation-ci runs the unittest file and the pytest directory separately.

Ran the branch merged into current main (bc7b2c0) through the validation-ci steps on Python 3.11 to 3.14: all green. I reproduced the four #408 cases on main first: two datasets named "" and a relationship with to: "" print Validation PASSED with exit 0, and 5000 nested parentheses in an expression or 3000 nested [ in the file end in a raw RecursionError. On this branch the first two fail with [Unique] and [Reference] lines and the other two exit 1 with the new diagnostics. The next layer down, a custom_extensions dict nested 150 and 300 levels (valid YAML, deep jsonschema recursion), already fails cleanly with a [Schema] error on both trees, so I don't see a remaining gap of the same kind. The 11 Ossie documents under examples/ and converters/** keep their exit codes.

Two small things, neither blocking:

• In validate_sql_expression the first attempt now has except (ParseError, TokenError): pass followed by except Exception: pass; the second covers the first. The noqa: BLE001 markers are inert too, since validation/ has no ruff configuration.
• The new comments are denser than the rest of the file and mostly restate the PR description and test comments. The one-line style used nearby would read more evenly.

Ordering note: #271 (approved, conflicting with main) edits the same block in main(), and merging both conflicts in validate.py and the pytest file. Whichever lands second has a small rebase.

LGTM.

[OGsiji]

Thanks for the thorough review! Addressed both nits in 2c8c4a5:

• Redundant except: collapsed the first parse attempt's except (ParseError, TokenError): pass into the broader except Exception: pass that already subsumed it. ParseError/TokenError stay imported for the second attempt's actual [SQL] diagnostic.
• Inert noqa: BLE001: dropped both, since validation/ has no ruff config.
• Comments: trimmed to the file's one-line style, keeping only the non-obvious "why" (schema has no minLength; deep input surfaces as RecursionError, not the parser/YAML error types).

No behaviour change; validation-ci steps (unittest 41, pytest 55, canonical example) stay green on 3.11 to 3.14.

On the #271 overlap: happy to take the second-mover rebase whenever it lands. The conflict is limited to the main() except ladder and the added pytest cases.

[jbonofre]

The is not None fix here only helps duplicate-name detection see an empty string. It doesn't fix the actual bug: a single (non duplicate) dataset/field/metric/relationship named "" still passes validation entirely, validate_unique_names only fires on 2+ occurrences, and nothing else checks for empty names.

I suggest fixing this at the schema level instead: add minLength: 1 to the name/from/to string properties in core-spec/ossie-schema.json (Dataset, Field, Metric, Relationship, SemanticModel). That rejects "" with a clear [Schema] error before semantic checks even run, and fixes this consistently across all four call sites instead of patching them individually.

[jbonofre]

The fix (about minLength: 1) is helping there too.

This data lookup ({d.get("name"): d for d in ... if d.get("name")}) still uses a truth filter, while the from_ds/to_ds checks two lines below were fixed to is not None. That inconsistency means a dataset actually named "" gets excluded from this dict, so any relationship referencing it is wrongly reporter as "unknown dataset ''" instead of being caught as an invalid name.

[jbonofre]

Broadening this from except (ParseError, TokenError) to bare exception Exception: pass silently swallows any exception type on the first parse attempt, not just the RecursionError this seems aimed at.
It means validate_sql_expression(123, 'ANSI_SQL', 'ctx') (non string expr) used to raise a loud TypeError. Now it's swallowed and the SELECT wrapped retry (parse_one('SELECT 123', ...)) spuriously succeeds, so the function reports the input as valid SQL.

I recommend catching RecursionError specifically here (in addition to ParseError/TokenError) rather than a bare Exception, so genuine bugs still surface instead of being masked as "valid".

[kayemkim]

Following up because one of the two nits in my review pointed the wrong way. I only said the except (ParseError, TokenError) clause was covered by the broader one. What I had in mind was keeping the narrow clause and adding RecursionError to it, not dropping it, and I should have spelled that out.

Measured on this branch merged into current main (df81044): validate_sql_expression(123, "ANSI_SQL", "ctx") returns None because the retry parses SELECT 123 without complaint, so a non-string expression is reported as valid SQL, which is the case raised above. Main raises TypeError there. Changing the first attempt to except (ParseError, TokenError, RecursionError): pass brings the TypeError back, still turns 5000 nested parentheses into the "too deeply nested" line, and both validation suites pass with it (55 pytest cases, unittest OK).

On the empty-name point I can confirm the gap. A document with a single dataset and a single field both named "" prints Validation PASSED on main and on this branch. My earlier reproduction only used the duplicate and dangling-reference shapes from #408, which is why it didn't show up. core-spec/ossie-schema.json has no minLength anywhere today, so minLength: 1 on name, from and to would be the first one, and with it the is not None changes become consistency fixes rather than the thing that closes the bug.

[OGsiji]

Thanks both. Reworked the fix and force-pushed a single commit (174491f) rebased onto current main.

@jbonofre point 1 (schema is the real fix): added minLength: 1 to every required identifier string in core-spec/ossie-schema.json: name on model/dataset/field/metric/relationship, plus dataset source and relationship from/to. Empty values now fail with a clear [Schema] ... should be non-empty before semantic checks run, so a single empty identifier is caught at the contract level, not just duplicates. (@kayemkim: confirmed your single-field-and-dataset "" repro now fails instead of printing Validation PASSED.)

@jbonofre point 2 (consistency): reverted the earlier is not None changes in validate_unique_names and validate_references. With the schema fix they were redundant, and reverting removes the inconsistency you flagged, where the dataset lookup used a truthiness filter while the endpoint checks used is not None. Both are plain again.

@kayemkim's exact suggestion on the SQL guard: the first parse attempt is now except (ParseError, TokenError, RecursionError): pass, and the SELECT-wrapped retry reports the "too deeply nested" line. validate_sql_expression(123, "ANSI_SQL", "ctx") raises TypeError again (verified), while 5000 nested parentheses still produce the diagnostic. No bare except Exception anywhere.

Rebase: carried over cleanly; OSSIE_SQL_2026 (#439) is preserved in the schema and accepted by the validator.

Tests: schema-level empty-identifier cases (unittest and pytest) replace the semantic ones, plus a non-string-expression case. validation-ci is green on 3.11 and 3.12 (unittest 41, pytest 55, canonical example passes).

[jbonofre]

@OGsiji awesome! Many thanks! I'm doing a new pass.

[jbonofre]

It's all good now. Thanks!

I will merge as soon as CI is green.