feat(validation): semantic checks for ontology documents

[kayemkim]

Summary

validate.py runs unique-name, reference, arity and SQL checks on core documents, but each one is guarded by datasets, so an ontology document only got the schema. Five defects, one at a time, each came back "Validation PASSED" on main: a concept declared twice, extends naming an undeclared concept, identify_by naming a relationship the concept does not have, a role played by an undeclared concept, and a QName iri whose prefix is not in prefixes.

This adds validate_ontology, run after the schema when the document has an ontology key:

• concept names are unique in the ontology, relationship names unique within their concept
• extends, identify_by and role references resolve to declared concepts and relationships. The built-in concepts from ontology.md (Any, Boolean, Date, DateTime, Decimal, Float, Integer, String) count as declared, and identify_by may name a relationship declared on a supertype, which is how the reference parser resolves it too
• a cycle check on extends
• a QName iri on a concept or relationship must use a prefix from the top-level prefixes map. An iri with // after the scheme or a second colon (http://..., urn:isbn:...) is read as a full IRI

The built-ins, the cycle check and the CI step are the three additions suggested on the dev@ thread. ontology_mappings are left to the schema until #458 settles where they live. One divergence to flag: the parser also treats AnyEntity as a built-in, which ontology.md does not list, so I went with the spec.

Testing

• validation/tests: 106 passed, running the workflow's steps locally on Python 3.11 to 3.14. 16 new tests cover each check, the built-ins, an inherited identify_by, cycles, full IRIs, malformed shapes and the CLI path.
• examples/flights.yaml, the converter's flights.yaml fixture and its round-trip snapshot all pass. The standalone flights.ontology.yaml proposed in Ontology: decouple Ontologies, Semantic Models, and their Mappings #458 passes as well.
• Seven single-defect documents each fail with one error on this branch and pass on main.
• The complete examples in ontology.md pass. The fragments introduced as snippets reference concepts declared elsewhere and fail as expected, so they are not a test target.

The validation CI gains a step running examples/flights.yaml against the ontology schema. It previously only validated the TPC-DS core example.

Written with LLM assistance; I ran every check above myself and read the diff against the neighbouring functions.

Related Issues

Closes #488. dev@ thread: https://lists.apache.org/thread/lltdz30fryypzgvgc0ldd2tbbbhrqt10

Checklist

Validation

• Validation rules in validation/ are updated if the spec changed
• New validation cases are covered by tests

Tests

• All existing tests pass (pytest / CI green)
• New functionality is covered by tests

Compliance

• ASF license headers are present on all new source files (no new files)
• No third-party dependencies are added without PMC/IPMC approval

[RyutoYoda]

The undeclared-prefix check here covers the follow-up I promised in #479 — thanks for picking it up.

One adjacent gap it doesn't touch: prefixes values declare "format": "iri", but the validator is built without format_checker=, so this passes:

prefixes:
  foaf: "not a valid iri at all !!!"

Adding format_checker=FormatChecker() alone wouldn't fix it either — jsonschema registers its iri checker only when the optional rfc3987 package is installed. iri is the only format keyword in either schema, so an inline check is probably lighter than taking a new dependency.

Happy to send a follow-up once this lands — it would collide with your validate.py hunk otherwise — or fold it in here if you'd rather.

[kayemkim]

Thanks, and glad the prefix check closes the loop from #479. I reproduced the gap: format: iri is the only format keyword in either schema, and with jsonschema 4.26 a plain FormatChecker() has no iri entry unless rfc3987 is installed, so the schema side cannot catch that value today.

I would keep it as your follow-up once this lands rather than fold it in, so this PR stays reference checks only and the two hunks do not fight. It would slot into validate_ontology next to the prefix lookup. A scheme plus no whitespace is probably all a prefix value needs to satisfy there. Full RFC 3987 is more than a namespace base has to pass.

[jbonofre]

This check treats any value shaped like x:y as a QName unless it contains // or a second colon. Legal absolute IRIs such as mailto:a@b.org, tel:+1555... and urn:x match that shape, so they are reported as iri uses undeclared prefix 'mailto' and validate.py exits 1 on a valid ontology.

Could we either:

• skip the check when the prefix is a known URI scheme (mailto, tel, urn, http, https, file, ...) or
• only flag it when the prefix is not declared and the value is not a valid absolute IRI?

The current tests only cover http:// and urn:isbn:..., so please add cases for mailto: and tel: (expecting no error), plus a genuine undeclared-prefix case (expecting an error).

[kayemkim]

Good catch, thanks. The check now treats a known URI scheme before the colon (mailto, tel, urn, did, doi and a few more) as a full IRI, case-insensitively, on top of the existing "//" and second-colon rules. The accepted cases gained urn:x, mailto:, tel: and an uppercase scheme, and there is a foaf:Order case next to them that still reports the undeclared prefix; the earlier ex:Order case covers the document with no prefixes at all. 141 validation tests pass locally.

[jbonofre]

LGTM!

The ontology validation looks solid: the built-in concepts match ontology.md and merging extends and relationship names across duplicate concepts behaves as the tests expect.

I rebased from main and resolved the conflict in test_validate.py.

Two minor notes:

• find_extends_cycles.visit is recursive, so an extends chain deeper than about 1000 concepts would raise RecursionError instead of reporting a cycle. That's unlikely in practice, so good for me.
• the ontology checks only run when the earlier checks pass, so a document with both a core error and an ontology error shows them in two rounds. It looks intentional to me, so good for me too 😄