Skip to content

perf: Resolve schema $refs once to speed up payload validation - #778

Open
IncognitoQuack wants to merge 1 commit into
mobilityhouse:masterfrom
IncognitoQuack:perf/inline-schema-refs
Open

IncognitoQuack wants to merge 1 commit into
mobilityhouse:masterfrom
IncognitoQuack:perf/inline-schema-refs

Conversation

@IncognitoQuack

Copy link
Copy Markdown

Changes included in this PR

Performance improvement for payload validation. Validation results and errors don't change.

Current behavior

The OCPP 2.0.1 and 2.1 schemas describe nested types with {"$ref": "#/definitions/..."}. jsonschema resolves each of these references again for every payload it validates. When profiling TransactionEvent validation, referencing lookups (Resolver.lookup() → pointer() → create_resource()) take about a third of the total time. Every incoming and outgoing message is validated, twice per round trip, so this adds up on a CSMS that serves many charging stations.

New behavior

Each schema's local references are resolved once, when it's first used. _validate_payload() checks payloads with the validator for this inlined schema. If a payload is invalid, it's validated again with the original validator from get_validator(), and that error is raised as before.

The inlined validator only decides whether a payload is valid. Errors always come from the original schema. This matters because str(SchemaValidationError) contains the failing subschema and ends up in the cause of the CallError. With inlined references, that text would change and could grow by several hundred KB for some v2.1 schemas.

Why the inlined validator accepts exactly the same payloads:

  • Draft4Validator ignores keywords next to $ref, so replacing {"$ref": X} with the target of X doesn't change semantics.
  • Only subschema keywords are walked (properties, items, allOf, ...). Values of enum, default etc. are never touched.
  • The original schema is used unmodified if something can't be inlined safely: a non-local or recursive $ref, a missing definition, or a nested id that changes the resolution scope. None of the bundled schemas hit this. A test checks that all of them are fully inlined.
  • Resolved definitions and subtrees without references are shared, not copied. For schemas without any $ref, the original validator is reused.

get_validator(), its cache and the schema of the validators it returns are unchanged.

Verification

  • Differential test across all 387 bundled schemas (1.6, 2.0.1, 2.1, both float and Decimal parsing): 216,720 generated payloads, valid ones and randomly mutated ones (wrong types, missing or extra keys, bad enum values, too long strings, ...), 168,269 of them invalid. Original and inlined validators agree on validity and on every error's validator, message, path, relative_schema_path and instance. 0 differences.
  • End to end, master vs this branch: the same 108,360 cases through _validate_payload(), hashing the raised exception class, description, details (including the cause text sent to the peer) and the payload afterwards. The SHA-256 digests are identical.

Benchmarks

Apple M5 Pro, Python 3.12, jsonschema 4.26. Best of 5.

_validate_payload():

message master this PR speedup
2.0.1 BootNotification 17.7 µs 11.2 µs 1.58x
2.0.1 StatusNotification 12.4 µs 9.2 µs 1.35x
2.0.1 TransactionEvent (10 sampled values) 275.4 µs 155.5 µs 1.77x
2.1 TransactionEvent (10 sampled values) 180.7 µs 101.9 µs 1.77x
1.6 (any, schemas barely use $ref) unchanged, within noise

Full ChargePoint.route_message() round trip (validate request, run handler, validate response, send):

master this PR speedup
2.0.1 BootNotification, ASYNC_VALIDATION = False 45.6 µs 33.5 µs 1.36x
2.0.1 TransactionEvent, ASYNC_VALIDATION = False 321.8 µs 202.7 µs 1.59x
2.0.1 BootNotification, ASYNC_VALIDATION = True 139.5 µs 120.9 µs 1.15x
2.0.1 TransactionEvent, ASYNC_VALIDATION = True 414.4 µs 291.1 µs 1.42x

Memory: the validators are cached per process, not per connection. Loading every schema of every version (387 in total) takes 3.84 MB on master and 4.62 MB with this PR. An application uses one or two versions and only loads the schemas it needs, so the real difference is much smaller.

Impact

No breaking changes. The public API and validation results are unchanged, and so are the raised exceptions and their details.

Checklist

  1. Does your submission pass the existing tests?
  2. Are there new tests that cover these additions/changes? In tests/test_messages.py: inlining, sharing, fallback for unresolvable refs and nested ids, all bundled schemas being inlined, and errors still being reported with the original schema. The last one fails if errors are raised by the inlined validator.
  3. Have you linted your code locally before submission?

jsonschema resolves every `$ref` again for each validated payload, which
takes up to 40% of the validation time of OCPP 2.0.1 and 2.1 payloads.

Inline references to `definitions` once, when a schema is loaded, and use
the resulting validator to check if a payload is valid. Invalid payloads
are validated again with the original validator, so raised errors don't
change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant