perf: Resolve schema $refs once to speed up payload validation - #778
Open
IncognitoQuack wants to merge 1 commit into
Open
IncognitoQuack wants to merge 1 commit into
IncognitoQuack wants to merge 1 commit into
Conversation
jsonschema resolves every `$ref` again for each validated payload, which takes up to 40% of the validation time of OCPP 2.0.1 and 2.1 payloads. Inline references to `definitions` once, when a schema is loaded, and use the resulting validator to check if a payload is valid. Invalid payloads are validated again with the original validator, so raised errors don't change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes included in this PR
Performance improvement for payload validation. Validation results and errors don't change.
Current behavior
The OCPP 2.0.1 and 2.1 schemas describe nested types with
{"$ref": "#/definitions/..."}.jsonschemaresolves each of these references again for every payload it validates. When profilingTransactionEventvalidation,referencinglookups (Resolver.lookup()→pointer()→create_resource()) take about a third of the total time. Every incoming and outgoing message is validated, twice per round trip, so this adds up on a CSMS that serves many charging stations.New behavior
Each schema's local references are resolved once, when it's first used.
_validate_payload()checks payloads with the validator for this inlined schema. If a payload is invalid, it's validated again with the original validator fromget_validator(), and that error is raised as before.The inlined validator only decides whether a payload is valid. Errors always come from the original schema. This matters because
str(SchemaValidationError)contains the failing subschema and ends up in thecauseof the CallError. With inlined references, that text would change and could grow by several hundred KB for some v2.1 schemas.Why the inlined validator accepts exactly the same payloads:
Draft4Validatorignores keywords next to$ref, so replacing{"$ref": X}with the target ofXdoesn't change semantics.properties,items,allOf, ...). Values ofenum,defaultetc. are never touched.$ref, a missing definition, or a nestedidthat changes the resolution scope. None of the bundled schemas hit this. A test checks that all of them are fully inlined.$ref, the original validator is reused.get_validator(), its cache and theschemaof the validators it returns are unchanged.Verification
floatandDecimalparsing): 216,720 generated payloads, valid ones and randomly mutated ones (wrong types, missing or extra keys, bad enum values, too long strings, ...), 168,269 of them invalid. Original and inlined validators agree on validity and on every error'svalidator,message,path,relative_schema_pathandinstance. 0 differences.mastervs this branch: the same 108,360 cases through_validate_payload(), hashing the raised exception class,description,details(including thecausetext sent to the peer) and the payload afterwards. The SHA-256 digests are identical.Benchmarks
Apple M5 Pro, Python 3.12, jsonschema 4.26. Best of 5.
_validate_payload():BootNotificationStatusNotificationTransactionEvent(10 sampled values)TransactionEvent(10 sampled values)$ref)Full
ChargePoint.route_message()round trip (validate request, run handler, validate response, send):BootNotification,ASYNC_VALIDATION = FalseTransactionEvent,ASYNC_VALIDATION = FalseBootNotification,ASYNC_VALIDATION = TrueTransactionEvent,ASYNC_VALIDATION = TrueMemory: the validators are cached per process, not per connection. Loading every schema of every version (387 in total) takes 3.84 MB on master and 4.62 MB with this PR. An application uses one or two versions and only loads the schemas it needs, so the real difference is much smaller.
Impact
No breaking changes. The public API and validation results are unchanged, and so are the raised exceptions and their details.
Checklist
tests/test_messages.py: inlining, sharing, fallback for unresolvable refs and nestedids, all bundled schemas being inlined, and errors still being reported with the original schema. The last one fails if errors are raised by the inlined validator.