Hi,
My problem:
I'm using the confluent_kafka python library and I want to use the schema validation feature when producing and consuming messages.
Internally confluent_kafka is using your nice jsonschema library but it seems as if this really slows down the kafka clients (example: producing 10,000 messages without validation takes about 0.0815 seconds and 0.4899 seconds with validation!). We have several million message each day, this slowdown will add up...
What I want to accomplish:
There is also a Rust-based JSON schema validator for python (https://pypi.org/project/jsonschema-rs/) which is - like most Rust-based libraries - blazingly fast and supports the kind of JSON schemas we use. I want to replace the "draft2020-12" validator implementation from jsonschema with the jsonschema-rs implementation.
What I don't understand
At first I thought it is just a matter of subclassing the Draft202012Validator or implementing the Validator Protocol with a class that delegates to the jsonschema-rs implementation and calling jsonschema.validators.validates to register it (so that confluent-kafka will use my implementation for "draft2020-12" schemas)
But then I've found this https://python-jsonschema.readthedocs.io/en/stable/creating/#creating-or-extending-validator-classes and subclassing directly even creates a warning messages. I don't quite understand the 'validators' parameter in jsonschema.validators.create or jsonschema.validators.extend. Do I have to define a validate method for each json schema element? I want just replace the complete validate method with a finished implementation... Is this possible? Can you point me to an example on how to use your intended way for creating or extending to do this?
Best regards,
Christian
Hi,
My problem:
I'm using the
confluent_kafkapython library and I want to use the schema validation feature when producing and consuming messages.Internally
confluent_kafkais using your nice jsonschema library but it seems as if this really slows down the kafka clients (example: producing 10,000 messages without validation takes about 0.0815 seconds and 0.4899 seconds with validation!). We have several million message each day, this slowdown will add up...What I want to accomplish:
There is also a Rust-based JSON schema validator for python (https://pypi.org/project/jsonschema-rs/) which is - like most Rust-based libraries - blazingly fast and supports the kind of JSON schemas we use. I want to replace the "draft2020-12" validator implementation from jsonschema with the jsonschema-rs implementation.
What I don't understand
At first I thought it is just a matter of subclassing the Draft202012Validator or implementing the Validator Protocol with a class that delegates to the jsonschema-rs implementation and calling
jsonschema.validators.validatesto register it (so that confluent-kafka will use my implementation for "draft2020-12" schemas)But then I've found this https://python-jsonschema.readthedocs.io/en/stable/creating/#creating-or-extending-validator-classes and subclassing directly even creates a warning messages. I don't quite understand the 'validators' parameter in
jsonschema.validators.createorjsonschema.validators.extend. Do I have to define a validate method for each json schema element? I want just replace the complete validate method with a finished implementation... Is this possible? Can you point me to an example on how to use your intended way for creating or extending to do this?Best regards,
Christian