I found xq use serde as its deserializer, so it is possible to support other format beside json (yaml for example).
cat a.yaml | xq --fmt yaml
cat b.data | xq --fmt msgpack
and one more step, support schemaless data with additional schema
cat b.bin | xq --fmt bincode --schema my-bincode-schema.json --schema-fmt json