Piqi 0.5.4 Released
There are two new major features in this release. Although not being immediately useful, they set an important foundation for future development.
The first one is the ability to serialize Piqi modules into the binary format in a way that ensures compatibility with future extensions of the Piqi language.
This is achieved by automatically tagging fields of the Piqi language data structures with hash-codes that are computed from field names. I learned this method from another powerful data serialization system for OCaml — ATD/biniou. The desired effect could also be achieved by assigning unique integer codes for each field manually, as it is done in Google Protocol Buffers, but the manual approach is less elegant and flexible for this use-case.
Another major feature of this release is the ability to embed type definitions in the data streams represented in Piq, JSON and the binary formats. This allows to produce self-describing data streams that include type definitions for the data contained in the stream. A new --embed-piqi flag for piqi convert command will do it automatically. Check the updated Piq documentation and Examples page to see how embedded Piqi modules are represented in Piq.
Release notes:
Now it is possible to embed Piqi type definitions into Piq streams to produce
self-describing data bundles. Piq streams with embedded type definitions can
be represented in Piq, Piq-JSON and Wire formats.
Piqi tools:
- New "piqi convert --embed-piqi" flag for embedding type information into
serialized Piq streams.
- Use hash-based wire codes for Piqi self-specification instead of
automatically-assigned ordinals to ensure forward and backward compatibility
of serialized .piqi specs with regard to future Piqi language extensions.
- Many internal improvements in Piqi compiler compiler, Piqi library
initialization and serialization of .piqi specs.
piqic:
- (ocaml, erlang) Generating more optimal code for variant parsing.
Bug fixes:
- Fixed an integer overflow bug in piqirun-ocaml library on 32-bit platform.
- Fixed missing location information in errors or warnings while converting
from JSON.
- Fixed incorrect field ordering during Piq input/output when Piqi
specification explicitly defines non-ordinal wire codes.
