History
In November 2008 I started a different project which eventually turned into Piqi.
The initial idea was to develop a query engine for Google Protocol Buffers that would support boolean queries over binary-encoded data structures. I named the project "Protoquery".
It was a lucky coincidence of several factors that led to the Protoquery project:
-
Google Protocol Buffers was released in public in July 2008. It is a brilliant tool built on top of some insightful ideas — the authors really "got it".
-
At the time I was working on query engines at work and I had some ideas that required a new playground.
-
I had been thinking about designing a programming language with advanced data model since 2003 but never gotten too far in actually making it happen. Starting a small sub-language project was a practical way to approach it.
I was planning to implement Protoquery compiler in OCaml and a low-level query engine in C. So I started looking for tools that would allow me to pass data definitions from C++ Protobuf library to OCaml and then pass compiled query representation down to the query engine implemented in C.
I quickly found a tool for the first part — passing data definitions from C++ to OCaml (it was sexp library). However, I couldn’t find tools that would allow automated mapping of rich OCaml data structures into something that can also be understood by low-level languages like C or C++.
In the absence of such tools I would typically choose some encoding and encode/decode necessary data structures manually. But this time, taking advantage of the project’s unconstrained nature, I decided to digress and build the missing tool. The idea was to reuse Protocol Buffers for reading data structures in C and to find a way to encode OCaml data structures into Protocol Buffers format. Basically what I needed was Protocol Buffers support for OCaml.
Since Protocol Buffers data model doesn’t support high-level features of OCaml type system such as variant type and lists (which I thought would be nice to have) I decided to create a higher-level type definition language and then map it into both OCaml and Protobuf types.
First implementation of the type specification language defined its own abstract syntax using a set of OCaml data structures (it is still defined this way as the initial step of Piqi bootstrap procedure). Then the self-definition was used to generate a parser for its concrete syntax (it is much more sophisticated today).
At some point I gave the name for the concrete syntax language. It originates from "ProtoQuery Intermediate (Language)" but since I disliked the idea of using acronym as a name I put "Intermediate" in between "Proto" and "Query" which shortens to "piq". And the type definition language became "piqi" which stands for "Piq Interfaces".
Initially, it meant to be a rather simple and straightforward tool. But while working on implementing piq/piqi functionality needed for Protoquery I started adding extra language features. I was getting more and more thrilled by Piqi challenges and perspectives so after a while I decided to postpone Protoquery to pursue the Piqi project.
The are plans for implementing Protoquery on top of the Piqi data model.
