Parser for arbitrary binary protocols

Rapid growth of IoT industry in recent years produced a plethora of new protocols with varying levels of standardization and adoption.

Quite a lot of these protocols are binary because:

• It allows for greater throughput

• On a device level these protocols are easier to implement

• On a device level they are more resource (CPU/RAM) efficient

Currently InterSystems IRIS offers several functions for work with binary data:

• zzdump

• $zhex

• $char

• $ascii

But protocol implementation is left as a task for the end user.

Removing the time-consuming need for protocol implementation would allow for easier and stronger positioning of InterSystems IRIS as an IoT platform.

One of the possible approaches to protocol implementation is declarative parsing. With declarative parsing user describes the data structure that he wants parsed (i.e.{ name: title, type: string, length: 5}...) in some declarative language (XML, JSON, YAML) and we use this to generate a concrete parser. Afterwards user passes protocol data to the concrete parser implementation and receives parsed values (as objects or locals or $lb).

Kaitai Struct is an OpenSource (MIT license) declarative parser implementation which allows user to generate parser code for several languages: C++, C#, Go, Java, JavaScript, Lua, Perl, PHP, Python, Ruby, Rust. It allows easy language extensibility by keeping most of the process language-agnostic and only the last step (concrete parser generation) is end-language specific.

I propose we add ObjectScript as a Kaitai Struct generation language. It would allow us to generate native ObjectScript parsers easily which would help with presenting InterSystems IRIS an IoT platform especially at a PoC stage.

Alternatives to Kaitai Struct:

• binpac - C++ only

• preon - Java only

The advantage of Kaitai Struct is that it's not aimed at one language form the beginning, so adding a new language is an expected procedure.

Article on the topic