IO Ninja – programmable terminal emulator/sniffer

The tool I am referring to is a programmable terminal emulator and sniffer called I/O Ninja (from now on I will omit the word “emulator” and simply say “terminal”). I suspect that the definition itself – “terminal/sniffer” – may sound unusual, if not weird. So I’m going to start with the story of its creation.

Bits of history

The very first product made by our company, Tibbo Technology, was the EM100 serial-to-Ethernet module.

A large portion of the development process revolved around designing and implementing network communication protocols for interaction between the module and a virtual serial port of a PC.

This was arduous, hard work. In our cruel world nothing works as intended – well, at least not immediately. So we spent long months of designing and debugging our communications. Here is the list of software tools that we employed:

• Serial terminal;
  
• TCP terminal;
  
• UDP terminal;
  
• Serial monitor;
  
• Ethernet sniffer.

We didn’t have any problems with Ethernet and serial data monitors – even free products available at that time fully satisfied our needs. Things weren’t as smooth on the terminal side of our work.

We found a plethora of free software terminal, but they all had their limitations. First, most of them were not that great at working with binary data. Some did not implement the binary mode at all, and those who did weren’t very convenient. Terminal software we tried also did not have a good HEX editor supporting select, copy, paste, undo, and redo operations. We also found that almost no terminal utilities allowed incoming TCP connections, and none offered support for broadcast UDP packets.

It was after my unsuccessful quest for the required toolset that I’ve got the idea to create a universal all-in-one software package that would incorporate all of the IO facilities we so desperately needed in our work. This is when I thought to myself: Wouldn’t it be awesome if this new software was scriptable and allowed adding custom protocol analyzers, syntax highlighters, and auto-response scripts (that wait for a packet, analyze it, generate and send a reply, wait again, and so on)?

Besides supporting “missing” socket modes, this utility demonstrated the great convenience of its innovative logging approach (which I since patented). The log was presented as a seamless HEX sheet with packets of the same kind stitched together.

I kept working on this utility in my spare time for a year or so. I improved the logging module and implemented a fully functional logging engine. When the engine was ready I recognized that it was suitable for use not only in terminals, but in sniffers and monitors as well. Convinced that I was on the right track, I kept working on the project and, after several more months of development we released our first official version of the product, now called I/O Ninja:

At this stage, I/O Ninja already allowed us to work with all the transports we needed (serial, TCP, UDP), and it did so in two capacities: as a terminal, and as a non-intrusive sniffer/monitor. Different kinds of I/O session were implemented as plugins, which were written, like the software itself, in C++. In theory, this plugin architecture allowed us to extend I/O Ninja’s functionality as much as we needed. We even through about publishing a plugin SDK and thus allow “advanced” users to create their own plugins. In reality we never got around to doing this. Why? Because we wanted to aim higher and implement real programmability.

What we wanted to do was to make I/O Ninja scriptable. Question was, what scripting language should we choose? Should it be Python? Lua? JavaScript? Some of its twins?

Well, just like we had a dream of having a scriptable terminal, we had a vision of what a perfect language for this terminal would look like. And two of the biggest “must haves” on the wish list for this perfect scripting language were:

It’s easy to see why these two features are so desirable. C is the standard choice for system, network, and other low-level programming. You can find the C implementation of any protocol, any checksum calculation algorithm, any cryptographic algorithm. On top of that, C-structs and pointer arithmetic offer the best way of dealing with binary buffers!

Now add to this the ability to run that code safely and not be bugged by pointer-induced problems – and you’ve got yourself a perfect language for writing I/O-related scripts.

I looked around in hopes of finding such a language. To my surprise, nothing was available. The scripting language I needed simply did not exist! I know, right? There are almost 700 languages on the Wikipedia page listing all notable programming languages, and none offered safe-scripting, pointer-supporting version of C!

To fix this ridiculous situation I have invented the Jancy scripting language. While featuring a high level of source and binary compatibility with C, Jancy provides safe pointers and safe pointer arithmetic. It also has a plethora of other delicious IO-related features such as built-in regular expression-based Lexer generator, scheduled function pointers, reactive programming, and much much more.

Once we’ve got the first prototype of Jancy up and running, we immediately started using it as a scripting engine for the new generation of our I/O Ninja software, which was released in late 2014.