In this article, we look at how messages can be sent from an Elixir to a C++ program. The reason why this is interesting is that it allows us to use Elixir and with it the excellent Erlang virtual machine for the core part of our application, and C++ for the User Interface. In this post we lay the foundation for such work.
Technique
Given an Elixir (Erlang) application, various interoperability options exist to integrate programs written in other languages. See the Interoperability Tutorial for how this can be done. While these options are great, they somewhat tightly couple the programs. We aim for a technique that is less specific to Elixir/Erlang.
We use nanomsg as a communication library, and MessagePack to encode and decode messages.
nanomsg supports various transports and scalability protocols. This is very useful, because we can support clients attached through TCP or IPC, and send messages to many clients at once. nanomsg also abstracts the handling of streams, and provides us with a nice message based interface.
MessagePack is used to encode messages. Such messages can be decoded by other languages that support MessagePack (which are many). However, if another serialization library such as Protocol Buffers or Cap’n Proto is preferred, MessagePack can be easily exchanged.
The Protocol
The protocol is extremely simple, but quite robust and flexible. Since nanomsg already provides access to individual messages, we no longer need to worry about handling individual packets. When we receive something in nanomsg, it is a complete message.
On top of this, we use the following protocol.
<<msg type :: size(8)>> <<msgpack data>>
That is, the first byte denotes the type of the following MessagePack encoded data. According to this type, we can decode the data on the receiving end.
The Elixir Side
For the Elixir side of things, we use enm from Basho to interface with nanomsg. Note that enm bundles and therefore supports a specific nanomsg version. At the time of this writing, the version is 0.5-beta. It is important to use the same version on the other side of the communication, otherwise things might not be compatible.
enm is an Erlang port driver for nanomsg. Fortunately, integrating this library is as easy as integrating a pure Elixir library, due to the excellent rebar support in Mix, the Elixir build tool.
For MessagePack, we use msgpax. There are other Elixir libraries for MessagePack if you want to experiment. In my tests, msgpax worked fine.
As usual with Elixir, we start by creating a new project and add the dependencies.
mix new enano
The mix.exs
file looks like this.
defmodule Enano.Mixfile do
use Mix.Project
def project do
[app: :enano,
version: "0.0.1",
elixir: "~> 1.0",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps]
end
def application do
[applications: [:logger, :enm, :msgpax],
mod: {Enano, []}]
end
defp deps do
[{:enm, github: "basho/enm"},
{:msgpax, "~> 0.7"}]
end
end
We can then fetch and compile the dependencies. Again, Mix will correctly build the enm port driver project out of the box, which I find amazing.
mix deps.get
mix deps.compile
The C++ Side
In C++, we create a simple program that receives messages from nanomsg, checks the type, and decodes the MessagePack data. It then prints the decoded values. If the received type equals 42, then the program ends.
#include <cstring>
#include <nanomsg/nn.h>
#include <nanomsg/pair.h>
#include <msgpack.hpp>
#include <iostream>
int main()
{
int s = nn_socket(AF_SP, NN_PAIR);
nn_bind(s, "ipc:///tmp/test.ipc");
void* buf = NULL;
int count;
while((count = nn_recv(s, &buf, NN_MSG, 0)) != -1) {
uint8_t type = 0;
std::memcpy(&type, buf, 1);
std::cout << "type is: " << (int)type << "\n";
if(type == 42) break;
msgpack::unpacked result;
msgpack::unpack(&result, (const char*)buf+1, count-1);
msgpack::object deserialized = result.get();
std::cout << deserialized << "\n";
nn_freemsg(buf);
std::cout << std::flush;
}
nn_close(s);
return 0;
}
Assuming the program is in a file called cnano.cc
, it can then be compiled
and run with
g++ -lnanomsg -lmsgpack -o cnano cnano.cc
./cnano
Sending Messages from Elixir
Once the C++ program has started, we can run some tests and send appropriate
messages from Elixir. We can do this interactively with iex
, Elixir’s
interactive shell.
We change to the Elixir project’s directory and invoke iex
to run our
project’s default task.
cd enano
iex -S mix
If all goes well, we are in the interactive shell, the enm and msgpax
applications are loaded, and we are ready to shoot messages to C++. First, we
create the pair
socket, connect it, and send a decoded message with an
arbitrary type to the receiver. The C++ program should print the values
accordingly. We then send type 42, which should end the C++ program.
iex> {:ok, data} = Msgpax.pack(["Greetings", 300, "Spartans"])
iex> {:ok, s} = :enm.pair
iex> :enm.connect(s, "ipc:///tmp/test.ipc")
iex> :enm.send(s, [<<2>>|data])
iex> :enm.send(s, <<42>>)
Hooray!
Note that in the example above, we use the IPC transport. For this to work, both programs need run on the same machine. If you want to try it on different machines, use TCP instead.
Code
A complete implementation of this example can be found on GitHub. It includes a Elixir and a C++/Qt program. The Qt program shows how messages can be received in a separate thread and posted to the Qt event loop. The messages are then read in the GUI thread and showed to the user.
The screenshot shows the Elixir shell where messages can be sent and a Qt window with a single text area.
Conclusion
Even though the code above is rather simple, I think it is conceptually interesting. Using nanomsg, we decouple different parts of our application. This is a bit like what Elixir/Erlang does with processes, but not specific to any language or environment.
At wisol, we currently use Qt5 and plain C++ to build our embedded applications. We are happy with Qt to build the UIs and want to keep it. The core software in C++ on the other hand is sometimes a bit tedious to build. We have enough experience with C++ to find our way around it, but I still want to experiment with alternatives. For me, Elixr/Erlang seems the best environment to build robust and highly available embedded systems.