Elixir says hello to C++

19Jun 2015

Elixir says hello to C++

In this article, we look at how messages can be sent from an Elixir to a C++ program. The reason why this is interesting is that it allows us to use Elixir and with it the excellent Erlang virtual machine for the core part of our application, and C++ for the User Interface. In this post we lay the foundation for such work.

Technique

Given an Elixir (Erlang) application, various interoperability options exist to integrate programs written in other languages. See the Interoperability Tutorial for how this can be done. While these options are great, they somewhat tightly couple the programs. We aim for a technique that is less specific to Elixir/Erlang.

We use nanomsg as a communication library, and MessagePack to encode and decode messages.

nanomsg supports various transports and scalability protocols. This is very useful, because we can support clients attached through TCP or IPC, and send messages to many clients at once. nanomsg also abstracts the handling of streams, and provides us with a nice message based interface.

MessagePack is used to encode messages. Such messages can be decoded by other languages that support MessagePack (which are many). However, if another serialization library such as Protocol Buffers or Cap’n Proto is preferred, MessagePack can be easily exchanged.

The Protocol

The protocol is extremely simple, but quite robust and flexible. Since nanomsg already provides access to individual messages, we no longer need to worry about handling individual packets. When we receive something in nanomsg, it is a complete message.

On top of this, we use the following protocol.

<<msg type :: size(8)>> <<msgpack data>>

That is, the first byte denotes the type of the following MessagePack encoded data. According to this type, we can decode the data on the receiving end.

The Elixir Side

For the Elixir side of things, we use enm from Basho to interface with nanomsg. Note that enm bundles and therefore supports a specific nanomsg version. At the time of this writing, the version is 0.5-beta. It is important to use the same version on the other side of the communication, otherwise things might not be compatible.

enm is an Erlang port driver for nanomsg. Fortunately, integrating this library is as easy as integrating a pure Elixir library, due to the excellent rebar support in Mix, the Elixir build tool.

For MessagePack, we use msgpax. There are other Elixir libraries for MessagePack if you want to experiment. In my tests, msgpax worked fine.

As usual with Elixir, we start by creating a new project and add the dependencies.

mix new enano

The mix.exs file looks like this.

defmodule Enano.Mixfile do
  use Mix.Project

  def project do
    [app: :enano,
     version: "0.0.1",
     elixir: "~> 1.0",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     deps: deps]
  end

  def application do
    [applications: [:logger, :enm, :msgpax],
     mod: {Enano, []}]
  end

  defp deps do
    [{:enm, github: "basho/enm"},
    {:msgpax, "~> 0.7"}]
  end
end

We can then fetch and compile the dependencies. Again, Mix will correctly build the enm port driver project out of the box, which I find amazing.

mix deps.get
mix deps.compile

The C++ Side

In C++, we create a simple program that receives messages from nanomsg, checks the type, and decodes the MessagePack data. It then prints the decoded values. If the received type equals 42, then the program ends.

#include <cstring>
#include <nanomsg/nn.h>
#include <nanomsg/pair.h>
#include <msgpack.hpp>
#include <iostream>

int main()
{
  int s = nn_socket(AF_SP, NN_PAIR);
  nn_bind(s, "ipc:///tmp/test.ipc");
  void* buf = NULL;
  int count;
  while((count = nn_recv(s, &buf, NN_MSG, 0)) != -1) {
    uint8_t type = 0;
    std::memcpy(&type, buf, 1);
    std::cout << "type is: " << (int)type << "\n";
    if(type == 42) break;
    
    msgpack::unpacked result;
    msgpack::unpack(&result, (const char*)buf+1, count-1);
    msgpack::object deserialized = result.get();
    std::cout << deserialized << "\n";
    nn_freemsg(buf);
    std::cout << std::flush;
  }
  nn_close(s);
  return 0;
}

Assuming the program is in a file called cnano.cc, it can then be compiled and run with

g++ -lnanomsg -lmsgpack -o cnano cnano.cc
./cnano

Sending Messages from Elixir

Once the C++ program has started, we can run some tests and send appropriate messages from Elixir. We can do this interactively with iex, Elixir’s interactive shell.

We change to the Elixir project’s directory and invoke iex to run our project’s default task.

cd enano
iex -S mix

If all goes well, we are in the interactive shell, the enm and msgpax applications are loaded, and we are ready to shoot messages to C++. First, we create the pair socket, connect it, and send a decoded message with an arbitrary type to the receiver. The C++ program should print the values accordingly. We then send type 42, which should end the C++ program.

iex> {:ok, data} = Msgpax.pack(["Greetings", 300, "Spartans"])
iex> {:ok, s} = :enm.pair
iex> :enm.connect(s, "ipc:///tmp/test.ipc")
iex> :enm.send(s, [<<2>>|data])
iex> :enm.send(s, <<42>>)

Hooray!

Note that in the example above, we use the IPC transport. For this to work, both programs need run on the same machine. If you want to try it on different machines, use TCP instead.

Code

A complete implementation of this example can be found on GitHub. It includes a Elixir and a C++/Qt program. The Qt program shows how messages can be received in a separate thread and posted to the Qt event loop. The messages are then read in the GUI thread and showed to the user.

Send messages from Elixir to Qt.

The screenshot shows the Elixir shell where messages can be sent and a Qt window with a single text area.

Conclusion

Even though the code above is rather simple, I think it is conceptually interesting. Using nanomsg, we decouple different parts of our application. This is a bit like what Elixir/Erlang does with processes, but not specific to any language or environment.

At wisol, we currently use Qt5 and plain C++ to build our embedded applications. We are happy with Qt to build the UIs and want to keep it. The core software in C++ on the other hand is sometimes a bit tedious to build. We have enough experience with C++ to find our way around it, but I still want to experiment with alternatives. For me, Elixr/Erlang seems the best environment to build robust and highly available embedded systems.

Tags: programming