Wow. It’s difficult to believe it’s been almost a week since I gave a talk about gossip protocols at Erlang User Conference in Stockholm. It was a fantastic event, great agenda, great topics, fantastic networking. EUC is one of those events you should attend, you will not regret it. No matter if you are interested in Erlang/Elixir or not. Big “Thank You” to all who made it happen.
EUC2015 was also a place of first public appearance of Gossiperl. This was the first time I was giving a public talk to such an audience, I’m not the best at this. Judging by the number of people attending my talk, the subject was good. Judging by the number of people who were there when I was wrapping up, I wasn’t terrible. I do hope that people got the value out of my 45 minutes. And finally, I was able to show Gossiperl.
At the end of my talk I’ve presented a couple of very simple demos:
- event subscribe / dissemination: events arriving only at interested nodes
- Gossiperl running over multicast on Raspberry Pi cluster
These are the very basic Gossiperl services covered in the documentation and the core of what Gossiperl is about. The core is ready. So, what are the next steps?
-
Top priority: Raft, in order to make Gossiperl fully data center aware. The idea behind this one is to make the data centers (
racksin Gossiperl notion) exchange the data between the physical locations only via seeds. This will ensure that the physical locations communicate via established, dedicated, streamlined channel. Currently, all members from different locations communicate with each other. This isn’t very effective because of the latency. As explained in my talk, it is not desired to have overlay members in, say, Oregon talk to members, say, in Singapore. The latency is expected but it should be controllable. This will be achieved with Raft. Only the seeds of those locations will talk to each other. The members of respective locations will talk only to local seeds. -
Priority number two: subscriptions. Gossiperl subscription message redelivery can be configured in two different ways. Either the delivery will be attempted selected number of times or indefinitely. In case of a fixed number, if a message can’t be delivered, the sender will never become aware of this fact. This problem will be addressed by solving the following issue.
-
Number three: subscriptions, again. Well, kind of. Ability to attach a
luggageto thedigest. Such information will be used to build an initial overview of the overlay with application specific data. Similar to membership but domain specific. Subscriptions are available only to members who already participate in an overlay. Any new joiners miss on this data and will never receive it. It’s possible to build a mechanism to resend such information but it is not straightforward. For data which should be available to all new joinersluggageis the answer. -
Next one is exometer. Exometer is de facto the standard for metrics in Erlang world. Like JMX for JVM. Currently only InfluxDB connector is available out of the box and exometer needs to be supported.
-
After that comes an easy one. The only encryption algorithm available (and hard coded) is
AES-256. This will become a configurable option with pluggable behaviours. Gossiperl will ship with configuration forAES-128,AES-192andAES-256. Other algorithms can still be added and selected on per overlay basis. -
The final one is: user interface improvements. Gossiperl ships with a REST API. Next to the REST API a web interface will be available. All management operations must be accessible through the browser and statistics will be streamed via web sockets.
Here we are. I’m sure there will be more.
This post would not be complete without words of appreciation to my managers at Virdata who supported me in this endeavour. You know who you are, thank you.