Embeddable scripting languages are for integration testing too!

tl;dr needed some fake clients, went the scripting route instead of building several little executables.

What?

At work, among other things, I'm working on a debugging/postmortem analysis component for a bigger, complex and not-so-tested system. This little service is really important and used during integration tests, so "it has to work".

While I managed to find and remove lots of bugs early (based TDD!), some behaviors of the service "as a whole" were not tested as much as I'd like.

Explaining the service a little more

Please, don't close the tab and keep reading. I need to explain some more details about the service I want to put under test.

This little "thing" receives messages via ZeroMQ, has to write them to files and stream the received information to other clients to do some realtime graphs/visual analysis, or whatever they prefer. Clients can communicate to the component using a simple, custom C++ library that "hides" the ZeroMQ client code, so we have to test it, that messages are received correctly and that the commands sent by clients are executed.

For the sake of simplicity, and to give you both an idea of what I'm discussing and a runnable proof of concept, I wrote a toy example on Bitbucket.

An example of the library code is the following:

void ClientClass::logValue(uint8_t value_to_send) {
  std::cout << "sending value: " << (int)value_to_send << std::endl;
  char *buffer = new char[2];
  buffer[0] = 1;
  buffer[1] = value_to_send;

  int res = zmq_send(socket, buffer, 2, 0);
  assert(res == 2);

  delete[] buffer;
}

The idea

At first, I tried to simulate as much cases as possible in a single "testing client" -a fake program to simulate actual clients-, but I understood the hard way that it was a very stupid idea: tests became difficult to extend very, very soon, and it was difficult to modify it to test something without breaking something else. After the enlightment, I started thinking about having several fake clients, each one tailored on the situation I wanted to test (message bursts, ntp client/server malfunctions, etc).

I didn't want to include these little executables to my daily dev build, though: it'd take too much time to do a full rebuild of the system, and time is precious. This constraint made me think about using a scripting language: "just write a script in some sort of DSL and you're done", I thought.

ChaiScript, or: extending an embeddable language

So, I started thinking about how to implement such a language. How difficult it could be? The more I though about it, the more use-cases and features I deemed necessary. From a bunch of log* functions, it evolved into a limited but general-purpose language with variables, functions and cycles. Building your own language is a cool, fun and "formative" experience, I didn't have the time to actually write an interpreter or a compiler and maintain it in addition to the real project. The right thing to do was to pick an existing solution and integrate it inside the real project.

So, after a little searching on the Wired, I came across ChaiScript, a simple Javascript-like language written in C++, designed to be embedded by C++ applications. The usual cmake . && make && make install will build and install it from sources.

Let's write the server first...

As said before, the server is just a simple ZeroMQ ROUTER in a DEALER-ROUTER communication scheme. For the sake of simplicity, in this article we will work on a simplified example, a proof of concept.

The protocol between the server and its clients is very simple: messages cannot be bigger than 20 bytes, and two kinds of payloads can be sent.

byte  0: message_type (0=ascii, 1=number)
bytes 1-20: payload

If message_type is 0, an ASCII string is sent to the server, if 1 it's a unsigned int number between 0 and 255 -only one byte will be used to store it-.

#include <zmq.h>
#include <iostream>
#include <cstring>

int main() {
  void *context = zmq_ctx_new();
  void *socket_ = zmq_socket(context, ZMQ_ROUTER);

  // set a timeout
  unsigned int recv_timeout_ms = 10000;
  zmq_setsockopt(socket_, ZMQ_RCVTIMEO, &recv_timeout_ms,
                 sizeof(recv_timeout_ms));

  if (zmq_bind(socket_, "tcp://127.0.0.1:6987") == -1) {
    std::cout << "err: could not bind, " << zmq_strerror(zmq_errno())
              << std::endl;
    return 1;
  }

  // needed to skip identity messages
  bool is_recv_message_identity = false;
  bool timeoutted = false;

  while (!timeoutted) {
    char buffer[20];
    bzero(buffer, 20 * sizeof(char));
    int recv_res = zmq_recv(socket_, buffer, 19, 0);

    if (recv_res == -1) {
      if (zmq_errno() == 11) { // EAGAIN
        timeoutted = true;
        continue;
      } else {
        std::cout << "err: could not receive [" << zmq_errno() << "] "
                  << zmq_strerror(zmq_errno()) << std::endl;
        return 2;
      }
    }

    // skip identity messages: they're not the payload we are looking for,
    //      and we don't need to do identity-based logic in this PoC.
    is_recv_message_identity = is_recv_message_identity ? false : true;
    if (is_recv_message_identity)
      continue;

    if (buffer[0] == '\0') {
      std::cout << "received a string!" << std::endl;
      std::cout << "the received string is: >>>" << std::string(buffer + 1)
                << "<<<" << std::endl;
    } else {
      std::cout << "received a number!" << std::endl;
      std::cout << "the received number is: " << (int)buffer[1] << std::endl;
    }
  }

  int sock_res = zmq_close(socket_);
  if (sock_res == -1)
    return sock_res;
  int ctx_res = zmq_ctx_shutdown(context);
  return ctx_res;
}

... then, the client script runner

Being a proof of concept discussed here, we'll keep things simple and limit our example to just two functions: logDebug(str) and logValue(unsigned int), that send a string or an unsigned int to the server via ZeroMQ.

#include <chaiscript/chaiscript.hpp>
#include <chaiscript/utility/utility.hpp>
#include <iostream>
#include <mahlib.h>

int main(int argc, char* argv[]) {
  // handle parameters. if no parameter is given, use "test.chai"
  // as default script name.
  std::string script_name = "test.chai";
  if(argc == 2){
    script_name = std::string(argv[1]);
  }
  else if(argc > 2){
    std::cout << "usage: " << argv[0] << " <script.chai>" << std::endl;
    return 1;
  }

  // create the ChaiScript interpreter
  chaiscript::ChaiScript chai;

  // extend ChaiScript with our ClientClass class,
  // so it can recognize our class' usage in the module,
  // call the right function at the right time, etc.
  chaiscript::ModulePtr m = chaiscript::ModulePtr(new chaiscript::Module());
  chaiscript::utility::add_class<ClientClass>(*m,
     "ClientClass",
     { chaiscript::constructor<ClientClass(const std::string)>()},
     { {chaiscript::fun(&ClientClass::logDebug), "logDebug"},
       {chaiscript::fun(&ClientClass::logValue), "logValue"}
     });
  chai.add(m);

  // the (augmented) interpreter picks the script and executes it.
  // it never asked for this, by the way.
  chai.eval_file(script_name);

  return 0;
}

Time to write scripts!

Let's now write some scripts and test the whole system!

// define a simple function
def foo(n) {
  return n*2+5
}

// define a client, directly inside the script!
var client = ClientClass("tcp://127.0.0.1:6987");

// call client's methods.
client.logValue(foo(33));
client.logDebug("from chai via zmq!");
client.logDebug("hey, what happens if I send a string that is obviously longer than 18 characters? eh? EEEEHHHH???");

What does the server prints out on the console?

# the client is run as:
# LD_LIBRARY_PATH="/home/winter/built/chai/lib/chaiscript/" ./client test.chai

[winter@timeofeve] [/dev/pts/4] [master]
[~/blogging/chai-poc]> make run_server
./server
received a number!
the received number is: 71
received a string!
the received string is: >>>from chai via zmq!<<<
received a string!
the received string is: >>>hey, what happens <<<

Let's now show another script we can run.

// define a client, directly inside the script!
var client = ClientClass("tcp://127.0.0.1:6987");

client.logDebug("hello!");

for(var i=65; i<92; ++i){
    client.logValue(i);
}

client.logDebug("goodbye!");

And the result is...

# the client is run as:
# LD_LIBRARY_PATH="/home/winter/built/chai/lib/chaiscript/" ./client wow.chai

[winter@timeofeve] [/dev/pts/4]
[~/blogging/chai-poc]> make run_server
./server
received a string!
the received string is: >>>hello!<<<
received a number!
the received number is: 65
[snip]
received a number!
the received number is: 91
received a string!
the received string is: >>>goodbye!<<<

What's the magic? We were able to write two scripts in a higher-level language and use the same application to run them, without even compiling once!

Would you want to test this automatically, you may write a simple script in Python that starts the server, launches the client with the right script and then checks the content of the files generated by the server! That's just an idea, though: your needs define how you'd write your integration tests!

The nice turnout: faster experimentation and bug-hunting

It didn't took long to discover -and be favourably surprised!- that the new scripted client allows for a lot of experimentation.

In particular, I'm now working on the graphical user interface of one of the systems that receive data from the server, and it let me check edge cases in the graphs interaction, in real time.

The workflow is very simple:

  1. Have a doubt/want to test a different use case
  2. Write a simple script, changing the order of the function calls, how data are generated, etc
  3. Run the new script, with the same client(!)
  4. Check how GUI reacts and fix glitches

So, you now have both some code that can be run to verify that issues are solved. Easy and useful, isn't it?

I'll throw it in too: as the whole thing is written in C++, I can use Suzuha (with a couple of ad-hoc function calls) inside the host application. Suzuha is a simple shim for gettimeofday(3) and settimeofday(3), so I can make clients send messages from the past (or the future!) without having to recompile them.

Nice. Isn't it a bit overkill though?

Probably. I just want to give my clients something that works. Being able to anticipate possible problems and be sure that bugs found on the field won't reappear anymore is very important, and I think that thorough testing is one of the best ways to achieve it.

Do you like the idea? Do you want to suggest something better, or share your experience? Send me a tweet , or consider offering me a Ko-Fi !

Hacktoberfest, or: help friends and get T-shirts

In one of my previous posts (the one about moesearch), I said I wanted to speak about the things I contributed to in October. I was waiting for the #hacktoberfest T-shirt. Now, that tee is here.

How did I earn it? Pushing patches and improvements to friend's projects. I'll show my contributions here, explaining why I made these Pull Requests, and a bit of history.

sa (simple ajax)

This little library is a thin wrapper around XMLHttpRequest objects, making AJAX requests very easy, even for someone like me that does not like using Javascript and doing "frontend programming". It's a nice alternative to jQuery and friends, when you just need to load a file in your custom homepage.

The first and only PR of the project brings a simple way to add a progress function to XMLHttpRequests objects. I found this functionality to be very useful to build a custom progress label while downloading a big JSON. Paolo (the project's owner) noticed that functionality and asked me to do a PR.

Docker-izing nerdzeu

nerdz.eu is not a forum, is not a social network... we only know what is _not_. At least we know its creator did a good job working on Docker-izing its discussion platform.

As every good Docker-er (yes, I just made it up), he knows to separate every single component. I wanted to try this new Docker thing, so I started installing and configuring nerdz.

nerdz-docker-camo #1 fixes a node problem being too strict with the SSL certificates: a certificate in the SSL "chain" was missing, and node rejects it entirely. Fortunately, it was fixed in v4.1.2, but v4.1.1 just came out in October, breaking lots of camo installations.

nerdz-docker #1 adds SSL support -modifying the nginx reverse proxy configuration-, and integrates the camo Docker I just linked a moment ago.

It was a fun experience, and got me interested in virtualization/replication systems like Docker or Vagrant.

Rubyin' hard: Yamazaki

Yamazaki #14 improves the project by adding more tests (and travis-ci support!), and a faster Database#include? method. Roxas Shadow is a Ruby pro, his advices are always backed by experience.

Moebius: helping a zeroMQ project

After a post on the ZeroMQ mailing list, I checked this "python zeromq-based dealer/router generic server" project. I didn't try it yet, but it's promising. At least, you can now run pip install moebius and you can start hacking: thank me later for moebius #5 :^)

In the meanwhile I worked to other projects too, but that's another story for another time (very soon, I promise!). So, let me know if you participated to the Hacktoberfest and what did you do!

Your mouse is a notification led!

I know, now you're wondering what the title means. "It makes no sense at all!", you're saying. Well, it makes sense... if you have a gaming mouse.

I'm an happy owner of a Steelseries Rival 300 Fade: it's a good mouse, the hand's grip is ok and feels good to use it. Another thing I like about this mouse is that it's supported by Steelseries Gamesense.

What is the Steelseries Gamesense? Gamesense is a component that lets third-party applications control the leds on Steelseries devices. In the linked blog post, you can see how health and ammo changes are reflected on the keyboard. My friends and I were hyped when that blog post came out, but I hadn't a chance to try it until now.

Some days ago, I had an idea: what if we can remotely check how much free space we have on our NAS/Raspberry Pi... just looking at the mouse? Light of the Free Disk Space does exactly that.

When there is a lot of space on your RPi, the mouse will show a relaxing blue light.

https://media.nerdz.eu/b_V5L1LyBwzT.jpg

When the free disk space starts to shrink (less than 50%), the led will have a frightening red light.

https://media.nerdz.eu/9h_vH64obIc1.jpg

Every info about the setup and the architecture can be found in the previously linked Github repository, and more details may come in the following days.

Yes, I did it for the laughs, to start using Flask and improve my sysadmin skills with the ArchLinux install that powers my Raspberry Pi. Nothing serious, nothing good, just a fun project.

Happy new Year to you too!

moesearch 0.2: from archive.moe to desustorage

Hello!

I'm back to talk about an update I just pushed to a personal project of mine, moesearch. This fine project is a requests-inspired library for a 4chan archive, desustorage.

A story first: it started a long time ago, when the Foolz Archive became archive.moe. Various techinical improvements to FoolFuuka (the engine that powers the most famous archives) brought a nice REST API developers could use to access the archive's data, removing the need to do painful and error-prone HTML scraping. It was matter of hours and the library was live on github.

Time passes, Hasumi decides he can't maintain the archive anymore (the costs were too high, its time not enough, hardware failure destroyed backups), so the desustorage guys take the project over.

Today, I thought: hey, what are these guys using? Turned out they're running FoolFuuka.

The library is now updated to version 0.2, and it's live on the cheese shop, or on the previously linked github repository: just pip-install it and try it!

See you next time!

PS: I'd like to talk about some other cool things I built or fixed too, but they have to wait until I receive my #hacktoberfest t-shirt :D

Suzuha: a shim for gettimeofday

In my daily work, I have to develop a logging system for realtime applications. Nothing too complex though, but it's nice to have when your system crashes and you don't know why.

I need to do a bit of integration testing, and I want to automatize it. Obviously, requirements talk about possible timing issues and some time-based business logic, so I have to test it, and using sleep is not an option.

How hard can it be to mock gettimeofday ? In C++, it requires a bit of work, but it's not impossible. At least, a bit more work than installing freezegun or time warp for projects in scripting languages such as Python or Ruby :D

After a fast search on github, I found two interesting projects: mocktime and gtod-shim. The main problem of the former is I don't want to change my actual code to call something different than gettimeofday. The latter was promising: a dynamic library you have to load at startup that changes the runtime behaviour of time functions, making it suitable to test time-related bugs in various applications.

After some modifications (mainly make users able to change the time shift using settimeofday and automated testing), suzuha is now released in beta. You can read more in the README, and check the unit tests.