Skip to content

cxx: add more flexible buffer classes

Stephan Lachnit requested to merge p-cxx-custom-buffer into main

Closes #86 (closed)

This is a rather big MR to fix three issues:

  1. Adds a proper payload_buffer class for message payloads instead of using zmq::message_t
  2. Fix that we always swap the payload of our message classes (closes #86 (closed)), instead of being able to choose
  3. Use ZeroMQ's zero-copy mechanism where possible

zmq_sbuffer

This is a rather simply class inherited from msgpack::sbuffer, but with the added benefit of creating zmq::message_t object (with and without releasing the buffer, so that we can both reuse message and use zero-copy when we don't reuse them). Since msgpack::sbuffer use std::free, we simply use this to free the buffer.

payload_buffer

This is the more interesting and complicated class. Essentially the idea is that we want zero-copy (not important for anything command related, but it gets important when sending data), but we do not know which object a user might want to send. To solve this issue, we use std::any to store the object. When it is time to delete the object, we can do so by deleting the std::any object.

However, since we take arbitrary object, we also need to get the pointer and size of the data. Thus we store a span and expect that the user passes a function that creates such a span from the object (note: I will add a default constructor for ranges of arithmetic types in !178).

There is also the tricky part that std::any requires that we have copy-able objects. This isn't the case for both zmq::message_t and msgpack::sbuffer. Luckily, we can trick std::any by just wrapping those in a shared pointer: in their move constructor, they simply copy the pointer and size of the buffer, so this is almost a free operation.

Further we need to take care of things like copy constructor and move assignments, which took a while to get right. Luckily we have the address sanitizers these days to ensure we don't leak memory or use freed memory.

TODO

  • Split buffer classes in separate headers
  • Check coverage, maybe add explicit tests
Edited by Stephan Lachnit

Merge request reports