Add support for multipart messages and remove custom framing for pub/sub

[cncfanatics]

I've changed the pub/sub code to remove the custom framing using a null byte and added code to send and receive multipart messages.
I've ensured API-level compatibility as well as making sure old versions can read data sent by the old one (thanks to the existing compatibility code there). But of course older versions won't be able to read multipart messages sent by this one.

[smira]

Hi @cncfanatics!

Thanks for the pull request. Actually first implementation was using multipart messages to deliver pub/sub. The problem with it was that with epgm (multicast) delivery parts of multipart messages from different hosts get intermixed in random way on the network level and come back from 0MQ in randomly cut and connected parts of messages. I'm not sure if 0MQ became reliable in this case. Probably we could use flag whether to use hack with 0 framing or real multipart messages?

[cncfanatics]

Hey @smira,

This sounds like a bug in libzmq, multipart messages should normally always arrive atomically.
Should probably be retested and if still present, log a bug report for it.

Do we have a unit test that reproduces this bug ?

[smira]

I believe it would be hard to write "unit-test", as it is only reproducible only with multiple hosts, multicast and moderately high load. I don't have test code to reproduce that. Probably we should involve ØMQ authors to help with that.

At the time I've been actively using ØMQ I was asking Martin Sustrik on that, and he confirmed that it's true by design. EPGM does datagram transfer, parts of multipart messages go as separate datagrams and get mixed on the way.

[phed]

Are there any plans to merge this?

[smira]

@phed I can't merge that, as it breaks wire level compatibility with old versions. If this PR had option to enable/disable new behavior, I'm ok to merge it.