ZeroMQ Asynchronous Messaging

First published — Jul 27, 2023
Last updated — Jul 27, 2023
#zeromq

ZeroMQ async messaging. Can run brokerless. Runs on top of ITC, IPC, PGM, TCP, TIPC, UDP, or VMCI. BSD sockets-like API. REQ-REP, PUB-SUB, and PUSH-PULL pattern.

Table of Contents

Introduction

ZeroMQ (also spelled ØMQ, 0MQ or ZMQ) is an asynchronous, high-performance messaging system.

Data is transferred in “messages”, relieving the user of worrying about low-level details.

Message queuing exists, but can run without a message broker (hence the “Zero” in ZeroMQ).

ZeroMQ provides “sockets” for communication — the same term as used in BSD sockets. However, ZeroMQ sockets are faster, more powerful, and generalized. They can run on top of BSD sockets, PGM (reliable multicast), inter-process communication (IPC), or inter-thread communication (ITC).

ZeroMQ sockets support many-to-many connections between endpoints.

Being based on messages, ZeroMQ offers choice of a couple different messaging patterns:

  1. Request–Reply. Connects a set of clients to a set of services. This is a standard remote procedure call and task distribution pattern, a service bus" topology
  2. Publish–Subscribe. Connects a set of publishers to a set of subscribers. This is a data distribution pattern / tree topology
  3. Push–Pull (pipeline). Connects nodes in a fan-out / fan-in pattern that can have multiple steps and loops. This is a parallel task distribution and collection pattern / parallelized pipeline topology
  4. Exclusive Pair. Connects two sockets in an exclusive pair. This is advanced low-level pattern for specific use cases.

ZeroMQ API is similar to the convenient BSD sockets API (send, recv), but hides low-level details and provides high-level patterns.

Comparison to BSD Sockets

For those at least vaguely familiar with standard BSD sockets, ZeroMQ differences can be summarized as follows:

  1. ZeroMQ enables distributed communication and message passing, with or without a central broker
  2. It supports various underlying socket types (ITC, IPC, PGM, TCP, TIPC, UDP, VMCI)
  3. It is not important which side calls bind, and which connect
  4. The listen call is implicit, it does not exist as a separate function
  5. Data is transferred as messages, user does not worry about low-level details
  6. Sockets can be connected to multiple peers at the same time, in that case Round-Robin (RR) is used to select them
  7. There is internal buffering, which is most visible at time at first connection and/or reconnection
  8. Reconnections work transparently as long as data transferred is conforming to the ZeroMQ protocol
  9. Being message-based, ZeroMQ requires use of one of supported messaging patterns (REQ-REP, PUB-SUB, PUSH-PULL)
  10. Data is sent in “frames”, blocks of data separated by null-byte. This is the only network overhead caused by ZeroMQ
  11. Framing can be disabled, for talking to traditional BSD sockets
  12. Clients select/filter the messages they are interested in (by default no messages). When filtering is enabled, it is done on the client side by checking whether a frame begins with the specified filter. By convention the filter is a string value
  13. ZeroMQ API is similar to the standard and convenient, synchronous BSD sockets API, but its internal implementation is fully asynchronous
  14. ZeroMQ autodetects async mechanisms available on the host. It supports kqueue (BSD) and epoll (Linux), falling back traditional select. Io_uring is not supported https://github.com/zeromq/libzmq/issues/4142 ,