Publishers
Overview
This guide covers various topics related to publishers:
- The basics
- Publisher lifecycle
- Protocol Differences
- Message properties and delivery metadata
- Publisher-side data safety topics (connection recovery, publisher confirms)
- Exception Handling
- Effects of Resource Alarms
- Unroutable Message Handling
- Metrics relevant for publishers
- Concurrency Consideration
- How to temporarily block all publishers
- How to troubleshoot common issues with publishers
and more.
The guide focuses on AMQP 0-9-1 and mentions key protocol-specific differences with other protocols supported by RabbitMQ: AMQP 1.0, MQTT and STOMP.
Terminology
The term "publisher" means different things in different contexts. In general in messaging a publisher (also called "producer") is an application (or application instance) that publishes (produces) messages. The same application can also consume messages and thus be a consumer at the same time.
Messaging protocols also have the concept of a lasting subscription for message delivery. Subscription is one term commonly used to describe such entity. Consumer is another. Messaging protocols supported by RabbitMQ use both terms but RabbitMQ documentation tends to prefer the latter.
The Basics
RabbitMQ is a messaging broker. It accepts messages from publishers, routes them and, if there were queues to route to, stores them for consumption or immediately delivers to consumers, if any.
Publishers publish to a destination that varies from protocol to protocol. In AMQP 0-9-1, publishers publish to exchanges. In AMQP 1.0, publishing happens on a link. In MQTT, publishers publish to topics. Finally, STOMP supports a variety of destination types: topics, queues, AMQP 0-9-1 exchanges. This is covered in more details in the protocol-specific differences section.
A publish message has to be routed to a queue (topic, etc). The queue (topic) may have online consumers. When the message is successfully routed to a queue and there is a consumer online that can accept more deliveries, the message will be sent to the consumer.
An attempt to publish to a non-existent queue (topic) will result in a channel-level
exception with the code of 404 Not Found
and render the channel it was attempted
on to be closed.
Publisher Lifecycle
Publishers are often long lived: that is, throughout the lifetime of a publisher it publishes multiple messages. Opening a connection or channel (session) to publish a single message is not optimal.
Publishers usually open their connection(s) during application startup. They often would live as long as their connection or even application runs.
Publishers can be more dynamic and begin publishing in reaction to a system event, stopping when they are no longer necessary. This is common with WebSocket clients used via Web STOMP and Web MQTT plugins, mobile clients and so on.
Protocol Differences
The process of publishing messages is quite similar in every protocol RabbitMQ supports. All four protocols allow the user to publish a message which has a payload (body) and one or more message properties (headers).
All four protocols also support an acknowledgement mechanism for publishers which allows the publishing application to keep track of the messages that have or haven't been successfully accepted by the broker, and continue publishing the next batch or retry publishing the current one.
The difference typically have more to do with the terminology used than the semantics. Message properties also vary from protocol to protocol.
AMQP 0-9-1
In AMQP 0-9-1, publishing happens on a channel to an exchange. The exchange uses a routing topology set up by defining bindings between one or more queues and the exchange, or source exchange and destination exchange. Successfully routed messages are stored in queues.
The role of each entity is covered in the AMQP 0-9-1 concepts guide.
Publisher confirms is the publisher acknowledgement mechanism.
There are several common types of publisher errors that are handled using different protocol features:
- Publishing to a non-existent exchange results in a channel error, which closes the channel so that no further publishing (or any other operation) is allowed on it.
- When a published message cannot be routed to any queue (e.g. because there are no bindings defined for the
target exchange), and the publisher set the
mandatory
message property tofalse
(this is the default), the message is discarded or republished to an alternate exchange, if any. - When a published message cannot be routed to any queue, and the publisher set the
mandatory
message property totrue
, the message will be returned to it. The publisher must have a returned message handler set up in order to handle the return (e.g. by logging an error or retrying with a different exchange)
AMQP 1.0
In AMQP 1.0 publishing happens within a context of a link.
MQTT
In MQTT, messages are published on a connection to a topic. The server side MQTT connection process routes messages via the topic exchange to queues.
When publisher chooses to use QoS 1, published messages are acknowledged by RabbitMQ using a PUBACK packet.
Publishers can provide a hint to the server that the published message on the topic must be retained (stored for future delivery to new subscribers). Only the latest published message for each topic can be retained.
The MQTT 5.0 PUBACK packet includes a reason code that tells the publisher whether publishing was successful. Reason codes returned by RabbitMQ include:
0 - Success
: All the queues the message was routed to successfully accepted the message.16 - No matching subscribers
: RabbitMQ could not route the message to any queue (because there are no bindings defined to the topic exchange).131 - Implementation specific error
: RabbitMQ rejected the message (for example when a target classic queue is unavailable).
In MQTT 3.1 and 3.1.1, other than closing the connection, there is no mechanism by which the server can communicate a publishing error to the client.
See the MQTT and MQTT-over-WebSockets guides to learn more.
STOMP
STOMP clients publish on a connection to one or more destinations which can have different semantics in case of RabbitMQ.
STOMP provides a way for the server to communicate an error in message processing back to the publisher. Its variation of publisher acknowledgements is called receipts, which is a feature clients enable when publishing.
See the STOMP guide, STOMP-over-WebSockets and the STOMP 1.2 spec to learn more.
Routing
AMQP 0-9-1
Routing in AMQP 0-9-1 is performed by exchanges. Exchanges are named routing tables. Table entries are called bindings. This is covered in more detail in the AMQP 0-9-1 concepts guide.
There are several built-in exchange types:
- Topic
- Fanout
- Direct (including the default exchange)
- Headers
The first three types are covered with examples in the tutorials.
More exchange types can be provided by plugins. Consistent hashing exchange, random routing exchange, internal event exchange and delayed message exchange are exchange plugins that ship with RabbitMQ. Like all plugins they must be enabled before they can be used.
Unroutable Message Handling
Clients might attempt to publish messages to destinations (exchanges, topics, queues) that do not exist. This section covers on how different protocols different in handling of such cases.
RabbitMQ collects and exposes metrics that can be used to detect publishers that publish unroutable messages.
AMQP 0-9-1
When a published message cannot be routed to any queue (e.g. because there are no bindings defined for the
target exchange), and the publisher set the mandatory
message property to false
(this is the default),
the message is discarded or republished to an alternate exchange, if any.
When a published message cannot be routed to any queue, and the publisher set the mandatory
message property to true
, the message will be returned to it. The publisher must have a returned
message handler set up in order to handle the return (e.g. by logging an error or retrying with
a different exchange).
Alternate Exchanges is an AMQP 0-9-1 exchange feature that lets clients handle messages that an exchange was unable to route (i.e. either because there were no bound queues or no matching bindings). Typical examples of this are detecting when clients accidentally or maliciously publish messages that cannot be routed "or else" routing semantics where some messages are handled specially and the rest by a generic handler.