Archive for the 'New Features' Category
October 7, 2021
RabbitMQ streams allow applications to convey detailled information thanks to the powerful message format they use.
Streams are a feature of their own, but they also fully integrate with the existing resources and protocols that RabbitMQ supports.
This blog post covers the interoperability of streams in RabbitMQ and explores the scenarios it unlocks.
September 13, 2021
RabbitMQ Streams provides server-side offset tracking for consumers.
This features allows a consuming application to restart consuming where it left off in a previous run.
This post covers the semantics of offset tracking and how it is implemented in the stream Java client.
July 28, 2021
RabbitMQ Streams Overview introduced streams, a new feature in RabbitMQ 3.9 and RabbitMQ Streams First Application provided an overview of the programming model with the stream Java client. This post covers how to deduplicate published messages in RabbitMQ Streams.
As deduplication is a critical and intricate concept, the post will walk you through this mechanism step by step, from a naive and somewhat broken publishing application to an optimized and reliable implementation.
July 23, 2021
RabbitMQ Streams Overview introduced streams, a new feature in RabbitMQ 3.9.
This post covers how client applications should connect to RabbitMQ nodes to get the most benefit from streams when the stream protocol is in use.
Streams are optimized for high throughput scenarios, that’s why technical details like data locality are critical to get the best out of your RabbitMQ cluster.
Client libraries can handle most of the details, but a basic understanding of how things work under the hood is essential when a setup involves extra layers like containers and load balancers. Keep reading if you want to learn more about streams and avoid some headaches when deploying your first stream applications!
July 19, 2021
RabbitMQ Streams Overview introduced streams, a new feature in RabbitMQ 3.9.
This post continues by showing how to use streams with the Java client.
We will write our first application that publishes messages to a stream, and then consumes them.
July 13, 2021
RabbitMQ 3.9 introduces a new type of data structure: streams. Streams unlock a set of use cases that could have been tedious to implement with “traditional” queues. Let’s discover in this post how streams expand the capabilities of RabbitMQ.
May 3, 2021
David Ansari & Gerhard Lazu
If you want to be notified when your RabbitMQ deployments have a problem, now you can set up the RabbitMQ monitoring and alerting that we have made available in the RabbitMQ Cluster Operator repository. Rather than asking you to follow a series of steps for setting up RabbitMQ monitoring & alerting, we have combined this in a single command. While this is a Kubernetes-specific quick-start, and you can use these Prometheus alerts outside of Kubernetes, the setup will require more consideration and effort on your part.
March 1, 2021
Diana Parra Corbacho
We have been constantly improving the monitoring capabilities that are built into RabbitMQ since shipping native Prometheus support in 3.8.0. Monitoring the broker and its clients is critically important for detecting issues before they affect the rest of the environment and, eventually, the end users.
RabbitMQ 3.8.10 exposes client authentication attempts metrics via both the Prometheus endpoint and the HTTP API.
November 17, 2020
We are pleased to announce that the RabbitMQ Operator for Kubernetes is now generally available. The RabbitMQ Operator makes it easy to provision
and manage RabbitMQ clusters consistently on any certified Kubernetes distribution. Operators inform the Kubernetes container orchestration
system how to provision and control specific applications. The Kubernetes (hereafter K8s) Operator pattern is a way to extend the K8s API and
state management to include the provisioning and management of custom resources – resources not provided in a default K8s deployment. In this
post, we’ll discuss how the Operator enables the K8s system to control a RabbitMQ cluster.
April 20, 2020
This is the first part of a series on quorum queues, our new replicated queue type. We’ll be covering everything from what quorum queues are, to hardware requirements, migration from mirrored queues and best practices.
Introducing Quorum Queues
Mirrored queues, also known as HA queues have been the de facto option for years when requiring extra data safety guarantees for your messages. Quorum queues are the next generation of replicated queue that aim to replace most use cases for mirrored queues and are available from the 3.8 release and onward.
In this blog series we’re going to cover the following:
November 11, 2019
RabbitMQ 3.8 has just been released and has some major new features which focus on reliability, operations, and observability.
You can find the new 3.8 release on the GitHub releases page which includes information about what is included in the release as well as various installation assets. See our upgrade guide for more information about upgrading to 3.8.0.
Our team dedicates this release to Joe Armstrong, the creator of Erlang. Joe’s work in the fields of concurrent and distributed systems benefits RabbitMQ to this day. Equally importantly, Joe was a rare example of a brilliant engineer who was also very humble and kind.
Let’s take a quick look at the new features in this release.
April 23, 2019
In this post we will cover feature flags, a new subsystem in RabbitMQ 3.8. Feature flags will allow a rolling cluster upgrade to the next minor version, without requiring all nodes to be stopped before upgrading.
April 10, 2018
In this post we’ll cover how the RabbitMQ Java client library gathers runtime metrics and sends them to monitoring systems like JMX and Datadog.
February 22, 2018
In this post we’ll cover a new configuration format available
in RabbitMQ 3.7.0.
February 18, 2018
In this blog post we’re going to take a closer look at a new
subsystem introduced in RabbitMQ 3.7.0.
February 5, 2018
After over 1 year in the works, RabbitMQ 3.7.0 has quietly shipped
right before the start of the holiday season. The release was heavily
inspired by the community feedback on 3.6.x. In this post weâ€™d like to
cover some of the highlights in this release.
October 18, 2017
The RabbitMQ team is happy to announce the release of version 2.0 of HOP, RabbitMQ HTTP API client for Java and other JVM languages. This new release introduce a new reactive client based on Spring Framework 5.0 WebFlux.
September 29, 2017
The RabbitMQ team is happy to announce the release of version 5.0 of the RabbitMQ Java Client. This new release is now based on Java 8 and comes with a bunch of interesting new features.
December 15, 2016
As of v3.7.0 Milestone 8,
RabbitMQ ships with a brand new set of CLI tools (
rabbitmq-plugins, and more), reworked from the ground-up. We had a few goals with this project.
November 30, 2016
Version 4.0 of the RabbitMQ Java Client brings support for runtime metrics. This can be especially useful to know how a client application is behaving. Let’s see how to enable metrics collection and how to monitor those metrics on JMX or even inside a Spring Boot application.
November 24, 2016
The RabbitMQ team is happy to announce the release of version 4.0 of the RabbitMQ Java Client. This new release does not introduce any breaking changes and comes with a bunch of interesting new features.
December 28, 2015
We are pleased to announce the immediate availability of RabbitMQ
3.6.0, a new version of the broker that comes packed with lot of
new features. Before
we go on, you can obtain it here:
This release brings many improvements in broker features, development
environment for our contributors, and security. Let’s take a look at
some of the most significant ones.
April 16, 2015
For a while people have looked for ways of implementing delayed
messaging with RabbitMQ. So far the accepted solution was to use a mix
of message TTL and
Dead Letter Exchanges as proposed
by James Carr
a while we have thought to offer an out-of-the-box solution for this,
and these past month we had the time to implement it as a
RabbitMQ Delayed Message Plugin.
October 30, 2014
“How much memory is my queue using?” That’s an easy question to ask, and a somewhat more complicated one to answer. RabbitMQ 3.4 gives you a clearer view of how queues use memory. This blog post talks a bit about that, and also explains queue memory use in general.
April 14, 2014
One of the goals for RabbitMQ 3.3 was that you should be able to find bottlenecks in running systems more easily. Older versions of RabbitMQ let you see that you were rate-limited but didn’t easily let you see why. In this blog post we’ll talk through some of the new performance indicators in version 3.3.
April 10, 2014
I warn you before we start: this is another wordy blog post about performance-ish changes in RabbitMQ 3.3. Still with us? Good.
So in the previous post I mentioned “a new feature which I’ll talk about in a future blog post”. That feature is consumer bias.
April 3, 2014
Well, we got the bad news out of the way yesterday, so today let’s talk about (some of) the good news: some types of publishing and consuming are now a great deal faster, especially in clusters.
April 2, 2014
What? Another “breaking things” post? Well, yes, but hopefully this should be less to deal with than the previous one. But there are enough slightly incompatible changes in RabbitMQ 3.3.0 that it’s worth listing them here.
January 23, 2014
Different services in our architecture will require a certain amount of resources for operation, whether these resources are CPUs, RAM or disk space, we need to make sure we have enough of them. If we don’t put limits on how many resources our servers are going to use, at some point we will be in trouble. This happens with your database if it runs out of file system space, your media storage if you fill it with images and never move them somewhere else, or your JVM if it runs out of RAM. Even your back up solution will be a problem if you don’t have a policy for expiring/deleting old backups. Well, queues are no exception. We have to make sure that our application won’t allow the queues to grow for ever. We need to have some strategy in place to delete/evict/migrate old messages.
December 16, 2013
With RabbitMQ 3.2.0 we introduced Consumer Priorities which not surprisingly allows us to set priorities for our consumers. This provides us with a bit of control over how RabbitMQ will deliver messages to consumers in order to obtain a different kind of scheduling that might be beneficial for our application.
When would you want to use Consumer Priorities in your code?
October 23, 2013
So we added support for federated queues in RabbitMQ 3.2.0. This blog post explains what they’re for and how to use them.
May 1, 2013
November 20, 2012
So we’ve talked about how RabbitMQ 3.0 can break things, but that’s not very positive. Let’s have a look at some of the new features! Just some of them - quite a lot changed in 3.0, and we don’t have all day…
November 19, 2012
RabbitMQ includes a bunch of cool new features. But in order to implement some of them we needed to change some things. So in this blog post I’m going to list some of those things in case you need to do anything about them.
September 12, 2012
I’ve written a plugin for RabbitMQ that adds support for the MQTT 3.1 protocol. MQ Telemetry Transport is a light-weight PUB/SUB protocol designed for resource-constrained devices and limited bandwidth situations, making it ideally suited to sensors and mobile devices. The implementation is a protocol adapter plugin, allowing MQTT clients to connect to a RabbitMQ broker simultaneously with clients implementing other protocols. We encourage projects that demand the combination of a low-overhead protocol on a robust, scalable broker with high reliability and enterprise features to consider this option.
May 14, 2012
For quite a while here, at RabbitMQ headquarters, we were struggling to
find a good way to expose messaging in a web browser. In the past we tried many
things ranging from the old-and-famous JsonRPC plugin (which basically
exposes AMQP via AJAX), to Rabbit-Socks (an attempt to create a generic
protocol hub), to the management plugin (which can be used for basic
things like sending and receiving messages from the browser).
Over time we’ve learned that the messaging on the web is very different
to what we’re used to. None of our attempts really addressed
that, and it is likely that messaging on the web will not be a fully
solved problem for some time yet.
That said, there is a simple thing RabbitMQ users keep on asking
about, and although not perfect, it’s far from the worst way do messaging
in the browser: exposing STOMP through Websockets.
December 20, 2011
The previous release of RabbitMQ (2.7.0) brought with it a better way of managing plugins, one-stop URI connecting by clients, thread-safe consumers in the Java client, and a number of performance improvements and bug-fixes. The latest release (2.7.1) is essentially a bug-fix release; though it also makes RabbitMQ compatible with Erlang R15B and enhances some of the management interface. The previous release didn’t get a blog post, so I’ve combined both releases in this one. (These are my own personal remarks and are NOT binding; errors of commission or omission are entirely my own – Steve Powell.)
October 27, 2011
Since the new persister arrived in RabbitMQ 2.0.0 (yes, it’s not so
new anymore), Rabbit has had a relatively good story to tell about
coping with queues that grow and grow and grow and reach sizes that
preclude them from being able to be held in RAM. Rabbit starts writing
out messages to disk fairly early on, and continues to do so at a
gentle rate so that by the time RAM gets really tight, we’ve done most
of the hard work already and thus avoid sudden bursts of
writes. Provided your message rates aren’t too high or too bursty,
this should all happen without any real impact on any connected
Some recent discussion with a client made us return to what we’d
thought was a fairly solved problem and has prompted us to make some
October 25, 2011
we introduced Highly Available
queues. These necessitated a
to AMQP, and a fair amount of
documentation, but to date, little has
been written on how they work.
September 9, 2011
While the firehose is quite a cool feature, I always thought that it was a shame we didn’t have a simple GUI to go on top and make it accessible to system administrators. So I wrote one. You can download it here.
June 22, 2011
Note: this blog post talks about the federation plugin preview that was released for RabbitMQ 2.5.0. If you’re using 2.6.0 or later, federation is part of the main release; get it the same way you would any other plugin.
Another day, another new plugin release 😃 Today it’s federation. If you want to skip this post and just download the plugin, go here. The detailed instructions are here.
The high level goal of federation is to scale out publish / subscribe messaging across WANs and administrative domains.
To do this we introduce the concept of the federation exchange. A federation exchange acts like a normal exchange of a given type (it can emulate the routing logic of any installed exchange type), but also knows how to connect to upstream exchanges (which might in turn themselves be federation exchanges).
March 23, 2011
RabbitMQ 2.4.0 introduced an extension that allows publishers to specify multiple routing keys in the CC and BCC message headers. The BCC header is removed from the message prior to delivery. Direct and topic exchanges are the only standard exchange types that make use of routing keys, therefore the routing logic of this feature only works with these exchange types.
February 10, 2011
In many messaging scenarios, you must not lose messages. Since AMQP gives few guarantees regarding message persistence/handling, the traditional way to do this is with transactions, which can be unacceptably slow. To remedy this problem, we introduce an extension to AMQP in the form of Lightweight Publisher Confirms.
February 7, 2011
RabbitMQ 2.3.1 introduces a couple of new plugin mechanisms, allowing you much more control over how users authenticate themselves against Rabbit, and how we determine what they are authorised to do. There are three questions of concern here:
- How does the client prove its identity over the wire?
- Where do users and authentication information (e.g. password hashes) live?
- Where does permission information live?
Question 1 is answered in the case of AMQP by SASL - a simple protocol for pluggable authentication mechanisms that is embedded within AMQP (and various other protocols). SASL lets a client and a server negotiate and use an authentication mechanism, without the “outer” protocol having to know any of the details about how authentication works.
SASL offers a number of “mechanisms”. Since the beginning, RabbitMQ has supported the PLAIN mechanism, which basically consists of sending a username and password over the wire in plaintext (of course possibly the whole connection might be protected by SSL). It’s also supported the variant AMQPLAIN mechanism (which is conceptually identical to PLAIN but slightly easier to implement if you have an AMQP codec lying around). RabbitMQ 2.3.1 adds a plugin system allowing you to add or configure more mechanisms, and we’ve written an example plugin which implements the SASL EXTERNAL mechanism.
January 19, 2011
I’m happy to announce that the AMQP 0.7 is released, as I promised in the previous blog post. So what are the changes?
December 1, 2010
We have been prototyping support for a new protocol, as is our wont. This one is called “AMQP 1.0 R0”, and it is the new issue from the AMQP working group (of which RabbitMQ, and latterly VMware, are a member). The “R0” indicates that it’s the first revision of a recommendation. The specification is incomplete: there are many TODOs, and to a large extent it is unproven. Those two facts are part of what prompted this prototyping.
October 19, 2010
Arriving in RabbitMQ 2.1.1, is support for bindings between exchanges. This is an extension of the AMQP specification and making use of this feature will (currently) result in your application only functioning with RabbitMQ, and not the myriad of other AMQP 0-9-1 broker implementations out there. However, this extension brings a massive increase to the expressivity and flexibility of routing topologies, and solves some scalability issues at the same time.
Normal bindings allow exchanges to be bound to queues: messages published to an exchange will, provided the various criteria of the exchange and its bindings are met, pass through the various bindings and be appended to the queue at the end of each binding. That’s fine for a lot of use cases, but there’s very little flexibility there: it’s always just one hop – the message being published to one exchange, with one set of bindings, and consequently one possible set of destinations. If you need something more flexible then you’d have to resort to publishing the same message multiple times. With exchange-to-exchange bindings, a message published once, can flow through any number of exchanges, with different types, and vastly more sophisticated routing topologies than previously possible.
October 18, 2010
Recently, Michael Bridgen and I implemented a bridge to connect the RabbitMQ broker with applications using 0MQ for messaging.
Here it is: http://github.com/rabbitmq/rmq-0mq
So: What kind of benefit the users can get by using both products in parallel?
September 7, 2010
The previously mentioned management plugin is now in a state where it’s worth looking at and testing. In order to make this easy, I’ve made a special once-only binary release just for the management plugin (in future we’ll make binary releases of it just like the other plugins). Download all the .ez files from here and install them as described here, then let us know what you think. (Update 2010-09-22: Note that the plugins referenced in this blog post are for version 2.0.0 of RabbitMQ. We’ve now released 2.1.0 - for this and subsequent versions you can get the management plugin from here).
August 27, 2010
Some three and a half years after we launched RabbitMQ, we have this week released RabbitMQ 2.0.
This means some big changes. The most important of these is our new Scalable Storage Engine. RabbitMQ has always provided persistence for failure recovery. But now, you can happily push data into Rabbit regardless of how much data is already stored, and you don’t need to worry about slow consumers disrupting processing. As the demands on your application grow, Rabbit can scale with you, in a stable, reliable way.
Before introducing RabbitMQ 2.0, let me reiterate that as Rabbit evolves you can count on the same high level of commitment to you as a customer or end user, regardless of whether you are a large enterprise, or a next-gen start-up, or an open source community. As always, get in touch if you need help or commercial support.
August 6, 2010
For a long time the management and monitoring capabilities built into RabbitMQ have consisted of rabbitmqctl. While it’s a reasonable tool for management (assuming you like the command line), rabbitmqctl has never been very powerful as a monitoring tool. So we’re going to build something better.
August 4, 2010
Since the beginning, RabbitMQ has implemented the 0-8 version of the AMQP specification. This was the first publicly-available version, but there’s been plenty of development since then. In particular, we’ve wanted to support the 0-9-1 version of AMQP for a while now.
August 3, 2010
Support for AMQP’s
basic.reject just landed on default. It’s taken this long because we couldn’t agree on a single set of semantics that followed the specification, was actually useful, and wasn’t too complicated to implement.