The previous release of RabbitMQ (2.7.0) brought with it a better way of managing plugins, one-stop URI connecting by clients, thread-safe consumers in the Java client, and a number of performance improvements and bug-fixes. The latest release (2.7.1) is essentially a bug-fix release; though it also makes RabbitMQ compatible with Erlang R15B and enhances some of the management interface. The previous release didn’t get a blog post, so I’ve combined both releases in this one. (These are my own personal remarks and are NOT binding; errors of commission or omission are entirely my own – Steve Powell.)
Prior to 2.7.0 if you wanted to use a plugin then a
.ez file was placed in the
plugins directory, and the broker restarted. Any plugin found in this directory was installed on startup. This meant two things: plugins weren’t supplied with the server (even our supported ones), and installing or uninstalling a plugin involved moving files about and ensuring other plugin dependencies were installed. Administration of plugins was unnecessarily messy.
In 2.7.0, we introduced a command
rabbitmq-plugins, to enable or disable any plugin in the
plugins directory. Simply issue the command
rabbitmq-plugins list to see what plugin files are there, and
rabbitmq-plugins enable <plugin-name> to use one the next time the broker starts. No files need to be moved around – they can stay in the plugins directory all the time – and all the rabbit plugins are now supplied with the server, disabled by default. The command also understands which plugins depend upon which others, and enables dependencies automatically. Using and managing plugins is now much easier. See the plugins page for more.
In 2.7.1 there is a fix to the
consistent-hash-exchange plugin which mis-routed messages when handling multiple exchanges.
All of the rabbit clients (.NET, Java and Erlang) clients accept an
amqp URI scheme for connections. This is a convenient one-stop shop allowing the username and password, hostname and port, and virtual host, to be supplied by a single URI or String parameter. For example:
See the individual client APIs for details.
In Release 2.7.0 the threading structure of the Java client has been significantly redesigned. Before this release there were restrictions in what could be done in the Java client’s
Consumer callback methods, and also on which application threads can call
Channel methods. These have been due to the underlying threading structure of the Java client which shared the channel thread with the callbacks. Locks on the channel and connection objects meant that calling
Channel methods directly from the
Consumer resulted in deadlocks (with a few exceptions, like acknowledgements). The
QueueingConsumer helper class was made available to isolate applications from some of these issues, at the cost of introducing another queue (in the Java client).
With the new threading structure, there are far fewer restrictions upon which application threads can call channel operations, because all
Consumer callbacks are executed on threads which are separate from the channel. In fact, it is now possible to configure the connection to manage a pool of threads used specifically for callbacks preserving the order of execution of these within each channel. Simple client applications can take the default which provides a small pool of callback threads and sophisticated clients can provide their own
ExecutorService object, which allows them to create and manage the size and behaviour of the thread pool themselves.
QueueingConsumer is now no longer necessary, for everything one would want to do as a result of a
Consumer callback can be done in the
Consumer method directly, without deadlocks. See the Java API Guide for more.
2.7.1 fixed a few irritations in the Java Client adjustments in 2.7.0: we inadvertently hid some of the API, and this has now been restored. There were also some potential resource leaks which we have now fixed.
A number of small performance improvements have been made to the server in both 2.7.0 and 2.7.1. These have been quite varied in scope, and I can only touch on some of them here.
Erlang offers a High Performance compiler (HiPE) for some platforms whereby Erlang modules can be compiled to native code. This compilation does not always produce a faster system, however, and is not supported by all Erlang environments and versions. In 2.7.0 we introduced a configuration option to use HiPE, and a re-compilation is performed automatically at server startup. Not every rabbit module is re-compiled only those that we have determined may benefit from this treatment. Although this option delays startup by some tens of seconds, it produces significant performance improvements at runtime which may be crucial for some larger rabbit installations.
This option is disabled by default, since it may actually affect behaviour (not that we have detected this), and the performance improvements are not tested on all the environments our users employ. However, if it works for you, go for it. If your Erlang environment does not support HiPE, there is a brief diagnostic message and the option is ignored.
We would be very interested in your experiences with this feature.
RabbitMQ deals with FIFO queues. If all goes well, the order the messages are put on a queue is the same as the order they are consumed. When a consumer fails, however, some of the messages it received may not have been acknowledged, and in this case these messages are re-queued so that they may be delivered again. In this way messages may appear to be re-ordered. Before this release, there was no guarantee of the order of re-queued messages.
From 2.7.0 the relative order of re-queued messages from a single consumer is preserved. Therefore, if another consumer receives them later, they will be consumed in the same order they originally appeared. Of course, if two or more consumers on the same queue fail, there is no guarantee that messages re-queued by distinct consumers will retain their relative order. But in the majority of cases where order matters, this guarantee should be enough.
A number of fixes for high availability features (mostly introduced in 2.7.0) were included in 2.7.1. These relate to some memory leaks; recovery of master queues; frequent restarting causing HA queues to fail; and promotion of slaves (to master) failing under some circumstances. The general quality of this code area is high, but the complex nature of the failure scenarios (which this feature is specifically designed to protect against) makes it a fruitful one for bugs to lurk. Nearly all the bugs fixed in 2.7.1 are due to rare or obscure combinations of recovery or restart events, and we confidently expect there to be few surprises left. Of course, High Availability doesn’t mean Guaranteed Availability, so there are going to be situations from which we cannot recover.
rabbitmqctl eval <expr>is new (in 2.7.1) to evaluate an arbitrary Erlang expression in the broker node.
As usual, the rabbit team welcome feedback of your experiences, good or bad. We encourage you to use the rabbitmq-discuss mailing list.
Written by: Steve Powell
Categories: New Features