Menu

Archive for year 2011

RabbitMQ 2.7.0 and 2.7.1 are released

December 20, 2011 by Steve Powell

The previous release of RabbitMQ (2.7.0) brought with it a better way of managing plugins, one-stop URI connecting by clients, thread-safe consumers in the Java client, and a number of performance improvements and bug-fixes. The latest release (2.7.1) is essentially a bug-fix release; though it also makes RabbitMQ compatible with Erlang R15B and enhances some of the management interface. The previous release didn’t get a blog post, so I’ve combined both releases in this one.  (These are my own personal remarks and are NOT binding; errors of commission or omission are entirely my own – Steve Powell.)

Read More...

Ponies, Dragons and Socks

November 30, 2011 by Marek Majkowski

We were wondering how to present SockJS and its possibilities to a wider audience. Having a working demo is worth much more than explaining dry theory, but what can you present if you are just a boring technologist, with no design skills whatsoever?

With questions like that it’s always good to open a history book and review previous generation of computer geeks with no artistic skills. What were they doing? On consoles with green letters they were playing geeky computer games, MUDs (Multi User Dungeons) were especially popular.

Hey, we can do that!

Read More...

Performance of Queues: when less is more

October 27, 2011 by Matthew Sackman

Since the new persister arrived in RabbitMQ 2.0.0 (yes, it’s not so new anymore), Rabbit has had a relatively good story to tell about coping with queues that grow and grow and grow and reach sizes that preclude them from being able to be held in RAM. Rabbit starts writing out messages to disk fairly early on, and continues to do so at a gentle rate so that by the time RAM gets really tight, we’ve done most of the hard work already and thus avoid sudden bursts of writes. Provided your message rates aren’t too high or too bursty, this should all happen without any real impact on any connected clients.

Some recent discussion with a client made us return to what we’d thought was a fairly solved problem and has prompted us to make some changes.

Read More...

High Availability in RabbitMQ: solving part of the puzzle

October 25, 2011 by Matthew Sackman

In RabbitMQ 2.6.0 we introduced Highly Available queues. These necessitated a new extension to AMQP, and a fair amount of documentation, but to date, little has been written on how they work.

Read More...

Keeping It Realtime Conference (Portland, OR)

October 19, 2011 by Marek Majkowski

There’s a lot of hot stuff happening in the web technology lately. JavaScript seems to be bearing the torch, both browser-side and server-side. At the RabbitMQ HQ we’re interested in developments in the wide world of messaging, and we’re particularly excited about the JavaScript angle on messaging - namely WebSockets and related technologies.

Read More...

PubSubHuddle 'Realtime Web' talk

September 26, 2011 by Marek Majkowski

I was asked to do a short presentation during the PubSubHuddle meetup. The talk was about current development of WebSockets, its issues and building web applications using them.

Read More...

Sizing your Rabbits

September 24, 2011 by Matthew Sackman

One of the problems we face at the RabbitMQ HQ is that whilst we may know lots about how the broker works, we don’t tend to have a large pool of experience of designing applications that use RabbitMQ and which need to work reliably, unattended, for long periods of time. We spend a lot of time answering questions on the mailing list, and we do consultancy work here and there, but in some cases it’s as a result of being contacted by users building applications that we’re really made to think about long-term behaviour of RabbitMQ. Recently, we’ve been prompted to think long and hard about the basic performance of queues, and this has lead to some realisations about provisioning Rabbits.

Read More...

PubSub huddle

September 16, 2011 by Michael Bridgen

All of a sudden there’s just one week to go until the PubSub huddle. It’s a one day conference, in London, about messaging. Not just RabbitMQ, but ZeroMQ, MQTT, XMPP and PuSH.

Read More...

SockJS - WebSocket emulation

September 13, 2011 by Marek Majkowski

WebSocket technology is catching up, but it will take a while before all browsers support it. In the meantime there are loads of projects that aim to substitute for WebSockets and enable ‘realtime’ capabilities for web apps. But all attempts solve only a part of the general problem, and there isn’t any single solution that works, is scalable and doesn’t require special deployment tricks.

Read More...

rabbitmq-tracing - a UI for the firehose

September 9, 2011 by Simon MacMullen

While the firehose is quite a cool feature, I always thought that it was a shame we didn’t have a simple GUI to go on top and make it accessible to system administrators. So I wrote one. You can download it here.

Read More...

RabbitMQ on Heroku

September 1, 2011 by Alexis Richardson

We are very pleased to announce the availability in beta of RabbitMQ as a Heroku add-on. With our RabbitMQ service on CloudFoundry, this extends our commitment to supporting the community of cloud application developers.

We believe that cloud messaging is fundamental in two senses. First as a core capability to build applications that scale to cloud use cases as explained in our blog post launching RabbitMQ on CloudFoundry. And second, because messaging can be extended to solve common problems like integration and data push. For example: to connect traditional on-premise applications with virtualized and cloud deployments.

Read More...

SockJS - web messaging ain't easy

August 22, 2011 by Marek Majkowski

The idea of ‘realtime web’ or messaging using web browsers has been around for quite some time. First it was called ‘long-polling’, then ‘Comet’, the latest incarnation is named ‘WebSockets’. Without doubt it’s going in a good direction, WebSockets is a neat technology.

But during the fight for realtime capabilities we’ve lost focus on what is really important how to actually use messaging. In the web context everything is request-response driven and marrying a typical web stack to asynchronous messaging isn’t easy.

Read More...

Using the RabbitMQ service on Cloud Foundry with Node.JS

August 16, 2011 by Michael Bridgen

Recently we launched a RabbitMQ service for Cloud Foundry, making it simple to spin up a message broker to use with your apps on Cloud Foundry. There are tutorials online for using it with Ruby on Rails and with Java apps using Spring. Here we are going to look at using the RabbitMQ service with Node.JS apps.

Read More...

RabbitMQ + Cloud Foundry: Cloud Messaging that Just Works

August 10, 2011 by David Wragg
Today we launched a RabbitMQ service on CloudFoundry.com. This service brings the messaging functionality of RabbitMQ to developers building applications on Cloud Foundry. You can read the main announcement over on the Cloud Foundry blog. There’s also an FAQ with more details on the Cloud Foundry knowledge base. CloudFoundry.com is a free beta service. So please register there (if you haven’t already), then take a look at the RabbitMQ service, try out the sample apps, and write your own.
Read More...

Puka - rethinking AMQP clients

July 8, 2011 by Marek Majkowski

I fundamentally disagree with the APIs exposed by our current AMQP client libraries.

There is a reason why they’re imperfect: we intentionally avoided innovation in APIs since the beginning. The purpose of our client libraries is to expose generic AMQP, not any one view of messaging. But, in my opinion, trying to map AMQP directly to client libraries APIs is just wrong and results in over-complication and abstractions hard to use.

There is no common ground: the client libraries blindly following AMQP model will be complex; easy to use client libraries must to be opinionated.

Read More...

ZeroMQ =/= Erlang

June 30, 2011 by Michael Bridgen

Recently I saw a tweet saying “ZeroMQ Erlangizes everything!” or some such. While I realise that not everything posted on the web is meant seriously, it does seem there is a stream of similar claims lately that ought to be dammed.

In the article Multi-threading Magic1, Pieter Hintjens and Martin Sustrik persuasively explain why concurrency is better served by message-passing than by locks and shared memory. And they are fair, I think, in their analysis – except for the insinuation that using ZeroMQ transforms your chosen programming language into a domestic Erlang.

Read More...

Federation plugin preview release

June 22, 2011 by Simon MacMullen

Note: this blog post talks about the federation plugin preview that was released for RabbitMQ 2.5.0. If you’re using 2.6.0 or later, federation is part of the main release; get it the same way you would any other plugin.

Another day, another new plugin release 😃 Today it’s federation. If you want to skip this post and just download the plugin, go here. The detailed instructions are here.

The high level goal of federation is to scale out publish / subscribe messaging across WANs and administrative domains.

To do this we introduce the concept of the federation exchange. A federation exchange acts like a normal exchange of a given type (it can emulate the routing logic of any installed exchange type), but also knows how to connect to upstream exchanges (which might in turn themselves be federation exchanges).

Read More...

RabbitMQ 2.5.0 released

June 16, 2011 by Jerry Kuch

The RabbitMQ team is delighted to announce the release of RabbitMQ 2.5.0.

Read More...

Can you hear the drums, Erlando?

May 17, 2011 by Matthew Sackman

Most of us at RabbitMQ HQ have spend time working in a number of functional languages in addition to Erlang, such as Haskell, Scheme, Lisp, OCaml or others. Whilst there is lots to like about Erlang, such as its VM/Emulator, there are inevitably features that we all miss from other languages. In my case, having spent a couple of years working in Haskell before returning to the RabbitMQ fold, all sorts of features are “missing”, such as laziness, type classes, additional infix operators, the ability to specify precedence of functions, fewer parenthesis, partial application, more consistent standard libraries and do-notation. That’s a fair list, and it’ll take me a while to get around to implementing them all in Erlang, but here are two for starters.

Read More...

Very fast and scalable topic routing - part 2

March 28, 2011 by Vlad Alexandru Ionescu

In our previous blog post we talked about a few approaches to topic routing optimization and described the two more important of these in brief. In this post, we will talk about a few things we tried when implementing the DFA, as well as some performance benchmarking we have done on the trie and the DFA.

Read More...

Sender-selected distribution

March 23, 2011 by Emile Joubert

RabbitMQ 2.4.0 introduced an extension that allows publishers to specify multiple routing keys in the CC and BCC message headers. The BCC header is removed from the message prior to delivery. Direct and topic exchanges are the only standard exchange types that make use of routing keys, therefore the routing logic of this feature only works with these exchange types.

Read More...

Ruby AMQP Benchmarks

March 1, 2011 by Jakub Stastny

I decided to run some benchmarks of my AMQP encoder/decoder (AMQ Protocol gem) against the old one in the AMQP gem to see whether it performs better or not. So far I did only the most basic optimisations like storing reusable values in constants, nothing special (yet).

I did two sets of benchmarks: CPU time benchmarking using my fork of RBench with support for custom formatters (like writing results into a YAML file) and memory benchmarking using Object.count_objects (Ruby 1.9).

Read More...

Introducing Publisher Confirms

February 10, 2011 by Alexandru Scvortov

In many messaging scenarios, you must not lose messages.  Since AMQP gives few guarantees regarding message persistence/handling, the traditional way to do this is with transactions, which can be unacceptably slow.  To remedy this problem, we introduce an extension to AMQP in the form of Lightweight Publisher Confirms.

Read More...

Who are you? Authentication and authorisation in RabbitMQ 2.3.1

February 7, 2011 by Simon MacMullen

RabbitMQ 2.3.1 introduces a couple of new plugin mechanisms, allowing you much more control over how users authenticate themselves against Rabbit, and how we determine what they are authorised to do. There are three questions of concern here:

  1. How does the client prove its identity over the wire?
  2. Where do users and authentication information (e.g. password hashes) live?
  3. Where does permission information live?

Question 1 is answered in the case of AMQP by SASL - a simple protocol for pluggable authentication mechanisms that is embedded within AMQP (and various other protocols). SASL lets a client and a server negotiate and use an authentication mechanism, without the “outer” protocol having to know any of the details about how authentication works.

SASL offers a number of “mechanisms”. Since the beginning, RabbitMQ has supported the PLAIN mechanism, which basically consists of sending a username and password over the wire in plaintext (of course possibly the whole connection might be protected by SSL). It’s also supported the variant AMQPLAIN mechanism (which is conceptually identical to PLAIN but slightly easier to implement if you have an AMQP codec lying around). RabbitMQ 2.3.1 adds a plugin system allowing you to add or configure more mechanisms, and we’ve written an example plugin which implements the SASL EXTERNAL mechanism.

Read More...

RabbitMQ, backing stores, databases and disks

January 20, 2011 by Matthew Sackman

From time to time, on our mailing list and elsewhere, the idea comes up of using a different backing store within RabbitMQ. The backing store is the bit that’s responsible for writing messages to disk (a message can be written to disk for a number of reasons) and it’s a fairly frequent suggestion to see what RabbitMQ would look like if its own backing store was replaced with another storage system.

Such a change would permit functionality that is not currently possible, for example out-of-band queue browsing, or distributed storage, but there is a fundamental difference in the nature of data storage and access patterns between a message broker such as RabbitMQ and a generic database. Indeed RabbitMQ deliberately does not store messages in such a database.

Read More...

Ruby AMQP 0.7 released!

January 19, 2011 by Jakub Stastny

I’m happy to announce that the AMQP 0.7 is released, as I promised in the previous blog post. So what are the changes?

Read More...

What's Going on with the Ruby AMQP Gem?

January 12, 2011 by Jakub Stastny

In the past year development of the AMQP gem was practicaly stagnating, as its original author Aman Gupta (@tmm1) was busy. A lot of bugs stayed unresolved, the code was getting old and out-dated and no new features or documentation were made.

At this point I started to talk with the RabbitMQ guys about possible collaboration on this. Actually originally I contacted VMware when I saw Ezra Zygmuntowicz looking for people to his cloud team, but when I found that VMware recently acquired the RabbitMQ project in London, I got interested. I signed the contract, switched from script/console to Wireshark and the RabbitMQ Tracer and since November I’ve been happily hacking on the AMQP and AMQ-Protocol gems.

Read More...