Menu

Archive for the 'Streams' Category

RabbitMQ 3.11 Feature Preview: Super Streams

July 13, 2022 by Arnaud Cogoluègnes

RabbitMQ 3.11 will bring a feature with one of the coolest names in its history: super streams. Super streams are a way to scale out by partitioning a large stream into smaller streams. They integrate with single active consumer to preserve message order within a partition.

This blog post gives an overview of super streams and the use cases they unlock. Read on to learn more, we value your feedback to make this feature the best it can be.

Read More...

RabbitMQ 3.11 Feature Preview: Single Active Consumer for Streams

July 5, 2022 by Arnaud Cogoluègnes

RabbitMQ 3.11 will bring a noteworthy feature to streams: single active consumer. Single active consumer provides exclusive consumption and consumption continuity on a stream. It is also critical to get the most out of super streams, our solution for partitioning, that provide scalability for streams.

Read on to find out more about single active consumer for streams and don’t hesitate to experiment with what is already available: try it, break it, tell us what you like and don’t like, what’s missing. Your feedback is essential to make this feature the best it can be.

Read More...

Interoperability in RabbitMQ Streams

October 7, 2021 by Arnaud Cogoluègnes

RabbitMQ streams allow applications to convey detailled information thanks to the powerful message format they use. Streams are a feature of their own, but they also fully integrate with the existing resources and protocols that RabbitMQ supports. This blog post covers the interoperability of streams in RabbitMQ and explores the scenarios it unlocks.

Read More...

Offset Tracking with RabbitMQ Streams

September 13, 2021 by Arnaud Cogoluègnes

RabbitMQ Streams provides server-side offset tracking for consumers. This features allows a consuming application to restart consuming where it left off in a previous run. This post covers the semantics of offset tracking and how it is implemented in the stream Java client.

Read More...

Message Deduplication with RabbitMQ Streams

July 28, 2021 by Arnaud Cogoluègnes

RabbitMQ Streams Overview introduced streams, a new feature in RabbitMQ 3.9 and RabbitMQ Streams First Application provided an overview of the programming model with the stream Java client. This post covers how to deduplicate published messages in RabbitMQ Streams.

As deduplication is a critical and intricate concept, the post will walk you through this mechanism step by step, from a naive and somewhat broken publishing application to an optimized and reliable implementation.

Read More...

Connecting to Streams

July 23, 2021 by Arnaud Cogoluègnes

RabbitMQ Streams Overview introduced streams, a new feature in RabbitMQ 3.9. This post covers how client applications should connect to RabbitMQ nodes to get the most benefit from streams when the stream protocol is in use.

Streams are optimized for high throughput scenarios, that’s why technical details like data locality are critical to get the best out of your RabbitMQ cluster. Client libraries can handle most of the details, but a basic understanding of how things work under the hood is essential when a setup involves extra layers like containers and load balancers. Keep reading if you want to learn more about streams and avoid some headaches when deploying your first stream applications!

Read More...

First Application With RabbitMQ Streams

July 19, 2021 by Arnaud Cogoluègnes

RabbitMQ Streams Overview introduced streams, a new feature in RabbitMQ 3.9. This post continues by showing how to use streams with the Java client. We will write our first application that publishes messages to a stream, and then consumes them.

Read More...

RabbitMQ Streams Overview

July 13, 2021 by Arnaud Cogoluègnes

RabbitMQ 3.9 introduces a new type of data structure: streams. Streams unlock a set of use cases that could have been tedious to implement with “traditional” queues. Let’s discover in this post how streams expand the capabilities of RabbitMQ.

Read More...