Message Queue Implementation using RabbitMQ
Introduction
Rapidly increasing count of electronic gadgets connected to the internet has given birth to a new IT jargon, Internet-of-things (IoT). It is a system of interconnected computing devices, digital and mechanical machines identified by a unique number/name, and its facility to transfer data over a network without requiring human-to-computer or human-to-human interaction. Therefore, this requirement of improving system integration triggered the development of Message-Brokers, which facilitate the inter-application communication technology to support building a common integration mechanism to support cloud native, microservices-based, serverless and hybrid cloud architectures.
Why Did Message Brokers emerge?
Can you imagine the existing volume of data for gadgets connected to the internet across the globe? Nowadays, approximately 12 billion Smart-Machines are connected to it. Bearing in mind that currently, around 7 billion people live on this planet, it is almost one-and-a-half devices per person. By the end of this year, this number could presumably surge to 200 billion, or even further. With technological development, like the building of Smart-Houses and other automatic systems, our everyday life becomes more and more digitized.
What is a message broker?
Wikipedia’s formal definition says: “A message broker translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver.”
Functional features for a message broker
- A message broker is computer software that enables apps, systems, and services to communicate with each other by exchanging information, irrespective of programming languages used to build them up or software/hardware platforms wherein they are deployed.
- Message brokers are software programs deployed in messaging middleware or message-oriented middleware (MOM) solutions.
- Message brokers can validate, store, route, and deliver messages to the intended destinations. This enables decoupling of services and processes within systems.
- Built-in “Message-Queues” hold intact messages until not delivered to intended destinations and ensure message reliability and guaranteed delivery.
- Message brokers provide queue managers to handle the interactions between multiple message queues, as well as services providing data routing, message translation, persistence, and client state management functionalities.
- Supports Asynchronous messaging, which prevents the loss of valuable data and enables systems to continue functioning even in the face of the intermittent connectivity or latency issues common on public networks.
Message broker implementation patterns
Message brokers can be configured in two basic message distribution patterns:
Publish/subscribe messaging: Often referred to as “Publisher/Subscriber”, wherein the producer of each message publishes it to a queue, and multiple message consumers subscribe to a queue from which they want to receive messages. All messages published to a queue are distributed to all the applications subscribed to it. It is a broadcast-style distribution method, depicting a one-to-many relationship between the message’s publisher and its consumers. For example, an airline was to circulate updates about the landing times or delay status of its flights, multiple parties could make use of the information: ground crews performing aircraft maintenance and refuelling, baggage handlers, flight attendants and pilots preparing for the plane’s next trip, and the operators of visual displays notifying the public. A pub/sub messaging style would be suitable for this scenario.
Note: Third possible pattern could be — Hybrid of both
Best Message Broker Software Tools
These tools are typically leveraged by IT professionals like system administrators, and software developers. Business organizations use this software to synchronise distributed applications, simplify coding dissimilar applications, improve performance, and automate communication-related tasks.
A software tool must possess below given to qualify as a Message Broker:
- Facilitate asynchronous messaging
- Store, deliver, and delete messages
- Document communication information
- Allow administrative control over messaging permissions
Major available tools
There are numerous message broker software tools available in the market which can be the best candidate for a given factors like Flexible Message Routing and Priority configurations, Message Replay, UI to monitor messages etc. E.g. RabbitMQ is a great fit for Flexible Message Routing and Priority configurations and Kafka is a great choice for message replay. Next, we will explore RabbitMQ as it covers the maximum share in the current market.
RabbitMQ a most widely deployed open source message broker
RabbitMQ is the most popular open-source message broker, having a huge number of production deployments world-wide. It is lightweight, and easy to deploy on-premises and in the cloud and execute on all major operating systems. It supports multiple messaging protocols, maximum developer platforms and can be deployed in distributed and federated configurations to satisfy high-scale, high-availability requirements.
RabbitMQ Features
Asynchronous Messaging
It supports multiple messaging protocols, message queuing, multiple exchange types, delivery acknowledgement, flexible routing to queues.
Developer Experience
It allows developers to deploy with Docker, BOSH, Chef and Puppet. Cross-language messaging with favourite programming languages such as: .NET, Java, PHP, Python, JavaScript, Ruby, and many others.
Distributed Deployment
Offers Clustered deployment for high availability and throughput and federate across multiple availability zones and regions.
Enterprise & Cloud Ready
It supports pluggable authentication, authorisation as well as supports LDAP and TLS. Lightweight and easy to deploy across public and private clouds.
Tools & Plugins
Variety of plugins and tools supporting continuous integration, operational metrics, and integration with other enterprise systems. Flexible plug-in approach for extending its functionality.
Management & Monitoring
It supports HTTP-API, command line tool, and UI for managing and monitoring itself.
.NET Libraries for RabbitMQ
Below are given two client libraries which help to integrate .NET applications with RabbitMQ very easily.
- RabbitMQ .NET Client (supports .NET Core and .NET 4.5.1+)
- RawRabbit, a higher-level client that targets ASP.NET next and supports .NET Core.
Note: Here in, I will be covering only RawRabbit by highlighting its core features and providing essential code snippets.
RawRabbit
It is a modern .NET framework for communication over RabbitMQ. The modular design and middleware-oriented architecture makes the client highly customizable while providing workable default for topology, routing and more.
Easily Configurable and extendable
RawRabbit can be easily configured with RawRabbitOptions, an option specifying object which allows registering client configuration, plugins, and permits overriding internal services.
var client = RawRabbitFactory.CreateSingleton(new RawRabbitOptions
{
ClientConfiguration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("rawrabbit.json")
.Build()
.Get<RawRabbitConfiguration>(),
Plugins = plugin => plugin
.UseProtobuf()
.UsePolly(config => config
.UsePolicy(queueBindPolicy, PolicyKeys.QueueBind)
.UsePolicy(queueDeclarePolicy, PolicyKeys.QueueDeclare)
.UsePolicy(exchangeDeclarePolicy, PolicyKeys.ExchangeDeclare)
),
DependencyInjection = ioc => ioc
.AddSingleton<IChannelFactory, CustomChannelFactory>()
});Configuring Publish/Subscribe
Just a few lines of code and you can set up strongly typed publish/subscribe.
var rawClient = RawRabbitFactory.CreateSingleton();
await rawClient.SubscribeAsync<BasicMessage>(async message =>
{
Console.WriteLine($"Received: { message.Prop}.");
});await rawClient.PublishAsync(new BasicMessage { Prop = "Hello, world!"});
Configuring Request/Response
RawRabbit’s ( RPC) request/response configuration uses the direct reply-to feature for better performance and lower resource allocation.
var rawClient = RawRabbitFactory.CreateSingleton();
rawClient.RespondAsync<BasicRequest, BasicResponse>(async request =>
{
return new BasicResponse();
});
var rawResponse = await client.RequestAsync<BasicRequest, BasicResponse>();Configuring Ack, Nack, Reject and Retry
Unlike many other clients, basic.ack, basic.nack and basic.reject are first class citizens in the message handling.
var client = RawRabbitFactory.CreateSingleton();
await client.SubscribeAsync<BasicMessage>(async message =>
{
if(UnableToProcessMessage(message))
{
return new Nack(requeue: true);
}
ProcessMessage(message)
return new Ack();
});In addition to the essential acknowledgements, RawRabbit also supports delayed retries
var client = RawRabbitFactory.CreateSingleton();
await client.SubscribeAsync<BasicMessage>(async message =>
{
try
{
ProcessMessage(message)
return new Ack();
}
catch (Exception e)
{
return Retry.In(TimeSpan.FromSeconds(30));
}
});Granular control over each call
Allow users to add or change properties in the IPipeContext to adapt calls for specific types of messages. This makes it possible to modify the topology features for calls, publish confirm timeout, consumer concurrency.
await subscriber.SubscribeAsync<BasicMessage>(received =>
{
receivedTcs.TrySetResult(received);
return Task.FromResult(true);
}, context => context
.UseSubscribeConfiguration(config => config
.Consume(consume => consume
.WithRoutingKey("CustomKey")
.WithConsumerTag("CustomTag")
.WithPrefetchCount(2)
.WithNoLocal(false))
.FromDeclaredQueue(queue => queue
.WithName("CustomQueue")
.WithAutoDelete()
.WithArgument(QueueArgument.DeadLetterExchange, "dlx"))
.OnDeclaredExchange(exchange=> exchange
.WithName("CustomExchange")
.WithType(ExchangeType.Topic))
));Originally published at https://www.neovasolutions.com on May 13, 2020.
Connect with us:
