A Pub/Sub Model Using Redis Stream

Redis is more than an in-memory caching database. Along with its various use cases, it offers the powerful Redis Stream feature, enabling it to function as a robust Pub/Sub model. In this article, we will explore how to leverage Redis Stream as the backbone of the Pub/Sub model. By publishing messages to a channel and subscribing to that channel, we can effectively utilize the capabilities of Redis Stream.

The Pub/Sub model we will implement can be used for bulk processing of various tasks. For example, it can be used for sending bulk SMS, email, heavy calculation, and overall does not hold the API request for a long time.

Application Overview with redis stream

We will have a Redis server. This Redis server provides a publishing client and an API server will publish messages to a channel using this client. On the other hand, for each worker, there will be individual subscription clients. Workers will subscribe to the channels and any time a message is published, it will read the message.

So according to the diagram, the API server will receive a request. Then according to the request, it will go to the Redis server and publish a message. The workers are always in the standby status and workers will consume the message.

Code Architecture

We will make use of 3 different containers, api-container, worker-container, and redis-container. Both the api-container and worker-container are connected with the same Redis server. The api-container uses the publisher to publish the message and the worker-container subscribed to the messages.

Code Snippet

We can simply publish a message,

const data = {
  uuid: Math.random().toString(),
  title: 'Testing redis stream',
  description: 'An article on Redis Stream from Ixora Solution Ltd.',
  timestamp: new Date().toISOString()
};
await publishClient.publish(REDIS_STREAM_CHANNEL, JSON.stringify(data));Code language: JavaScript (javascript)

To subscribe to a message,

await subscribeClient.subscribe(REDIS_STREAM_CHANNEL, message => {
  console.log(message);
});Code language: JavaScript (javascript)

Now it is important, although we are using the same Redis database, we need a different Redis client, can not use the same client for both publish and subscribe messages.

For getting a Redis client,

import { createClient } from 'redis';

const publishClient = createClient({
  socket: {
    host: process.env.REDIS_HOST,
    port: process.env.REDIS_DOCKER_PORT as unknown as number
  },
  password: process.env.REDIS_PASSWORD
}); Code language: JavaScript (javascript)

Workflow

First, clone the code and run the app docker-compose up

From the browser, when we go to http://localhost:3000/

A message will be streamed to the Redis and our worker will consume the message as well as print it in the console.

Features

This is a basic implementation of Redis. However, Redis has many advanced features like regular queue service,

• Can scale the server vertically and horizontally
• Allow acknowledgment from the consumer as it is being processed or processed
• Allow setting a TTL, so if there is no acknowledgment from the consumer, the publisher makes the message visible again to be proceeded by a different consumer

Conclusion

A complete code base is already in Github.

Experience the iXora Solution difference as your trusted offshore software development partner. We’re here to empower your vision with dedicated, extended, and remote software development teams. Our agile processes and tailored software development services optimize your projects, ensuring efficiency and success. At iXora Solution, we thrive in a dynamic team culture and experience innovation in the field of custom-made software development.

Have specific project requirements? Personalized or customized software solutions! You can contact iXora Solution expert teams for any consultation or coordination from here. We are committed to maximizing your business growth with our expertise as a custom software development and offshore solutions provider. Let’s make your goals a reality.
Thanks for your patience!