Application Scaling Using Redis

Data in this digitized world is growing at an exponential pace, doubling itself in size every few years. Regular RDBMS and older databases are simply lacking behind. The frequent read and writes on disc storage database costs a performance drop in applications. To process huge amounts of data in the desired timeline, we need features like fast processing, in-memory computations, in-memory storage (NoSQL), etc. There are various business scenarios where we need to store frequently required data into an in-memory store, to scale up applications to larger extend. Below listed examples, depict some cases where in-memory storage is required:

Horizontal scale (Scale Out): If we are using application servers in a cluster, then we need to have session replication across the clusters. This would require all the sessions to be replicated on all other clusters and result into high replication traffic and thus have an adverse impact on application scalability.

We can solve this problem easily by using a centralized in-memory session store, where all cluster servers will check for the user session. This will also help to scale out the application as we can add any number of clusters as and when load increases.

Media publishing: Some NoSQL’s support key/value storage and that value can range up to 512 MB. This feature enables us to serve media content (audios/videos and images) from in-memory storage for faster access. This is beneficial especially for Live streaming sites.

Frequent Reads: We can access frequently required key value data into NoSQL In-Memory store as well. This will significantly improve performance as we can avoid many reads and writes to disc based RDBMS’s.

Public/Subscribe Model: Some scenarios require event publishing and subscribing mechanisms. Example: Sending blogs to all followers who have subscribed to a particular topic.

Using Redis NoSQL we can solve all of the above-mentioned problems. Redis supports data persistence and has an advanced key-value cache and store. It takes a snapshot of the in-memory store and restores the state back in situations involving a server crash/server restart. Let us look at the features supported by Redis:

• Key-value cache: Redis acts as a data structure server where the key may be associated with values like- strings, hashes, lists, sets, sorted sets, bitmaps and hyper logs. Its value size can range up to 512 MB.
• Transactions: Redis transactions are atomic in nature, meaning, all the commands would be processed or none of them would pass. This property would make sure that if a client loses the connection before the MULTI command is invoked, none of the operations related to that transaction would be performed.
• Pub/Sub: The publish/subscribe messaging characterizes the messages into channels, thus enabling subscribers to receive only the messages they have expressed their interest in
• Persistence: Redis supports Persist data on time intervals (RDB) or logs every operation in a specific format (AOF) so that the server can restore data for re-executing commands
• Replication: Redis works as a Master/Slave architecture; generally slaves are exact copy of masters
• Automatic failover: Redis ensures high availability of the application with Sentinel. With Sentinel, a user can implement Redis deployment where during a failure, a new master can be configured to take over and ensure continuity in operations.

Horizontal scaling in Redis

As the above diagram shows, Redis can be used for two purposes; first is a centralized session store and the other where most frequent read only data of primary database can be cached into Redis e.g. when user logs into the system, we can cache users roles, permissions, groups into Redis cache and on logout it would be deleted. When user logs into the application, a session is created onto Redis. Later on, as requests may go on other cluster servers, they will check the session on Redis and if it exists, serve the request. Using this approach, we can add more clusters if server load exceeds threshold limit and helps to scale the application horizontally.