Sources

Unfortunately, though, Redis key-value stores don't always work the way they should. You may run into issues like slow performance due to low hit rates and poorly sharded data. Problems like these must be identified and fixed, otherwise, what's the point of paying for an in-memory key-value store if it's not living up to its full potential? … ### Large JSON keys Using large JSON keys instead of Redis hashes is another common Redis issue. It happens when you use a single key to hold a JSON value as a string, causing lookups in your apps to be very inefficient. A simple solution is to hold the data in a hash so you get a full lookup using a single field in O(1) complexity. … ### Poorly sharded data Redis clusters spread their data across many nodes. When you use a Redis cluster with a general-purpose hash instead of using multiple keys, your cluster can suffer a performance hit. This happens because the key is stored on a single node, and in a high-scale environment, the pressure will fall on that node instead of being distributed between all of the nodes in the cluster. The result is that the node becomes a performance bottleneck. As a real-world example, consider a cluster that stores user data in a hash, where the key is the user ID. An authentication server that performs a lot of lookups on the user ID will place heavy pressure on the node that stores the key. A solution would be to spread the hashed data to multiple keys across nodes, letting Redis's sharding algorithm distribute the pressure. … Here, we perform pipelined requests on three individual keys. The requests execute on a single node, and if one of the keys is not on that specific node, the commands with the keys that are on that node will return a response while the others will return the MOVED error. A quick fix is to use hashtags in the key structure, which means simply adding curly brackets around the part of the key that we want to hash by will cause the sharding algorithm to direct the values to the same node: … - **Multiple points of failure**: When something goes wrong in Redis, there are typically multiple potential causes. For example, high latency could stem from increased system load, lack of available memory, or poorly structured requests, to name just a few possibilities. To monitor and troubleshoot effectively, you need to be able to explore each potential root cause quickly. This requires the ability to correlate and analyze a variety of data points.

### 2 - all the data types you need in one place Redis is a NoSQL database. This means you won’t get the old SQL transactions, tables, foreign and unique key-contraints, etc. I was ‘raised’ as a developer in a time where SQL was the only option, so we did everything with it. While it is convenient you don’t have to think about data consistency in your code, there are serious drawbacks as well, mostly in terms of speed, data size, and scalability. When I reflect on old projects, I wonder how often I really needed that absolute - stop the world - level of forced consistency in the database layer.

1. **Memory Consumption**: Redis is an in-memory data store, which means it can consume a significant amount of memory. If not monitored and managed properly, this can lead to high memory usage, potentially causing the system to slow down or even crash. Example: A Redis instance running on a server with 16GB of RAM might start to experience performance issues if it consumes more than 8GB of memory, leaving insufficient space for the operating system and other applications. 2. **Persistence Issues**: Redis offers several persistence options like RDB snapshots and AOF logs. Misconfiguration or issues with these mechanisms can lead to data loss. Example: If Redis is configured to persist data only through RDB snapshots and the server crashes before a snapshot is taken, any data changes since the last snapshot will be lost. 3. **Network Latency**: Redis performance can be severely impacted by network latency, especially in distributed environments where Redis instances are spread across multiple servers or data centers. Example: A Redis cluster spanning across two geographically distant data centers might experience high latency due to the physical distance between the servers, leading to slow response times. 4. **Configuration Errors**: Misconfigurations in Redis settings, such as maxmemory policies, timeout settings, or binding IP addresses, can lead to unexpected behavior or security vulnerabilities. Example: Setting the `maxmemory` policy too low might cause Redis to evict keys prematurely, leading to data loss or application errors. 5. **Concurrency Issues**: While Redis is generally good at handling concurrent operations, improper use of commands that modify data (like INCR, HSET, etc.) without proper locking mechanisms can lead to race conditions and inconsistent data. Example: Two clients simultaneously updating the same key without proper locking might result in one of the updates being overwritten, leading to data inconsistency.

### Memory-intensive and not ideal for large datasets While Redis’s approach to storing data in RAM contributes to its speed, this has its consequences. RAM is significantly more expensive than disk storage, which means that using Redis for large datasets can become expensive, especially when scaling up. Companies that store and process terabytes of data must make significant investments when working with Redis. This is why it’s rarely used as a standalone solution and is often paired with other databases to balance performance and cost. ### Manual memory management Redis does not automatically manage memory like relational databases do. Developers must manually configure eviction policies to decide what happens when memory is full. This disadvantage is addressed once you move to Redis Cloud since it’s a fully-managed Redis service.

The inverse of this is that Redis becomes a major single point of failure, making reliability especially important. After doing some research into deployment architectures for Redis, it appears that it only supports master-slave replication, with a single master handling writes and X number of read-only slaves. Replication happens over a TCP/IP port, while password authentication can be enabled between the nodes to increase security. There is a clustering solution in development, but it is not production ready 2. There are a number of issues with the current Redis master-slave architecture: "There are several problems that surface when a slave attempts to synchronize off a master that manages a large dataset (about 25GB or more). Firstly, the master server may require a lot of RAM, even up to 3X the size of the database, during the snapshotting. Although also true for small databases, this requirement becomes harder to fulfill the bigger the database grows. Secondly, the bigger the dataset is, the longer and harder it is to fork another process for snapshotting purposes, which directly affects the main server process. This phenomenon is called "latency due to fork" and is explained here and at redis.io." … 4 Naturally you would only want to run Redis on a private network. Within that network however, I have many different projects and developers using that same resource, and sadly Redis provides no authentication system beyond a single global password from which to tell them apart. 5 A single rogue client issuing a *flushall* command for example would wipe out the databases of all users of the service, not cool. My thoughts on this presently is that I will have to build an authentication system in front of Redis myself as part of my public API. … session.save_handler = redis session.save_path = tcp://127.0.0.1:6379 Finally, we have long term storage. For me this is the problem child of the bunch, as I do not think that the replication model in Redis will support very large data sets. I know that replication lag will increase with larger data sets, as will the overhead of carrying out the replication. Until the clustered solution becomes production ready, I am not ready to use Redis as a full replacement for MySQL just yet.

Disclaimer – all of these problems arose from our use case and not because Redis is somewhat flawed. Like any piece of software it requires understanding and research before deployed in any decent production environment. We have a data collecting pipeline with the following requirements: Given our requirements we **started to use Redis cluster from the start**. We chose it over single master/replica because we couldn’t fit our 800M+ keys on a single instance and because Redis cluster provides high availability kinda out of the box (you still need to create the cluster with ``` redis-trib.rb ``` or ``` redis-cli --cluster create ``` ). Also, such beefy nodes are very hard to manage – loading of the dataset would take about an hour, the snapshot would take a long time So, I’ve setup Redis cluster and this time I did it without cross replication because I’ve used Google Cloud instances and because cross replication is very tedious to configure and painful to maintain.

## Common Redis® Challenges When dealing with Redis®, there can be various problems such as performance bottlenecks and scaling issues that may affect the efficacy of your instances. It is important to understand these challenges properly to find solutions for them. Examples include slow command execution time which leads to poor latency and increased memory usage. Or even having limitations when trying vertical/horizontal scalability while ensuring availability at all times. By monitoring conditions closely, you’ll have better control over how your Redis® systems are running so they remain top-notch in terms of reliability and performance. ### Redis® Performance Bottlenecks Analyzing some key metrics such as memory usage, command processing throughput, active connections and cache hit ratio can help identify Redis® performance bottlenecks. To improve the speed of your system, there are various options you may want to consider – like using slowlogs for tracking down commands that take too long, optimizing hash functions, enabling TCP Keepalive or investigating eviction bursts. Taking these measures should result in a better functioning application with fewer lags and faster response times, which leads to improved overall Redis® performance.

# The Redis Exodus: Why We're Returning to Database-Backed Queues ## — Commercial License Changes and the Return to Second-Generation Queue Management Something interesting happened in our industry recently. Redis, the in-memory data store that became synonymous with modern web architecture, suddenly feels less inevitable than it once did. The licensing changes and operational costs have prompted many e-commerce platforms to reconsider what was once an obvious choice. In the Rails community especially, the "Redis or nothing" mentality is giving way to something more nuanced. … |Licensing|Previously fully open source.|Commercial feature licensing changes raise future uncertainty concerns.| **Key Point**: Redis still excels in many use cases like caching and pub/sub. However, the reality in 2025 is that more workloads can conclude "persistent layer in DB, queues in DB too" is sufficient. ## Rethinking Infrastructure Costs and Maintenance Load **Physical Costs:** Being in-memory means data volume equals memory requirements. As application count and transactions increase, costs scale accordingly. **Operational Costs:** Building and maintaining Redis Sentinel/Cluster configurations, handling failures, and managing version upgrades requires specialized knowledge separate from RDB operations. Even cloud services require ongoing management.

Scalability is another challenge with Redis. Though Redis offers mechanisms such as Redis Cluster, managing large-scale deployments remains complex and resource-intensive. Redis was originally designed as a single-instance in-memory data store. While clustering can distribute data, it introduces additional complexity in terms of cluster management, memory, and CPU overhead. Redis struggles to take advantage of multi-core processors, meaning scaling up requires more instances, which increases both hardware costs and operational complexity. As data grows, Redis’ single-threaded design, while working well enough in smaller environments, doesn’t take advantage of today’s multi-core architectures, often leading to underutilized hardware. Operational overhead is also a concern. Maintaining high availability and data consistency requires careful handling. In self-managed environments, administrators need to configure and monitor replication, failover processes, and persistence mechanisms, all of which add complexity and increase the chances of downtime. While Redis supports replication and persistence through features such as AOF (Append-Only Files) and snapshots, these mechanisms are not foolproof. For mission-critical workloads, where data loss is unacceptable, Redis' persistence models may fall short, especially as replication can introduce latency and affect consistency in real-time applications. In addition to scalability and operational challenges, Redis may not have all the features some companies need. For instance, advanced data consistency and complex queries are areas where Redis often falls behind. Redis provides eventual consistency at best, which works well for many caching and session management scenarios. However, for applications that demand strong consistency or transactional guarantees, Redis’ replication mechanisms, designed to offer basic data redundancy, can introduce latency or inconsistency during network partitions, making it more difficult to provide a real-time response. Redis also lacks built-in support for advanced querying (such as joins, aggregations, or full-text search), which limits its use cases to simpler key-value and caching scenarios. This can be a roadblock for applications that need more sophisticated data processing capabilities. This is particularly true with the Redis Community Edition, which lacks the enterprise-level support and advanced features available in Redis Enterprise. Without enterprise support, organizations may have trouble addressing performance bottlenecks, troubleshooting issues, or receiving critical security updates. As businesses grow and their needs become more complex, they often require richer functionality, such as full-text search, time-series data management, or graph processing, which Redis doesn't natively provide. Organizations may need to evaluate whether their current Redis setup meets their evolving requirements or whether a database with more features would be a better fit. … ### Data volume constraints Redis, as an in-memory data store, requires all data to reside in RAM for high performance. While Redis Cloud offers managed instances with higher memory capacity, the total amount of data you can handle is limited by the available RAM on your Redis instance. This becomes a challenge when your dataset grows beyond the memory limits of your infrastructure, resulting in additional costs and potential performance degradation. - **Memory overhead**: Redis is designed for speed, but storing large datasets can use a lot of memory. If you're managing millions of keys or large objects, the memory overhead per Redis instance increases, putting pressure on both your infrastructure and budget. - **Ephemeral storage**: Redis stores data primarily in memory. While Redis provides persistence mechanisms such as RDB snapshots and AOF logs, they are not foolproof. These methods can lead to data loss in the event of a crash or unexpected failure, especially when persistence is disabled for performance reasons. … - **Replication overhead**: Redis replication creates read replicas of a primary node, but the replication process can introduce latency, especially in geographically distributed clusters. Any network disruptions or high-latency conditions between the primary and secondary Redis instances can cause inconsistent or delayed data across your replicas. - **Cluster management**: Redis Cluster helps scale Redis horizontally, but managing a Redis Cluster adds additional overhead. When scaling your Redis instances or performing online migrations, managing the partitioning of data and rebalancing the cluster uses a lot of resources and is prone to errors. ### Migration complexity Migrating Redis data, especially in large-scale or mission-critical environments, is complicated to run. - **Data consistency**: During migration or scaling, maintaining data consistency across Redis instances becomes a challenge. Although Redis supports online migration techniques, such as the Migrate command, ensuring consistent data transfers while maintaining availability is tricky and requires planning. - **Instance coordination**: When migrating or scaling, coordination between Redis instances is important. Redis’ single-threaded nature means coordinating large datasets across multiple instances can use a lot of resources, and improper synchronization can lead to downtime or data inconsistencies. When considering Redis for larger or more complex use cases, it is important to evaluate the technical limitations related to data volume, schema flexibility, replication, and migration. While Redis provides high-performance data access, operational challenges related to memory management, data consistency, and high availability can limit its effectiveness for certain workloads. By understanding these constraints, system architects and DBAs can make more informed decisions about whether Redis can meet their long-term needs or if an alternative solution may be more suitable.

- **Large databases on a single shard** — keep shards under 25 GB or 25K ops/sec > - **Direct connections without a proxy** — use a connection proxy to prevent reconnect floods > - **Caching keys without TTL** — always set expiration on cache keys to prevent unbounded growth > - **Hot keys** — distribute frequently accessed data across multiple shards > - **Using the KEYS command** — use `SCAN` or Redis Search instead > - **Storing JSON blobs in strings** — use HASH structures or Redis JSON > - **Running ephemeral Redis as a primary database** — enable persistence and high availability > - **Endless replication loops** — tune replica and client buffers for large active databases Devs don't just use Redis, they love it. ... - How to identify the most critical Redis anti-patterns in your application - Why single-shard deployments and direct connections cause reliability problems - The performance impact of missing TTLs, hot keys, and the `KEYS` command - Best practices for data modeling with HASH structures and Redis JSON ... ## # Anti-pattern summary |#|Anti-pattern|Severity|Impact| |--|--|--|--| |1|Large database on a single shard|High|Slow failover, long backup/recovery| |2|Connecting directly to Redis instances|High|Reconnect floods, forced failovers| |3|Incorrect replica count (open source)|Medium|Split-brain risk| |4|Serial single operations (no pipelining)|Medium|Increased latency, wasted round-trips| |5|Caching keys without TTL|High|Unbounded memory growth, eviction storms| |6|Endless replication loop|Medium|Replication never completes| |7|Hot keys|High|Single-node bottleneck in clusters| |8|Using the KEYS command|High|Blocks Redis, O(N) full scan| |9|Ephemeral Redis as primary database|High|Data loss, downtime on restart| |10|Storing JSON blobs in strings|Medium|Expensive parsing, no atomic field updates| |11|HASH without considering query patterns|Medium|Limited filtering, full scans required| … ## # 1. Large databases running on a single shard/Redis instance **What is the single-shard anti-pattern?** Running a large dataset on one Redis instance means that failover, backup, and recovery all take significantly longer. If that single instance goes down, the blast radius covers your entire dataset. With large databases running on a single shard/Redis instance, there are chances that the fail over, backup and recovery all will take longer. Hence, it's always recommended to keep shards to recommended sizes. General conservative rule of thumb is 25Gb or 25K Ops/Second. … ## # 2. Connecting directly to Redis instances **What is the direct-connection anti-pattern?** When many clients connect directly to Redis without a proxy, a reconnect flood after a network hiccup can overwhelm the single-threaded Redis process and force a failover. With many clients, a reconnect flood will be able to simply overwhelm a single threaded Redis process and force a failover. … ## # 5. Caching keys without TTL **What is the missing-TTL anti-pattern?** Storing cache keys without an expiration means they accumulate indefinitely. Over time this leads to unbounded memory growth, increased eviction pressure, and potentially out-of-memory errors. Redis functions primarily as a key-value store. It is possible to set timeout values on these keys. … ## # 9. Running Ephemeral Redis as a primary database **What is the ephemeral-primary anti-pattern?** Using Redis as your application's primary database without enabling persistence or high availability means a restart results in complete data loss, and any downtime takes your entire application offline. Redis is often used as a primary storage engine for applications. Unlike using Redis as a cache, using Redis as a primary database requires two extra features to be effective.

## Anti-Pattern 1: Treating Redis as a Primary DatabaseDaThis is the root of many problems. Redis is fast, flexible, and easy to use. That makes it tempting to store more and more critical data in it. Over time, Redis stops being a cache and quietly becomes the system of record. Then a restart happens. Or a failover. Or a misconfiguration. And suddenly people realize Redis was holding data that could not be easily recovered. Redis can persist data, but its persistence model is not designed to replace a traditional database for most workloads. Using Redis as the primary store for critical business data without strong guarantees is a gamble. If losing Redis would cause irreversible data loss, you are likely using it incorrectly. ## Anti-Pattern 2: Missing or Infinite TTLsitFew Redis mistakes are as common or as subtle. Keys get added without TTLs because “this data never changes” or “we will invalidate it manually.” Months later, assumptions change. Data changes. Bugs are introduced. Without TTLs, bad data lives forever. Over time, memory usage creeps up. Eviction becomes unpredictable. Debugging becomes painful because nobody knows which keys are still relevant. … ## Anti-Pattern 3: Using Redis as a Dumping Ground for Large Objects ORedis is optimized for many small values. It is not optimized for a few massive ones. Storing large JSON blobs, binary payloads, or entire documents in Redis often starts innocently. It reduces database calls. It simplifies code. Then serialization costs grow. Network traffic increases. Latency spikes. Evictions become expensive. Persistence slows down. Large values amplify every Redis operation. They also make tuning and scaling harder. If a value is large and long lived, Redis may not be the right place for it. ## Anti-Pattern 4: Poor Key Design and Unbounded CardinalitydiKeys are easy to create. That is part of the problem. Teams include user input directly in keys. Search queries. URLs. Session identifiers. Anything that seems unique. The result is unbounded key growth. Redis does not warn you when cardinality explodes. It simply allocates memory until it can’t. Good Redis systems have predictable key counts. Bad ones grow forever until eviction behavior becomes chaotic. If you cannot predict how many keys will exist, you probably have a key design problem. … ## Anti-Pattern 7: Overusing Lua Scripts SLua scripts are powerful. They allow atomic operations and complex logic. They also block Redis while running. Teams sometimes move business logic into Redis using Lua because it feels fast and elegant. Over time, scripts grow. Data sizes grow. Execution time grows. Then Redis starts blocking under load. Lua should be used sparingly and carefully. If a script’s runtime depends on data size or unbounded loops, it is a liability. … ## Anti-Pattern 12: Over-Optimizing Too EarlyooSome teams aggressively tune Redis before they understand their workload. They tweak memory policies. They change persistence settings. They add clustering and sharding prematurely. This often adds complexity without solving real problems. Redis works extremely well with sensible defaults. Premature optimization can introduce more failure modes than it removes. Measure first. Optimize second. ## Anti-Pattern 13: Mixing Critical and Non-Critical DatacaPutting everything into one Redis instance feels convenient. Cache data. Locks. Queues. Sessions. Feature flags. Counters. Over time, eviction pressure and performance characteristics collide. Evictions intended for cache data affect critical locks. Memory pressure impacts queues. Different data has different priorities. Mixing them increases blast radius. Separating concerns, either logically or physically, reduces risk. … ## Anti-Pattern 15: Assuming Redis Problems Are Redis’s Fault’sThis may be the most subtle anti-pattern. When things go wrong, Redis gets blamed. In reality, Redis is often doing exactly what it was told to do. Bad key design. Missing TTLs. Expensive commands. Poor client behavior. Redis amplifies design decisions. Good decisions scale smoothly. Bad decisions fail loudly. … ## Summary SRedis anti-patterns are dangerous because they do not look dangerous at first. They look like convenience. Like speed. Like progress. Over time, they turn Redis from a reliable accelerator into a source of instability. The good news is that most of these mistakes are avoidable once you know what to look for. Redis rewards discipline. It punishes shortcuts.

### Performance Limitations One of the main limitations of using Redis in development projects is its performance limitations. While Redis is known for its high speed and low latency, it may not be suitable for all use cases, especially when dealing with large datasets or high throughput requirements. As the amount of data stored in Redis increases, the performance may degrade, leading to increased response times and potential bottlenecks in your application. … ### Scalability Challenges Another limitation of using Redis in development projects is scalability challenges. While Redis is designed to be fast and efficient, scaling it horizontally to handle increasing workloads can be a complex and challenging process. Horizontal scaling in Redis involves setting up multiple instances and implementing sharding techniques to distribute data across different nodes. This process can be time-consuming and require careful planning to ensure data consistency and high availability. … While Redis offers many benefits for developers, including fast performance, scalability, and high availability, it is essential to be aware of its limitations and drawbacks when using it in development projects. Performance limitations, scalability challenges, and data persistence issues are some of the key factors to consider when evaluating the use of Redis in your application. … ## Exploring the Limitations of Using Redis in Development Projects ### Difficulty with Complex Data Structures While Redis excels at storing simple key-value pairs, it can be challenging to work with more complex data structures. For example, Redis does not have built-in support for nested data structures like arrays or objects. This can make it difficult to represent and manipulate data that requires more complex relationships. In cases where developers need to work with complex data structures, they may need to implement additional logic to serialize and deserialize the data into a format that Redis can handle. This can add complexity to the codebase and potentially impact the performance of the application. Additionally, Redis lacks support for certain data types like sets, maps, and graphs. While developers can work around these limitations by utilizing Redis commands and data structures creatively, it may not always be the most efficient or elegant solution. ### Performance Concerns While Redis is known for its high performance, there are certain scenarios where it may not be the best choice for optimizing speed. For example, when dealing with large datasets that exceed the available memory capacity, Redis can start paging data to disk, which can significantly impact performance. Another performance consideration is the network overhead of using Redis in a distributed environment. When Redis is deployed across multiple nodes or data centers, there can be latency issues that affect the overall performance of the application. Developers should also be mindful of the potential for data loss in Redis. While Redis offers persistence options like snapshots and append-only files, there is still a risk of data loss if these mechanisms are not properly configured or maintained. ### Scaling Challenges Scaling Redis can present challenges for developers, especially when it comes to ensuring high availability and data consistency. While Redis supports replication and clustering for scalability, setting up and managing these configurations can be complex and time-consuming. Developers also need to consider the cost implications of scaling Redis. As the volume of data and traffic increases, so too does the infrastructure required to support it. This can result in higher operational costs and potentially limit the scalability of the application. Furthermore, making changes to the data schema in Redis can be tricky when dealing with a large distributed system. Developers need to carefully plan and execute these changes to avoid data inconsistencies and downtime. While Redis offers many benefits for developers looking to improve the speed and efficiency of their applications, it is important to be aware of its limitations and drawbacks. By understanding the challenges of working with complex data structures, managing performance concerns, and addressing scaling challenges, developers can make informed decisions about when and how to use Redis in their development projects. … ### High Memory Usage Another limitation of Redis is its high memory usage. Since Redis stores all data in memory, it can quickly consume a large amount of RAM, especially as the size of your dataset grows. This can be a significant drawback for applications with limited memory resources or those running on cloud platforms where memory costs can add up quickly. To mitigate this issue, developers can implement strategies such as data sharding, data compression, or using Redis in combination with a disk-based database to offload less frequently accessed data. However, these solutions add complexity to the application architecture and may require additional development effort to maintain. ### Lack of Built-In Security Features One potential drawback of using Redis is its lack of built-in security features. By default, Redis has no authentication mechanism enabled, which means that anyone who can access the server can read, modify, or delete data stored in Redis. This can pose a security risk for applications that handle sensitive or confidential information. … The lack of built-in support for transactions in Redis can be a drawback for some projects. You gotta be careful with your data consistency, or you might run into issues. I've had instances where Redis has hit its memory limit and started evicting keys to make space for new data. Not fun when you're relying on that data being there when you need it.