Boosting Performance and Reliability: Mastering Redis Cache and Eviction Strategies

Concept of Cache

Cache is an intermediate layer between an application and physical data, designed to reduce the frequency of accessing physical data sources and thereby enhance the performance of the application.

Data inside the cache is a copy of the data present in the physical data source. The application reads and writes data from the cache during runtime, and at specific intervals, the cache data is synchronized with the data in the physical data source.

For example, in a typical scenario where we directly query a MySQL database, high concurrency can lead to slower database performance. To alleviate this issue, we create a Cache layer between the application and MySQL. This allows requests to first access the cache, significantly reducing the pressure on the database and improving performance.

Distributed Cache

In simple terms, a cache system that can span across multiple processes is called a distributed cache.

In the development of distributed systems, interactions between different systems occur at the process level. When a cache system can function across processes, it is termed a distributed cache. Two popular distributed cache technologies in the market are Memcached and Redis, and their differences are as follows:

Performance:

• Redis is single-threaded.
• Memcached is multi-threaded.

Memory Space and Data Size:

• Memcached: The maximum memory size can be modified, and it uses the LRU algorithm for eviction.
• Redis: Besides modifying the maximum memory, Redis caches data in memory, and some data can exceed the physical space limit using virtual memory features (similar to using an external disk as a data storage source).

Operations:

• Memcached: Supports a single data type, “String,” used for caching.
• Redis: Supports a variety of data types (String, Hash, Set, Sorted Set, etc.). It can be used for various operations, such as implementing features like check-ins, nearby people (using bitmap and geo features), reducing server-side operations, and minimizing IO read and write.

Reliability:

• Memcached: Does not support persistence, so data is lost in case of power failure. It can only be used for caching and needs to be refilled from the relational database.
• Redis: Supports persistence through mechanisms like RDB (Redis Database) and AOF (Append-Only File). Data can be loaded from the persisted disk and stored in the cache. It can handle single-point failures and implement mechanisms like master-slave, clustering, and sentinels.

Use Cases:

• Memcached: Suitable for reducing database pressure in read-heavy or read-only caching scenarios.
• Redis: Can also reduce database pressure and be used for caching in read-heavy and write-heavy scenarios. Additionally, it can be used for implementing complex business features like scoring, check-ins, and nearby people.

Considerations:

Memcached:

• Stores a single key-value pair, and the maximum value size is limited to 1MB.
• Suitable for scenarios where high reliability is not a critical requirement.

Redis:

• Stores key-value pairs, with a maximum value size of 512MB (4.2 billion bytes).
• Offers high reliability and can be considered an in-memory database.
• Supports multiple data types, making it suitable for various special business scenarios (e.g., bitmap for storing check-ins, geo for nearby people, set for friend lists).

Storage Method

When using Redis as a cache, there are two storage methods: storing data as Strings or using Hashes.

Which method is better depends on the specific scenario:

• String Storage: Simple and suitable for fixed data with a small size, such as storing basic user information (username, nickname, avatar, age, etc.). Data needs to be serialized during storage and deserialized during retrieval, but the overhead is negligible for small data sizes.
• Hashes Storage: More suitable when some properties of the data may change. For example, in a restaurant data set, there may be attributes like likeVotes and dislikeVotes, which change frequently. Using Hashes avoids the serialization overhead during storage.

Cache Exception Handling:

Cache-related exceptions can lead to critical issues like database crashes when a large number of requests flood in. The three classic cache problems are cache breakdown, cache penetration, and cache avalanche.

Cache Breakdown:

1. Cause: A hot cache data with an expired time is accessed by a large number of requests simultaneously, leading to cache misses. As a result, all these requests access the database directly, overwhelming it.
2. Solutions:
   • Never expire cache data.
   • Implement logical expiration.
   • Perform service degradation.

Cache Penetration:

1. Cause: When a user requests data that is neither in the cache nor in the database, it leads to cache misses in both cases, and the subsequent requests keep failing to build cache data.
2. Solutions:
   • Use Bloom Filters.
   • Store empty values in the cache.
   • Employ mutex locks (not recommended due to potential performance overhead).

Cache Avalanche:

1. Cause: A large number of cache data expires or Redis fails, causing all requests to bypass the cache and directly access the database, leading to a surge in database pressure and potential crashes.
2. Solutions:
   • Randomize expiration times.
   • Use dual caches as backups (rarely used).
   • Employ mutex locks (not recommended).
   • Perform service degradation.

Overall, the most critical issue is the failure of Redis nodes. Configuring master-slave, clustering, and sentinels can help handle node failures and maintain cache availability.

Solution:

To address the issues related to cache and Redis node failures, one of the effective approaches is to configure Redis with a master-slave setup, clustering, and sentinel mechanism.

1. Master-Slave Configuration: In a master-slave configuration, Redis uses replication to create multiple copies (slaves) of the master node’s data. The master node is responsible for handling write operations, while the slave nodes replicate the data from the master and handle read operations. This setup provides redundancy and load balancing, as read requests can be distributed among multiple slave nodes.
2. Redis Clustering: Redis clustering allows you to distribute your data across multiple Redis nodes in a way that ensures scalability and high availability. Each node in the cluster is responsible for a subset of the data. By using Redis clustering, you can increase the overall capacity and throughput of your cache system. Additionally, it provides failover and automatic resharding for handling node failures and rebalancing the data distribution.
3. Sentinel Mechanism: The Redis Sentinel is a monitoring and notification system designed to ensure high availability for Redis. It continuously monitors the health of Redis instances in a master-slave setup and automatically promotes a slave to a new master if the current master fails. The Sentinel system provides automatic failover and helps maintain data consistency in the presence of node failures.

Cache Eviction:

Redis cache parameters, such as maximum memory, expiration time, and eviction mechanisms, can be configured in the redis.conf file.

maxmemory <bytes> //set max memory

When the memory exceeds the maximum limit, the cache system will adopt a memory eviction strategy and perform evictions as follows.

Memory Eviction Strategy

Up to now, Redis has provided us with a total of eight different memory eviction strategies, with six of them being available since the early days.

# volatile-lru -> Evict using approximated LRU, only keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU, only keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key having an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.
#
# LRU means Least Recently Used
# LFU means Least Frequently Used
#
# Both LRU, LFU and volatile-ttl are implemented using approximated
# randomized algorithms.
#
# Note: with any of the above policies, Redis will return an error on write
#       operations, when there are no suitable keys for eviction.
#
#       At the date of writing these commands are: set setnx setex append
#       incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
#       sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
#       zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
#       getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy noeviction

Default is no eviction, and commonly used in development is volatile-lru.

1. volatile-lru: Evicts the least recently used data from the dataset with expiration time set.
2. volatile-ttl: Evicts data from the dataset with expiration time set, selecting data that is about to expire.
3. volatile-random: Randomly evicts data from the dataset with expiration time set.
4. volatile-lfu: Uses the LFU algorithm to evict key-value pairs from the dataset with expiration time set.
5. allkeys-lru: Evicts the least recently used data from all datasets.
6. allkeys-random: Randomly evicts data from all datasets.
7. no-eviction: Disables eviction and returns an error on write requests if Redis reaches its memory limit.

Note that there are six strategies mentioned here. “volatile” and “allkeys” define whether the eviction applies to datasets with expiration time or all datasets. The subsequent “lru,” “ttl,” and “random” represent three different eviction policies. Additionally, there is one strategy, “no-eviction,” which permanently retains data and does not perform evictions.

Usage rules:

• If the data exhibits a power-law distribution, where some data has high access frequency while others have low access frequency, use “allkeys-lru.”
• If the data exhibits a uniform distribution, where all data has equal access frequency, use “allkeys-random.”

Implementation of Eviction Mechanism

Since eviction involves removing data, the cache system needs to delete certain data and make room for new entries. Redis implements the following deletion strategies:

Lazy Deletion:

• When the memory is not full, Redis adopts lazy deletion. It doesn’t spawn an additional thread to handle deletions, ensuring high performance. Upon requesting a key, Redis checks if the key has expired and returns the result accordingly. If a key is found to be expired, it is deleted at that moment.

Timed Deletion:

• At regular intervals, Redis performs timed deletion to remove expired keys. This scheduled process periodically checks for keys with expiration times and deletes them if they have expired.

Active Deletion:

• In situations where memory is full, Redis actively removes surplus keys. When the cache reaches its memory limit, Redis prioritizes evicting keys based on specific eviction policies, such as LRU (Least Recently Used), LFU (Least Frequently Used), or random eviction. This process ensures that new data can be stored in the cache when it reaches its capacity.

These deletion strategies ensure that expired or unnecessary data is removed from the cache, allowing Redis to accommodate new data and maintain efficient memory management.