Let’s Write a Composable, Easy-to-Use Caching Package in Python

Easy, user-friendly caching that tailors to all your needs

Python choosing a caching strategy (image by ChatGPT + amateuristic edits by author)

In this article we’ll walk through the process of building a caching system in python from the ground up. The goal is to…

• better understand caching and explore various caching strategies

• understand why we went for a composition-approach over inheritance

• learn how and when to apply a cache effectively

We’ll focus on building a user-friendly package that allows you to easily add caches to your code. Additionally, we’ll provide a simple means to extend the cache by providing it with self-designed behavior. Let’s code!

---

Before we begin..

This article details creating caches that are part of the the Cachr package on PyPI. All code can be found in this Github repo. Contributions are always welcome; feel free to submit bug reports, bug fixes, feature requests, documentation improvements or enhancements!

---

Basics: How Caches Work and Terminology

Caches are high-speed data storage layers designed to temporarily hold data, making it quicker to retrieve. For example, imagine storing product details from a web store in a cache after fetching them from the database. When a user requests the same product information again, you can just quickly retrieve it from the cache; significantly reducing respond time and easing the load on your database.

Cache Capacity

It isn’t advisable to allow your cache to grow without limits as it can eventually consume all available memory. For this reason we provide a capacity to our cache. A cache with a capacity of 5 will only allow 5 items in the cache. When a new item needs to be added, we first need to evict an existing item to make room.

Choosing What to evict

When your cache is full and you want to add a new item, we have to determine which item to evict. The simplest approach is to evict the “oldest” item. Other strategies include evicting the Least Recently Used (LRU) item, the Least Frequently Used (LFU), or even removing an item at random.

This decision-making process is know as a caching strategy or replacement policy. In this article we’ll develop a package that comes pre-packaged with a few strategies, while also giving you the flexibility to create and apply your own.

Expiring Cache Items

Can an item live in your cache forever? In some cases it might be useful to put a new item in your cache with an “expiration date”, after which it will be automatically removed. This is particularly useful for caching time-sensitive data, such as authentication tokens.

For example, if an authentication token is valid for 9 hours, we can cache the results of our “get-token-function” with an expiration time of 9 hours. This way, after the initial request, the token can be quickly retrieved from the cache for the next 9 hours, rather than making repeated HTTP request to obtain the token. This approach speeds up your system and reduces unnecessary network calls.

---

Writing the Cache

Our goal is to develop a package that contains several pre-built caches that are easy to use and efficient. We also want our package to empower users to create custom caches tailored to their specific needs. This process needs to be straightforward yet flexible enough to accommodate various caching requirements.

General Approach

We’ll start by writing a basic caching class that can be initialized with a caching strategy. This caching strategy determines the cache’s behavior such as how items are stored and evicted. Furthermore, we make it easy for users to create and apply their own caching strategy, allowing them to fully customize the cache’s functionality to suit their particular use cases.

In technical terms: our cache will be built with the strategy design pattern that uses composition to encapsulate the cache’s desired behavior in specific classes.