API Reference#
|
Rate limiting with commonly used storage backends |
|
Rate limiting strategies |
|
Implementations of storage backends to be used with |
|
Asynchronous rate limiting strategies |
|
Implementations of storage backends to be used with |
Strategies#
Synchronous Strategies#
The available built in rate limiting strategies which expect
a single parameter: a subclass of Storage.
Provided by limits.strategies
- class FixedWindowRateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
Reference: Fixed Window
- test(item: RateLimitItem, *identifiers: str) bool[source]#
Check if the rate limit can be consumed
- get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats[source]#
Query the reset time and remaining amount for the limit
- class FixedWindowElasticExpiryRateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
Reference: Fixed Window with Elastic Expiry
- get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats#
Query the reset time and remaining amount for the limit
- class MovingWindowRateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
Reference: Moving Window
- test(item: RateLimitItem, *identifiers: str) bool[source]#
Check if the rate limit can be consumed
- get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats[source]#
returns the number of requests remaining within this limit.
All strategies implement the same abstract base class:
- class RateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
- abstract hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#
Consume the rate limit
- abstract test(item: RateLimitItem, *identifiers: str) bool[source]#
Check the rate limit without consuming from it.
- abstract get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats[source]#
Query the reset time and remaining amount for the limit
Async Strategies#
These variants should be used in for asyncio support. These strategies
expose async variants and expect a subclass of limits.aio.storage.Storage
Provided by limits.aio.strategies
- class FixedWindowRateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
Reference: Fixed Window
- async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#
Consume the rate limit
- async test(item: RateLimitItem, *identifiers: str) bool[source]#
Check if the rate limit can be consumed
- async get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats[source]#
Query the reset time and remaining amount for the limit
- class FixedWindowElasticExpiryRateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
Reference: Fixed Window with Elastic Expiry
- async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#
Consume the rate limit
- Parameters:
item¶ – a
limits.limits.RateLimitIteminstanceidentifiers¶ – variable list of strings to uniquely identify the limit
cost¶ – The cost of this hit, default 1
- async get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats#
Query the reset time and remaining amount for the limit
- class MovingWindowRateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
Reference: Moving Window
- async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#
Consume the rate limit
- async test(item: RateLimitItem, *identifiers: str) bool[source]#
Check if the rate limit can be consumed
- async get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats[source]#
returns the number of requests remaining within this limit.
All strategies implement the same abstract base class:
- class RateLimiter(storage: Storage | limits.aio.storage.Storage)[source]#
- abstract async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#
Consume the rate limit
- abstract async test(item: RateLimitItem, *identifiers: str) bool[source]#
Check if the rate limit can be consumed
- abstract async get_window_stats(item: RateLimitItem, *identifiers: str) WindowStats[source]#
Query the reset time and remaining amount for the limit
Storage#
Storage Factory function#
Provided by limits.storage
- storage_from_string(storage_string: str, **options: float | str | bool) Storage | Storage[source]#
Factory function to get an instance of the storage class based on the uri of the storage. In most cases using it should be sufficient instead of directly instantiating the storage classes. for example:
from limits.storage import storage_from_string memory = from_string("memory://") memcached = from_string("memcached://localhost:11211") redis = from_string("redis://localhost:6379")
The same function can be used to construct the Async Storage variants, for example:
from limits.storage import storage_from_string memory = storage_from_string("async+memory://") memcached = storage_from_string("async+memcached://localhost:11211") redis = storage_from_string("async+redis://localhost:6379")
- Parameters:
storage_string¶ – a string of the form
scheme://host:port. More details about supported storage schemes can be found at Storage schemeoptions¶ – all remaining keyword arguments are passed to the constructor matched by
storage_string.
- Raises:
ConfigurationError – when the
storage_stringcannot be mapped to a registeredlimits.storage.Storageorlimits.aio.storage.Storageinstance.
Synchronous Storage#
Provided by limits.storage
In-Memory Storage#
- class MemoryStorage(uri: str | None = None, **_: str)[source]#
rate limit storage using
collections.Counteras an in memory storage for fixed and elastic window strategies, and a simple list to implement moving window strategy.- STORAGE_SCHEME: List[str] | None = ['memory']#
The storage schemes to register against this implementation
- incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
Redis Storage#
- class RedisStorage(uri: str, connection_pool: redis.connection.ConnectionPool | None = None, **options: float | str | bool)[source]#
Rate limit storage with redis as backend.
Depends on redis.
- Parameters:
uri¶ – uri of the form
redis://[:password]@host:port,redis://[:password]@host:port/db,rediss://[:password]@host:port,redis+unix:///path/to/socketc. This uri is passed directly toredis.from_url()except for the case ofredis+unix://where it is replaced withunix://.connection_pool¶ – if provided, the redis client is initialized with the connection pool and any other params passed as
optionsoptions¶ – all remaining keyword arguments are passed directly to the constructor of
redis.Redis
- Raises:
ConfigurationError – when the redis library is not available
- DEPENDENCIES: Dict[str, Version | None] | List[str] = {'redis': <Version('3.0')>}#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
Redis Cluster Storage#
- class RedisClusterStorage(uri: str, **options: float | str | bool)[source]#
Rate limit storage with redis cluster as backend
Depends on redis.
Changed in version 2.5.0: Cluster support was provided by the redis-py-cluster library which has been absorbed into the official redis client. By default the
redis.cluster.RedisClusterclient will be used however if the version of the package is lower than4.2.0the implementation will fallback to trying to userediscluster.RedisCluster.- Parameters:
uri¶ – url of the form
redis+cluster://[:password]@host:port,host:portoptions¶ – all remaining keyword arguments are passed directly to the constructor of
redis.cluster.RedisCluster
- Raises:
ConfigurationError – when the redis library is not available or if the redis cluster cannot be reached.
- DEFAULT_OPTIONS: Dict[str, float | str | bool] = {'max_connections': 1000}#
Default options passed to the
RedisCluster
- DEPENDENCIES: Dict[str, Version | None] | List[str] = {'redis': <Version('4.2.0')>, 'rediscluster': <Version('2.0.0')>}#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#
returns the starting point and the number of entries in the moving window
- incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int#
increments the counter for a given rate limit key
- reset() int | None[source]#
Redis Clusters are sharded and deleting across shards can’t be done atomically. Because of this, this reset loops over all keys that are prefixed with ‘LIMITER’ and calls delete on them, one at a time.
Warning
This operation was not tested with extremely large data sets. On a large production based system, care should be taken with its usage as it could be slow on very large data sets
Redis Sentinel Storage#
- class RedisSentinelStorage(uri: str, service_name: str | None = None, use_replicas: bool = True, sentinel_kwargs: Dict[str, float | str | bool] | None = None, **options: float | str | bool)[source]#
Rate limit storage with redis sentinel as backend
Depends on redis package
- Parameters:
uri¶ – url of the form
redis+sentinel://host:port,host:port/service_nameservice_name¶ – sentinel service name (if not provided in
uri)use_replicas¶ – Whether to use replicas for read only operations
sentinel_kwargs¶ – kwargs to pass as
sentinel_kwargstoredis.sentinel.Sentineloptions¶ – all remaining keyword arguments are passed directly to the constructor of
redis.sentinel.Sentinel
- Raises:
ConfigurationError – when the redis library is not available or if the redis master host cannot be pinged.
- STORAGE_SCHEME: List[str] | None = ['redis+sentinel']#
The storage scheme for redis accessed via a redis sentinel installation
- DEPENDENCIES: Dict[str, Version | None] | List[str] = {'redis.sentinel': <Version('3.0')>}#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#
returns the starting point and the number of entries in the moving window
Memcached Storage#
- class MemcachedStorage(uri: str, **options: str | Callable[[], MemcachedClientP])[source]#
Rate limit storage with memcached as backend.
Depends on pymemcache.
- Parameters:
uri¶ – memcached location of the form
memcached://host:port,host:port,memcached:///var/tmp/path/to/sockoptions¶ – all remaining keyword arguments are passed directly to the constructor of
pymemcache.client.base.PooledClientorpymemcache.client.hash.HashClient(if there are more than one hosts specified)
- Raises:
ConfigurationError – when pymemcache is not available
- get_client(module: module, hosts: List[Tuple[str, int]], **kwargs: str) MemcachedClientP[source]#
returns a memcached client.
- property storage: MemcachedClientP#
lazily creates a memcached client instance using a thread local
- incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
- DEPENDENCIES: Dict[str, Version | None] | List[str] = []#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
MongoDB Storage#
- class MongoDBStorage(uri: str, database_name: str = 'limits', **options: int | str | bool)[source]#
Rate limit storage with MongoDB as backend.
Depends on pymongo.
New in version 2.1.
- Parameters:
uri¶ – uri of the form
mongodb://[user:password]@host:port?..., This uri is passed directly toMongoClientdatabase_name¶ – The database to use for storing the rate limit collections.
options¶ – all remaining keyword arguments are merged with
DEFAULT_OPTIONSand passed to the constructor ofMongoClient
- Raises:
ConfigurationError – when the pymongo library is not available
- STORAGE_SCHEME: List[str] | None = ['mongodb', 'mongodb+srv']#
The storage schemes to register against this implementation
- DEFAULT_OPTIONS: Dict[str, int | str | bool] = {'connectTimeoutMS': 1000, 'serverSelectionTimeoutMS': 1000}#
Default options passed to
MongoClient
- DEPENDENCIES: Dict[str, Version | None] | List[str] = ['pymongo']#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- reset() int | None[source]#
Delete all rate limit keys in the rate limit collections (counters, windows)
- incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
- check() bool[source]#
Check if storage is healthy by calling
pymongo.mongo_client.MongoClient.server_info()
Etcd Storage#
- class EtcdStorage(uri: str, max_retries: int = 5, **options: str)[source]#
Rate limit storage with etcd as backend.
Depends on etcd3.
- Parameters:
uri¶ – etcd location of the form
etcd://host:port,max_retries¶ – Maximum number of attempts to retry in the case of concurrent updates to a rate limit key
options¶ – all remaining keyword arguments are passed directly to the constructor of
etcd3.Etcd3Client
- Raises:
ConfigurationError – when etcd3 is not available
- DEPENDENCIES: Dict[str, Version | None] | List[str] = ['etcd3']#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
Async Storage#
Provided by limits.aio.storage
Async In-Memory Storage#
- class MemoryStorage(uri: str | None = None, **_: str)[source]#
rate limit storage using
collections.Counteras an in memory storage for fixed and elastic window strategies, and a simple list to implement moving window strategy.New in version 2.1.
- STORAGE_SCHEME: List[str] | None = ['async+memory']#
The storage scheme for in process memory storage for use in an async context
- async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
- async get_num_acquired(key: str, expiry: int) int[source]#
returns the number of entries already acquired
- async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#
returns the starting point and the number of entries in the moving window
Async Redis Storage#
- class RedisStorage(uri: str, connection_pool: coredis.ConnectionPool | None = None, **options: float | str | bool)[source]#
Rate limit storage with redis as backend.
Depends on coredis
New in version 2.1.
- Parameters:
uri¶ –
uri of the form:
async+redis://[:password]@host:portasync+redis://[:password]@host:port/dbasync+rediss://[:password]@host:portasync+unix:///path/to/socketc…
This uri is passed directly to
coredis.Redis.from_url()with the initialasyncremoved, except for the case ofasync+redis+unixwhere it is replaced withunix.connection_pool¶ – if provided, the redis client is initialized with the connection pool and any other params passed as
optionsoptions¶ – all remaining keyword arguments are passed directly to the constructor of
coredis.Redis
- Raises:
ConfigurationError – when the redis library is not available
- STORAGE_SCHEME: List[str] | None = ['async+redis', 'async+rediss', 'async+redis+unix']#
The storage schemes for redis to be used in an async context
- DEPENDENCIES: Dict[str, Version | None] | List[str] = {'coredis': <Version('3.4.0')>}#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
- async check() bool[source]#
Check if storage is healthy by calling
coredis.Redis.ping()
Async Redis Cluster Storage#
- class RedisClusterStorage(uri: str, **options: float | str | bool)[source]#
Rate limit storage with redis cluster as backend
Depends on coredis
New in version 2.1.
- Parameters:
uri¶ – url of the form
async+redis+cluster://[:password]@host:port,host:portoptions¶ – all remaining keyword arguments are passed directly to the constructor of
coredis.RedisCluster
- Raises:
ConfigurationError – when the coredis library is not available or if the redis host cannot be pinged.
- STORAGE_SCHEME: List[str] | None = ['async+redis+cluster']#
The storage schemes for redis cluster to be used in an async context
- DEFAULT_OPTIONS: Dict[str, float | str | bool] = {'max_connections': 1000}#
Default options passed to
coredis.RedisCluster
- async reset() int | None[source]#
Redis Clusters are sharded and deleting across shards can’t be done atomically. Because of this, this reset loops over all keys that are prefixed with ‘LIMITER’ and calls delete on them, one at a time.
Warning
This operation was not tested with extremely large data sets. On a large production based system, care should be taken with its usage as it could be slow on very large data sets
- DEPENDENCIES: Dict[str, Version | None] | List[str] = {'coredis': <Version('3.4.0')>}#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- async check() bool#
Check if storage is healthy by calling
coredis.Redis.ping()
Async Redis Sentinel Storage#
- class RedisSentinelStorage(uri: str, service_name: str | None = None, use_replicas: bool = True, sentinel_kwargs: Dict[str, float | str | bool] | None = None, **options: float | str | bool)[source]#
Rate limit storage with redis sentinel as backend
Depends on coredis
New in version 2.1.
- Parameters:
uri¶ – url of the form
async+redis+sentinel://host:port,host:port/service_nameoptional¶ (sentinel_kwargs,) – sentinel service name (if not provided in uri)
use_replicas¶ – Whether to use replicas for read only operations
optional¶ – kwargs to pass as
sentinel_kwargstocoredis.sentinel.Sentineloptions¶ – all remaining keyword arguments are passed directly to the constructor of
coredis.sentinel.Sentinel
- Raises:
ConfigurationError – when the coredis library is not available or if the redis primary host cannot be pinged.
- STORAGE_SCHEME: List[str] | None = ['async+redis+sentinel']#
The storage scheme for redis accessed via a redis sentinel installation
- DEPENDENCIES: Dict[str, Version | None] | List[str] = {'coredis.sentinel': <Version('3.4.0')>}#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#
returns the starting point and the number of entries in the moving window
- async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int#
increments the counter for a given rate limit key
- async reset() int | None#
This function calls a Lua Script to delete keys prefixed with ‘LIMITER’ in block of 5000.
Warning
This operation was designed to be fast, but was not tested on a large production based system. Be careful with its usage as it could be slow on very large data sets.
- async check() bool[source]#
Check if storage is healthy by calling
coredis.Redis.ping()on the replica.
Async Memcached Storage#
- class MemcachedStorage(uri: str, **options: float | str | bool)[source]#
Rate limit storage with memcached as backend.
Depends on emcache
New in version 2.1.
- Parameters:
- Raises:
ConfigurationError – when emcache is not available
- STORAGE_SCHEME: List[str] | None = ['async+memcached']#
The storage scheme for memcached to be used in an async context
- DEPENDENCIES: Dict[str, Version | None] | List[str] = ['emcache']#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
Async MongoDB Storage#
- class MongoDBStorage(uri: str, database_name: str = 'limits', **options: float | str | bool)[source]#
Rate limit storage with MongoDB as backend.
Depends on motor
New in version 2.1.
- Parameters:
uri¶ – uri of the form
async+mongodb://[user:password]@host:port?..., This uri is passed directly toAsyncIOMotorClientdatabase_name¶ – The database to use for storing the rate limit collections.
options¶ – all remaining keyword arguments are merged with
DEFAULT_OPTIONSand passed to the constructor ofAsyncIOMotorClient
- Raises:
ConfigurationError – when the motor or pymongo are not available
- STORAGE_SCHEME: List[str] | None = ['async+mongodb', 'async+mongodb+srv']#
The storage scheme for MongoDB for use in an async context
- DEFAULT_OPTIONS: Dict[str, float | str | bool] = {'connectTimeoutMS': 1000, 'serverSelectionTimeoutMS': 1000}#
Default options passed to
AsyncIOMotorClient
- DEPENDENCIES: Dict[str, Version | None] | List[str] = ['motor.motor_asyncio', 'pymongo']#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
- async reset() int | None[source]#
Delete all rate limit keys in the rate limit collections (counters, windows)
- async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
- async check() bool[source]#
Check if storage is healthy by calling
motor.motor_asyncio.AsyncIOMotorClient.server_info()
Async Etcd Storage#
- class EtcdStorage(uri: str, max_retries: int = 5, **options: str)[source]#
Rate limit storage with etcd as backend.
Depends on aetcd.
- Parameters:
uri¶ – etcd location of the form
async+etcd://host:port,max_retries¶ – Maximum number of attempts to retry in the case of concurrent updates to a rate limit key
options¶ – all remaining keyword arguments are passed directly to the constructor of
aetcd.client.Client
- Raises:
ConfigurationError – when aetcd is not available
- DEPENDENCIES: Dict[str, Version | None] | List[str] = ['aetcd']#
The python modules this class has a dependency on. Used to lazily populate the
dependencies
Abstract storage classes#
- class Storage(uri: str | None = None, **options: float | str | bool)[source]#
Base class to extend when implementing a storage backend.
- abstract incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
- class MovingWindowSupport[source]#
Abstract base for storages that intend to support the moving window strategy
Async variants#
- class Storage(uri: str | None = None, **options: float | str | bool)[source]#
Base class to extend when implementing an async storage backend.
New in version 2.1.
- abstract async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#
increments the counter for a given rate limit key
Rate Limits#
Provided by limits
Parsing functions#
- parse(limit_string: str) RateLimitItem[source]#
parses a single rate limit in string notation (e.g.
1/secondor1 per second)- Parameters:
limit_string¶ – rate limit string using Rate limit string notation
- Raises:
ValueError – if the string notation is invalid.
- parse_many(limit_string: str) List[RateLimitItem][source]#
parses rate limits in string notation containing multiple rate limits (e.g.
1/second; 5/minute)- Parameters:
limit_string¶ – rate limit string using Rate limit string notation
- Raises:
ValueError – if the string notation is invalid.
Rate limit granularities#
All rate limit items implement RateLimitItem by
declaring a GRANULARITY
- class RateLimitItem(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
defines a Rate limited resource which contains the characteristic namespace, amount and granularity multiples of the rate limiting window.
- Parameters:
amount¶ – the rate limit amount
multiples¶ – multiple of the ‘per’
GRANULARITY(e.g. ‘n’ per ‘m’ seconds)namespace¶ – category for the specific rate limit
- GRANULARITY: ClassVar[Granularity]#
A tuple describing the granularity of this limit as (number of seconds, name)
- classmethod check_granularity_string(granularity_string: str) bool[source]#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY
- class RateLimitItemPerSecond(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
per second rate limited resource.
- classmethod check_granularity_string(granularity_string: str) bool#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY
- class RateLimitItemPerMinute(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
per minute rate limited resource.
- classmethod check_granularity_string(granularity_string: str) bool#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY
- class RateLimitItemPerHour(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
per hour rate limited resource.
- classmethod check_granularity_string(granularity_string: str) bool#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY
- class RateLimitItemPerDay(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
per day rate limited resource.
- classmethod check_granularity_string(granularity_string: str) bool#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY
- class RateLimitItemPerMonth(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
per month rate limited resource.
- classmethod check_granularity_string(granularity_string: str) bool#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY
- class RateLimitItemPerYear(amount: int, multiples: int | None = 1, namespace: str = 'LIMITER')[source]#
per year rate limited resource.
- classmethod check_granularity_string(granularity_string: str) bool#
Checks if this instance matches a granularity_string of type
n per hour,n per minuteetc, by comparing withGRANULARITY