API Reference#

limits

Rate limiting with commonly used storage backends

limits.strategies

Rate limiting strategies

limits.storage

Implementations of storage backends to be used with limits.strategies.RateLimiter strategies

limits.aio.strategies

Asynchronous rate limiting strategies

limits.aio.storage

Implementations of storage backends to be used with limits.aio.strategies.RateLimiter strategies

Strategies#

The available built in rate limiting strategies which expect a single parameter: a subclass of Storage.

Provided by limits.strategies

class FixedWindowRateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#

Reference: Fixed Window

hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

  • cost – The cost of this hit, default 1

test(item: RateLimitItem, *identifiers: str) bool[source]#

Check if the rate limit can be consumed

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int][source]#

Query the reset time and remaining amount for the limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

Returns

(reset time, remaining)

class FixedWindowElasticExpiryRateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#

Reference: Fixed Window with Elastic Expiry

hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

  • cost – The cost of this hit, default 1

get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int]#

Query the reset time and remaining amount for the limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

Returns

(reset time, remaining)

test(item: RateLimitItem, *identifiers: str) bool#

Check if the rate limit can be consumed

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

class MovingWindowRateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#

Reference: Moving Window

hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

  • cost – The cost of this hit, default 1

Returns

(reset time, remaining)

test(item: RateLimitItem, *identifiers: str) bool[source]#

Check if the rate limit can be consumed

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int][source]#

returns the number of requests remaining within this limit.

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

Returns

tuple (reset time, remaining)

All strategies implement the same abstract base class:

class RateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#
abstract hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

  • cost – The cost of this hit, default 1

abstract test(item: RateLimitItem, *identifiers: str) bool[source]#

Check the rate limit without consuming from it.

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

abstract get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int][source]#

Query the reset time and remaining amount for the limit

Parameters
  • item – The rate limit item

  • identifiers – variable list of strings to uniquely identify this instance of the limit

Returns

(reset time, remaining)

These variants should be used in for asyncio support. These strategies expose async variants and expect a subclass of limits.aio.storage.Storage

Provided by limits.aio.strategies

class FixedWindowRateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#

Reference: Fixed Window

async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

  • cost – The cost of this hit, default 1

async test(item: RateLimitItem, *identifiers: str) bool[source]#

Check if the rate limit can be consumed

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

async get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int][source]#

Query the reset time and remaining amount for the limit

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

Returns

reset time, remaining

class FixedWindowElasticExpiryRateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#

Reference: Fixed Window with Elastic Expiry

async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – a limits.limits.RateLimitItem instance

  • identifiers – variable list of strings to uniquely identify the limit

  • cost – The cost of this hit, default 1

async get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int]#

Query the reset time and remaining amount for the limit

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

Returns

reset time, remaining

async test(item: RateLimitItem, *identifiers: str) bool#

Check if the rate limit can be consumed

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

class MovingWindowRateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#

Reference: Moving Window

async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

  • cost – The cost of this hit, default 1

async test(item: RateLimitItem, *identifiers: str) bool[source]#

Check if the rate limit can be consumed

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

async get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int][source]#

returns the number of requests remaining within this limit.

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

Returns

(reset time, remaining)

All strategies implement the same abstract base class:

class RateLimiter(storage: Union[Storage, limits.aio.storage.Storage])[source]#
abstract async hit(item: RateLimitItem, *identifiers: str, cost: int = 1) bool[source]#

Consume the rate limit

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

  • cost – The cost of this hit, default 1

abstract async test(item: RateLimitItem, *identifiers: str) bool[source]#

Check if the rate limit can be consumed

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

abstract async get_window_stats(item: RateLimitItem, *identifiers: str) Tuple[int, int][source]#

Query the reset time and remaining amount for the limit

Parameters
  • item – the rate limit item

  • identifiers – variable list of strings to uniquely identify the limit

Returns

(reset time, remaining))

Storage#

Storage Factory function#

Provided by limits.storage

storage_from_string(storage_string: str, **options: Union[float, str, bool]) Union[Storage, Storage][source]#

Factory function to get an instance of the storage class based on the uri of the storage. In most cases using it should be sufficient instead of directly instantiating the storage classes. for example:

from limits.storage import storage_from_string

memory = from_string("memory://")
memcached = from_string("memcached://localhost:11211")
redis = from_string("redis://localhost:6379")

The same function can be used to construct the Async Storage variants, for example:

from limits.storage import storage_from_string

memory = storage_from_string("async+memory://")
memcached = storage_from_string("async+memcached://localhost:11211")
redis = storage_from_string("asycn+redis://localhost:6379")
Parameters
  • storage_string – a string of the form scheme://host:port. More details about supported storage schemes can be found at Storage scheme

  • options – all remaining keyword arguments are passed to the constructor matched by storage_string.

Raises

ConfigurationError – when the storage_string cannot be mapped to a registered limits.storage.Storage or limits.aio.storage.Storage instance.

Synchronous Storage#

Provided by limits.storage

In-Memory#

class MemoryStorage(uri: Optional[str] = None, **_: str)[source]#

rate limit storage using collections.Counter as an in memory storage for fixed and elastic window strategies, and a simple list to implement moving window strategy.

incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

get(key: str) int[source]#
Parameters

key – the key to get the counter value for

clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

get_num_acquired(key: str, expiry: int) int[source]#

returns the number of entries already acquired

Parameters
  • key – rate limit key to acquire an entry in

  • expiry – expiry of the entry

get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

check() bool[source]#

check if storage is healthy

reset() Optional[int][source]#

reset storage to clear limits

Redis#

class RedisStorage(uri: str, connection_pool: Optional[redis.connection.ConnectionPool] = None, **options: Union[float, str, bool])[source]#

Rate limit storage with redis as backend.

Depends on redis.

Parameters
  • uri – uri of the form redis://[:password]@host:port, redis://[:password]@host:port/db, rediss://[:password]@host:port, redis+unix:///path/to/sock etc. This uri is passed directly to redis.from_url() except for the case of redis+unix:// where it is replaced with unix://.

  • connection_pool – if provided, the redis client is initialized with the connection pool and any other params passed as options

  • options – all remaining keyword arguments are passed directly to the constructor of redis.Redis

Raises

ConfigurationError – when the redis library is not available

STORAGE_SCHEME: Optional[List[str]] = ['redis', 'rediss', 'redis+unix']#

The storage scheme for redis

incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

get(key: str) int[source]#
Parameters

key – the key to get the counter value for

clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number to increment by

get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

check() bool[source]#

check if storage is healthy

get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

reset() Optional[int][source]#

This function calls a Lua Script to delete keys prefixed with ‘LIMITER’ in block of 5000.

Warning

This operation was designed to be fast, but was not tested on a large production based system. Be careful with its usage as it could be slow on very large data sets.

Redis Cluster#

class RedisClusterStorage(uri: str, **options: Union[float, str, bool])[source]#

Rate limit storage with redis cluster as backend

Depends on redis.

Changed in version 2.5.0: Cluster support was provided by the redis-py-cluster library which has been absorbed into the official redis client. By default the redis.cluster.RedisCluster client will be used however if the version of the package is lower than 4.2.0 the implementation will fallback to trying to use rediscluster.RedisCluster.

Parameters
  • uri – url of the form redis+cluster://[:password]@host:port,host:port

  • options – all remaining keyword arguments are passed directly to the constructor of redis.cluster.RedisCluster

Raises

ConfigurationError – when the redis library is not available or if the redis cluster cannot be reached.

STORAGE_SCHEME: Optional[List[str]] = ['redis+cluster']#

The storage scheme for redis cluster

DEFAULT_OPTIONS: Dict[str, Union[float, str, bool]] = {'max_connections': 1000}#

Default options passed to the RedisCluster

acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number to increment by

check() bool#

check if storage is healthy

clear(key: str) None#
Parameters

key – the key to clear rate limits for

get(key: str) int#
Parameters

key – the key to get the counter value for

get_expiry(key: str) int#
Parameters

key – the key to get the expiry for

get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

reset() Optional[int][source]#

Redis Clusters are sharded and deleting across shards can’t be done atomically. Because of this, this reset loops over all keys that are prefixed with ‘LIMITER’ and calls delete on them, one at a time.

Warning

This operation was not tested with extremely large data sets. On a large production based system, care should be taken with its usage as it could be slow on very large data sets

Redis Sentinel#

class RedisSentinelStorage(uri: str, service_name: Optional[str] = None, use_replicas: bool = True, sentinel_kwargs: Optional[Dict[str, Union[float, str, bool]]] = None, **options: Union[float, str, bool])[source]#

Rate limit storage with redis sentinel as backend

Depends on redis package

Parameters
  • uri – url of the form redis+sentinel://host:port,host:port/service_name

  • service_name – sentinel service name (if not provided in uri)

  • use_replicas – Whether to use replicas for read only operations

  • sentinel_kwargs – kwargs to pass as sentinel_kwargs to redis.sentinel.Sentinel

  • options – all remaining keyword arguments are passed directly to the constructor of redis.sentinel.Sentinel

Raises

ConfigurationError – when the redis library is not available or if the redis master host cannot be pinged.

STORAGE_SCHEME: Optional[List[str]] = ['redis+sentinel']#

The storage scheme for redis accessed via a redis sentinel installation

DEFAULT_OPTIONS: Dict[str, Union[float, str, bool]] = {'socket_timeout': 0.2}#

Default options passed to Sentinel

get(key: str) int[source]#
Parameters

key – the key to get the counter value for

acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number to increment by

clear(key: str) None#
Parameters

key – the key to clear rate limits for

get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

reset() Optional[int]#

This function calls a Lua Script to delete keys prefixed with ‘LIMITER’ in block of 5000.

Warning

This operation was designed to be fast, but was not tested on a large production based system. Be careful with its usage as it could be slow on very large data sets.

check() bool[source]#

Check if storage is healthy by calling aredis.StrictRedis.ping on the slave.

Memcached#

class MemcachedStorage(uri: str, **options: Union[str, Callable[[], MemcachedClientP]])[source]#

Rate limit storage with memcached as backend.

Depends on pymemcache.

Parameters
  • uri – memcached location of the form memcached://host:port,host:port, memcached:///var/tmp/path/to/sock

  • options – all remaining keyword arguments are passed directly to the constructor of pymemcache.client.base.PooledClient or pymemcache.client.hash.HashClient (if there are more than one hosts specified)

Raises

ConfigurationError – when pymemcache is not available

STORAGE_SCHEME: Optional[List[str]] = ['memcached']#

The storage scheme for memcached

get_client(module: module, hosts: List[Tuple[str, int]], **kwargs: str) MemcachedClientP[source]#

returns a memcached client.

Parameters
  • module – the memcached module

  • hosts – list of memcached hosts

property storage: limits.typing.MemcachedClientP#

lazily creates a memcached client instance using a thread local

get(key: str) int[source]#
Parameters

key – the key to get the counter value for

clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

check() bool[source]#

Check if storage is healthy by calling the get command on the key limiter-check

reset() Optional[int][source]#

reset storage to clear limits

MongoDB#

class MongoDBStorage(uri: str, database_name: str = 'limits', **options: Union[int, str, bool])[source]#

Rate limit storage with MongoDB as backend.

Depends on pymongo.

New in version 2.1.

Parameters
  • uri – uri of the form mongodb://[user:password]@host:port?..., This uri is passed directly to MongoClient

  • database_name – The database to use for storing the rate limit collections.

  • options – all remaining keyword arguments are merged with DEFAULT_OPTIONS and passed to the constructor of MongoClient

Raises

ConfigurationError – when the pymongo library is not available

DEFAULT_OPTIONS: Dict[str, Union[int, str, bool]] = {'connectTimeoutMS': 1000, 'serverSelectionTimeoutMS': 1000, 'socketTimeoutMS': 1000}#

Default options passed to MongoClient

reset() Optional[int][source]#

Delete all rate limit keys in the rate limit collections (counters, windows)

clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

get(key: str) int[source]#
Parameters

key – the key to get the counter value for

incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

check() bool[source]#

Check if storage is healthy by calling pymongo.mongo_client.MongoClient.server_info()

get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

Async Storage#

Provided by limits.aio.storage

In-Memory#

class MemoryStorage(uri: Optional[str] = None, **_: str)[source]#

rate limit storage using collections.Counter as an in memory storage for fixed and elastic window strategies, and a simple list to implement moving window strategy.

New in version 2.1.

STORAGE_SCHEME: Optional[List[str]] = ['async+memory']#

The storage scheme for in process memory storage for use in an async context

async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

async get(key: str) int[source]#
Parameters

key – the key to get the counter value for

async clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

async acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

async get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

async get_num_acquired(key: str, expiry: int) int[source]#

returns the number of entries already acquired

Parameters
  • key – rate limit key to acquire an entry in

  • expiry – expiry of the entry

async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

async check() bool[source]#

check if storage is healthy

async reset() Optional[int][source]#

reset storage to clear limits

Redis#

class RedisStorage(uri: str, connection_pool: Optional[coredis.ConnectionPool] = None, **options: Union[float, str, bool])[source]#

Rate limit storage with redis as backend.

Depends on coredis

New in version 2.1.

Parameters
  • uri

    uri of the form:

    • async+redis://[:password]@host:port

    • async+redis://[:password]@host:port/db

    • async+rediss://[:password]@host:port

    • async+unix:///path/to/sock etc…

    This uri is passed directly to coredis.Redis.from_url() with the initial async removed, except for the case of async+redis+unix where it is replaced with unix.

  • connection_pool – if provided, the redis client is initialized with the connection pool and any other params passed as options

  • options – all remaining keyword arguments are passed directly to the constructor of coredis.Redis

Raises

ConfigurationError – when the redis library is not available

STORAGE_SCHEME: Optional[List[str]] = ['async+redis', 'async+rediss', 'async+redis+unix']#

The storage schemes for redis to be used in an async context

async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

async get(key: str) int[source]#
Parameters

key – the key to get the counter value for

async clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

async acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

async get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

async check() bool[source]#

Check if storage is healthy by calling coredis.Redis.ping()

async reset() Optional[int][source]#

This function calls a Lua Script to delete keys prefixed with ‘LIMITER’ in block of 5000.

Warning

This operation was designed to be fast, but was not tested on a large production based system. Be careful with its usage as it could be slow on very large data sets.

async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

Redis Cluster#

class RedisClusterStorage(uri: str, **options: Union[float, str, bool])[source]#

Rate limit storage with redis cluster as backend

Depends on coredis

New in version 2.1.

Parameters
  • uri – url of the form async+redis+cluster://[:password]@host:port,host:port

  • options – all remaining keyword arguments are passed directly to the constructor of coredis.RedisCluster

Raises

ConfigurationError – when the coredis library is not available or if the redis host cannot be pinged.

STORAGE_SCHEME: Optional[List[str]] = ['async+redis+cluster']#

The storage schemes for redis cluster to be used in an async context

DEFAULT_OPTIONS: Dict[str, Union[float, str, bool]] = {'max_connections': 1000}#

Default options passed to coredis.RedisCluster

async reset() Optional[int][source]#

Redis Clusters are sharded and deleting across shards can’t be done atomically. Because of this, this reset loops over all keys that are prefixed with ‘LIMITER’ and calls delete on them, one at a time.

Warning

This operation was not tested with extremely large data sets. On a large production based system, care should be taken with its usage as it could be slow on very large data sets

async acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

async check() bool#

Check if storage is healthy by calling coredis.Redis.ping()

async clear(key: str) None#
Parameters

key – the key to clear rate limits for

async get(key: str) int#
Parameters

key – the key to get the counter value for

async get_expiry(key: str) int#
Parameters

key – the key to get the expiry for

async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

Redis Sentinel#

class RedisSentinelStorage(uri: str, service_name: Optional[str] = None, use_replicas: bool = True, sentinel_kwargs: Optional[Dict[str, Union[float, str, bool]]] = None, **options: Union[float, str, bool])[source]#

Rate limit storage with redis sentinel as backend

Depends on coredis

New in version 2.1.

Parameters
  • uri – url of the form async+redis+sentinel://host:port,host:port/service_name

  • optional (sentinel_kwargs,) – sentinel service name (if not provided in uri)

  • use_replicas – Whether to use replicas for read only operations

  • optional – kwargs to pass as sentinel_kwargs to coredis.sentinel.Sentinel

  • options – all remaining keyword arguments are passed directly to the constructor of coredis.sentinel.Sentinel

Raises

ConfigurationError – when the coredis library is not available or if the redis primary host cannot be pinged.

STORAGE_SCHEME: Optional[List[str]] = ['async+redis+sentinel']#

The storage scheme for redis accessed via a redis sentinel installation

DEFAULT_OPTIONS: Dict[str, Union[float, str, bool]] = {'stream_timeout': 0.2}#

Default options passed to Sentinel

async acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

async clear(key: str) None#
Parameters

key – the key to clear rate limits for

async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • amount – the number to increment by

async reset() Optional[int]#

This function calls a Lua Script to delete keys prefixed with ‘LIMITER’ in block of 5000.

Warning

This operation was designed to be fast, but was not tested on a large production based system. Be careful with its usage as it could be slow on very large data sets.

async get(key: str) int[source]#
Parameters

key – the key to get the counter value for

async get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

async check() bool[source]#

Check if storage is healthy by calling coredis.StrictRedis.ping() on the replica.

Memcached#

class MemcachedStorage(uri: str, **options: Union[float, str, bool])[source]#

Rate limit storage with memcached as backend.

Depends on emcache

New in version 2.1.

Parameters
  • uri – memcached location of the form async+memcached://host:port,host:port

  • options – all remaining keyword arguments are passed directly to the constructor of emcache.Client

Raises

ConfigurationError – when emcache is not available

STORAGE_SCHEME: Optional[List[str]] = ['async+memcached']#

The storage scheme for memcached to be used in an async context

async get(key: str) int[source]#
Parameters

key – the key to get the counter value for

async clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

async get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

async check() bool[source]#

Check if storage is healthy by calling the get command on the key limiter-check

async reset() Optional[int][source]#

reset storage to clear limits

MongoDB#

class MongoDBStorage(uri: str, database_name: str = 'limits', **options: Union[float, str, bool])[source]#

Rate limit storage with MongoDB as backend.

Depends on motor

New in version 2.1.

Parameters
  • uri – uri of the form async+mongodb://[user:password]@host:port?..., This uri is passed directly to AsyncIOMotorClient

  • database_name – The database to use for storing the rate limit collections.

  • options – all remaining keyword arguments are merged with DEFAULT_OPTIONS and passed to the constructor of AsyncIOMotorClient

Raises

ConfigurationError – when the motor or pymongo are not available

STORAGE_SCHEME: Optional[List[str]] = ['async+mongodb', 'async+mongodb+srv']#

The storage scheme for MongoDB for use in an async context

DEFAULT_OPTIONS: Dict[str, Union[float, str, bool]] = {'connectTimeoutMS': 1000, 'serverSelectionTimeoutMS': 1000, 'socketTimeoutMS': 1000}#

Default options passed to AsyncIOMotorClient

async reset() Optional[int][source]#

Delete all rate limit keys in the rate limit collections (counters, windows)

async clear(key: str) None[source]#
Parameters

key – the key to clear rate limits for

async get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

async get(key: str) int[source]#
Parameters

key – the key to get the counter value for

async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

async check() bool[source]#

Check if storage is healthy by calling motor.motor_asyncio.AsyncIOMotorClient.server_info()

async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#

returns the starting point and the number of entries in the moving window

Parameters
  • key (str) – rate limit key

  • expiry (int) – expiry of entry

Returns

(start of window, number of acquired entries)

async acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

Abstract storage classes#

class Storage(uri: Optional[str] = None, **options: Union[float, str, bool])[source]#

Base class to extend when implementing a storage backend.

STORAGE_SCHEME: Optional[List[str]]#

The storage schemes to register against this implementation

abstract incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

abstract get(key: str) int[source]#
Parameters

key – the key to get the counter value for

abstract get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

abstract check() bool[source]#

check if storage is healthy

abstract reset() Optional[int][source]#

reset storage to clear limits

abstract clear(key: str) None[source]#

resets the rate limit key

Parameters

key – the key to clear rate limits for

class MovingWindowSupport[source]#

Abstract base for storages that intend to support the moving window strategy

acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

Async variants#

class Storage(uri: Optional[str] = None, **options: Union[float, str, bool])[source]#

Base class to extend when implementing an async storage backend.

New in version 2.1.

STORAGE_SCHEME: Optional[List[str]]#

The storage schemes to register against this implementation

abstract async incr(key: str, expiry: int, elastic_expiry: bool = False, amount: int = 1) int[source]#

increments the counter for a given rate limit key

Parameters
  • key – the key to increment

  • expiry – amount in seconds for the key to expire in

  • elastic_expiry – whether to keep extending the rate limit window every hit.

  • amount – the number to increment by

abstract async get(key: str) int[source]#
Parameters

key – the key to get the counter value for

abstract async get_expiry(key: str) int[source]#
Parameters

key – the key to get the expiry for

abstract async check() bool[source]#

check if storage is healthy

abstract async reset() Optional[int][source]#

reset storage to clear limits

abstract async clear(key: str) None[source]#

resets the rate limit key

Parameters

key – the key to clear rate limits for

class MovingWindowSupport[source]#

Abstract base for storages that intend to support the moving window strategy

async acquire_entry(key: str, limit: int, expiry: int, amount: int = 1) bool[source]#
Parameters
  • key – rate limit key to acquire an entry in

  • limit – amount of entries allowed

  • expiry – expiry of the entry

  • amount – the number of entries to acquire

async get_moving_window(key: str, limit: int, expiry: int) Tuple[int, int][source]#

returns the starting point and the number of entries in the moving window

Parameters
  • key – rate limit key

  • expiry – expiry of entry

Returns

(start of window, number of acquired entries)

Rate Limits#

Provided by limits

Parsing functions#

parse(limit_string: str) RateLimitItem[source]#

parses a single rate limit in string notation (e.g. 1/second or 1 per second)

Parameters

limit_string – rate limit string using Rate limit string notation

Raises

ValueError – if the string notation is invalid.

parse_many(limit_string: str) List[RateLimitItem][source]#

parses rate limits in string notation containing multiple rate limits (e.g. 1/second; 5/minute)

Parameters

limit_string – rate limit string using Rate limit string notation

Raises

ValueError – if the string notation is invalid.

Rate limit granularities#

All rate limit items implement RateLimitItem by declaring a GRANULARITY

class RateLimitItem(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

defines a Rate limited resource which contains the characteristic namespace, amount and granularity multiples of the rate limiting window.

Parameters
  • amount – the rate limit amount

  • multiples – multiple of the ‘per’ GRANULARITY (e.g. ‘n’ per ‘m’ seconds)

  • namespace – category for the specific rate limit

GRANULARITY: ClassVar[limits.limits.Granularity]#

A tuple describing the granularity of this limit as (number of seconds, name)

classmethod check_granularity_string(granularity_string: str) bool[source]#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int[source]#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str[source]#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.


class RateLimitItemPerSecond(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

per second rate limited resource.

classmethod check_granularity_string(granularity_string: str) bool#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.

GRANULARITY: ClassVar[limits.limits.Granularity] = Granularity(seconds=1, name='second')#

A second

class RateLimitItemPerMinute(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

per minute rate limited resource.

classmethod check_granularity_string(granularity_string: str) bool#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.

GRANULARITY: ClassVar[limits.limits.Granularity] = Granularity(seconds=60, name='minute')#

A minute

class RateLimitItemPerHour(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

per hour rate limited resource.

GRANULARITY: ClassVar[limits.limits.Granularity] = Granularity(seconds=3600, name='hour')#

An hour

classmethod check_granularity_string(granularity_string: str) bool#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.

class RateLimitItemPerDay(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

per day rate limited resource.

GRANULARITY: ClassVar[limits.limits.Granularity] = Granularity(seconds=86400, name='day')#

A day

classmethod check_granularity_string(granularity_string: str) bool#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.

class RateLimitItemPerMonth(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

per month rate limited resource.

GRANULARITY: ClassVar[limits.limits.Granularity] = Granularity(seconds=2592000, name='month')#

A month

classmethod check_granularity_string(granularity_string: str) bool#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.

class RateLimitItemPerYear(amount: int, multiples: Optional[int] = 1, namespace: str = 'LIMITER')[source]#

per year rate limited resource.

GRANULARITY: ClassVar[limits.limits.Granularity] = Granularity(seconds=31104000, name='year')#

A year

classmethod check_granularity_string(granularity_string: str) bool#

Checks if this instance matches a granularity_string of type n per hour, n per minute etc, by comparing with GRANULARITY

get_expiry() int#
Returns

the duration the limit is enforced for in seconds.

key_for(*identifiers: str) str#

Constructs a key for the current limit and any additional identifiers provided.

Parameters

identifiers – a list of strings to append to the key

Returns

a string key identifying this resource with each identifier appended with a ‘/’ delimiter.

Exceptions#

exception ConfigurationError[source]#

exception raised when a configuration problem is encountered

with_traceback()#

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.