Storage Backends#
Supported versions#
limits is tested and known to work with the following versions of the dependency libraries and the associated storage versions.
The CI tests against these versions on a nightly basis and you can see the results in github.
Dependency versions:
redis>3,!=4.5.2,!=4.5.3,<6.0.0
Dependency versions (async):
coredis>=3.4.0,<5
7.0 6.2.6 6.0.16
Redis with SSL
7.0 6.2.6 6.0.16
6.2.6 6.0.16 7.0
Dependency versions:
redis>=4.2.0,!=4.5.2,!=4.5.3
Dependency versions (async):
coredis>=3.4.0,<5
7.0 6.2.6 6.0.16
Dependency versions:
pymemcache>3,<5.0.0
Dependency versions (async):
emcache>=0.6.1;python_version<"3.11" # not yet supported emcache>=1;python_version>="3.11" # not yet supported
1.6.15 1.6.6 1.5.16 1.4.34
Dependency versions:
pymongo>4.1,<5
Dependency versions (async):
motor>=3,<4
6.0.3 5.0.3 4.4.9 4.2.17
Storage scheme#
limits uses a url style storage scheme notation (similar to the JDBC driver connection string notation) for configuring and initializing storage backends. This notation additionally provides a simple mechanism to both identify and configure the backend implementation based on a single string argument.
The storage scheme follows the format {scheme}://{parameters}
limits.storage.storage_from_string() is provided to
lookup and construct an instance of a storage based on the storage scheme. For example:
import limits.storage
uri = "redis://localhost:9999"
options = {}
redis_storage = limits.storage.storage_from_string(uri, **options)
The additional options key-word arguments are passed as is to the constructor of the storage and handled differently by each implementation. Please refer to the API documentation in the Storage section for details.
Examples#
In-Memory Storage#
The in-memory storage (MemoryStorage) takes no parameters so the only relevant value is memory://
Memcached Storage#
Requires the location of the memcached server(s). As such
the parameters is a comma separated list of {host}:{port} locations such as
memcached://localhost:11211 or
memcached://localhost:11211,localhost:11212,192.168.1.1:11211 etc…
or a path to a unix domain socket such as memcached:///var/tmp/path/to/sock
Depends on: pymemcache
Redis Storage#
Requires the location of the redis server and optionally the database number.
redis://localhost:6379 or redis://localhost:6379/n (for database n).
If the redis server is listening over a unix domain socket you can use redis+unix:///path/to/sock
or redis+unix:///path/to/socket?db=n (for database n).
If the database is password protected the password can be provided in the url, for example
redis://:foobared@localhost:6379 or redis+unix://:foobered/path/to/socket if using a UDS..
For scenarios where a redis connection pool is already available and can be reused, it can be provided
in options, for example:
pool = redis.connections.BlockingConnectionPool.from_url("redis://.....")
storage_from_string("redis://", connection_pool=pool)
Depends on: redis
Redis+SSL Storage#
The official Redis client redis supports redis connections over SSL with the scheme
You can add ssl related parameters in the url itself, for example:
rediss://localhost:6379/0?ssl_ca_certs=./tls/ca.crt&ssl_keyfile=./tls/client.key.
Depends on: redis
Redis+Sentinel Storage#
Requires the location(s) of the redis sentinal instances and the service-name
that is monitored by the sentinels.
redis+sentinel://localhost:26379/my-redis-service
or redis+sentinel://localhost:26379,localhost:26380/my-redis-service.
If the sentinel is password protected the username and/or password can be provided in the url,
for example redis+sentinel://:sekret@localhost:26379/my-redis-service
When authentication details are provided in the url they will be used for both the sentinel and as connection arguments for the underlying redis nodes managed by the sentinel.
If you need fine grained control it is recommended to use the additional options
arguments. More details can be found in the API documentation for RedisSentinelStorage (or the aysnc version: RedisSentinelStorage).
Depends on: redis
Redis Cluster Storage#
Requires the location(s) of the redis cluster startup nodes (One is enough).
redis+cluster://localhost:7000
or redis+cluster://localhost:7000,localhost:7001
If the cluster is password protected the username and/or password can be provided in the url,
for example redis+cluster://:sekret@localhost:7000,localhost:7001
Depends on: redis
MongoDB Storage#
Requires the location(s) of a mongodb installation using the uri schema described by the Mongodb URI Specification
Examples:
Local instance:
mongodb://localhost:27017/Instance with SSL:
mongodb://mymongo.com/?tls=trueLocal instance with SSL & self signed/invalid certificate:
mongodb://localhost:27017/?tls=true&tlsAllowInvalidCertificates=true
Depends on: pymongo
Etcd Storage#
Requires the location of an etcd node
Example: etcd://localhost:2379
Depends on: etcd3
Async Storage#
New in version 2.1.
When using limits in an async code base the same uri schema can be used
to query for an async implementation of the storage by prefixing the
scheme with async+.
For example:
async+redis://localhost:6379/0async+rediss://localhost:6379/0async+redis+cluster://localhost:7000,localhost:7001async+redis+sentinel://:sekret@localhost:26379/my-redis-serviceasync+memcached://localhost:11211async+etcd://localhost:2379async+memory://
For implementation details of currently supported async backends refer to Async Storage