apeye.rate_limiter

Rate limiters for making calls to external APIs in a polite manner.

Attention

This module has the following additional requirements:

cachecontrol[filecache]>=0.12.6
lockfile>=0.12.2

These can be installed as follows:

python -m pip install apeye[limiter]

Classes:

HTTPCache(app_name[, expires_after])

Cache HTTP requests for up to 28 days and limit the rate of requests to no more than 5/second.

RateLimitAdapter([cache, cache_etags, …])

Custom cachecontrol.adapter.CacheControlAdapter to limit the rate of requests to 5 per second.

Functions:

rate_limit([min_time, logger])

Decorator to force a function to run no less than min_time seconds after it last ran.
class HTTPCache(app_name, expires_after=datetime.timedelta(days=28))[source]

Cache HTTP requests for up to 28 days and limit the rate of requests to no more than 5/second.

Parameters

  • app_name (str) – The name of the app. This dictates the name of the cache directory.

  • expires_after (timedelta) – The maximum time to cache responses for. Default datetime.timedelta(days=28).

Attributes:

app_name

The name of the app.

cache_dir

The location of the cache directory on disk.

caches

Mapping of function names to their caches.

Methods:

clear()

Clear the cache.
app_name

Type:    str

The name of the app. This dictates the name of the cache directory.

cache_dir

Type:    PathPlus

The location of the cache directory on disk.

caches

Type:    Dict[str, Dict[str, Any]]

Mapping of function names to their caches.

clear()[source]

Clear the cache.

Return type

bool

Returns

True to indicate success. False otherwise.

rate_limit(min_time=0.2, logger=None)[source]

Decorator to force a function to run no less than min_time seconds after it last ran. Used for rate limiting.

Parameters

  • min_time (float) – The minimum interval between subsequent runs of the decorated function. Default 0.2, which gives a maximum rate of 5 calls per second.

  • logger (Optional[Logger]) – Optional logger to log information about requests to. Defaults to the root logger.

Return type

Callable[[Callable], Any]

class RateLimitAdapter(cache=None, cache_etags=True, controller_class=None, serializer=None, heuristic=None, cacheable_methods=None, *args, **kw)[source]

Bases: CacheControlAdapter

Custom cachecontrol.adapter.CacheControlAdapter to limit the rate of requests to 5 per second.

Parameters

  • cache (BaseCache | None) – Default None.

  • cache_etags (bool) – Default True.

  • controller_class (type[CacheController] | None) – Default None.

  • serializer (Serializer | None) – Default None.

  • heuristic (BaseHeuristic | None) – Default None.

  • cacheable_methods (Collection[str] | None) – Default None.

Methods:

rate_limited_send(*args, **kwargs)

Wrapper around CacheControlAdapter.send to limit the rate of requests.

send(request[, cacheable_methods])

Send a request.
rate_limited_send(*args, **kwargs)[source]

Wrapper around CacheControlAdapter.send to limit the rate of requests.

Return type

Response

send(request, cacheable_methods=None, **kwargs)[source]

Send a request.

Use the request information to see if it exists in the cache and cache the response if we need to and can.

Parameters
Return type

Response