Subject: [httpapi] RateLimit Header client implementations

From: Darrel Miller <[email-masked]>
Date: Wed, 30 Dec 2020 03:16:51 +0000

Hey Roberto,

Do you have links to implementations of client applications that use some variant of the rate-limiting headers?
I know there are plenty of examples of server side implementations that will return these headers, but I am finding very few examples of clients taking advantage of these headers.

>From what I understand there are a few scenarios where clients can use these headers:

  1.  Optimizing polling frequency:  i.e. check quota and window and determine optimum calling interval
  2.  Prioritization between client tasks:  Don’t let low priority tasks starve high priority tasks of quota
  3.  Enable user agents to avoid hitting rate limit due to some kind of punitive behavior.

Are you aware of other scenarios?  Are there implementations of these scenarios?

While reviewing the goals for the draft I was wondering if they could be improved.

  *   The goals of this proposal are:
  *
  *   1. Standardizing the names and semantic of rate-limit headers;
  *   2. Improve resiliency of HTTP infrastructures simplifying the enforcement and the adoption of rate-limit headers;
  *   3. Simplify API documentation avoiding expliciting rate-limit fields semantic in documentation.

The second goal seems only indirectly related to what the draft actually enables.  A phrase in the introduction paragraph seems like a more accurate goal:

  *   communicate service quotas so that the client can throttle its requests and prevent 4xx or 5xx responses.

I also thing the third goal would be clearer stated as:

  *   Simplify API documentation by eliminating the need to define custom quota related header fields.

Darrel

---

From: Roberto Polli <[email-masked]>
Date: Wed, 30 Dec 2020 16:28:23 +0100

Hi Darrel,

Thanks for your support and interest in the spec!
replies inline and on gh-issues.

> Do you have links to implementations of client applications that use some variant of the rate-limiting headers?
Since there is not a standard way of providing those headers, client
libraries are hard to find.
I saw that some cloud providers bake their custom models in their client, eg:
- https://cloud.google.com/solutions/rate-limiting-strategies-techniques
- https://octokit.github.io/octokit.rb/Octokit/Client/RateLimit.html

> I know there are plenty of examples of server side implementations that will return these headers,
> but I am finding very few examples of clients taking advantage of these headers.
In my experience, clients implement the desired behavior per-service
or per-api-gateway.

> [..] scenarios [..]:
> 1) Optimizing polling frequency:  i.e. check quota and window and determine optimum calling interval
Yes, this is quite useful when you have a continuous flow of  (queued)
requests that can be tuned.

> 2) Prioritization between client tasks:  Don’t let low priority tasks starve high priority tasks of quota.
Yes, see https://github.com/ioggstream/draft-polli-ratelimit-headers/issues/69#issuecomment-548323736
replace user with whatever quota key you want.

```
# pseudocode
for user, request_data in queue:
  if user.has_quota():  # checks headers, ratelimit-reset, retry-after, whatever
    response, headers = make_request(conn, request_data, user)
    user.update_quota(headers)
```

> 3) Enable user agents to avoid hitting rate limit due to some kind of punitive behavior.
This was probably the main goal for this spec: specifically when
orchestrating some kind of
"live" transactions (eg. payment, ... ) hitting a ratelimit can be
problematic. In those cases
the "consumer" service has a way to know whether to accept a request
that may be blocked
at some point.

> Are you aware of other scenarios?
Your scenarios seem to cover all the main ones; @Alex Martinez Ruiz is welcome
to join the discussion.

>  Are there implementations of these scenarios?
I saw only custom implementations. My experience is mainly related to
scenario 3)
 availability respect to performance.

A ratelimit landscape is here:
- https://github.com/ietf-wg-httpapi/ratelimit-headers/issues/25
but we can file another issue for client implementations.

## Improving goals

> 1. Standardizing the names and semantic of rate-limit headers;
> 2. Improve resiliency of HTTP infrastructures simplifying the enforcement and the adoption of rate-limit headers;
> 3. Simplify API documentation avoiding expliciting rate-limit fields semantic in documentation.
Let me see...

> The second goal seems only indirectly related to what the draft actually enables.
> A phrase in the introduction paragraph seems like a more accurate goal:
> communicate service quotas so that the client can throttle its requests and prevent 4xx or 5xx responses.
I stated the overall goal of the specification, but I'm open to a more
technical perspective.

> Simplify API documentation by eliminating the need to define custom quota related header fields.

While this is true, what I want to get rid of is the limit
communicated via pdf or web pages, eg.
https://developer.service.hmrc.gov.uk/api-documentation/docs/reference-guide
Let me know if this makes sense to you.

I filed a specific issue
https://github.com/ietf-wg-httpapi/ratelimit-headers/issues/26
so we can track down easily this thread.

Have a nice day,
R.

-- 
Roberto Polli
API Expert
M. +39 3406522736
MID
DIPARTIMENTO PER LA
TRASFORMAZIONE
DIGITALE
Presidenza del Consiglio dei Ministri - Ministro per l'innovazione
tecnologica e la digitalizzazione
https://innovazione.gov.it/

Il Dipartimento per la Trasformazione Digitale, salvo eccezioni,
comunica con le altre Amministrazioni via posta elettronica ordinaria
e non posta elettronica certificata, in conformità a quanto previsto
dall’art.47 del Codice dell’Amministrazione Digitale.