Rate Limiting, Throttling & Circuit Breakers

Why Rate Limiting is Critical for Backend APIs

Every backend system has finite CPU, memory, and database connection limits. Without rate limiting, a runaway client loop or a DDoS attack can exhaust thread pools and take down an entire platform.

Rate limiting provides three vital protections:

1. Prevents Cascading Outages: Dampens sudden traffic spikes before they hit database query queues.
2. Fair Resource Allocation: Prevents a single high-volume tenant from monopolizing shared multi-tenant resources.
3. Monetization Enforcement: Enforces usage quotas across free and enterprise API billing tiers.

---

Comparing Rate Limiting Algorithms

Algorithm	How It Works	Strengths	Trade-Offs
Token Bucket	Tokens refill at constant rate up to a max capacity; each request consumes 1 token.	Handles bursts smoothly; memory efficient.	Requires synchronized atomic operations (Redis/D1).
Leaky Bucket	Requests enter a queue and leak out at a constant processing rate.	Guarantees smooth outbound flow.	Drops burst traffic immediately when the queue is full.
Fixed Window	Counts requests in a fixed time block (e.g. 100 req / minute).	Simplest to implement.	Vulnerable to double-limit bursts at window edges.
Sliding Window Log	Tracks timestamps of individual requests.	Mathematically precise.	High memory footprint at scale.

---

Standard IETF Rate Limiting Headers

When a client makes a request, your API should communicate their current budget using standardized response headers:

HTTP/1.1 200 OK
RateLimit-Limit: 100
RateLimit-Remaining: 94
RateLimit-Reset: 38

When the client exceeds their quota, respond with 429 Too Many Requests and specify the cooldown duration using the Retry-After header:

HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 30

{
  "type": "https://buildrestapi.com/errors/rate-limit-exceeded",
  "title": "Too Many Requests",
  "status": 429,
  "detail": "You have exceeded your limit of 100 requests per minute. Please retry in 30 seconds."
}

---

Circuit Breakers: Preventing Cascading Failures

When a downstream database or microservice experiences degraded performance (e.g. queries timing out after 10 seconds), sending continuous incoming requests will trigger a complete system collapse.

A Circuit Breaker monitors failure rates and transitions across three states:

1. Closed (Normal): All traffic passes through to the downstream service.
2. Open (Tripped): When error rates cross a threshold (e.g. 50% failures over 10 seconds), the circuit trips. Incoming requests immediately fail-fast with 503 Service Unavailable, shielding the database.
3. Half-Open (Testing): After a cooldown period, a small percentage of requests are permitted through to probe whether the downstream dependency has recovered.