Rate Limiting as a Service

Part of the worlds-biggest-software-project initiative.

An open, AI-native rate-limiting platform with token bucket and sliding window algorithms, customer quotas, and compute-aware metering for modern API and LLM traffic.

Rate Limiting as a Service is a managed platform for enforcing configurable limits on API traffic. It is designed for engineering teams who need accurate distributed rate limiting, multi-tenant quota plans, and AI-aware metering without rebuilding the infrastructure themselves. The platform exposes a synchronous evaluation API that gateways call before processing requests, backed by a low-latency distributed counter store.

Why Rate Limiting as a Service?

Incumbent gateways are heavy: Tools like Zuplo and Tyk bundle full API management — overkill for teams that need only limit enforcement.
DIY Redis logic is brittle: Custom INCR/EXPIRE or sorted-set implementations rarely handle distributed counter accuracy, multi-dimensional quotas, or graceful degradation correctly.
Edge libraries are not enterprise-grade: Upstash Rate Limit is purpose-built for serverless runtimes but has no developer portal, API key management, or quota plan management.
Self-hosted operational burden: Apache APISIX requires running etcd alongside Redis and the gateway; no managed cloud offering exists for the OSS path.
Traditional limits miss AI traffic: Request-count quotas are insufficient when a single LLM call may consume vastly different amounts of compute depending on prompt length or model selection.

Key Features

Algorithms and Enforcement

Fixed-window, sliding-window, and token-bucket algorithms selectable per route or API key
Leaky-bucket and concurrent-connection limits for traffic shaping
Per-key, per-IP, per-endpoint, per-consumer, and per-service dimensions
Standard HTTP 429 responses with Retry-After and X-RateLimit-* headers
Configurable fail-open vs. fail-closed behaviour when the counter store is unavailable

Distributed Counter Infrastructure

Redis-backed distributed counters for cluster-wide consistent enforcement
Sub-10 ms p99 check latency targets
Hot-reload of policy changes without gateway restart
Multi-region counter configuration to minimise check latency for globally distributed workloads

Quota and Plan Management

Hierarchical quota model: user-within-organisation aggregated limits
Plan tier management to assign quotas to customer accounts via the control plane
Per-period reset (hourly, daily, monthly) for quota policies
Real-time usage dashboard per customer and per plan tier
Bypass rules for internal services and trusted partners

AI-Native Metering

Token-based limits counting LLM tokens consumed per request, not just request counts
Per-request token consumption customisation (consume N tokens per request rather than always 1)
Cost-based limits capping monthly spend rather than request counts
Integration hooks for LLM provider metering APIs

Operations and Developer Experience

REST API control plane for creating and updating policies without redeployment
Prometheus metrics endpoint for alerting on 429 rates and counter check latency
Webhook notifications when customers approach or exceed thresholds
Self-serve customer usage portal showing consumption against quota

AI-Native Advantage

Traditional rate limiters count HTTP requests, but a single AI agent call can consume thousands of LLM tokens or significant compute. This platform meters tokens and cost directly, supports cost-based monthly caps, and enables anomaly detection that flags customers whose usage pattern deviates sharply from their historical baseline before they breach limits. Predictive quota exhaustion alerts and natural-language policy authoring ("allow 1,000 requests per minute for Pro customers, 100 for Free") further close the gap between operator intent and enforced configuration.

Tech Stack & Deployment

Counter store: Redis (cluster or sentinel) for distributed counter state
Control plane: REST API for policy CRUD; configuration changes hot-reloaded without restart
Deployment modes: self-hosted gateway with Kubernetes-native integration, plus optional edge enforcement via Cloudflare Worker or Vercel Edge middleware
Observability: Prometheus metrics, Grafana dashboards, and webhook event streams
SDKs: Library integrations for Next.js middleware, Hono, Remix, Astro, and other TypeScript/JavaScript runtimes

Market Context

Gartner projected in 2025 that more than 30% of the growth in API demand by 2026 would originate from AI and LLM tooling, validating a shift toward compute-aware quota enforcement. Existing options span proprietary SaaS gateways (Zuplo, Cloudflare Rate Limiting), open-source gateways with broader scope (Tyk MPL-2.0, Apache APISIX Apache 2.0), and serverless libraries (Upstash Rate Limit, MIT). Primary buyers are platform and infrastructure teams at API-first companies, B2B SaaS vendors enforcing plan tiers, and AI product teams metering agent traffic.

Project Status

This project is in the research and specification phase.
Contributions, feedback, and domain expertise are welcome.

Contributing

We welcome contributions from developers, domain experts, and potential users. See CONTRIBUTING.md for guidelines.

Important: All contributions must be your own original work or clearly attributed open-source material with a compatible licence. Copyright infringement and licence violations will not be tolerated and will result in immediate removal of the offending contribution. If you are unsure whether a piece of code, text, or other material is safe to contribute, open an issue and ask before submitting.

Licence

Licence to be determined. See discussion for context.