Posted in Security Operations

🚀 Rate Limiting Microservice | Update #1: Building a Distributed, Policy-Driven Rate Limiting System Over…

Naman Mahna — Software Engineer

🚀 Rate Limiting Microservice | Update #1: Building a Distributed, Policy-Driven Rate Limiting System

Over three weeks, I’ve built an independent service to centralise rate-limit enforcement across applications and API gateways, with configurable, identity-aware decisions and concurrency-safe state management.

WHAT I HAVE ENGINEERED

1. Four rate-limiting algorithms

• Fixed Window Counter tracks request counts and window start times.

• Sliding Window Log stores timestamped entries in a Redis Sorted Set; expired entries are removed by score before evaluating the active window.

• Token Bucket maintains token balance and update time, with time-based replenishment.

• Leaky Bucket maintains water level and update time, with time-based leakage and capacity enforcement.

2. Atomic Redis state management

Redis Hashes hold Fixed Window, Token Bucket and Leaky Bucket state; a Sorted Set stores Sliding Window Log timestamps. Lua scripts run via EVAL perform reads, time calculations, quota checks and mutations atomically preventing concurrent requests from interleaving decision and update. Redis server time and PEXPIRE provide reliable timing and state expiry.

3. Database-backed policy engine

FastAPI, asynchronous SQLAlchemy and PostgreSQL underpin client-specific policies resolved by resource, HTTP method, route, identity type and quota scope. Exact routes take precedence over wildcard fallbacks; policies select the algorithm, its configuration and per-identity or shared quota scope.

4. Service authentication and authorisation

Credentials use an ID and a cryptographically generated secret (secrets). HMAC-SHA256 with a server-side key derives the verifier, checked using constant-time comparison. Credential controls include key version, activation, not-before, expiry and revocation. The authenticated service identity is then checked against its assigned permissions.

5. End-to-end decision pipeline

The API validates, authenticates and authorises each call, resolves a policy, builds a namespaced Redis key and dispatches its Lua script. The result is normalised into an allow/deny response with remaining quota and retry timing.

TECHNOLOGY STACK

Python 3.14, FastAPI, Pydantic, async SQLAlchemy, PostgreSQL/NeonDB, Redis/Lua, Alembic, Docker, Kubernetes/K3s, Helm, Kubernetes Secrets, Pytest, Ruff, MyPy, Locust, Prometheus and Grafana.

WHAT’S NEXT

Concurrency, load and security testing, including penetration testing, then benchmarking.

I am integrating the service with Kong API Gateway in my separate AI DevOps project. Kong requests a centralized rate-limit decision before forwarding upstream. Integration is ongoing.

Next: architecture diagrams, low-level design, integration and test results

Repository: https://github.com/namankali/rate-limiting-service

#Python #FastAPI #Redis #PostgreSQL #BackendEngineering #DistributedSystems #Microservices #Kubernetes #CloudNative #PlatformEngineering #APIManagement #PythonDeveloper #BackendDeveloper #CloudEngineering #SystemDesign

🚀 Rate Limiting Microservice | Update #1: Building a Distributed, Policy-Driven Rate Limiting System Over…

4 reactions · 0 comments

Topics: #platformengineering, #python, #redis, #distributedsystems, #backenddeveloper, #kubernetes, #microservices, #pythondeveloper, #backendengineering, #postgresql

Read and join the discussion on Cyberr