From a45cf30fe449067ad9ae717c5eae6ac9037e1ff1 Mon Sep 17 00:00:00 2001 From: Ramkumar Ethiraj <64294490+rmkr-dev@users.noreply.github.com> Date: Sun, 1 Nov 2026 05:04:56 +0000 Subject: [PATCH 1/2] Add API rate limiting and quota guardrails skill --- skillbook/skills.json | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/skillbook/skills.json b/skillbook/skills.json index 5ab3c83..e6829a5 100644 --- a/skillbook/skills.json +++ b/skillbook/skills.json @@ -898,6 +898,27 @@ ], "formatNotes": "All three. Copilot: path-scoped `.github/instructions/datasource.instructions.md` with applyTo for `**/application*.yml`, `**/application*.properties`, `**/*Repository.java`, and `**/k8s/**`, or `.github/skills/hikari-connection-pool-guardrails/SKILL.md`; one always-on line in `.github/copilot-instructions.md` (\"pool size is a budget: pods at HPA max times maximum-pool-size must fit the database\"). Claude Code: `.claude/skills/hikari-connection-pool-guardrails/SKILL.md`, plus a CLAUDE.md line naming the database connection limit and HPA max. Codex: a \"Database connections\" section in root `AGENTS.md` (or the same SKILL.md under `.agents/skills/`). Pair with spring-data-jpa-query-guardrails for query shape, aks-hpa-requests-limits-guardrails for replica counts, java-virtual-threads-spring-guardrails for concurrency, and spring-boot-micrometer-prometheus-guardrails for metrics. This skill owns how many connections exist and how long anyone may hold one.", "body": "# Skill: HikariCP Connection Pool Guardrails for Spring Boot\n\n## Goal\nEvery service holds a small, fixed number of database connections that fits inside the database's limit even when the deployment is fully scaled out. Requests fail fast with a clear error instead of queueing for 30 seconds, no connection is held across a remote call, dead connections are retired before the network or the database kills them, and pool pressure is visible on a dashboard before users feel it.\n\n## When to apply\n- Adding a datasource to a Spring Boot service, or changing replica counts, HPA limits, or database tier\n- Seeing `SQLTransientConnectionException: ... Connection is not available, request timed out after 30000ms`\n- Seeing `FATAL: sorry, too many clients already` or `remaining connection slots are reserved` on PostgreSQL\n- Latency spikes that line up with `hikaricp_connections_pending` above zero\n- Moving to virtual threads, PgBouncer, or a managed database proxy\n\n## Sizing is a budget, not a guess\nStart from the database, not from the app:\n\n```text\nusable = database max_connections - reserved (admin, replication, migrations, other apps)\nothers = sum over other clients of (pool size * max pods)\nper_pod_max = floor((usable - others) / (HPA maxReplicas + rollout maxSurge))\n```\n\nRules of thumb:\n- A pool larger than the database can do useful work with makes things slower, not faster. HikariCP's own sizing guidance starts near `(cores * 2) + effective disks` **for the database server**, shared by all clients.\n- Most request-response services do well with 5 to 20 connections per pod. Bigger numbers usually hide slow queries or long transactions.\n- Count every pool: each datasource, each replica, batch jobs, migration jobs, and rolling-update surge pods (old and new pods overlap during a deploy).\n\n## Baseline rules\n1. **Fixed-size pool.** Set `maximum-pool-size` from the budget and leave `minimum-idle` unset (it then equals the maximum). A fixed pool avoids connection storms when traffic returns. Name it with `pool-name` so logs and metrics say which pool.\n2. **Fail fast.** `connection-timeout` well under the caller's timeout, typically 1 to 5 seconds, never the 30-second default behind a 10-second ingress timeout. Return 503 with a retry hint on `SQLTransientConnectionException`, not 500.\n3. **Retire connections before something else does.** `max-lifetime` a few minutes shorter than the shortest of: database idle or session limits, proxy or firewall idle timeouts, and credential rotation (for example Entra token lifetime when using passwordless auth). Set `keepalive-time` below any idle timeout on the path (Azure load balancers drop idle TCP flows after a few minutes by default). Keep `idle-timeout` irrelevant by keeping the pool fixed.\n4. **Short transactions.** Set `spring.jpa.open-in-view=false`, so connections are not held for view rendering. No HTTP calls, message publishing that waits for a broker, sleeps, or user think time inside `@Transactional`. Read-only paths use `@Transactional(readOnly = true)`. Batch jobs commit in chunks.\n5. **Time limits on the database side.** A per-role `statement_timeout` and `idle_in_transaction_session_timeout` on PostgreSQL (or the equivalent on your engine), plus a JDBC `socketTimeout` slightly above the statement timeout. A runaway query should end on the server, not only in the client.\n6. **Leak detection where it is cheap.** `leak-detection-threshold` (for example 20 to 60 seconds) in dev and test, and in production only while investigating. A leak warning includes the stack trace of the borrower; fix the code, do not raise the threshold.\n7. **Validation without extra round trips.** Use a JDBC4 driver and do not set `connection-test-query`. Leave `auto-commit` at its default unless the whole codebase manages transactions explicitly, and then set it once and document it.\n8. **Proxies and PgBouncer.** In transaction pooling mode, session state does not survive between transactions: no session-level `SET`, advisory locks, `LISTEN`, or temp tables across transactions. With pgjdbc, disable server-side prepared statements (`prepareThreshold=0`) unless the proxy explicitly supports protocol-level prepared statements. Size the app pool against the proxy's client limit, and the proxy's server pool against the database.\n9. **Virtual threads do not add connections.** With `spring.threads.virtual.enabled=true`, thousands of requests can wait on the same small pool. Treat the pool as the concurrency limit, keep `connection-timeout` short, and add a bulkhead or rate limit in front of database-heavy endpoints rather than growing the pool. Check that the JDBC driver version in use does not pin carrier threads during I/O.\n10. **Separate pools for separate jobs.** Long reports, batch exports, or migrations get their own small pool or their own deployment, so they cannot starve request traffic. Flyway or Liquibase runs as a single job or init step, not in every replica at once.\n11. **Measure it.** Micrometer exposes `hikaricp_connections_active`, `_idle`, `_pending`, `_max`, `hikaricp_connections_acquire_seconds`, `hikaricp_connections_usage_seconds`, and `hikaricp_connections_timeout_total`. Alert on timeouts above zero and on pending above zero for several minutes; graph usage time p95 to find long holders.\n12. **Clean shutdown.** `server.shutdown=graceful` with a `preStop` delay, so in-flight requests finish and the pool closes after them; termination grace period longer than the drain plus the longest statement timeout.\n\n## Copy-ready baseline (application.yml)\n```yaml\nspring:\n jpa:\n open-in-view: false\n datasource:\n url: jdbc:postgresql://${DB_HOST}:5432/${DB_NAME}?socketTimeout=35&ApplicationName=${APP_NAME}\n hikari:\n pool-name: ${APP_NAME}-primary\n maximum-pool-size: ${DB_POOL_MAX:10} # from the connection budget\n connection-timeout: 3000 # ms; well under the request timeout\n max-lifetime: 1500000 # ms; below DB, proxy, and credential limits\n keepalive-time: 120000 # ms; below any idle timeout on the path\n validation-timeout: 1000\n # leak-detection-threshold: 30000 # dev/test, or prod while investigating\n```\n\nDatabase side (PostgreSQL, per application role):\n```sql\nALTER ROLE app_svc SET statement_timeout = '30s';\nALTER ROLE app_svc SET idle_in_transaction_session_timeout = '60s';\nALTER ROLE app_svc CONNECTION LIMIT 120; -- the budget for this service\n```\n\n## Review checklist\n- [ ] Connection budget written down: database limit, reserved slots, HPA max per service, surge pods\n- [ ] `maximum-pool-size` set from the budget; `minimum-idle` unset or equal\n- [ ] `connection-timeout` below the caller's timeout; 503 on pool exhaustion\n- [ ] `max-lifetime` and `keepalive-time` below every idle and session limit on the path\n- [ ] `open-in-view` off; no remote calls inside transactions\n- [ ] Server-side statement and idle-in-transaction timeouts per role\n- [ ] PgBouncer or proxy settings match the pooling mode\n- [ ] Pool metrics on the dashboard, alerts on timeouts and sustained pending\n- [ ] Batch, report, and migration work isolated from request traffic\n- [ ] Graceful shutdown and termination grace period cover the drain\n\n## Anti-patterns\n- Raising `maximum-pool-size` to fix timeouts without finding the slow query or long transaction\n- Pool size copied between services with different replica counts\n- Default 30-second `connection-timeout` behind a shorter gateway timeout\n- `minimum-idle: 0` with a large maximum, causing reconnect storms after quiet periods\n- Leak detection threshold raised until the warnings stop\n- Calling another service, or waiting on a queue, inside `@Transactional`\n\n## Copy-ready SKILL.md stub\n```markdown\n---\nname: hikari-connection-pool-guardrails\ndescription: Use when configuring or reviewing database connection pooling in a Spring Boot service: HikariCP pool size from a connection budget across all replicas at HPA max, fixed pool, connection timeout under the request timeout, maxLifetime and keepalive below idle limits, short transactions with open-in-view off, database-side statement timeouts, leak detection, PgBouncer transaction-mode settings, virtual-thread contention, pool metrics and alerts, and graceful shutdown.\n---\n\nFollow HikariCP Connection Pool Guardrails: size maximum-pool-size from the database connection limit divided across every pod at HPA max, including surge pods and other services, and keep the pool fixed. Set connection-timeout to a few seconds, below the caller's timeout, and map pool exhaustion to 503. Keep max-lifetime and keepalive-time below every database, proxy, load balancer, and credential limit. Turn open-in-view off, keep transactions short, and never call remote services inside one. Put statement and idle-in-transaction timeouts on the database role. Use leak detection in dev and test. Under PgBouncer transaction mode avoid session state and disable server-side prepared statements unless supported. With virtual threads, treat the pool as the concurrency limit. Isolate batch and migration work, publish HikariCP metrics with alerts on timeouts and pending, and shut down gracefully.\n```\n" + }, + { + "id": "api-rate-limiting-guardrails", + "title": "API Rate Limiting and Quota Guardrails for Spring Boot", + "whenToUse": "Adding or reviewing rate limits and quotas for a Spring Boot API on Kubernetes — choosing edge, distributed, in-process, or quota-ledger layers, keying limits on an authenticated client or tenant, token buckets with burst per route class and cost weighting (including LLM token cost), Redis-backed limits that hold across replicas with a store timeout and a documented fail-open or fail-closed choice, fast 429 responses with Retry-After and problem details, outbound limits below provider quotas, exemptions in config, and metrics, alerts, and tests.", + "tools": [ + "copilot", + "claude-code", + "codex" + ], + "tags": [ + "java", + "spring-boot", + "rate-limiting", + "redis", + "api", + "aks", + "security" + ], + "formatNotes": "All three. Copilot: path-scoped `.github/instructions/rate-limiting.instructions.md` with applyTo for `**/*Filter.java`, `**/*Interceptor.java`, `**/*Controller.java`, and `**/application*.yml`, or `.github/skills/api-rate-limiting-guardrails/SKILL.md`; one always-on line in `.github/copilot-instructions.md` (\"limits key on authenticated identity and live in Redis when they span replicas; throttled requests get 429 with Retry-After\"). Claude Code: `.claude/skills/api-rate-limiting-guardrails/SKILL.md`, plus a CLAUDE.md line naming the limiter library and store. Codex: a \"Rate limits\" section in root `AGENTS.md` (or the same SKILL.md under `.agents/skills/`). Pair with spring-security-oauth2-resource-server for the identity the limit keys on, spring-boot-cache-redis-guardrails for Redis, llm-cost-latency-budget-guardrails for token budgets, and spring-boot-http-client-guardrails for outbound retries. This skill owns who may call how often and what they hear when they cannot.", + "body": "# Skill: API Rate Limiting and Quota Guardrails for Spring Boot\n\n## Goal\nEvery public or shared API protects itself and its dependencies with limits that are keyed to a real caller identity, enforced consistently across all replicas, and explained to the caller. A throttled request gets a fast `429 Too Many Requests` with `Retry-After`, never a timeout or a 500, and every limit is visible on a dashboard before anyone complains.\n\n## When to apply\n- Exposing an API to other teams, partners, tenants, or the internet\n- Fronting a costly or fragile dependency: an LLM provider, a database, a third-party API with its own quota\n- Seeing one noisy client degrade everyone else, or retry storms after an outage\n- Adding per-tenant plans, monthly quotas, or token budgets\n- Moving from an in-memory limiter to many replicas behind an HPA\n\n## Pick the layer on purpose\n| Layer | Good for | Watch out for |\n|---|---|---|\n| Edge (API gateway, ingress, WAF) | Coarse per-IP or per-key flood protection | Cannot see app-level identity or cost |\n| Service, distributed (Bucket4j + Redis, Spring Cloud Gateway `RedisRateLimiter`) | Per-client, per-tenant, per-route limits that hold across replicas | Redis latency and outage behavior |\n| Service, in-process (Resilience4j `RateLimiter`, Bucket4j local) | Protecting an outbound dependency from this pod | Limit multiplies by replica count |\n| Quota ledger (database table, usage events) | Daily or monthly plans, billing, token budgets | Not a burst control; pair with a rate limit |\n\nUse more than one: edge for floods, service-level for fairness, outbound limits for dependencies, a ledger for plans.\n\n## Baseline rules\n1. **Key on authenticated identity.** Limit by client id (`azp`/`appid`), tenant, or API key after authentication. Fall back to IP only for anonymous endpoints, and never trust `X-Forwarded-For` unless the ingress overwrites it.\n2. **Token bucket with explicit burst.** Define rate and burst separately (for example 50 requests per second, burst 100). Write limits per route class, not one global number: reads, writes, expensive searches, and LLM calls differ.\n3. **Cost-weighted where cost varies.** Consume more tokens for expensive calls (large page sizes, bulk endpoints, LLM requests weighted by estimated tokens). Reconcile after the call when the real cost is known.\n4. **Shared state for shared limits.** With more than one replica, a per-client limit lives in Redis (atomic Lua or Bucket4j `ProxyManager`), not in each pod's memory. An in-process limit is fine only when the intent is \"per pod\".\n5. **Decide the failure mode.** When the limiter store is down, choose fail-open (keep serving, alert) or fail-closed (reject) per route and write it down. Put a short timeout (tens of milliseconds) on limiter calls so the limiter never becomes the slowest dependency.\n6. **Clear responses.** Return 429 with `Retry-After` in seconds and a problem-details body (`application/problem+json`) naming the limit, never the internal key. Optionally add the IETF `RateLimit-Policy` / `RateLimit` headers so clients can pace themselves. Use 503 for \"we are overloaded\", 429 for \"you are over your limit\".\n7. **Throttle before work.** Check the limit right after authentication, before reading large bodies, opening transactions, or calling downstream services.\n8. **Outbound limits match the provider.** For a third-party or LLM quota, enforce a client-side limiter below the provider's limit, honor their 429 and `Retry-After`, and back off with jitter instead of immediate retries.\n9. **Quotas are separate from rate limits.** Daily or monthly allowances go in a durable ledger with idempotent increments, reset on a documented boundary and time zone, and clear headers or errors when exhausted.\n10. **Exemptions are explicit.** Health probes, internal callers, and break-glass clients are listed in config with an owner, never hard-coded.\n11. **Measure it.** Counters for allowed and rejected requests by route and client class (not raw client id as a high-cardinality tag), limiter latency, and store errors. Alert when rejects jump or a top client nears its limit.\n12. **Test it.** Unit tests for bucket math with a fake clock; an integration test with Testcontainers Redis that runs two app instances against one limit; a load test that proves 429s arrive fast and good clients are unaffected.\n\n## Copy-ready baseline (Bucket4j + Redis, config shape)\n```yaml\nratelimit:\n enabled: true\n store: redis # or local for per-pod limits\n store-timeout: 50ms\n on-store-failure: fail-open # per route override allowed\n key: client-id # client-id | tenant | api-key | ip\n policies:\n default: { capacity: 100, refill: 50, per: 1s }\n search: { capacity: 20, refill: 10, per: 1s, routes: [\"/api/search/**\"] }\n llm-calls: { capacity: 60, refill: 60, per: 1m, cost: estimated-tokens / 1000 }\n exempt-clients: [] # owner and reason for each entry\n```\n\n```java\n// After authentication, before any work\nConsumptionProbe probe = bucket.tryConsumeAndReturnRemaining(cost);\nif (!probe.isConsumed()) {\n long retrySeconds = Math.max(1, TimeUnit.NANOSECONDS.toSeconds(probe.getNanosToWaitForRefill()));\n throw new RateLimitedException(policyName, retrySeconds); // mapped to 429 + Retry-After\n}\n```\n\n## Review checklist\n- [ ] Limit key is an authenticated identity, IP only for anonymous routes\n- [ ] Rate and burst defined per route class, cost-weighted where cost varies\n- [ ] Shared store for limits meant to span replicas; store timeout set\n- [ ] Fail-open or fail-closed decided and documented per route\n- [ ] 429 with `Retry-After` and problem-details body; 503 reserved for overload\n- [ ] Limit checked before body parsing, transactions, and downstream calls\n- [ ] Outbound limiter below provider quota; provider 429 honored with backoff and jitter\n- [ ] Quotas in a durable ledger with a documented reset boundary\n- [ ] Exemptions in config with owners\n- [ ] Metrics and alerts on rejects, near-limit clients, and store errors\n- [ ] Tests with a fake clock, multi-instance Redis test, and a load test\n\n## Anti-patterns\n- An in-memory limiter behind an HPA, so the real limit grows with replica count\n- Keying on `X-Forwarded-For` straight from the client\n- Returning 500 or letting requests time out instead of a fast 429\n- One global limit for every route regardless of cost\n- Retrying a provider 429 immediately in a loop\n- Raw client ids as metric tags, exploding cardinality\n- Treating a monthly quota as a rate limit, or a rate limit as billing\n\n## Copy-ready SKILL.md stub\n```markdown\n---\nname: api-rate-limiting-guardrails\ndescription: Use when adding or reviewing rate limits and quotas in a Spring Boot API: pick edge, distributed, in-process, or ledger layers; key limits on authenticated client or tenant; token bucket with burst per route class and cost weighting; Redis-backed limits across replicas with a store timeout and a documented fail-open or fail-closed choice; 429 with Retry-After and problem details; outbound limits below provider quotas; quota ledgers; exemptions in config; metrics, alerts, and tests.\n---\n\nFollow API Rate Limiting and Quota Guardrails: key every limit on an authenticated identity (IP only for anonymous routes, never a client-supplied forwarded header). Define rate and burst per route class and weight expensive calls by cost. Keep limits that must span replicas in Redis with a short store timeout and a written fail-open or fail-closed choice per route. Check limits right after authentication and before any work. Throttled requests get 429 with Retry-After and a problem-details body; overload gets 503. Keep outbound limiters below provider quotas and back off with jitter on their 429s. Keep daily or monthly quotas in a durable ledger. List exemptions in config with owners. Publish allowed and rejected counts by route and client class, alert on spikes, and test with a fake clock, two instances against one Redis, and a load test.\n```\n" } ] } From be667170cf1ca28449a59250649969081e21c0dd Mon Sep 17 00:00:00 2001 From: Ramkumar Ethiraj <64294490+rmkr-dev@users.noreply.github.com> Date: Sun, 1 Nov 2026 05:04:56 +0000 Subject: [PATCH 2/2] Add prompt for adding rate limits to a Spring Boot API --- promptbook/prompts.json | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/promptbook/prompts.json b/promptbook/prompts.json index f28cea8..c76ce32 100644 --- a/promptbook/prompts.json +++ b/promptbook/prompts.json @@ -900,6 +900,27 @@ ], "formatNotes": "All three. Point the tool at `application*.yml`, `@Transactional` classes and repositories, the Deployment and HPA manifests, and paste `SHOW max_connections;`, a `pg_stat_activity` count grouped by `application_name` and `state`, and a screenshot or export of HikariCP metrics if you have them. Copilot: the datasource path instructions plus those files in agent mode (or save it as `.github/prompts/tune-hikari-connection-pool.prompt.md`). Claude Code: the hikari-connection-pool-guardrails skill plus the files. Codex: AGENTS.md database connections section, then the diff. The prompt proposes a config and code diff; it never changes database settings itself.", "body": "Task: Size and harden this Spring Boot service's database connection pool so it fits the database at full scale, fails fast under pressure, and shows its health on a dashboard. Finish with a connection budget, a config and code diff, database-side settings to review, and a load-test plan.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service and datasources: {{SERVICE}} (e.g. orders-api, one primary datasource)\n- Database engine and tier: {{DATABASE}} (e.g. Azure Database for PostgreSQL Flexible Server, 4 vCores)\n- Database max_connections and reserved slots: {{DB_LIMIT}}\n- Other clients of the same database: {{OTHER_CLIENTS}} (services, jobs, their pool sizes and replica counts)\n- Replicas: {{REPLICAS}} (HPA min and max, rollout maxSurge)\n- Request timeout at the edge: {{EDGE_TIMEOUT}} (e.g. ingress 15s, client 10s)\n- Network path: {{PATH}} (direct, PgBouncer transaction mode, private endpoint, load balancer idle timeout)\n- Credentials: {{CREDENTIALS}} (password, or passwordless token with lifetime)\n- Threads: {{THREADS}} (platform threads with Tomcat max, or virtual threads)\n- Symptoms, if any: {{SYMPTOMS}} (e.g. connection-not-available timeouts, too many clients, p99 spikes)\n\nRequirements:\n\n1. Connection budget\n - Table: client | pool size | replicas at max | surge | total\n - usable = {{DB_LIMIT}} minus reserved; show the per-pod maximum for {{SERVICE}} and whether today's config exceeds it\n - Flag pools copied between services with different replica counts\n\n2. Holding time audit\n - Find `@Transactional` methods and repositories that call HTTP clients, publish and wait on messages, sleep, stream large results, or loop over remote calls\n - Check `spring.jpa.open-in-view` and lazy loading in controllers\n - Table: method | what holds the connection | estimated hold time | fix\n\n3. Pool configuration\n - `maximum-pool-size` from the budget, fixed pool (`minimum-idle` unset), `pool-name`\n - `connection-timeout` below {{EDGE_TIMEOUT}}; map `SQLTransientConnectionException` to 503 with Retry-After\n - `max-lifetime` and `keepalive-time` below every limit in {{PATH}} and {{CREDENTIALS}}, with the reasoning\n - `leak-detection-threshold` in dev and test profiles only\n - If {{PATH}} includes PgBouncer transaction mode: `prepareThreshold=0` (unless supported), no session state across transactions\n - If {{THREADS}} is virtual threads: pool as the concurrency limit, a bulkhead on database-heavy endpoints\n - JDBC `socketTimeout` and `ApplicationName` in the URL\n\n4. Database-side settings (output only)\n - Per-role `statement_timeout`, `idle_in_transaction_session_timeout`, and `CONNECTION LIMIT` matching the budget\n - Note where the managed service sets server parameters instead of SQL\n\n5. Observability\n - Micrometer HikariCP metrics on the actuator endpoint already in use\n - Alerts: `hikaricp_connections_timeout_total` increase above zero; `hikaricp_connections_pending` above zero for 5 minutes; usage p95 above a threshold\n - One dashboard row: active, idle, pending, max, acquire p95, usage p95\n\n6. Shutdown and isolation\n - `server.shutdown=graceful`, `preStop` delay, termination grace period above drain plus statement timeout\n - Batch, report, or migration work on its own pool or deployment; migrations run once, not per replica\n\n7. Proof\n - Load test at expected peak and at 2x: no timeouts at peak; at 2x, requests fail in about the connection timeout with 503, not after 30s\n - Kill a database connection or restart the proxy: pool recovers without restarting pods\n - Rolling deploy at HPA max: total connections stay under the budget\n - Leak test in dev: a deliberate leak produces a leak warning with the stack trace\n\nConstraints:\n- Do not change database server parameters or roles; output the statements to review.\n- Do not raise the pool size to hide slow queries; list the queries instead.\n- Do not invent limits, timeouts, or replica counts; use placeholders and list what must be looked up.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Connection budget table with the per-pod maximum\n2. Holding time audit table with fixes\n3. Patch: application.yml per profile, exception mapping, code changes for long transactions, manifests\n4. Database-side statements to review\n5. Alerts and dashboard definitions\n6. Proof plan with expected results\n7. Risks and follow-ups (e.g. queries to tune, services to resize, a proxy to consider)\n\nStart with the budget. Shorten how long connections are held before you change how many there are.\n" + }, + { + "id": "add-api-rate-limiting", + "title": "Add Rate Limiting and Quotas to a Spring Boot API", + "whenToUse": "Design and add rate limits (and quotas when needed) to a Spring Boot API: a limit table per route class keyed on authenticated identity, Redis-backed limits that hold across replicas with a store timeout and failure mode, cost-weighted buckets, 429 with Retry-After and problem details, outbound limits below provider quotas, metrics and alerts, tests with a fake clock and two instances, and a shadow-mode rollout.", + "tools": [ + "copilot", + "claude-code", + "codex" + ], + "tags": [ + "java", + "spring-boot", + "rate-limiting", + "redis", + "api", + "aks", + "security" + ], + "formatNotes": "All three. Point the tool at the security config and auth filters, controllers, outbound clients, `application*.yml`, and the Deployment and HPA manifests, and paste provider quota pages or numbers and any traffic graph you have. Copilot: the rate-limiting path instructions plus those files in agent mode (or save it as `.github/prompts/add-api-rate-limiting.prompt.md`). Claude Code: the api-rate-limiting-guardrails skill plus the files. Codex: AGENTS.md rate limits section, then the diff. The prompt ships shadow mode first and never enforces a limit it could not justify.", + "body": "Task: Add rate limiting (and, if needed, quotas) to this Spring Boot API so one caller cannot starve others or exhaust a dependency, limits hold across every replica, and throttled callers get a fast, clear 429. Finish with a limit design table, a code and config diff, tests, and a rollout plan.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service and routes: {{SERVICE}} (e.g. orders-api; list route classes such as reads, writes, search, bulk, LLM calls)\n- Caller identity: {{IDENTITY}} (e.g. JWT azp/appid claim, tenant claim, API key header; anonymous routes)\n- Replicas: {{REPLICAS}} (HPA min and max)\n- Shared store available: {{STORE}} (e.g. Azure Cache for Redis, none)\n- Edge layer: {{EDGE}} (ingress, API Management, Front Door/WAF; what it already limits)\n- Downstream quotas: {{DOWNSTREAM}} (provider limits such as requests or tokens per minute)\n- Plans or quotas: {{PLANS}} (e.g. per-tenant daily request quota, monthly token budget, reset time zone)\n- Current symptoms: {{SYMPTOMS}} (e.g. one client spikes, provider 429s, retry storms)\n\nRequirements:\n\n1. Limit design\n - Table: route class | key | rate | burst | cost per call | layer | store | on store failure\n - Explain each number from {{DOWNSTREAM}}, capacity, or a stated assumption; mark assumptions\n - Say which limits are per pod on purpose and which must be shared\n\n2. Identity and key\n - Derive the key from {{IDENTITY}} after authentication; IP only for anonymous routes\n - Confirm how the client IP is obtained (ingress overwrite, trusted proxies) before using it\n - Exemptions in config with owner and reason\n\n3. Implementation\n - A filter or interceptor placed after authentication and before body parsing, transactions, and downstream calls\n - Bucket4j with a Redis `ProxyManager` (or the existing gateway's Redis limiter) for shared limits; local buckets only for per-pod limits\n - Store timeout in tens of milliseconds; fail-open or fail-closed per route as designed, with a metric and log when it happens\n - Cost-weighted consumption where calls differ in cost; reconcile after the call if the real cost is known later\n - 429 with `Retry-After` (seconds) and an `application/problem+json` body; optional `RateLimit-Policy` / `RateLimit` headers\n - Outbound: client-side limiter below {{DOWNSTREAM}}, honor provider 429 and `Retry-After`, backoff with jitter, no unbounded retries\n\n4. Quotas (only if {{PLANS}} is set)\n - Durable ledger with idempotent increments, a documented reset boundary and time zone, and a clear error when exhausted\n - Keep quotas separate from burst limits\n\n5. Observability\n - Counters for allowed and rejected by route class and client class (no raw client ids as tags), limiter latency, store errors, fail-open events\n - Alerts: reject rate jump, a client above 80% of its quota, store errors above zero\n - A structured log line on reject with route class, policy, and a hashed key\n\n6. Tests\n - Unit: bucket math with a fake clock, cost weighting, Retry-After value\n - Integration: Testcontainers Redis, two app contexts sharing one limit, store outage path for both failure modes\n - Contract: 429 body and headers match the documented shape\n - Load: at the limit, good clients unaffected and rejects return within milliseconds\n\n7. Rollout\n - Shadow mode first: count would-be rejects without rejecting, compare with traffic\n - Turn on per route class, highest risk first; publish limits and headers in API docs\n - Rollback switch in config\n\nConstraints:\n- Do not invent provider quotas, plan sizes, or traffic numbers; use placeholders and list what must be looked up.\n- Do not key limits on client-supplied forwarded headers.\n- Do not add a new datastore if {{STORE}} is none; propose per-pod limits plus edge limits and say what that gives up.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Limit design table with assumptions marked\n2. Patch: filter or interceptor, limiter config, exception mapping, outbound limiter, application.yml per profile\n3. Quota ledger schema and code (if needed)\n4. Metrics, alerts, and log fields\n5. Tests\n6. Rollout plan with the shadow-mode check\n7. Risks and follow-ups\n\nStart with the design table. Ship shadow mode before enforcing.\n" } ] }