Add API rate limiting skill and prompt

promptbook/prompts.json

@@ -900,6 +900,27 @@
       ],
       "formatNotes": "All three. Point the tool at `application*.yml`, `@Transactional` classes and repositories, the Deployment and HPA manifests, and paste `SHOW max_connections;`, a `pg_stat_activity` count grouped by `application_name` and `state`, and a screenshot or export of HikariCP metrics if you have them. Copilot: the datasource path instructions plus those files in agent mode (or save it as `.github/prompts/tune-hikari-connection-pool.prompt.md`). Claude Code: the hikari-connection-pool-guardrails skill plus the files. Codex: AGENTS.md database connections section, then the diff. The prompt proposes a config and code diff; it never changes database settings itself.",
       "body": "Task: Size and harden this Spring Boot service's database connection pool so it fits the database at full scale, fails fast under pressure, and shows its health on a dashboard. Finish with a connection budget, a config and code diff, database-side settings to review, and a load-test plan.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service and datasources: {{SERVICE}}  (e.g. orders-api, one primary datasource)\n- Database engine and tier: {{DATABASE}}  (e.g. Azure Database for PostgreSQL Flexible Server, 4 vCores)\n- Database max_connections and reserved slots: {{DB_LIMIT}}\n- Other clients of the same database: {{OTHER_CLIENTS}}  (services, jobs, their pool sizes and replica counts)\n- Replicas: {{REPLICAS}}  (HPA min and max, rollout maxSurge)\n- Request timeout at the edge: {{EDGE_TIMEOUT}}  (e.g. ingress 15s, client 10s)\n- Network path: {{PATH}}  (direct, PgBouncer transaction mode, private endpoint, load balancer idle timeout)\n- Credentials: {{CREDENTIALS}}  (password, or passwordless token with lifetime)\n- Threads: {{THREADS}}  (platform threads with Tomcat max, or virtual threads)\n- Symptoms, if any: {{SYMPTOMS}}  (e.g. connection-not-available timeouts, too many clients, p99 spikes)\n\nRequirements:\n\n1. Connection budget\n   - Table: client | pool size | replicas at max | surge | total\n   - usable = {{DB_LIMIT}} minus reserved; show the per-pod maximum for {{SERVICE}} and whether today's config exceeds it\n   - Flag pools copied between services with different replica counts\n\n2. Holding time audit\n   - Find `@Transactional` methods and repositories that call HTTP clients, publish and wait on messages, sleep, stream large results, or loop over remote calls\n   - Check `spring.jpa.open-in-view` and lazy loading in controllers\n   - Table: method | what holds the connection | estimated hold time | fix\n\n3. Pool configuration\n   - `maximum-pool-size` from the budget, fixed pool (`minimum-idle` unset), `pool-name`\n   - `connection-timeout` below {{EDGE_TIMEOUT}}; map `SQLTransientConnectionException` to 503 with Retry-After\n   - `max-lifetime` and `keepalive-time` below every limit in {{PATH}} and {{CREDENTIALS}}, with the reasoning\n   - `leak-detection-threshold` in dev and test profiles only\n   - If {{PATH}} includes PgBouncer transaction mode: `prepareThreshold=0` (unless supported), no session state across transactions\n   - If {{THREADS}} is virtual threads: pool as the concurrency limit, a bulkhead on database-heavy endpoints\n   - JDBC `socketTimeout` and `ApplicationName` in the URL\n\n4. Database-side settings (output only)\n   - Per-role `statement_timeout`, `idle_in_transaction_session_timeout`, and `CONNECTION LIMIT` matching the budget\n   - Note where the managed service sets server parameters instead of SQL\n\n5. Observability\n   - Micrometer HikariCP metrics on the actuator endpoint already in use\n   - Alerts: `hikaricp_connections_timeout_total` increase above zero; `hikaricp_connections_pending` above zero for 5 minutes; usage p95 above a threshold\n   - One dashboard row: active, idle, pending, max, acquire p95, usage p95\n\n6. Shutdown and isolation\n   - `server.shutdown=graceful`, `preStop` delay, termination grace period above drain plus statement timeout\n   - Batch, report, or migration work on its own pool or deployment; migrations run once, not per replica\n\n7. Proof\n   - Load test at expected peak and at 2x: no timeouts at peak; at 2x, requests fail in about the connection timeout with 503, not after 30s\n   - Kill a database connection or restart the proxy: pool recovers without restarting pods\n   - Rolling deploy at HPA max: total connections stay under the budget\n   - Leak test in dev: a deliberate leak produces a leak warning with the stack trace\n\nConstraints:\n- Do not change database server parameters or roles; output the statements to review.\n- Do not raise the pool size to hide slow queries; list the queries instead.\n- Do not invent limits, timeouts, or replica counts; use placeholders and list what must be looked up.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Connection budget table with the per-pod maximum\n2. Holding time audit table with fixes\n3. Patch: application.yml per profile, exception mapping, code changes for long transactions, manifests\n4. Database-side statements to review\n5. Alerts and dashboard definitions\n6. Proof plan with expected results\n7. Risks and follow-ups (e.g. queries to tune, services to resize, a proxy to consider)\n\nStart with the budget. Shorten how long connections are held before you change how many there are.\n"
+    },
+    {
+      "id": "add-api-rate-limiting",
+      "title": "Add Rate Limiting and Quotas to a Spring Boot API",
+      "whenToUse": "Design and add rate limits (and quotas when needed) to a Spring Boot API: a limit table per route class keyed on authenticated identity, Redis-backed limits that hold across replicas with a store timeout and failure mode, cost-weighted buckets, 429 with Retry-After and problem details, outbound limits below provider quotas, metrics and alerts, tests with a fake clock and two instances, and a shadow-mode rollout.",
+      "tools": [
+        "copilot",
+        "claude-code",
+        "codex"
+      ],
+      "tags": [
+        "java",
+        "spring-boot",
+        "rate-limiting",
+        "redis",
+        "api",
+        "aks",
+        "security"
+      ],
+      "formatNotes": "All three. Point the tool at the security config and auth filters, controllers, outbound clients, `application*.yml`, and the Deployment and HPA manifests, and paste provider quota pages or numbers and any traffic graph you have. Copilot: the rate-limiting path instructions plus those files in agent mode (or save it as `.github/prompts/add-api-rate-limiting.prompt.md`). Claude Code: the api-rate-limiting-guardrails skill plus the files. Codex: AGENTS.md rate limits section, then the diff. The prompt ships shadow mode first and never enforces a limit it could not justify.",
+      "body": "Task: Add rate limiting (and, if needed, quotas) to this Spring Boot API so one caller cannot starve others or exhaust a dependency, limits hold across every replica, and throttled callers get a fast, clear 429. Finish with a limit design table, a code and config diff, tests, and a rollout plan.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service and routes: {{SERVICE}}  (e.g. orders-api; list route classes such as reads, writes, search, bulk, LLM calls)\n- Caller identity: {{IDENTITY}}  (e.g. JWT azp/appid claim, tenant claim, API key header; anonymous routes)\n- Replicas: {{REPLICAS}}  (HPA min and max)\n- Shared store available: {{STORE}}  (e.g. Azure Cache for Redis, none)\n- Edge layer: {{EDGE}}  (ingress, API Management, Front Door/WAF; what it already limits)\n- Downstream quotas: {{DOWNSTREAM}}  (provider limits such as requests or tokens per minute)\n- Plans or quotas: {{PLANS}}  (e.g. per-tenant daily request quota, monthly token budget, reset time zone)\n- Current symptoms: {{SYMPTOMS}}  (e.g. one client spikes, provider 429s, retry storms)\n\nRequirements:\n\n1. Limit design\n   - Table: route class | key | rate | burst | cost per call | layer | store | on store failure\n   - Explain each number from {{DOWNSTREAM}}, capacity, or a stated assumption; mark assumptions\n   - Say which limits are per pod on purpose and which must be shared\n\n2. Identity and key\n   - Derive the key from {{IDENTITY}} after authentication; IP only for anonymous routes\n   - Confirm how the client IP is obtained (ingress overwrite, trusted proxies) before using it\n   - Exemptions in config with owner and reason\n\n3. Implementation\n   - A filter or interceptor placed after authentication and before body parsing, transactions, and downstream calls\n   - Bucket4j with a Redis `ProxyManager` (or the existing gateway's Redis limiter) for shared limits; local buckets only for per-pod limits\n   - Store timeout in tens of milliseconds; fail-open or fail-closed per route as designed, with a metric and log when it happens\n   - Cost-weighted consumption where calls differ in cost; reconcile after the call if the real cost is known later\n   - 429 with `Retry-After` (seconds) and an `application/problem+json` body; optional `RateLimit-Policy` / `RateLimit` headers\n   - Outbound: client-side limiter below {{DOWNSTREAM}}, honor provider 429 and `Retry-After`, backoff with jitter, no unbounded retries\n\n4. Quotas (only if {{PLANS}} is set)\n   - Durable ledger with idempotent increments, a documented reset boundary and time zone, and a clear error when exhausted\n   - Keep quotas separate from burst limits\n\n5. Observability\n   - Counters for allowed and rejected by route class and client class (no raw client ids as tags), limiter latency, store errors, fail-open events\n   - Alerts: reject rate jump, a client above 80% of its quota, store errors above zero\n   - A structured log line on reject with route class, policy, and a hashed key\n\n6. Tests\n   - Unit: bucket math with a fake clock, cost weighting, Retry-After value\n   - Integration: Testcontainers Redis, two app contexts sharing one limit, store outage path for both failure modes\n   - Contract: 429 body and headers match the documented shape\n   - Load: at the limit, good clients unaffected and rejects return within milliseconds\n\n7. Rollout\n   - Shadow mode first: count would-be rejects without rejecting, compare with traffic\n   - Turn on per route class, highest risk first; publish limits and headers in API docs\n   - Rollback switch in config\n\nConstraints:\n- Do not invent provider quotas, plan sizes, or traffic numbers; use placeholders and list what must be looked up.\n- Do not key limits on client-supplied forwarded headers.\n- Do not add a new datastore if {{STORE}} is none; propose per-pod limits plus edge limits and say what that gives up.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Limit design table with assumptions marked\n2. Patch: filter or interceptor, limiter config, exception mapping, outbound limiter, application.yml per profile\n3. Quota ledger schema and code (if needed)\n4. Metrics, alerts, and log fields\n5. Tests\n6. Rollout plan with the shadow-mode check\n7. Risks and follow-ups\n\nStart with the design table. Ship shadow mode before enforcing.\n"
     }
   ]
 }