Add JVM container memory skill and prompt

promptbook/prompts.json

@@ -921,6 +921,27 @@
       ],
       "formatNotes": "All three. Point the tool at the security config and auth filters, controllers, outbound clients, `application*.yml`, and the Deployment and HPA manifests, and paste provider quota pages or numbers and any traffic graph you have. Copilot: the rate-limiting path instructions plus those files in agent mode (or save it as `.github/prompts/add-api-rate-limiting.prompt.md`). Claude Code: the api-rate-limiting-guardrails skill plus the files. Codex: AGENTS.md rate limits section, then the diff. The prompt ships shadow mode first and never enforces a limit it could not justify.",
       "body": "Task: Add rate limiting (and, if needed, quotas) to this Spring Boot API so one caller cannot starve others or exhaust a dependency, limits hold across every replica, and throttled callers get a fast, clear 429. Finish with a limit design table, a code and config diff, tests, and a rollout plan.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service and routes: {{SERVICE}}  (e.g. orders-api; list route classes such as reads, writes, search, bulk, LLM calls)\n- Caller identity: {{IDENTITY}}  (e.g. JWT azp/appid claim, tenant claim, API key header; anonymous routes)\n- Replicas: {{REPLICAS}}  (HPA min and max)\n- Shared store available: {{STORE}}  (e.g. Azure Cache for Redis, none)\n- Edge layer: {{EDGE}}  (ingress, API Management, Front Door/WAF; what it already limits)\n- Downstream quotas: {{DOWNSTREAM}}  (provider limits such as requests or tokens per minute)\n- Plans or quotas: {{PLANS}}  (e.g. per-tenant daily request quota, monthly token budget, reset time zone)\n- Current symptoms: {{SYMPTOMS}}  (e.g. one client spikes, provider 429s, retry storms)\n\nRequirements:\n\n1. Limit design\n   - Table: route class | key | rate | burst | cost per call | layer | store | on store failure\n   - Explain each number from {{DOWNSTREAM}}, capacity, or a stated assumption; mark assumptions\n   - Say which limits are per pod on purpose and which must be shared\n\n2. Identity and key\n   - Derive the key from {{IDENTITY}} after authentication; IP only for anonymous routes\n   - Confirm how the client IP is obtained (ingress overwrite, trusted proxies) before using it\n   - Exemptions in config with owner and reason\n\n3. Implementation\n   - A filter or interceptor placed after authentication and before body parsing, transactions, and downstream calls\n   - Bucket4j with a Redis `ProxyManager` (or the existing gateway's Redis limiter) for shared limits; local buckets only for per-pod limits\n   - Store timeout in tens of milliseconds; fail-open or fail-closed per route as designed, with a metric and log when it happens\n   - Cost-weighted consumption where calls differ in cost; reconcile after the call if the real cost is known later\n   - 429 with `Retry-After` (seconds) and an `application/problem+json` body; optional `RateLimit-Policy` / `RateLimit` headers\n   - Outbound: client-side limiter below {{DOWNSTREAM}}, honor provider 429 and `Retry-After`, backoff with jitter, no unbounded retries\n\n4. Quotas (only if {{PLANS}} is set)\n   - Durable ledger with idempotent increments, a documented reset boundary and time zone, and a clear error when exhausted\n   - Keep quotas separate from burst limits\n\n5. Observability\n   - Counters for allowed and rejected by route class and client class (no raw client ids as tags), limiter latency, store errors, fail-open events\n   - Alerts: reject rate jump, a client above 80% of its quota, store errors above zero\n   - A structured log line on reject with route class, policy, and a hashed key\n\n6. Tests\n   - Unit: bucket math with a fake clock, cost weighting, Retry-After value\n   - Integration: Testcontainers Redis, two app contexts sharing one limit, store outage path for both failure modes\n   - Contract: 429 body and headers match the documented shape\n   - Load: at the limit, good clients unaffected and rejects return within milliseconds\n\n7. Rollout\n   - Shadow mode first: count would-be rejects without rejecting, compare with traffic\n   - Turn on per route class, highest risk first; publish limits and headers in API docs\n   - Rollback switch in config\n\nConstraints:\n- Do not invent provider quotas, plan sizes, or traffic numbers; use placeholders and list what must be looked up.\n- Do not key limits on client-supplied forwarded headers.\n- Do not add a new datastore if {{STORE}} is none; propose per-pod limits plus edge limits and say what that gives up.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Limit design table with assumptions marked\n2. Patch: filter or interceptor, limiter config, exception mapping, outbound limiter, application.yml per profile\n3. Quota ledger schema and code (if needed)\n4. Metrics, alerts, and log fields\n5. Tests\n6. Rollout plan with the shadow-mode check\n7. Risks and follow-ups\n\nStart with the design table. Ship shadow mode before enforcing.\n"
+    },
+    {
+      "id": "right-size-jvm-container-memory",
+      "title": "Right-Size JVM Memory and GC for a Kubernetes Pod",
+      "whenToUse": "Fix OOMKilled restarts, Java OutOfMemoryErrors, wasted memory, or GC and startup trouble in a Java service on Kubernetes: find every JVM flag and the effective values, build a measured memory budget (heap, metaspace, code cache, threads, direct, native, headroom), diagnose kernel OOMKilled versus Java OOM, set MaxRAMPercentage, request equal to limit, visible processors and collector, safe heap dumps, then prove it with Native Memory Tracking under load and add working-set and OOMKilled alerts.",
+      "tools": [
+        "copilot",
+        "claude-code",
+        "codex"
+      ],
+      "tags": [
+        "java",
+        "jvm",
+        "kubernetes",
+        "aks",
+        "memory",
+        "gc",
+        "performance"
+      ],
+      "formatNotes": "All three. Point the tool at the Dockerfile or buildpack config, entrypoint scripts, the Deployment, Kustomize or Helm values that set resources and JVM env vars, and `application*.yml`, and paste `kubectl describe pod` last state, working-set graphs, GC logs, or NMT output if you have them. Copilot: the JVM container path instructions plus those files in agent mode (or save it as `.github/prompts/right-size-jvm-container-memory.prompt.md`). Claude Code: the jvm-container-memory-guardrails skill plus the files. Codex: AGENTS.md JVM in containers section, then the diff. The prompt measures before it raises any limit and marks every number as measured or estimated.",
+      "body": "Task: Right-size memory and GC for this Java service on Kubernetes so it stops getting OOMKilled (or stops wasting memory), the heap and non-heap fit the container limit with stated headroom, and the JVM sees the CPU it is given. Finish with a memory budget table, a manifest and image diff, a measurement plan, and alerts.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service: {{SERVICE}}  (e.g. orders-api; Spring Boot version, web stack such as MVC, WebFlux, or Netty clients)\n- JDK and image: {{JDK}}  (e.g. Temurin 21 JRE on Ubuntu, distroless Java 21, Paketo buildpack image)\n- Current resources: {{RESOURCES}}  (requests and limits for CPU and memory, HPA min and max)\n- Current JVM flags: {{FLAGS}}  (Dockerfile CMD/ENTRYPOINT, JAVA_TOOL_OPTIONS, JDK_JAVA_OPTIONS, Helm or Kustomize values)\n- Threads: {{THREADS}}  (platform or virtual threads; known pool sizes such as Tomcat, Kafka consumers, executors)\n- Latency goal: {{SLO}}  (e.g. p99 under 300 ms, batch throughput only)\n- Symptoms: {{SYMPTOMS}}  (e.g. OOMKilled exit 137 every few days, OutOfMemoryError: Direct buffer memory, slow startup, long GC pauses, CPU throttling)\n- Evidence: {{EVIDENCE}}  (pod events, `kubectl describe` last state, working-set graphs, GC logs, NMT output if any)\n\nRequirements:\n\n1. Find every JVM setting\n   - List all places flags are set and which one wins (explicit `-Xmx` beats percentages; later flags override earlier ones)\n   - Show the effective values: heap max, processor count, selected collector (from `PrintFlagsFinal`, `jcmd VM.flags`, or by reasoning from the limits if you cannot run them, marked as reasoning)\n   - Flag accidental SerialGC (under 2 visible processors or about 1792 MiB) and a JVM that sees all node cores because there is no CPU limit\n\n2. Memory budget\n   - Table: region | current | measured or estimated | cap flag | notes, for heap, metaspace, code cache, thread stacks, direct buffers, GC overhead, native and malloc, headroom\n   - Mark every number as measured or estimated; do not invent measurements\n   - Derive the limit as heap plus non-heap plus 10-20% headroom, and `MaxRAMPercentage` as heap divided by limit\n\n3. Diagnose the symptom\n   - Kernel OOMKilled (exit 137, no Java stack trace) versus Java `OutOfMemoryError` (heap, Metaspace, Direct buffer memory, unable to create native thread): explain which it is from {{EVIDENCE}} and what that implies\n   - If the gap between heap and working set is large, name the likely regions and how to confirm\n\n4. Changes\n   - Heap from `MaxRAMPercentage` (and `InitialRAMPercentage` if a fixed heap suits {{SLO}}); remove `-Xmx` equal to or near the limit\n   - Memory request equal to the limit\n   - `ActiveProcessorCount` or a CPU limit so visible processors match the CPU given; justify the CPU limit choice\n   - Collector choice with a reason tied to heap size and {{SLO}}\n   - `MaxDirectMemorySize` if Netty, gRPC, or NIO buffers are used; bounded thread pools if {{THREADS}} is platform threads\n   - `ExitOnOutOfMemoryError`; heap dumps to a sized `emptyDir` or volume with restricted access and a cleanup step\n   - All flags in one environment variable; if the image uses the Paketo memory calculator, tune it instead of adding `-Xmx`\n   - `startupProbe` sized from startup time measured under the CPU limit\n   - GC logging to stdout at a level the log pipeline can afford\n\n5. Measurement plan\n   - Load test with `-XX:NativeMemoryTracking=summary`: baseline after warm-up, peak traffic, `summary.diff`; compare NMT committed with the pod working set\n   - Soak test long enough to catch slow native growth\n   - Record heap after GC, GC pause p99, throughput, working set at peak, and startup time before and after\n\n6. Observability\n   - Alerts: working set above 90% of the limit for 10 minutes; any OOMKilled restart; GC pause p99 above a threshold from {{SLO}}\n   - Dashboard row: working set vs limit, heap used after GC, non-heap by pool, direct buffers, live threads, GC pause and overhead, CPU throttling\n   - No memory-based autoscaling for this JVM; propose CPU or request-rate scaling if memory is used today\n\nConstraints:\n- Do not invent limits, measurements, or traffic numbers; use placeholders and list what must be measured.\n- Do not \"fix\" OOMKilled by only raising the limit; show what grew.\n- Do not move heap dumps off the cluster or into tickets; they can hold secrets and personal data.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Effective settings today (where set, which wins, values)\n2. Memory budget table with measured or estimated marked\n3. Diagnosis of the symptom\n4. Patch: Deployment or Kustomize patch, Dockerfile or buildpack env, probe changes\n5. Measurement plan with expected results\n6. Alerts and dashboard definitions\n7. Risks and follow-ups (e.g. leaks to chase, pools to bound, startup work such as CDS)\n\nStart with the effective settings. Measure before you raise any limit.\n"
     }
   ]
 }