diff --git a/promptbook/prompts.json b/promptbook/prompts.json index c76ce32..3e3d3eb 100644 --- a/promptbook/prompts.json +++ b/promptbook/prompts.json @@ -921,6 +921,27 @@ ], "formatNotes": "All three. Point the tool at the security config and auth filters, controllers, outbound clients, `application*.yml`, and the Deployment and HPA manifests, and paste provider quota pages or numbers and any traffic graph you have. Copilot: the rate-limiting path instructions plus those files in agent mode (or save it as `.github/prompts/add-api-rate-limiting.prompt.md`). Claude Code: the api-rate-limiting-guardrails skill plus the files. Codex: AGENTS.md rate limits section, then the diff. The prompt ships shadow mode first and never enforces a limit it could not justify.", "body": "Task: Add rate limiting (and, if needed, quotas) to this Spring Boot API so one caller cannot starve others or exhaust a dependency, limits hold across every replica, and throttled callers get a fast, clear 429. Finish with a limit design table, a code and config diff, tests, and a rollout plan.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service and routes: {{SERVICE}} (e.g. orders-api; list route classes such as reads, writes, search, bulk, LLM calls)\n- Caller identity: {{IDENTITY}} (e.g. JWT azp/appid claim, tenant claim, API key header; anonymous routes)\n- Replicas: {{REPLICAS}} (HPA min and max)\n- Shared store available: {{STORE}} (e.g. Azure Cache for Redis, none)\n- Edge layer: {{EDGE}} (ingress, API Management, Front Door/WAF; what it already limits)\n- Downstream quotas: {{DOWNSTREAM}} (provider limits such as requests or tokens per minute)\n- Plans or quotas: {{PLANS}} (e.g. per-tenant daily request quota, monthly token budget, reset time zone)\n- Current symptoms: {{SYMPTOMS}} (e.g. one client spikes, provider 429s, retry storms)\n\nRequirements:\n\n1. Limit design\n - Table: route class | key | rate | burst | cost per call | layer | store | on store failure\n - Explain each number from {{DOWNSTREAM}}, capacity, or a stated assumption; mark assumptions\n - Say which limits are per pod on purpose and which must be shared\n\n2. Identity and key\n - Derive the key from {{IDENTITY}} after authentication; IP only for anonymous routes\n - Confirm how the client IP is obtained (ingress overwrite, trusted proxies) before using it\n - Exemptions in config with owner and reason\n\n3. Implementation\n - A filter or interceptor placed after authentication and before body parsing, transactions, and downstream calls\n - Bucket4j with a Redis `ProxyManager` (or the existing gateway's Redis limiter) for shared limits; local buckets only for per-pod limits\n - Store timeout in tens of milliseconds; fail-open or fail-closed per route as designed, with a metric and log when it happens\n - Cost-weighted consumption where calls differ in cost; reconcile after the call if the real cost is known later\n - 429 with `Retry-After` (seconds) and an `application/problem+json` body; optional `RateLimit-Policy` / `RateLimit` headers\n - Outbound: client-side limiter below {{DOWNSTREAM}}, honor provider 429 and `Retry-After`, backoff with jitter, no unbounded retries\n\n4. Quotas (only if {{PLANS}} is set)\n - Durable ledger with idempotent increments, a documented reset boundary and time zone, and a clear error when exhausted\n - Keep quotas separate from burst limits\n\n5. Observability\n - Counters for allowed and rejected by route class and client class (no raw client ids as tags), limiter latency, store errors, fail-open events\n - Alerts: reject rate jump, a client above 80% of its quota, store errors above zero\n - A structured log line on reject with route class, policy, and a hashed key\n\n6. Tests\n - Unit: bucket math with a fake clock, cost weighting, Retry-After value\n - Integration: Testcontainers Redis, two app contexts sharing one limit, store outage path for both failure modes\n - Contract: 429 body and headers match the documented shape\n - Load: at the limit, good clients unaffected and rejects return within milliseconds\n\n7. Rollout\n - Shadow mode first: count would-be rejects without rejecting, compare with traffic\n - Turn on per route class, highest risk first; publish limits and headers in API docs\n - Rollback switch in config\n\nConstraints:\n- Do not invent provider quotas, plan sizes, or traffic numbers; use placeholders and list what must be looked up.\n- Do not key limits on client-supplied forwarded headers.\n- Do not add a new datastore if {{STORE}} is none; propose per-pod limits plus edge limits and say what that gives up.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Limit design table with assumptions marked\n2. Patch: filter or interceptor, limiter config, exception mapping, outbound limiter, application.yml per profile\n3. Quota ledger schema and code (if needed)\n4. Metrics, alerts, and log fields\n5. Tests\n6. Rollout plan with the shadow-mode check\n7. Risks and follow-ups\n\nStart with the design table. Ship shadow mode before enforcing.\n" + }, + { + "id": "right-size-jvm-container-memory", + "title": "Right-Size JVM Memory and GC for a Kubernetes Pod", + "whenToUse": "Fix OOMKilled restarts, Java OutOfMemoryErrors, wasted memory, or GC and startup trouble in a Java service on Kubernetes: find every JVM flag and the effective values, build a measured memory budget (heap, metaspace, code cache, threads, direct, native, headroom), diagnose kernel OOMKilled versus Java OOM, set MaxRAMPercentage, request equal to limit, visible processors and collector, safe heap dumps, then prove it with Native Memory Tracking under load and add working-set and OOMKilled alerts.", + "tools": [ + "copilot", + "claude-code", + "codex" + ], + "tags": [ + "java", + "jvm", + "kubernetes", + "aks", + "memory", + "gc", + "performance" + ], + "formatNotes": "All three. Point the tool at the Dockerfile or buildpack config, entrypoint scripts, the Deployment, Kustomize or Helm values that set resources and JVM env vars, and `application*.yml`, and paste `kubectl describe pod` last state, working-set graphs, GC logs, or NMT output if you have them. Copilot: the JVM container path instructions plus those files in agent mode (or save it as `.github/prompts/right-size-jvm-container-memory.prompt.md`). Claude Code: the jvm-container-memory-guardrails skill plus the files. Codex: AGENTS.md JVM in containers section, then the diff. The prompt measures before it raises any limit and marks every number as measured or estimated.", + "body": "Task: Right-size memory and GC for this Java service on Kubernetes so it stops getting OOMKilled (or stops wasting memory), the heap and non-heap fit the container limit with stated headroom, and the JVM sees the CPU it is given. Finish with a memory budget table, a manifest and image diff, a measurement plan, and alerts.\n\nContext placeholders (fill from the repo or ask only if missing):\n- Service: {{SERVICE}} (e.g. orders-api; Spring Boot version, web stack such as MVC, WebFlux, or Netty clients)\n- JDK and image: {{JDK}} (e.g. Temurin 21 JRE on Ubuntu, distroless Java 21, Paketo buildpack image)\n- Current resources: {{RESOURCES}} (requests and limits for CPU and memory, HPA min and max)\n- Current JVM flags: {{FLAGS}} (Dockerfile CMD/ENTRYPOINT, JAVA_TOOL_OPTIONS, JDK_JAVA_OPTIONS, Helm or Kustomize values)\n- Threads: {{THREADS}} (platform or virtual threads; known pool sizes such as Tomcat, Kafka consumers, executors)\n- Latency goal: {{SLO}} (e.g. p99 under 300 ms, batch throughput only)\n- Symptoms: {{SYMPTOMS}} (e.g. OOMKilled exit 137 every few days, OutOfMemoryError: Direct buffer memory, slow startup, long GC pauses, CPU throttling)\n- Evidence: {{EVIDENCE}} (pod events, `kubectl describe` last state, working-set graphs, GC logs, NMT output if any)\n\nRequirements:\n\n1. Find every JVM setting\n - List all places flags are set and which one wins (explicit `-Xmx` beats percentages; later flags override earlier ones)\n - Show the effective values: heap max, processor count, selected collector (from `PrintFlagsFinal`, `jcmd VM.flags`, or by reasoning from the limits if you cannot run them, marked as reasoning)\n - Flag accidental SerialGC (under 2 visible processors or about 1792 MiB) and a JVM that sees all node cores because there is no CPU limit\n\n2. Memory budget\n - Table: region | current | measured or estimated | cap flag | notes, for heap, metaspace, code cache, thread stacks, direct buffers, GC overhead, native and malloc, headroom\n - Mark every number as measured or estimated; do not invent measurements\n - Derive the limit as heap plus non-heap plus 10-20% headroom, and `MaxRAMPercentage` as heap divided by limit\n\n3. Diagnose the symptom\n - Kernel OOMKilled (exit 137, no Java stack trace) versus Java `OutOfMemoryError` (heap, Metaspace, Direct buffer memory, unable to create native thread): explain which it is from {{EVIDENCE}} and what that implies\n - If the gap between heap and working set is large, name the likely regions and how to confirm\n\n4. Changes\n - Heap from `MaxRAMPercentage` (and `InitialRAMPercentage` if a fixed heap suits {{SLO}}); remove `-Xmx` equal to or near the limit\n - Memory request equal to the limit\n - `ActiveProcessorCount` or a CPU limit so visible processors match the CPU given; justify the CPU limit choice\n - Collector choice with a reason tied to heap size and {{SLO}}\n - `MaxDirectMemorySize` if Netty, gRPC, or NIO buffers are used; bounded thread pools if {{THREADS}} is platform threads\n - `ExitOnOutOfMemoryError`; heap dumps to a sized `emptyDir` or volume with restricted access and a cleanup step\n - All flags in one environment variable; if the image uses the Paketo memory calculator, tune it instead of adding `-Xmx`\n - `startupProbe` sized from startup time measured under the CPU limit\n - GC logging to stdout at a level the log pipeline can afford\n\n5. Measurement plan\n - Load test with `-XX:NativeMemoryTracking=summary`: baseline after warm-up, peak traffic, `summary.diff`; compare NMT committed with the pod working set\n - Soak test long enough to catch slow native growth\n - Record heap after GC, GC pause p99, throughput, working set at peak, and startup time before and after\n\n6. Observability\n - Alerts: working set above 90% of the limit for 10 minutes; any OOMKilled restart; GC pause p99 above a threshold from {{SLO}}\n - Dashboard row: working set vs limit, heap used after GC, non-heap by pool, direct buffers, live threads, GC pause and overhead, CPU throttling\n - No memory-based autoscaling for this JVM; propose CPU or request-rate scaling if memory is used today\n\nConstraints:\n- Do not invent limits, measurements, or traffic numbers; use placeholders and list what must be measured.\n- Do not \"fix\" OOMKilled by only raising the limit; show what grew.\n- Do not move heap dumps off the cluster or into tickets; they can hold secrets and personal data.\n- Minimal diff; no unrelated refactors.\n- Optimize for GitHub Copilot, Claude Code, and Codex: give concrete files and a checklist.\n\nOutput format:\n1. Effective settings today (where set, which wins, values)\n2. Memory budget table with measured or estimated marked\n3. Diagnosis of the symptom\n4. Patch: Deployment or Kustomize patch, Dockerfile or buildpack env, probe changes\n5. Measurement plan with expected results\n6. Alerts and dashboard definitions\n7. Risks and follow-ups (e.g. leaks to chase, pools to bound, startup work such as CDS)\n\nStart with the effective settings. Measure before you raise any limit.\n" } ] } diff --git a/skillbook/skills.json b/skillbook/skills.json index e6829a5..97c2a96 100644 --- a/skillbook/skills.json +++ b/skillbook/skills.json @@ -919,6 +919,27 @@ ], "formatNotes": "All three. Copilot: path-scoped `.github/instructions/rate-limiting.instructions.md` with applyTo for `**/*Filter.java`, `**/*Interceptor.java`, `**/*Controller.java`, and `**/application*.yml`, or `.github/skills/api-rate-limiting-guardrails/SKILL.md`; one always-on line in `.github/copilot-instructions.md` (\"limits key on authenticated identity and live in Redis when they span replicas; throttled requests get 429 with Retry-After\"). Claude Code: `.claude/skills/api-rate-limiting-guardrails/SKILL.md`, plus a CLAUDE.md line naming the limiter library and store. Codex: a \"Rate limits\" section in root `AGENTS.md` (or the same SKILL.md under `.agents/skills/`). Pair with spring-security-oauth2-resource-server for the identity the limit keys on, spring-boot-cache-redis-guardrails for Redis, llm-cost-latency-budget-guardrails for token budgets, and spring-boot-http-client-guardrails for outbound retries. This skill owns who may call how often and what they hear when they cannot.", "body": "# Skill: API Rate Limiting and Quota Guardrails for Spring Boot\n\n## Goal\nEvery public or shared API protects itself and its dependencies with limits that are keyed to a real caller identity, enforced consistently across all replicas, and explained to the caller. A throttled request gets a fast `429 Too Many Requests` with `Retry-After`, never a timeout or a 500, and every limit is visible on a dashboard before anyone complains.\n\n## When to apply\n- Exposing an API to other teams, partners, tenants, or the internet\n- Fronting a costly or fragile dependency: an LLM provider, a database, a third-party API with its own quota\n- Seeing one noisy client degrade everyone else, or retry storms after an outage\n- Adding per-tenant plans, monthly quotas, or token budgets\n- Moving from an in-memory limiter to many replicas behind an HPA\n\n## Pick the layer on purpose\n| Layer | Good for | Watch out for |\n|---|---|---|\n| Edge (API gateway, ingress, WAF) | Coarse per-IP or per-key flood protection | Cannot see app-level identity or cost |\n| Service, distributed (Bucket4j + Redis, Spring Cloud Gateway `RedisRateLimiter`) | Per-client, per-tenant, per-route limits that hold across replicas | Redis latency and outage behavior |\n| Service, in-process (Resilience4j `RateLimiter`, Bucket4j local) | Protecting an outbound dependency from this pod | Limit multiplies by replica count |\n| Quota ledger (database table, usage events) | Daily or monthly plans, billing, token budgets | Not a burst control; pair with a rate limit |\n\nUse more than one: edge for floods, service-level for fairness, outbound limits for dependencies, a ledger for plans.\n\n## Baseline rules\n1. **Key on authenticated identity.** Limit by client id (`azp`/`appid`), tenant, or API key after authentication. Fall back to IP only for anonymous endpoints, and never trust `X-Forwarded-For` unless the ingress overwrites it.\n2. **Token bucket with explicit burst.** Define rate and burst separately (for example 50 requests per second, burst 100). Write limits per route class, not one global number: reads, writes, expensive searches, and LLM calls differ.\n3. **Cost-weighted where cost varies.** Consume more tokens for expensive calls (large page sizes, bulk endpoints, LLM requests weighted by estimated tokens). Reconcile after the call when the real cost is known.\n4. **Shared state for shared limits.** With more than one replica, a per-client limit lives in Redis (atomic Lua or Bucket4j `ProxyManager`), not in each pod's memory. An in-process limit is fine only when the intent is \"per pod\".\n5. **Decide the failure mode.** When the limiter store is down, choose fail-open (keep serving, alert) or fail-closed (reject) per route and write it down. Put a short timeout (tens of milliseconds) on limiter calls so the limiter never becomes the slowest dependency.\n6. **Clear responses.** Return 429 with `Retry-After` in seconds and a problem-details body (`application/problem+json`) naming the limit, never the internal key. Optionally add the IETF `RateLimit-Policy` / `RateLimit` headers so clients can pace themselves. Use 503 for \"we are overloaded\", 429 for \"you are over your limit\".\n7. **Throttle before work.** Check the limit right after authentication, before reading large bodies, opening transactions, or calling downstream services.\n8. **Outbound limits match the provider.** For a third-party or LLM quota, enforce a client-side limiter below the provider's limit, honor their 429 and `Retry-After`, and back off with jitter instead of immediate retries.\n9. **Quotas are separate from rate limits.** Daily or monthly allowances go in a durable ledger with idempotent increments, reset on a documented boundary and time zone, and clear headers or errors when exhausted.\n10. **Exemptions are explicit.** Health probes, internal callers, and break-glass clients are listed in config with an owner, never hard-coded.\n11. **Measure it.** Counters for allowed and rejected requests by route and client class (not raw client id as a high-cardinality tag), limiter latency, and store errors. Alert when rejects jump or a top client nears its limit.\n12. **Test it.** Unit tests for bucket math with a fake clock; an integration test with Testcontainers Redis that runs two app instances against one limit; a load test that proves 429s arrive fast and good clients are unaffected.\n\n## Copy-ready baseline (Bucket4j + Redis, config shape)\n```yaml\nratelimit:\n enabled: true\n store: redis # or local for per-pod limits\n store-timeout: 50ms\n on-store-failure: fail-open # per route override allowed\n key: client-id # client-id | tenant | api-key | ip\n policies:\n default: { capacity: 100, refill: 50, per: 1s }\n search: { capacity: 20, refill: 10, per: 1s, routes: [\"/api/search/**\"] }\n llm-calls: { capacity: 60, refill: 60, per: 1m, cost: estimated-tokens / 1000 }\n exempt-clients: [] # owner and reason for each entry\n```\n\n```java\n// After authentication, before any work\nConsumptionProbe probe = bucket.tryConsumeAndReturnRemaining(cost);\nif (!probe.isConsumed()) {\n long retrySeconds = Math.max(1, TimeUnit.NANOSECONDS.toSeconds(probe.getNanosToWaitForRefill()));\n throw new RateLimitedException(policyName, retrySeconds); // mapped to 429 + Retry-After\n}\n```\n\n## Review checklist\n- [ ] Limit key is an authenticated identity, IP only for anonymous routes\n- [ ] Rate and burst defined per route class, cost-weighted where cost varies\n- [ ] Shared store for limits meant to span replicas; store timeout set\n- [ ] Fail-open or fail-closed decided and documented per route\n- [ ] 429 with `Retry-After` and problem-details body; 503 reserved for overload\n- [ ] Limit checked before body parsing, transactions, and downstream calls\n- [ ] Outbound limiter below provider quota; provider 429 honored with backoff and jitter\n- [ ] Quotas in a durable ledger with a documented reset boundary\n- [ ] Exemptions in config with owners\n- [ ] Metrics and alerts on rejects, near-limit clients, and store errors\n- [ ] Tests with a fake clock, multi-instance Redis test, and a load test\n\n## Anti-patterns\n- An in-memory limiter behind an HPA, so the real limit grows with replica count\n- Keying on `X-Forwarded-For` straight from the client\n- Returning 500 or letting requests time out instead of a fast 429\n- One global limit for every route regardless of cost\n- Retrying a provider 429 immediately in a loop\n- Raw client ids as metric tags, exploding cardinality\n- Treating a monthly quota as a rate limit, or a rate limit as billing\n\n## Copy-ready SKILL.md stub\n```markdown\n---\nname: api-rate-limiting-guardrails\ndescription: Use when adding or reviewing rate limits and quotas in a Spring Boot API: pick edge, distributed, in-process, or ledger layers; key limits on authenticated client or tenant; token bucket with burst per route class and cost weighting; Redis-backed limits across replicas with a store timeout and a documented fail-open or fail-closed choice; 429 with Retry-After and problem details; outbound limits below provider quotas; quota ledgers; exemptions in config; metrics, alerts, and tests.\n---\n\nFollow API Rate Limiting and Quota Guardrails: key every limit on an authenticated identity (IP only for anonymous routes, never a client-supplied forwarded header). Define rate and burst per route class and weight expensive calls by cost. Keep limits that must span replicas in Redis with a short store timeout and a written fail-open or fail-closed choice per route. Check limits right after authentication and before any work. Throttled requests get 429 with Retry-After and a problem-details body; overload gets 503. Keep outbound limiters below provider quotas and back off with jitter on their 429s. Keep daily or monthly quotas in a durable ledger. List exemptions in config with owners. Publish allowed and rejected counts by route and client class, alert on spikes, and test with a fake clock, two instances against one Redis, and a load test.\n```\n" + }, + { + "id": "jvm-container-memory-guardrails", + "title": "JVM Container Memory and GC Guardrails for AKS", + "whenToUse": "Writing or reviewing JVM flags, Dockerfiles, and Kubernetes resources for Java services on AKS — a memory budget where heap plus metaspace, code cache, thread stacks, direct buffers, and native memory fit under the container limit, MaxRAMPercentage instead of -Xmx equal to the limit, memory request equal to limit, the processor count the JVM actually sees (CPU limit or ActiveProcessorCount, no accidental SerialGC), collector choice, ExitOnOutOfMemoryError with access-controlled heap dumps, JVM flags in one place, startup CPU, Native Memory Tracking under load, and working-set and OOMKilled alerts.", + "tools": [ + "copilot", + "claude-code", + "codex" + ], + "tags": [ + "java", + "jvm", + "kubernetes", + "aks", + "memory", + "gc", + "performance" + ], + "formatNotes": "All three. Copilot: path-scoped `.github/instructions/jvm-container.instructions.md` with applyTo for `**/Dockerfile*`, `**/k8s/**`, `**/kustomization.yaml`, `**/deployment*.yaml`, and `**/application*.yml`, or `.github/skills/jvm-container-memory-guardrails/SKILL.md`; one always-on line in `.github/copilot-instructions.md` (\"heap is a percentage of the container limit, never -Xmx equal to it; heap plus non-heap plus headroom must fit\"). Claude Code: `.claude/skills/jvm-container-memory-guardrails/SKILL.md`, plus a CLAUDE.md line naming the JDK, collector, and where JVM flags are set. Codex: a \"JVM in containers\" section in root `AGENTS.md` (or the same SKILL.md under `.agents/skills/`). Pair with aks-hpa-requests-limits-guardrails for requests, limits, and scaling, k8s-probes-graceful-shutdown-guardrails for startup probes, container-image-hardening for the base image, java-virtual-threads-spring-guardrails for thread counts, and spring-boot-micrometer-prometheus-guardrails for JVM metrics. This skill owns how the JVM fits inside its container.", + "body": "# Skill: JVM Container Memory and GC Guardrails for AKS\n\n## Goal\nEvery Java service knows its memory budget: heap plus everything that is not heap fits under the container limit with room to spare, the JVM sees the CPU count you intended, the garbage collector is chosen on purpose, and a Java `OutOfMemoryError` ends the process cleanly with evidence instead of a silent `OOMKilled` (exit code 137) with nothing in the logs.\n\n## When to apply\n- Writing or reviewing the Dockerfile, entrypoint, `JAVA_TOOL_OPTIONS` / `JDK_JAVA_OPTIONS`, or `resources` block for a JVM workload\n- Pods restart with `OOMKilled` while heap graphs look fine\n- Slow startup, failing startup probes, or long GC pauses after a CPU limit change\n- Moving to a new JDK, a new base image, cgroup v2 node pools, virtual threads, or Netty-based clients\n- Someone proposes `-Xmx` equal to the memory limit, or \"just double the limit\"\n\n## Know what the limit covers\nThe kernel enforces the memory limit on the whole process (resident set plus page cache charged to the cgroup), not on the heap. The JVM only sizes the heap from the limit. Budget the rest explicitly:\n\n```text\ncontainer limit >= heap max\n + metaspace (classes; Spring apps often 100-250 MiB)\n + code cache (JIT; up to ReservedCodeCacheSize, 240 MiB by default)\n + thread stacks (platform threads x committed stack)\n + direct / NIO buffers (Netty, gRPC, HTTP clients; defaults to heap max)\n + GC structures (G1/ZGC bookkeeping, a few % of heap)\n + native libs, malloc arenas, mapped files\n + headroom (10-20%)\n```\n\nMeasure, then decide. Typical results for a Spring Boot service land the heap at 50-70% of the limit, not 75-90%.\n\n## Baseline rules\n1. **Container-aware JDK only.** Use a current LTS JDK whose cgroup v2 support is in place (AKS Ubuntu 22.04+ and Azure Linux node pools are cgroup v2). Confirm with `java -Xlog:os+container=info -version` inside the image, and `cat /sys/fs/cgroup/memory.max` in the pod.\n2. **Size heap as a share of the limit, not a hard-coded number.** Set `-XX:MaxRAMPercentage` (start near 60-70 and adjust from measurement) so a limit change resizes the heap. Never set `-Xmx` equal to the limit. If you must pin `-Xmx`, also pin the limit next to it and review them together.\n3. **Memory request equals memory limit.** Memory is not compressible; a request below the limit invites node pressure evictions. Change both together. A fixed heap (`-XX:InitialRAMPercentage` equal to the max) avoids resize churn for latency-sensitive services; the memory is reserved anyway.\n4. **Cap what can grow unbounded.** Set `-XX:MaxDirectMemorySize` when Netty, gRPC, or NIO buffers are in play; set `-XX:MaxMetaspaceSize` only after measuring (a cap below real need turns into `OutOfMemoryError: Metaspace`). Bound thread pools; with platform threads, each thread costs stack memory. Consider `MALLOC_ARENA_MAX=2` for glibc images with many threads after measuring the gain.\n5. **CPU the JVM can see.** Current JDKs derive the processor count from the CPU limit (quota), not from requests or shares. With no CPU limit, the JVM sees every core on the node and sizes GC and common pool threads for it. Either set a CPU limit, or set `-XX:ActiveProcessorCount` to match the CPU you actually give the pod. Below 2 processors or about 1792 MiB of memory the JVM silently picks SerialGC.\n6. **Choose the collector on purpose.** G1 for most request/response services; generational ZGC for large heaps or strict tail-latency goals (it costs more CPU and memory); Parallel for batch throughput; Serial only for tiny single-core tools. Write the choice and the reason in the Dockerfile or manifest comment.\n7. **Fail fast and keep evidence.** `-XX:+ExitOnOutOfMemoryError` so a broken JVM restarts instead of limping. `-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dumps` on a mounted volume sized for the heap (an `emptyDir` with a `sizeLimit` and a collection step, or a persistent volume). Heap dumps contain secrets, tokens, and personal data: restrict access, never upload them to tickets or public storage, and delete them after analysis.\n8. **One place for JVM flags.** Put flags in one environment variable (`JDK_JAVA_OPTIONS` for the `java` launcher, or `JAVA_TOOL_OPTIONS` when a buildpack or tool expects it), set in the manifest or image, not split across Dockerfile `CMD`, entrypoint scripts, and Helm values. If you use Paketo buildpacks, its memory calculator sizes the heap from the limit and `BPL_JVM_THREAD_COUNT` / `BPL_JVM_HEAD_ROOM`; tune those instead of adding a competing `-Xmx`.\n9. **Startup needs CPU.** JIT and class loading burn CPU in the first minute. A tight CPU limit throttles startup and fails probes. Use a `startupProbe` sized for the measured startup time, and consider Class Data Sharing or the JDK AOT cache (Spring Boot supports building a CDS archive) before raising limits.\n10. **Measure under load, not at idle.** Enable `-XX:NativeMemoryTracking=summary` in a load test (it costs a few percent, keep it off by default in production), take `jcmd 1 VM.native_memory baseline`, drive peak traffic, then `jcmd 1 VM.native_memory summary.diff scale=MB`. Compare NMT's committed total to the pod working set; the gap is native memory NMT cannot see, and it belongs in the budget.\n11. **Watch the right graphs.** Container: `container_memory_working_set_bytes` against the limit, and the last termination reason (`OOMKilled`). JVM (Micrometer): `jvm_memory_used_bytes` by area and pool, `jvm_buffer_memory_used_bytes`, `jvm_threads_live_threads`, `jvm_gc_pause_seconds`, and GC overhead. Alert on working set above 90% of the limit and on any OOMKilled restart, not only on heap usage.\n12. **Do not scale on JVM memory.** A JVM keeps heap it has grown into, so memory-based autoscaling rarely scales down. Scale on CPU, request rate, or queue depth.\n\n## Copy-ready baseline (Deployment snippet)\n```yaml\ncontainers:\n - name: app\n image: myregistry.azurecr.io/orders-api:1.4.2\n # ActiveProcessorCount matches the CPU limit; keep it if the CPU limit is removed\n env:\n - name: JDK_JAVA_OPTIONS\n value: >-\n -XX:MaxRAMPercentage=65\n -XX:InitialRAMPercentage=65\n -XX:MaxDirectMemorySize=192m\n -XX:+UseG1GC\n -XX:ActiveProcessorCount=2\n -XX:+ExitOnOutOfMemoryError\n -XX:+HeapDumpOnOutOfMemoryError\n -XX:HeapDumpPath=/dumps\n -Xlog:gc*:stdout:time,uptime,level,tags\n resources:\n requests: { cpu: \"1\", memory: 1536Mi }\n limits: { cpu: \"2\", memory: 1536Mi } # memory request == limit\n volumeMounts:\n - { name: dumps, mountPath: /dumps }\nvolumes:\n - name: dumps\n emptyDir: { sizeLimit: 2Gi }\n```\n\nCheck what actually took effect:\n```bash\nkubectl exec deploy/orders-api -- java -XX:+PrintFlagsFinal -version | grep -E 'MaxHeapSize|ActiveProcessorCount|Use(G1|Z|Serial|Parallel)GC '\nkubectl exec deploy/orders-api -- jcmd 1 VM.flags\nkubectl exec deploy/orders-api -- jcmd 1 GC.heap_info\n```\n`jcmd 1` assumes the JVM is PID 1 and the image ships `jcmd`. JRE-only or distroless images need `kubectl debug` with a JDK ephemeral container that shares the process namespace.\n\n## Review checklist\n- [ ] Memory budget written down: heap, metaspace, code cache, threads, direct, native, headroom\n- [ ] Heap set as a percentage of the limit (or pinned next to the limit with a comment)\n- [ ] Memory request equals memory limit\n- [ ] Processor count the JVM sees matches the CPU given to the pod; not silently on SerialGC\n- [ ] Collector chosen with a written reason\n- [ ] Direct memory capped where Netty or NIO is used; thread pools bounded\n- [ ] ExitOnOutOfMemoryError on; heap dumps go to a sized, access-controlled volume\n- [ ] JVM flags in one place; buildpack calculator not fighting a manual `-Xmx`\n- [ ] Startup probe sized from measured startup under the CPU limit\n- [ ] NMT measurement from a load test recorded; working-set and OOMKilled alerts exist\n\n## Anti-patterns\n- `-Xmx` equal to (or above) the container memory limit\n- Raising the limit after every OOMKilled without measuring what grew\n- No CPU limit and no `ActiveProcessorCount`, so a 2-core pod sizes GC threads for a 64-core node\n- Memory request far below the limit on a JVM that never gives heap back\n- Heap dumps written to the container's writable layer, or copied to a shared drive\n- Autoscaling a JVM on memory utilization\n- Flags set in three places that override each other\n\n## Copy-ready SKILL.md stub\n```markdown\n---\nname: jvm-container-memory-guardrails\ndescription: Use when writing or reviewing JVM settings, Dockerfiles, or Kubernetes resources for Java services on AKS: memory budget of heap plus metaspace, code cache, threads, direct buffers and native memory under the container limit; MaxRAMPercentage instead of -Xmx equal to the limit; memory request equal to limit; CPU count the JVM sees and ActiveProcessorCount; collector choice; ExitOnOutOfMemoryError and access-controlled heap dumps; one place for JVM flags; startup CPU; NMT measurement under load; working-set and OOMKilled alerts.\n---\n\nFollow JVM Container Memory and GC Guardrails: write a memory budget where heap plus metaspace, code cache, thread stacks, direct buffers, native memory, and 10-20% headroom fit under the container limit. Size heap with MaxRAMPercentage (start near 60-70, adjust from measurement), never -Xmx equal to the limit. Set memory request equal to the limit. Make the processor count the JVM sees match the pod's CPU (CPU limit or ActiveProcessorCount) and avoid accidental SerialGC. Choose G1, generational ZGC, or Parallel with a written reason. Cap direct memory where Netty or NIO is used. Turn on ExitOnOutOfMemoryError and write heap dumps to a sized, access-controlled volume. Keep JVM flags in one environment variable. Size the startup probe for startup under the CPU limit. Measure with NMT under load, compare with the pod working set, and alert on working set above 90% of the limit and on OOMKilled restarts. Do not autoscale on JVM memory.\n```\n" } ] }