[Vicente Ferrara (vicentefb)]

TRL's GRPOTrainer trains Qwen2.5-0.5B on MBPP on a CPU host, every completion runs inside a gVisor actor, and the reward is the share of the task's asserts that pass. Only the sandbox calls changed; the module docstring maps each one to its agent-sandbox counterpart.

What the example adds: a smoke mode that checks the wiring without loading the model; a JSONL record of every create, boot, eval, suspend, wake and heal with wall time, plus report.py for p50/p95/max per event and per step; a first-request retry on wake, since Substrate wakes on the first request rather than on an explicit resume; a working-set knob (random-byte ballast per actor) for suspend and wake cost by size; and a multi-endpoint router that splits the reward fleet across clusters.

examples/grpo/README.md has the prerequisites, the run ladder, what to record, and the measured results: ten-step runs on a 4-node cluster and on a 139-node cluster with ac5c5ee0 Substrate, 560 evaluations with no state lost, and the parking cost per step by working set, with a four-worker rerun that separates the placement effect from the build effect.

[jessicaochen]

I recommend clarifying that this is an RL Loop where all the pieces are in one python process on a CPU machine.

Usually when I hear Real RL Loop, I do expect disaggrated and accelerator usage. Timing & load from a single python process on CPU machine probably does not look like what a classic RL loop would produce.

[Jaana Dogan (rakyll)]

This example doesn't belong to this repo, see #109.