Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions skills/bounding-heavy-commands/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,11 @@ A check prints its lines in a fixed form too:
```
bounded-run: check slots=<n> slot=<n>M free-slots=<n> free=<n>M unused=<n>M cpus=<n> cpus-used=<n> cpus-headroom=<n> cpus-free=<n> line=<free|busy> fits=<n>M
bounded-run: check held unit=<name> budget=<n>M used=<n>M cpus=<n> age=<n>s dir=<directory> command=<command>
bounded-run: check waiting unit=<name> for=memory budget=<n>M slots=<n> age=<n>s dir=<directory> command=<command>
bounded-run: check budget=<n>M slots=<n> headroom=<n>M starts=<yes|no>
```

The first line gives the slots of the queue and their size, the slots that are free now, the free memory, the unused budgets of the running commands, the processors of the machine, the processors that the running bounded commands can use together (`cpus-used`), the processors that the check keeps for the programs outside the queue (`cpus-headroom`, a tenth of `cpus`, rounded up), and the processors that are left for a new bounded command (`cpus-free`: `cpus` less `cpus-headroom` and `cpus-used`, and at least 0). `line=busy` means that a command waits in the line. `fits` is the largest budget whose memory, with its own headroom, is free now. It does not count the slots or the line; `starts` counts all three. Each `held` line is one bounded command that runs now: its budget, the memory it uses, the processors it can use, how long it has run, its working directory, and its command. The held lines, `cpus-used`, and `cpus-free` need the hard cap, so without it there are no held lines and `cpus-used` and `cpus-free` are `-`. Each `budget` line is one `--memory`, in the order given: the slots that it needs, the headroom that the wait keeps free beside that budget, and `starts=yes` when a command of that budget would start now. A `-` means that the check cannot read or count the value at that moment.
The first line gives the slots of the queue and their size, the slots that are free now, the free memory, the unused budgets of the running commands, the processors of the machine, the processors that the running bounded commands can use together (`cpus-used`), the processors that the check keeps for the programs outside the queue (`cpus-headroom`, a tenth of `cpus`, rounded up), and the processors that are left for a new bounded command (`cpus-free`: `cpus` less `cpus-headroom` and `cpus-used`, and at least 0). `line=busy` means that a command waits in the line. `fits` is the largest budget whose memory, with its own headroom, is free now. It does not count the slots or the line; `starts` counts all three. Each `held` line is one bounded command that runs now: its budget, the memory it uses, the processors it can use, how long it has run, its working directory, and its command. The held lines, `cpus-used`, and `cpus-free` need the hard cap, so without it there are no held lines and `cpus-used` and `cpus-free` are `-`. Each `waiting` line is one bounded command that had its turn in the queue and now waits for free memory: its budget, the slots that it holds, how long it has waited, its working directory, and its command. It keeps its slots while it waits, so it does not lose its turn to a command behind it in the line. A `waiting` line means that the machine is short of memory, not of slots: the programs outside the queue hold most of the memory, and the command waits for the memory that a running bounded command frees when it ends. Where the hard cap is available, that command has a `held` line. It starts when that memory is free, or exits 75 when no bounded command runs any more. Each `budget` line is one `--memory`, in the order given: the slots that it needs, the headroom that the wait keeps free beside that budget, and `starts=yes` when a command of that budget would start now. A `-` means that the check cannot read or count the value at that moment.

The exit code of the wrapper is the command's exit code. Exit code 125, with a `bounded-run:` line that names a fault and with no line for the budget and the peak, means that the wrapper itself failed (see §When the wrapper cannot run).

Expand Down Expand Up @@ -158,14 +159,14 @@ A command that the project adds later gets its entry when an agent first needs i

## Rules for the wait

A long wait means that the machine is full. It is never a fault to repair: the queue has no state that can go wrong between runs, and an agent that repairs it breaks it. Waiting commands take turns in a line. Only the command at the head of the line tries for slots, so the smaller commands behind it do not take the slots that it waits for. The line has no fixed order, and a head that sees no slot come free for 5 minutes lets the commands behind it try first. So a small command can wait behind a large one, also when a slot is free. Unless an instruction says otherwise (§When an instruction sends a command around the wrapper), the only correct moves are to keep the run in the queue and to tell the user that the machine is full and the command is queued. After a long wait, run the wrapper with `--check`, and in that message name each command of its `held` lines, with its directory and its age, so that the user can see what holds the machine. A subagent tells its dispatcher in its report.
A long wait means that the machine is full. It is never a fault to repair: the queue has no state that can go wrong between runs, and an agent that repairs it breaks it. Waiting commands take turns in a line. Only the command at the head of the line tries for slots, so the smaller commands behind it do not take the slots that it waits for. The line has no fixed order, and a head that sees no slot come free for 5 minutes lets the commands behind it try first. So a small command can wait behind a large one, also when a slot is free. Unless an instruction says otherwise (§When an instruction sends a command around the wrapper), the only correct moves are to keep the run in the queue and to tell the user that the machine is full and the command is queued. After a long wait, run the wrapper with `--check`, and in that message name each command of its `held` and `waiting` lines, with its directory and its age, so that the user can see what holds the machine. When the check has `waiting` lines, also say that the machine is short of memory, not of slots: the programs outside the queue hold most of the memory, and the waiting commands start when a running bounded command ends and frees its memory. A subagent tells its dispatcher in its report.

- **Run a bounded command in the background, with no short timeout around it.** It can wait in the queue for minutes before it starts. Where the harness requires a timeout, use the longest one that it permits. When that timeout ends a run, start the run again through the wrapper, and say in your report that the timeout ended the run and that the code did not fail. A timeout that you calculate from the run time of the command does not count the wait. A timeout that fires during the wait kills a command that never ran, and the result reads like a failure of the code.
- **Plan the time as if the commands run one after the other.** Two bounded commands that start together are safe, because the queue puts them in sequence when the machine cannot hold both. So a parallel start does not always save time, and a promise to the user that depends on it is a guess.
- **Do not lower a budget to start sooner.** A smaller budget does not make the command smaller. It moves the failure inside the command: on Linux as a kill at the hard cap, elsewhere as the swap that the queue exists to prevent. A row at a smaller `--cpus` is not a lower budget: it is a smaller command, measured, and it is the way to start sooner (§The record).
- **Do not run the plain command because the queue is slow.** A slow queue means the machine is full. The plain command is how it freezes. This includes a narrower form of the command: one test file outside the queue is a command that the queue cannot count.
- **Do not set an environment variable whose name starts with `BOUNDED_RUN_`.** The usage text of the wrapper lists variables that change how the queue counts memory, where it keeps its locks, and whether the hard cap applies. They are for a person who tunes the whole machine in a shell profile, and for the tests of the wrapper. A call that sets one does not share the queue with the other sessions, or it runs with no cap. A note in your project memory that recommends one is wrong: delete the note, and tell the user that you did.
- **Do not touch the lock files of the wrapper.** The wrapper keeps them in a directory of its own. Do not look for that directory. Do not delete, move, or edit a file in it, and do not look for the process that holds one of these lock files. The lock files are empty and permanent. They hold no process ID. The kernel holds each lock and releases it at the moment its holder stops, also when the holder crashes, so a stale lock cannot exist. A deleted lock file makes two commands hold one slot. The `held` lines of `--check` are the way to name what holds the queue: do not look further, and never stop a held command, because it belongs to another session or to the user.
- **Do not touch the lock files of the wrapper.** The wrapper keeps them in a directory of its own. Do not look for that directory. Do not delete, move, or edit a file in it, and do not look for the process that holds one of these lock files. The lock files hold no process ID. The kernel holds each lock and releases it at the moment its holder stops, also when the holder crashes, so a stale lock cannot exist. A deleted slot file makes two commands hold one slot, and a deleted wait file hides a command that waits for memory from `--check`. The `held` and `waiting` lines of `--check` are the way to name what holds the queue and what waits for memory: do not look further, and never stop a held or waiting command, because it belongs to another session or to the user.
- **Do not sleep and retry around a collision, not even one time.** A port in use or a tool's lock means two commands wanted one resource. Add the resource to the record as an `--exclusive` name and run through the wrapper. A subagent adds the option to its own command line and names the resource in its report, so that its dispatcher corrects the record. Do not stop the process that holds the resource: it belongs to a different session. When a run with the `--exclusive` name fails on the same collision, the holder did not go through the wrapper or belongs to a different repository: tell the user, or your dispatcher, what holds the resource, and wait for the answer.

### After exit code 75
Expand All @@ -192,7 +193,7 @@ A subagent does not see your project memory. It knows the bounded commands only
On a machine on which the wrapper can run, every dispatch prompt for a subagent that will run a heavy command MUST include:

1. The bounded command line for each heavy command, copied from the record, with the `--cpus` and the budget of its default row. When its entry has more than one row, also the `--cpus` and the budget of each row, and its check line with one `--memory` for each row.
2. This rule, in these words: "Run the bounded commands as written. You can make three changes: the `--cpus` and the `--memory` of a row that this prompt gives for the command, a narrower argument at the end of the line, such as one test file, and one more `--exclusive <name>` option after a collision on a port or a lock, which you name in your report. When this prompt gives a command more than one row, run its check line before each run. When `cpus-free` is `-`, use the largest row with `starts=yes`. Otherwise, use the largest row with `starts=yes` whose `--cpus` is not more than `cpus-free`, and when no such row fits, use the smallest row with `starts=yes`. When no row has `starts=yes`, use the default row. Do not run the plain form of a bounded command or of a part of it. For a heavy command that is not in this prompt, use the same wrapper with no `--memory` option. If the file of the wrapper does not exist, stop and report that. Set no environment variable whose name starts with `BOUNDED_RUN_`. A command can wait in the queue for minutes before it starts, so run it in the background and do not put a short timeout around it. A long wait means that the machine is full; it is not a fault to repair. Exit code 75 with a `bounded-run: no-memory` line means that the command did not start, and that the same budget gives 75 again: run the largest row of this prompt whose budget is not more than `fits`, or, when no row fits, report the line. Say in your report how long you waited. After a long wait, run the same wrapper with `--check`, and name in your report each command of its `held` lines, with its directory and its age."
2. This rule, in these words: "Run the bounded commands as written. You can make three changes: the `--cpus` and the `--memory` of a row that this prompt gives for the command, a narrower argument at the end of the line, such as one test file, and one more `--exclusive <name>` option after a collision on a port or a lock, which you name in your report. When this prompt gives a command more than one row, run its check line before each run. When `cpus-free` is `-`, use the largest row with `starts=yes`. Otherwise, use the largest row with `starts=yes` whose `--cpus` is not more than `cpus-free`, and when no such row fits, use the smallest row with `starts=yes`. When no row has `starts=yes`, use the default row. Do not run the plain form of a bounded command or of a part of it. For a heavy command that is not in this prompt, use the same wrapper with no `--memory` option. If the file of the wrapper does not exist, stop and report that. Set no environment variable whose name starts with `BOUNDED_RUN_`. A command can wait in the queue for minutes before it starts, so run it in the background and do not put a short timeout around it. A long wait means that the machine is full; it is not a fault to repair. Exit code 75 with a `bounded-run: no-memory` line means that the command did not start, and that the same budget gives 75 again: run the largest row of this prompt whose budget is not more than `fits`, or, when no row fits, report the line. Say in your report how long you waited. After a long wait, run the same wrapper with `--check`, and name in your report each command of its `held` and `waiting` lines, with its directory and its age. A `waiting` line is a command that holds its slots and waits for memory, because the programs outside the queue hold most of it: say in your report that the machine is short of memory, not of slots."
3. Each command that an instruction exempts from the wrapper, in its plain form, in a list of its own, with the instruction that exempts it. The rule of item 2 does not apply to that list. An instruction of the user that covers one run goes to the one subagent that does that run, and to no other.

Do not dispatch fewer subagents to protect the machine. The queue protects it. Subagents that wait in the queue cost time, not memory. On a machine on which the wrapper cannot run, the next section says how to dispatch.
Expand Down
Loading
Loading