[earakely-scale]

agent-env run BUNDLE --set envs/NAME.KEY=VALUE sets a key of one of the bundle's envs for one run, so a bundle can be compared across drivers, base computers or tool sets without being copied or edited. Computer envs get a tools key, so which tools their agents see is a setting like any other.

--set envs/NAME.KEY=VALUE

• --set envs/desktop.driver=cua-driver sets driver in the env in folder envs/desktop, as its env.toml would. The env name is matched against the bundle's env folders, longest first, so a name with dots works.

• VALUE is read as TOML, and a bare word is a string: --set 'envs/desktop.display={ width = 1280, height = 720 }'. KEY can be dotted into a table: --set envs/desktop.computer_settings.timezone=UTC.

• The flag is repeatable, and settings apply in the order given.

• Every setting names its target. There's no bare KEY=VALUE form, so no kind is a default target. Other kinds can come later under their own folder prefix (tasks/…, agents/…) without overlapping this one.

• These are written under ids ending in the settings:

  • the envs it changes;
  • every env or task that names one of them, such as a computer env composing a changed app;
  • every eval that names one of those tasks.

  For example, desktop(desktop.driver cua-driver, desktop.tools standard). A value an id can't hold is shown as 8 hex characters of its sha256.

  Other tasks and evals keep their ids. Names don't change, so --task probe still selects the task, and a reference by name reaches the variant; a reference by id is repointed. A variant composes and reuses images like any env. The plan output says what was renamed: --set envs/desktop.driver=reference: 1 env and 1 task written under ids ending in (desktop.driver reference).

• Refusals before anything is written:

  • a setting that isn't envs/NAME.KEY=VALUE, or whose envs/NAME names no env;
  • a key or value the env's type refuses, with the same message a bad env.toml gets (the plan runs each type's accept_toml), such as "unknown key 'drivr'; a computer env takes …";
  • a run whose selected tasks deploy nothing the settings change.

  A driver and computer that can't compose are refused by the composer's gate as the run writes the env, before any task runs. A dry run writes nothing, so it doesn't catch that; that was already true of any new computer env.

The computer env's tools

tools = "standard" | "driver" | "both" gives the env's agents (the default role) one of:

• standard: AgentEnv's computer_* tools alone;
• driver: the driver's own tools alone;
• both: everything it serves. This is the default and changes nothing.

ComputerEnv.deploy applies it through the driver's tool-access extension, the same one modify_env_tool_access uses. It runs inside deploy_through_provider, through a new then hook, so a driver that can't take it fails the deploy and the computer is torn down. A reset applies it again, since rebooting the computer drops the rules. A task's own tool-access steps run after it and win. The key is stored only when set, so existing env documents don't change.

The drivers and computers READMEs give the --set examples and the tools values.

Also: exporting a disk on a machine of another arch

exported_disk, which carries a computer's disk to a Modal VM, ran docker create on the disk's image without naming a platform. The image's index holds the disk's own platform alone (linux/amd64), so on an arm64 machine whose Docker didn't already have the image, the create failed with "no matching manifest for linux/arm64/v8". The new Modal VM test found this. The export now creates for the disk's own platform, which the Modal VM provider declares in disk_image_platforms.

Not in this PR

• Each setting takes one value. A matrix of values (one variant per value), --repeat, and running the combinations that work while refusing the rest come with M3.3. Changing several envs, such as every computer env of a bundle, takes one --set each for now.
• Only envs the bundle holds change; a store env named by id stays as it is.
• The website docs' computer env key list needs tools. That's a follow-up in the website repo.

Verified

• Unit tests:

  • tst/unit/bundle/variants_test.py: renaming and what stays, repointing by id, TOML and bare values, dotted keys, an app change renaming the computer env, every refusal (form, missing env, the env type's own key and value checks, nothing selected), a materialized variant, and the CLI dry run.
  • test_computer_env.py: tools at deploy and again after a reset for each value, a refused value, and a driver without tool access failing the deploy and tearing the computer down.
  • installed_test's no-BUNDLE usage test now covers --set.
  • The full unit suite on Linux (c6i.4xlarge), rebased on main 41f0e09: 13,518 passed, 8 skipped.

• Integration tests, tst/integration/computer/set_test.py (int_test_slow). Each runs the new set test bundle (the ubuntu-24.04 base with cua-driver) through agent-env run, with a verifier that lists what the agents' role is offered before and after a reset_env:

  • each tools value against the real cua-driver on local Docker: unset (7 computer_* + 66 of the driver's own), standard (exactly the 7) and driver (exactly the 66), before and after the reset;
  • --set envs/desktop.driver=reference: the plan line, the variant's id and its reference driver, the bundle's own env never written, and only computer_* tools offered;
  • tools=standard on a Modal VM: deployed as a disk, the 7 tools kept through a reset in place. It is gated on the remote_sandbox capability.

  Run results: the four local cases passed on the c6i.4xlarge box (2 min 24 s), where the Modal case skipped as agentenv-capability-missing: remote_sandbox. The Modal case passed from an arm64 laptop with the CI Modal keys (2 min 32 s).

• End to end on the same box, local Docker:

  • hf://earakely-scale/cua-driver-bench --task probe --set envs/desktop.driver=reference passed, as probe(desktop.driver reference).
  • The rows below come from the shipped desktop example with a verifier that lists what the agents' role sees.

--set envs/desktop.…	computer_* tools	the driver's own
driver=cua-driver	7	66
driver=cua-driver tools=standard	7	0
driver=cua-driver tools=driver	0	66
driver=cua-driver tools=standard, deploy then reset_env	7	0
tools=standard (reference driver)	14	0

• End to end on modal_vm (laptop as the Modal client, app agentenv-pr510). The disk was composed for amd64 on the laptop, and each env deployed on ModalVmSandboxProvider as the amd64 disk:
  • --set envs/desktop.driver=cua-driver --set envs/desktop.tools=standard: 7 computer_* tools, none of the driver's own.
  • The same with a reset_env before the check: 7 and 0 after the reset.
  • tools=driver: 0 and 66.
  • The probe under driver=cua-driver passed.
  • --set envs/desktop.driver=macos-harness was refused as the env was written: "macos-harness supports no platform ubuntu-24.04 is". A bare --set driver=x was refused before anything was written.

Refs FD-4362

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no blocking issue remains.

Summary

Adds run-only env settings through --set envs/NAME.KEY=VALUE and lets computer envs choose which tools their agents see.

• A run can set env keys and write the affected bundle entries as variants.
• Computer envs can choose whether agents get computer tools, driver tools, or both.
• Modal VM disk exports now request the disk image's platform from Docker.
• Bundle tests share setup helpers instead of keeping duplicate copies.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["Run with --set"] --> B["Parse and check env settings"]
  B --> C["Rename changed envs and dependent entries"]
  C --> D["Repoint references and selections"]
  D --> E["Plan and write the variant"]
  E --> F["Deploy the computer"]
  F --> G["Apply the env's tools choice"]
  G --> H["Run task steps"]
  H --> I["Reset the computer"]
  I --> G

Loading

_{Reviews (6) · Last reviewed commit: "test(run): --set and a computer env's to..." · Reviewed by Greptile}