Skip to content

feat(computer): a computer reaches the run's websites by their names (FD-4683) - #448

Merged
earakely-scale merged 13 commits into
mainfrom
wloo/website-computer-routing
Nov 5, 2026
Merged

earakely-scale merged 13 commits into
mainfrom
wloo/website-computer-routing

Conversation

@williamloo-scale

@williamloo-scale williamloo-scale commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

The computer side of this change; its gateway side landed first as #505. Rebased onto main without #444–#447.

A deployed computer reaches the run's websites by their own names (https://notes.test/), opt-in.

  • Boot order with site_routing: the boot waits only for the driver to answer, then the sites are routed, the held boot apps launched, the computer readied and its display checked, so a browser computer, whose page driver's screen is its browser's page, boots too; a reset does the same.
  • Name table (computer/utils/sites.py): each site's names claimed once, a site's names over the network allowlist. Live sites are left out until they land (FD-4945).
  • Per-deploy CA, name-constrained to those names, signs a certificate for each site.
  • site_proxy.py, standard library only, runs on the computer's python3: it maps the names to loopback, trusts the CA (system store, Firefox policy, NSS) and proxies 443/80 (and DNS for wildcards) to the site's gateway route with X-Forwarded-Host/-Proto and AgentEnv-Site-Root (feat(gateway): WebSockets on a site's route, a site served at a front's own root, and the provider's headers kept from sites (FD-4683) #505), rewriting redirects and cookies and relaying WebSockets.
  • env_params.site_routing (true, or env ids): deploy_env fills the table from the websites deployed before the computer, read from their cards; once the computer is ready the provider installs the proxy and records the table, the CA and each name's check. A reset routes the new computer under a new CA. Without site_routing nothing changes.

Before merging

  • Fits main's WebsiteBrowserEnvironment (feat(computer): WebsiteBrowserEnvironment, the base class for website browsers #449): this proxy answers on loopback through the hosts file, so a website browser in a computer, which bypasses loopback and has no AGENTENV_SITE_PROXY set there yet, reaches the sites through it. Setting AGENTENV_SITE_PROXY in a computer (M5.5/M5.7) would need a forward proxy, which this isn't.
  • Linux computers only. The proxy has been checked from a Mac against the real Notes site over HTTPS, but not yet inside a deployed Linux computer.

Test

  • CI on this head. Locally: env, task-step and provider units 6,113 passed (two macOS-only disk-launcher tests fail on main too); protocol 1,219 passed; lint clean.
  • End to end on a VM (local Docker, an SDK site at slack.e2e448.test behind a gateway), inside two deployed Linux computers with site_routing:
    • Desktop (ubuntu-24.04, the reference driver, Chromium): the name resolves to loopback; a page, base_url (https://slack.e2e448.test, no prefix), a trailing-slash redirect (307 onto the name), a session cookie (Path=/, no domain), a WebSocket (101 and an echo), port 80 (308 to https) and 50 requests at once all answer through the proxy over TLS the system trusts; the site's data plane and card are 404; Chromium opens the site by its name. The proxy killed is back within 20 s; a gateway restart, a site restart and a paused gateway recover; a reset routes the fresh computer under a new CA, Chromium and the WebSocket working again.
    • A browser computer (agentenv's Chromium alone, its page driver's screen its browser's page): it deploys in 5 s and opens the site by its name, before and after a reset. The run found the two boot fixes here: the boot waited for a ready computer whose browser's launch it held, and the held browser was looked up by the computer's own name.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The changes appear safe to merge; the shared boot path preserves the required order.

Summary

Adds opt-in website routing inside Linux computers, including local names, trusted certificates, and an HTTP/WebSocket proxy.

  • Opted-in computers reach earlier websites by their declared names.
  • Browser computers launch their boot apps after site routing is ready.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Driver answers] --> B{Site routing enabled?}
  B -->|Yes| C[Install names, certificates, and proxy]
  C --> D[Launch held apps]
  D --> E[Wait for computer readiness]
  E --> F[Check display]
  B -->|No: already ready| F
  F --> G[Return deployment or finish reset]
Loading

Reviews (28) · Last reviewed commit: "refactor(computer): one sequence for wha..." · Reviewed by Greptile

@williamloo-scale williamloo-scale changed the title computer: reach the run's websites by their names (FD-4683) feat(computer): a computer reaches the run's websites by their names (FD-4683) Nov 3, 2026
@williamloo-scale
williamloo-scale marked this pull request as ready for review November 3, 2026 00:24
@williamloo-scale
williamloo-scale requested a review from a team as a code owner November 3, 2026 00:24
Comment thread src/agent_env/env/gateway/rest_proxy.py Resolved

@earakely-scale earakely-scale left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've reviewed this against our plan for websites in the computer. Three parts are better than what we'd planned, and we've changed our design docs to follow them:

  • the CA made for each deploy, limited to the run's site names, with its key discarded after signing;
  • the transparent loopback routing;
  • the site table's rules.

The latest commits already fix three things we'd found: Chromium's trust (the CACertificates policy), the check's time budget, and WebSockets through the gateway. Four things are inline. Three smaller ones are below, and are optional.

Smaller things:

  • Leaf keys: RootChannel.write (sites.py:324) sends them base64-encoded in a bash -c argument, through a staging file in the VM host's /tmp. write_secret_file avoids both.
  • site_routing validation: run_env_params (computer/env.py:172) uses the value before ComputerParams validates it. "web-env" becomes one env id per character, and 1 raises a TypeError.
  • Rewrites:
    • a scheme-relative Location: //gw/website/notes/x is passed through unchanged (site_proxy.py:181), so the browser leaves the site's name for the gateway;
    • Set-Cookie: ...; Path = /website/notes/x, with spaces around =, isn't rewritten (site_proxy.py:200).

Comment thread src/agent_env/env/gateway/gateway.py Resolved
williamloo-scale added a commit that referenced this pull request Nov 3, 2026
… traffic/v1 (FD-4683)

This PR's change, rebased as one commit onto #448.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@williamloo-scale
williamloo-scale force-pushed the wloo/website-computer-routing branch from a50d72f to a767a25 Compare November 3, 2026 21:50
@williamloo-scale
williamloo-scale force-pushed the wloo/website-live-replay branch from cb672e5 to 306587d Compare November 3, 2026 21:50
Comment thread src/agent_env/env/gateway/rest_proxy.py Resolved
williamloo-scale added a commit that referenced this pull request Nov 3, 2026
… traffic/v1 (FD-4683)

This PR's change, rebased as one commit onto #448.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@williamloo-scale
williamloo-scale force-pushed the wloo/website-computer-routing branch from a767a25 to d160d31 Compare November 3, 2026 22:52
@williamloo-scale
williamloo-scale force-pushed the wloo/website-live-replay branch from 306587d to 2b782e9 Compare November 3, 2026 22:52
@williamloo-scale
williamloo-scale force-pushed the wloo/website-live-replay branch from 2b782e9 to 0d613b8 Compare November 3, 2026 23:51
@williamloo-scale
williamloo-scale force-pushed the wloo/website-computer-routing branch from d160d31 to 5953d5a Compare November 3, 2026 23:53
williamloo-scale added a commit that referenced this pull request Nov 4, 2026
… traffic/v1 (FD-4683)

This PR's change, rebased as one commit onto #448.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@williamloo-scale
williamloo-scale force-pushed the wloo/website-live-replay branch from 0d613b8 to 9495ed2 Compare November 4, 2026 00:32
@williamloo-scale
williamloo-scale force-pushed the wloo/website-computer-routing branch from 5953d5a to 9f2a5ca Compare November 4, 2026 00:32
@williamloo-scale

williamloo-scale commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

@earakely-scale thanks for the review. Everything you raised is now addressed in #448:

Inline (replied to and resolved on each thread):

  • Boot apps vs routing (55c29836): with site_routing, the boot environment holds each app's boot launch. Readiness counts a held app as ready. Once the sites are routed, the provider launches the held apps through computer-app/v1 launch, a framework-only call, then waits until the computer is ready. Chromium's start URLs and Firefox's policies therefore see the hosts entries, the CA and the proxy already in place. A reset does the same.
  • Proxy as the session user (55c29836): serve drops to nobody where the base has it, else to the session user. A root keep loop restarts it. Its log is root-owned beside the config, opened O_NOFOLLOW with mode 0600, so a planted link is refused.
  • OPTIONS on website routes (55c29836): both /website/<name> routes accept OPTIONS, so a cross-name CORS preflight reaches the site.
  • _ in a site's name: already handled. fallback_name maps _ to - before .localhost, so slack_website gets slack-website.localhost.

The smaller three (faf8e7c2):

  • Leaf keys: the routing config holds each site's leaf key and the provider's upstream headers. It now goes through RootChannel.write_secret, which writes with write_secret_file over the exec's stdin into a staging folder only its owner can open. It is then docker cp'd into the container's root-only /etc/agentenv/sites, and the staged copy is removed. It never appears on a command line or in a shared /tmp path. Tested by test_the_config_holding_the_keys_is_written_over_stdin_into_a_private_staging_folder_never_as_text.

  • site_routing validation: run_env_params refuses anything other than true or a list of env-id strings before using it, so "web-env", 1 and ["web-env", 2] each raise a clear ValueError. Tested by test_a_site_routing_that_is_neither_true_nor_a_list_of_env_ids_is_refused_before_it_is_read.

  • Rewrites:

    • a scheme-relative Location: //gw/website/notes/x is read with the page's scheme and moved onto the site's name, while //other.example/... is left alone;
    • Set-Cookie: ...; Path = /website/notes/x is rewritten whatever the spacing around = or the case of path.

    Covered by test_only_a_location_on_the_sites_route_is_moved_onto_its_name and test_a_cookie_path_on_the_route_is_rooted_however_it_is_spelled.

Each new test fails on the code before its fix.

williamloo-scale added a commit that referenced this pull request Nov 4, 2026
… traffic/v1 (FD-4683)

This PR's change, rebased as one commit onto #448.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@williamloo-scale
williamloo-scale force-pushed the wloo/website-computer-routing branch from f813a59 to faf8e7c Compare November 4, 2026 05:10
@williamloo-scale
williamloo-scale force-pushed the wloo/website-live-replay branch from 04d3f23 to 956163f Compare November 4, 2026 05:10
williamloo-scale added a commit that referenced this pull request Nov 4, 2026
… traffic/v1 (FD-4683)

This PR's change, rebased as one commit onto #448.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@williamloo-scale
williamloo-scale force-pushed the wloo/website-computer-routing branch from faf8e7c to 27a865f Compare November 4, 2026 05:16
…te at its own root, and the provider's headers kept from sites

The gateway side of a computer reaching the run's websites by their names, lifted from #448:

- A WebSocket to /website/<name>/ is relayed to the site, under the HTTP route's rules: refused at the paths the site's card reserves, with the role and forwarded address the gateway writes.
- AgentEnv-Site-Root: a front that serves a site's route at a root of its own, as a computer's site proxy does, gets no X-Forwarded-Prefix.
- The headers the sandbox provider in front of the gateway is sent, its credentials, are taken off every request the gateway forwards (AGENTENV_PROVIDER_REQUEST_HEADERS).
- website/v1 browse_url: where a composer serves a site to a browser.

Co-Authored-By: William Loo <william.loo@scale.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 2 commits November 5, 2026 09:58
…es either

The website browser's CDP answer, its discovery and its WebSocket relay forwarded a request's headers on routes of their own, before the withheld filter; BrowserRoutes now takes the withheld names and drops them on all three.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(FD-4683)

The computer side of #448, on the gateway's site routes:

- env_params.site_routing (true, or env ids): deploy_env fills the computer's sites from the websites the run deployed before it, read from their cards (Env.run_env_params).
- The name table (computer/utils/sites.py): each site's names claimed once, a site's names over the network's allowlist; two sites claiming a name are refused.
- A CA made for the deploy, name-constrained to those names, signs a certificate for each site.
- site_proxy.py, standard library only, runs on the computer's python3: it maps the names to loopback, trusts the CA (system store, Firefox policy, NSS) and proxies 443 and 80 (and DNS for wildcards) to the site's gateway route with X-Forwarded-Host, -Proto and AgentEnv-Site-Root, rewriting redirects and cookies and relaying WebSockets.
- Once the computer is ready the provider installs the proxy and records the table, the CA and each name's check; boot apps launch once the sites are routed; a reset routes the new computer under a new CA.

Live sites are left out until they land (FD-4945).

Co-Authored-By: William Loo <william.loo@scale.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale force-pushed the wloo/website-computer-routing branch from 27a865f to 1f97138 Compare November 5, 2026 17:59
@earakely-scale
earakely-scale changed the base branch from wloo/website-live-replay to website-gateway-site-root November 5, 2026 17:59
@greptile-apps

greptile-apps Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Comments Outside Diff

These findings could not be posted inline.

  • P1 Failed checks leave sites empty packages/agentenv-protocol/src/agentenv_protocol/website/conformance.py:191 ▶

    _data_plane() marks the exported data as restored before calling data/add. If that request fails before writing anything-for example, on a temporary server error-the earlier data/reset has already cleared the site, and the finally block skips its recovery attempt.

    Mark restoration complete only after it succeeds, and preserve the export if recovery also fails.

  • P1 Saved website state gets skipped src/agent_env/env/legacy_protocol.py:89 ▶

    backing_servers() uses the card's display name as environment_name, although loading expects the name in the environment definition. For a service named notes whose card says Notes, it exports state under Notes and adds a second, cardless notes entry. That entry is skipped, while MultiEnv.load_environment_artifact() cannot match Notes and skips the saved state on reload.

    Match listed children to their service routes and keep the definition's name for snapshot artifacts.

…, and is readied after its held apps launch

A deploy routing sites holds the launch of the computer's boot apps until the names, CA and proxy are in place, but its boot waited for the computer to be ready, and a page driver, whose screen is its browser's page, can't be ready before the browser launches. The boot now waits only for the driver to answer (ready(answering=True)), and the computer is readied, its display then checked, once the sites are routed and the held apps launched; a reset does the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 2 commits November 5, 2026 11:06
client_headers is what the gateway may pass on of a connection's headers, minus the hop-by-hop set and the provider's credentials; upstream_headers builds on it, and the CDP routes' discovery and WebSocket relay call it instead of filtering their own copies, so a header withheld once is withheld on every route.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from website-gateway-site-root to main November 5, 2026 19:23
earakely-scale and others added 3 commits November 5, 2026 11:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ders, whoever runs the tests

CodeBuild runs the unit tier as root, which reads a file whatever its mode, so chmod 0 didn't stand for the session user; the read is refused instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… with its server, and the record keeps the ready card

On a VM, a browser computer routing sites failed at deploy: its page driver answers before the browser's server does and lists the browser by its name alone, so launch wasn't on the card the boot read. The deploy now waits until the driver lists each held app with its server (boot.listed), launches them, and keeps the card the ready computer answers on its record, in place of the boot's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 4 commits November 5, 2026 12:05
…d lists, though the computer has its name

A browser alone is a computer named for its app, chromium, and invoke by that name reached the computer's own card, which offers no launch; the deploy now passes the listed child.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… startup limits

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tes the forwarded host

On a boat gateway, the site behind it named the gateway's public host in base_url and redirects: boat's ingress, between the computer's site proxy and the gateway, sets X-Forwarded-Host to its own. The site proxy now names the browser's origin in AgentEnv-Site-Root, and the gateway takes a Site-Root naming an origin as the site's address ahead of X-Forwarded-Host/-Proto; any other value is the root flag as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e the driver answers

_finish_boot routes the sites, launches the held boot apps and checks the display, in that order, for both, so the order lives in one place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 41f0e09 into main Nov 5, 2026
61 of 63 checks passed
@earakely-scale
earakely-scale deleted the wloo/website-computer-routing branch November 5, 2026 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants