Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CODING_STANDARDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,13 @@ Grant the least the function does, scoped by the stack's tag
(`ci-runner:stack`) or its own resources, never `*` on a write action
without a condition. A new condition is checked live: some EC2 calls don't
carry the keys their docs suggest (see `docs/traps.md`).

A role a job can reach, a task role or an instance profile, is **very
restricted**: it names each action it needs, never `*`, grants no IAM action
and no `iam:PassRole`, and writes only to the stack's own resources. Adding one
changes a security control, so it goes to the owner first. Today none exists:
a job finds no AWS credentials, and `test/conventions.test.js` keeps it so.
The runner container shares the instance's network, and a job can start a
container that does too through the Docker socket, so a job reaches the
instance metadata service; an instance profile would hand its credentials to
any job.
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,12 @@ the host's Docker daemon, so a `docker run -v` source is a path on the host.
`/tmp` and the runner's home (`/home/runner`: `$HOME`, and the work directory
`_work` with the checkout and `$RUNNER_TEMP`) are at the same paths on both, so
a bind mount from any of them works as it does on a GitHub-hosted runner,
`mktemp` files and `~/` config included. Both are on the 50 GB root disk. A mount from anywhere else (`/opt`,
`/etc`, `/var`) gets an empty host path. Nothing is pinned: every boot pulls
`mktemp` files and `~/` config included. Both are on the 50 GB root disk. The job shares
the instance's network, as it does for `/tmp` and `$HOME`: its `localhost` is the
instance's, so a port it publishes with `docker run -p` is reachable at `localhost`,
and a server it starts on `127.0.0.1` is reachable from `docker build --network host`,
as on a GitHub-hosted runner. A mount from anywhere else (`/opt`, `/etc`, `/var`) gets
an empty host path. Nothing is pinned: every boot pulls
`:latest`, and the image is rebuilt weekly, for linux/amd64 and linux/arm64,
each built and smoke-tested on its own native runner.

Expand Down
5 changes: 3 additions & 2 deletions docs/superpowers/specs/2026-11-12-ci-runner-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,8 +210,9 @@ IPv4 address.
overrides or `CreateFleet` of type `instant`; whatever launches one instance
with that spread.
- 50 GB gp3 root volume, deleted on termination.
- **IMDSv2 required, hop limit 1**, so the Docker containers a job starts
can't reach the instance metadata service.
- **IMDSv2 required, hop limit 1**: a Docker container a job starts on the
default bridge can't reach the instance metadata service. One started with
`--network host` shares the instance's network, and can.
- `InstanceInitiatedShutdownBehavior: terminate`.
- **Caps:** at most **6 Spot and 2 On-Demand** CI runners at once, across all
repos, counted by tag. It is a soft cap: two launches at the same moment may
Expand Down
2 changes: 1 addition & 1 deletion functions/cloud-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ docker run --rm --user 0 --entrypoint cp -v ${HOME_DIR}:/seed ${image} -a ${HOME
install -d -m 1777 /var/tmp/ci-runner
mount --bind /var/tmp/ci-runner /tmp || exit 1

docker run -d --init --name ci-runner \\
docker run -d --init --name ci-runner --network host \\
-v /var/run/docker.sock:/var/run/docker.sock \\
-v /tmp:/tmp \\
-v ${HOME_DIR}:${HOME_DIR} \\
Expand Down
12 changes: 10 additions & 2 deletions template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -739,7 +739,11 @@ Resources:
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs: { VolumeSize: 50, VolumeType: gp3, DeleteOnTermination: true, Encrypted: true }
# IMDSv2 only, one hop: a job's containers can't reach the metadata service.
# IMDSv2 only, one hop. It still stops a container on another network from
# reaching IMDS. The runner container shares the instance's network, and so
# does any container a job starts with --network host through the Docker
# socket: they reach IMDS, and the user-data (the JIT config) with it. The
# instance has no profile, so IMDS holds no credentials (CODING_STANDARDS.md, IAM).
MetadataOptions:
HttpEndpoint: enabled
HttpTokens: required
Expand All @@ -766,7 +770,11 @@ Resources:
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs: { VolumeSize: 50, VolumeType: gp3, DeleteOnTermination: true, Encrypted: true }
# IMDSv2 only, one hop: a job's containers can't reach the metadata service.
# IMDSv2 only, one hop. It still stops a container on another network from
# reaching IMDS. The runner container shares the instance's network, and so
# does any container a job starts with --network host through the Docker
# socket: they reach IMDS, and the user-data (the JIT config) with it. The
# instance has no profile, so IMDS holds no credentials (CODING_STANDARDS.md, IAM).
MetadataOptions:
HttpEndpoint: enabled
HttpTokens: required
Expand Down
8 changes: 8 additions & 0 deletions test/cloud-config.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,14 @@ test('a job sees /tmp and its home at the same paths as the host docker daemon',
assert.match(out, /ghcr\.io\/zeroae\/ci-runner:latest \.\/run\.sh --jitconfig /);
});

test('the runner container shares the instance network, so a job reaches its localhost', () => {
// A port a job publishes with `docker run -p`, and a server it starts on
// 127.0.0.1 for `docker build --network host`, are on the instance's loopback.
const out = renderCloudConfig({ jitConfig: JIT });
const run = out.slice(out.indexOf('docker run -d'), out.indexOf('./run.sh'));
assert.match(run, / --network host /);
});

test('the host home is seeded from the image before it is mounted over the runner', () => {
// The image keeps the runner itself in /home/runner; an empty host dir
// mounted there would hide run.sh.
Expand Down
13 changes: 13 additions & 0 deletions test/conventions.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,19 @@ test('every IAM role and policy is PowerUserPB-${AWS::StackName}-*, and every ro
assert.match(read('Makefile'), /--capabilities CAPABILITY_NAMED_IAM/);
});

test('no runner has a role a job can reach (CODING_STANDARDS.md, IAM)', () => {
// A job can run a container on the instance's network (the runner's own, or one
// it starts through the socket), so IMDS would hand an instance profile to any
// job; a task role reaches a job the same way. Adding
// one is a security-control change for the owner first, so this fails until it is made.
const runners = resources.filter((r) => ['AWS::ECS::TaskDefinition', 'AWS::EC2::LaunchTemplate'].includes(r.type));
assert.ok(runners.length >= 3, 'the runner task definitions and launch templates');
for (const r of runners) {
assert.doesNotMatch(r.body, /TaskRoleArn|IamInstanceProfile/, `${r.name} gives a job a role (see CODING_STANDARDS.md, IAM)`);
}
assert.ok(!resources.some((r) => r.type === 'AWS::IAM::InstanceProfile'), 'an instance profile (see CODING_STANDARDS.md, IAM)');
});

// Read but deliberately not set by the template: a test or hand-run override.
const OPTIONAL_ENV = new Set(['IDLE_SECONDS']);

Expand Down
Loading