Skip to content

Qualify the filter sandbox on Fargate with a maintained harness #24

Closed

Description

@sodre

Parent

#21

What to build

A Fargate qualification harness kept in the repo, and its first passing run. Docker's default profile reproduces Fargate's refusal of namespaces, but not its kernel: Fargate 1.4.0 runs 6.1 with Landlock ABI v2 and Yama scope 1, while CI hosts run newer kernels. The allow-list depends on the kernel, so both test seams have to pass on real Fargate before we ask the reporter on #18 to qualify.

Start from the prototype's CloudFormation stack (fargate.yaml on prototype/18-isolation-probe) and move it out of the throwaway branch as a maintained qualification tool. The Gateway never depends on it. The stack covers:

  • an ECR repository;
  • a small public VPC with one subnet, an internet gateway and no NAT;
  • a cluster and a log group;
  • an execution role named PowerUserPB-<stack>-exec with the arn:aws:iam::aws:policy/PowerUserAccess boundary, as the zeroae-code/ZAEPowerUserAccess SSO role requires;
  • Fargate 1.4.0 task definitions for X86_64 and ARM64.

The prototype builds amd64 images on an arm64 host without emulation by cross-compiling and only copying files in the target stage. Keep that approach or replace it.

This needs our AWS account and a fresh SSO login, which is why it's ready for a human, or for an agent given credentials.

Acceptance criteria

  • The harness (stack template, image build and a run script) lives in the repo outside the product code, with a short README on how to deploy, run and tear it down
  • On Fargate 1.4.0 X86_64, the worker sandbox containment test passes under the filter sandbox
  • On Fargate 1.4.0 X86_64, the runtime test passes under "filter", with the test grant authority and mTLS site inside the task
  • On Fargate 1.4.0 ARM64, the containment test passes
  • On Fargate 1.4.0 ARM64, the runtime test passes
  • The run's output (kernel, Landlock ABI, layers logged, test results) is recorded on this ticket
  • Gateway v0.2.0 cannot start a grant worker on ECS Fargate 1.4.0 #18 gets a comment asking the reporter to qualify, linking the build to try
  • The prototype stack countersign-isolation-probe-18 (us-east-1) is deleted once the harness replaces it

Decided in: #21 (spec); ADR 0001; the probe and research on prototype/18-isolation-probe
Live step at merge: none

Activity

  1. added theissue type on Nov 20, 2026
  2. self-assigned this
    on Nov 20, 2026
  3. sodre commented on Nov 20, 2026

    @sodre
    MemberAuthor

    Qualification run: Fargate 1.4.0, x86-64 and arm64

    qualification/fargate/run.sh run, from feat/24-fargate-harness at 30bedc2 (image revision v0.2.0-10-g30bedc2), in us-east-2 on 2026-11-20. It exited 0, and both tasks exited 0.

    x86-64 arm64
    Platform Fargate Linux 1.4.0 Fargate Linux 1.4.0
    Kernel 6.1.190 x86_64 6.1.190 aarch64
    User namespaces refused refused
    Yama ptrace_scope 1 1
    Layers the Gateway logged non-dumpable, Landlock ABI v2, seccomp allow-list non-dumpable, Landlock ABI v2, seccomp allow-list
    Containment test 25 passed, 0 failed 23 passed, 0 failed
    Runtime test under "filter" sidecar and shared mTLS passed; filter-killed grant worker fenced sidecar and shared mTLS passed; filter-killed grant worker fenced

    The first run, before any fix, failed on both architectures, and not because of the filter sandbox: the Gateway issued each leaf with notBefore at the current second, and curl in the same task, reading a clock up to 10 ms behind on this kernel, rejected a leaf issued just after a second began as not yet valid. The same pull request fixes that in the issuer (certificates now start a minute before they are issued, never before their CA).

    The prototype stack countersign-isolation-probe-18 (us-east-1) is deleted.

    x86-64 task log
    ==== Fargate x86_64
    Platform: Linux: 1.4.0
    Architecture: x86_64
    Image digest: sha256:1cb2f15f97d04d644c372d6169f0751dca95d4d234275636d4d891f97f353432
    Status: STOPPED
    Stopped: Essential container in task exited
    == Host
    Countersign revision: v0.2.0-10-g30bedc2
    Kernel: 6.1.190 x86_64
    uid / gid: 65531 / 65531
    CapEff: <account>0000
    NoNewPrivs: 0
    Seccomp: 2
    Yama ptrace_scope: 1
    LSMs: unreadable
    == The host refuses namespaces
    refused, and the image has no bubblewrap
    == Worker sandbox containment test
    test a_confined_grant_worker_reports_its_layers ... ok
    test without_landlock_it_says_so_and_is_still_confined ... ok
    test a_host_that_denies_landlock_is_reported_as_without_it_and_still_confined ... ok
    test a_layer_the_host_refuses_fails_confinement_by_name ... ok
    test its_stream_arrives_and_its_threads_work ... ok
    test opening_a_file_is_refused ... ok
    test reading_the_supervisors_environment_is_refused ... ok
    test reading_the_supervisors_memory_is_refused ... ok
    test reading_the_other_grant_workers_memory_is_refused ... ok
    test an_unconfined_process_cannot_read_a_grant_workers_memory ... ok
    test the_grant_worker_is_confined_before_it_is_sent_anything ... ok
    test the_grant_worker_starts_on_a_host_without_clone3 ... ok
    test its_threads_work_on_a_host_without_clone3 ... ok
    test starting_another_process_is_refused ... ok
         this host has no clone3: the call fails unconfined too
    test starting_another_process_by_clone3_fails_as_unsupported ... ok
    test having_the_kernel_signal_another_process_is_refused ... ok
    test returning_from_a_signal_handler_works ... ok
    test a_tcp_connection_is_refused ... ok
    test a_udp_datagram_is_refused ... ok
    test running_a_program_is_refused ... ok
    test signalling_another_process_is_refused ... ok
    test signalling_itself_works ... ok
    test a_forbidden_system_call_kills_it_with_sigsys ... ok
    test a_system_call_in_the_x32_range_kills_it ... ok
    test a_system_call_from_a_foreign_architecture_kills_it ... ok
    25 passed, 0 failed
    == Runtime test under the filter sandbox
    Gateway worker sandbox "filter": non-dumpable, Landlock ABI v2, seccomp allow-list
    Gateway connection 679e449d-3e6a-4f6c-9966-808cf527efc5 ended at ingress (None)
    Gateway worker admission unavailable (Some(Other))
    Gateway CONNECT e63a5dda-acfa-403c-b6db-0ecbd6e512d8 unavailable
    sidecar Gateway mTLS passed
    Gateway worker sandbox "filter": non-dumpable, Landlock ABI v2, seccomp allow-list
    Gateway connection fa129015-66fa-4831-a2cf-a9e91dc46e6f ended at ingress (Some(UnexpectedEof))
    Gateway connection 868b1853-a9a9-4cbc-8daa-1357c7865431 ended at ingress (Some(UnexpectedEof))
    Gateway connection 966135fb-505d-42cd-8b82-914c96f81474 ended at ingress (Some(UnexpectedEof))
    Gateway connection 992608b3-9ee3-464a-b3b5-fd06113a5f8c ended at ingress (Some(UnexpectedEof))
    Gateway connection 07cd30e1-96ce-4b52-804b-b2d15770dc10 ended at ingress (None)
    Gateway worker admission unavailable (Some(Other))
    Gateway CONNECT 9bb79d6c-4793-4bb3-b11d-3cbc1a781409 unavailable
    Gateway connection 7ebb5f3a-4229-4faa-993b-5382dc48c401 ended at ingress (Some(UnexpectedEof))
    Gateway connection 5516e16e-e940-4eab-a18b-85a60f8c780f ended at ingress (Some(UnexpectedEof))
    Gateway connection da2924d7-c0c3-430c-a646-dca41cdcc285 ended at fence (Some(Other))
    Gateway connection bae45296-5bc9-443c-a84f-77d0b1a6bcbd ended at ingress (Some(Other))
    shared Gateway mTLS passed
    Gateway worker sandbox "filter": non-dumpable, Landlock ABI v2, seccomp allow-list
    Gateway connection b94ec9c7-3a90-49fc-b04a-8263ab2cca02 ended at ingress (Some(UnexpectedEof))
    Gateway grant alice: grant worker killed by the worker sandbox's filter (SIGSYS); grant fenced
    Gateway worker admission unavailable (Some(Other))
    Gateway CONNECT a20343c6-d486-49ed-a7ef-ba9489800e53 unavailable
    Gateway connection 85dc1166-5a9b-41a5-ae1a-69298a71362a ended at ingress (Some(UnexpectedEof))
    filter-killed grant worker fenced
    == Qualified: the containment test and the runtime test passed under the filter sandbox
    ==== Fargate x86_64 exit code: 0
    
    arm64 task log
    ==== Fargate arm64
    Platform: Linux: 1.4.0
    Architecture: arm64
    Image digest: sha256:a786127d6dba35c9b0d3b05a7eac1842ade19a94b3a67d3f6a0ce3f8301e8130
    Status: STOPPED
    Stopped: Essential container in task exited
    == Host
    Countersign revision: v0.2.0-10-g30bedc2
    Kernel: 6.1.190 aarch64
    uid / gid: 65531 / 65531
    CapEff: <account>0000
    NoNewPrivs: 0
    Seccomp: 2
    Yama ptrace_scope: 1
    LSMs: unreadable
    == The host refuses namespaces
    refused, and the image has no bubblewrap
    == Worker sandbox containment test
    test a_confined_grant_worker_reports_its_layers ... ok
    test without_landlock_it_says_so_and_is_still_confined ... ok
    test a_host_that_denies_landlock_is_reported_as_without_it_and_still_confined ... ok
    test a_layer_the_host_refuses_fails_confinement_by_name ... ok
    test its_stream_arrives_and_its_threads_work ... ok
    test opening_a_file_is_refused ... ok
    test reading_the_supervisors_environment_is_refused ... ok
    test reading_the_supervisors_memory_is_refused ... ok
    test reading_the_other_grant_workers_memory_is_refused ... ok
    test an_unconfined_process_cannot_read_a_grant_workers_memory ... ok
    test the_grant_worker_is_confined_before_it_is_sent_anything ... ok
    test the_grant_worker_starts_on_a_host_without_clone3 ... ok
    test its_threads_work_on_a_host_without_clone3 ... ok
    test starting_another_process_is_refused ... ok
         this host has no clone3: the call fails unconfined too
    test starting_another_process_by_clone3_fails_as_unsupported ... ok
    test having_the_kernel_signal_another_process_is_refused ... ok
    test returning_from_a_signal_handler_works ... ok
    test a_tcp_connection_is_refused ... ok
    test a_udp_datagram_is_refused ... ok
    test running_a_program_is_refused ... ok
    test signalling_another_process_is_refused ... ok
    test signalling_itself_works ... ok
    test a_forbidden_system_call_kills_it_with_sigsys ... ok
    23 passed, 0 failed
    == Runtime test under the filter sandbox
    Gateway worker sandbox "filter": non-dumpable, Landlock ABI v2, seccomp allow-list
    Gateway connection 79147043-2010-42f4-9e68-7f6c6bbbb0c2 ended at ingress (None)
    Gateway worker admission unavailable (Some(Other))
    Gateway CONNECT b6dd641c-f25a-4a2a-9fd0-4ddf19c1b2f8 unavailable
    sidecar Gateway mTLS passed
    Gateway worker sandbox "filter": non-dumpable, Landlock ABI v2, seccomp allow-list
    Gateway connection 2b447474-e8e8-4ec6-9c8b-a9947a469a23 ended at ingress (Some(UnexpectedEof))
    Gateway connection 31d8fe1e-10ee-4a07-ac91-16c70c79822f ended at ingress (Some(UnexpectedEof))
    Gateway connection 2eff8135-d4b1-458c-8421-35d47389b14d ended at ingress (Some(UnexpectedEof))
    Gateway connection 139fb0ac-3787-4565-93da-1f3a0f871aa2 ended at ingress (Some(UnexpectedEof))
    Gateway connection 2ffa103a-dc26-4dc7-9b30-c275e53ddaf2 ended at egress (None)
    Gateway worker admission unavailable (Some(Other))
    Gateway CONNECT 7dc4f837-8fd3-4561-96eb-7060e7a80509 unavailable
    Gateway connection 72eac1f7-e860-4de2-b7a3-68966cfcf413 ended at ingress (Some(UnexpectedEof))
    Gateway connection a3320b04-775e-435b-a1f0-ca9334cef938 ended at ingress (Some(UnexpectedEof))
    Gateway connection 3b6c8ff5-120e-42c8-b79e-1585ea288f21 ended at fence (Some(Other))
    Gateway connection 9bf61e45-bdae-4782-a4c5-cf8a8fe80a52 ended at ingress (Some(Other))
    shared Gateway mTLS passed
    Gateway worker sandbox "filter": non-dumpable, Landlock ABI v2, seccomp allow-list
    Gateway connection e07ac2be-a9da-4e4f-aee1-335c3ae784f6 ended at ingress (Some(UnexpectedEof))
    Gateway grant alice: grant worker killed by the worker sandbox's filter (SIGSYS); grant fenced
    Gateway worker admission unavailable (Some(Other))
    Gateway CONNECT dd940d3e-e5fc-4a64-899f-d7d56b3df39b unavailable
    Gateway connection e72181cb-dd8c-428f-a622-4a2c9340b72c ended at ingress (Some(UnexpectedEof))
    Gateway connection dc03442c-fdee-4500-8588-f5f4d5cb452d ended at ingress (Some(UnexpectedEof))
    filter-killed grant worker fenced
    == Qualified: the containment test and the runtime test passed under the filter sandbox
    ==== Fargate arm64 exit code: 0
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions