ci: tidy CI workflows and Python tooling after multi-aspect review

.changes/ci-repository-server-only-deploy.md

@@ -1,11 +1,12 @@
 ---
 kind: chore
+pr: 438
 ---
 Deploy firmware through the repository server only, and drop CI's direct CDN access
 
 ## Release Note
 CI no longer uploads firmware to the CDN or advances the channel pointer files itself. The whole deployment is now three calls to the repository server: start a release, upload every board's artifacts, publish, and the run fails on the first one that does not succeed. Where artifacts are stored, which hostname serves them and the moment a channel starts offering a version are decisions the server makes behind one API call, so CI no longer holds a storage credential and can no longer leave objects, pointer files and the release index disagreeing about what is live.
 
-What remains of that path moved out of YAML into `.github/scripts/repo_server_publish.py` and `.github/scripts/repo_server_token.py`, joining `get-vars.py` and the build scripts. All three now share `.github/scripts/gha.py` for workflow commands, step outputs and the interpreter floor. HTTP and multipart are `requests`, and `get-vars.py` parses tags with the `semver` package instead of a local regex and comparison class. Dependencies are pinned by digest in `.github/scripts/requirements.txt` and installed with `--require-hashes`, so the job holding the publish credential cannot pick up whatever the index happens to serve. The staged release is a resource with a lifetime, and a `try/finally` expresses that directly, where the four YAML steps it replaces had to coordinate through `GITHUB_ENV` and an `always()` cleanup step to get the same guarantee.
+What remains of that path moved out of YAML into `.github/scripts/repo_server_publish.py` and `.github/scripts/repo_server_token.py`, joining `get_vars.py` and the build scripts. All three now share `.github/scripts/gha.py` for workflow commands, step outputs and the interpreter floor. HTTP and multipart are `requests`, and `get_vars.py` parses tags with the `semver` package instead of a local regex and comparison class. Dependencies are pinned by digest in `.github/scripts/requirements.txt` and installed with `--require-hashes`, so the job holding the publish credential cannot pick up whatever the index happens to serve. The staged release is a resource with a lifetime, and a `try`/`except` that discards on failure expresses that directly, where the four YAML steps it replaces had to coordinate through `GITHUB_ENV` and an `always()` cleanup step to get the same guarantee.
 
-The `cdn-deploy` and `cdn-bump` jobs are gone, along with the `cdn-upload-firmware`, `cdn-upload-version-info`, `cdn-bump-version` and `sftp-mirror` composite actions and the `cdn` environment they read their Bunny credentials from. Publishing is now driven by the deploy gate instead of the `OPENSHOCK_REPO_SERVER_PUBLISH` variable, which is retired: a release tag or a nightly/manual develop deploy publishes, and every other build does the same init-upload-discard dry run as before. The staged-draft-release check that guarded the CDN upload now guards the publish. The GitHub release and the API announcement still run last, in that order, each chained on the publish succeeding.
+The `cdn-deploy` and `cdn-bump` jobs are gone, along with the `cdn-upload-firmware`, `cdn-upload-version-info`, `cdn-bump-version` and `sftp-mirror` composite actions and the `cdn` environment they read their Bunny credentials from. Publishing is now driven by the deploy gate instead of the `OPENSHOCK_REPO_SERVER_PUBLISH` variable, which is retired: a release tag or a nightly/manual develop deploy publishes, and every other build does not upload to the server. The staged-draft-release check that guarded the CDN upload now guards the publish. The GitHub release and the API announcement still run last, in that order, each chained on the publish succeeding.

.github/actions/build-compilationdb/action.yml

@@ -1,44 +1,32 @@
 name: build-compilationdb
 description: >-
   Emit build/OpenShock-Core-V2/compile_commands.json with native ESP-IDF for the
-  canonical OpenShock-Core-V2 config, configuring only or also compiling.
+  canonical OpenShock-Core-V2 config.
 inputs:
   version:
     required: true
     description: 'Current firmware version'
-  mode:
+  target:
     required: false
-    default: configure
+    default: reconfigure
     description: >-
-      configure - CMake only, which writes compile_commands.json.
-      build - also compile, for tools that trace the compiler (CodeQL).
+      The idf.py target: reconfigure runs CMake only, which writes compile_commands.json;
+      app also compiles, for tools that trace the compiler (CodeQL).
 
 runs:
   using: composite
   steps:
     - uses: ./.github/actions/setup-esp-idf
 
-    - name: Validate mode
-      shell: bash
-      run: |
-        set -euo pipefail
-        case "${{ inputs.mode }}" in
-          configure|build) ;;
-          *) echo "::error::mode must be 'configure' or 'build', got '${{ inputs.mode }}'"; exit 1 ;;
-        esac
-
     # ccache stays off: it would hide compiler invocations from CodeQL's tracer.
-    - name: Generate compilation database (OpenShock-Core-V2 / esp32s3)
+    - name: Generate compilation database (OpenShock-Core-V2)
       shell: bash
       run: |
         set -euo pipefail
-        if [ "${{ inputs.mode }}" = "build" ]; then
-          python3 scripts/build.py OpenShock-Core-V2 app
-        else
-          python3 scripts/build.py OpenShock-Core-V2 reconfigure
-        fi
+        python scripts/build.py OpenShock-Core-V2 "$TARGET"
         test -f build/OpenShock-Core-V2/compile_commands.json \
           || { echo "::error::compile_commands.json was not generated"; exit 1; }
       env:
+        TARGET: ${{ inputs.target }}
         OPENSHOCK_FW_VERSION: ${{ inputs.version }}
         OPENSHOCK_FW_GIT_COMMIT: ${{ github.sha }}

.github/actions/build-firmware/action.yml

@@ -5,7 +5,7 @@ description: >-
 inputs:
   variant:
     required: true
-    description: 'Build variant id, as get-vars.py emits it (e.g. esp32s3-6832dbd5)'
+    description: 'Build variant id, as get_vars.py emits it (e.g. esp32s3-6832dbd5)'
   version:
     required: true
     description: 'Current firmware version'
@@ -18,13 +18,21 @@ runs:
 
     # One store per variant and IDF version. Entries are immutable, so the key ends in the run and
     # restore-keys picks the newest store; only develop saves, so every other ref restores develop's.
+    - name: Compute ccache key
+      id: ccache-key
+      shell: bash
+      env:
+        IDF_VERSION: ${{ steps.idf.outputs.version }}
+        VARIANT: ${{ inputs.variant }}
+      run: echo "key=ccache-v2-$RUNNER_OS-$IDF_VERSION-$VARIANT-" >> "$GITHUB_OUTPUT"
+
     - name: Restore ccache
       uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9  # v6.1.0
       with:
         path: ~/.cache/ccache
-        key: ccache-v2-${{ runner.os }}-${{ steps.idf.outputs.version }}-${{ inputs.variant }}-${{ github.run_id }}-${{ github.run_attempt }}
+        key: ${{ steps.ccache-key.outputs.key }}${{ github.run_id }}-${{ github.run_attempt }}
         restore-keys: |
-          ccache-v2-${{ runner.os }}-${{ steps.idf.outputs.version }}-${{ inputs.variant }}-
+          ${{ steps.ccache-key.outputs.key }}
 
     # Code targets only: the static0 image is the staticfs job's.
     - name: Build firmware
@@ -33,8 +41,9 @@ runs:
         set -euo pipefail
         export IDF_CCACHE_ENABLE=1
         ccache --zero-stats >/dev/null 2>&1 || true
-        python3 scripts/build.py --variant "${{ inputs.variant }}" app bootloader partition-table
+        python scripts/build.py --variant "$VARIANT" app bootloader partition-table
       env:
+        VARIANT: ${{ inputs.variant }}
         OPENSHOCK_FW_VERSION: ${{ inputs.version }}
         # Fixed: the build directory is on every compile command line, so it is part of each ccache hash.
         OPENSHOCK_BUILD_DIR: build/firmware
@@ -47,28 +56,37 @@ runs:
     # A near-zero hit rate on an unchanged tree means something per-commit is being force-included.
     - name: ccache statistics
       shell: bash
-      run: ccache --show-stats --verbose || ccache -s
+      env:
+        VARIANT: ${{ inputs.variant }}
+      run: |
+        ccache --show-stats --verbose || ccache -s
+        {
+          echo "### ccache ($VARIANT)"
+          echo '```'
+          ccache --show-stats || true
+          echo '```'
+        } >> "$GITHUB_STEP_SUMMARY"
 
     # develop only: another ref's entry is restorable by that ref alone and would only spend the 10 GB.
     - name: Save ccache
       if: ${{ github.ref == 'refs/heads/develop' }}
       uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9  # v6.1.0
       with:
         path: ~/.cache/ccache
-        key: ccache-v2-${{ runner.os }}-${{ steps.idf.outputs.version }}-${{ inputs.variant }}-${{ github.run_id }}-${{ github.run_attempt }}
+        key: ${{ steps.ccache-key.outputs.key }}${{ github.run_id }}-${{ github.run_attempt }}
 
     # dist/<variant>/ so the package job can merge every variant into one tree; app.bin is the OTA name.
     - name: Stage partition images
       shell: bash
       env:
-        B: build/firmware
-        OUT: dist/${{ inputs.variant }}
+        BUILD_DIR: build/firmware
+        OUT_DIR: dist/${{ inputs.variant }}
       run: |
         set -euo pipefail
-        mkdir -p "$OUT"
-        cp "$B/firmware.bin"                        "$OUT/app.bin"
-        cp "$B/bootloader/bootloader.bin"           "$OUT/bootloader.bin"
-        cp "$B/partition_table/partition-table.bin" "$OUT/partition-table.bin"
+        mkdir -p "$OUT_DIR"
+        cp "$BUILD_DIR/firmware.bin"                        "$OUT_DIR/app.bin"
+        cp "$BUILD_DIR/bootloader/bootloader.bin"           "$OUT_DIR/bootloader.bin"
+        cp "$BUILD_DIR/partition_table/partition-table.bin" "$OUT_DIR/partition-table.bin"
 
     # variant_*, not firmware_*: repo-server-publish downloads firmware_* and must not get unstamped images.
     - name: Upload partition images

.github/actions/build-frontend/action.yml

@@ -15,34 +15,29 @@ runs:
       with:
         node-version-file: ./frontend/package.json
 
-    - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413  # v6.1.0
-      name: Install pnpm
+    - name: Install pnpm
+      uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413  # v6.1.0
       with:
-          cache: true
-          cache_dependency_path: ./frontend/pnpm-lock.yaml
-          package_json_file: ./frontend/package.json
-          run_install: |
-            - recursive: true
-              cwd: ./frontend
-              args: [--frozen-lockfile, --strict-peer-dependencies]
+        cache: true
+        cache_dependency_path: ./frontend/pnpm-lock.yaml
+        package_json_file: ./frontend/package.json
+        run_install: |
+          - recursive: true
+            cwd: ./frontend
+            args: [--frozen-lockfile, --strict-peer-dependencies]
 
     - name: Check
       if: ${{ inputs.checks == 'true' }}
       working-directory: ./frontend
       shell: bash
       run: pnpm run check
 
-    # FIXME: Playwirght is not working: Error: browserType.launch: Executable doesn't exist at /home/runner/.cache/ms-playwright/chromium_headless_shell-1161/chrome-linux/headless_shell
-    #    - name: Install playwright
-    #      working-directory: ./frontend
-    #      shell: bash
-    #      run: pnpx playwright install --with-deps
-
+    # TODO: run e2e tests once playwright installs on the runner
     - name: Test
       if: ${{ inputs.checks == 'true' }}
       working-directory: ./frontend
       shell: bash
-      run: pnpm test:unit # TODO: Change back to pnpm test when playwright is working
+      run: pnpm test:unit
 
     - name: Build
       working-directory: ./frontend

.github/actions/build-staticfs/action.yml

@@ -8,13 +8,12 @@ runs:
   steps:
     - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97  # v7.0.0
       with:
-        python-version-file: .github/scripts/.python-version
+        python-version-file: .python-version
         cache: 'pip'
 
-    # gen_staticfs.py needs only littlefs-python; the rest of requirements.txt is for the merge.
     - name: Install Python dependencies
       shell: bash
-      run: pip install "$(grep -E '^littlefs-python==' requirements.txt)"
+      run: pip install --require-hashes -r requirements.txt
 
     # Sized from partitions/ota_4mb.csv, the table every board uses.
     - name: Build static filesystem image

.github/actions/package-firmware/action.yml

@@ -15,12 +15,12 @@ runs:
   steps:
     - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97  # v7.0.0
       with:
-        python-version-file: .github/scripts/.python-version
+        python-version-file: .python-version
         cache: 'pip'
 
     - name: Install Python dependencies
       shell: bash
-      run: pip install -r requirements.txt
+      run: pip install --require-hashes -r requirements.txt
 
     # Each variant artifact holds one <variant-id>/ directory, so merging them keeps variants apart.
     - name: Download variant builds

.github/actions/repo-server-publish/action.yml

@@ -1,13 +1,15 @@
 name: repo-server-publish
 description: >-
   Drives one phase of a firmware release on the repository server.
-  `init` opens the release and returns its id; `publish` and `dry-run` upload every board's artifacts and then promote or discard; `abort` closes a release whose build never got that far.
-  Nothing before the ending differs between `publish` and `dry-run`, so a dry run exercises the whole publish path.
+  `init` opens the release and returns its id; `publish` uploads every board's artifacts and promotes it; `abort` closes a release whose build never got that far.
 
 outputs:
   release-id:
-    description: 'The staged release id, set by `init` and passed to a later `publish`, `dry-run` or `abort`.'
+    description: 'The staged release id, set by `init` and passed to a later `publish` or `abort`.'
     value: ${{ steps.run.outputs.release-id }}
+  outcome-unknown:
+    description: "'true' when `publish` got no response, so the release may be live and must not be aborted."
+    value: ${{ steps.run.outputs.outcome-unknown }}
 
 inputs:
   server-url:
@@ -23,22 +25,21 @@ inputs:
     description: 'Release channel: stable, beta or develop. Required by `init`.'
     required: false
   boards:
-    description: 'Newline-separated boards to publish. Required by `init`, `publish` and `dry-run`.'
+    description: 'Newline-separated boards to publish. Required by `init` and `publish`.'
     required: false
   release-id:
-    description: 'Release opened by an earlier `init`. Required by `publish`, `dry-run` and `abort`.'
+    description: 'Release opened by an earlier `init`. Required by `publish` and `abort`.'
     required: false
   mode:
-    description: '`init` opens the release; `publish` promotes it; `dry-run` aborts it, leaving nothing staged; `abort` closes one the build never reached.'
-    required: false
-    default: 'dry-run'
+    description: '`init` opens the release; `publish` uploads and promotes it; `abort` closes one the build never reached.'
+    required: true
 
 runs:
   using: composite
   steps:
 
     - name: Download firmware artifacts
-      if: ${{ inputs.mode == 'publish' || inputs.mode == 'dry-run' }}
+      if: ${{ inputs.mode == 'publish' }}
       uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333  # v8.0.2
       with:
         path: artifacts
@@ -56,5 +57,4 @@ runs:
         BOARDS: ${{ inputs.boards }}
         RELEASE_ID: ${{ inputs.release-id }}
         MODE: ${{ inputs.mode }}
-        ARTIFACTS_DIR: artifacts
       run: python .github/scripts/repo_server_publish.py

.github/actions/repo-server-token/action.yml

@@ -16,16 +16,7 @@ outputs:
 runs:
   using: composite
   steps:
-    - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97  # v7.0.0
-      with:
-        python-version-file: .github/scripts/.python-version
-        cache: 'pip'
-        cache-dependency-path: .github/scripts/requirements.txt
-
-    # Same pinned set as repo-server-publish; pip no-ops when the job already has them.
-    - name: Install script dependencies
-      shell: bash
-      run: pip install --require-hashes -r .github/scripts/requirements.txt
+    - uses: ./.github/actions/setup-ci-scripts
 
     - id: mint
       shell: bash

.github/actions/setup-ci-scripts/action.yml

@@ -0,0 +1,15 @@
+name: setup-ci-scripts
+description: Installs Python and the hash-pinned dependencies of the scripts in .github/scripts.
+
+runs:
+  using: composite
+  steps:
+    - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97  # v7.0.0
+      with:
+        python-version-file: .python-version
+        cache: 'pip'
+        cache-dependency-path: .github/scripts/requirements.txt
+
+    - name: Install script dependencies
+      shell: bash
+      run: pip install --require-hashes -r .github/scripts/requirements.txt