Skip to content

ci: compile, render and package pull requests that change the app - #336

Merged
M4n0x merged 2 commits into
mainfrom
ci/pull-request-release-checks
Oct 26, 2026
Merged

M4n0x merged 2 commits into
mainfrom
ci/pull-request-release-checks

Conversation

@M4n0x

@M4n0x M4n0x commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A pull request ran only a debug build, the tests and commitlint. Build release, Docs images, the packaging scripts and several examples were checked only on main. A pull request could break them, merge, and then fail on main, where a failed run publishes no beta, and a stable release needs a beta.

Fixes #221.

Changes

Pull requests that change the app now run the release jobs. On pull requests, Check revision reads the changed files from the merge commit (git diff HEAD^ HEAD, depth-2 checkout without credentials) and sets two outputs:

Output Set when the pull request changes Jobs it adds
app Sources/, Resources/, Package.swift, Package.resolved, scripts/ (the files scripts/unchanged-app.sh compares) or ci.yml Build release, then Docs images
packaging Resources/, scripts/ or ci.yml Package

Docs-only and tests-only pull requests run what they ran before.

  • Build release runs as before on main. On a pull request it restores main's release cache but saves none of its own (actions/cache/restore + actions/cache/save). The repository's Actions caches total 10.7 GB against the 10 GB limit, with a 250 MB debug cache per pull request. Adding a 128 MB release cache per pull request would push main's caches out sooner.
  • Docs images is unchanged and runs whenever Build release does. Its docs-images artifact shows reviewers the bar, popups and Settings as the pull request draws them.
  • Package (new, pull requests only) unpacks the release binary as the beta does. It runs set-version.sh and STABLEBAR_RELEASE_BUILD=1 ./scripts/package.sh without secrets, so the app is ad-hoc signed. Then it checks the DMG (hdiutil verify), the app's signature (codesign --verify --deep --strict) and the stamped version and build number. Last, it runs the packaged app's --validate Resources/config.json, which loads its bundled themes. Signing and notarization stay in Beta and Release, and no pull request job gets a secret.
  • Test gets a Check script syntax step that parses each scripts/*.sh and examples/**/*.sh with the shell its first line names (zsh -n, sh -n). shellcheck doesn't support zsh, which every script in scripts/ uses.

Tests.

  • everyExampleLoadsBesideTheExamplePackages finds the examples itself, so a new example needs no entry. It installs every package in examples/packages. It then loads every configuration (examples/*/, examples/*.json) and module (through an include, as the guide shows) with ConfigurationLoader, the same call StableBar --validate makes. The installed packages sit beside each configuration, so presentation finds its theme. Newly covered: window-clearance, menu-bar-delay, essentials.json, the sketchybar-app-font module and the system-monitor package. This check is a test rather than a --validate step because Test runs on every pull request and swift test runs it locally.
  • packageSchemaListsTheSamePropertiesAsTheManifestDecoder compares package.schema.json with the coding keys of WidgetPackageManifest and WidgetPackageSetting, as the existing test does for stablebar.schema.json.

CONTRIBUTING.md says when the new jobs run. The Test and Commit messages job names, the Beta and Release jobs and the workflow's name are unchanged.

Cost. Check revision adds a shallow checkout of about 3 s. A pull request that changes Swift code adds Build release (about 3.5 min, in parallel with Test) and then Docs images (about 2 min). One that changes scripts, resources or the workflow also adds Package, in parallel with Docs images. These are macOS runners, which were already queueing: on the last main push, Build release waited 10 minutes for one.

Testing

Local:

  • swift test --filter passes for the two new tests and the two existing schema and examples tests.
  • I broke window-clearance, presentation, the module and package.schema.json on purpose. Each break failed with the example's name, for example window-clearance: bar.keepWindowsClaer: Unknown property.. Then I restored the files.
  • The syntax step passes on the current scripts and fails on a script with a missing fi.
  • An ad-hoc app made by build.sh passes codesign --verify --deep --strict, and --validate Resources/config.json from it prints Valid configuration: 3 widgets in 3 files. set-version.sh on a copy of the plist stamps 0.0.0-pr.1 and 123.

CI on this pull request: this PR changes ci.yml, so every new job ran here (run 40365094013):

  • Check revision found the changes and set both outputs.
  • Build release restored main's release cache, compiled in 277 s, and skipped the cache save as intended.
  • Docs images drew the images in 2 min.
  • Package built StableBar.dmg with create-dmg. hdiutil verify reported the checksum VALID, the signature and the stamped version passed, and the packaged app printed Valid configuration: 3 widgets in 3 files.
  • Test: the new tests and the script syntax step passed. The first attempt failed on the known timing flake commandOutputHasABoundedSize (5.43 s against a 5 s bound, in CommandTests, which this PR doesn't touch), and the rerun of that job passed.

Wall time on this run: Build release took 5.5 min, Docs images followed with 2 min, and Package then waited 5 min for a macOS runner, so the last check finished 12 min after the push. Test alone took 4.5 min.

Not checked: create-dmg locally, because it drives Finder. That runs only here in CI. The Developer ID path is unchanged and needs the secrets.

Merging

Merge commit or rebase merge. The test: and ci: commits release nothing and touch no app files, so main builds no beta for them.

Claude Opus 5.5 via Claude Code

M4n0x added 2 commits October 26, 2026 18:03
…oder

The examples test listed its examples by hand and missed window-clearance,
menu-bar-delay, essentials.json, the sketchybar-app-font module and the
system-monitor package. It now finds them itself: every package installs, and
every configuration and module loads beside the installed packages, as
StableBar --validate loads it.

package.schema.json is now compared with the manifest's and setting's coding
keys, as stablebar.schema.json is with the configuration decoders.

Refs #221
Build release and Docs images ran only on main, so a pull request could
break them, merge, and leave main without a beta, which stable needs.

They now run on pull requests that change Sources, Resources, Package.swift,
Package.resolved, scripts or this workflow. A change to the scripts,
resources or workflow also runs Package, which builds the DMG as the beta
does, ad-hoc signed, and checks its signature, stamped version and starter
configuration. Test parses every script with the shell its first line names.

Pull requests restore main's release cache without saving their own, so they
don't evict main's caches.

Refs #221
@M4n0x
M4n0x requested a review from Seb-GRAF as a code owner October 26, 2026 17:03
@M4n0x
M4n0x merged commit 431b8ff into main Oct 26, 2026
15 of 16 checks passed
@M4n0x
M4n0x deleted the ci/pull-request-release-checks branch October 26, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pull request checks miss breakage in release builds, docs images, scripts and examples

1 participant