perf(desktop-gpui): rebuild only changed screenshot privacy masks

[flamboh]

Note

🤖 GPT-6 Astra and Claude Opus 5.5 on behalf of Oliver

Problem and change

Dragging one screenshot privacy mask currently rebuilds the filtered images for every mask and drops every previous overlay from the GPUI atlas. This caches completed results per annotation, so moving one mask filters one region and preserves the other masks' Arc<RenderImage> values.

Cache keys include an explicit source-frame revision, frame dimensions, the clipped integer pixel region, mask type, effective level, content rectangle, crop, and padding. Cached entries are matched one-to-one (exact request first, then annotation ID), so duplicate annotation IDs never share or lose an image. Results remain in annotation order. Removed masks are evicted immediately; an atlas image is dropped only when no replacement overlay still references it. Pending work is cancelled on changed requests, including deletion and returning to an already-cached position, and the completion-time refresh remains in place.

While a build is in flight, move/level/type/add/reorder requests no longer cancel it: the build completes, installs (reconciled one-to-one against the current overlays), and the completion-time refresh then builds the latest state, so at most one build runs and at most one latest request waits. Previously every drag tick dropped the in-flight build, which at drag rates could starve installation for up to a second. Removals, delete-all, frame revision/size, crop, padding, content-rect changes and ambiguous duplicate-ID changes still cancel immediately, so a removed mask can never be re-installed by a late build.

The cache matching and clipping logic live in a GPUI-free module in cap-gpui-kernels. The filtering kernel, image quality, and export behavior are unchanged.

In the running editor

On main, dragging a privacy mask with many masks on screen leaves its blur behind: the selection handles move, but the filtered overlay stays at the old position for up to a second, leaving the text under the mask's new position readable. Same 16-mask drag, recorded from the GPUI editor at 120 Hz (the dragged mask moves from the inbox card onto the chat message):

https://gh-file-drop-api-prod-galwoqjslzlnws6s.oliver-boorstein.workers.dev/f/a83638863c41a4ee/before-after-blur-16.mp4

https://gh-file-drop-api-prod-galwoqjslzlnws6s.oliver-boorstein.workers.dev/f/4107f50f7167c51e/before-after-pixelate-16.mp4

Measured in the release GPUI app (M4 Pro, macOS 27.2) with temporary, uncommitted instrumentation: a driver calls the editor's real move/mouse-up handlers for 120 ticks at 120 Hz on a 3840×2160 screenshot with 640×360 masks, 3 runs per configuration. Catch-up is request → first GPUI paint showing overlays at least that new; the max gap is the longest stretch of the drag with no overlay installed.

Masks	Catch-up mean / p95 ms, main → this PR	Longest gap ms, main → this PR	Drag CPU s, main → this PR
blur 4	14.7 / 35.5 → 7.7 / 16.5	102 → 45	1.02 → 0.46
blur 8	517 / 964 → 8.7 / 16.8	1001 → 31	1.79 → 0.47
blur 16	552 / 1000 → 20.3 / 34.4	1002 → 39	3.50 → 0.44
pixelate 16	517 / 963 → 17.1 / 34.4	1001 → 43	2.00 → 0.45
mixed 16	541 / 987 → 19.0 / 34.3	1001 → 39	2.74 → 0.44
blur 1	6.9 / 11.5 → 8.4 / 19.8	25 → 36	0.48 → 0.46
pixelate 1	5.8 / 11.3 → 20.2 / 34.5	31 → 34	0.43 → 0.43

Across all 12 configurations (blur/pixelate/mixed × 1/4/8/16) the longest gap is now ≤ 52 ms versus up to ~1 s on main. The trade-off is at one mask, where there is nothing to cache: a request that arrives mid-build now waits for that build instead of restarting, so mean catch-up rises (e.g. 5.8 → 20.2 ms for pixelate) while p95 stays ≈ 35 ms and the longest gap is unchanged. Identical one-mask configurations varied 8–25 ms between runs, so part of that is scheduling noise.

Also exercised in the running editor: deleting a mask while a build containing it is executing (its region is never re-installed across 117 subsequent paints), delete/undo/redo, crop, padding and delete-all mid-drag, duplicate annotation IDs (never fewer overlays than masks on any paint), atlas retirement (0 of 1,021 dropped images still referenced), and a 600-tick drag holding exactly 16 live mask images. The harness bypasses OS input, hit testing and pointer capture.

CPU microbenchmark

Drag one mask among N on a deterministic 3840×2160 RGBA screenshot: 640×360 masks, level 16, 120 measured drag events after 12 warmup pairs, before/after order alternated. "Before" reproduces the old builder exactly (overlay images and geometry only); every overlay's geometry and bytes are compared against it after every event, outside timing. Numbers are CPU overlay preparation per drag event (filtering, cache matching, allocation, retired-image release); they exclude GPUI scheduling, atlas upload and painting.

Mean ms per drag event, before → after:

Mode	Masks	Apple M4 Pro (macOS, arm64)	AMD Ryzen 5 PRO 5650U (Ubuntu, x86_64)	Filter calls
blur	1	1.58 → 1.59 (1.00×)	4.66 → 4.53 (1.03×)	120 → 120
blur	4	6.52 → 1.64 (3.98×)	19.15 → 4.54 (4.22×)	480 → 120
blur	8	13.16 → 1.67 (7.87×)	40.21 → 4.81 (8.35×)	960 → 120
blur	16	26.28 → 1.69 (15.51×)	81.23 → 4.84 (16.80×)	1920 → 120
pixelate	1	0.81 → 0.80 (1.00×)	3.52 → 3.09 (1.14×)	120 → 120
pixelate	4	3.35 → 0.85 (3.94×)	13.40 → 3.13 (4.29×)	480 → 120
pixelate	8	6.76 → 0.87 (7.75×)	27.54 → 3.22 (8.56×)	960 → 120
pixelate	16	13.72 → 0.89 (15.37×)	54.14 → 3.23 (16.78×)	1920 → 120

Repeat runs on both machines were within a few percent (M4 Pro 16-mask blur 15.5–15.9×; Ryzen 16.7–16.9×). One mask is the control: there is nothing to reuse, so it should not change. Frame replacement invalidates every mask and is not expected to benefit. On the Ryzen, 16 blur masks previously cost ~81 ms of preparation per drag event.

Reproduce

From apps/desktop-gpui (GPUI workspace):

cargo run --release -p cap-gpui-kernels --example screenshot-mask-cache-benchmark

Without the GPUI/FFmpeg toolchain (e.g. on Linux), from the repository root:

cargo +stable run --locked --release \
  --manifest-path apps/desktop-gpui/kernels/benchmarks/mask-cache/Cargo.toml \
  --example screenshot-mask-cache-benchmark

The standalone package compiles the production mask_cache.rs and screenshot-kernel sources and depends only on image, at the version in the desktop lockfile.

Validation

From apps/desktop-gpui on macOS (Rust 1.97.0):

• cargo fmt --all: clean.
• cargo check --all-targets: passes.
• cargo test -p cap-gpui-kernels: 34 passed, 1 ignored.
• cargo test -p cap-desktop-gpui --bin cap-gpui -- screenshot mask: 60 passed, 1 ignored, including a regression test for duplicate IDs + deletion + reordering + retirement against real GPUI images.
• cargo clippy -p cap-gpui-kernels --all-targets -- -D warnings: clean. Workspace cargo clippy --all-targets -- -D warnings reports nothing in files touched here; its only failures are existing lints in untouched files on main (e.g. diagnostics.rs, editor_timeline.rs, updates.rs).
• Linux x86_64: standalone tests (20 passed) and the benchmark above.
• After coalescing: cargo test -p cap-gpui-kernels 42 passed; cargo test -p cap-desktop-gpui --bin cap-gpui -- screenshot mask 61 passed; no clippy diagnostics in touched files.

Cache tests cover coalesce-vs-cancel decisions, moving one of N masks, deletion, reordering, overlaps, duplicate IDs, level/type changes, frame/crop/padding invalidation, fractional/clipped rectangles, and identical requests.

Related work

#2529 optimizes screenshot-export/frame kernels; #2493 targets unannotated exports; #2490 scissors GPU video-mask draws. This change targets the GPUI screenshot editor's per-mask overlay cache.

Implemented with GPT-6 Astra (Codex harness), reviewed by GPT-6.1 Sol, and measured in the running app by GPT-6.1 Sol, and validated with Claude Opus 5.5 (Claude Code) in T3 Code.

Confidence Score: 5/5

The PR appears safe to merge; no actionable defects were found.

Summary

This PR caches completed screenshot masks separately, so changing one mask can reuse the others.

• The screenshot editor rebuilds only mask overlays whose pixels changed.
• Mask edits let one overlay build finish while newer requests wait.

No actionable new issues were found. The previous and current commits have different merge bases with the target commit, so this review covered the full PR diff. No previous review threads were supplied. Checks were based on code inspection; builds and UI checks were not run.

_{Reviews (3) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile}

[vercel]

@flamboh is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[richiemcilroy]

Thanks for the PR. Greptile's 5/5 was for 7a7ab58, and new commits have landed since, so the current head hasn't been scored. We only merge once the current head is 5/5. Please comment @greptileai to request a re-review and get it back to 5/5.

[flamboh]

@greptileai review

[flamboh]

@richiemcilroy greptile 5/5!

thanks for making awesome software

[flamboh]

@greptileai review