perf(windows): fence-waited readbacks, overlapped MF export decode, cheaper screen capture by cesarfrancots · Pull Request #2541 · CapSoftware/Cap

cesarfrancots · 2026-10-14T02:33:12Z

Summary

This PR speeds up the Rust media pipeline on Windows: preview/playback rendering, MP4 export, and studio/instant screen recording. It also adds scripts/capbench.py, the A/B runner used to measure every change against the CI performance fixture (CapSoftware/cap-performance-fixtures). Rendered output is unchanged, and repeated exports are now byte-identical.

Each commit stands alone, so this can be split into smaller PRs if preferred.

Commit	What and why
`fix(rendering): find the bundled DXC from example binaries`	Binaries in `target/<profile>/examples` fell back to FXC. That cost about 13 s of shader compiles per editor open and skewed every example benchmark.
`chore: add capbench`	An A/B runner around the existing binaries: `cap export`, `cap record` + `cap-test validate`, and the playback/decode/startup examples. It records wall time, whole-process-tree CPU time and peak commit through a Windows job object. Variants are interleaved with alternating order, and each variant gets its own fixture copy. It reports the median change with a bootstrap 95% CI (verdicts need ≥5 runs per side) and the SSIM of exports. Also adds `--timer-1ms` to the playback example. Its tests now run in `sync-tests.yml` next to the recording-reliability harness tests.
`fix(wgpu-hal): give each DX12 fence wait its own event`	Backports gfx-rs/wgpu#8273 into the vendored hal. `Device::wait` shared one auto-reset event per device. With concurrent waiters, a wakeup could go to the wrong thread, and the other waiter then slept until the 60 s timeout while holding wgpu-core's fence lock. The desktop app shares one device between the editor, preparing-preview and screenshot renderers.
`perf(rendering): wait on the GPU fence for readbacks`	Readbacks sleep-polled `map_async`, and outside `timeBeginPeriod` every sleep rounds up to the ~15.6 ms Windows timer tick. They now block once on the submission fence (`WaitForSubmissionIndex` in `spawn_blocking`, under the existing timeout).
`perf(rendering): serve Media Foundation frame requests during readahead`	The MF decoder now answers cache hits between samples and slides its readahead window with them, so export's sequential reads overlap decoding. A frame is answered only once the decoder has moved past it. That makes VFR pts collisions (two samples rounding to one frame number) resolve the same way every run. Before, repeated exports differed by a source frame at a few dozen positions.
`chore(export): log NV12 export loop stage timings`	MP4 export logs the existing `CAP_RENDER_LOOP_STATS` counters plus time spent blocked on the encoder, as the render farm already does.
`perf(recording): pool the CPU frames Windows screen capture copies into`	Each captured frame was copied into a freshly allocated frame (8 MB at 1080p), paying page faults and zeroing (~2 ms per frame). Buffers now come from an `AVBufferPool`.
`perf(recording): reuse the Windows cursor image while its handle is unchanged`	The cursor thread rendered, trimmed, PNG-encoded and hashed the cursor on every tick. It now reuses the previous result while `GetCursorInfo` reports the same `HCURSOR`.
`perf(desktop-gpui): drop the export timer-resolution hold`	The 1 ms timer hold existed only for the readback sleep removed above. CLI exports measure the same with and without it (8.62 vs 8.70 s).

Results

Test machine: Ryzen 9 9950X3D, RTX 4070 Ti, Windows 11, fixture 3024x1964 + camera, release codegen. All rows are medians of 5 interleaved reps. Base is origin/main f6164c1 plus the DXC fix and capbench. Every change shown is outside its bootstrap 95% CI.

Benchmark	Base	This PR
Playback pipeline (FFmpeg decode, as the Windows editor uses)	31.4 fps	95.4 fps
Playback pipeline under the editor's 1 ms timer	34.5 fps	94.9 fps
Render + readback p50	24.8 ms	3.9 ms
Pipeline p95	45.8 ms	16.0 ms
Scrub p50 / p95	77.8 / 108.5 ms	56.7 / 76.6 ms
Editor first preview	36.4 ms	14.3 ms
MF random access avg / p95	84.5 / 109.4 ms	67.7 / 87.7 ms
Export 1080p60 wall	11.23 s	8.62 s (−23%)
Export 1080p60 steady rate	193 fps	266 fps

The recording numbers come from an earlier run on main 3d19c5c. The recording code in this PR is identical to what was measured there.

Benchmark	Before	After
Studio, 60 fps	0.309 cores	0.248 cores (−20%), same delivered fps
Instant, 30 fps	0.205 cores	0.167 cores (−18%)
Cursor tick	~250 µs	~0.5 µs

Trade-off: export CPU time goes up (1080p60: 17.1 → 19.6 s, +14%) because decode now overlaps rendering. Most of the increase is NVIDIA driver threads running concurrently. 4K exports are NVENC-bound and unchanged.

Testing

cargo fmt --all.
cargo clippy --lib -D warnings on the touched root-workspace crates, plus the playback example.
- --all-targets reports two lints that already exist on main, in code this PR doesn't touch: crates/export/src/mp4.rs (irrefutable let ... else in a test) and crates/recording/src/capture_pipeline.rs (items after the test module).
cap-desktop-gpui type-checks. Its clippy run (stable 1.95, -D warnings) reports 6 lints, none in editor_export.rs: diagnostics.rs, editor_timeline.rs, editor_edits.rs, editor_sidebar/animated_gradient.rs, editor_window.rs and updates.rs.
Unit tests for cap-rendering, cap-export, cap-recording and scap-ffmpeg on Windows, including:
- New tests for the MF frame-selection rule.
- A new test that pooled frames recycle buffers, including across a resize.
python -B scripts/test-capbench.py.
Two exports of the fixture produce byte-identical files (SHA-256).
On both builds, studio recordings went through cap-test validate, which reports frame count and duration. Instant recordings were measured for CPU only.
Not reproduced locally: the DX12 lost-wakeup stall. The backport is the upstream fix.

Follow-ups found but not done here

The MF decoder now answers requests from several places with slightly different lookup rules. A single request/decode state machine, with the policy extracted behind a fake sample source, would make the readahead and VFR rules unit-testable.
Zero-copy MF frames are not recycled: every frame creates a texture and opens a shared handle. Recycling needs D3D11/D3D12 fence synchronisation.
Frame selection still rounds the decoder's pts to frame numbers. For VFR sources, about a third of exported frames differ from what the player shows at that time. Timestamp-based selection would fix it.
wgpu-core 25 holds the device's fence read lock during a blocking wait. A queue.submit from another renderer on the same device waits until the GPU finishes the waited submission. With the hal fix this is bounded by GPU time.
The benchmark examples print human-readable reports that capbench parses. A --json mode would remove the scraping. capbench now warns when an expected metric is missing.
ScreenFrame::Scaled (downscaled capture) still allocates a fresh frame for every captured frame.
The cursor recorder could memoise HCURSOR → cursor id in its loop and skip the clone and hash too.

🤖 Generated with Claude Code

Confidence Score: 5/5

The latest changes appear safe to merge; no new actionable issues were found.

Summary

This PR speeds up Windows rendering, decoding, and capture, and adds an A/B benchmark runner.

Rendered frames arrive as soon as their GPU work finishes.
Media Foundation serves frame requests while it reads ahead.
Windows screen capture reuses buffers for copied frames.
Windows recording reuses the cursor image while its handle stays the same.
Cap performance runs can be compared across builds with one benchmark runner.
MP4 exports log how long their render and encoder stages take.
Example binaries can find the bundled DXC shader compiler.

_{Reviews (12) · Last reviewed commit: "fix(scripts): refuse binaries inside the..." · Reviewed by Greptile}

Examples live in target/<profile>/examples, next to deps/, but the DXC lookup only stepped up from deps/. Examples fell back to FXC, which took ~13s to compile the shaders on every editor open and skewed every benchmark built as an example.

…marks scripts/capbench.py drives the existing measurement binaries (cap export, cap record + cap-test validate, and the cap-editor playback/decode/startup examples) against the CI performance fixture and records wall time, whole-process-tree CPU time and peak commit (Windows job object), plus per-benchmark metrics parsed from their output. Variants (builds and/or env overrides) are interleaved per repetition with alternating order so drift and thermal state hit both sides equally; `compare` reports the median change with a bootstrap 95% interval and SSIM of the exported files. Single-shot runs on the reference machine varied by ~25%, which is larger than most individual changes. The playback example gains --timer-1ms to measure under the 1ms timer resolution the editor holds while playing.

Backport of gfx-rs/wgpu#8273 (wgpu 27). Device::wait signalled the device's single auto-reset idler event, so when several threads waited on one device a wakeup could reach the wrong waiter and the other one slept until the 60s CLEANUP_WAIT_MS timeout while wgpu-core held the fence lock, blocking every queue.submit on that device. The desktop app shares one device between editor, preparing-preview and screenshot renderers, and readbacks are about to block on the fence, so concurrent waits become routine.

…-polling Preview and export readbacks polled map_async with yield, then sleep(100us), then sleep(1ms). Outside timeBeginPeriod every sleep rounds up to the ~15.6ms Windows timer tick, which pinned render+readback at ~24ms regardless of resolution. Block on the submission's fence in spawn_blocking (device.poll WaitForSubmissionIndex) so the wait ends when the GPU signals. capbench, 5 interleaved reps (Ryzen 9 9950X3D, RTX 4070 Ti, fixture 3024x1964 + camera): playback pipeline 31.3 -> 92.2 fps, render p50 24.2 -> 4.1 ms, p95 46.0 -> 17.0 ms scrub p50/p95 79.6/123.5 -> 63.5/90.2 ms editor first frame 38.7 -> 13.9 ms exports unchanged

After answering a request the MF decoder reads ahead up to 8-30 frames before it looks at its channel again. A sequential reader (export, playback prefetch) therefore waited out the rest of the batch for its next frame, then found the decoder idle while it consumed the cached ones, so decode and render alternated instead of overlapping. Poll for requests between samples, answer cache hits immediately and slide the readahead window with them; misses are handed back to the normal request path. Answering from the cache mid-batch exposed a timing dependence that already existed: the decoders number samples at the recorded fps (59/57/30 on the fixture), so a VFR stream regularly has two samples rounding to the same frame and the later one replaces the first in the cache. Whether a request got the first or the second depended on whether it arrived before the second was decoded, so repeated exports differed by a source frame at a few dozen positions. A frame is now answered only once the decoder has moved past it, which always yields the last sample for that frame; repeated exports are byte-identical. capbench, interleaved, same build with the window slide toggled (n=4): export 1080p60 9.79 -> 8.45 s wall (-14%), 15.7 -> 18.0 s CPU export 720p30 8.11 -> 7.72 s wall (-5%), 13.7 -> 15.2 s CPU and against the previous build (n=5): MF random access 87.7 -> 69.1 ms avg, 119 -> 97 ms p95. The extra CPU is mostly NVIDIA driver threads now running concurrently with rendering; 4K exports are NVENC-bound and unchanged.

…OP_STATS The renderer already accumulates per-stage time (decode wait, render, next-frame decode, join, send) behind CAP_RENDER_LOOP_STATS=1, but only the render farm read it. MP4 exports now log it at completion, together with the time the forwarder spent blocked on the encoder, which is what showed the 1080p export to be decode-bound and 4K to be NVENC-bound.

Every captured frame was copied into a freshly allocated FFmpeg frame (8 MB at 1080p, 33 MB at 4K). Allocations that large come straight from new OS pages on Windows, so each frame also paid the page faults and zeroing, measured at ~2 ms per 1080p frame (~9 ms at 4K) against ~0.45 ms for the copy itself. Take the buffers from an AVBufferPool instead; they return to the pool when the encoder and the duplicate-frame holder drop them. capbench, 10 s recordings of a 60 fps test pattern, 5 interleaved reps (with the cursor change that follows; instant mode has no cursor thread, so its number is this change alone): studio 60 fps 0.309 -> 0.248 cores (-20%), delivered fps unchanged instant 30 fps 0.205 -> 0.167 cores (-18%) peak commit +7 MB (the pooled buffers)

…nchanged The cursor thread rendered the cursor into a DIB, trimmed it, PNG-encoded it and hashed the PNG on every tick (~32 Hz) only to find the same id. That costs ~250 us per tick (~0.8% of a core for the whole recording) against ~0.5 us for returning the previous result when GetCursorInfo reports the same HCURSOR.

Windows exports held a 1 ms timer resolution because the NV12 readback wait slept in 1 ms steps. Readbacks now block on the GPU fence and nothing on the export's per-frame path sleeps: CLI exports (1080p60, 5 interleaved reps) take 8.62 s without the hold and 8.70 s with it, so the process no longer raises its timer resolution for the length of an export.

vercel · 2026-10-14T02:33:20Z

@cesarfrancots is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

…ailure logs `cap-test validate` exits 0 for an invalid recording, so read its `valid` and `errors` and count such runs as failures instead of comparing them. Failure logs were written under the work directory that is deleted after the run; copy them to <out>/failures/ and record that path.

richiemcilroy · 2026-10-14T06:44:29Z

Thanks for the PR. Greptile currently has this at 4/5, and we only merge PRs once Greptile scores the current head 5/5. Please address the remaining Greptile feedback, then comment @greptileai to request a re-review, and repeat until it reaches 5/5.

cesarfrancots · 2026-10-14T14:28:53Z

@greptileai

cesarfrancots · 2026-10-14T14:45:41Z

@richiemcilroy Thanks. The Greptile feedback is addressed and all review threads are resolved. Greptile now scores the current head (610ba39) 5/5.

…peline

cesarfrancots · 2026-10-16T04:55:08Z

@greptileai

… colour matrix

cesarfrancots · 2026-10-16T05:06:25Z

@greptileai

A benchmark that exits 0 without its metrics (decode-benchmark does when it cannot create a decoder) was kept with a warning, and its short wall time could make a broken build look faster. Such runs are now invalid runs, like failed ffprobe or recording validation. The job's CPU and memory figures stay optional because they exist only on Windows, and instant recordings no longer list delivered_fps and captured_ratio, which cap-test validate never reports for them.

cesarfrancots · 2026-10-16T05:14:51Z

@greptileai

…peline

cesarfrancots · 2026-10-16T13:53:01Z

@greptileai

… with failed reads A missing cap-test or ffprobe raised out of the run loop after earlier benchmarks had finished, so nothing was saved; each bench now names the helpers it needs and the startup check finds them before any run. decode-benchmark exits 0 and times only the reads that succeeded, so failed sequential reads, seeks or random reads now make the run invalid.

cesarfrancots · 2026-10-16T14:01:42Z

@greptileai

…peline

cesarfrancots · 2026-10-16T15:21:31Z

@greptileai

…tory cmd_run deletes <out>/work before copying the fixture into it, so a fixture reused from an earlier --keep-work run under the same --out was deleted before it was copied. Overlapping fixture and work paths now stop the run before anything is deleted.

cesarfrancots · 2026-10-16T15:29:11Z

@greptileai

run_once writes each log inside the work directory, which is deleted at the end unless --keep-work, so --keep-logs alone kept nothing. Kept logs are now copied to <out>/logs, like failure logs to <out>/failures.

cesarfrancots · 2026-10-16T15:30:28Z

@greptileai

…peline The readahead path this branch adds to the Media Foundation decoder now maps request times with request_frame, like main's other decoder paths.

cesarfrancots · 2026-10-18T02:02:15Z

@greptileai

richiemcilroy · 2026-10-18T02:09:21Z

Thanks @cesarfrancots! Greptile is at 4/5 here. We only merge once Greptile reaches 5/5, so please address its findings in the summary above (or reply to them if you think one is wrong), then comment @greptileai review to re-run it. If the branch is behind main, please merge or rebase on main first. Once it is 5/5 we will run the full verification suite on it.

…tion test after_submit takes the submission index on this branch; the macOS-only test main added still called it with the queue alone, so cap-rendering's tests did not compile on macOS.

…runs with failed frames A --bin-dir inside <out>/work passed the binary checks and was then deleted with the work directory. playback-pipeline-benchmark exits 0 and times only the frames that decoded and rendered, so runs reporting decode or render failures are now invalid, like decode-benchmark runs with failed reads.

cesarfrancots · 2026-10-18T02:12:45Z

@greptileai

cesarfrancots · 2026-10-18T02:14:06Z

Thanks @richiemcilroy! All of Greptile's findings are addressed and resolved: the macOS surface_completion_wakes_without_device_polling test now passes the submission index (e26f981), and capbench rejects playback runs with failed frames and binaries inside the work directory (e00c11e). The branch also includes the latest main (cc3f995).

@greptileai review

…adback waits PendingReadback::wait and the macOS PendingSurface::wait polled, yielded nine times, then slept 100 µs, which tokio rounds up to a whole millisecond, so a frame whose GPU work had finished could still wait for the next timer tick. cap-test's performance suite showed it as render times split into two bands one tick apart. Both waits now block on the frame's own submission with device.poll(WaitForSubmissionIndex) on a blocking thread. They keep the readback deadlines: each wait gets only the time left under the playback or export deadline, measured with readback_waited, and wgpu's own 60 s wait is retried until the export deadline runs out. PendingSurface::wait takes the submission index from its caller's submit. In the suite, against main: median render time -57.1%, suite CPU -7.1%. Yielding instead of sleeping, as the NV12 export readback does, removed the bands too but used 10.7% more suite CPU than main. The vendored DX12 Device::wait waited on one auto-reset event per device, so concurrent waiters on the shared device could lose a wakeup and hold wgpu-core's fence lock until the 60 s timeout. This backports gfx-rs/wgpu#8273, an event per wait, byte-identical to the hunk in CapSoftware#2541.

cesarfrancots added 9 commits October 13, 2026 19:18

greptile-apps Bot reviewed Oct 14, 2026

View reviewed changes

Comment thread scripts/capbench.py Outdated

Comment thread scripts/capbench.py

cesarfrancots mentioned this pull request Oct 14, 2026

refactor(rendering): drive the Media Foundation decoder from one request state machine #2545

Open

docs(export): note that the loop stats are process-wide

610ba39

cesarfrancots mentioned this pull request Oct 14, 2026

perf(video-decode): pipeline the Media Foundation GPU readback #2583

Open

This was referenced Oct 15, 2026

perf(recording): take the cursor position from the recorder's mouse sample #2621

Merged

perf(recording): reuse one X11 connection and unchanged cursor PNGs #2622

Open

harshav167 mentioned this pull request Oct 16, 2026

perf(rendering): cut Metal command buffers 70% and block on frame submissions #2634

Open

Merge remote-tracking branch 'origin/main' into perf/windows-media-pi…

5fd3a90

…peline

greptile-apps Bot reviewed Oct 16, 2026

View reviewed changes

Comment thread crates/rendering/src/decoder/media_foundation.rs Outdated

fix(rendering): tag readahead Media Foundation frames with the source…

470f90d

… colour matrix

greptile-apps Bot reviewed Oct 16, 2026

View reviewed changes

Comment thread scripts/capbench.py

Merge remote-tracking branch 'origin/main' into perf/windows-media-pi…

f7aa44c

…peline

greptile-apps Bot reviewed Oct 16, 2026

View reviewed changes

Comment thread scripts/capbench.py Outdated

Merge remote-tracking branch 'origin/main' into perf/windows-media-pi…

b95bfa6

…peline

greptile-apps Bot reviewed Oct 16, 2026

View reviewed changes

Comment thread scripts/capbench.py

fix(scripts): keep successful-run logs when --keep-logs is set

03c72d9

run_once writes each log inside the work directory, which is deleted at the end unless --keep-work, so --keep-logs alone kept nothing. Kept logs are now copied to <out>/logs, like failure logs to <out>/failures.

Merge remote-tracking branch 'origin/main' into perf/windows-media-pi…

9afd214

…peline The readahead path this branch adds to the Media Foundation decoder now maps request times with request_frame, like main's other decoder paths.

greptile-apps Bot reviewed Oct 18, 2026

View reviewed changes

Comment thread crates/rendering/src/frame_pipeline.rs

Comment thread scripts/capbench.py

cesarfrancots added 2 commits October 17, 2026 20:11

fix(rendering): pass the submission index in the macOS surface comple…

e26f981

…tion test after_submit takes the submission index on this branch; the macOS-only test main added still called it with the queue alone, so cap-rendering's tests did not compile on macOS.

Conversation

cesarfrancots commented Oct 14, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Results

Testing

Follow-ups found but not done here

Confidence Score: 5/5

Summary

Uh oh!

vercel Bot commented Oct 14, 2026

Uh oh!

Uh oh!

Uh oh!

richiemcilroy commented Oct 14, 2026

Uh oh!

cesarfrancots commented Oct 14, 2026

Uh oh!

cesarfrancots commented Oct 14, 2026

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

cesarfrancots commented Oct 16, 2026

Uh oh!

cesarfrancots commented Oct 18, 2026

Uh oh!

Uh oh!

Uh oh!

richiemcilroy commented Oct 18, 2026

Uh oh!

cesarfrancots commented Oct 18, 2026

Uh oh!

cesarfrancots commented Oct 18, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cesarfrancots commented Oct 14, 2026 •

edited by greptile-apps Bot

Loading