Skip to content

perf(rendering): read recording metadata from container headers - #2498

Merged
richiemcilroy merged 3 commits into
CapSoftware:mainfrom
x1xhlol:devin/1791909220-header-media-probe
Oct 19, 2026
Merged

richiemcilroy merged 3 commits into
CapSoftware:mainfrom
x1xhlol:devin/1791909220-header-media-probe

Conversation

@x1xhlol

@x1xhlol x1xhlol commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Opening a recording in the editor called avformat_find_stream_info on every display, camera and audio file. That call decodes frames to fill in stream info, so for a 3024x1964 H.264 display track it software-decodes video just to learn width, height, fps and duration. MP4 and Ogg headers already contain those values.

This PR opens files with avformat_open_input only and reads metadata from codecpar and the stream/container durations:

// ProjectRecordingsMeta: Video::new / Audio::new
if let Some(meta) = open_headers(path).and_then(|input| Self::from_headers(&input, start_time)) {
    return Ok(meta);
}
// unchanged full probe
let input = ffmpeg::format::input(path)?;
Self::from_input(&input, start_time)
  • from_headers returns None (and we fall back to the existing full probe) when the header is incomplete: zero width/height, missing avg_frame_rate, missing sample rate/channels, or no usable duration.
  • header_duration_secs reproduces FFmpeg's container duration calculation from stream start_time/duration, so header and full-probe durations match.
  • Fragmented/directory inputs keep using open_fragmented_input.
  • AVAssetReaderDecoder::run (macOS) did a second full probe per decoder just to validate dimensions before signalling ready. It now uses header_video_metadata first and keeps the old probe as the fallback.

Tests generate media with ffmpeg-next and check that header metadata matches the full probe for H.264 MP4, Ogg Opus, fragmented AAC .m4a (written with the recorder's frag_keyframe+empty_moov+default_base_moof flags) and WAV. They also check that MPEG-4 video and a fragmented .m4a with no fragments yet both fall back to the full probe, and that a missing file still errors. On the fixture's four .ogg tracks, header and full-probe values match exactly.

Benchmarks

cargo run --release -p cap-editor --example editor-startup-benchmark -- --recording-path <fresh copy of cap-performance-fixtures/reference-recording.cap> --runs 10 --profile-stages. Three rounds, alternating before/after, fresh fixture copy each run. Before = eefa6ef, after = this branch. Times are means of the per-round averages, in ms.

macOS (Apple M4 Pro virtual machine, macOS 26.6.2, 12 vCPU)

Stage Before After
Recordings metadata 53.3 2.1
Create segments 28.1 1.4
Total startup-equivalent 94.3 47.7
Startup p50 85.2 42.6
First preview 105.0 104.3

Editor max RSS went from about 145 to 107 MiB. Layer warmup runs in parallel and used to finish hidden behind the probe, so it now shows up as the remaining startup cost (about 2 ms to 29 ms in that column).

Linux (8 vCPU, llvmpipe)

Stage Before After
Recordings metadata 75.6 1.9
Total startup-equivalent 160.2 89.5
Startup 173.5 104.1
First preview 94.1 90.7

1080p export on the same Mac (export also builds ProjectRecordingsMeta): 7.59 s before, 7.49 s after over 2 alternating runs each, which is within noise. The outputs were identical in shape (1664x1080 H.264, 1850 frames, AAC 1445 frames).

Checked on macOS: cargo fmt --all --check, cargo clippy -p cap-rendering --all-targets -- -D warnings, cargo test -p cap-rendering project_recordings, cargo check -p cap-editor -p cap-export, cargo check in apps/desktop-gpui. The same commands minus the macOS-only path also pass on Linux. Not measured on physical Apple hardware or Windows. Windows takes the same ProjectRecordingsMeta path.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable defects were established.

Summary

Reads recording metadata from container headers before trying the existing full probe.

  • Recording metadata reads usable media headers before probing the full track.
  • macOS video decoders check headers before running a full probe.

No new actionable issues were found. The previous audio-test concern is addressed. Review checks were static; tests and benchmarks were not rerun.

Reviews (3) · Last reviewed commit: "test(rendering): cover Ogg Opus and frag..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

@x1xhlol is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

Comment thread crates/rendering/src/project_recordings.rs
@x1xhlol

x1xhlol commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai review

@richiemcilroy

Copy link
Copy Markdown
Member

Thanks for this! It now conflicts with main after today's merges. Could you merge or rebase onto the latest main? Once it's up to date I'll run the full engine verify and real-capture suites on it and merge if they pass.

@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1791909220-header-media-probe branch from 3e2d4e1 to b1e82e9 Compare October 18, 2026 18:06
@x1xhlol

x1xhlol commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

should be done

@richiemcilroy

Copy link
Copy Markdown
Member

@greptileai review

@richiemcilroy richiemcilroy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: exports are frame- and audio-identical to the base on 1080p/4K fixtures (-1.1 to -2.0% instructions), and header metadata matches the fully probed metadata (size, fps, duration, sample rate, channels) on all 435 media files of a real recordings library. CI green, Greptile 5/5. Thanks!

@richiemcilroy
richiemcilroy merged commit b996752 into CapSoftware:main Oct 19, 2026
44 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants