[kixelated]

Finishes quest/m0/audio-source-tracks.md.

Problem

Microphones and system audio had no track of their own, so the studio could not record one in its own file or replay it. Cameras, displays, windows, images, and colors already had one (moq-dev/studio#21).

Approach

• publish::Audio (rs/moq-scene/src/publish.rs) encodes a source's sound as Opus into its nested catalog under dev.moq.studio, beside the picture sources. It works like publish::Video: it encodes only while the source is used, which means on program or with its track being read. Leaving use marks a gap. A capture gap re-stamps the next sound, so the track's timeline keeps the gap.
• An audio source is on program while the program mix plays it: it is unmuted and not left out of that mix. Program::on_program (rs/moq-studio/src/engine.rs) adds those sources to the set of sources shown in a scene on air. Only microphones and system audio count as heard. A replay source's track holds its pictures, so its strip on the program mix does not keep it encoding.
• Driver::listen feeds the bus and the encoder from the one capture::Producer (moq-dev/studio#27). It reads each buffer from the capture consumer, after the capture chain and before the strip's gain, mute, and delay. The stall check that consumer did for moq-dev/studio#31 stays in the same loop. If the capture fails, the track ends with that error.
• Isolated recording needed no recorder change: it already records any source listing that has audio renditions. Replay now lets you pick microphones and system audio. A local source replays with or without --connect, since its track is in the local broadcast; only the program's replay needs a relay. Program::source returns a sound-only target. A take with nothing encoded is refused with Take's reason. A slowed-down replay of sound alone is refused, because it would play nothing.
• publishes in rs/moq-scene/src/source.rs now matches every variant explicitly, and its stale doc comment is rewritten.
• Publish::show is renamed Publish::set_on_program, because "shown" no longer covers sources that are heard.

Encoder load: Opus runs on the CPU and holds no GPU encode session. Measured on an Apple M4 in release, encoding 60 s of 48 kHz input through publish::Audio took 617 ms for stereo (1.0% of one core) and 326 ms for mono (0.5%). An unused source costs about 5 ms per minute (0.01%). It adds no GPU copies and does not touch frame time.

Impact

• Microphones and system audio list an Opus rendition in the local catalog's dev.moq.studio sources, by source name. The track stays local and never reaches a relay.
• Start recording with each source too now writes a file for each microphone and system audio as well.
• The Replay panel lists microphones and system audio. Their replays and clips hold sound only, and play only at 100%.
• Replaying or clipping a camera, display, window, microphone, or system audio no longer needs --connect.
• Sources panel: microphones and system audio now show the state of their own track.
• No change to the scene file or flags. README.md documents all of this.
• API: moq_scene::publish::Publish::show is renamed set_on_program. Its only consumer is moq-studio.

Alternatives

• moq-audio's encode::Control (capture-backed, encodes on demand and keeps raw history) can encode only while subscribed or all the time. It cannot express "on program or subscribed", it compiles only with the capture feature (so default-feature tests could not cover it), and it would encode up to 30 s of held history for a recording that subscribes. encode::Producer behind the same use rule as video needs no moq change.
• Skipped a GPU-free engine test of Program::on_program: the engine tests need a GPU. The rule underneath (Sources::plays) and publish::Audio are covered in moq-scene.

Choices made without asking

• The track carries the capture's own layout and rate (mono or stereo), encoded by Config::from_input, rather than the bus's stereo layout.
• Only microphones and system audio count as heard on program. Broadcast and browser sources publish no track, and a replay source's track is its pictures.
• The audio-filters quest now records the tap point (filters go before the source track), and publish-independently restates the "used" rule this quest held.

Follow-ups

None.

Validation

• just ci: passes after merging main (quest: settle keyframes; plan a media file source and stinger transitions #39, feat(studio): a recording picks its mix #41, feat(scene): broadcast sources play in sync on the publisher's timestamps #38).
• cargo clippy --locked -p moq-scene -p moq-studio --all-targets --features moq-scene/capture,moq-studio/capture,moq-studio/openh264 -- -D warnings: clean.
• New test publish::tests::an_audio_encodes_while_used (paused clock via kio::clock). It checks that the rendition is listed before anything encodes, that nothing encodes while unused, that sound encodes on program and while read, that the timestamps land on the program clock, that a capture gap stays on the track, that a replay take holds the encoded span, that a take of nothing is refused, and that the track ends cleanly.

Review

• Reviewer: Codex, gpt-6.1-sol, medium effort (quest review, round 3)
• Reviewed head: d73781b
• Findings: none. Rounds 1 (3595a77) and 2 (c2d45a7) also had none.
• Online (pullfrog):
  • rs/moq-studio/src/replay.rs:82: fixed in c2d45a7. Local sources replay without --connect, and only the program needs a relay.
  • README.md:643: wording suggestion applied in 254d197.

(Written by Claude Opus 5.5)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 43 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 5edc8c02-26af-4502-9289-43277e202a00

Commits

Reviewing files that changed from the base of the PR and between 88423f8 and d73781b.

Files selected for processing: 12

• README.md
• quest/m0/README.md
• quest/m0/audio-filters.md
• quest/m0/audio-source-tracks.md
• quest/m0/publish-independently.md
• quest/m1/compose/README.md
• quest/m1/compose/layout.md
• rs/moq-scene/src/publish.rs
• rs/moq-scene/src/source.rs
• rs/moq-studio/src/engine.rs
• rs/moq-studio/src/replay.rs
• rs/moq-studio/src/ui.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[kixelated]

Automated review by review (OpenAI)

Reviewed head 3595a77883c6b0e9df4e3d7982b138839cd2195f against base 6d1b7242cc32910ad12ac3ec73b972ca3ab94730.

No actionable findings identified in this source review. The direction is sound: the studio-specific on-program/subscriber policy wraps the existing upstream encoder and shared capture rather than adding another capture or codec implementation. I traced capture fanout, unique track allocation, use transitions and discontinuities, failure handling, and the recording/replay consumers, including sound-only replay's speed restriction.

Verification limits: GitHub-only inspection of the diff, surrounding source, pinned moq implementation and tests; no build, tests, hardware capture/playback or performance measurements run. quest context could not be run in this GitHub-only review. The current Check job was still running when checked, so this does not establish CI success or validate the reported benchmarks.

[pullfrog]

Important

The new microphone/system-audio tracks are local, but replay::targets gates every Of::Source on --connect, so replaying one (or saving its clip) fails offline with a message that does not hold for a local source.

Reviewed changes

• Per-source Opus track: Audio in rs/moq-scene/src/publish.rs advertises an Opus rendition from the shared capture input and encodes only while the source is on program (shown or heard) or read by a recording, marking capture gaps and leaving use.
• Heard on program: Program::on_program adds running microphones and system audio that the program mix plays to the set that keeps encoding, alongside shown scene items.
• Recording and replay: source tracks flow through the existing record listing; Program::source now exposes an audio rendition, and Feed::replay restricts sound-only takes to real time.
• Docs and quests: README Audio/Replay/Health sections updated, and the audio-source-tracks quest completed.

Technical details

# Source replay needlessly requires a relay

## Affected sites
- `rs/moq-studio/src/replay.rs:82` — `Some(_) => output()?.source(capture)` short-circuits on `self.output`, which is `Some` only with `--connect`.
- `README.md` (Replay) — "a replay needs `--connect`" generalizes the program's relay requirement to local sources.

## Required outcome
- A local source (camera, microphone, system audio, replay) can be replayed and saved as a clip without a relay: `Program.sources` is a local `Publish` broadcast created in `publish()` regardless of `--connect`, and `Program::source` reads it directly.
- `Of::Program` keeps its existing `output()` gate, since the program's catalog is the published broadcast.

## Suggested approach (optional)
- Use `self.program.source(capture)` for `Of::Source` (the `.with_context(...)` chain already handles `None`) and leave the `output()` closure for `Of::Program`; update the README sentence that generalizes the limitation.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using big-pickle (free) ｜ 𝕏}

[kixelated]

Automated review by review (OpenAI)

Reviewed head c2d45a71d1f0fb32aa3664af8b9962d7e70f96cc against base 8fdb2b2757e0f366de392b0f7e36dcbd3560a3bf, focusing on the changes since my review of 3595a77883c6b0e9df4e3d7982b138839cd2195f and separating the merged base changes.

No new actionable findings. The offline-source replay fix is sound: Engine::targets now reads local source tracks directly from self.program.source(capture), while only Of::Program checks for a relay. The engine creates the local source broadcast and passes its publisher into source context regardless of --connect, so this fixes both replay and clip selection without changing program replay's gate. The README now matches that distinction.

Verification limits: GitHub-only source review of the delta and replay/source initialization paths. No build, tests, hardware capture/playback, or performance measurements run; quest context could not be executed. Author-reported local passes were not independently reproduced. The current Check job was still running when checked.

(Written by review (OpenAI))

[pullfrog]

ℹ️ No critical issues — one minor documentation clarity suggestion inline.

Reviewed changes

• Offline source replay: Engine::targets now reads Program::source for every local source instead of gating on self.output, so cameras, microphones, system audio, and replay sources replay and clip without a relay; only Of::Program still requires --connect.
• Explicit LUT refusal: a Source::Lut target now fails with a "no picture or sound" message instead of falling through to the local-source path.
• Docs: the README and targets doc comment now state that a local source keeps its track without a relay, and only a replay of the program needs --connect.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using big-pickle (free) ｜ 𝕏}