Repository navigation
feat: encode from DMA-BUFs, with GPU scale and convert, and change the bitrate - #3
Merged
Merged
Conversation
This adds the two modules an encoder needs to take frames it did not allocate. They are also the "VPP wrapper" that moq-video's VAAPI quest lists as a prerequisite. `dmabuf::DmaBuf` describes a single-object DRM PRIME buffer by its DRM format, modifier and, where the producer knows it, its YUV color space. It maps the DRM format to the VA-API fourcc, which differs for packed RGB and for planar 4:2:0, and imports the buffer as a surface without copying. The object size comes from `lseek(SEEK_END)` on the descriptor, because iHD refuses a packed RGB import whose size is zero (it works the size out itself for YUV). `DmaBuf::from_prime` turns an export from `Surface::export_prime` back into one. `vpp::Processor` drives `VAEntrypointVideoProc`. A blit stretches the input over the output and converts to the output's format, so one call scales, converts RGB to NV12, converts between YUV color spaces, or re-tiles into the driver's preferred layout. The processor keeps one context per destination size, drops a context after a failed blit, and refuses surfaces from another display, whose ids it would otherwise misread as its own. `color::Color` names the matrix and range on each side of a conversion. `ExportedFrame::from_surface` exports a processed NV12 surface the way the decoder exports its pictures, so a consumer handles both the same way. Tests, which skip without a VA-API device: a retile round trip, and a half-dark, half-bright surface scaled to half size and read back to check it was scaled rather than cropped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`Encoder::encode_dmabuf` takes a frame that a capture or a decoder already holds on the GPU. An NV12 buffer at the encoder's size, a whole number of macroblocks, and in the stream's color space is imported and encoded in place. Anything else importable is blitted through the video processor into the encoder's own surface, scaled to its size and converted to its color space. Examples are packed RGB from a screen capture, YUYV, NV12 at another size or in another space, and a 1080-row buffer. The last case matters because the encoder reads 1088 rows, and a producer that puts the chroma plane right after the last visible row has nothing there to read. The encode body now takes any surface, so both paths and the CPU upload share it. `Encoder::set_bitrate` retunes the rate controller from the next frame. The rate control parameters already went out with every frame, so no IDR is needed. The next frame sets the rate control reset flag, and the coded buffer grows when the new rate could overflow it. Also: - `Config::color` is written into the SPS as the VUI colour description. It defaults to BT.601 up to 576 lines and BT.709 above. - Packed SPS and PPS headers are negotiated through `VAConfigAttribEncPackedHeaders` where the driver offers them. iHD took them without being asked, but a driver that follows the spec would write its own SPS, without our VUI and level. - The level follows the frame size and macroblock rate instead of being fixed at 4. Level 4 was too low for 1440p and 4K streams. - frame_num wraps at MaxFrameNum, and the slice header carries only the low bits of the picture order count. Before, a group longer than 2048 frames depended on the driver masking them. - Where the device has no `VAEntrypointEncSlice` for H.264 Main, the encoder opens the low-power one, which is the only one some Intel parts expose. - `encode_nv12` rejects a short buffer instead of panicking on it. Validated on Intel Meteor Lake (iHD 26.1.5). The tests decode with ffmpeg and skip without it or without a device: - CPU and DMA-BUF input decode at 35 dB luma PSNR or better. - A scaled DMA-BUF decodes at 25 dB or better against a point-sampled reference. - A 1080p buffer is encoded through the blit. - BGRX and a BT.709 NV12 buffer convert into BT.601 and BT.709 streams, with every channel within 6 code values. - The VUI label is checked with ffprobe. - IDRs follow the group length and restart after a forced keyframe. - A refused import leaves the encoder usable. - The entrypoint choice is covered. - A bitrate drop from 2 Mbps to 250 kbps shows up in the picture sizes, with no IDR. The low-power entrypoint itself is untested, since this device exposes the full one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Frando
marked this pull request as ready for review
September 24, 2026 07:10
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Frando
added a commit
to Frando/moq
that referenced
this pull request
Sep 24, 2026
The patch followed the `pr/encode-dmabuf` branch of a personal fork, so a push there would change what `main` builds. It now names the commit the lock already resolved to, which leaves the lock's source line as the only change there. `deny.toml` rejects unknown git sources, so the fork is allowed there until the moq-vaapi release that carries moq-dev/vaapi#3 replaces the patch. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Contributor
|
Landing this as-is. Check passed on 8ae97af. The branch is mergeable with no conflicts, and there is no review feedback to address. CodeRabbit did not review (the repo is under the star threshold). I read the DMA-BUF import, VPP blit, and encoder paths (in-place NV12 vs. blit, bitrate reset, packed VUI, level selection, frame_num/POC wrap) and did not find a blocker. GPU tests skip on CI, which the PR already documents. Enabling squash auto-merge, matching the previous merges on this repo. (Written by Grok) |
kixelated
pushed a commit
to Frando/moq
that referenced
this pull request
Sep 24, 2026
The VAAPI encoder took every frame through `Surface::to_i420`. A PipeWire screen capture or a VA-API decode was therefore downloaded, converted on the CPU, and uploaded again. The encoder's header said its bitstream had never been validated, and it refused bitrate changes. This uses moq-vaapi's new DMA-BUF import and video post-processing (moq-dev/vaapi#3), pinned by git until that is released: - A `Surface::DmaBuf` is encoded on the GPU. NV12 at the encoder's size is imported and encoded in place. Packed RGB, other sizes, and YUV labelled with another color space are blitted through VPP into the encoder's surface in the stream's color space. - A layout (format and modifier) the driver refuses is logged once and goes through the CPU from then on. A failure before the driver sees the buffer, such as a fence timeout, affects only that frame. - `Surface::resize` of a DMA-BUF scales on the GPU through VPP when `vaapi` is on, and hands back an NV12 DMA-BUF the encoder imports in turn. A simulcast publisher resizing one capture into several renditions never downloads it. It falls back to the CPU resize when VPP is missing or refuses the buffer, and it labels colors the way the CPU resize does. - `set_bitrate` retunes the rate controller from the next frame. Before, a VAAPI publisher stopped following the congestion estimate at the first `BitrateUnsupported`. - The stream's VUI carries `Config::color`. The VA-API export wrapping moves from the decoder into `frame/vaapi.rs`, where the decoder, the encoder and the resize share it. Validated on Intel Meteor Lake (iHD 26.1.5). The new tests skip without a VA-API device and decode with openh264. They cover: - CPU frames. - A VA-API decode re-encoded from its DMA-BUFs. - Packed RGB buffers at and above the encoder's size. - A refused buffer encoded and resized through the CPU. - The SPS color description. Each plane decodes within 8 code values on average of the source, or 10 after two generations of coding. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kixelated
pushed a commit
to Frando/moq
that referenced
this pull request
Sep 24, 2026
The patch followed the `pr/encode-dmabuf` branch of a personal fork, so a push there would change what `main` builds. It now names the commit the lock already resolved to, which leaves the lock's source line as the only change there. `deny.toml` rejects unknown git sources, so the fork is allowed there until the moq-vaapi release that carries moq-dev/vaapi#3 replaces the patch. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds what moq-video's VAAPI quest (
quest/m4/video-vaapi.md) lists as missing: DMA-BUF import, a VPP wrapper, and an encoder that takes GPU frames and changes its bitrate. The capabilities come from the VAAPI encoder iroh-live had before moving to moq-video. That encoder was built on cros-codecs; this one is rebuilt on this crate's own encoder.Two commits:
dmabufandvppmodules.DmaBufimports a DRM PRIME buffer as a surface without copying.vpp::Processorblits one surface into another, which scales, converts RGB to NV12, converts between YUV color spaces, or re-tiles.ExportedFrame::from_surfaceexports a processed surface the way the decoder exports its pictures.encode_dmabuf: an aligned NV12 buffer is encoded in place; anything else goes through VPP into the encoder's surface.set_bitrate: no IDR, sets the rate-control reset flag.Config::color.Validated on Intel Meteor Lake (iHD 26.1.5). The new tests decode with ffmpeg and skip without it or without a VA-API device:
On CI without a GPU these tests skip, so they pass without testing anything. The low-power entrypoint is untested, because this device exposes the full one.
The moq-video side is moq-dev/moq#4012, which needs a release of this.
Not in here: an HEVC encoder, and pre-generated bindings (the quest's other two asks).
🤖 Generated with Claude Code