Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 29 additions & 20 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,26 +18,35 @@ because it turns other people's test suites red.

- **Films: the `webm` format.** A WebM video with an AV1 picture and no
sound, at the exact size you ask for, like every other format. Its length
is a setting of its own, independent of the size: an hour at thirty frames
a second fits in less than a megabyte (977 128 B at the smallest), because
a film that shows one picture costs a few bytes for every frame after the
first. Set `duration`,
`frame_rate` (whole rates from 1 to 60), `keyframe_interval` - how far
apart the frames a player can start from are - `width`, `height` and
`quality`. The picture is the gradient with the self describing label the
image formats draw, and it stays the same for the whole film. It can be up
to 4096 pixels wide and at most 4096x2304 pixels in all, the largest the
built in encoder writes correctly - a larger one is refused with a pair
that fits. The manifest
says what a test can check: `duration_ms`, `frame_count`, `frame_rate`,
`keyframe_count`, `width`, `height`, `compression: av1` and `audio: false`.
A length that does not end on a frame is refused with the two nearest that
do - at 30 frames a second lengths go in steps of 100ms. Below the
smallest film the settings allow, the refusal says how small it can be and
which settings make it smaller. AV1 plays in current browsers, and an older
player or a pipeline that expects H.264 may refuse it - which is a test
worth having. The window lists it under Video.
- **A length of time as a setting.** `duration` and `keyframe_interval` take
is a setting of its own, independent of the size. The picture moves: every
`change_interval`, one second unless you say otherwise, the clock in it
moves on and a square takes a step across it, so a player that plays it
shows a running clock and one that freezes shows a stopped one. The clock
reads the time the picture starts, the way a player's position does -
`00:00:07`, with milliseconds when the changes do not fall on whole
seconds. Each change is a whole new picture in bytes and in coding time,
and the frames between changes cost a few bytes each, so an hour at thirty
frames a second with a change every second fits in about a megabyte
(1 075 831 B at the smallest), and a `change_interval` as long as the film
or longer keeps one picture throughout (977 434 B for that hour). Set
`duration`, `change_interval`, `frame_rate` (whole rates from 1 to 60),
`keyframe_interval` - how far apart the frames a player can start from
are - `width`, `height` and `quality`. The picture is the gradient with the
self describing label the image formats draw, with the clock under the
label and the square below. On a picture too small for the clock the film
says so in the manifest. It can be up to 4096 pixels wide and at most
4096x2304 pixels in all, the largest the built-in encoder writes
correctly - a larger one is refused with a pair that fits. The manifest says what a
test can check: `duration_ms`, `frame_count`, `frame_rate`,
`keyframe_count`, `change_count`, `change_interval_ms`, `width`, `height`,
`compression: av1` and `audio: false`. A length that does not end on a
frame is refused with the two nearest that do - at 30 frames a second
lengths go in steps of `100ms`. Below the smallest film the settings allow,
the refusal says how small it can be and which settings make it smaller.
AV1 plays in current browsers, and an older player or a pipeline that
expects H.264 may refuse it - which is a test worth having. The window
lists it under Video.
- **A length of time as a setting.** `duration`, `change_interval` and `keyframe_interval` take
`10s`, `1m30s`, `1h`, `500ms` or `59.9s`, to the millisecond. A bare number
is refused rather than read as seconds, and so is a length that does not
land on a whole millisecond.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -606,7 +606,7 @@ recipe. `tfg formats <id>` prints the allowed range or list for each:
| `avif`, `jpg`, `jxl` | `width`, `height`, `quality` |
| `ico` | `width`, `height`, `embed` |
| `wav` | `sample_rate`, `bit_depth`, `channels`, `content` |
| `webm` | `width`, `height`, `duration`, `frame_rate`, `keyframe_interval`, `quality` |
| `webm` | `width`, `height`, `duration`, `change_interval`, `frame_rate`, `keyframe_interval`, `quality` |
| `zip` | `entries`, `entry_format`, `entry_size`, `compression`, `depth`, `directory_entries`, `password`, `encryption` |
| `targz` | `entries`, `entry_format`, `entry_size`, `compression`, `depth`, `directory_entries`, `entry_mode`, `entry_owner` |
| `docx` | `paragraphs` |
Expand Down
12 changes: 12 additions & 0 deletions internal/format/imagelabel/imagelabel.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,15 @@ func drawGlyph(img draw.Image, g string, x, y, scale int, ink color.Color) {
func Fits(width, chars int) bool {
return scaleFor(width, chars) > 0
}

// BandHeight is how tall the band Draw paints for a label of chars characters
// across width is, before it is cut to the picture - 0 when the label does not
// fit. A film draws a second line under the first and asks this before it
// draws, so planning can say whether that line will be there.
func BandHeight(width, chars int) int {
s := scaleFor(width, chars)
if s == 0 {
return 0
}
return glyphHeight*s + 2*pad
}
144 changes: 99 additions & 45 deletions internal/format/video/choose.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,25 +22,28 @@ type Rung struct {
// 640x360 for the reason AVIF stops at 640x480: a larger picture costs every
// file of a run its encoding time, and somebody who wants Full HD says so.
//
// Ceiling is a tenth above the largest tile the rung's picture coded to in two
// passes - every seed offset there is with every length the label can have,
// either format's name in it, and the label absent, then 512 seeds far from
// zero, because the label's text moves with the whole seed and not only with
// the offset. Measured by tools/probes/videoladder at the default quality on
// 2026-10-06 (docs/WIDEO-2026-10-06.md section 13).
// Ceiling is a tenth above the largest tile any picture of a film on the rung
// coded to, and it is the reserve of every one of them. Measured by
// tools/probes/videoladder at the default quality on 2026-10-06
// (docs/WIDEO-2026-10-06.md section 15), in three passes. First every seed
// offset there is with every length the label can have, either format's name
// in it, and the label absent, in both shapes of the clock. Then the eight
// heaviest of those at a hundred clocks, every step of the square and the
// longest times a day shows. Then 512 seeds far from zero, because the
// label's text moves with the whole seed and not only with the offset.
//
// The tenth is not caution for its own sake. On four rungs of eleven the far
// seeds coded larger than the largest of the first pass, by up to 3.7 percent
// (256x144: 4185 B, then 4338 B), so the first pass alone would have been a
// ceiling a seed nobody tried could pass. A tenth is close to three times the
// worst of that. A ceiling too high costs a smaller picture than there was
// The tenth is not caution for its own sake. On 640x360 the far seeds coded
// larger than the first pass by 1.9 percent (15194 B, then 15478 B), and
// before the pictures moved, on four rungs of eleven, by up to 3.7 percent
// (section 13), so a first pass alone would have been a ceiling a seed nobody
// tried could pass. A ceiling too high costs a smaller picture than there was
// room for and nothing else. One too low would let planning promise a size
// writing cannot keep, so writing checks and says so rather than trusting
// this table.
// writing cannot keep, so writing checks every picture and says so rather
// than trusting this table.
var Ladder = []Rung{
{640, 360, 14644}, {426, 240, 8500}, {320, 180, 6192}, {256, 144, 4772},
{160, 90, 2953}, {80, 45, 385}, {40, 23, 218}, {16, 9, 83},
{4, 3, 38}, {2, 2, 21}, {1, 1, 6},
{640, 360, 17026}, {426, 240, 10905}, {320, 180, 8191}, {256, 144, 6754},
{160, 90, 3951}, {80, 45, 1454}, {40, 23, 825}, {16, 9, 149},
{4, 3, 46}, {2, 2, 31}, {1, 1, 6},
}

// Choice is the picture a film shows.
Expand All @@ -49,12 +52,12 @@ type Choice struct {
Seed uint64
Label string
QIndex int
// Coded is the picture when planning had to code it - a size named by hand,
// or a quality the ceilings were not measured at. Nil means the picture is
// a ladder rung and writing codes it.
Coded *Coded
// Ceiling is the largest tile the plan allowed for, which writing holds
// the coded picture to.
// First is the film's first picture when planning had to code it - a size
// named by hand, or a quality the ceilings were not measured at. Nil means
// the picture is a ladder rung and writing codes it.
First *Coded
// Ceiling is the most tile bytes the plan allowed each picture, which
// writing holds every picture to.
Ceiling int
// Named is whether the request gave the picture's size. A picture chosen
// to fit is the largest that does, so only a named one can be asked to be
Expand All @@ -65,20 +68,67 @@ type Choice struct {
// Labelled is whether this picture carries its label.
func (c Choice) Labelled() bool { return Labelled(c.Width, c.Label) }

// Code is the picture's coded form, coding it when planning did not.
func (c Choice) Code() (Coded, error) {
if c.Coded != nil {
return *c.Coded, nil
// ClockShown is whether this film's pictures show the clock.
func (c Choice) ClockShown(t Timeline) bool { return ClockShown(c.Width, c.Height, c.Label, t) }

// sampleReserve codes the sample of a film's pictures, SampleChanges, and
// gives back the first picture, which writing reuses, and the reserve every
// picture is held to: the largest of the sample when the sample is the whole
// film, and a tenth above it when it is not.
//
// The first picture alone is no base for it. The pictures of one film differ
// only in the clock and the square, and on a small picture those are much of
// it: the largest of a film came out up to 42 percent above its first (48x32)
// and 9 percent above the largest of its first ten (52x20), because the
// clock's other digits change later. Against the spread sample the largest of
// whole films of 240 to 3600 pictures, from 16x16 to 640x360, five seeds and
// both shapes of the clock, came out at most 4.0 percent above (48x32) -
// tools/probes/videomotion/spread, 2026-10-06, docs/WIDEO-2026-10-06.md
// section 15. A tenth is two and a half times that, the margin the ladder's
// ceilings keep over their own measurement. The gradient does not move between
// pictures, by the owner's decision of the same day, because moving it changed
// a picture's size by up to 1.9 times.
func sampleReserve(w, h int, seed uint64, label string, t Timeline, qindex int) (Coded, int, error) {
p := newPainter(w, h, seed, label, t)
sample := SampleChanges(t)
var first Coded
largest := 0
for _, c := range sample {
coded, err := Encode(p.paint(c), qindex)
if err != nil {
return Coded{}, 0, err
}
if c == 0 {
first = coded
}
largest = max(largest, coded.Size())
}
coded, err := Encode(Picture(c.Width, c.Height, c.Seed, c.Label), c.QIndex)
if err != nil {
return Coded{}, err
if int64(len(sample)) == t.Changes() {
return first, largest, nil
}
return first, largest + (largest+9)/10, nil
}

// SampleChanges is the pictures planning codes to settle the reserve of a film
// whose picture it cannot take from the ladder: every one of a film of ten
// pictures or fewer, and of a longer film ten, one at each step of the square,
// in walks spread from its start to its end, so the sample sees the square
// everywhere and the clock with the digits of the whole film.
func SampleChanges(t Timeline) []int64 {
n := t.Changes()
if n <= squareSteps {
out := make([]int64, n)
for c := range out {
out[c] = int64(c)
}
return out
}
if coded.Size() > c.Ceiling {
return Coded{}, core.Defect(fmt.Errorf("video: a %dx%d picture coded to a %d B tile and its rung allows %d B, so the file planned around it cannot be kept",
c.Width, c.Height, coded.Size(), c.Ceiling))
walks := n / squareSteps
out := make([]int64, squareSteps)
for k := range out {
out[k] = int64(k)*(walks-1)/(squareSteps-1)*squareSteps + int64(k)
}
return coded, nil
return out
}

// Choose settles the picture for a request, and the stream planning sizes the
Expand Down Expand Up @@ -109,25 +159,27 @@ func Choose(formatID string, r format.Request, s Settings, label string, fits fu
}

// onRung is the picture of one rung and the stream planning sizes it by: the
// rung's bound, or - when the request named a quality the ceilings were not
// measured at - the picture itself, coded. That is the slow road AVIF takes
// for the same reason, and only runs that asked for it pay.
// rung's ceiling for every picture, or - when the request named a quality the
// ceilings were not measured at - a reserve from a sample of the film's
// pictures, coded (sampleReserve). That is the slow road AVIF takes for the
// same reason, and only runs that asked for it pay.
func onRung(base Choice, rung Rung, s Settings) (Choice, Stream, error) {
c := base
c.Width, c.Height, c.Ceiling = rung.Width, rung.Height, rung.Ceiling
if !s.QualityNamed {
return c, Bound(s.Timeline, rung.Ceiling, rung.Width, rung.Height), nil
return c, NewStream(s.Timeline, rung.Ceiling, rung.Width, rung.Height), nil
}
coded, err := Encode(Picture(c.Width, c.Height, c.Seed, c.Label), c.QIndex)
first, reserve, err := sampleReserve(c.Width, c.Height, c.Seed, c.Label, s.Timeline, c.QIndex)
if err != nil {
return Choice{}, Stream{}, err
}
c.Coded, c.Ceiling = &coded, coded.Size()
return c, NewStream(s.Timeline, coded, c.Width, c.Height), nil
c.First, c.Ceiling = &first, reserve
return c, NewStream(s.Timeline, c.Ceiling, c.Width, c.Height), nil
}

// named codes a picture whose size the request gave, at planning, because
// nothing but coding it says how big it is.
// named settles a film whose picture size the request gave by coding a sample
// of its pictures at planning, because nothing but coding them says how big
// they are (sampleReserve).
func named(formatID string, r format.Request, s Settings, c Choice) (Choice, Stream, error) {
w, err := imagedim.Value(formatID, imagedim.SettingWidth, r.Properties, maxWidth, Ladder[0].Width)
if err != nil {
Expand All @@ -143,12 +195,12 @@ func named(formatID string, r format.Request, s Settings, c Choice) (Choice, Str
if err := checkOneTile(formatID, w, h); err != nil {
return Choice{}, Stream{}, err
}
coded, err := Encode(Picture(w, h, c.Seed, c.Label), c.QIndex)
first, reserve, err := sampleReserve(w, h, c.Seed, c.Label, s.Timeline, c.QIndex)
if err != nil {
return Choice{}, Stream{}, err
}
c.Width, c.Height, c.Coded, c.Ceiling, c.Named = w, h, &coded, coded.Size(), true
return c, NewStream(s.Timeline, coded, w, h), nil
c.Width, c.Height, c.First, c.Ceiling, c.Named = w, h, &first, reserve, true
return c, NewStream(s.Timeline, c.Ceiling, w, h), nil
}

// checkJointLimits asks the registry's own declaration, so the refusal, the
Expand Down Expand Up @@ -205,6 +257,8 @@ func Facts(c Choice, s Settings) map[string]any {
"frame_rate": s.FPS,
"keyframe_count": s.Keys(),
"keyframe_interval_ms": s.KeyEvery * 1000 / int64(s.FPS),
"change_count": s.Changes(),
"change_interval_ms": s.ChangeMs(),
"quality": s.Quality,
"compression": "av1",
"audio": false,
Expand Down
23 changes: 14 additions & 9 deletions internal/format/video/doc.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Package video is what the video formats share: the timeline, the picture,
// Package video is what the video formats share: the timeline, the pictures,
// and the AV1 stream they carry. WebM and MP4 are thin containers around it,
// the way docx, xlsx and pptx are thin around opc.
//
Expand All @@ -9,21 +9,26 @@
// a picture again without coding it again. The probe that settled this and
// every measurement under it is docs/WIDEO-2026-10-06.md sections 9 to 11.
//
// The stream has one shape, and the shape is the point:
// A film shows a new picture at every change - the clock in it moves on and a
// square takes a step, every change_interval (section 15) - and between
// changes it shows the same picture again. Each picture is coded on its own,
// because nothing in gav1d predicts one picture from another. The stream has
// one shape, and the shape is the point:
//
// a key frame, shown - where a player can start
// the same picture again, - an intra only frame, not shown, kept in a
// hidden and showable slot a later frame can show
// the picture as a hidden - an intra only frame, not shown, kept in a
// copy, then shown slot, and the frame that shows it: after
// each key frame and at each change
// show_existing_frame - every other frame, three bytes each
//
// A shown key frame cannot be shown a second time - the specification sets its
// showable_frame to 0 and libaom refuses the stream that tries ("Buffer does
// not contain a showable frame"). gav1d's own decoder and Chromium both play
// that stream without a word, which is why neither of them is a witness here.
// The hidden copy costs the picture's bytes once more and no second encoding,
// because it carries the very same tile.
// The hidden copy after a key frame costs the picture's bytes once more and no
// second encoding, because it carries the very same tile.
//
// That shape is what makes a length independent of the size: an hour at thirty
// frames a second is about a megabyte, because 108 000 of its frames are three
// bytes each.
// That shape is what makes a length independent of the size: frames between
// changes are three bytes each, so the bytes go to the pictures, and a film
// that changes rarely or never is small whatever its length.
package video
Loading
Loading