Repository navigation
AUS-1012: Publish the benchmarks from the cpu-performance job, regenerated from data - #172
Merged
Merged
Conversation
…, ingest and generated README tables
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear: AUS-1012 (parent AUS-1001; supersedes AUS-985). Built on #171 (AUS-1011), now merged.
The published benchmarks now come from one Hugging Face Jobs
cpu-performancerun (AMD EPYC 7R13, 32 cores, whole host) instead of the desktop, and republishing them is a pipeline rather than a transcription.Pipeline
JSONRPC_BENCH_RESULTSis set (TestServer_Console/BenchResults.cs;benchmarks/Baselinehas its own writer). Rows carry a stable id, throughput and the CPU usage from AUS-1011: CPU-utilisation axis per benchmark row, derived machine description, Hugging Face Jobs runner #171.benchmarks/hf/run.sh <ref> publishruns the README's policy on the job host: the scaling gate once, two rounds of--sync,--async 3 1,--async 3 N,--kestrel,--kestrel asyncand--compare, three legacytruns, two baseline runs and five--sweep 2runs, every run and every JSON file framed on stdout.fetch.pyextracts them.benchmarks/charts/ingest.py <dir>folds the JSON intobenchmarks.json(measured fields only; ids, labels, notes and groups stay hand-maintained; idempotent) and copies the sweep files.render.pyrenders the SVGs, the explorer and the README tables between<!-- benchmarks:<set> -->markers (README.md and Json-Rpc/README.md);render.py --checkfails on a stale block, alt text or chart.Environment.ProcessorCount).Republishing is:
launch.sh <ref> cpu-performance publish,fetch.py,ingest.py,render.py, commit.Data
Job
6ab9b3936b030d633f69b0fb(https://huggingface.co/jobs/astnh/6ab9b3936b030d633f69b0fb), commit 6c3c272, 2026-09-28. Gate 12.2 / 12.1 / 13.1 at 32 workers, pass. Sync 44.2 M to 45.7 M at 32 threads; Kestrel TCP 18.5 M at 32 clients; the 2.0 string API 9.8 M to 11.1 M against 1.2.3 at 1.29 M to 2.41 M (the two baseline runs differ widely; the ratios use the medians). The WebAssembly table stays a browser measurement on the desktop and says so.Checks
render.py --check, the 43 chart tests, the request-path checker, the site build and the solution build and tests (1220 on net8.0 and net10.0) pass; a secondingest.pyon the job directory reports no change.