A pure-Rust engine for training, running, serving, and evaluating small causal Transformer language models (SLMs) on the CPU, with no GPU. Models you train are GPT-2-architecture GGUF files that also run in llama.cpp, Ollama, and LM Studio.
Ferrum is #![forbid(unsafe_code)] throughout. The project contains the
tensors, the hand-written and gradient-checked backpropagation, a byte-level BPE
tokenizer, int8 quantization-aware training (QAT), a GGUF model format, a
multi-core matmul engine, a corpus-preparation tool, an HTTP inference server,
and a LightEval module. There is no NumPy, no BLAS, and no CUDA. The core also
compiles to WebAssembly, so a model you train can run on a server, a laptop, a
Raspberry Pi, or in a browser tab.
Ferrum keeps its dependencies few. The engine, ferrum_core, is std-only
apart from two pure-Rust Unicode tables, and a crate is admitted anywhere only
where a measurement shows a gain (see the table below and
Manual §2). It is built
for small models. It is not a runtime for frontier LLMs.
| Property | What it means |
|---|---|
| Few dependencies, each with a reason | ferrum_core has two (unicode-general-category and unicode-normalization, pure-Rust tables) and hand-writes its threading and matmul kernels. The CLI adds sha2 and serde_json, the server axum/tokio/serde, the browser crate wasm-bindgen, and the GUI Tauri plus a few helper crates. A crate is admitted only if it is faster on a benchmark and keeps results deterministic and wasm32 builds working. rayon, ndarray, and faer were measured and none is in the tree (Manual §2.2.1). |
| CPU only, deterministic | Every matmul splits across a worker pool. Training, generation, and evaluation give bit-for-bit identical results at any pool size on one platform (override with FERRUM_NUM_THREADS), and a test enforces this. Across operating systems the last float bits can differ (the bits recorded on Linux did not match on macOS and Windows in CI). One knob changes the trained weights: the number of data-parallel training workers, --threads (auto = the core count). Fix it to reproduce a run on another machine of the same platform. |
| GGUF only | Every file Ferrum writes is GGUF v3: models, training checkpoints, tokenizer files, and pre-tokenized corpora. The old FINF .bin format is removed. |
| Runs in llama.cpp, Ollama, and LM Studio | A trained BPE SLM is a gpt2-architecture GGUF. The same file gives the same greedy continuation in Ferrum and in llama.cpp (a test checks this). Models can be as small as about 0.1M parameters. |
| Quantization-aware | QAT simulates GGUF Q8_0 exactly, so an int8 file reloads bit-identical to the model you trained. int4 (Q4_0) export is a lossy option. |
| Byte-level BPE with a GPT-2 pre-tokenizer | Any UTF-8 text round-trips. llama.cpp tokenizes prompts exactly as Ferrum does. `< |
| A real data pipeline | train_transformer prepare cleans, dedups, and splits a corpus and writes a hash manifest. DVC reproduces fetch, prepare, train, and eval from params.yaml. |
| Corpora larger than RAM | train_transformer tokenize writes a pre-tokenized GGUF token file (2 bytes per token); train tokens.gguf streams windows from disk and gives the same model, bit for bit, as training on the text (Manual §6.11). |
| Serving and standard evaluation | ferrum-serve exposes log-likelihood and generation over HTTP for Ferrum models and imported Llama/Qwen GGUFs, on loopback by default, with an optional bearer token (FERRUM_TOKEN) that a non-loopback bind requires. A LightEval module evaluates Ferrum models through it. |
| Llama/Qwen GGUF in and out | Import Llama/Qwen checkpoints, run them, fine-tune them with AdamW, and export them back to GGUF. |
One limit to know. Ferrum's tabular models (the ferrum-ml MLP classifiers) and character-level models are also GGUF files. They do not run in llama.cpp, because llama.cpp has no MLP-classifier architecture and no character-level tokenizer. Only BPE transformer SLMs run there.
- Workspace layout
- Installation
- Quick start
- Run your model elsewhere
- How big a model can I train?
- Documentation
- Building and testing
- License
| Path | Kind | What it is |
|---|---|---|
ferrum_core |
library | The engine: tensors, layers, the transformer trainer, tokenizer, quantization, the GGUF model codecs, and the GGUF reader/writer plus Llama/Qwen runner and trainer |
slm_cli |
binary train_transformer |
Train, generate, evaluate, inspect, prepare and tokenize data; export-ollama, export-lmstudio; run-gguf, eval-gguf, finetune-gguf, export-gguf for external checkpoints |
slm_server |
binary ferrum-serve |
HTTP inference server (axum): log-likelihood, rolling log-likelihood, generation |
slm_wasm |
cdylib + rlib | wasm-bindgen bindings that run a trained SLM in the browser (Slm text API, TransformerSLMModel), plus a no-framework demo page and Web Worker in slm_wasm/demo/ |
eval/lighteval/ |
Python package | FerrumModel and custom tasks for Hugging Face LightEval 0.13.0, plus run.sh |
dvc.yaml, params.yaml, dvc.lock |
DVC pipeline | fetch, prepare, train, eval (and an opt-in lighteval stage in eval/lighteval/dvc.yaml) |
scripts/ |
shell | fetch.sh (pinned download with sha256 check), bench_slm.sh, build_wasm.sh, verify_runtimes.sh |
tests |
integration tests | Cross-crate integration and regression tests, including the llama.cpp/LM Studio compatibility test |
ferrum_gui |
Tauri app (built separately) | Ferrum SLM Studio, a desktop GUI; see Manual §10 |
ferrum_gui is deliberately outside the Cargo workspace because it needs
heavy system WebView libraries. Build it from its own directory.
Sibling folders. Two folders next to this repository hold code that used to live here (see Status §2):
../ferrum-mlholds the machine-learning demos that are not about SLMs: a tabular MLP classifier/regressor, its CSV loader, ten datasets, pretrained models (now GGUF), and a browser binding. It builds onferrum_coreby path.../ferrum-junkis an archive of removed features.../ferrum-gpuholds the plan (PLAN.md) for a GPU-capable edition of this engine. There is no code yet.
| Requirement | Version | Notes |
|---|---|---|
| Rust + Cargo | 1.87 or newer, edition 2021 | Install via rustup. The dependency tree needs 1.85 (sha2), and the code uses is_multiple_of, stable since 1.87. Built and tested with 1.96.0; older toolchains are untested. |
| OS | Linux (x86_64 gated in CI; aarch64 also tested), macOS, Windows or Windows-on-Arm (built and tested by a non-gating CI job that is green on all of them, see Status §5) | Pure CPU code; the RAM guard (it refuses a run that would not fit) works on Linux only |
curl |
any | Only for scripts/fetch.sh (Samples 1 and 2) |
| GNU coreutils | Linux | The samples use sha256sum, md5sum, and timeout; they are written for Linux |
| Python 3.10+ | optional | Only for DVC and LightEval (Samples 2 and 8) |
wasm32-unknown-unknown |
optional | Only for browser builds (slm_wasm) |
| WebView system libraries | optional | Only for the desktop GUI; see Manual §10.2 |
| llama.cpp, Ollama, LM Studio | optional | Only to run Ferrum models there (Sample 7) |
You need only Rust to train, evaluate, generate, and serve. DVC and Python are optional.
git clone https://github.com/thomascherickal/Ferrum
cd Ferrum
cargo build --workspace --release # LTO + 1 codegen unit; use it for real training
cargo test --workspace --release # all unit + integration tests should passDebug builds are several times slower. Use --release for any real training or
for running GGUF models.
cargo install --path slm_cli # installs `train_transformer`
cargo install --path slm_server # installs `ferrum-serve`[dependencies]
ferrum_core = { git = "https://github.com/thomascherickal/Ferrum" }ferrum_core forbids unsafe code. Its dependencies are the few listed in
Manual §2; cargo deny check
covers their licences.
# 1. Download the pinned TinyStories file (22,502,601 bytes, sha256 is checked).
scripts/fetch.sh \
https://huggingface.co/datasets/roneneldan/TinyStories/resolve/f54c09fd23315a6f9c86f9dc80f725de7d8f9c64/TinyStoriesV2-GPT4-valid.txt \
6874bae9a4c1a4e7edcf0e53b86c17817e9cf881fc75ff2368da457b80c0585d \
data/raw/tinystories.txt
# 2. Clean, dedup, and split it. Writes train/val/test.txt and manifest.json.
train_transformer prepare --input data/raw/tinystories.txt --out data/prepared \
--dedup --nfkc --min-chars 50 --max-chars 20000
# 3. Train the 507,552-parameter reference model.
train_transformer train data/prepared/train.txt models/slm.gguf \
--context 128 --embed 96 --heads 4 --blocks 3 --hidden 320 --vocab 1024 \
--tokens 2e6 --lr 5e-3 --cosine --warmup 50 --clip 1 \
--val 0.02 --patience 3 --tokenizer models/tokenizer.gguf
# 4. Use it.
train_transformer generate models/slm.gguf "Once upon a time" --max-tokens 100 --temp 0.7
train_transformer eval models/slm.gguf data/prepared/test.txt --json metrics/test.json
train_transformer info models/slm.ggufInterrupted? Add --checkpoint run.fckp --checkpoint-every 500 to the training
command, then repeat the same command with --resume run.fckp. To stop on
purpose, pass --max-seconds N or create the file <model>.stop; the run
checkpoints and exits cleanly (Manual §6.6).
SAMPLES.md walks through every step with real output, and
Manual §6 explains every flag.
The CLI is strict. An unknown flag, a value that does not parse, or a missing
--resume file stops the command with an error that names the problem, and so
does an extra positional argument (info m.gguf extra). train_transformer --version prints the version, and train_transformer <command> --help prints
that command's flags.
python3 -m venv .venv && .venv/bin/pip install -r requirements-dataops.txt
.venv/bin/dvc repro # fetch -> prepare -> train -> eval
.venv/bin/dvc metrics showEdit params.yaml and run dvc repro again to rerun only the stages that
changed. Training is deterministic, so the same inputs give the same
models/slm.gguf on any machine of the same platform (the lock hashes were recorded on Linux x86_64; other operating systems can differ in the last float bits): the pipeline pins the data-parallel shard count (train.threads in params.yaml; Manual §6.8).
use ferrum_core::{GenerativeSLM, Rng, TransformerConfig};
let corpus = std::fs::read_to_string("corpus.txt")?;
let cfg = TransformerConfig {
context_len: 64, embed_dim: 64, num_heads: 4, num_blocks: 2, hidden_dim: 128,
vocab_size: 512, weight_tying: true, lr: 0.003, batch_size: 16,
max_tokens: 5_000_000, cosine_lr: true, warmup_steps: 100, grad_clip: 1.0,
..TransformerConfig::default()
};
let mut rng = Rng::new(1337);
let slm = GenerativeSLM::train_transformer_config_threaded(&corpus, &cfg, 0, &mut rng, |ep, loss| {
println!("epoch {ep}: loss {loss:.4}");
})?;
slm.save("model.gguf")?; // int8 GGUF (gpt2 layout)
let s = slm.score("The cat sat on the", " mat")?; // log p(continuation | context)
println!("logprob {:.3} over {} tokens", s.logprob, s.n_tokens);train_transformer run-gguf qwen2-0_5b.gguf "Explain rust ownership" --quant int4 --max 64
train_transformer finetune-gguf smollm2-135m.gguf corpus.txt tuned.flck --epochs 1 --lr 1e-4
train_transformer export-gguf smollm2-135m.gguf tuned.gguf --resume tuned.flck --quant q8_0Supported architectures are llama and qwen2. See
Manual §9.
A Ferrum BPE SLM is a normal gpt2 GGUF. Use it with other runtimes directly,
or with a one-command export.
# llama.cpp
llama-completion -m models/slm.gguf -p "Once upon a time" -n 16 --temp 0 -no-cnv --no-warmup </dev/null 2>/dev/null
# One-command exports (Ferrum gpt2 models only: each refuses anything else)
train_transformer export-ollama models/slm.gguf ferrum-tiny # writes a Modelfile, runs `ollama create`
train_transformer export-lmstudio models/slm.gguf # links it into LM Studio with `lms import -l`Versions verified so far: llama.cpp b11521, Ollama v0.40.2 (by hand; no test or CI job
runs Ollama since 2026-11-04), LM Studio 0.4.18.
Each tool is found from a --bin flag (export commands), then the env var
(FERRUM_LLAMA_BIN, FERRUM_OLLAMA, FERRUM_LMS), then PATH, then the default
install location (~/.local/opt/llama.cpp-b11521, ~/.local/bin/ollama,
~/.lmstudio/bin/lms).
The exact commands, and what to expect from each tool, are in
SAMPLES.md §7
and Manual §9.5.
Serve a model over HTTP with ferrum-serve --model models/slm.gguf and
evaluate it with LightEval: Manual §9.7 and
Manual §9.8.
Measured on an Intel i5-1135G7 (4 cores / 8 threads, 15 GiB RAM) on 2026-10-26 with int8 QAT, batch 16, and 8 threads. Compute is the binding limit. RAM is not, and a 2 GB GPU makes no difference because Ferrum never uses it.
| Goal | Reachable size |
|---|---|
| Heavily over-trained, minutes to hours | 0.1M–2M parameters |
| Compute-optimal (about 20 tokens/param) in 24 h | about 4M |
| Compute-optimal in a week | about 12M |
| Compute-optimal in a month | about 27M (extrapolated) |
| Fine-tune an imported GGUF (full AdamW, f32) | about 135M comfortably; 0.5B is tight |
The sizes come from short runs of four configurations, after the speed work in this release (Manual §6.7). A TinyStories-class model (1–3M parameters on about 470M tokens) is a 2–4 day job. The capability report estimates what 8 to 128 GB machines and recent Intel, Ryzen and Snapdragon processors can do. Manual §6.7 has the throughput table, and Status §3 has the assessment.
Two reference sizes are used in the docs:
| Config | Flags | Parameters |
|---|---|---|
| Reference 500K | --context 128 --embed 96 --heads 4 --blocks 3 --hidden 320 --vocab 1024 |
507,552 |
| Smallest supported reference | --context 64 --embed 64 --heads 4 --blocks 2 --hidden 128 --vocab 512 --tie |
103,936 |
--vocab N is a ceiling: a small corpus supplies fewer BPE merges, and train
then prints vocab: 282 (requested 512, limited by the corpus). The counts
above assume a corpus large enough to fill the vocabulary, as the TinyStories
reference does (Manual §6.2).
| Document | Read it for |
|---|---|
| MANUAL.md | Everything about using Ferrum, from first principles through corpus preparation, training, generation, evaluation, the GGUF format, running in other runtimes, DVC, serving, LightEval, the GUI, and deployment, to the API reference, benchmarks, FAQ, and an honest critique |
| SAMPLES.md | Ten executed walk-throughs with real commands and output: quick start, DVC, data cleaning, training controls, evaluation, generation, serving and other runtimes, LightEval, benchmarking, and the library API |
| STATUS.md | Where the project stands: what is implemented and tested, known issues, the roadmap, the history, and the backward-compatibility record (§7) |
| GLOSSARY.md | A beginner's guide to LLM terms |
cargo doc -p ferrum_core --open |
Rustdoc for every public item |
cargo build --workspace # everything except the GUI
cargo test --workspace --release # unit + integration tests
cargo test -p slm_cli --release --test determinism # 1 thread vs N threads, identical output
cargo bench --bench gemm # matmul/decode microbenchmark
cargo clippy --workspace --all-targets -- -D warnings
cargo deny check # licences and advisories for the dependencies
bash scripts/build_wasm.sh # slm_wasm -> slm_wasm/pkg/ and the demo (needs wasm-bindgen)
python3 -m http.server -d slm_wasm/demo 8000 # the browser demo: http://localhost:8000/ (Manual §11.3)
(cd ferrum_gui && cargo tauri dev) # the desktop app
train_transformer prepare --input corpus.txt --out data/prepared # clean + split + manifest
cargo run --release -p slm_server -- --model models/slm.gguf # ferrum-serve on 127.0.0.1:8080
.venv/bin/dvc repro # whole data-to-metrics pipeline
python3 -m unittest eval/lighteval/test_ferrum_model.py # LightEval module unit tests
eval/lighteval/run.sh --model models/slm.gguf --test data/prepared/test.txt --out metrics/lighteval.jsonRuntime compatibility tests skip a tool that is not installed. Set
FERRUM_REQUIRE_RUNTIMES=1 and a missing tool becomes a test failure
instead. CI and pre-release runs set it. Tool discovery is described above.
CI (.github/workflows/ci.yml) runs build and test, rustfmt, clippy, JS syntax
checks, cargo-deny, a wasm32 build of slm_wasm, the LightEval module's import
check, and the GUI crate's own format, lint, and test jobs. A runtimes job
runs the compatibility tests with FERRUM_REQUIRE_RUNTIMES=1: llama.cpp b11521
is required there, and FERRUM_SKIP_LMSTUDIO=1 is its one exemption (LM Studio is
required locally before a release). No test or CI job runs Ollama. Python and DVC are
otherwise not in CI.
Auto-generated from METRICS.md · Last updated: 2026-11-20 · Commit: ef41165 (2026-11-19)
| Metric Dimension | Value / Measurement | Description / Invariant |
|---|---|---|
| Repository Scope | Pure-Rust SLM Engine & Tauri GUI | Training, inference, GGUF import/export, serving, and evaluation |
| Workspace Architecture | 5 Crates + GUI + Wasm + Python | ferrum_core, slm_cli, slm_server, slm_wasm, ferrum_gui |
| Total Tracked Code Lines | 93,666 lines (114,861 total across 402 files) | Across Rust, JS, Python, HTML, CSS, Shell, and configuration |
| Rust Codebase Volume | 67,969 pure code lines | 267 Rust files (80,363 total physical lines) |
| Memory Safety Invariant | 100% Safe Rust (#![forbid(unsafe_code)]) |
Zero unsafe blocks in any crate root or dependency module |
| Total Test Suite Volume | 1,422+ automated tests | Rust (1,018), GUI (225), Node/DOM (352+), Python (64) |
| Test Suite Health | 100% Pass (0 failed, 2 ignored) | Automated CI line coverage suites execute in ~4m 50s (< 5m) |
| Rust Line Coverage | 99.89% (34,967 / 35,007 lines) | LLVM-cov gated floor: 99.00% |
| GUI Backend Line Coverage | 100.00% (6,472 / 6,472 lines) | Tauri IPC backend coverage floor: 100.00% |
| Frontend Line Coverage | 100.00% (14 / 14 UI files) | Node.js DOM unit, flow, accessibility & streaming suites |
| Python Coverage | 100.00% (153 / 153 statements) | LightEval adapter stubs & evaluation scaffolding |
| Language | Files | Code Lines | Comments | Blank Lines | Total Lines | Code Share |
|---|---|---|---|---|---|---|
| Rust (.rs) | 267 | 65,633 | 9,717 | 5,013 | 80,363 | 70.1% |
| Markdown (.md) | 32 | 12,149 | 1,454 | 3,082 | 16,685 | 13.0% |
| JavaScript (.js) | 55 | 7,348 | 701 | 548 | 8,597 | 7.8% |
| JSON (.json) | 12 | 5,183 | 0 | 0 | 5,183 | 5.5% |
| Python (.py) | 11 | 1,056 | 25 | 212 | 1,293 | 1.1% |
| HTML (.html) | 2 | 807 | 0 | 34 | 841 | 0.9% |
| Shell (.sh) | 7 | 528 | 102 | 102 | 732 | 0.6% |
| YAML (.yml, .yaml) | 5 | 489 | 46 | 17 | 552 | 0.5% |
| TOML (.toml) | 9 | 224 | 42 | 44 | 310 | 0.2% |
| CSS (.css) | 2 | 249 | 33 | 23 | 305 | 0.3% |
| Total Repository | 402 | 93,666 | 12,120 | 9,075 | 114,861 | 100.0% |
| Crate Name | Purpose / Role | Files | Code Lines | Comments | Blank Lines | Total Lines | Unsafe Blocks |
|---|---|---|---|---|---|---|---|
ferrum_core |
Core neural tensor math, transformer layers, KV cache, tokenizer, BPE | 116 | 32,400 | 4,493 | 2,492 | 39,385 | 0 (forbid) |
slm_cli |
CLI subcommands: train, generate, eval, gguf-export, finetune | 53 | 13,176 | 847 | 899 | 14,922 | 0 (forbid) |
slm_server |
OpenAI-compatible HTTP serving daemon (/v1/models, /v1/completions) |
20 | 4,479 | 337 | 339 | 5,155 | 0 (forbid) |
slm_wasm |
WebAssembly bindings for browser-based client inference & web worker | 5 | 1,169 | 184 | 122 | 1,475 | 0 (forbid) |
ferrum_gui |
Tauri 2.0 desktop frontend & IPC bridge (hardware detection, sizing) | 49 | 12,603 | 1,176 | 847 | 14,626 | 0 (forbid) |
tests |
Workspace integration, runtime compatibility, and E2E test suite | 24 | 4,142 | 344 | 314 | 4,800 | 0 (forbid) |
| Total Rust | Workspace & GUI | 267 | 67,969 | 7,381 | 5,013 | 80,363 | 0 |
| Test Suite | Target Scope | Line Coverage | Function Coverage | Region / Branch Coverage | Suite Duration | Gate Status |
|---|---|---|---|---|---|---|
| Rust Workspace | scripts/coverage.sh --rust |
99.89% (34,967 / 35,007) | 99.92% (3,813 / 3,816) | 99.00% (68,795 / 69,489) | ~4m 15s | PASS (Floor: 99.0%) |
| Ferrum GUI Backend | scripts/coverage.sh --gui |
100.00% (6,472 / 6,472) | 100.00% (711 / 711) | 99.11% (11,539 / 11,643) | ~21s | PASS (Floor: 100.0%) |
| Frontend UI (DOM / Node) | scripts/coverage.sh --js |
100.00% (14 / 14 files) | 100.00% | 93.81% (branch) | ~6s | PASS (Floor: 100.0%) |
| Evaluation Suite (Python) | scripts/coverage.sh --py |
100.00% (153 / 153 stmts) | 100.00% | 100.00% | ~6s | PASS (Floor: 100.0%) |
| Combined Full Suite | scripts/coverage.sh --all |
> 99.9% Overall | > 99.9% Overall | > 98.9% Overall | ~4m 50s | ALL GATES PASS |
For comprehensive details, memory bounds, and modularization logs, see METRICS.md.
MIT. See LICENSE. The default corpus (TinyStories V2) is CDLA-Sharing-1.0. Ferrum downloads it and never commits it.
