Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1290 commits
Select commit Hold shift + click to select a range
9166cea
opencl: fix OOB tail access for adreno noshuffle gemv kernels (#30421)
lhez Oct 15, 2026
e36df60
sycl : replace oneAPI package by DLE+oneDNN package (#30131)
arthw Oct 15, 2026
47f5a64
vulkan : fix concurrent first device init (#29766)
losewayy Oct 15, 2026
82e1e2f
jinja : write nan and inf in tojson as json.dumps does (#30483)
ykhrustalev Oct 15, 2026
08698c8
jinja : improve context copy constructor performance (#30562)
Ninja3047 Oct 15, 2026
3e4dfa8
models : add maion-coder (#29778)
mailkv23 Oct 15, 2026
fd55cb9
fix(ui): take the agentic turns default from DEFAULT_AGENTIC_CONFIG (…
allozaur Oct 15, 2026
711780f
ci : bump ty to 0.0.88 (#30587)
CISC Oct 15, 2026
4ed1c85
ggml : bump version to 0.27.0
ggerganov Oct 15, 2026
c0cb53a
sync : ggml
ggerganov Oct 15, 2026
b0d2391
server: prefer empty slots and keep states overwritten by child tasks…
ServeurpersoCom Oct 15, 2026
ea736c1
server: save the draft state in the slot file (#30574)
ServeurpersoCom Oct 15, 2026
a6b91d8
model : Mellum - read MoE gating params from GGUF, unify RoPE (#30578)
Xarbirus Oct 15, 2026
61ad80b
server: fix the off-by-two in the SWA reuse threshold (#30580)
ServeurpersoCom Oct 15, 2026
a11b653
scripts : add ci-status.sh to list active CI runs (#30586)
ggerganov Oct 15, 2026
cd0a5b6
models : add Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B support …
frozenblade1224 Oct 15, 2026
620d222
llama.cpp : bump version to 0.7.0 (#30603)
ggerganov Oct 15, 2026
b76b133
pi : expand AI disclosure with hardware and stats (#30615)
ggerganov Oct 15, 2026
ba3c05f
model: add Muse Spark Open (text, vision and audio) (#30616)
ServeurpersoCom Oct 15, 2026
2f729cf
mtmd: lightonocr does not require longest edge to always be 1540, nat…
staghado Oct 15, 2026
e287bd8
convert : fix regression in mapping of Granite speech tensors (#30628)
mrutkows Oct 15, 2026
59d5fad
convert : fix Granite Switch block_count for MultiSwitch checkpoints …
planetf1 Oct 15, 2026
2851b3b
mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629)
ngxson Oct 15, 2026
43d57fb
[SYCL] Enable ssl in build script and CI (#30456)
arthw Oct 16, 2026
e707790
sycl: support reordered Q8_0 MoE weights (#29337)
newjordan Oct 16, 2026
b6334e3
Limit max CUDA pinned host buffer size to 4 GB (#30449)
gaugarg-nv Oct 16, 2026
6afa9a3
llama : share the k-pool input views between layers (#30613)
am17an Oct 16, 2026
b797b9c
hexagon: tensor repacking rework and support for tiled and flat quant…
max-krasnyansky Oct 16, 2026
5e03a1f
graph : add precision handling for MELLUM architecture (#30584)
Xarbirus Oct 16, 2026
f12aaa3
Fix failing RocM tests (#30611)
mailkv23 Oct 16, 2026
a8d9a7e
vulkan: fix gpu-visible host memory issues (#30468)
0cc4m Oct 16, 2026
88aaf32
tests : catch vanished rows in the fusion baseline (#30618)
ggerganov Oct 16, 2026
13de11e
sycl: wide stores for the q4_K/q5_K weight dequant (#29696)
Titaniumtown Oct 16, 2026
c0083e3
sycl : reduce the build threads to half (#30236)
arthw Oct 16, 2026
c804754
ggml-cpu: refactor s309x repack helper functions (#30588)
taronaeo Oct 16, 2026
3dbcb7f
cpu : reduce AVX2 tiled accumulators together (#30296)
trivalik Oct 16, 2026
0f43762
convert : support text-only checkpoints in pplx-decider (#30575)
final-bench Oct 16, 2026
13b8048
common : read ngram cache parts by reference instead of copying (#30061)
jadidbourbaki Oct 16, 2026
623c163
CUDA: HIP: rms_norm: add variants that load float4 at a time and stor…
IMbackK Oct 16, 2026
495c580
gguf-py : add numpy Q1_0 and Q2_0 de/quantization support (#30630)
SomeNobody21112 Oct 16, 2026
a30fb49
[SYCL]: fix memory errors in mul_mat, split buffer, host pool (#29889)
mctylr-gh Oct 16, 2026
b84ea5c
convert : speed up model endian conversion and add more datatype supp…
taronaeo Oct 16, 2026
9a16f1d
model : MTP support for Mellum2.1 (#30612)
Xarbirus Oct 16, 2026
ceefd16
hexagon: correct HMX FA sinks (#30581)
jhen0409 Oct 16, 2026
f336359
Add more benchmarks to llama-eval (#26174)
pwilkin Oct 16, 2026
2a66703
convert : respect Mistral format for DeepseekV2Model (#30669)
aiqubits Oct 16, 2026
7182ef7
common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client (#21752)
texasich Oct 16, 2026
23f7aab
model : Granite Switch Decision (#30676)
gabe-l-hart Oct 16, 2026
01fc3aa
model: add Beam (Reflection) (#30670)
danielhanchen Oct 16, 2026
2f2763d
opencl: Add MxFP4 MoE bin kernel for Adreno X1 devices (#30691)
shawngu-quic Oct 16, 2026
27c7a76
hexagon: vectorize softplus with HVX (#30435)
huangwei858 Oct 16, 2026
6420cf5
ci: update device matrix for Snapdragon jobs by removing SM8750 (#30620)
zhiyuan8 Oct 16, 2026
ea0c4c2
skills: add slice-model (#30677)
ngxson Oct 17, 2026
f74e2fa
sycl: fix prompt-processing regression after Q4_0/Q8_0 weight reorder…
marcusds Oct 17, 2026
1268eda
model : Granite switch decision 256 options (#30693)
gabe-l-hart Oct 17, 2026
83f9389
ci : added ABI reports as github artifact (#30619)
sandrohanea Oct 17, 2026
62318cd
model : add Kolibri-1 (Aleph-Alpha) support (#30673)
riteme Oct 17, 2026
f2f667b
cuda : add grouped experts top-k fusion (#29181)
CISC Oct 17, 2026
9d1e036
cuda : fix -Wtautological-compare in fattn vec dispatch (#30519)
angt Oct 17, 2026
10b8b75
tests: give the random model generator a per-tensor filler table (#30…
ServeurpersoCom Oct 17, 2026
2744e48
CUDA: faster MMVQ for low-bit types (#30517)
am17an Oct 17, 2026
c0fd786
ui : Discover & Download Models from HF Hub (#29584)
allozaur Oct 17, 2026
9d1878a
ggml-cuda: HIP: use __builtin_amdgcn_perm for q1_0 unpacking (#29927)
thelittlefireman Oct 17, 2026
797de63
hexagon: add support for Q1_0 and Q2_0 (#30122)
tboinovski1 Oct 17, 2026
848bbda
hexagon: add support for hadamard MUL_MAT and GET_ROW for flat and ti…
tboinovski1 Oct 18, 2026
547208d
Restore ESIMD kernels on FFN path (#30239)
malsbat Oct 18, 2026
5aeac37
ggml-cuda: fattn do not check oob if k_VKQ_max = (k_VKQ_max - nbatch_…
thelittlefireman Oct 18, 2026
0878f7c
ggml: support non-contiguous SUM (CPU, CUDA, Vulkan, WebGPU) (#28851)
rssndr Oct 18, 2026
ff5a233
cuda : fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain (#3…
am17an Oct 18, 2026
fe8841a
hexagon: allow multistream FA (#30218)
jhen0409 Oct 19, 2026
31288c9
hexagon: FLASH_ATTN_EXT improvements for longer context and SWA model…
max-krasnyansky Oct 19, 2026
99db719
CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec (#30573)
invaliddayta Oct 19, 2026
7db0d3d
pi : turn .pi/gg into a system prompt addendum (#30762)
ggerganov Oct 19, 2026
27af581
tests : update Metal fusion baseline for beam (#30764)
ggerganov Oct 19, 2026
8f97d82
tools : reject truncated GGUF tensor data in split and export tools (…
ppAlexa Oct 19, 2026
2b57148
ui: Fix markdown tables and lists being cut off on phones (#30687)
allozaur Oct 19, 2026
412a363
qwen3 : bidirectional attention and dense_2 output projection (#30607)
fzowl Oct 19, 2026
4c1004c
cuda: cap MMQ Stream-K blocks at available work (#30553)
joelagnel Oct 19, 2026
42b7003
ggml-cpu: vectorize ggml_v_silu and ggml_v_expf (#30791)
taronaeo Oct 19, 2026
4866db3
CUDA: fix warning for nonexistent extra semicolon (#30774)
JohannesGaessler Oct 19, 2026
fda00f1
tests: add big-endian conversion tests (#30705)
taronaeo Oct 19, 2026
f3fb2fc
cann : refactor `#ifdef` blocks to avoid unreachable code after `retu…
rauletorresc Oct 19, 2026
098ce52
metal : fix moe_reduce fusion race when dst aliases its inputs (#30795)
ggerganov Oct 19, 2026
33d6135
moe-cache : replace MoE LRU cache with a 50/50 SLRU cache (#30637)
eapache Oct 19, 2026
7951f60
vendor : update cpp-httplib to 0.62.0 (#30785)
cabelo Oct 19, 2026
f174553
server : fix swa checkpoint invalidation logic (#30666)
tdakhran Oct 19, 2026
0ce8cd1
jinja : implement `unique` filter (#30819)
aldehir Oct 19, 2026
6fa4f6c
opencl: add get_rows for i32, cpy for i32 and f32 (#30756)
lhez Oct 19, 2026
1b29621
opencl: add cooperative-k dp4a gemm for q4_0, q4_k and q6_k (#30741)
lhez Oct 19, 2026
4c3c581
hybrid dna tokenizer fixes (#30825)
kashif Oct 19, 2026
ab7ed0d
ui : fix attachment text preview clipped when a line is longer than t…
A-Raphie Oct 19, 2026
7adf160
server: add --connect, allow p2p access to server outside LAN (#28277)
ngxson Oct 19, 2026
3ffb629
ci : re-enable type/lint-check for skills (#30713)
CISC Oct 19, 2026
921257f
ci, docs: add WASI CI and docs (#30754)
MendyBerger Oct 19, 2026
f711c8f
mtmd : match the transformers Qwen-VL resize (#30736)
boshjerns Oct 19, 2026
3bb63cb
sycl : fix get_rows on reordered quantized weights (#30430)
skipzx-lab Oct 19, 2026
1d88cb0
ui: hide the native clear button in SearchInput (#30719)
aiqubits Oct 20, 2026
dd002b9
sycl: fix compiler warnings - missing prototypes, dead code, extra se…
mctylr-gh Oct 20, 2026
8466395
hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (…
max-krasnyansky Oct 20, 2026
7879491
tests : make MoE router selection independent of the backend noise (#…
ggerganov Oct 20, 2026
d8d9297
vocab : add \p{M} to lookaheads in Tekken and GPT-4o pre-tokenizer re…
tkran2 Oct 20, 2026
707616f
ci: Fix --strict mode for API check (#30623)
ckastner Oct 20, 2026
704366d
metal : fix missing MoE kernel variant for 9 experts per token (#3061…
Caladrel65 Oct 20, 2026
d806193
server : pass id_slot through the Anthropic /v1/messages conversion (…
pcvelz Oct 20, 2026
13786fb
CUDA: Clear CUDA graphs after legacy memory pool flush (#26771)
bokrosbalint Oct 20, 2026
83c2d01
spec : save the dynamic ngram cache file (#22055)
petersid2022 Oct 20, 2026
522f2b6
ggml-zdnn: introduce flash attention op (#30512)
taronaeo Oct 20, 2026
b684498
HIP/CUDA: Fix reduction in mmf getting tile widths not devisiable by …
IMbackK Oct 20, 2026
119f68d
ggml-cpu : select the NEON fp32->fp16 conversion (#30505)
bernardladenthin Oct 20, 2026
8115b4c
sycl : replace memset due to not always ordered (#30812)
arthw Oct 20, 2026
a01ce9f
opencl: add several gemm variants for small batches and some fixes (#…
lhez Oct 20, 2026
b38960f
ci : remove superfluous webgpu jobs (#30868)
CISC Oct 20, 2026
9df0b73
model : drop the redundant precision overrides in the Kolibri-1 route…
ggerganov Oct 20, 2026
df11702
metal : optimize argmax and pool_1d (#30862)
ggerganov Oct 20, 2026
36140db
mtmd : read LFM2 tiling params from GGUF metadata (#30840)
tdakhran Oct 20, 2026
1ba568c
server: fix missing output_index in Responses API SSE events for comp…
Renzishi Oct 20, 2026
4e20138
server: Router wasn't propagating the client disconnects fully to the…
Me42019 Oct 20, 2026
39da130
ggml-cuda : support ssm_conv kernel size 7 (#30873)
tdakhran Oct 20, 2026
5a9a34d
server : avoid repeated mmproj memory margin addition on sleep/resume…
TANGBUDU Oct 20, 2026
5ba3b9f
mtmd: fix gemma 4 ffn act (#30882)
ngxson Oct 20, 2026
7e3eeee
model : add H2O-Lightning-4B decision model (#30361)
pseudotensor Oct 20, 2026
4a45de4
mtmd : lfm2 tiles and layout match transformers (#30261)
ykhrustalev Oct 20, 2026
b684b49
hexagon: fix VTCM size query on images that need layout out-params (#…
tdakhran Oct 20, 2026
ead9f32
server: Support logprobs with speculative decoding (#27196)
pnb Oct 20, 2026
839a572
qwen: reshape output to 2d instead of 3d (#30823)
am17an Oct 20, 2026
52ffa02
model : support Nemotron-3(.5)-Embed (#30877)
tokinasin Oct 20, 2026
84e66d0
refactor : move LFM2 shortconv and FFN blocks into a shared LFM base …
tdakhran Oct 20, 2026
1242be6
server: add realtime transcription API (#30636)
ngxson Oct 20, 2026
5c0b49a
server : add POST /tts endpoint for TTS models (#26603)
ngxson Oct 20, 2026
9a3b91c
server : fix usage.completion_tokens undercount with n > 1 (#29486)
unbug Oct 20, 2026
339f324
opencl: fix rope program build error with newer Clang (#30893)
bashbaug Oct 21, 2026
493253f
Sycl iq3 crash fix (#30604)
anantshri Oct 21, 2026
8baabd5
llama : scale moe down projection per expert token (#30861)
ggerganov Oct 21, 2026
479a527
ggml-cpu : compute f32 gelu_quick and geglu_quick without the fp16 ta…
boshjerns Oct 21, 2026
96caa80
sycl: collapse duplicate need_check launch bodies in mmq (#30739)
Titaniumtown Oct 21, 2026
74b115c
sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / …
arthw Oct 21, 2026
b531e4c
opencl: add dp4a GEMM variants to vector load activation (#30753)
lhez Oct 21, 2026
0efe8e4
hexagon: neon-vectorized and threaded repack (tiling) (#30901)
max-krasnyansky Oct 21, 2026
ea9e344
metal : fix memory leaks and cleanup logic in GPU capture path (#30890)
nikwen Oct 21, 2026
eb182d8
llama : place d2t by its set_rows use so it can stay on the GPU (#30912)
am17an Oct 21, 2026
824eaec
llama : optimize vocabulary loading (#30679)
mykhailen Oct 21, 2026
f849095
vendor: update BoringSSL to 0.20261020.0 (#30889)
cabelo Oct 21, 2026
261700b
server : forward llama.cpp sampling params on /v1/messages (#30856)
Dgarc359 Oct 21, 2026
0a1ef30
mtmd: add state object for mtmd_gen (#30902)
ngxson Oct 21, 2026
9b56ccf
ggml-opencl : add IM2COL_3D support (#30895)
kurquhar Oct 21, 2026
9febe63
jinja: support hexadecimal string escapes (#30929)
alexey-rsi Oct 21, 2026
0756e83
sycl: q8 fast dequant (#30775)
92es Oct 21, 2026
d454a20
llama: create new fused op for QSA mask (#30640)
am17an Oct 21, 2026
7775ffb
model: Add support for SenseNova U1 model (#28919)
Maphist0 Oct 21, 2026
99fb997
jinja : fix namespace attributes shadowed by dict methods (#30941)
tdakhran Oct 21, 2026
a37cc46
models : add Flint model (FlintForCausalLM) (#30928)
alexey-rsi Oct 21, 2026
c7fc509
opencl: add F32-to-I32 ARGMAX (#30897)
kurquhar Oct 22, 2026
a6b83d4
ui : key draft sidecar badges by model instead of repo (#30919)
allozaur Oct 22, 2026
0682a57
lora : support prism.hadamard folded weights (#30910)
vieduy Oct 22, 2026
84746c5
ui: Add "Show tool call in progress" setting (#30681)
allozaur Oct 22, 2026
4912ac2
dflash : load optional FC input scale (#30945)
hthadicherla Oct 22, 2026
0b1281a
webgpu : add Q2_0 support (#30914)
teddygood Oct 22, 2026
e3797b1
sycl: merge the fp16 and fp32 dequant switches (#30740)
Titaniumtown Oct 22, 2026
dfeea52
kleidiai : add Q8_0 SVE dotprod GEMV and SVE i8mm GEMM kernels (#30808)
chaxu01 Oct 22, 2026
bf039c8
models : Record layer inputs for all granite models (#30957)
gabe-l-hart Oct 22, 2026
e197cef
vocab : use Tekken pre-tokenizer for pixtral rather than the Llama 3 …
tkran2 Oct 22, 2026
d88040c
vocab : add clean_spaces GGUF key (#30940)
tkran2 Oct 22, 2026
7170cad
sycl: fix CONV_2D wrong results for non-contiguous input (#30363)
mctylr-gh Oct 22, 2026
1f4c2ee
ci : be able to reconfigure HTTP port for server pytest workers (#30972)
ggerganov Oct 22, 2026
5f5acfe
ci : disable slow WebGPU tests (#30963)
ggerganov Oct 22, 2026
a5d2227
server : support typed content input for RERANK endpoint (#29625)
timothywang21 Oct 22, 2026
0693dbf
nemotron : support non-standard GQA, BPE tokenizer and domyn-small pr…
martin-cimmino Oct 22, 2026
d772569
vulkan: replace std::aligned_alloc with posix_memalign for termux com…
bill88t Oct 22, 2026
d81d0d8
Multi-column on Intel XMX matrix engines refactored (#29864)
budiony Oct 22, 2026
d9ee007
model : fix qwen4exp layer inputs for DFlash drafters (#30927)
ggerganov Oct 22, 2026
5c19a5a
glm5 : optimize inputs (#30932)
ggerganov Oct 22, 2026
36cbf41
Fix !30793
gaugarg-nv Oct 22, 2026
19ab5a5
clef : include <algorithm> for std::find and std::copy (#30981)
kleinbem Oct 22, 2026
a08ab75
mtmd : add MiniCPM-OCR 1.0 support (#30979)
tc-mb Oct 22, 2026
970b51c
opencl: add depthwise conv2d (#30900)
kurquhar Oct 22, 2026
d814cbb
server: realtime API: send complete after committed audio drains (#30…
ngxson Oct 22, 2026
184f653
server: Substitute rerank placeholder from the original template only…
Pengkun-ZHU Oct 22, 2026
7377bad
sycl: Add graph record and replay (#28725)
malsbat Oct 23, 2026
0a9ef00
sycl: Replace host waits in fattn-mkl with explicit dependencies (#30…
malsbat Oct 23, 2026
643b0cd
hexagon: Op Fusion refactoring, optimizations and new fusions like `B…
max-krasnyansky Oct 23, 2026
f9599d0
vocab : fix Nemotron-3.5-Embed tokenization (#30974)
tokinasin Oct 23, 2026
1ab09d6
Extend probabilistic sampling for DFlash2 drafts (#30663)
praneshgo Oct 23, 2026
f64a498
sycl : pass device into mul_mat_uses_onednn (#31014)
Titaniumtown Oct 23, 2026
f5027fb
CANN: missing `const` qualifiers added for consistency (#30933)
rauletorresc Oct 23, 2026
f00c908
ci : add behavior check workflow (#31036)
CISC Oct 23, 2026
4231b23
ggml : fix time overflow in ggml_time_us (#30969)
frozenblade1224 Oct 23, 2026
e1e3b2c
ui : bump packages to clear npm audit (#31038)
allozaur Oct 23, 2026
36e36e6
hexagon: add CEIL op (#31006)
max-krasnyansky Oct 23, 2026
280141e
CUDA: chunk graphs to amortize graph launch (#30804)
am17an Oct 23, 2026
da2ee1e
vendor : update LibreSSL to 4.4.0 [no ci] (#31049)
angt Oct 23, 2026
5c31142
qwen4exp : build the QSA selection without the kq_mask (#30965)
sanmai Oct 23, 2026
06230cb
allows the usage of 4-10 gpus for stepfun and laguna in tensor paral…
krampenschiesser Oct 23, 2026
9ff0d7e
CUDA: fix build after chunked CUDA graphs (#30804) (#31053)
sreeram-11 Oct 23, 2026
1a24df1
ggm-cuda: Enforce DP4A MMQ tile height multiple of warp size (#30837)
thelittlefireman Oct 23, 2026
9145ddb
hip: fix bf16 WMMA operand vector type for gfx11/gfx12 (#30885)
waltercool Oct 23, 2026
21ec307
common : allow concurrent downloads across processes (#29312)
angt Oct 23, 2026
0ae0e19
hexagon: add SGN op (#30346)
kurquhar Oct 23, 2026
9ae73e6
mtmd : add Soprano TTS support (#28607)
hans00 Oct 23, 2026
edc3538
mtmd : lfm2 resize matches torchvision (#30943)
ykhrustalev Oct 24, 2026
7894e8b
mtmd: move build_inp_temporal_merge to base class (#31065)
ngxson Oct 24, 2026
1ddffe9
HIP: add byte permute fallback when __builtin_amdgcn_perm is unavaila…
aobolensk Oct 24, 2026
8e5a58d
vulkan: ggml src1 precision support (#30656)
0cc4m Oct 24, 2026
6b9ba4f
ci : fix data race between runners sharing the model cache (#31046)
ggerganov Oct 24, 2026
3642238
tests : add speculative rejection draft sampling test (#30213)
danbev Oct 24, 2026
701fd25
ggml : allow MUL_MAT_ID -1 index to skip the computation (#26631)
ngxson Oct 24, 2026
3032209
llama : deny -sm tensor for arches with moe amax rescaling (#31081)
ggerganov Oct 24, 2026
4d32b29
ci : exclude test-speculative-rejection-sampling from macos-latest-ar…
ggerganov Oct 24, 2026
df9beac
tests : fix test-llama-context for large n_batch (#31083)
sanmai Oct 24, 2026
73adf89
minimax-m3 : forward quant scales to the graph (fixes #31063) (#31097)
samgsx Oct 24, 2026
71255fe
ci : pass per-runner results dir to self-hosted jobs on T4 runners (#…
ggerganov Oct 24, 2026
ea1d7d6
fix(ui): align hover card and context popup styling with dropdown men…
allozaur Oct 24, 2026
e168e04
ui: Models Manager as Route + UI improvements (#30988)
allozaur Oct 24, 2026
9a0fde9
rpc : add bounds check in rdma_recv (#30399)
sohaibharraoui Oct 24, 2026
3f03b17
metal : remove mul_mm_id amax rescaling (#30924)
ggerganov Oct 24, 2026
f3269ef
server: refactor server entrypoint (#31113)
ngxson Oct 24, 2026
78a8ae1
hexagon: KV-head team split for multicore HMX flash attention (#31055)
ebateni Oct 24, 2026
d327bf1
jinja : fix select and selectattr argument handling (#31059)
JoeyPatricio Oct 24, 2026
74bca39
model: DSV4.1 flash (#31086)
am17an Oct 25, 2026
b3fa699
chat : reject JSON nested more than 128 levels before walking it (#30…
ykhrustalev Oct 25, 2026
344be0a
ci : update CPU job assignments (#31136)
ggerganov Oct 25, 2026
b083c38
opencl: add initial q2_k support (#30999)
lhez Oct 25, 2026
9a4580b
opencl: add topk and lightning indexer (#31026)
lhez Oct 25, 2026
f0fb598
hexagon: generic dims collapse/reshape for unary, binary and matmul o…
max-krasnyansky Oct 25, 2026
459a1ba
vulkan : one warp tile per subgroup in the small matmul tiles (#30106)
ianloic Oct 25, 2026
e321bda
vulkan: Add a larger tile size for conv3d (coopmat2 only) (#29920)
jeffbolznv Oct 25, 2026
a55c16e
ggml-cuda: fuse qwen MRoPE + set-rows (#31025)
am17an Oct 25, 2026
c49edf0
CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/u…
anujj Oct 25, 2026
7790a92
tools : reject truncated GGUF tensor data in merge (#31044)
shreya-11733 Oct 25, 2026
fa3645f
llama : respect no_alloc in all memory caches (#31087)
ggerganov Oct 25, 2026
f1168ac
grammar: limit recursion depth to prevent stack overflow (#29495) (#2…
Hrushikesh-ramilla Oct 25, 2026
5c965fd
llama-quant : fix 4D tensor quantization using ggml_nrows() and name-…
anthony-wss Oct 25, 2026
63412d8
llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31…
trunk-trick Oct 25, 2026
1a0b56c
ci : disable test-llama-context with fa off in the webgpu (#30977)
yomaytk Oct 25, 2026
2280f0f
qwen: merge tensors on convert (#31028)
am17an Oct 25, 2026
6050b46
tests : adjust default threads for test-llama-context (#31157)
ggerganov Oct 25, 2026
48c9f08
common, grammar: add recursive depth limit, make it a requirement (#3…
ngxson Oct 25, 2026
d854d98
cuda : use std::string instead of VLA for archName (#30463)
angt Oct 25, 2026
e880914
Merge the upstream-merge candidate into current prism
bri-prism Oct 29, 2026
3d53bad
Merge upstream master (d854d98ba) into prism
bri-prism Oct 29, 2026
d0f258c
graph: keep the direct recurrent-state gather when there is one state
bri-prism Oct 29, 2026
0085e51
cuda: handle SWIGLU_CLAMP in the PQ2_0 multi-column fused mat-vec
bri-prism Oct 29, 2026
c2734e3
cuda: handle SWIGLU_CLAMP in the PTQ1_0 warp-per-row mat-vec epilogue
bri-prism Oct 29, 2026
0c7907e
Merge the DSpark script fixes from the previous upstream-merge branch
bri-prism Oct 29, 2026
61cb746
qwen35moe: use narrow_last for the last-layer row crop
bri-prism Oct 29, 2026
c897791
graph: initialize mtp_chain_embd in declaration order
bri-prism Oct 29, 2026
b4c2b0b
speculative: drop two row_bytes locals left unused by the merge
bri-prism Oct 29, 2026
1734d53
tests: build the DSpark eval chat message by field, and skip the DSpa…
bri-prism Oct 29, 2026
c4595c5
tests: expect empty unions to parse as any schema
bri-prism Oct 29, 2026
3b9f6c3
Merge prism (#406, #413, #414, #415) into the upstream merge
bri-prism Oct 29, 2026
5a469f3
cuda: try the fork's exact MoE weighted reduce before upstream's fuse…
bri-prism Oct 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 3 additions & 2 deletions .devops/cann.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ ENTRYPOINT ["/app/tools.sh"]
# ==============================================================================
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

ENTRYPOINT [ "/app/llama-cli" ]

Expand All @@ -155,8 +155,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

HEALTHCHECK --interval=5m CMD [ "curl", "-f", "http://localhost:8080/health" ]

Expand Down
5 changes: 3 additions & 2 deletions .devops/cpu.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -114,8 +114,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
5 changes: 3 additions & 2 deletions .devops/cuda.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -123,8 +123,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
18 changes: 12 additions & 6 deletions .devops/intel.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG ONEAPI_VERSION=2025.3.3-0-devel-ubuntu24.04
ARG ONEAPI_VERSION=2026.1.1-devel-ubuntu24.04
ARG BUILD_DATE=N/A
ARG APP_VERSION=N/A
ARG APP_REVISION=N/A
Expand All @@ -21,6 +21,11 @@ RUN LLAMA_BUILD_NUMBER="$APP_VERSION" npm run build

FROM docker.io/intel/deep-learning-essentials:$ONEAPI_VERSION AS build

RUN echo "Download oneDNN installation package" && \
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/94c3dbac-0852-45be-a57d-21c204cada3e/intel-onednn-2026.0.2.46_offline.sh -O intel-onednn_offline.sh && \
echo "Install oneDNN components" && \
bash intel-onednn_offline.sh -s -a --silent --eula accept

ARG GGML_SYCL_F16=ON
ARG LEVEL_ZERO_VERSION=1.28.2
ARG LEVEL_ZERO_UBUNTU_VERSION=u24.04
Expand All @@ -45,7 +50,7 @@ RUN if [ "${GGML_SYCL_F16}" = "ON" ]; then \
fi && \
echo "Building with dynamic libs" && \
cmake -B build -DGGML_NATIVE=OFF -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DLLAMA_BUILD_TESTS=OFF ${OPT_SYCL_F16} && \
cmake --build build --config Release -j$(nproc)
cmake --build build --config Release -j$((($(nproc)+1)/2))

RUN mkdir -p /app/lib && \
find build -name "*.so*" -exec cp -P {} /app/lib \;
Expand All @@ -59,7 +64,7 @@ RUN mkdir -p /app/full \
&& cp requirements.txt /app/full \
&& cp .devops/tools.sh /app/full/tools.sh

FROM docker.io/intel/deep-learning-essentials:$ONEAPI_VERSION AS base
FROM docker.io/intel/oneapi-toolkit:$ONEAPI_VERSION AS base

ARG BUILD_DATE=N/A
ARG APP_VERSION=N/A
Expand Down Expand Up @@ -102,7 +107,7 @@ RUN mkdir /tmp/neo/ && cd /tmp/neo/ \
&& dpkg --install *.deb

RUN apt-get update \
&& apt-get install -y libgomp1 curl ffmpeg \
&& apt-get install -y libgomp1 curl ffmpeg openssl libssl-dev \
&& apt autoremove -y \
&& apt clean -y \
&& rm -rf /tmp/* /var/tmp/* \
Expand Down Expand Up @@ -141,7 +146,7 @@ ENTRYPOINT ["/app/tools.sh"]
FROM base AS light

COPY --from=build /app/lib/ /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -151,9 +156,10 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/lib/ /app
COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
20 changes: 13 additions & 7 deletions .devops/musa.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
ARG UBUNTU_VERSION=22.04
# This needs to generally match the container host's environment.
ARG MUSA_VERSION=rc4.3.0
# Target the MUSA build image
ARG BASE_MUSA_DEV_CONTAINER=docker.io/mthreads/musa:${MUSA_VERSION}-devel-ubuntu${UBUNTU_VERSION}-amd64
ARG BASE_MUSA_DEV_CONTAINER=registry.mthreads.com/mcconline/musa_sdk:5.2.0-devel-ubuntu${UBUNTU_VERSION}-s5000

ARG BASE_MUSA_RUN_CONTAINER=docker.io/mthreads/musa:${MUSA_VERSION}-runtime-ubuntu${UBUNTU_VERSION}-amd64
ARG BASE_MUSA_RUN_CONTAINER=registry.mthreads.com/mcconline/musa_sdk:5.2.0-runtime-ubuntu${UBUNTU_VERSION}-s5000

ARG BUILD_DATE=N/A
ARG APP_VERSION=N/A
Expand Down Expand Up @@ -37,7 +36,10 @@ RUN apt-get update && \
python3-pip \
git \
libssl-dev \
libgomp1
libgomp1 \
musa-mualg-5-2 \
musa-muthrust-5-2 \
libmthreads-compute

WORKDIR /app

Expand Down Expand Up @@ -80,13 +82,16 @@ LABEL org.opencontainers.image.created=$BUILD_DATE \
org.opencontainers.image.source=$IMAGE_SOURCE

RUN apt-get update \
&& apt-get install -y libgomp1 curl ffmpeg \
&& apt-get install -y libgomp1 curl ffmpeg libmthreads-compute \
&& apt autoremove -y \
&& apt clean -y \
&& rm -rf /tmp/* /var/tmp/* \
&& find /var/cache/apt/archives /var/lib/apt/lists -not -name lock -type f -delete \
&& find /var/cache -type f -delete

# The MUSA runtime image does not register its library directory
RUN echo "/usr/local/musa/lib" > /etc/ld.so.conf.d/musa-runtime.conf && ldconfig

COPY --from=build /app/lib/ /app

### Full
Expand Down Expand Up @@ -115,7 +120,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -125,8 +130,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
12 changes: 6 additions & 6 deletions .devops/nix/package.nix
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
]
&& blas.meta.available,
useCuda ? config.cudaSupport,
useMetalKit ? stdenv.isAarch64 && stdenv.isDarwin,
useMetalKit ? stdenv.hostPlatform.isAarch64 && stdenv.hostPlatform.isDarwin,
# Increases the runtime closure size by ~700M
useMpi ? false,
useRocm ? config.rocmSupport,
Expand Down Expand Up @@ -92,7 +92,7 @@ let

cudaBuildInputs = with cudaPackages; [
cuda_cudart
cuda_cccl # <nv/target>
cccl # <nv/target>
libcublas
];

Expand Down Expand Up @@ -166,7 +166,7 @@ effectiveStdenv.mkDerivation (finalAttrs: {
# `xcrun` is used find the path of the Metal compiler, which is varible
# and not on $PATH
# see https://github.com/ggml-org/llama.cpp/pull/6118 for discussion
__noChroot = effectiveStdenv.isDarwin && useMetalKit && precompileMetalShaders;
__noChroot = effectiveStdenv.hostPlatform.isDarwin && useMetalKit && precompileMetalShaders;

nativeBuildInputs =
[
Expand All @@ -181,10 +181,10 @@ effectiveStdenv.mkDerivation (finalAttrs: {
autoAddDriverRunpath
]
++ optionals (effectiveStdenv.hostPlatform.isGnu && enableStatic) [ glibc.static ]
++ optionals (effectiveStdenv.isDarwin && useMetalKit && precompileMetalShaders) [ xcrunHost ];
++ optionals (effectiveStdenv.hostPlatform.isDarwin && useMetalKit && precompileMetalShaders) [ xcrunHost ];

buildInputs =
optionals effectiveStdenv.isDarwin darwinBuildInputs
optionals effectiveStdenv.hostPlatform.isDarwin darwinBuildInputs
++ optionals useCuda cudaBuildInputs
++ optionals useMpi [ mpi ]
++ optionals useRocm rocmBuildInputs
Expand Down Expand Up @@ -245,7 +245,7 @@ effectiveStdenv.mkDerivation (finalAttrs: {

# Configurations that are known to result in build failures. Can be
# overridden by importing Nixpkgs with `allowBroken = true`.
broken = (useMetalKit && !effectiveStdenv.isDarwin);
broken = (useMetalKit && !effectiveStdenv.hostPlatform.isDarwin);

description = "Inference of LLaMA model in pure C/C++${descriptionSuffix}";
homepage = "https://github.com/ggml-org/llama.cpp/";
Expand Down
24 changes: 14 additions & 10 deletions .devops/openvino.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
ARG OPENVINO_VERSION_MAJOR=2026.3
ARG OPENVINO_VERSION_FULL=2026.3.0.22451.bd8d6542e3c
ARG OPENVINO_VERSION_MAJOR=2026.4.1
ARG OPENVINO_VERSION_FULL=2026.4.1.22982.07f9c262b05
ARG UBUNTU_VERSION=24.04

# Intel GPU driver versions. https://github.com/intel/compute-runtime/releases
ARG IGC_VERSION=v2.38.2
ARG IGC_VERSION_FULL=2_2.38.2+22051
ARG COMPUTE_RUNTIME_VERSION=26.27.39122.11
ARG COMPUTE_RUNTIME_VERSION_FULL=26.27.39122.11-0
ARG IGC_VERSION=v2.41.5
ARG IGC_VERSION_FULL=2_2.41.5+22716
ARG COMPUTE_RUNTIME_VERSION=26.35.39758.10
ARG COMPUTE_RUNTIME_VERSION_FULL=26.35.39758.10-0
ARG IGDGMM_VERSION=22.10.0

# Intel NPU driver versions. https://github.com/intel/linux-npu-driver/releases
ARG NPU_DRIVER_VERSION=v1.35.0
ARG NPU_DRIVER_FULL=v1.35.0.20260722-29947505341
ARG LIBZE1_VERSION=1.28.2-1~24.04~ppa1
ARG NPU_DRIVER_VERSION=v1.38.0
ARG NPU_DRIVER_FULL=v1.38.0.20260910-34487311128
ARG LIBZE1_VERSION=1.32.0-1~24.04~ppa1

# Optional proxy build arguments
ARG http_proxy=
Expand Down Expand Up @@ -90,6 +90,9 @@ RUN bash -c "source ${OpenVINO_DIR}/setupvars.sh && \
cmake -B build/ReleaseOV -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DLLAMA_BUILD_TESTS=OFF \
-DGGML_NATIVE=OFF \
-DGGML_BACKEND_DL=ON \
-DGGML_CPU_ALL_VARIANTS=ON \
-DGGML_OPENVINO=ON && \
cmake --build build/ReleaseOV --parallel "

Expand Down Expand Up @@ -170,7 +173,7 @@ RUN --mount=type=cache,target=/var/cache/intel-npu,sharing=locked \
fi; \
DEB=/var/cache/intel-npu/libze1_${LIBZE1_VERSION}_amd64.deb; \
if [ ! -f "$DEB" ]; then \
wget -q -O "$DEB" https://snapshot.ppa.launchpadcontent.net/kobuk-team/intel-graphics/ubuntu/20260606T100000Z/pool/main/l/level-zero-loader/libze1_${LIBZE1_VERSION}_amd64.deb; \
wget -q -O "$DEB" https://snapshot.ppa.launchpadcontent.net/kobuk-team/intel-graphics/ubuntu/20260830T100000Z/pool/main/l/level-zero-loader/libze1_${LIBZE1_VERSION}_amd64.deb; \
fi; \
mkdir /tmp/npu/ && cd /tmp/npu/ && tar -xf "$TGZ" && cp "$DEB" .; \
apt-get update; \
Expand Down Expand Up @@ -224,6 +227,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app/

Expand Down
5 changes: 3 additions & 2 deletions .devops/rocm.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -136,8 +136,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
1 change: 1 addition & 0 deletions .devops/s390x.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ ENTRYPOINT [ "/llama.cpp/bin/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

WORKDIR /llama.cpp/bin

Expand Down
5 changes: 3 additions & 2 deletions .devops/vulkan.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -117,8 +117,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
5 changes: 3 additions & 2 deletions .devops/zendnn.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -107,8 +107,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
2 changes: 1 addition & 1 deletion .ecrc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"Exclude": ["^\\.gitmodules$", "stb_image\\.h"],
"Exclude": ["^\\.gitmodules$", "stb_image\\.h", "examples/test-cmake/build/", "examples/test-cmake/build-subdir/"],
"Disable": {
"IndentSize": true
}
Expand Down
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ insert_final_newline = unset
trim_trailing_whitespace = unset
insert_final_newline = unset

[vendor/**.patch]
trim_trailing_whitespace = unset

[tools/ui/**]
indent_style = unset
indent_size = unset
Expand Down
2 changes: 2 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ exclude =
# Do not traverse examples and tools
examples,
tools,
# Do not traverse .venv directories
.venv*,
# Do not include package initializers
__init__.py,
# No need to traverse our git directory
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
blank_issues_enabled: true
blank_issues_enabled: false
contact_links:
- name: Got an idea?
url: https://github.com/ggml-org/llama.cpp/discussions/categories/ideas
Expand Down
Loading
Loading