Skip to content

ggml-cpu : AVX-512 VNNI and AVX2 vec_dot for PTQ1_0 x Q8_0 on x86 (rebased #250, about 2x CPU) - #392

Merged
bri-prism merged 1 commit into
prismfrom
rebase/250-on-prism
Oct 24, 2026
Merged

bri-prism merged 1 commit into
prismfrom
rebase/250-on-prism

Conversation

@bri-prism

Copy link
Copy Markdown
Collaborator

Rebased version of #250 by @lenny76. The original head conflicts with #248 (duplicate function definition); this is the author's commit rebased onto prism with no other changes.

AVX-512 VNNI, AVX2+AVX-VNNI and AVX2 vec_dot for PTQ1_0 x Q8_0 on x86. Each tier runs its own path (checked in the disassembly).

Validation on prism:

Follow-up nit for the author: add __AVX512DQ__ to the AVX-512 guard.

PTQ1_0 had only the generic scalar vec_dot on x86. Add SIMD kernels:
every 128-bit lane holds one source run of trit bytes times one power
of 3, so the unpack needs one in-lane shuffle (no VBMI). Trit t is
(3*v)>>8, computed with two byte compares; the dot uses t in {0,1,2}
and subtracts sum(y).

- AVX-512BW + VNNI: 64 weights per vector, dpbusd
- AVX2: one q8_0 sub-block per vector, dpbusd with AVX-VNNI or
  AVX512-VL VNNI, else maddubs + madd

Other x86 CPUs keep the generic kernel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bri-prism
bri-prism merged commit 30bc022 into prism Oct 24, 2026
7 checks passed
@github-actions github-actions Bot added the ggml label Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants