Repository navigation
ggml-cuda: Add GCN HIP sparse fattn - #30370
thelittlefireman wants to merge 4 commits into
Conversation
a9d72b5 to
e835868
Compare
1ad19e5 to
2dcd206
Compare
|
/bot review |
Automated code reviewReview of PR #30370 (ggml-cuda: Add GCN HIP sparse fattn). Checklists run: General, ggml/backend, Security, Tests (touches Summary of the changeThe PR wires the existing sparse flash-attention infrastructure ( CorrectnessI verified the tricky parts and they check out:
No correctness bug found. Will slow the review(point 1) Missing perf data. The PR description says "TODO add benchmark". Per project convention, new kernel paths are expected to come with concrete numbers (throughput on realistic shapes). This matters doubly here because (point 2) PR description is essentially empty. No overview of why GCN needs this (no mma available pre-CDNA, dense tile reads the whole cache), no linked issue or related prior work (the NVIDIA/CDNA mma sparse path). Also the template requires describing how AI was used when disclosing "YES" - please fill that in. (point 3) (point 4) No (point 5) Test duplication across backends. The ~26 new eval cases (tests/test-backend-ops.cpp:13152+) run on every backend in CI, but only exercise the new sparse path on GCN hardware, which CI does not have; on other backends several shapes (256/256, 512/512, 576/512 with Nits(point 6) In (point 7) Notes
This review was generated automatically by pi coding agent using |
Assisted-by: ChatGPT
Assisted-by: ChatGPT
89cd77e to
2c22414
Compare
Assisted-by: ChatGPT
Overview
Enable sparse at fattn on GCN. This could be extended to other arch without wmma.
Additional information
qwen QSA grid (256/256, GQA 12, F16 K/V, n_kv_max=2048)
256/256 GQA 12, KV=32768, n_kv_max=2048
(decode nq=1, ~25% budget)
End-to-end model benchmarks (llama-bench, t/s, higher is better)
Requirements