Repository navigation
feat(debug-files): Upload DXIL shader binaries - #3489
Closed
tustanivsky wants to merge 9 commits into
Closed
tustanivsky wants to merge 9 commits into
tustanivsky wants to merge 9 commits into
Conversation
Detect DXC shader artifacts before handing files to symbolic: - dxil: DXIL containers (DXBC magic); debug ID from the 32-hex file stem, code ID from the Aftermath hash in sentry-shader-manifest.tsv next to the files - dxilpdb: DXC shader PDBs (MSF magic plus ILDN part in the first 64 KiB), which symbolic cannot parse; debug ID from the file stem Add the two kinds as --type values, send an optional code_id in the assemble request, and tolerate object kinds unknown to symbolic in the assemble response.
Keep the manifest as a plain map loaded once per directory, build the DXIL match with an optional ShaderInfo instead of a cache and entry types, and expose code_id and shader_group only through Assemblable. Update the --type help snapshots for the two new kinds.
A UE project uploads about 14,000 shader binaries per build; one assemble request with all of them runs for many minutes on the server and gets cut off by proxies. Split the request in both the first attempt and the poll loop and merge the responses.
Only the DXIL binary is consumed on the server, so the shader PDB type goes until line numbers are supported.
tustanivsky
marked this pull request as ready for review
November 4, 2026 08:18
Member
|
Hello @tustanivsky, and thank you for your contribution! Unfortunately, as stated at the top of our README.md, we are no longer actively developing this CLI, and in particular, we no longer accept feature requests. We have a new CLI, which will replace this one in the near future. Please feel free to open an issue and/or a PR for the new CLI. |
BYK
pushed a commit
to getsentry/toolkit
that referenced
this pull request
Nov 4, 2026
## Summary Sentry can now name the shader that caused a GPU crash, but only if it has the game's compiled shaders. Unreal Engine builds export them as `.dxil` files (one per shader, over ten thousand per game). This PR lets `sentry debug-files upload` upload them as `dxil` debug files and makes the upload path fast enough for sets of that size. This is the toolkit version of getsentry/sentry-cli#3489, which was implemented earlier to test with the Unreal SDK which has not migrated to the Sentry toolkit yet (getsentry/sentry-unreal#1680). ## Changes - The scanner recognises a DXIL container by its header, before the files reach the object parser, which does not know shaders. The debug id is the 32-hex compiler hash the engine uses as the file name. `dxil` joins the `--type` list; `--id` accepts the bare file name or the hyphenated id. Shaders have none of the symbol/unwind/source features the other formats are filtered on, so those filters do not apply to them. - The GPU crash decoder identifies a shader by NVIDIA's own hash of it, which only NVIDIA's Windows library can compute. The Sentry Unreal plugin computes it at build time and writes `sentry-shader-manifest.tsv` next to the shaders. The CLI reads that manifest once per directory and sends the hash and the shader group as the new `code_id` and `shader_group` assemble fields. Without a manifest the shader is uploaded without them. - Assemble requests are sent in batches of a thousand files. - Missing chunks are uploaded in batched requests, packed up to the server's `chunksPerRequest` and `maxRequestSize` limits, with a chunk shared by several files sent once. They were sent one request per chunk, so ten thousand small files took a lot of time. - The assemble request is repeated right after the chunks are delivered, and the processing wait starts from that point. Before, the wait started before the upload; when the upload alone outlasted it, the command exited without the request that starts processing, and reported the files as not found or timed out although every chunk was on the server. The last three points apply to every debug file type. Files that are not shaders otherwise take exactly the old path. `debug-files find` and `check` do not learn about shaders here. ## Possible Follow-ups Shader PDBs would add source file and line context to GPU crashes. They are left out for now because the server cannot use them yet, but the manifest already lists the PDB that belongs to each binary. ## Related - getsentry/sentry#129684 adds the `dxil` debug file format and the assemble fields - getsentry/teapot#22 names the shader from the binary - getsentry/sentry-unreal#1678 exports and hashes the shaders after the cook - getsentry/sentry-cli#3489 is the legacy CLI version of this change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Sentry can now show which shader was running when a game crashed on the GPU, but only if it has the game's compiled shaders. Games built with Unreal Engine export those shaders as DXIL files, one per compiled shader, next to the other build outputs. This PR teaches
debug-files uploadto upload them as debug files, the same way it uploads PDBs or dSYMs for native crashes.Key Changes
--type dxiluploads DXIL shader binaries. A file is recognised by its DXIL container header, and its debug ID is the compiler hash the engine uses as the file name.code_idandshader_groupassemble fields. A binary without a manifest row is still uploaded, just without them.Older servers reject the new type like any other unknown one.
Side Note
A shader set is much larger than a typical set of debug files: a UE 5.8 game produces around ten thousand DXIL files per build, mostly a few kilobytes each. Uploading them one by one was never the problem, since chunk upload and dedup already handle that, but a single assemble request for all of them ran for minutes on the server, which is where the batching comes from. With it, a re-upload of eleven thousand already-known files completes in seconds.
Possible Follow-ups
Shader PDBs would add source file and line context to GPU crashes. They are left out for now because the server cannot use them yet, but the manifest already lists the PDB that belongs to each binary.
Related Items
dxildebug file format and the assemble fields