Repository navigation
Honor DTO_SUBMIT_BOF only on work queues configured with block-on-fault - #45
Merged
Merged
Conversation
A descriptor carrying IDXD_OP_FLAG_BOF is rejected with INVALID_FLAGS (0x11) by a work queue configured with block_on_fault=0, so a caller that asked for DTO_SUBMIT_BOF got a guaranteed failure on such a queue and had to know the queue configuration itself. Record each work queue's block_on_fault attribute at discovery (accel-config, or the sysfs attribute for DTO_WQ_LIST) and set BOF on explicit submits and batch members only when the queue allows it. On a queue without block-on-fault the flag is dropped and a page fault stops the operation partway (0x03), exactly as without the flag, so callers handle one failure mode instead of two. The init trace reports the attribute per work queue. test_async_bof now learns the queue's setting from a BOF copy into untouched pages (completes with block-on-fault, stops at the first untouched page without) and checks that a BOF submit is never rejected; the batch case uses a cold destination so the member is actually stopped and redone by dto_batch_poll on a block_on_fault=0 queue.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A descriptor carrying
IDXD_OP_FLAG_BOFis rejected with INVALID_FLAGS (0x11) by a work queue configured withblock_on_fault=0, so a caller that asked forDTO_SUBMIT_BOFgot a guaranteed failure on such a queue and had to know the queue configuration itself. In practice this means an application has to pick its flags per deployment, and gets silent software fallback when it guesses wrong.This records each work queue's
block_on_faultattribute at discovery (via accel-config, or the sysfs attribute on theDTO_WQ_LISTpath) and sets BOF on explicit submits and batch members only when the queue allows it. On a queue without block-on-fault the flag is dropped and a page fault stops the operation partway (status 0x03), exactly as without the flag, so callers handle one failure mode instead of two.DTO_SUBMIT_BOFthus means "block on faults when the work queue allows it". The init trace reports the attribute per work queue.Batch members settle BOF at
dto_batch_submit, in the same place CC is already stripped for devices without cache-control support, so a batch built before the queue is known gets the right flags.test_async_bofnow learns the queue's setting from a BOF copy into untouched pages (completes with block-on-fault, stops at the first untouched page without) and checks that a BOF submit is never rejected. The batch case uses a cold destination so the member is actually stopped and redone bydto_batch_pollon ablock_on_fault=0queue. The documented 0x11 row indto.his removed; README and TESTING.md are updated.