Repository navigation
FEAT: Add native ASSERT prompt generation - #3271
Open
Richard Lundeen (richlundeen) wants to merge 10 commits into
Open
Richard Lundeen (richlundeen) wants to merge 10 commits into
Richard Lundeen (richlundeen) wants to merge 10 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Check group prompt contracts before attack construction. Retry whitespace-only system prompts and use canonical behavioral inputs for seed storage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Preserve complete-group storage checks and canonical named-prompt identity. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Share named prompt slot, parameter, and schema validation with attack construction. Bind adaptive candidates to fresh group-specific attacks before selection, preserve full technique identity and saved choices, and pass prepared inline groups through Execute. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Separate restored-name registration from proposed-name recovery so both retain their assertions within the existing E2E timeout. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Preserve context-based construction, named-prompt binding, child technique labels, and the incoming Operations acceptance tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Restore saved names and source occurrences for adaptive and baseline groups. Reject unbound or mismatched named prompts at the atomic boundary while preserving compound child identities. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ede3dafb-0eaa-472d-9638-2bf9e8661437
Preserve named seed-group prompts and run-scoped technique editing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Roman Lutz (romanlutz)
approved these changes
Oct 24, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Assert (https://github.com/responsibleai/ASSERT) allows users to specify policies using english language and test those policies.
This PR adds native ASSERT-style prompt generation using PyRIT targets, normalizers, and seeds.
AssertGeneratorresearches an English behavior requirement, builds a policy taxonomy, resolves coverage dimensions, and returns validated nativeAttackSeedGroupobjects.SeedPromptinputs. Existing technique factories bind those inputs through constructor-levelAttackAdversarialConfig; existing attacks own JSON replies, feedback, and turn handling.AssertProviderfor memory storage. Group associations, literal text, response schemas, identity, and bounded generation references survive serialization and storage.seed_datasets/generated, preserving the publicTargetObjectiveProviderimport. Add lazy exports and retain MIT attribution to ASSERT v0.3.0, revision04b713fe73879208369750b3bd5ad53544f3eecc(2026-09-04).This PR covers generation only. ASSERT-specific scoring and foundry scenario integration are separate work. No ASSERT runtime dependency or dependency-manifest changes are included.