Skip to content

feat(comply): fill TODO fields from a provider answers file - #136

Merged
lab700xdev merged 2 commits into
mainfrom
feat/comply-answers
Nov 15, 2026
Merged

lab700xdev merged 2 commits into
mainfrom
feat/comply-answers

Conversation

@lab700xdev

Copy link
Copy Markdown
Contributor

What

aisbom comply packs leave most fields as TODO (provider) because a file scan cannot read them (provider identity, release dates, training process, FDA support dates). Today a provider fills them by hand and the next run overwrites that work.

This adds a provider-maintained answers file:

aisbom comply ./models --framework eu-ai-act --write-answers-template comply.yaml
# fill in comply.yaml, then on every run:
aisbom comply ./models --framework eu-ai-act --answers comply.yaml

How it behaves

  • Every Annex XI and Article 53 field has a stable id (76 in all, listed in docs/comply.md). The id registry is built from the document builders, so ids, docs and template can't drift.
  • Answers are written as given and tagged (provider-supplied). A value the scan read is never replaced: both appear, scan first, and the pack README lists those fields.
  • The pack README counts scan-read, provider-supplied and TODO fields separately.
  • FDA: components entries fill support_level / end_of_support_date for every component that matches all given keys (name, version, purl, sha256, bom_ref). component-support.csv gains an additive support_source column.
  • The file is validated before the scan: invalid YAML, duplicate keys, unknown top-level keys or field ids, non-scalar values and component entries without a match key exit 1 and list every problem. Unknown ids get a "did you mean" suggestion.
  • Warnings, not errors: a component entry that matches nothing (hashes and versions change between releases), and verdict words such as "compliant" in provider text, which is written as given.
  • --write-answers-template refuses to overwrite an existing file.
  • Pack READMEs, the CLI closing line and the main README point to the workflow. The template links the docs online, because a pip install ships only the package.

Compatibility

  • Without --answers, a pack differs from main only by the template version line (2026-11-15), a new "Filling in the TODOs" section in the pack README, and the empty support_source CSV column (checked by diffing old vs new packs for demo_data).
  • PyYAML is now a declared dependency. It was already installed for every user via spdx-tools; the lockfile changes only its content hash. Loading uses a SafeLoader subclass.
  • No telemetry change. The answers file is read locally and never sent.

Verification

  • poetry run pytest --cov=aisbom --cov-fail-under=85: 1651 passed, 94.39% (aisbom/comply.py 99%). 57 new tests in tests/test_comply_answers.py.
  • aisbom bypass-scorecard --check passes.
  • CI smoke test chore(deps-dev): bump pytest-cov from 6.3.0 to 7.0.0 #14 now writes a template, fills one field, reruns with --answers and checks the tagged value and README count.

`aisbom comply --answers comply.yaml` merges a provider-maintained YAML
file into the pack, so answers a scan cannot read are written once and
kept across runs. `--write-answers-template PATH` writes a commented
starter file with every field id and the scanned components.

- Every Annex XI and Article 53 field has a stable id, derived from the
  builders themselves; docs/comply.md lists all 76.
- Answers are tagged (provider-supplied). A scan-read value is never
  replaced: both are shown, scan first, and the pack README lists them.
- The README index counts scan-read, provider-supplied and TODO fields
  separately.
- FDA component-support.csv takes support_level / end_of_support_date
  per component (matched on name, version, purl, sha256 or bom_ref) and
  gains an additive support_source column.
- Malformed YAML, duplicate keys, unknown ids and bad values fail with
  exit 1 before the scan, listing every problem. Unmatched component
  entries and verdict words in provider text are warnings.
- The template refuses to overwrite an existing file.
- Pack READMEs, the CLI closing line and the main README point users to
  the workflow; the template links the docs online.
- PyYAML declared as a direct dependency (already installed via
  spdx-tools; lockfile content hash only). TEMPLATE_VERSION 2026-11-15.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T04:08:42.844013Z 93271c3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93271c3254

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aisbom/cli.py Outdated
Comment on lines +1821 to +1822
if write_answers_template:
Path(write_answers_template).write_text(comply_mod.answers_template(doc))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject template paths that collide with generated pack files

When the template path names a pack artifact that does not exist before the run, such as --output-dir pack --write-answers-template pack/sbom.json, the initial existence check passes. The command then writes the SBOM and overwrites it here with YAML, returning success with a corrupted documentation pack. The same issue affects the generated Markdown and FDA artifacts. Reject paths that resolve to pack output files before writing, or create the template exclusively so an artifact created during the run cannot be overwritten.

Useful? React with 👍 / 👎.

… files

`--write-answers-template pack/sbom.json` passed the exists check before
the scan, then overwrote the freshly written SBOM with YAML. An answers
file sitting where the pack writes (e.g. pack/README.md) would likewise
be destroyed by the run.

Both paths are now resolved against every file the pack writes (VEX and
CSV included) and refused with exit 1 before scanning. The template is
also created exclusively, so a file that appears during the run is not
overwritten. A test keeps the guarded file list equal to what the pack
actually writes.
@lab700xdev
lab700xdev merged commit 4ac6ab5 into main Nov 15, 2026
2 checks passed
@lab700xdev
lab700xdev deleted the feat/comply-answers branch November 15, 2026 04:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant