Skip to content

Add support for proper lockless rendering, and other related commands - #430

Open
Dan Streetman (ddstreet) wants to merge 71 commits into
microsoft:mainfrom
ddstreet:lockless-rendering
Open

Dan Streetman (ddstreet) wants to merge 71 commits into
microsoft:mainfrom
ddstreet:lockless-rendering

Conversation

@ddstreet

Copy link
Copy Markdown
Contributor

This adds support for lockless rendering, in a controlled per-component transition.

During the transition, the tool will look for a file named lockless-components.txt in the same dir as the azldev.toml file (i.e., the top dir of the repo). If a component name is found inside this file, the component is considered a 'lockless' component; otherwise, the component is a legacy 'locked' component. Since our repo currently has an empty lockless-components.txt file, all our components are currently considered 'locked', and none of these updates to azldev should have any effect at all on any 'locked' component.

Until all components have been transitioned over, this adds a new command azldev comp make-lockless, which adds the specified component name(s) to the lockless-components.txt file, and runs azldev comp refresh-upstream-commit for them, to generate the component's default upstream-commit toml file. After that, the component's 'lock' file will no longer be used for anything (unless the component is removed from the lockless-components.txt file).

After transitioning all components to lockless, we should remove support for 'locked' files from azldev, and remove the azldev comp make-lockless command; and then we can remove the lockless-components.txt file from our main repo.

Most operations are the same for lockless components, but notably:

  • render operations should now work correctly, and should update the release value and generate the changelog entry correctly.
  • render operations are now slightly different, instead of backwards-searching the git commits to try to find text for a changelog entry, azldev will now prompt the user to provide a changelog entry or autorelease git commit message. This may be avoided by providing the message using the new --changelog-comment parameter.
  • render operations now will git commit the rendered files, unless the --no-git-commit parameter is used.
  • render operations now require no uncommitted changes in the repo (although uncommitted changes in the specs/ dir other than the rendering component's dist-git dir are ignored). this can be forced with --allow-uncommitted
  • renders no longer use mock for anything.
  • renders for autorelease components do not pre-rpmautospec the rendered dist-git dir.
  • there is a new command azldev comp dist-git-repo which converts the committed dist-git dir for a component into an industry-standard dist-git repo (at the path specified to --output-dir).

Besides those listed above, I have probably forgot some other changes.

Note this is still undergoing copilot testing, and there may be additional changes required; but this should not impact any components that still use 'lock' files, so we can slowly migrate components to lockless one-at-a-time and address any transition issues as we find them.

Copilot AI balanced review requested due to automatic review settings November 8, 2026 01:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Skipped local components can still have generated TOML files deleted or flagged as stale, and shared single-branch cloning can break locked renders/builds using valid commits pinned on other branches.

2 open issues
What changed in this PR

This PR introduces per-component lockless workflows through lockless-components.txt, allowing gradual migration alongside legacy lock-based operations.

Changes:

  • Add mode-aware configuration, validation, and the make-lockless migration command.
  • Support host-based rendering, release/changelog updates, and committed dist-git preparation, querying, building, and extraction.
  • Update regression tests, agent guidance, and CLI documentation; parallelize Mage validation.

This pull request requires changes. The following issues remain unresolved:

  • Critical — internal/app/azldev/cmds/component/refreshupstreamcommit.go:173: The orphan pass can delete generated TOML files for configured locked local components, or report them as stale under --check-only. Exempt configured locked components while preserving cleanup for genuinely removed components.
  • Moderate — internal/app/azldev/cmds/component/query_lockless.go:62: Querying all components serially clones and rewrites the full project history for each lockless component. Reuse shared repository objects or bulk extraction while retaining separate rpmautospec repositories.
  • Moderate — internal/app/azldev/cmds/component/render_release_carryover.go:39: An autorelease-to-static transition can restore the old %autorelease macro and skip the required static bump. Resolve the previous release to a concrete baseline instead.
  • Moderate — internal/utils/git/git.go:211: Shared single-branch cloning breaks locked workflows pinned to commits unavailable on the selected branch or through tags. Make this behavior opt-in or fetch the pinned commit before checkout.
File Description
schemas/​azldev.schema.json Clarify filtering behavior by lock mode.
pkg/​app/​azldev_cli/​azldev.go Update command-registration comments.
magefiles/​mageutil/​mageutil.go Resolve tools without changing process directory.
magefiles/​magefile.go Parallelize post-build validation.
internal/​utils/​git/​git.go Add single-branch cloning behavior.
internal/​utils/​git/​git_test.go Verify branch-clone arguments with mocks.
internal/​utils/​dirdiff/​types.go Add ignored-path options.
internal/​utils/​dirdiff/​dirdiff.go Apply exclusions during directory traversal.
internal/​utils/​dirdiff/​dirdiff_test.go Cover exclusions and unreadable entries.
internal/​upstreamcommit/​store.go Omit empty generated TOML headers.
internal/​upstreamcommit/​store_test.go Cover compact TOML and quoted names.
internal/​rpm/​spec/​structural_edit.go Preserve Release-tag formatting.
internal/​rpm/​spec/​legacy_edit.go Preserve Release-tag formatting.
internal/​rpm/​spec/​editor.go Share formatting-preserving tag replacement.
internal/​rpm/​spec/​edit_test.go Cover Release formatting across editors.
internal/​rpm/​mock/​mock.go Support and check the rpmautospec plugin.
internal/​rpm/​mock/​mock_test.go Cover plugin enablement and absence.
internal/​providers/​sourceproviders/​sourcemanager.go Forward local-config exclusion options.
internal/​providers/​sourceproviders/​sourcemanager_test.go Cover local-config exclusion.
internal/​providers/​sourceproviders/​localcontentsprovider.go Exclude TOML defining local spec paths.
internal/​providers/​sourceproviders/​localcontentsprovider_test.go Cover defining versus unrelated TOML files.
internal/​projectconfig/​render.go Clarify mode-specific file filtering.
internal/​projectconfig/​project.go Track and validate lockless components.
internal/​projectconfig/​lockmodes.go Read project component selections.
internal/​projectconfig/​loader.go Select merge semantics per component.
internal/​projectconfig/​configfile.go Defer partial lockless component validation.
internal/​projectconfig/​config.go Load selections and retain lock-directory configuration.
internal/​projectconfig/​config_test.go Cover mixed-mode loading and validation.
internal/​lockfile/​store.go Accept consistency-check exemptions.
internal/​lockfile/​show.go Skip ignored historical locks.
internal/​lockfile/​show_test.go Cover ignored corrupt historical locks.
internal/​lockfile/​lockfile.go Exempt lockless components from consistency checks.
internal/​lockfile/​lockfile_test.go Cover exemptions and orphan detection.
internal/​app/​azldev/​env.go Expose per-component lock-mode selection.
internal/​app/​azldev/​env_test.go Cover component selections and internal overrides.
internal/​app/​azldev/​core/​testutils/​testenv.go Update internal-mode comments.
internal/​app/​azldev/​core/​sources/​upstream_provenance.go Export resolver interface and reuse supplied provenance.
internal/​app/​azldev/​core/​sources/​sourceprep_test.go Cover acquisition ordering and overlay effects.
internal/​app/​azldev/​core/​sources/​render_process.py Allow processing without spectool.
internal/​app/​azldev/​core/​sources/​mockprocessor.go Carry spectool-skip settings into JSON.
internal/​app/​azldev/​core/​sources/​bumpspec.go Export RPM-wrapper generation.
internal/​app/​azldev/​core/​components/​resolver.go Skip lock population and validation for lockless components.
internal/​app/​azldev/​core/​componentbuilder/​componentbuilder.go Build directly from prepared dist-git repositories.
internal/​app/​azldev/​core/​componentbuilder/​componentbuilder_test.go Cover dist-git build options and preparation bypass.
internal/​app/​azldev/​cmds/​docs/​agent.go Serve mixed-workflow agent guidance.
internal/​app/​azldev/​cmds/​component/​update.go Restrict lock updates to locked components.
internal/​app/​azldev/​cmds/​component/​update_test.go Cover skipped components and preserved locks.
internal/​app/​azldev/​cmds/​component/​render_test.go Cover new render flags and dirty-worktree handling.
internal/​app/​azldev/​cmds/​component/​render_release_carryover.go Carry committed release state and baserelease definitions.
internal/​app/​azldev/​cmds/​component/​render_host.go Run host RPM tools and route locked mock processing.
internal/​app/​azldev/​cmds/​component/​refreshupstreamcommit.go Filter refresh work to lockless components.
internal/​app/​azldev/​cmds/​component/​refreshupstreamcommit_test.go Cover per-component refresh selection.
internal/​app/​azldev/​cmds/​component/​query.go Combine locked and lockless query workflows.
internal/​app/​azldev/​cmds/​component/​query_lockless.go Extract and process committed dist-git specs.
internal/​app/​azldev/​cmds/​component/​preparesources.go Dispatch preparation by component mode.
internal/​app/​azldev/​cmds/​component/​preparesources_test.go Adapt option-validation coverage.
internal/​app/​azldev/​cmds/​component/​preparerctx.go Select preparation context per component.
internal/​app/​azldev/​cmds/​component/​mockprocessor.go Require mock only for locked rendering.
internal/​app/​azldev/​cmds/​component/​mockproc.go Clarify locked-render dependency comments.
internal/​app/​azldev/​cmds/​component/​makelockless.go Add migration selection and pin refresh.
internal/​app/​azldev/​cmds/​component/​lockmodes.go Filter component sets by lock mode.
internal/​app/​azldev/​cmds/​component/​list.go Remove mode-dependent command construction.
internal/​app/​azldev/​cmds/​component/​legacy.go Remove obsolete no-op command wrappers.
internal/​app/​azldev/​cmds/​component/​legacy_internal_test.go Remove no-op command tests.
internal/​app/​azldev/​cmds/​component/​history.go Skip lockless components in legacy history reports.
internal/​app/​azldev/​cmds/​component/​diffsources.go Remove mode-dependent command construction.
internal/​app/​azldev/​cmds/​component/​component.go Register both workflows and new commands.
internal/​app/​azldev/​cmds/​component/​component_test.go Cover shared registration and history selection.
internal/​app/​azldev/​cmds/​component/​cmdoptions.go Simplify component flag registration.
internal/​app/​azldev/​cmds/​component/​changed_upstreamcommit.go Scope historical input comparisons to lockless components.
internal/​app/​azldev/​cmds/​component/​changed_upstreamcommit_internal_test.go Cover skipped locked comparison inputs.
internal/​app/​azldev/​cmds/​component/​changed_test.go Update mixed-mode help assertions.
internal/​app/​azldev/​cmds/​component/​build.go Dispatch lockless builds to committed dist-git.
internal/​app/​azldev/​cmds/​component/​build_lockless.go Extract dist-git, download sources, and build.
internal/​app/​azldev/​cmds/​component/​build_lockless_internal_test.go Cover lockless builds, rejected flags, and cleanup.
internal/​app/​azldev/​app.go Remove the global lockless CLI selector.
internal/​app/​azldev/​app_test.go Cover removed-flag rejection and pre-parsing.
internal/​app/​azldev/​agentskill/​doc.go Describe mixed-workflow catalogs.
internal/​app/​azldev/​agentskill/​content/​withoutlockfile/​refresh-upstream-commit.md.tmpl Remove the mode-specific refresh template.
internal/​app/​azldev/​agentskill/​content/​withoutlockfile/​comp-toml.md.tmpl Update lockless render and configuration guidance.
internal/​app/​azldev/​agentskill/​content/​withoutlockfile/​build-component.md.tmpl Document committed dist-git workflows.
internal/​app/​azldev/​agentskill/​content/​update-component.md.tmpl Limit update guidance to locked components.
internal/​app/​azldev/​agentskill/​content/​remove-component.md.tmpl Correct query workflow guidance.
internal/​app/​azldev/​agentskill/​content/​comp-toml.md.tmpl Document mixed-mode configuration and rendering.
internal/​app/​azldev/​agentskill/​content/​build-component.md.tmpl Document lockless preparation and builds.
internal/​app/​azldev/​agentskill/​agentskill.go Include refresh guidance in the default catalog.
docs/​user/​reference/​config/​config-file.md Document per-component include ordering.
docs/​user/​reference/​cli/​azldev.md Remove obsolete global flag documentation.
docs/​user/​reference/​cli/​azldev_version.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_repo.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_repo_query.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_repo_compare.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_project.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_project_new.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_project_init.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_package.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_package_list.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image_test.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image_list.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image_inject-files.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image_customize.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image_build.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_image_boot.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_docs.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_docs_markdown.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_docs_agent.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_docs_agent_show.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_docs_agent_install.md Document mixed-workflow agent output.
docs/​user/​reference/​cli/​azldev_config.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_config_generate-schema.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_config_dump.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_component.md List migration, refresh, and dist-git commands.
docs/​user/​reference/​cli/​azldev_component_update.md Document lockless update exclusions.
docs/​user/​reference/​cli/​azldev_component_test.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_component_render.md Document lockless rendering and new flags.
docs/​user/​reference/​cli/​azldev_component_refresh-upstream-commit.md Add refresh command reference.
docs/​user/​reference/​cli/​azldev_component_query.md Document committed lockless query inputs.
docs/​user/​reference/​cli/​azldev_component_prepare-sources.md Document lockless preparation behavior.
docs/​user/​reference/​cli/​azldev_component_metadata.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_component_make-lockless.md Add migration command reference.
docs/​user/​reference/​cli/​azldev_component_list.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_component_history.md Document lockless history exclusions.
docs/​user/​reference/​cli/​azldev_component_dist-git-repo.md Add dist-git extraction command reference.
docs/​user/​reference/​cli/​azldev_component_diff-sources.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_component_changed.md Document per-component comparison selection.
docs/​user/​reference/​cli/​azldev_component_build.md Document lockless build requirements.
docs/​user/​reference/​cli/​azldev_component_add.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_completion.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_completion_zsh.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_completion_powershell.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_completion_fish.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_completion_bash.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced_wget.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced_mock.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced_mock_shell.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced_mock_build-rpms.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced_mcp.md Remove obsolete inherited flag documentation.
docs/​user/​reference/​cli/​azldev_advanced_download-sources.md Remove obsolete inherited flag documentation.
docs/​user/​README.md Link extraction and per-component migration guides.
docs/​user/​how-to/​extract-dist-git-repo.md Explain committed history extraction and safety.
docs/​user/​how-to/​build-component.md Add lockless build workflow requirements.
docs/​user/​explanation/​config-system.md Explain per-component merge semantics.
docs/​developer/​reference/​build.md Document parallel Mage validation.
.github/​instructions/​agent-skills.instructions.md Update catalog maintenance guidance.

Review effort: Balanced

Comment thread internal/utils/git/git.go Resolved
@ddstreet

Copy link
Copy Markdown
Contributor Author

i'll get my copilot to investigate the above issues that copilot found

@ddstreet

Copy link
Copy Markdown
Contributor Author

I also have a azurelinux repo branch I'm using for testing here: https://github.com/ddstreet/azurelinux/commits/lockless-testing/

Copilot AI balanced review requested due to automatic review settings November 8, 2026 03:03
@ddstreet

Copy link
Copy Markdown
Contributor Author

i'll get my copilot to investigate the above issues that copilot found

pushed 4 new commits to address the issues copilot found above.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The lockless preparation path can delete an existing output directory during a forced dry run because cleanup occurs before the dry-run check.

1 open issue
2 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 8, 2026 03:34
@ddstreet

Copy link
Copy Markdown
Contributor Author

added commit to address last round of feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Lockless builds reintroduce explicitly undefined macros, and source preparation can fail when a pinned upstream commit exists only on another branch.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Medium severity Apply undefine semantics to lockless mock options

internal/​app/​azldev/​cmds/​component/​build_lockless.go:62

Lockless builds pass the raw With, Without, and Defines values to mock without applying Build.Undefines. For example, an inherited defines.dist = ".azl3" with undefines = ["dist"] is correctly omitted from the rendered macros file, but this path reintroduces it through --define dist .azl3 in both SRPM and RPM builds. Filter the options using the same undefine semantics as buildMacrosMap before forwarding them, so transitioning a component does not change its effective build macros.

Medium severity Fetch pinned commits missing from the single-branch clone

internal/​providers/​sourceproviders/​fedorasourceprovider.go:133

Lockless source preparation fails when an explicit upstream pin is reachable only from another branch and is not fetched through a tag. The single-branch clone omits that commit, and checkoutTargetCommit never fetches it before checkout. Keep a multi-branch clone when effectiveCommit is set, or explicitly fetch the pinned commit.

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 8, 2026 03:44
@ddstreet

Copy link
Copy Markdown
Contributor Author

added 4 more commits based on copilot investigation

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Critical findings omitted from the published review involve incorrect macro resolution and source retrieval and pinned-commit checkout failures, requiring human assessment.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 8, 2026 05:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Unpublished findings identify a critical regression in cross-branch commit pinning and failures on ignored unreadable directories, warranting human assessment.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 8, 2026 06:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to evaluate unpublished findings concerning cross-branch pinned-commit checkout failures and lost error diagnostics when the mock probe fails.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 8, 2026 06:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because unpublished findings identify pinned-commit checkout failures and ignored macro undefines that can disrupt source preparation and builds.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 8, 2026 16:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because lockless builds forward raw options that can reinstate macros excluded by build.undefines, causing build settings to diverge from the rendered spec.

No open issues 🎉

Review effort: Balanced

@ddstreet

Copy link
Copy Markdown
Contributor Author

Copilot can you check all your previous reviews and comments, and give a summary of anything you still see as needing action?

Prepare lockless components directly in the requested output directory
using the locked source acquisition, macro, overlay and hash pipeline.
Read release/changelog history from the configured rendered-specs tree
without generating synthetic history or committing prepared changes.

Honor source-skipping and missing-hash flags, and remove Git metadata only
after successful preparation when --without-git is requested. Preserve
partial output on failure and use the shared --force output guard.

Update command help, generated docs, agent skills and regression tests.

Validation: mage fix all, mage unit and mage docs passed. Quality checks
in mage all passed; scenario tests and mage scenarioUpdate remain blocked
by the existing Docker-provider/$XDG_RUNTIME_DIR initialization failure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Resolve the current upstream commit and distro from working-tree TOML
during lockless source preparation. Continue reading the previous render's
configuration and component dist-git snapshot from committed history.

Keep render's committed-input behavior unchanged. Cover modified and
untracked TOML and overlay inputs, and verify that dirty dist-git specs
and sidecars do not affect release/changelog baselines.

Update user documentation, generated help and emitted agent skills.

Validation: mage fix all, mage unit and mage docs passed. Quality checks
in mage all passed; scenario tests and mage scenarioUpdate remain blocked
by the existing Docker-provider/$XDG_RUNTIME_DIR initialization failure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Limit the lockless render preflight to changes outside the configured
project.rendered-specs-dir and changes in selected component dist-git
directories. Ignore changes in unrelated rendered output while keeping
the --allow-uncommitted override and locked-only behavior unchanged.

Use literal Git pathspecs to preserve path boundaries, handle renamed
files and inspect the whole repository from nested project directories.
Update regression tests, generated help, user docs and agent skills.

Validation: mage fix all, mage unit and mage docs passed. Native Git
scope checks passed all 34 cases. Quality checks in mage all passed;
scenario tests and mage scenarioUpdate remain blocked by the existing
Docker-provider/$XDG_RUNTIME_DIR initialization failure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Exempt configured locked components from generated upstream-commit orphan
checks and pruning, including local components. Keep cleanup for components
that have actually been removed.

Cover normal refresh and check-only preservation alongside genuine orphan
cleanup with in-memory regression tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Restore branch selection without excluding commits on other branches so
locked renders and builds retain support for their existing pins.

Opt into single-branch fetching explicitly for lockless preparation and
snapshot identity resolution. Cover both clone modes and option forwarding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Calculate the previous complete release from committed component history
instead of restoring its autorelease macro into a new static spec. Resolve
its generated changelog before the normal static bump as well.

Preserve version-reset and no-dist behavior, propagate calculation errors,
and remove temporary repositories on both success and failure. Cover both
spec editors and document the extraction prerequisite.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Clone one immutable project snapshot for each bulk query, then extract
component histories concurrently using shared objects instead of repeating
full project clones serially.

Repack each filtered repository and remove its alternate object dependency
before passing it into Mock. Cover bulk and mixed queries, extraction
failures, independent object stores, and scratch cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
The lockless prepare-sources path called CheckOutputDir before checking
for dry-run mode. With '--force', that helper removes a nonempty output
directory through the non-dry-run-aware filesystem, so a dry run could
delete existing output.

During a dry run, still reject a nonempty output directory when
'--force' is not set, but return before any cleanup or preparation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ae64263c-02b0-4e70-89b2-030ec0389a00
Restore non-symlink collision protection for locked render aliases while
retaining replacement of occupied alias paths for lockless components.

Cover file and directory collisions in both modes and retain the existing
relative and dangling-symlink regressions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Restore recursive copying with preserved file modes for locked components.
Keep rename-based publication and staged Git history for lockless renders.

Cover cross-device rename failures, staged-file retention, and permissions
with in-memory filesystem tests for both modes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Restrict root Git and rebuild-marker exclusions to lockless rendering.
Remove staged Git metadata before comparing locked output, including when
file filtering is disabled, without changing existing output in check-only.

Cover locked and lockless metadata drift, staging cleanup, and nested
markers using the render completion path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
Restore normalized Release tag updates by default in both spec editors.
Opt into preserving casing and alignment for lockless preparation,
changelog carryover, autorelease transitions, and source overlays.

Test both editor modes and mutation APIs, plus source-preparer overlays
and generated release bumps in locked and lockless modes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3dc8e9aa-35cd-42a2-9377-a73317ee4a51
diff-sources builds its source preparer without lockless history options,
so Release overlays on lockless components were rewritten as
'Release: <value>' even though render and build preserve the original
casing and spacing.

Add a preparer option that requests preserved Release formatting and set
it for lockless components in diff-sources. Locked components keep the
normalized rewrite.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 16503351-1bf3-4693-96f1-26283e4fc641
Add UpdateExistingTagFunc, which computes each matching tag's new value
from its current value, and GetTags, which returns every matching tag
value in spec order. Both are implemented for the legacy and structural
editors and exposed on the Spec facade.

UpdateExistingTag is now a wrapper that returns the same value for every
tag, so its output and errors are unchanged for existing callers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 16503351-1bf3-4693-96f1-26283e4fc641
Lockless rendering read only the last Release tag and wrote its updated
value to every Release tag. Specs with conditional Release tags, such as
yara's '%if %{with release}' / '%else' pair, had every branch replaced
by the last one, changing the default build's NEVR.

Compute the autorelease base once and apply it to each Release tag,
keeping each tag's own options. Static carryover now copies committed
Release values by position and fails when the upstream and committed
tag counts differ. Specs that mix '%autorelease' and static Release
tags, and committed specs with differing '%autorelease' tags converted
to a static release, now fail with a specific error.

Locked components do not use this code path. Lockless specs with a
single Release tag render exactly as before.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 16503351-1bf3-4693-96f1-26283e4fc641
Historical project snapshots are written against older config schemas.
Strictly decoding them fails once azldev drops a field, for example the
removed [test-suites] table in older images.toml files. That broke the
lockless render upstream history lookup for every component last
rendered before the removal, so render commits fell back to a generic
"Update <name> to version <ver>" message instead of the upstream log.

Always ignore unknown fields when loading historical configuration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8b620ba1-9249-46b7-bfb4-78616d6f1ec1
Lockless render validation queried existing and new source NEVRs with plain
rpmspec, which expands an unprocessed `%autorelease -b N` to N. Builds
instead filter the rendered directory's history and let rpmautospec count
commits, so valid base bumps were rejected (e.g. SymCrypt, telegraf) and
the check did not reflect the release actually built.

For unprocessed %autorelease specs, reconstruct the filtered rendered
history and run `rpmautospec process-distgit` before querying. The existing
spec uses committed history; the new spec adds the prepared output as the
next commit unless its tree is unchanged. Merge-free histories are replayed
into a temporary repository that reads project objects via alternates and
never writes to the project; histories with merges fall back to cloning and
running `git filter-repo --subdirectory-filter`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8b620ba1-9249-46b7-bfb4-78616d6f1ec1
rpmdev-bumpspec bumps every line starting with 'Release:', including
text inside %changelog, such as a quoted spec snippet in a previous
entry. Lockless renders then rewrote that historical changelog text.

Before running rpmdev-bumpspec for a lockless component, temporarily
mask %changelog lines that start with 'Release:' and restore them
afterward, also restoring the original spec if the bump fails. Specs
without such lines are bumped exactly as before, and locked components
are unaffected because they use a separate bumpspec path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 16503351-1bf3-4693-96f1-26283e4fc641
Use the source manager to retrieve configured payloads before downloading
remaining dist-git manifest entries from lookaside. Preserve download-origin
fallback, custom generation, hash validation, and disable-origins policy
without replaying overlays or replacing the committed rendered spec.

Cover origin downloads with and without a manifest, disabled origins, and
custom-provider dispatch. Update build guidance for both skill catalogs.

Validation: mage fix all, mage unit, and mage all build/quality checks
passed. Container scenarios were blocked by Docker socket permissions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Compare each component's committed rendered tree before filtering unchanged
inputs. Include spec, sidecar, manifest, and no-change-rebuild marker changes
so render-only commits enter all-component rebuild selection. Continue to
report sources-change only for manifest content changes.

Cover output-only changes, added/removed renders, unchanged component trees,
and unrelated changes using in-memory Git repositories. Document the
additional change-detection input.

Validation: mage fix all, mage unit, and mage all build/quality checks
passed. Container scenarios were blocked by Docker socket permissions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Include captured stderr when the plugin prerequisite probe fails. Keep the
original error wrapped so callers can still inspect its exit status.

Add a regression test for a nonzero probe exit that reports diagnostics and
prevents the SRPM build from starting.

Validation: mage fix all, mage unit, and mage all build/quality checks
passed. Container scenarios were blocked by Docker socket permissions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Do not limit clones to one branch when the component has an effective
upstream commit. Explicit pins and locked commits may exist only on other
branches and cannot be checked out from a single-branch clone. Keep the
single-branch optimization for floating sources.

Test generated clone arguments and checkout targets for floating, explicit,
and locked pins without invoking external commands. Document pin behavior.

Validation: mage fix all, mage unit, and mage all build/quality checks
passed. Container scenarios were blocked by Docker socket permissions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Stop forwarding live build defines and with/without flags to Mock when
building committed dist-git. The rendered spec already loads the resolved
macro file, including TOML undefine semantics. Remove the redundant builder
option path for both SRPM and binary builds while retaining local repos,
rpmautospec, and no-check behavior.

Test that conflicting live macro settings do not reach Mock or overwrite
the committed macro file. Document the committed-spec source of macros.

Validation: mage fix all, mage check all, and the full unit suite passed.
The build and default quality checks passed; Docker-enabled container
scenarios are still running separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Emit resolved build and provenance macros as sorted, lazy %define entries
before user overlays. Do not write an .azl.macros file or inject its load
directive and Source9999 tag for lockless components.

Keep file-based macro loading unchanged for locked components. Cover empty
values, deferred expansion, with/without flags, and skipped overlays. Update
CLI documentation and emitted agent skills for both workflows.

Unit, quality, and changed-line mutation checks pass. Scenario failures for
stale global-flag and schema snapshots also reproduce before this change.
Network-dependent scenarios were not completed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Regenerate local and container help, MCP, and schema snapshots to match
removal of the global --without-lockfile flag and the updated lockless
file-filter description.

Verify with mage scenarioUpdate and mage scenario, both using
TESTING_DISABLE_NETWORK_TESTS=1. The full network-dependent suite remains
unverified after its earlier stall during Mock chroot cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 83cf8df4-30c0-474c-8dca-e3b3a2e54873
Return build results directly and wrap errors from the shared Git path helper.\n\nValidation: mage docs, mage unit, and mage check all passed.\nContainer scenarios are blocked by Docker socket permissions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7e9e3287-a989-4992-970c-595c177a8505
Copilot AI balanced review requested due to automatic review settings November 12, 2026 17:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

End-to-end validation with real RPM/rpmautospec, git-filter-repo, and Mock tooling is unavailable, leaving release/changelog correctness, SRPM history handling, and compatibility across locked and lockless components unverified.

1 open issue

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants