Skip to content

feat(authoring)!: native topic authoring and scoped Push - #504

Open
nkemms wants to merge 5 commits into
mainfrom
users/nkemd/topic-authoring-native-split
Open

nkemms wants to merge 5 commits into
mainfrom
users/nkemd/topic-authoring-native-split

Conversation

@nkemms

@nkemms nkemms commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Description

Add native DA-GA topic authoring and session-led scoped Push for the selected editable Dev agent.

  • Share the validated selected-agent connection and bounded AgentBuilder operations.
  • Create inactive topics from ordinary YAML, update mapped topics without losing newer edits, and save separately approved instruction corrections.
  • Keep review, consent, operation sequencing, exclusions, recovery and outcome reporting in session guidance. Scripts return bounded operation evidence.
  • Replace legacy Dataverse Push and eval-scenario materialization. Keep evaluation deployment with its existing selected-set owner and direct Power Automate authoring outside ADK.
  • Include authoring entry points, shared presentation and handoff contracts, onboarding, lifecycle telemetry and regression tests.

This is layer 1 of the three-PR split of the integration PR. It targets main. Native testing/debugging follows in layer 2; topic deletion follows in layer 3. Testing handoffs check whether the native runtime is installed and report acceptance pending when absent. Topic deletion remains unavailable here, including through Push and cleanup.

The original PR and branch remain unchanged. The split uses the final integration snapshot, not historical intermediate policies.

Related issue

Source integration: #495

Type of change

  • New feature
  • Breaking change: retire legacy Dataverse Push and eval-scenario materialization.
  • Documentation update
  • Refactor / cleanup

Testing

  • Focused Python suite, including retained legacy-driver compatibility: 942 passed, 1 known source failure.
  • The existing test_every_canonical_capability_is_actually_emitted failure names evaluation_delete, workflow_create, workflow_test and workflow_update, matching the integration source. This split does not change that separate inventory issue.
  • Extension checks: 60 passed.
  • Ruff and whitespace checks passed.
  • Markdown checks passed for the split-specific Push, README, cleanup and shared-contract edits. Broader Markdownlint reports four existing errors in source instructions/onboarding documents, reproduced on the unchanged integration source; unrelated formatting was preserved.
  • No new live-agent writes or runtime trials were performed for the split. Earlier runtime evidence belongs to the complete integration snapshot, not an independently exercised intermediate release.

Checklist

  • Feature code, relevant guidance and tests are kept together.
  • Existing capture workspace and integration branch are preserved.
  • No publication, agent mutation, merge or force-push is included.

Keep native writes bounded and session-led. Include authoring consent, instruction alignment, workspace acknowledgement and operation evidence. Retire legacy Dataverse Push and eval-scenario materialization. Native testing and deletion follow in dependent PRs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 45972120-90da-4a86-b7bb-6c8ce20c4261
Merge current main while preserving Planner routing, scoped topic authoring and owner-specific presentation contracts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 45972120-90da-4a86-b7bb-6c8ce20c4261

@daeunJe0ng Dawn Jeong (daeunJe0ng) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: native topic authoring and scoped Push

Verdict: request changes. Scope safety holds up: Push only accepts an all-topics/ or all-evaluations/ scope, checks each component's ID and schema name, and stops on new or deleted files in scope. One data-loss issue blocks merge, and two items conflict with the design doc (ESS-ADK-Topic-Authoring-Developer-Design, draft dated 2026-11-08; if SharePoint has a newer version, some design notes below may be out of date).

Blocking

1. A stale local draft can overwrite newer edits in Copilot Studio (scripts/topic_authoring.py:620-642)

  • update takes the expected remote state (expectedDialog) from the fresh --expected read and never compares it with .baseline/topics/<file>.
  • Why it matters: if someone edits the topic in Copilot Studio after the last sync, the merge sees "remote equals expected" and sends the old local draft, wiping out that edit. This breaks the design's "Preserved work" property (section 3.3) and the promise in update/SKILL.md:3. plan_workspace_topic_updates (line 446) already uses the baseline correctly.
  • test_update_sends_captured_yaml_with_fresh_read_baseline (tests/scripts/test_topic_authoring.py:441) locks in this behavior.
  • agent_instructions.py update has the same gap: it replaces the whole instructions value without checking the baseline.
  • Suggested fix: stop with a "remote changed since last sync" error when the fresh read differs from the converted .baseline (or force a refreshed review). Add a test where baseline, remote, and draft all differ.

Should fix in this PR

2. Telemetry accepts unchecked model-supplied values (scripts/emit_capability.py:84-89)

  • --agent-id, --outcome, --error-code, and --error-category pass straight through to telemetry; the docstring says values "pass through unchanged".
  • Why it matters: model-written free text (exception messages, record names) can reach remote telemetry. Design sections 4.2.3 and 4.3 forbid new outcome vocabulary and free-text diagnostics.
  • Suggested fix: allowlist the values listed in push-experience.md (as --deploy-target already does), require --agent-id to be a GUID, and add negative tests.

3. A blank account silently picks the first cached account (scripts/setup_existing_da.py:1684)

  • selected_account = account or accounts[0]: with --account "", the code binds to the first cached account, or raises IndexError if none exist.
  • Why it matters: design section 3.1 says the session must not guess an account. A wrong account means the wrong tenant or agent.
  • Suggested fix: reject a blank account explicitly and use account if account is not None else .... Add a test.

4. New test fails on Windows checkouts (tests/scripts/test_authoring_guidance.py:73-86)

  • test_topic_actions_companion_artifacts_remain_byte_identical hashes on-disk bytes. With core.autocrlf=true and no .gitattributes, catalog.provisional.json checks out with CRLF and the hash differs (bfe14cdb... vs the committed 81bbfb41...).
  • Suggested fix: add src/reference/topic-actions/*.json -text to .gitattributes, or normalize line endings before hashing.

5. VS Code extension package not rebuilt (tools/ess-maker-profile/extension/package.json:5)

  • Command titles drop "(Coming Soon)", but the version stays 0.4.37 and the shipped ess-maker-profile-0.4.37.vsix is unchanged.
  • Why it matters: the installer installs the VSIX, so users keep seeing "Coming Soon" and the old Push description.
  • Suggested fix: bump the version, rebuild the VSIX, remove the old one, and add a changelog entry (per tools/ess-maker-profile/README.md).

6. Raw response body in an error message (scripts/minimalbot_evaluation.py, read_components)

  • The error now embeds response={json.dumps(body)}, which reaches stderr and the model's context.
  • Suggested fix: report status, request ID, and service error code instead.

7. push.py topic path is weaker than the new authoring path (scripts/push.py:149, 194-197)

  • It builds its client with MinimalBotEvaluationClient.from_config rather than connect_selected_dev_workspace, so it skips the Dev, editable, tenant, and account checks. The Workday connect actions still call it with --yes.
  • It prints "Updated and verified N" even when no write happened (writeAttempted: false). Design section 3.7 says zero updated components alone does not prove "already current".
  • Suggested fix: route it through the same workspace binding, and report "already current" separately from "written".

Minor / nits

  • agent_instructions.py acknowledge_instruction_baseline rewrites the whole .baseline/agent.mcs.yml with yaml.safe_dump, dropping comments and formatting. Design section 3.6 says unrelated baseline metadata stays untouched. Replace only the instructions value.
  • topic_authoring.py:572-588 blocks same-name creates, while design section 3.2 calls them advisory. The code matches this PR's experience contract, so the design doc probably needs the update.
  • push.py --activate with no matching Workday dialogs returns no_local_changes rather than an explicit "nothing to activate".
  • topics/delete/SKILL.md:7-16 and workflows/delete/SKILL.md:7-16 say "not available, stop" and then "you MUST push the deletion via push.py ... NEVER stop". Topic deletion is rewritten in #506; the workflow delete skill is not covered by any PR in the stack.

Questions

  1. Topic create checks display-name collisions only. What happens if the schema name {agentSchema}.topic.{stem} matches an existing unmapped component? Each run also generates a new uuid4, so a repeated uncertain create makes a second component.
  2. Do install_ootb_topics.py:42,331 (push.py --only-from) and connect/shared/lifecycle-runner.md:863-864 (push.py --only "{ROLLBACK_PUSH_GLOB}") still fit the new all-topics/ scope rule?
  3. Non-DA agents now get "Dataverse Push is retired". Is a migration note planned for this breaking change?

Tests run

python -m pytest tests/scripts/test_topic_authoring.py tests/scripts/test_authoring_guidance.py tests/setup/test_da_setup_router.py -q: all passed except the Windows line-ending test in item 4.

Not verified: live Copilot Studio behavior; the tests mock the AgentBuilder API.

Nkem Dockery and others added 3 commits November 17, 2026 20:24
Validate Push lifecycle telemetry at the CLI boundary, reject blank account and existing schema identities, redact read errors, and report verified no-write results accurately. Clarify retained-draft reconciliation and route prepared sample drafts through Create review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 45972120-90da-4a86-b7bb-6c8ce20c4261
Remove executable deletion instructions for unavailable workflow operations and retain the inactive telemetry capability without an entry point. Normalize companion catalog newlines before source-integrity checks while preserving schema validation and authoring guidance coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 45972120-90da-4a86-b7bb-6c8ce20c4261
Ship version 0.4.38 from the current source, replacing the stale VSIX and aligning the release check and changelog.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 45972120-90da-4a86-b7bb-6c8ce20c4261
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants