Skip to content

fix(evaluations): migrate maker connection discovery to PPAPI - #514

Draft
Apurva Banka (apurvabanka) wants to merge 1 commit into
mainfrom
fix/ppapi-maker-connections-7985151
Draft

Apurva Banka (apurvabanka) wants to merge 1 commit into
mainfrom
fix/ppapi-maker-connections-7985151

Conversation

@apurvabanka

Copy link
Copy Markdown
Contributor

Description

Migrate Dataverse-backed evaluation maker connection discovery from the legacy PowerApps API to the supported environment-scoped PPAPI /connectivity/connectors/{connector}/connections route. This covers Copilot Studio profile discovery for listing/running evaluations and optional agent-tool connection discovery.

  • Reuse the ESS-owned 417219b4-3a7d-42a2-bdb1-972bd8281a02 public client and existing authentication helpers with the named delegated Connectivity.Connections.Read scope and shared token-cache path.
  • Bind discovery to the configured environment/ring and authenticated tenant/account; preserve Connected-status checks, explicit profile validation, creator/shared visibility, and connection selection.
  • Opt evaluation discovery into complete pagination with same-environment/same-resource continuation validation and loop detection. A denied or malformed later page fails discovery instead of returning partial success. No fixed 1,000-item page size is assumed.
  • Support Default-<GUID> environment identities using the existing host derivation.
  • Preserve existing native readiness callers' single-attempt behavior. Admin discovery, BAP/Flow/Dataverse authentication, and the native MinimalBot evaluation transport are unchanged.

The maker contract is recorded in ADO task 7985149. Environment-wide delegated admin discovery remains unresolved and is deliberately not substituted with maker-visible results or S2S authentication. This is only the maker portion of task 7985151; neither task nor the parent story is completed by this PR. No new permission grant or deployment is included; existing-grant reconciliation and controlled live verification remain separate.

Telemetry impact review: no telemetry impact. No CLI entry point, outcome/status bucket, stable telemetry field, scope/category dimension, or emitter contract is added or changed.

Related issue

Refs ADO task 7985151 (maker portion only) and contract task 7985149.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor / cleanup

Testing

All commands ran from the repository root. Tests used ESS_ADK_TELEMETRY=off; new discovery tests use fake MSAL, temporary cache state, and blocked network requests.

  • python -m pytest tests\scripts\test_evaluation_runs.py tests\scripts\test_agentbuilder.py tests\flightcheck\test_pp_admin_client.py tests\flightcheck\checks\test_servicenow_da_hrsd.py tests\flightcheck\checks\test_servicenow_da_itsm.py -q --tb=short — passed: 401 tests.
  • python -m ruff check solutions\ess-maker-skills\scripts\agentbuilder.py solutions\ess-maker-skills\scripts\evaluation_runs.py tests\scripts\test_agentbuilder.py tests\scripts\test_evaluation_runs.py — passed.
  • python -m compileall -q solutions\ess-maker-skills\scripts\agentbuilder.py solutions\ess-maker-skills\scripts\evaluation_runs.py — passed.
  • git --no-pager diff --check — passed.

Focused cases cover exact client/scope/authority and account selection, wrong-account/tenant and auth-denial failures, target binding, shared-profile selection, default environments, short/empty continuation pages, unsafe/repeated continuations, and failures after a successful first page. Existing admin-client and native ServiceNow connector regressions remain passing.

The owner-linked current OpenAPI collection/item schema was inspected while implementing the client behavior. No live tenant sign-in/API call, permission deployment, evaluation start, or full E2E authorization verification was performed. Full repository suites were not run.

Checklist

  • My code follows the existing style
  • I have added/updated tests where applicable
  • I have updated documentation as needed

Static validation (samples/ only)

Validation
- YAML parse: N-A
- AdaptiveDialog kind: N-A
- XML parse: N-A
- Filename convention (new): N-A
- Folder convention (new, incl. README.md): N-A
- Diff scope (samples/ only): N-A
- Secrets / internal URLs: N-A

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 61fd0c13-7208-4a76-ab52-1029d3184801
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant