Skip to content

Fix FlightCheck source attribution for checkpoint and profile runs - #494

Open
amilandi wants to merge 3 commits into
mainfrom
amilandin/fix-flightcheck-invocation-source
Open

amilandi wants to merge 3 commits into
mainfrom
amilandin/fix-flightcheck-invocation-source

Conversation

@amilandi

@amilandi amilandi commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix FlightCheck invocation-source attribution and address the follow-up review on merged PR #486.

  • Default checkpoint/profile and other direct CLI runs to invocationSource=cli. Preserve explicit adk, installer, and connect sources, and explicitly pass connect from the Connect lifecycle runner and ServiceNow controller.
  • Honor FlightCheck's parsed --no-telemetry flag for the entire SDK invocation, including authentication session starts, API events, and script crash diagnostics. Nested boundaries cannot re-enable it; suppression is checked before starting asynchronous transport threads and restored afterward. ServiceNow and maker-diagnostics child runs retain their explicit opt-out.
  • Make every registered CLI's telemetry-boundary import fail-open, including import-time runtime/syntax failures, without replacing the original CLI exception or exit behavior.
  • Classify errors through the nearest mapped base in __mro__, covering missing files, permissions, JSON decoding, subprocess failures, and requests HTTP errors. Payloads still contain only fixed messages/codes, never exception text or class names.
  • Register flightcheck_setup.py, and require every __main__ script to be registered or explicitly excluded. Removed scripts must also leave the registry/exclusion inventory.

Telemetry impact

The invocation-source fix changes attribution only for future targeted runs without an explicit source. Historical connect events cannot be retroactively distinguished.

The follow-ups reuse the existing script-error event and fields; subprocess errors gain a fixed process category/code. No global exception hooks or arbitrary error text are introduced. Schema stays at main's 1.8.0; #436 owns its pending 1.9.0 bump and must retain the merged crash-diagnostic additions. #495 must remove its deleted CLI registrations/exclusion together with the scripts.

Verification

Focused offline runs passed: 412 SDK tests, 265 FlightCheck CLI/checkpoint/profile/consent tests, and 672 Connect/lifecycle/maker-diagnostics tests. Coverage includes asynchronous opt-out, nested suppression, consent restoration, all 33 registered entry-point guards with failed telemetry imports, exception preservation, fixed-text MRO classification, and CLI coverage inventory.

The repository's pinned Ruff 0.16.0 pre-commit gate and git diff --check passed. No live collector or customer backend was used. Remote CI is separate.

Avery Milandin and others added 3 commits November 10, 2026 22:50
Default checkpoint and profile runs to cli. Require Connect callers to opt in explicitly, and preserve caller-provided sources in telemetry and compact results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70ddf006-7f8d-48e7-9afa-3fbae73b3864
Honor FlightCheck invocation opt-out, keep telemetry imports fail-open, classify mapped exception bases, and enforce CLI coverage decisions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 70ddf006-7f8d-48e7-9afa-3fbae73b3864
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant