Skip to content

agentconfig-mcp: add shared LPC operation telemetry - #436

Open
Asher Johnson Hogan (asherjh) wants to merge 8 commits into
mainfrom
users/asherj/adk-core-lpc-observability
Open

Asher Johnson Hogan (asherjh) wants to merge 8 commits into
mainfrom
users/asherj/adk-core-lpc-observability

Conversation

@asherjh

Copy link
Copy Markdown
Contributor

Description

Add shared LPC operation telemetry.

Measure landing-page tool reliability and usage with one content-free completion record while keeping shared HTTP behavior and existing telemetry callers compatible. The shared infrastructure is opt-in; this PR integrates LPC only, with Org Announcements reserved for a separate follow-up.

  • Extend the existing adk.api.call event with optional schema-1.8 server and usage metadata. Existing callers omit the new fields and retain their outcome/error semantics.
  • Observe all ten LPC business tools through native FastMCP's public call_tool boundary, covering argument validation, handler execution, returned tool errors, output conversion, unknown tools, exceptions, and cancellation without changing results or propagation.
  • Capture request-local tenant identity, monotonic duration, and one generated operation UUID retained through buffer redelivery. Pre-authentication failures remain unattributed rather than inheriting another call's tenant.
  • Accumulate actual HTTP attempts, replays, single-flight credential refreshes, retry exhaustion, and response-point statuses without changing authentication, retry, timeout, idempotency, or exception metadata behavior. Unobserved core clients emit nothing.
  • Record provenance-safe configuration presence, actual discovery/configuration/draft counts, and code-owned widget values. Do not collect arbitrary configuration keys, content, search terms, URLs, backend messages, exception text, or personal identifiers. Missing or malformed shapes omit measurements; explicit empty lists retain zero counts.
  • Keep the schema-v2 client bridge app-only on LPC, with its existing parameters and accepted/rejected verdict, and exclude it from business-call records.
  • Document canonical server cohorts, active installations rather than unique people, separate client/server denominators, delivery limitations, and window-local deduplication that retains ID-less failures.

All 14 implementation files are included: six SDK/core/LPC source files, five test files, and three documentation/query files. No Org Announcements source, Graph client, manifests, visual surfaces, or monitor configuration are changed. There are no additional event families, per-attempt streams, or distributed tracing framework.

Type of change

  • New feature (non-breaking change which adds functionality)
  • Documentation update
  • Refactor / cleanup

Testing

  • Combined offline validation: 586 passed, 22 expected skips on Python 3.14.5 with native MCP 1.29.0. The skips are live-backend tests and one POSIX-only permission test.
  • Native in-memory MCP protocol coverage verifies the registered public boundary, result/exception preservation, pre-handler validation, cancellation, returned errors, and output-conversion failures.
  • Regressions cover all ten LPC tools, legacy/Org-style emitter compatibility, concurrent identity and counter isolation, single-flight and failed refresh accounting, non-idempotent retry policy, privacy fixtures, opt-out, asynchronous collector failures, and stable operation IDs through buffered redelivery.
  • The repository's staged-file pre-commit run gate passed using its pinned Ruff 0.16.0 hook. Source/test lint, syntax compilation, git diff --check, and standalone SDK/core imports without MCP also passed.

Combined test command, from the repository root:

python -m pytest tests/test_adk_telemetry.py tests/test_adk_mcp_report_client_events.py tests/mcp/agentconfig tests/mcp/agentconfig_core tests/scripts/test_mcp_config.py tests/scripts/test_setup_preserves_title_id.py tests/scripts/test_validate_branding.py -q --tb=short

No real backend or collector was used. KQL examples were reviewed locally, not executed against live production. Org integration is not exercised here. The remote CI matrix is pending and is not claimed to have passed; local Python differs from CI's Python 3.11 environment. Asynchronous completion records and accepted bridge batches do not guarantee collector delivery or establish process availability.

Checklist

  • My code follows the existing style
  • I have added/updated tests where applicable
  • I have updated documentation as needed

Measure landing-page tool reliability and usage with one content-free completion record while keeping shared HTTP behavior and existing telemetry callers compatible. Make the core instrumentation opt-in so Org Announcements can integrate separately.

- Add optional MCP and usage fields to the existing API-call event schema.
- Observe all ten LPC tools with request-local identity, timing, and HTTP summaries.
- Preserve the app-only schema-v2 client bridge and exclude it from business records.
- Add native protocol, privacy, concurrency, retry, and compatibility regressions.
- Document canonical cohorts, installation metrics, and delivery limitations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fe7b727c-3609-48cc-8c27-becc4fd98998
Internal telemetry analysis belongs in private repositories. The existing public disclosure covers the collection categories used by this instrumentation.

- Restore the existing README and telemetry investigation guide.
- Restore the existing KQL queries.
- Preserve the shared SDK/core/LPC instrumentation and tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fe7b727c-3609-48cc-8c27-becc4fd98998
Keep shared LPC operation telemetry compatible with the current session lifecycle and Stay up to date configuration APIs.

- Merge the latest main into the LPC telemetry branch.
- Retain MCP instrumentation and current configuration guidance.
- Validate API projection, ordered buffer replay, and asynchronous delivery with session-start events.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 25307f66-8638-4a01-a8e7-24bf0a5d8f06
Link each widget invocation and every backend HTTP attempt without conflating retries or concurrent calls. Keep telemetry content-free while preserving existing transport behavior and schema-v2 bridge compatibility.

- Accept validated root/request correlation metadata with independent server execution IDs.
- Emit distinct HTTP attempt IDs and records across retries and 401 refresh replays.
- Return LPC correlation metadata and project optional client bridge correlation IDs.
- Cover native MCP, concurrency, error/cancellation, opt-out, and buffered redelivery.
- Preserve legacy telemetry field shapes and branch-specific SDK behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fe7b727c-3609-48cc-8c27-becc4fd98998
…herj/adk-core-lpc-observability

# Conflicts:
#	solutions/ess-maker-skills/scripts/adk_telemetry.py
#	tests/test_adk_telemetry.py

Co-authored-by: asherjh <100732777+asherjh@users.noreply.github.com>
Canonical lowercase identifiers keep request metadata and client telemetry joinable when callers supply uppercase UUIDs.

- Normalize validated incoming MCP correlation and client request IDs.
- Normalize the three optional client correlation fields while preserving malformed-field omission and legacy operation ID validation.
- Cover uppercase metadata through native MCP execution and client telemetry ingestion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fe7b727c-3609-48cc-8c27-becc4fd98998
@amilandi

Copy link
Copy Markdown
Contributor

Coordination from the follow-up review on merged #486: #494 (8021abe) retains main SCHEMA_VERSION=1.8.0 and adds no payload fields. Please keep this PR's 1.9.0 bump when reconciling main, together with the merged script-error constants/registry and the follow-up opt-out/MRO fixes. The new invocation-scoped suppression must be checked before spawning transport threads because ContextVar state does not flow to those threads. No LPC changes were copied into #494.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants