Skip to content

FND-3856: add pyatlan_v9 creators for the 7 Sigma asset types - #1087

Open
cmgrote wants to merge 2 commits into
mainfrom
FND-3856
Open

cmgrote wants to merge 2 commits into
mainfrom
FND-3856

Conversation

@cmgrote

@cmgrote cmgrote commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

✨ Description

Adds creator() for the 7 Sigma asset types in pyatlan_v9: SigmaWorkbook, SigmaPage, SigmaDataElement, SigmaDataElementField, SigmaDataset, SigmaDataModel, SigmaDataModelColumn. Follows the Cognos exemplar (#1078): overlay + parity tests + integration test.

SigmaPage.creator(
    name="Summary",
    workbook_qualified_name="default/sigma/1712345678/wb1",
    workbook_name="Sales Overview",
    page_id="pg1",
)
# qualified_name == "default/sigma/1712345678/wb1/pg1"
  • Exact app parity: each creator emits the qualifiedName atlan-sigma-app emits today (FND-3838 census), including the dataModels segment for data models. A different string would re-key existing Sigma assets on every tenant.
  • Shape follows Table / AirflowTask: Sigma qualifiedNames are nested, so child creators take the parent's qualifiedName plus the source id. Ancestor qualifiedNames and connection_qualified_name are derived from it. Parent and ancestor names are required params, because the qualifiedName holds ids and validate(for_creation=True) requires the denormalised names.
  • Parent relationship is set via Related<Parent>(unique_attributes={"qualifiedName": ...}) (sigma_workbook / sigma_page / sigma_data_element / sigma_data_model), as the app does, so creator output passes validate(for_creation=True).
  • Blank inputs raise (validate_required_fields), so the app's 'unknown' placeholder collision (sigma.py:89) cannot happen. A parent qualifiedName with the wrong segment count (or, for data model columns, no dataModels segment) raises ValueError.
  • asset_source_id is set on SigmaDataModel (data model id) and SigmaDataModelColumn (column id), matching the app.
  • Overlays in pyatlan_v9/model/assets/_overlays/sigma_*.py; the 7 classes are regenerated from atlanhq/models with these overlays (pkl sdkOnly=python + pre-commit, the same steps as sync-python-sdk.yaml). Only the 7 Sigma classes are taken from the regeneration; other drift from models master is left to the sync workflow.

Linear: FND-3856 (analysis table in the issue comments)
Paired models PR: https://github.com/atlanhq/models/pull/2082 — adds the matching qualifiedNameFormula, which drives the regenerated _QUALIFIED_NAME_PATTERN. Merge order: this PR first, then models#2082. The sync bot regenerates from atlan-python main's overlays when models master changes, so merged in this order its PR is a no-op for the Sigma classes; the other way round it would regenerate them without the creators.


🧩 Type of change

  • 🚀 New feature (non-breaking change that adds functionality)

✅ How has this been tested?

  • tests_v9/unit/model/sigma_*_test.py, one file per type (118 tests): literal app qualifiedName, validate(for_creation=True) (including the pattern), parent unique_attributes and denormalised attrs, None / blank for every param, malformed parent qualifiedNames, create alias, updater (+ missing inputs), trim_to_required.
  • App vs creator: 1,792 records run through atlan-sigma-app's own SigmaTransformer (all 7 types, ids with - _ . ~) give qualifiedNames, serialised parent refs ({typeName, uniqueAttributes}), denormalised parent qns / names and assetSourceId identical to the creators; 0 mismatches (app at 550376c).
  • Mutation check: changing the dataModels segment fails the parity tests.
  • Generator fidelity: models master + main's overlays reproduces main's 7 Sigma classes byte-for-byte; models#2082 + these overlays reproduces this PR's classes byte-for-byte.
  • tests_v9/integration/sigma_asset_test.py: creates a Sigma connection and every Sigma type through its creator, retrieves each and asserts the exact qualifiedName, connector, parent link and denormalised attrs, then purges. Collected locally; runs in CI.
  • pytest tests_v9/unit: 8028 passed, 5 skipped. pre-commit clean on all changed files.

CI (run 39278697037, with the run_pyatlan_v9_integration_tests label)

  • Green for this PR's files: v9-integration-tests (tests_v9/integration/sigma_asset_test.py) and v9-qa-checks-and-unit-tests on 3.9–3.14.
  • Pre-existing, not from this PR: 44 other integration jobs fail (sync v9, async v9 aio/* and legacy tests/integration), plus some cancelled jobs. Each of the 44 also fails or is cancelled on the same day's runs for FND-3857 (39278661620) or FND-3846 (39277871485). This PR touches only the Sigma files in pyatlan_v9 and tests_v9.

📋 Checklist

  • My code follows the project’s style guidelines
  • I’ve performed a self-review of my code
  • There are no new warnings from my changes
  • I’ve added tests to cover my changes
  • All new and existing tests pass locally

🤖 Generated with Claude Code

https://claude.ai/code/session_0166vmu2bYFX2P4LFGH3GR4F

Overlays for SigmaWorkbook, SigmaPage, SigmaDataElement,
SigmaDataElementField, SigmaDataset, SigmaDataModel and
SigmaDataModelColumn. Each creator emits the qualifiedName
atlan-sigma-app emits today, sets the parent relationship and the
denormalised parent attributes, and raises on blank inputs.
Classes regenerated from atlanhq/models#2082.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166vmu2bYFX2P4LFGH3GR4F
@linear

linear Bot commented Oct 14, 2026

Copy link
Copy Markdown

FND-3856

…/trim

- Parent relationships use Related<Parent>(unique_attributes=...), which
  serialises to exactly the {typeName, uniqueAttributes} ref
  atlan-sigma-app emits (qualified_name= also added a top-level
  qualifiedName key).
- Typed updater() and trim_to_required() on every Sigma overlay.
- Split sigma_test.py into one tests_v9/unit/model/<type>_test.py per
  type, adding updater, trim and missing/blank cases for every param.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166vmu2bYFX2P4LFGH3GR4F
@cmgrote cmgrote added the run_pyatlan_v9_integration_tests To trigger pyatlan v9 (msgspec-based) integration tests suite (name the workflow checks) label Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run_pyatlan_v9_integration_tests To trigger pyatlan v9 (msgspec-based) integration tests suite (name the workflow checks)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant