Repository navigation
Conversation
…el/others get static demo catalog
…image installs the gam extra (AI-6)
…g-for-synced-ad-servers # Conflicts: # CHANGELOG.md # src/ad_seller/interfaces/api/deps.py # src/ad_seller/interfaces/api/main.py # src/ad_seller/interfaces/cli/main.py # src/ad_seller/services/catalog_service.py
Kishore-2572
left a comment
There was a problem hiding this comment.
Live-tested the safety-critical part given this codebase's history with #100: simulated an ad-server call failing mid-fetch with a real product already persisted — confirmed the real product survives, the failure path never reaches the pruning logic.
Closed the cross-repo gap live: drove the real unmodified persistence/serving functions with realistic fake GAM inventory into a real DB, started a real server against it, confirmed
GET /productsreturns real synced products not the static catalog, then got a correct quote through real buyer-agent code end to end.GAM discovery fix and Dockerfile
gamextra both confirmed in code. Full suite: 1726 passed, 0 failed, matches the PR's numbers. Approving on substance — no blocking issues found.
|
A successful sync that returns no inventory deletes the stored products, and the next catalog read serves the static defaults again. |
…talog, not the static demo products (AI-6) A successful sync that finds zero inventory items left zero synced_product:* keys, indistinguishable from "never synced" -- build_synced_product_catalog() fell back to the 13-product static demo catalog, silently serving fake products as if real. Add a synced_catalog:completed marker, set only on the real ad-server success path (never the mock path). The catalog is now treated as real/synced if EITHER a synced_product:* key exists (covers products persisted mid-sync, before the marker is set) OR the completion marker is present (covers a completed sync that genuinely found nothing) -- only "neither" still falls back to the static catalog. Found by aleksUIX in PR #117 review. Verified with a new unit regression test plus a full e2e pass (real FastAPI app, real sqlite storage, real sync method, only the GAM network call faked): sync with items -> empty sync -> simulated transient failure -> recovery sync, confirmed correct at each step, including from a second independent server process reading the same storage.
…report it as failed (AI-6) persist_synced_product/_upsert_synced_package write straight to storage one item at a time inside the loop. An exception partway through left every item already written by THIS run permanently in storage, even though the sync is reported as failed -- contradicting the existing claim that a failure never silently changes the synced layer. Capture each key's prior value before writing it, and on any exception, roll back precisely: restore the prior value if the key already existed (a re-synced item can share an id with one an earlier successful sync persisted), or delete it if this run introduced it fresh. Same fix applied to both the product loop and the package loop (identical structural risk). Reproduced live: synced A+B cleanly, then a sync that persists C and crashes before D -- A and B survived as expected, but C leaked in anyway. Fixed; added as two permanent regression tests (the leak case, and the re-synced-existing-id-restores-correctly case). Full suite: 1729 passed, 0 failed.
|
Good catch, thanks for catching this! Fixed — a sync completing with zero items now serves an honest empty catalog instead of falling back to the demo one, via a Also found and fixed a related issue while verifying: a sync that crashes partway through persisting items was leaving the already-persisted new items in storage despite being reported as failed. Now rolled back precisely on failure. Both covered by new tests, verified end-to-end. Commits: |
|
An empty successful sync now stays empty. A finished real sync sets synced_catalog:completed, including when the ad server returns no items, and the catalog read returns that empty catalog instead of the static defaults. The new test covers a one-product sync followed by an empty success. |
Problem
GET /products(and everything reading the catalog) served the static 13-product default catalog even after a real ad-server sync (GAM/FreeWheel/S3) had run.ProductSetupFlow.state.productsis per-flow-instance and never outlives the request, and the catalog service's in-memory cache is per-worker-process (infra/docker/Dockerfileruns uvicorn with--workers 2), so at most one of two worker processes ever saw synced data. Docs claimed GAM/FreeWheel support that the code didn't deliver.Fix
Three layers, all required together for a real GAM deployment to actually work:
synced_product:{id}storage key (deliberately separate from the genericproduct:{id}keynegotiation_service/media_kit_servicealready write, which is a narrower proposal-time snapshot) and are served fresh from storage on every catalog read when a real ad server is configured, matching how synced packages already worked. Stale products no longer re-seeded by a sync run are pruned instead of lingering as phantom inventory. CSV-mode and no-ad-server-configured behavior is unchanged.GAMRestClient.connect()calledgoogleapiclient.discovery.build("admanager", "v1", ...)with nodiscoveryServiceUrl. GAM self-hosts its own discovery document rather than registering centrally, so every real connection raisedUnknownApiNameOrVersionand silently degraded to mocks. Fixed by pointingbuild()atadmanager.googleapis.com/$discovery/rest?version=v1directly.infra/docker/Dockerfileonly installed theredisextra; thegamextra (googleads/google-api-python-client/google-auth) was never in the shipped production image, so even with (1) and (2) fixed, the container wouldImportErrorbefore ever reaching GAM. Fixed by installing".[redis,gam]".All three verified together end to end against a real GAM network with real service-account credentials, rebuilding the real Dockerfile from scratch with zero manual installs:
list_inventory()returned 100 real ad units,POST /packages/synccompleted with zero warnings,GET /productsserved 50 real synced products.Not yet verified: FreeWheel's real-data path — no real FreeWheel credentials available. Its
list_inventory()looks architecturally sound (real OAuth 2.1 MCP tool call) but unproven live.Reconciliation with main
This branch had drifted behind
main. Mergedorigin/mainin, resolving conflicts inCHANGELOG.md,interfaces/api/deps.py,interfaces/api/main.py,interfaces/cli/main.py, andservices/catalog_service.py— all were additive overlaps (new unrelated routes/fixes landing alongside this branch's own catalog work), not competing logic changes.Reconciling against main's newer
default_price_floor_cpmsetting (AI-8) surfaced one real gap:tests/unit/test_synced_product_catalog.py's hand-rolled settings fixture predated that field, so the second sync in the prune test crashed before pruning ever ran — masking as a "pruning doesn't work" failure when the actual cause was an unrelated settings-fixture gap. Fixed by adding the field to the fixture.Testing
/healthand/productsrespond correctly post-merge.