fix: defer proof delivery after durable handoff

[ben-kaufman]

This PR:

1. Refreshes existing Paykit display projections from one validated state observation.
2. Returns local payment completion after durable proof handoff without waiting for remote proof delivery.

Related work: SDK #183, iOS #914, and Server #78. This consumer pins the published rc73 prerelease.

Description

• Uses the published rc73 SDK observation snapshot in the existing contact and payment-request refresh workers so display records belong to one validated identity and key generation, while keeping action, authorization, eligibility, and dispatch getters authoritative.
• Completes proven successful payments after durable activity and exact-proof persistence so remote proof delivery does not delay local completion; the existing lifecycle-owned reconciliation and persisted queue deliver proofs after restart.

Proof persistence failure retains the existing immediate-delivery fallback. If neither persistence nor delivery succeeds, the unresolved follow-up and payment guard remain. Unknown payment outcomes remain unresolved; per-request execution claims, save-before-send ordering, exact proof/ciphertext handling, and duplicate-payment prevention are unchanged. Cancellation retains native-operation ownership until the native call completes; refreshes retain runtime, identity, sign-in, and key-generation guards.

The normal display request-snapshot read phase changes from three getter calls (identity validation, request list, and linked peers) to one SDK observation transaction. Message intake, authoritative action reads, and checkpoint-driven rereads remain separate and unchanged. Contact-only loading already used one transaction; reading a full observation adds projection/filtering work, so no contact speedup is claimed. Existing warm StateFlows, coalescing, and persistence are reused, with no new cache or cold-start instant-display promise. Observation failures preserve existing warm display; cold failures do not claim a successful refresh.

Out of Scope

• SDK/FFI implementation and artifact publication: owned by pubky/paykit-rs#183; this consumer pins the published rc73 artifact.
• Homeserver and Noise behavior, safety waits, payment retries, forced unlocks, and new UI copy.
• New persistent caches, additional UI behavior, or replacing authoritative getters globally.
• Minimum-amount PR fix: validate minimum paykit request amounts #1457 and other independently owned Android changes.
• Total linking/payment/unlock speedup claims, releases, and merges.

Design

N/A — no UI changes.

Preview

QA Notes

Journeys

N/A — not drivable; see Manual Tests.

The payment-request journey README documents these fault-driven checks. Matching iOS coverage belongs to the iOS consumer change.

Manual Tests

• In an authorized disposable regtest fixture, hold remote proof delivery after proven payment success → accepted activity and exact proof are durable before local completion; no remote unlock is claimed and no additional payment is dispatched — independent payment-acceptance, durable-storage, and remote-submission fault controls not in Capabilities.
• Restart before releasing the proof-delivery hold, then restore the original identity and release it → the same proof is delivered without another payment or duplicate activity — restart delivery with controlled remote submission not in Capabilities.
• Repeat with an unknown payment outcome, then separately fail both proof persistence and delivery → unresolved follow-up and payment guard remain; no fresh payment becomes eligible — payment-outcome and storage fault injection not in Capabilities.
• Switch identity or sign out before delivery; separately reset disposable wallet state → no proof is sent under another identity and delayed work does not repopulate reset state — controlled in-flight delivery and disposable reset fixture not in Capabilities.

Automated Checks

• added PaykitPaymentProofRepoTest.kt - verifies local completion after successful durable writes during blocked submission, exact-proof delivery after reconstruction, failure fallback, and identity/reset isolation.
• added PaykitPaymentRequestRepoTest.kt - verifies readable warm flows during blocked refresh, identity clearing, warm-state retention on snapshot failure, and honest cold failure.
• added PubkyRepoTest.kt - verifies failed observations preserve warm contacts and stale results from an ended same-identity sign-in stay hidden.
• added PaykitSdkServiceTest.kt - verifies observation identity validation, error propagation, queued runtime replacement, and cancellation without overlapping native operations.
• updated PaykitKeyGenerationTest.kt - verifies observation key generation matches the active key.
• updated PaykitPaymentRequestRepoSubscriptionTest.kt, ContactSaveSessionChangeTest.kt, and AppViewModelSendFlowTest.kt - retains request/contact action coverage and verifies proof reconciliation remains lifecycle-owned and coalesced during active payment submission.
• updated PubkyRepoTest.kt - verifies cold contact failure ends loading without marking a successful observation.
• ran matched baseline/candidate proof-handoff fixtures with 20 successful file writes plus fsync and a deterministic 5,000 ms virtual remote-submission hold: baseline p50/p95/max local-completion wait was 5,000/5,000/5,000 ms; candidate was 0/0/0 ms of injected virtual wait. This is an isolation/recovery correctness benchmark, not Android Keystore latency, wall-clock speed, or real-network performance.
• ran matched warm-read fixtures during blocked refresh: 20 reads took 0 ms of injected virtual wait on both baseline and candidate. Warm display isolation already existed; no warm-cache speedup is claimed.
• ran forced fresh canonical compile and full tests against published GitHub Maven rc73 with configuration/build caches disabled and no init script or local override: 13/13 compile tasks and 44/44 test tasks executed; all 3,955 tests passed with zero failures, errors, or skips. Gradle cached AAR SHA-256 exactly matches the canonical registry artifact: 44cc0ad08e6689c5a820d399d49ad67f2d69f8ff3039f322107525fa97411725.
• ran canonical format and final lint sequentially; complete Detekt reports contain 1,980 findings versus 1,981 on verified master, with zero introduced findings. All 223 compiler diagnostics, the manifest-removal warning, and the Gradle deprecation notice match baseline. Unrelated baseline formatter rewrites were excluded.

One fresh combined source correctness and quality review and bounded integration follow-up are complete. Controlled payment/restart fault checks above remain incomplete, not passed.

Remote Measurements and Payment QA

• Published canonical rc72/rc73 native artifacts against the actual staging Homeserver: stored-request refresh measured midpoint medians 2,618.688 ms versus 879.058 ms across 12 samples per arm in ABBA blocks; all 24 passed. Fixture: 32 contacts, empty request/peer/receipt collections, no injected delays or shortened safety waits. Timer covers the repository refresh and model publication, not rendered UI, cold startup, populated history, linking or unlock. Contacts-only and already-cached display have no demonstrated improvement.
• A separately authorized isolated regtest trial reached native Accepted, but immediate bookkeeping failed with Winning transaction fee is unavailable before this PR's changed deferred-delivery branch. The activity/fee path is byte-identical to base. The fixture omitted the normal post-Accepted wallet synchronization and periodic recovery, so it does not prove a persistent production blocker or explain the reported stuck-wallet incident. No successful payment-completion speedup was measured. The separate validated fee-retention fix is fix: retain prepared transaction fees #1459; unknown-outcome and duplicate-payment guards remain unchanged.

Reviewer-Reported Device Coverage

• jvsena42's exact-head approval and device report approves afbf14d. Device measurements were made on 268293a against 3cac93b, using linked Pixel_9/Pixel_Tablet staging-regtest wallets and non-suspending debugger logpoints; the later hardware-only follow-up was not exercised. With three historic requests, periodic poll medians were 4.36 s versus 7.59 s, and snapshot-read medians were 1.63 s versus 4.54 s. This is a small sequential baseline/candidate comparison, not populated-history, full unlock, or payment-speedup evidence.
• The reviewer reports a 1,200-sat Lightning payment followed by airplane mode 0.5 s after Success and force-stop before a proof-queued log. After relaunch, one proof was queued and cleared, one Sent activity remained, the payee showed Proof submitted, and no second payment was observed. This supports a bounded normal-payment restart case, not the independently controlled acceptance/storage/submission fault matrix above. Hardware Shop, identity/reset fault cases and large-history coverage remain open; automated checks do not substitute for those device journeys.
• Performance follow-up remains open: one Lightning pending-path run took 56.7 s from PaymentSuccessful to Success, without a matching pending-path baseline. The pending branch already awaits full refresh on the base revision, while this PR changes downstream reconciliation; that establishes neither a new regression nor its absence. Home-resume medians were 13.2 s versus 10.7 s with overlapping ranges, and reconciliation counts were 3-5 versus 1-3 per payment. Normal accepted-to-Success timings were similar on both builds; no end-to-end payment speedup is claimed. The reviewer also observed the winning-fee-unavailable error on both builds, separately addressed by fix: retain prepared transaction fees #1459; it is not proof that this performance PR caused it.

Hardware Proof Handoff Follow-up

An exact hardware proof whose activity was successfully reconstructed in this process can proceed to delivery without repeating the remote transaction lookup. This evidence is process-local, bound to the complete immutable proof, cleared on backup restore/empty store and pruned when proofs change or leave the store. Restored proofs still reconstruct activity before delivery; identity, request, wallet and positive-acceptance checks are unchanged.

The second-lookup-unavailable regression failed before the correction. Final validation: 77 proof-repository tests and all 3,956 unit tests passed; one test added and one existing test extended, none removed. Final compile passed; complete Detekt/format findings remain 1,980 versus 1,981 on verified base with no introduced findings, and incremental compiler diagnostics match baseline. One fresh combined review and its bounded lint-correction recheck found no actionable issues. No device or payment test was run for this follow-up. Existing performance and device gates remain open.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable defects were found.

Summary

This PR uses one checked Paykit observation to refresh contact and payment-request displays. It also lets successful payments finish locally once their exact proof has been saved.

• Successful payments finish locally after their proof is saved.
• Contact and payment-request displays refresh from one checked snapshot.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Proven successful payment] --> B[Save exact proof]
  B --> C{Save succeeded?}
  C -->|Yes| D[Finish locally]
  D --> E[Lifecycle reconciliation or restart]
  E --> F[Check original identity]
  F --> G[Submit saved proof]
  C -->|No| H[Try immediate delivery]
  H --> I{Delivery succeeded?}
  I -->|Yes| D
  I -->|No| J[Retry saving proof]
  J --> K[Keep unresolved guard if both fail]

Loading

_{Reviews (1) · Last reviewed commit: "fix: consolidate paykit reads and defer ..." · Reviewed by Greptile}

[github-actions]

Regtest APK

Built from afbf14d (run).

Download bitkit-dev-debug universal APK (expires in 30 days).

[talosmachina]

No findings. Bumps paykit to rc73, moves the contact and payment-request display refreshes onto the single observeState() transaction, and lets proven Lightning, on-chain and hardware completions return once the proof is persisted, leaving delivery to reconcile(). Reviewed 268293a, full tier. Reasoned from the code and CI; per the repo's agent rules this branch is not built here.

What I checked, and 4 candidates I ruled out

Read in full: the diff of all 17 files; persistAndSubmit, reconcile, reconcileProof, reconcileLightningProof, reconcileOnchainProof, reconcileHardwareOnchainProof, submitReady, completeOnchainPayment and paymentRequestStateChanges in PaykitPaymentProofRepo.kt; refresh and fetchRequestSnapshot in PaykitPaymentRequestRepo.kt; loadContacts, currentSignIn and isCurrent in PubkyRepo.kt; observeState, withPaykitKey and refreshPaykitKey in PaykitSdkService.kt; observe_state and state_observation_in_transaction in pubky/paykit-rs#183.

Call sites traced: completeLightningPayment (handlePaymentSuccessful), completeOnchainPayment (completeOnchainPaymentProofInBackground, completeRecoveredOnchainPayment), the hardware original-payment observer, every reconcile() caller (refreshIncomingPaykitPaymentRequests, the not-dispatched send error, BackupRepo).

CI: CI (build, unit tests), build-local, lint, detekt and Greptile green at this head; the e2e-tests-local shards were still pending when I read them.

Ruled out

• A persisted proof that nothing delivers: the persist bumps PaykitPaymentProofStore.backupStateVersion, paymentRequestStateChanges sees proofData move from null to the preimage or txid, and the observer now passes reconcileProofs = true, so reconcile() reaches submitReady. That covers the one-time on-chain path too, where completeOnchainPaymentProofInBackground only refreshes when billingPeriod != null. The Lightning path also gets the FULL refresh right after completeLightningPayment.
• Reconcile looping on its own writes: submitReady removing the delivered proof changes the observed state once, then distinctUntilChanged holds; an undeliverable proof is not re-persisted by reconcile(), so offline does not spin. Two concurrent reconciles are covered by the alreadyQueued check under operationMutex.
• Changed meaning of completed for the Shop follow-up: persistAndSubmit already returned true on a successful persist before this change, so completeAcceptedShopFollowup fires on the same condition; only the wait for submitReady is gone, and the persist-failure branch still delivers immediately.
• Contacts on a session without the Paykit secret: observe_state requires the identity secret, while the old contact_records() needed only an initialized identity, so a restored session whose local key cannot be read (a Ring-adopted identity while the Ring provider fails the trust check) now fails the contacts load where it used to succeed. Warm contacts are kept and the trigger is a transient provider failure, so I ruled it out, narrowly.

Merge confidence: 4/5, the contacts candidate above was ruled out only narrowly and the e2e shards had not finished.

[jvsena42]

Code scan at 268293a: no HIGH or MEDIUM finding, 1 LOW inline. The device result follows in a second review; not approving until it has run.

Checked and clean:

• Durable handoff. persistAndSubmit(..., awaitDelivery = false) returns true only after persist(allProofs) succeeded, so the completed proof is in the keychain before any caller clears or reports state. If the persist fails it falls back to the immediate submitReady and a second persist; if both fail it returns false, the started proof is kept and prepare rejects a second payment.
• No second payment. prepare() treats a stored proof with proofData as started, so a deferred proof blocks a new payment of the same request until submitReady removes it. Subscriptions are also checked against the SDK record's proofs.
• A paid request does not resurface. Every snapshot apply reads completedRequestProofKindsAwaitingSubmission and inFlightRequestIds from the proof store, and the refresh checkpoint re-syncs when the store changed during the read.
• Crash between broadcast and delivery. Lightning and default-wallet on-chain proofs are persisted under the mutex and delivered by reconcile() on the next start; a Lightning proof that never got that far is completed from LDK's payment list.
• Delivery trigger and retry. No new timer or polling: the persist changes completedProofKinds, which runs a stored refresh with reconcileProofs = true; a change during payment submission is re-emitted when the gate clears. A failed submitReady is not re-persisted, so there is no persist/reconcile loop, and reconcile() returns early when nothing is pending.
• Cross-identity. reconcile() filters to the SDK's current identity and submitReady re-checks it under the same lock before submitPaymentProof; another identity's proofs stay stored until that identity is active.
• Cancellation. The new paths use runSuspendCatching; observeState goes through completeSdkCall, so a cancelled observer cannot overlap the next native call.
• loadContacts. The sign-in generation check discards a load from an ended sign-in of the same identity; a failed load keeps the warm contacts.

⚠️ Not verified: the paykit-android 0.1.0-rc73 binary. observeState does not exist in rc72, and I read the rc73 source diff but not the AAR, so "the observation returns what contactRecords() and listPaymentRequests() returned" rests on the source.

CI is green at this head.

Device gate (partial): 268293a — not run yet. Pending, against a master build on the same emulators: request-refresh time with the single observation, time from payment accepted to Success against the proof-delivery log line for a Lightning and a one-time on-chain request, reconcile runs per payment, and the airplane-mode restart check (one delivery, no duplicate activity). Performance-relevant: the PR states a speed purpose and adds a reconcile() to every proof-state change.

[jvsena42]

No HIGH or MEDIUM finding at afbf14d, and no LOW left open: the double hardware observation from my code-scan review is fixed in afbf14d. Approving.

afbf14d read: observedHardwareProofs is added only after persistAndSubmit retained the completed proof, reconcileCompletedHardwareProof delivers a proof in the set with submitReady and sends any other through the observation as before, and the set is cleared in restoreBackup and on an empty store and pruned against every load and persist. It is only touched under operationMutex (reconcileProof runs inside the lock taken in reconcile()), and a restored verified proof still rebuilds its activity (restoring an observed hardware proof requires activity reconstruction again). CI is green at afbf14d.

The device run below was made at 268293a. The one commit since changes only the hardware proof path, which the run did not exercise.

Device gate: 268293a — journeys: none listed; perf: poll refresh 4.36 s median (master 7.59 s), snapshot read 1.63 s (master 4.54 s), Home-resume refresh 13.2 s (master 10.7 s, ranges overlap); Lightning accepted → Success 0.03 s (master 0.02 s); on-chain accepted → Success 0.02 s (master 0.02 s); Success → proof queued +26 s (master +15 to +30 s); 3–5 reconcile() runs per payment (master 1–3); idle 2 min on Home: 1 reconcile / 7 refreshes (master 2 / 5); background 1 min: 0 / 0 on both; no crash, ANR or OOM on either build; Figma: no screen changed.
Skipped — covered by CI at afbf14d: unit tests, lint, detekt, e2e (all green)

Setup: Pixel_9 (payer, fresh wallet and identity, 304,712-sat channel) and Pixel_Tablet (payee) linked as contacts, staging regtest, master (3cac93b) installed and measured first, then this head installed over it. The refresh and reconcile paths log nothing on success, so they were timed with non-suspending debugger logpoints; payment events come from the app log (UTC).

	master	this PR
Periodic poll with 3 historic requests, total	7.59 s (5.35–18.47), n=7	4.36 s (2.84–9.04), n=10
Snapshot read (allPaymentRequests / observeState → applied)	4.54 s (3.42–7.14)	1.63 s (1.09–3.54)
Lightning 1,000 sats, swipe → Success	20.9 s	96.4 s (pending branch, below)
Lightning 1,500 sats after 2 min idle, swipe → Success	30.1 s	42.0 s
On-chain 2,000 sats, swipe → Success	40.8 s	37.8 s

The refresh claim holds: the single observation is about 2.8× faster on the read and the whole poll is faster. On both builds Success already arrives right after acceptance and the proof is queued 15–30 s later, so for node-wallet payments the ordering is the same as on master.

Durable handoff, on this build: paid a 1,200-sat Lightning request, turned on airplane mode 0.5 s after Success and force-stopped before any Queued line. After relaunch: one Queued a Paykit payment proof for private delivery (11:01:31.681), the proof store deleted at 11:01:49.871, no second Paying bolt11 or PaymentSuccessful, one "Sent" activity row on the payer, "Proof submitted" on the payee, and the request not payable again.

One slow run, unverified as a regression. The first Lightning payment on this build took 56.7 s from PaymentSuccessful to the Success screen. The staging LSP held the HTLC for 12.5 s, so the send went through the "Lightning payment pending" branch, which awaits a full refreshIncomingPaykitPaymentRequests() before navigating; that refresh ended after the proof delivery. The branch is the same code on master and master never took it during the run, so there is no baseline. A master run in which the payment stays pending past the send timeout would settle it.

Log lines for the slow run and the normal path

This PR, pending branch:

10:40:36.122 DEBUG [AppViewModel.kt:4460] Swipe to pay event, checking send confirmation conditions
10:40:57.972 VERBOSE [LightningRepo.kt:259] Operation called: 'payInvoice'
10:40:59.034 WARN  [PaykitPaymentRequestRepo.kt:576] Failed to refresh incoming Paykit payment requests [RequestUnavailable='Payment request is unavailable']
10:41:00.108 DEBUG [LightningService.kt:1037] Paying bolt11: lnbcrt1p4vedacdq…
10:41:10.588 INFO  [AppViewModel.kt:5060] Lightning payment pending
10:41:15.805 DEBUG [AppViewModel.kt:1822] LDK-node event received in AppViewModel: {"type":"PaymentSuccessful",…}
10:41:15.815 INFO  [Keychain.kt:154] Upserted value for key 'PAYKIT_PENDING_PAYMENT_PROOFS'
10:41:46.735 INFO  [PaykitPaymentProofRepo.kt:1200] Queued a Paykit payment proof for private delivery
10:41:54.883 DEBUG [Keychain.kt:170] Deleted value for key 'PAYKIT_PENDING_PAYMENT_PROOFS'
10:42:12.386 INFO  [Keychain.kt:154] Upserted value for key 'PAYKIT_ACCEPTED_PAYMENT_REQUESTS'
reconcile() entered 10:41:10.941, 10:41:16.182, 10:41:16.223, 10:41:50.562; onSendSuccess 10:42:12.812 (logpoints, host clock)

Nothing is logged between 10:41:15.815 and 10:41:46.735 (31 s), nor between 10:41:54.883 and 10:42:12.386 (17.5 s).

master, normal path:

10:03:02.822 Swipe to pay event …
10:03:17.541 Operation called: 'payInvoice'
10:03:19.614 Paying bolt11: lnbcrt1p4vet65d…
10:03:23.706 LDK-node event received in AppViewModel: {"type":"PaymentSuccessful",…}
10:03:39.253 [PaykitPaymentProofRepo.kt:1197] Queued a Paykit payment proof for private delivery
10:03:48.002 Deleted value for key 'PAYKIT_PENDING_PAYMENT_PROOFS'
handlePaymentSuccessful 10:03:24.048, onSendSuccess 10:03:24.052 (logpoints, host clock)

Also seen, the same on both builds: Failed to finish accepted on-chain send locally [IllegalArgumentException='Winning transaction fee is unavailable'] on the on-chain payment (LightningRepo.kt:1679; master 10:07:40.400, this PR 10:46:51.816). Swipe → payInvoice took 14.7 s and 23.9 s on master and 21.9 s, 36.0 s and 22.8 s here; too few samples to call.

⚠️ Not run: a hardware Shop payment (the path afbf14d changes). It needs a paykit-server issuer fixture, which is not available on my setup; that coverage is left to the e2e CI and the unit tests added in afbf14d.
⚠️ Not verified: master's first Lightning receipt on the payee was closed before it was read; every other payment showed "Proof submitted" on the payee.

[piotr-iohk]

QA review

Scope: Full review of the complete PR diff against merge base 3cac93b, at afbf14d.

No new actionable code findings.

Successful Paykit payments persist the exact proof and return before remote delivery. Lightning completion and recurring on-chain completion then refresh, and that refresh reconciles and submits the stored proof. One-time on-chain and hardware proofs are submitted by the proof-state observer while polling is active, or by the next foreground full refresh. A failed persist still attempts immediate delivery. A different identity, or a cleared proof store, does not submit.

A hardware proof observed in this process is delivered without a second chain lookup. Restoring the wallet clears that handoff, so a restored proof is submitted only after the original transaction is observed again. Display refresh reads one rc73 observation (paykit-rs 0eff3d4, tag v0.1.0-rc73). Its payment requests use the same non-blocked filter as listPaymentRequests. Payment actions, eligibility, and proof submission still use the authoritative getters. The same display-observation and local-completion flow was compared with merged bitkit-ios#914 at 532eb15: iOS starts submission inside completion, and Android submits from reconcile. Contact loading uses that observation and ignores a result from a sign-in that has already ended. A failed observation leaves the current contact list in place.

Validation: CI, lint, and e2e succeeded on this head. The new repository tests were inspected, not executed in this review.

Device testing: not performed in this review. The existing device report on this PR was run on 268293a and left the hardware path changed in afbf14d unexercised.

Ready for device testing.

Suggested additional test cases

• Android, disposable regtest, paired hardware wallet and a Shop issuer. Pay one Shop on-chain request from that hardware wallet. Local completion finishes without waiting for issuer delivery, one Sent activity is stored, and after reconcile the payee shows one submitted proof and the request is not payable again.
• Force-stop the payer after local completion and before the proof is queued, then relaunch the same identity with the hardware wallet disconnected. No second payment and no duplicate activity appear. The proof stays unsent until that original transaction can be observed again, and it is then delivered once.