Skip to content

perf: optimize dmnlist construction and payye validation - #7934

Open
knst wants to merge 5 commits into
dashpay:developfrom
knst:perf-sync-p1
Open

knst wants to merge 5 commits into
dashpay:developfrom
knst:perf-sync-p1

Conversation

@knst

@knst knst commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

Significant part of block validation is building new lists of masternodes and validation of block's payee.
It's not waste, but it could be done more efficient way. This PR aim to optimize some bottlenecks.

What was done?

  • select the block's payee once during block validation
  • fetch the previous block's masternode list once per block
  • make ForEachMN and ForEachMNShared templates to help compilers do its job
  • diff masternode lists through immer's structural diff

This checks are done in the main validation thread and not parallelized, so it should make overall sync ~5% faster for all post-dip3 block, for blocks between v20 and mn_rr even more.

Extra fix provided by pastapastapasta@:

  • construct the gsl::not_null block-index argument before std::optional::emplace
  • keep the captured source location in Dash code instead of libc++'s construct_at.h

This fixes the macOS check-stdlib-paths failure for #7934.

How Has This Been Tested?

Some blocks close to tip had been invalidated and re-considered after [10k blocks].

event develop PR
m_dmnman.ProcessBlock 7.27s 5.67s
ProcessSpecialTxsInBlock 58.56s (6.20ms/blk) 55.76s (5.91ms/blk)
IsBlockPayeeValid 1.27s (0.13ms/blk) 0.05s (0.01ms/blk)
Dash specific 1.47s (0.16ms/blk) 0.24s (0.03ms/blk)
Connect total 67.66s (7.17ms/blk) 64.14s (6.79ms/blk)
Connect block 70.26s (7.44ms/blk) 66.62s (7.06ms/blk)

Breaking Changes

N/A

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

knst added 4 commits October 22, 2026 16:44
BuildDiff walked the whole new list and looked every masternode up in the old one to find the few entries a block changes, and walked the old list once more whenever something was removed: two hash lookups per masternode, ~4000 per block, for a handful of changes.

The new list is the old one plus those changes and the two share all other nodes, so immer::diff visits only the subtrees that differ. The added, removed and updated sets are the same; the early exit of the removal walk is no longer needed.
…do its job

Both took the callback as std::function, so every per-block walk paid a type-erased call per masternode and the lambda could not be inlined into the loop. Take the callable as a template parameter.
ProcessSpecialTxsInBlock fetched the list at the previous block for the payment check, BuildNewListFromBlock fetched it again to rebuild from, and CDeterministicMNManager::ProcessBlock a third time to build the diff.
Each fetch copies the list and runs the cache bookkeeping.
Pass the one already fetched into the rebuild and into ProcessBlock.
FinishListForBlock selected the payee from the previous block's list to update its last-paid height, and IsBlockPayeeValid selected it again from the same list to check the coinbase, each a full walk (two before MN_RR).
GetMasternodePayments also walked the list once more in GetCounts() just to see whether it is empty.

RebuildListFromBlock hands the payee it selected to its caller, ProcessSpecialTxsInBlock records it in MNListUpdates next to the lists it was chosen from, and the payment check uses it. Block templates and the RPC select the payee themselves as before.
@knst knst added this to the 24.1 milestone Oct 22, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T08:34:55.042192Z 1602784 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

Potential PR merge conflicts

This is advisory only. It does not block CI, but it marks PRs that will likely need a rebase depending on merge order.

If this PR merges first

These open PRs will likely need a rebase:

@thepastaclaw

thepastaclaw commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Final review complete — no blockers (commit 1602784) · triage: normal

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b1fcf9d3-d479-4e7a-ae00-bf96c6d17093
Commits

Reviewing files that changed from the base of the PR and between ca14f98 and 1602784.

Files selected for processing: 1
  • src/node/miner.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The change computes deterministic masternode-list diffs with immer::diff and passes the previous list into block processing. List rebuilding selects the payee from the previous list and stores it in MNListUpdates. Payment generation and block payee validation receive that payee explicitly. Mining and the masternode_payments RPC obtain the payee from the prior-block list. The list API also adds an entry count and changes two iteration callbacks to forwarding-reference templates.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CSpecialTxProcessor
  participant CDeterministicMNList
  participant MNListUpdates
  participant Chainstate
  participant GetMasternodePayments
  CSpecialTxProcessor->>CDeterministicMNList: select payee from previous list
  CDeterministicMNList-->>CSpecialTxProcessor: selected payee
  CSpecialTxProcessor->>MNListUpdates: store payee with old and rebuilt lists
  Chainstate->>GetMasternodePayments: pass list and payee for payment validation
Loading
Merge Risk: Minimal · up to 16027

No merge-blocking risk is established; the changed payee flow preserves existing behavior.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 15.38% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 26 functions across 10 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly identifies the main performance changes: optimizing deterministic masternode list construction and payee validation. The typo in “payye” does not prevent understanding.
Description check Passed The description directly explains the performance goals, implemented changes, testing method, measured results, and related build fix.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

Static verification of ca14f98 found no actionable correctness or consensus regressions in the list-diff, previous-list reuse, or cached-payee changes. Approval is withheld because mac-build / Build source fails check-stdlib-paths on an embedded libc++ construct_at.h SDK path; the corresponding base job passed, and changed C++ template instantiations could affect this check, so the failure cannot yet be dismissed as unrelated. Linux standard, ASan, and TSan tests and the multiprocess build remain pending; no local builds or tests were run.

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — The diff makes moderately cross-cutting optimizations to masternode-list diffing and cached payee propagation through block validation, but is not sufficiently large or intricate to meet the critical tier's combined criteria.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — dash-core-commit-history (completed, effort high); agent phase2-reviewer

@thepastaclaw thepastaclaw added the pastaclaw:commented thepastaclaw's latest review was comment-only label Oct 22, 2026
@thepastaclaw thepastaclaw removed the pastaclaw:commented thepastaclaw's latest review was comment-only label Oct 26, 2026

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

The complete PR range was verified at the exact head. The masternode-list diff, payee propagation, payment generation, incremental builder, and mining call-site changes preserve the existing semantics; no actionable correctness, consensus, security, or runtime issue was found. Builds, lint, and formatting checks reported success, while several source-test checks were still in progress in the supplied CI snapshot.

🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This is a cross-cutting performance refactor across deterministic masternode-list construction and block payee validation, but it does not directly alter consensus rules, funds movement, cryptography, networking deserialization, or storage migrations.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — dash-core-commit-history (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — dash-core-commit-history (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.

No unresolved findings remain from the prior review on this head.

@thepastaclaw thepastaclaw added the pastaclaw:approved thepastaclaw's latest review approved this PR label Oct 26, 2026

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failed after approval — commit 1602784

The standing review approved this commit; this follow-up moves the verdict to COMMENT: checks that failed after the review may relate to this change, and the evidence does not rule it out.

Failing checks:

Why it is unclear: The failing feature_llmq_signing.py --spork21 test exercises broad block, masternode, and payment validation paths that this PR changes, so a behavioral interaction is plausible. However, the provided log tail contains no failure assertion, race report, or stack trace identifying a changed path; all other tests passed and the failure occurred during shutdown, so the evidence is insufficient to classify it as related.

Nothing was re-reviewed: the findings of the standing review are unchanged. The next push gets a full review; a re-run that passes does not restore the approval.

Review provenance

Source: CI-check lane gpt-6.1-sol (agent: triage); no reviewer or verifier lanes ran for this follow-up

  • Verdict moved because CI failed on this commit after the review; failure logs read: 1 GitHub Actions job(s)

@thepastaclaw thepastaclaw added pastaclaw:commented thepastaclaw's latest review was comment-only and removed pastaclaw:approved thepastaclaw's latest review approved this PR labels Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pastaclaw:commented thepastaclaw's latest review was comment-only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants