From 0b056c200ceb6fba3e206fe4481d9d6d34995a02 Mon Sep 17 00:00:00 2001 From: lklimek <842586+lklimek@users.noreply.github.com> Date: Wed, 17 Jun 2026 08:00:25 +0200 Subject: [PATCH 001/149] fix(rpc-json)!: resolve Core 23 nested platform addresses (breaking field/accessor renames) (#808) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: read Core 23 nested platform addresses in rpc-json DMNState Dash Core 23.1 moved the platform P2P/HTTPS ports out of the deprecated top-level platformP2PPort/platformHTTPPort keys into a nested `addresses` object whose `platform_p2p`/`platform_https` sub-keys hold "host:port" arrays. Without reading them, platform_p2p_port deserialized to None on Core-23 entries, dropping or mis-porting validators downstream. Add MasternodeAddresses and backfill the existing port fields from the addresses object when the legacy port is absent or zero, via a private DMNStateIntermediate (mirroring DMNStateDiffIntermediate) and inside the existing TryFrom for DMNStateDiff. Legacy pre-23 JSON is untouched. Co-Authored-By: Claude Opus 4.8 (1M context) * refactor(rpc-json): resolve platform ports via accessors; drop in-deserialize backfill Match the plain-derive style of the rest of the crate: deserialize DMNState and DMNStateDiff with no logic in the deserialize path. The nested Core 23 `addresses` object now lands as a plain public field instead of being folded into the legacy port fields during deserialization. Port resolution moves into accessor methods on both DMNState and DMNStateDiff: resolved_platform_p2p_port / resolved_platform_http_port. They check the new `addresses` object first (first non-zero, parseable port) and fall back to the deprecated top-level platformP2PPort/platformHTTPPort — new-first, legacy-fallback. Removed: the DMNStateIntermediate mirror struct and its `#[serde(from)]`, the `backfill_port` helper, and the in-deserialize backfill blocks in both From/TryFrom impls. DMNState regains its plain `#[derive(Deserialize)]`; DMNStateDiff keeps its pre-existing `try_from` with `addresses` flowing through as raw data. apply_diff now propagates `addresses` so a stored DMNState stays resolvable after diffs. Co-Authored-By: Claude Opus 4.8 (1M context) * refactor(rpc-json): platform address accessors return (host, port), clearable diff Address self-review on the Core 23 platform-address work. Rename the raw legacy fields on DMNState and DMNStateDiff to legacy_platform_p2p_port / legacy_platform_http_port (serde wire renames intact) so callers reach for the accessors instead of the deprecated fields. The resolver accessors become platform_p2p_address / platform_http_address and return Option<(String, u32)> instead of a bare port. On DMNState the legacy fallback pairs the port with the node IP from `service`, since Dash deploys platform services on the masternode's core IP; on DMNStateDiff there is no node IP to pair with, so the legacy path is not resolved. Zero ports never surface: they are dropped in both the addresses path and the legacy fallback. Ports parse through u16 then widen to u32, rejecting out-of-range values like "host:70000". DMNStateDiff::addresses becomes Option>, mirroring pose_ban_height: None = unchanged, Some(None) = cleared, Some(Some(_)) = set. A custom serde helper distinguishes absent / null / object on the wire. compare_to_newer_dmn_state now encodes a Some->None clear as Some(None) and apply_diff applies the inner value, so the clear survives the round-trip. Tests cover diff addresses propagation through apply_diff, the Some->None clear round-trip, zero-port-to-None, and out-of-range rejection. Co-Authored-By: Claude Opus 4.8 (1M context) * chore: add deprecated annotation to legacy fields * fix(rpc-json): tighten legacy platform-port resolution and deprecate DMNState fields Address a fresh review pass on the Core 23 platform-address work. - Mirror the `#[deprecated]` annotation onto DMNState::legacy_platform_p2p_port / legacy_platform_http_port so both DMNState and DMNStateDiff signal the same deprecation; add `#[allow(deprecated)]` to the two accessor methods, the apply_diff write sites, and the tests that read the fields to keep `clippy -D warnings` green. - legacy_platform_address now validates through `u16::try_from`, so an out-of-range legacy port (e.g. platformP2PPort: 70000) yields None instead of Some((ip, 70000)), matching the addresses path and the accessor's documented in-range contract. - parse_host_port rejects unbracketed multi-colon (bare IPv6) hosts rather than silently mangling them via rsplit_once; bracketed IPv6 and IPv4 still parse. The doc now states the supported forms accurately. Tests: wire-level `addresses: null` -> Some(None) (and absent -> None) round trip, out-of-range legacy port rejection, and parse_host_port IPv6 handling. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(rpc-json): reject empty host in parse_host_port; deprecate intermediate legacy ports Addresses CodeRabbit review on PR #808: - parse_host_port now rejects empty hosts (":36656" -> None) - DMNStateDiffIntermediate legacy_platform_*_port mirror the #[deprecated] of DMNState/DMNStateDiff Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- rpc-json/src/lib.rs | 517 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 496 insertions(+), 21 deletions(-) diff --git a/rpc-json/src/lib.rs b/rpc-json/src/lib.rs index 6416f2857..5d4e7015e 100644 --- a/rpc-json/src/lib.rs +++ b/rpc-json/src/lib.rs @@ -2052,6 +2052,51 @@ pub struct GetMasternodePaymentsResult { pub masternodes: Vec, } +/// Nested `addresses` object on Core 23+ masternode entries. +/// +/// Each purpose maps to an array of `"host:port"` strings. Dash Core 23 moved the +/// platform ports here, away from the deprecated top-level `platformP2PPort`/ +/// `platformHTTPPort` keys. Unknown purposes are ignored. +#[derive(Clone, PartialEq, Eq, Debug, Default, Deserialize, Serialize)] +pub struct MasternodeAddresses { + #[serde(default)] + pub core_p2p: Vec, + #[serde(default)] + pub platform_p2p: Vec, + #[serde(default)] + pub platform_https: Vec, +} + +impl MasternodeAddresses { + /// First valid `(host, port)` from a `"host:port"` array, if any. + /// + /// "Valid" means parseable and with a non-zero in-range port; zero ports are + /// skipped because Dash Core uses `0` as a "not set" sentinel. + fn first_valid_host_port(addrs: &[String]) -> Option<(String, u32)> { + addrs.iter().filter_map(|a| parse_host_port(a)).find(|(_, p)| *p != 0) + } +} + +/// Splits a `"host:port"` string into `(host, port)` with a non-fabricated host. +/// +/// Supports IPv4 (`1.2.3.4:9999`) and bracketed IPv6 (`[2001:db8::1]:9999`). The +/// final `:port` segment is parsed through [`u16`] then widened to `u32`, rejecting +/// values outside the TCP/UDP port range (e.g. `"host:70000"`). Unbracketed +/// multi-colon hosts (bare IPv6) are rejected rather than silently mangled. An empty +/// host (e.g. `":36656"`) is also rejected. Returns `None` when no colon is present, +/// the host is empty or ambiguous, or the suffix is not a valid `u16`. +fn parse_host_port(addr: &str) -> Option<(String, u32)> { + let (host, port) = addr.rsplit_once(':')?; + if host.is_empty() { + return None; + } + if host.contains(':') && !(host.starts_with('[') && host.ends_with(']')) { + return None; + } + let port = port.parse::().ok()?; + Some((host.to_string(), u32::from(port))) +} + #[serde_as] #[derive(Clone, PartialEq, Eq, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] @@ -2080,10 +2125,54 @@ pub struct DMNState { rename = "platformNodeID" )] pub platform_node_id: Option<[u8; 20]>, + #[deprecated(note = "Core 23+ nested addresses.platform_p2p should be used instead")] #[serde(default, rename = "platformP2PPort")] - pub platform_p2p_port: Option, + pub legacy_platform_p2p_port: Option, + #[deprecated(note = "Core 23+ nested addresses.platform_https should be used instead")] #[serde(default, rename = "platformHTTPPort")] - pub platform_http_port: Option, + pub legacy_platform_http_port: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub addresses: Option, +} + +impl DMNState { + /// Resolved platform P2P `(host, port)`. + /// + /// Prefers the Core 23+ nested `addresses.platform_p2p` entry, returning its + /// host and port verbatim. Falls back to the deprecated top-level + /// `platformP2PPort`, pairing it with the node IP from [`service`](Self::service) + /// because Dash deploys platform services on the masternode's core IP. Returns + /// `None` when no source yields a non-zero in-range port. + #[allow(deprecated)] + pub fn platform_p2p_address(&self) -> Option<(String, u32)> { + self.addresses + .as_ref() + .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_p2p)) + .or_else(|| self.legacy_platform_address(self.legacy_platform_p2p_port)) + } + + /// Resolved platform HTTPS `(host, port)`. + /// + /// Prefers the Core 23+ nested `addresses.platform_https` entry, returning its + /// host and port verbatim. Falls back to the deprecated top-level + /// `platformHTTPPort`, pairing it with the node IP from [`service`](Self::service) + /// because Dash deploys platform services on the masternode's core IP. Returns + /// `None` when no source yields a non-zero in-range port. + #[allow(deprecated)] + pub fn platform_http_address(&self) -> Option<(String, u32)> { + self.addresses + .as_ref() + .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_https)) + .or_else(|| self.legacy_platform_address(self.legacy_platform_http_port)) + } + + /// Pairs a legacy platform port with the node IP, dropping zero, absent, and + /// out-of-`u16`-range ports so the result honors the TCP/UDP port range. + fn legacy_platform_address(&self, port: Option) -> Option<(String, u32)> { + port.and_then(|p| u16::try_from(p).ok()) + .filter(|&p| p != 0) + .map(|p| (self.service.ip().to_string(), u32::from(p))) + } } #[derive(Clone, PartialEq, Eq, Debug, Deserialize)] @@ -2103,13 +2192,19 @@ pub struct DMNStateDiff { pub pub_key_operator: Option>, pub operator_payout_address: Option>, pub platform_node_id: Option<[u8; 20]>, - pub platform_p2p_port: Option, - pub platform_http_port: Option, + #[deprecated(note = "Core 23+ nested addresses.platform_p2p should be used instead")] + pub legacy_platform_p2p_port: Option, + #[deprecated(note = "Core 23+ nested addresses.platform_https should be used instead")] + pub legacy_platform_http_port: Option, + /// Three-state nested addresses: `None` = unchanged, `Some(None)` = cleared, + /// `Some(Some(_))` = set. Mirrors [`pose_ban_height`](Self::pose_ban_height). + pub addresses: Option>, } impl TryFrom for DMNStateDiff { type Error = encode::Error; + #[allow(deprecated)] fn try_from(value: DMNStateDiffIntermediate) -> Result { let DMNStateDiffIntermediate { service, @@ -2123,10 +2218,11 @@ impl TryFrom for DMNStateDiff { owner_address, voting_address, platform_node_id, - platform_p2p_port, - platform_http_port, + legacy_platform_p2p_port, + legacy_platform_http_port, payout_address, pub_key_operator, + addresses, } = value; let owner_address = owner_address @@ -2188,12 +2284,43 @@ impl TryFrom for DMNStateDiff { pub_key_operator, operator_payout_address, platform_node_id, - platform_p2p_port, - platform_http_port, + #[allow(deprecated)] + legacy_platform_p2p_port, + #[allow(deprecated)] + legacy_platform_http_port, + addresses, }) } } +impl DMNStateDiff { + /// Resolved platform P2P `(host, port)` carried by this diff, if any. + /// + /// Returns the host and port from the Core 23+ nested `addresses.platform_p2p` + /// entry. The legacy top-level `platformP2PPort` is not resolved here: a diff + /// carries no node IP to pair it with, so a host would have to be fabricated. + /// Read the legacy port via [`legacy_platform_p2p_port`](Self::legacy_platform_p2p_port). + pub fn platform_p2p_address(&self) -> Option<(String, u32)> { + self.addresses + .as_ref() + .and_then(|a| a.as_ref()) + .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_p2p)) + } + + /// Resolved platform HTTPS `(host, port)` carried by this diff, if any. + /// + /// Returns the host and port from the Core 23+ nested `addresses.platform_https` + /// entry. The legacy top-level `platformHTTPPort` is not resolved here: a diff + /// carries no node IP to pair it with, so a host would have to be fabricated. + /// Read the legacy port via [`legacy_platform_http_port`](Self::legacy_platform_http_port). + pub fn platform_http_address(&self) -> Option<(String, u32)> { + self.addresses + .as_ref() + .and_then(|a| a.as_ref()) + .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_https)) + } +} + impl DMNState { pub fn compare_to_older_dmn_state(&self, older: &DMNState) -> Option { older.compare_to_newer_dmn_state(self) @@ -2272,15 +2399,27 @@ impl DMNState { } else { None }, - platform_p2p_port: if self.platform_p2p_port != newer.platform_p2p_port { + #[allow(deprecated)] + legacy_platform_p2p_port: if self.legacy_platform_p2p_port + != newer.legacy_platform_p2p_port + { + has_diff = true; + newer.legacy_platform_p2p_port + } else { + None + }, + #[allow(deprecated)] + legacy_platform_http_port: if self.legacy_platform_http_port + != newer.legacy_platform_http_port + { has_diff = true; - newer.platform_p2p_port + newer.legacy_platform_http_port } else { None }, - platform_http_port: if self.platform_http_port != newer.platform_http_port { + addresses: if self.addresses != newer.addresses { has_diff = true; - newer.platform_http_port + Some(newer.addresses.clone()) } else { None }, @@ -2304,8 +2443,11 @@ impl DMNState { pub_key_operator, operator_payout_address, platform_node_id, - platform_p2p_port, - platform_http_port, + #[allow(deprecated)] + legacy_platform_p2p_port, + #[allow(deprecated)] + legacy_platform_http_port, + addresses, .. } = diff; self.pose_revived_height = pose_revived_height; @@ -2338,12 +2480,18 @@ impl DMNState { self.platform_node_id = Some(platform_node_id); } - if let Some(platform_p2p_port) = platform_p2p_port { - self.platform_p2p_port = Some(platform_p2p_port); + #[allow(deprecated)] + if let Some(legacy_platform_p2p_port) = legacy_platform_p2p_port { + self.legacy_platform_p2p_port = Some(legacy_platform_p2p_port); + } + + #[allow(deprecated)] + if let Some(legacy_platform_http_port) = legacy_platform_http_port { + self.legacy_platform_http_port = Some(legacy_platform_http_port); } - if let Some(platform_http_port) = platform_http_port { - self.platform_http_port = Some(platform_http_port); + if let Some(addresses) = addresses { + self.addresses = addresses; } } } @@ -2885,14 +3033,19 @@ pub struct DMNStateDiffIntermediate { pub voting_address: Option, #[serde(default, rename = "platformNodeID")] pub platform_node_id: Option, + #[deprecated(note = "Core 23+ nested addresses.platform_p2p should be used instead")] #[serde(default, rename = "platformP2PPort")] - pub platform_p2p_port: Option, + pub legacy_platform_p2p_port: Option, + #[deprecated(note = "Core 23+ nested addresses.platform_https should be used instead")] #[serde(default, rename = "platformHTTPPort")] - pub platform_http_port: Option, + pub legacy_platform_http_port: Option, #[serde(default)] pub payout_address: Option, #[serde(default, deserialize_with = "deserialize_hex_opt")] pub pub_key_operator: Option>, + // Three-state: missing field = None, `null` = Some(None), object = Some(Some(_)). + #[serde(default, deserialize_with = "deserialize_addresses_2opt")] + pub addresses: Option>, } #[derive(Clone, PartialEq, Debug, Deserialize)] @@ -3251,6 +3404,20 @@ where Ok(Some(Some(val as u32))) } +/// Deserializes the present-but-clearable nested `addresses` object. +/// +/// Paired with `#[serde(default)]`, the field absence yields `None` (unchanged), +/// a JSON `null` yields `Some(None)` (cleared), and an object yields +/// `Some(Some(_))` (set) — the canonical serde double-`Option` pattern. +fn deserialize_addresses_2opt<'de, D>( + deserializer: D, +) -> Result>, D::Error> +where + D: Deserializer<'de>, +{ + Ok(Some(Option::::deserialize(deserializer)?)) +} + #[cfg(test)] mod tests { use dashcore::hashes::Hash; @@ -3258,7 +3425,8 @@ mod tests { use serde_json::json; use crate::{ - ExtendedQuorumListResult, MasternodeListDiff, MnSyncStatus, QuorumType, deserialize_u32_opt, + DMNState, DMNStateDiff, ExtendedQuorumListResult, MasternodeAddresses, MasternodeListDiff, + MnSyncStatus, QuorumType, deserialize_u32_opt, parse_host_port, }; #[test] @@ -3435,6 +3603,313 @@ mod tests { ); } + #[test] + #[allow(deprecated)] + fn dmn_state_core23_addresses_resolve_platform_ports() { + // Core 23 entry: legacy platformP2PPort/platformHTTPPort absent, ports live + // in the nested `addresses` object. Raw fields stay None; accessors resolve. + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "addresses": { + "core_p2p": ["192.0.2.1:9999"], + "platform_p2p": ["192.0.2.2:36656"], + "platform_https": ["192.0.2.2:443"] + } + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(state.legacy_platform_p2p_port, None, "raw legacy field deserialized as-is"); + assert_eq!(state.legacy_platform_http_port, None, "raw legacy field deserialized as-is"); + assert_eq!( + state.platform_p2p_address(), + Some(("192.0.2.2".to_string(), 36656)), + "p2p resolved from addresses" + ); + assert_eq!( + state.platform_http_address(), + Some(("192.0.2.2".to_string(), 443)), + "http resolved from addresses" + ); + } + + #[test] + #[allow(deprecated)] + fn dmn_state_diff_core23_addresses_resolve_platform_ports() { + // updatedMNs entry carrying only the new `addresses` object. + let json = r#"{ + "addresses": { + "platform_p2p": ["192.0.2.2:36656"], + "platform_https": ["192.0.2.2:443"] + } + }"#; + let diff: DMNStateDiff = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(diff.legacy_platform_p2p_port, None, "raw legacy diff field deserialized as-is"); + assert_eq!( + diff.legacy_platform_http_port, None, + "raw legacy diff field deserialized as-is" + ); + assert_eq!( + diff.platform_p2p_address(), + Some(("192.0.2.2".to_string(), 36656)), + "diff p2p resolved from addresses" + ); + assert_eq!( + diff.platform_http_address(), + Some(("192.0.2.2".to_string(), 443)), + "diff http resolved from addresses" + ); + } + + #[test] + fn dmn_state_legacy_platform_ports_resolve_to_node_ip() { + // Pre-23 entry: legacy top-level keys, no `addresses`. Accessors fall back to + // the legacy port paired with the node IP from `service`. + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "platformP2PPort": 26656, + "platformHTTPPort": 443 + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert!(state.addresses.is_none(), "no addresses object present"); + assert_eq!( + state.platform_p2p_address(), + Some(("192.0.2.1".to_string(), 26656)), + "p2p resolved from legacy paired with node IP" + ); + assert_eq!( + state.platform_http_address(), + Some(("192.0.2.1".to_string(), 443)), + "http resolved from legacy paired with node IP" + ); + } + + #[test] + #[allow(deprecated)] + fn dmn_state_zero_legacy_port_resolves_to_addresses() { + // Transitional entry: legacy port present but zero -> addresses wins (new-first). + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "platformP2PPort": 0, + "addresses": { + "platform_p2p": ["192.0.2.2:36656"] + } + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(state.legacy_platform_p2p_port, Some(0), "raw legacy zero deserialized as-is"); + assert_eq!( + state.platform_p2p_address(), + Some(("192.0.2.2".to_string(), 36656)), + "zero legacy yields to addresses" + ); + } + + #[test] + #[allow(deprecated)] + fn dmn_state_zero_legacy_port_no_addresses_resolves_to_none() { + // Legacy port present but zero and no `addresses` -> accessor returns None, + // never `(host, 0)`. + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "platformP2PPort": 0 + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(state.legacy_platform_p2p_port, Some(0), "raw legacy zero deserialized as-is"); + assert_eq!(state.platform_p2p_address(), None, "zero legacy with no addresses -> None"); + } + + #[test] + fn dmn_state_out_of_range_port_rejected() { + // A port above the u16 range must be rejected rather than truncated/accepted. + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "addresses": { + "platform_p2p": ["192.0.2.2:70000"] + } + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(state.platform_p2p_address(), None, "out-of-range port rejected"); + } + + #[test] + fn dmn_state_legacy_out_of_range_port_rejected() { + // A legacy port above the u16 range must be rejected, matching the addresses + // path, so the accessor honors its documented in-range invariant. + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "platformP2PPort": 70000 + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(state.platform_p2p_address(), None, "out-of-range legacy port rejected"); + } + + #[test] + fn dmn_state_no_ports_resolve_to_none() { + // No addresses and no legacy ports -> accessors return None. + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5" + }"#; + let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); + assert_eq!(state.platform_p2p_address(), None, "no source -> None"); + assert_eq!(state.platform_http_address(), None, "no source -> None"); + } + + fn dmn_state_with_legacy_p2p_zero() -> DMNState { + let json = r#"{ + "service": "192.0.2.1:9999", + "registeredHeight": 123456, + "revocationReason": 0, + "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", + "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", + "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", + "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", + "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", + "platformP2PPort": 0 + }"#; + serde_json::from_str(json).expect("expected to deserialize json") + } + + #[test] + fn dmn_state_apply_diff_propagates_addresses() { + // Stored entry has a zero legacy port and no addresses; a diff carrying a + // nested `addresses` object must make the merged state resolvable. + let mut state = dmn_state_with_legacy_p2p_zero(); + assert_eq!(state.platform_p2p_address(), None, "unresolvable before diff"); + + let diff = DMNStateDiff { + service: None, + registered_height: None, + last_paid_height: None, + consecutive_payments: None, + pose_penalty: None, + pose_revived_height: None, + pose_ban_height: None, + revocation_reason: None, + owner_address: None, + voting_address: None, + payout_address: None, + pub_key_operator: None, + operator_payout_address: None, + platform_node_id: None, + #[allow(deprecated)] + legacy_platform_p2p_port: None, + #[allow(deprecated)] + legacy_platform_http_port: None, + addresses: Some(Some(MasternodeAddresses { + core_p2p: vec![], + platform_p2p: vec!["192.0.2.2:36656".to_string()], + platform_https: vec![], + })), + }; + + state.apply_diff(diff); + assert_eq!( + state.platform_p2p_address(), + Some(("192.0.2.2".to_string(), 36656)), + "diff addresses propagated and resolvable" + ); + } + + #[test] + fn dmn_state_diff_clears_addresses() { + // A Some -> None transition must survive the compare/apply round-trip: the + // diff carries `Some(None)` and applying it clears the stored addresses. + let mut newer = dmn_state_with_legacy_p2p_zero(); + newer.addresses = Some(MasternodeAddresses { + core_p2p: vec![], + platform_p2p: vec!["192.0.2.2:36656".to_string()], + platform_https: vec![], + }); + let older = dmn_state_with_legacy_p2p_zero(); + + let diff = + newer.compare_to_newer_dmn_state(&older).expect("addresses change yields a diff"); + assert_eq!(diff.addresses, Some(None), "clear is encoded as Some(None)"); + + let mut applied = newer; + applied.apply_diff(diff); + assert!(applied.addresses.is_none(), "Some(None) diff clears stored addresses"); + } + + #[test] + fn dmn_state_diff_addresses_null_wire_clears() { + // Wire-level three-state: `null` -> Some(None) (clear), absent -> None + // (unchanged). Exercises `deserialize_addresses_2opt` through the intermediate. + let diff: DMNStateDiff = + serde_json::from_str(r#"{"addresses": null}"#).expect("expected to deserialize json"); + assert_eq!(diff.addresses, Some(None), "null wire -> Some(None) (clear)"); + + let diff: DMNStateDiff = + serde_json::from_str(r#"{}"#).expect("expected to deserialize json"); + assert_eq!(diff.addresses, None, "absent wire -> None (unchanged)"); + } + + #[test] + fn parse_host_port_ipv6() { + // Bracketed IPv6 keeps host intact; unbracketed (ambiguous) is rejected. + assert_eq!( + parse_host_port("[2001:db8::1]:9999"), + Some(("[2001:db8::1]".to_string(), 9999)), + "bracketed IPv6 parses host + port" + ); + assert_eq!(parse_host_port("2001:db8::1"), None, "unbracketed IPv6 rejected"); + assert_eq!( + parse_host_port("192.0.2.1:9999"), + Some(("192.0.2.1".to_string(), 9999)), + "IPv4 still parses" + ); + // Empty host must be rejected. + assert_eq!(parse_host_port(":36656"), None, "empty host rejected"); + assert_eq!(parse_host_port(":443"), None, "empty host rejected"); + } + #[test] fn deserialize_mnsync_status() { let json_value = json!({ From 64d9267d414da2c01c604c12d95dd1b12a764bb1 Mon Sep 17 00:00:00 2001 From: Ivan Shumkov Date: Thu, 18 Jun 2026 16:29:43 +0700 Subject: [PATCH 002/149] fix(key-wallet): honor account index in DashPay derivation path (#813) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AccountType::derivation_path() for DashpayReceivingFunds / DashpayExternalAccount pushed a hardcoded `0'` for the account segment and discarded the variant's own `index` field (destructured with `..`). The DashPay friendship path is m/9'/coin'/15'/account'// (DIP-15 "Account Reference" + DIP-14), where the account segment is the sender's DashPay account — see DIP-15's multi-account example (personal = account 0, professional = account 1). dashj's FriendKeyChain already parameterizes this segment; we were the outlier. Effect: a multi-account wallet derived every relationship under account 0, so a non-zero account watched the wrong addresses and would miss funds. Now the segment uses `*index`. Account 0 — the only value in use today — is byte-identical to the old path, so this is backward-compatible; it only unlocks non-zero accounts. Test would have caught this in CI: test_dashpay_account_index_in_derivation_path asserts the account segment equals the index for accounts {0,1,7,42} on both the receiving and external channels (plus id ordering). ✖ before fix (1/7/42 derived 0'), ✔ after. Co-authored-by: Claude Opus 4.8 (1M context) --- key-wallet/src/account/account_type.rs | 22 +++++-- key-wallet/src/tests/account_tests.rs | 79 ++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 6 deletions(-) diff --git a/key-wallet/src/account/account_type.rs b/key-wallet/src/account/account_type.rs index f5ce4961c..d027317bf 100644 --- a/key-wallet/src/account/account_type.rs +++ b/key-wallet/src/account/account_type.rs @@ -470,11 +470,13 @@ impl AccountType { ])) } Self::DashpayReceivingFunds { + index: account_index, user_identity_id, friend_identity_id, - .. } => { - // Base DashPay root + account 0' + user_id/friend_id (non-hardened per DIP-14/DIP-15) + // Base DashPay root + account' + user_id/friend_id (non-hardened per DIP-14/DIP-15). + // The account is the sender's DashPay account (DIP-15 "Account Reference"), + // hardened; defaults to 0 but may differ for multi-account wallets. let mut path = match network { Network::Mainnet => { DerivationPath::from(crate::dip9::DASHPAY_ROOT_PATH_MAINNET) @@ -483,7 +485,10 @@ impl AccountType { DerivationPath::from(crate::dip9::DASHPAY_ROOT_PATH_TESTNET) } }; - path.push(ChildNumber::from_hardened_idx(0).map_err(crate::error::Error::Bip32)?); + path.push( + ChildNumber::from_hardened_idx(*account_index) + .map_err(crate::error::Error::Bip32)?, + ); path.push(ChildNumber::Normal256 { index: *user_identity_id, }); @@ -493,11 +498,13 @@ impl AccountType { Ok(path) } Self::DashpayExternalAccount { + index: account_index, user_identity_id, friend_identity_id, - .. } => { - // Base DashPay root + account 0' + friend_id/user_id (non-hardened per DIP-14/DIP-15) + // Base DashPay root + account' + friend_id/user_id (non-hardened per DIP-14/DIP-15). + // The account is the sender's DashPay account (DIP-15 "Account Reference"), + // hardened; defaults to 0 but may differ for multi-account wallets. let mut path = match network { Network::Mainnet => { DerivationPath::from(crate::dip9::DASHPAY_ROOT_PATH_MAINNET) @@ -506,7 +513,10 @@ impl AccountType { DerivationPath::from(crate::dip9::DASHPAY_ROOT_PATH_TESTNET) } }; - path.push(ChildNumber::from_hardened_idx(0).map_err(crate::error::Error::Bip32)?); + path.push( + ChildNumber::from_hardened_idx(*account_index) + .map_err(crate::error::Error::Bip32)?, + ); path.push(ChildNumber::Normal256 { index: *friend_identity_id, }); diff --git a/key-wallet/src/tests/account_tests.rs b/key-wallet/src/tests/account_tests.rs index d21b238bf..1732823db 100644 --- a/key-wallet/src/tests/account_tests.rs +++ b/key-wallet/src/tests/account_tests.rs @@ -549,3 +549,82 @@ fn test_account_derivation_path_uniqueness() { paths.push(path_str); } } + +#[test] +fn test_dashpay_account_index_in_derivation_path() { + use crate::bip32::ChildNumber; + + // The DashPay friendship path is m/9'/coin'/15'/account'// (DIP-15 "Account + // Reference" + DIP-14 256-bit indices). The account segment is the sender's DashPay + // account and MUST reflect the account index carried on the account type — not a fixed + // 0'. A multi-account wallet that derived every relationship under account 0 would + // watch the wrong addresses and miss funds paid to a non-zero account. Account 0 must + // stay byte-identical to the legacy single-account path (backward compatibility). + let network = Network::Testnet; + let user_id = [0x11u8; 32]; + let friend_id = [0x22u8; 32]; + + for account in [0u32, 1, 7, 42] { + // Receiving funds: identity ids ordered user/friend. + let recv = AccountType::DashpayReceivingFunds { + index: account, + user_identity_id: user_id, + friend_identity_id: friend_id, + }; + let recv_path = recv.derivation_path(network).unwrap(); + let recv_comps: Vec = recv_path.clone().into(); + assert_eq!(recv_comps.len(), 6, "root(3) + account' + two 256-bit ids"); + assert_eq!( + recv_comps[3], + ChildNumber::Hardened { + index: account + }, + "receiving account segment must honor the account index" + ); + assert_eq!( + recv_comps[4], + ChildNumber::Normal256 { + index: user_id + } + ); + assert_eq!( + recv_comps[5], + ChildNumber::Normal256 { + index: friend_id + } + ); + assert!( + recv_path.to_string().starts_with(&format!("m/9'/1'/15'/{}'/", account)), + "receiving path should carry account {}': got {}", + account, + recv_path + ); + + // External account: the reverse channel, ids ordered friend/user. + let ext = AccountType::DashpayExternalAccount { + index: account, + user_identity_id: user_id, + friend_identity_id: friend_id, + }; + let ext_comps: Vec = ext.derivation_path(network).unwrap().into(); + assert_eq!( + ext_comps[3], + ChildNumber::Hardened { + index: account + }, + "external account segment must honor the account index" + ); + assert_eq!( + ext_comps[4], + ChildNumber::Normal256 { + index: friend_id + } + ); + assert_eq!( + ext_comps[5], + ChildNumber::Normal256 { + index: user_id + } + ); + } +} From ceee4a9b40d0ed8b0402144d841ff5d7ead3774c Mon Sep 17 00:00:00 2001 From: Ivan Shumkov Date: Fri, 19 Jun 2026 17:57:41 +0700 Subject: [PATCH 003/149] revert(rpc-json)!: resolve Core 23 nested platform addresses (#814) This reverts commit 0b056c200ceb6fba3e206fe4481d9d6d34995a02. --- rpc-json/src/lib.rs | 517 ++------------------------------------------ 1 file changed, 21 insertions(+), 496 deletions(-) diff --git a/rpc-json/src/lib.rs b/rpc-json/src/lib.rs index 5d4e7015e..6416f2857 100644 --- a/rpc-json/src/lib.rs +++ b/rpc-json/src/lib.rs @@ -2052,51 +2052,6 @@ pub struct GetMasternodePaymentsResult { pub masternodes: Vec, } -/// Nested `addresses` object on Core 23+ masternode entries. -/// -/// Each purpose maps to an array of `"host:port"` strings. Dash Core 23 moved the -/// platform ports here, away from the deprecated top-level `platformP2PPort`/ -/// `platformHTTPPort` keys. Unknown purposes are ignored. -#[derive(Clone, PartialEq, Eq, Debug, Default, Deserialize, Serialize)] -pub struct MasternodeAddresses { - #[serde(default)] - pub core_p2p: Vec, - #[serde(default)] - pub platform_p2p: Vec, - #[serde(default)] - pub platform_https: Vec, -} - -impl MasternodeAddresses { - /// First valid `(host, port)` from a `"host:port"` array, if any. - /// - /// "Valid" means parseable and with a non-zero in-range port; zero ports are - /// skipped because Dash Core uses `0` as a "not set" sentinel. - fn first_valid_host_port(addrs: &[String]) -> Option<(String, u32)> { - addrs.iter().filter_map(|a| parse_host_port(a)).find(|(_, p)| *p != 0) - } -} - -/// Splits a `"host:port"` string into `(host, port)` with a non-fabricated host. -/// -/// Supports IPv4 (`1.2.3.4:9999`) and bracketed IPv6 (`[2001:db8::1]:9999`). The -/// final `:port` segment is parsed through [`u16`] then widened to `u32`, rejecting -/// values outside the TCP/UDP port range (e.g. `"host:70000"`). Unbracketed -/// multi-colon hosts (bare IPv6) are rejected rather than silently mangled. An empty -/// host (e.g. `":36656"`) is also rejected. Returns `None` when no colon is present, -/// the host is empty or ambiguous, or the suffix is not a valid `u16`. -fn parse_host_port(addr: &str) -> Option<(String, u32)> { - let (host, port) = addr.rsplit_once(':')?; - if host.is_empty() { - return None; - } - if host.contains(':') && !(host.starts_with('[') && host.ends_with(']')) { - return None; - } - let port = port.parse::().ok()?; - Some((host.to_string(), u32::from(port))) -} - #[serde_as] #[derive(Clone, PartialEq, Eq, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] @@ -2125,54 +2080,10 @@ pub struct DMNState { rename = "platformNodeID" )] pub platform_node_id: Option<[u8; 20]>, - #[deprecated(note = "Core 23+ nested addresses.platform_p2p should be used instead")] #[serde(default, rename = "platformP2PPort")] - pub legacy_platform_p2p_port: Option, - #[deprecated(note = "Core 23+ nested addresses.platform_https should be used instead")] + pub platform_p2p_port: Option, #[serde(default, rename = "platformHTTPPort")] - pub legacy_platform_http_port: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - pub addresses: Option, -} - -impl DMNState { - /// Resolved platform P2P `(host, port)`. - /// - /// Prefers the Core 23+ nested `addresses.platform_p2p` entry, returning its - /// host and port verbatim. Falls back to the deprecated top-level - /// `platformP2PPort`, pairing it with the node IP from [`service`](Self::service) - /// because Dash deploys platform services on the masternode's core IP. Returns - /// `None` when no source yields a non-zero in-range port. - #[allow(deprecated)] - pub fn platform_p2p_address(&self) -> Option<(String, u32)> { - self.addresses - .as_ref() - .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_p2p)) - .or_else(|| self.legacy_platform_address(self.legacy_platform_p2p_port)) - } - - /// Resolved platform HTTPS `(host, port)`. - /// - /// Prefers the Core 23+ nested `addresses.platform_https` entry, returning its - /// host and port verbatim. Falls back to the deprecated top-level - /// `platformHTTPPort`, pairing it with the node IP from [`service`](Self::service) - /// because Dash deploys platform services on the masternode's core IP. Returns - /// `None` when no source yields a non-zero in-range port. - #[allow(deprecated)] - pub fn platform_http_address(&self) -> Option<(String, u32)> { - self.addresses - .as_ref() - .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_https)) - .or_else(|| self.legacy_platform_address(self.legacy_platform_http_port)) - } - - /// Pairs a legacy platform port with the node IP, dropping zero, absent, and - /// out-of-`u16`-range ports so the result honors the TCP/UDP port range. - fn legacy_platform_address(&self, port: Option) -> Option<(String, u32)> { - port.and_then(|p| u16::try_from(p).ok()) - .filter(|&p| p != 0) - .map(|p| (self.service.ip().to_string(), u32::from(p))) - } + pub platform_http_port: Option, } #[derive(Clone, PartialEq, Eq, Debug, Deserialize)] @@ -2192,19 +2103,13 @@ pub struct DMNStateDiff { pub pub_key_operator: Option>, pub operator_payout_address: Option>, pub platform_node_id: Option<[u8; 20]>, - #[deprecated(note = "Core 23+ nested addresses.platform_p2p should be used instead")] - pub legacy_platform_p2p_port: Option, - #[deprecated(note = "Core 23+ nested addresses.platform_https should be used instead")] - pub legacy_platform_http_port: Option, - /// Three-state nested addresses: `None` = unchanged, `Some(None)` = cleared, - /// `Some(Some(_))` = set. Mirrors [`pose_ban_height`](Self::pose_ban_height). - pub addresses: Option>, + pub platform_p2p_port: Option, + pub platform_http_port: Option, } impl TryFrom for DMNStateDiff { type Error = encode::Error; - #[allow(deprecated)] fn try_from(value: DMNStateDiffIntermediate) -> Result { let DMNStateDiffIntermediate { service, @@ -2218,11 +2123,10 @@ impl TryFrom for DMNStateDiff { owner_address, voting_address, platform_node_id, - legacy_platform_p2p_port, - legacy_platform_http_port, + platform_p2p_port, + platform_http_port, payout_address, pub_key_operator, - addresses, } = value; let owner_address = owner_address @@ -2284,43 +2188,12 @@ impl TryFrom for DMNStateDiff { pub_key_operator, operator_payout_address, platform_node_id, - #[allow(deprecated)] - legacy_platform_p2p_port, - #[allow(deprecated)] - legacy_platform_http_port, - addresses, + platform_p2p_port, + platform_http_port, }) } } -impl DMNStateDiff { - /// Resolved platform P2P `(host, port)` carried by this diff, if any. - /// - /// Returns the host and port from the Core 23+ nested `addresses.platform_p2p` - /// entry. The legacy top-level `platformP2PPort` is not resolved here: a diff - /// carries no node IP to pair it with, so a host would have to be fabricated. - /// Read the legacy port via [`legacy_platform_p2p_port`](Self::legacy_platform_p2p_port). - pub fn platform_p2p_address(&self) -> Option<(String, u32)> { - self.addresses - .as_ref() - .and_then(|a| a.as_ref()) - .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_p2p)) - } - - /// Resolved platform HTTPS `(host, port)` carried by this diff, if any. - /// - /// Returns the host and port from the Core 23+ nested `addresses.platform_https` - /// entry. The legacy top-level `platformHTTPPort` is not resolved here: a diff - /// carries no node IP to pair it with, so a host would have to be fabricated. - /// Read the legacy port via [`legacy_platform_http_port`](Self::legacy_platform_http_port). - pub fn platform_http_address(&self) -> Option<(String, u32)> { - self.addresses - .as_ref() - .and_then(|a| a.as_ref()) - .and_then(|a| MasternodeAddresses::first_valid_host_port(&a.platform_https)) - } -} - impl DMNState { pub fn compare_to_older_dmn_state(&self, older: &DMNState) -> Option { older.compare_to_newer_dmn_state(self) @@ -2399,27 +2272,15 @@ impl DMNState { } else { None }, - #[allow(deprecated)] - legacy_platform_p2p_port: if self.legacy_platform_p2p_port - != newer.legacy_platform_p2p_port - { - has_diff = true; - newer.legacy_platform_p2p_port - } else { - None - }, - #[allow(deprecated)] - legacy_platform_http_port: if self.legacy_platform_http_port - != newer.legacy_platform_http_port - { + platform_p2p_port: if self.platform_p2p_port != newer.platform_p2p_port { has_diff = true; - newer.legacy_platform_http_port + newer.platform_p2p_port } else { None }, - addresses: if self.addresses != newer.addresses { + platform_http_port: if self.platform_http_port != newer.platform_http_port { has_diff = true; - Some(newer.addresses.clone()) + newer.platform_http_port } else { None }, @@ -2443,11 +2304,8 @@ impl DMNState { pub_key_operator, operator_payout_address, platform_node_id, - #[allow(deprecated)] - legacy_platform_p2p_port, - #[allow(deprecated)] - legacy_platform_http_port, - addresses, + platform_p2p_port, + platform_http_port, .. } = diff; self.pose_revived_height = pose_revived_height; @@ -2480,18 +2338,12 @@ impl DMNState { self.platform_node_id = Some(platform_node_id); } - #[allow(deprecated)] - if let Some(legacy_platform_p2p_port) = legacy_platform_p2p_port { - self.legacy_platform_p2p_port = Some(legacy_platform_p2p_port); - } - - #[allow(deprecated)] - if let Some(legacy_platform_http_port) = legacy_platform_http_port { - self.legacy_platform_http_port = Some(legacy_platform_http_port); + if let Some(platform_p2p_port) = platform_p2p_port { + self.platform_p2p_port = Some(platform_p2p_port); } - if let Some(addresses) = addresses { - self.addresses = addresses; + if let Some(platform_http_port) = platform_http_port { + self.platform_http_port = Some(platform_http_port); } } } @@ -3033,19 +2885,14 @@ pub struct DMNStateDiffIntermediate { pub voting_address: Option, #[serde(default, rename = "platformNodeID")] pub platform_node_id: Option, - #[deprecated(note = "Core 23+ nested addresses.platform_p2p should be used instead")] #[serde(default, rename = "platformP2PPort")] - pub legacy_platform_p2p_port: Option, - #[deprecated(note = "Core 23+ nested addresses.platform_https should be used instead")] + pub platform_p2p_port: Option, #[serde(default, rename = "platformHTTPPort")] - pub legacy_platform_http_port: Option, + pub platform_http_port: Option, #[serde(default)] pub payout_address: Option, #[serde(default, deserialize_with = "deserialize_hex_opt")] pub pub_key_operator: Option>, - // Three-state: missing field = None, `null` = Some(None), object = Some(Some(_)). - #[serde(default, deserialize_with = "deserialize_addresses_2opt")] - pub addresses: Option>, } #[derive(Clone, PartialEq, Debug, Deserialize)] @@ -3404,20 +3251,6 @@ where Ok(Some(Some(val as u32))) } -/// Deserializes the present-but-clearable nested `addresses` object. -/// -/// Paired with `#[serde(default)]`, the field absence yields `None` (unchanged), -/// a JSON `null` yields `Some(None)` (cleared), and an object yields -/// `Some(Some(_))` (set) — the canonical serde double-`Option` pattern. -fn deserialize_addresses_2opt<'de, D>( - deserializer: D, -) -> Result>, D::Error> -where - D: Deserializer<'de>, -{ - Ok(Some(Option::::deserialize(deserializer)?)) -} - #[cfg(test)] mod tests { use dashcore::hashes::Hash; @@ -3425,8 +3258,7 @@ mod tests { use serde_json::json; use crate::{ - DMNState, DMNStateDiff, ExtendedQuorumListResult, MasternodeAddresses, MasternodeListDiff, - MnSyncStatus, QuorumType, deserialize_u32_opt, parse_host_port, + ExtendedQuorumListResult, MasternodeListDiff, MnSyncStatus, QuorumType, deserialize_u32_opt, }; #[test] @@ -3603,313 +3435,6 @@ mod tests { ); } - #[test] - #[allow(deprecated)] - fn dmn_state_core23_addresses_resolve_platform_ports() { - // Core 23 entry: legacy platformP2PPort/platformHTTPPort absent, ports live - // in the nested `addresses` object. Raw fields stay None; accessors resolve. - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "addresses": { - "core_p2p": ["192.0.2.1:9999"], - "platform_p2p": ["192.0.2.2:36656"], - "platform_https": ["192.0.2.2:443"] - } - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(state.legacy_platform_p2p_port, None, "raw legacy field deserialized as-is"); - assert_eq!(state.legacy_platform_http_port, None, "raw legacy field deserialized as-is"); - assert_eq!( - state.platform_p2p_address(), - Some(("192.0.2.2".to_string(), 36656)), - "p2p resolved from addresses" - ); - assert_eq!( - state.platform_http_address(), - Some(("192.0.2.2".to_string(), 443)), - "http resolved from addresses" - ); - } - - #[test] - #[allow(deprecated)] - fn dmn_state_diff_core23_addresses_resolve_platform_ports() { - // updatedMNs entry carrying only the new `addresses` object. - let json = r#"{ - "addresses": { - "platform_p2p": ["192.0.2.2:36656"], - "platform_https": ["192.0.2.2:443"] - } - }"#; - let diff: DMNStateDiff = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(diff.legacy_platform_p2p_port, None, "raw legacy diff field deserialized as-is"); - assert_eq!( - diff.legacy_platform_http_port, None, - "raw legacy diff field deserialized as-is" - ); - assert_eq!( - diff.platform_p2p_address(), - Some(("192.0.2.2".to_string(), 36656)), - "diff p2p resolved from addresses" - ); - assert_eq!( - diff.platform_http_address(), - Some(("192.0.2.2".to_string(), 443)), - "diff http resolved from addresses" - ); - } - - #[test] - fn dmn_state_legacy_platform_ports_resolve_to_node_ip() { - // Pre-23 entry: legacy top-level keys, no `addresses`. Accessors fall back to - // the legacy port paired with the node IP from `service`. - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "platformP2PPort": 26656, - "platformHTTPPort": 443 - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert!(state.addresses.is_none(), "no addresses object present"); - assert_eq!( - state.platform_p2p_address(), - Some(("192.0.2.1".to_string(), 26656)), - "p2p resolved from legacy paired with node IP" - ); - assert_eq!( - state.platform_http_address(), - Some(("192.0.2.1".to_string(), 443)), - "http resolved from legacy paired with node IP" - ); - } - - #[test] - #[allow(deprecated)] - fn dmn_state_zero_legacy_port_resolves_to_addresses() { - // Transitional entry: legacy port present but zero -> addresses wins (new-first). - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "platformP2PPort": 0, - "addresses": { - "platform_p2p": ["192.0.2.2:36656"] - } - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(state.legacy_platform_p2p_port, Some(0), "raw legacy zero deserialized as-is"); - assert_eq!( - state.platform_p2p_address(), - Some(("192.0.2.2".to_string(), 36656)), - "zero legacy yields to addresses" - ); - } - - #[test] - #[allow(deprecated)] - fn dmn_state_zero_legacy_port_no_addresses_resolves_to_none() { - // Legacy port present but zero and no `addresses` -> accessor returns None, - // never `(host, 0)`. - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "platformP2PPort": 0 - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(state.legacy_platform_p2p_port, Some(0), "raw legacy zero deserialized as-is"); - assert_eq!(state.platform_p2p_address(), None, "zero legacy with no addresses -> None"); - } - - #[test] - fn dmn_state_out_of_range_port_rejected() { - // A port above the u16 range must be rejected rather than truncated/accepted. - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "addresses": { - "platform_p2p": ["192.0.2.2:70000"] - } - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(state.platform_p2p_address(), None, "out-of-range port rejected"); - } - - #[test] - fn dmn_state_legacy_out_of_range_port_rejected() { - // A legacy port above the u16 range must be rejected, matching the addresses - // path, so the accessor honors its documented in-range invariant. - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "platformP2PPort": 70000 - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(state.platform_p2p_address(), None, "out-of-range legacy port rejected"); - } - - #[test] - fn dmn_state_no_ports_resolve_to_none() { - // No addresses and no legacy ports -> accessors return None. - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5" - }"#; - let state: DMNState = serde_json::from_str(json).expect("expected to deserialize json"); - assert_eq!(state.platform_p2p_address(), None, "no source -> None"); - assert_eq!(state.platform_http_address(), None, "no source -> None"); - } - - fn dmn_state_with_legacy_p2p_zero() -> DMNState { - let json = r#"{ - "service": "192.0.2.1:9999", - "registeredHeight": 123456, - "revocationReason": 0, - "ownerAddress": "yPBWCdMRY5PsS3hJzs7csbdWQVRR85yxUz", - "votingAddress": "ySM11LUD65Bi4p1gm68XLkdWc65TBKRzvQ", - "payoutAddress": "yX4Ve7Q8Y4jscV4LZJD8HVCHKyePzR3MhA", - "pubKeyOperator": "8ed3f0c208efbcfc815cbfb94490dc68cf2e29d44dd9f8a91e20e06057aa110d7062c8ab7ccc85a9ff0c88760157f563", - "platformNodeID": "f2dbd9b0a1f541a7c44d34a58674d0262f5feca5", - "platformP2PPort": 0 - }"#; - serde_json::from_str(json).expect("expected to deserialize json") - } - - #[test] - fn dmn_state_apply_diff_propagates_addresses() { - // Stored entry has a zero legacy port and no addresses; a diff carrying a - // nested `addresses` object must make the merged state resolvable. - let mut state = dmn_state_with_legacy_p2p_zero(); - assert_eq!(state.platform_p2p_address(), None, "unresolvable before diff"); - - let diff = DMNStateDiff { - service: None, - registered_height: None, - last_paid_height: None, - consecutive_payments: None, - pose_penalty: None, - pose_revived_height: None, - pose_ban_height: None, - revocation_reason: None, - owner_address: None, - voting_address: None, - payout_address: None, - pub_key_operator: None, - operator_payout_address: None, - platform_node_id: None, - #[allow(deprecated)] - legacy_platform_p2p_port: None, - #[allow(deprecated)] - legacy_platform_http_port: None, - addresses: Some(Some(MasternodeAddresses { - core_p2p: vec![], - platform_p2p: vec!["192.0.2.2:36656".to_string()], - platform_https: vec![], - })), - }; - - state.apply_diff(diff); - assert_eq!( - state.platform_p2p_address(), - Some(("192.0.2.2".to_string(), 36656)), - "diff addresses propagated and resolvable" - ); - } - - #[test] - fn dmn_state_diff_clears_addresses() { - // A Some -> None transition must survive the compare/apply round-trip: the - // diff carries `Some(None)` and applying it clears the stored addresses. - let mut newer = dmn_state_with_legacy_p2p_zero(); - newer.addresses = Some(MasternodeAddresses { - core_p2p: vec![], - platform_p2p: vec!["192.0.2.2:36656".to_string()], - platform_https: vec![], - }); - let older = dmn_state_with_legacy_p2p_zero(); - - let diff = - newer.compare_to_newer_dmn_state(&older).expect("addresses change yields a diff"); - assert_eq!(diff.addresses, Some(None), "clear is encoded as Some(None)"); - - let mut applied = newer; - applied.apply_diff(diff); - assert!(applied.addresses.is_none(), "Some(None) diff clears stored addresses"); - } - - #[test] - fn dmn_state_diff_addresses_null_wire_clears() { - // Wire-level three-state: `null` -> Some(None) (clear), absent -> None - // (unchanged). Exercises `deserialize_addresses_2opt` through the intermediate. - let diff: DMNStateDiff = - serde_json::from_str(r#"{"addresses": null}"#).expect("expected to deserialize json"); - assert_eq!(diff.addresses, Some(None), "null wire -> Some(None) (clear)"); - - let diff: DMNStateDiff = - serde_json::from_str(r#"{}"#).expect("expected to deserialize json"); - assert_eq!(diff.addresses, None, "absent wire -> None (unchanged)"); - } - - #[test] - fn parse_host_port_ipv6() { - // Bracketed IPv6 keeps host intact; unbracketed (ambiguous) is rejected. - assert_eq!( - parse_host_port("[2001:db8::1]:9999"), - Some(("[2001:db8::1]".to_string(), 9999)), - "bracketed IPv6 parses host + port" - ); - assert_eq!(parse_host_port("2001:db8::1"), None, "unbracketed IPv6 rejected"); - assert_eq!( - parse_host_port("192.0.2.1:9999"), - Some(("192.0.2.1".to_string(), 9999)), - "IPv4 still parses" - ); - // Empty host must be rejected. - assert_eq!(parse_host_port(":36656"), None, "empty host rejected"); - assert_eq!(parse_host_port(":443"), None, "empty host rejected"); - } - #[test] fn deserialize_mnsync_status() { let json_value = json!({ From 95a3c8ff04d75d7abe324db4af01e8521092aff9 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Mon, 22 Jun 2026 17:46:09 +1000 Subject: [PATCH 004/149] fix(dash-spv): prevent filter sync crash on new block after full sync (#816) When a wallet boots already fully synced, `start_download` takes its early return before assigning `processing_height`, leaving it at the initial 0. The next block that extends the chain then drives `try_create_lookahead_batches`, which falls back to `processing_height` when no batches are active and creates a lookahead batch starting at height 0. Loading filters from height 0, which were never stored under checkpoint sync, reads an empty leading segment and panics in `SegmentCache::get_items` (debug) or returns sentinel filters as real data (release). Set `processing_height` to the computed `scan_start` before the early-return check so the scan frontier is recorded on every path. The genesis case (`committed_height == 0`) keeps starting at the wallet birth height because `scan_start` already encodes that distinction. --- dash-spv/src/sync/filters/manager.rs | 52 +++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index 2464466b2..4128333f1 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -191,6 +191,13 @@ impl self.progress.filter_header_tip_height() { // Only emit FiltersSyncComplete if we've also reached the chain tip @@ -218,7 +225,6 @@ impl Date: Wed, 24 Jun 2026 13:52:50 +0700 Subject: [PATCH 005/149] feat(key-wallet): add Signer::extended_public_key (#817) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Signer trait exposes sign_ecdsa + public_key, the latter returning only the compressed leaf point. A caller that needs a BIP-32 extended public key at a hardened path — e.g. DashPay contact-xpub derivation under m/9'/coin'/15'/account'/sender/recipient — could not obtain the chain code without the wallet's resident root private key, forcing the seed to be made resident. Add a required extended_public_key(path) -> ExtendedPubKey method so a Keychain/hardware-backed signer can return the point + chain code + parent fingerprint at a (possibly hardened) path, letting the caller non-hardened-derive descendants offline with no resident seed. A signer that cannot export an xpub at a hardened path returns an error rather than panicking. The two in-repo InMemorySigner test signers derive it from their root xprv (ExtendedPubKey::from_priv); the NoDigestSigner test stubs are unreachable. Pinned by a test asserting the signer's xpub at a hardened path equals the wallet's own derive_extended_public_key. Required (not defaulted): the trait's associated Error type is opaque, so a default body cannot construct a Self::Error to return; there are no production Signer impls in this crate. Co-authored-by: Claude Opus 4.8 (1M context) --- key-wallet/src/signer.rs | 17 +++++++- .../managed_wallet_info/asset_lock_builder.rs | 19 +++++++++ .../transaction_building.rs | 40 +++++++++++++++++++ 3 files changed, 75 insertions(+), 1 deletion(-) diff --git a/key-wallet/src/signer.rs b/key-wallet/src/signer.rs index a5a97f804..3435289aa 100644 --- a/key-wallet/src/signer.rs +++ b/key-wallet/src/signer.rs @@ -14,7 +14,7 @@ use async_trait::async_trait; use secp256k1::{ecdsa, PublicKey}; -use crate::bip32::DerivationPath; +use crate::bip32::{DerivationPath, ExtendedPubKey}; /// A signing method a [`Signer`] can perform. /// @@ -125,4 +125,19 @@ pub trait Signer: Send + Sync { /// keys) that the caller later references when signing Platform state /// transitions. async fn public_key(&self, path: &DerivationPath) -> Result; + + /// Return the BIP-32 extended public key at `path` — the public point + /// plus the chain code and parent fingerprint, so the caller can + /// non-hardened-derive a whole range of descendants from a single + /// (possibly hardened) request without further signer round-trips + /// (e.g. DashPay contact payment addresses under + /// `m/9'/coin'/15'/account'/sender/recipient`). + /// + /// Distinct from [`Self::public_key`], which returns only the leaf point + /// (no chain code). A signer that cannot export an extended public key at + /// a hardened path should return an error rather than panic. + async fn extended_public_key( + &self, + path: &DerivationPath, + ) -> Result; } diff --git a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs index 8f8fa9bb5..56c9dad4c 100644 --- a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs @@ -469,6 +469,19 @@ mod tests { .map_err(|e| e.to_string())?; Ok(secp256k1::PublicKey::from_secret_key(&secp, &xpriv.private_key)) } + + async fn extended_public_key( + &self, + path: &DerivationPath, + ) -> Result { + let secp = secp256k1::Secp256k1::new(); + let xpriv = self + .root + .to_extended_priv_key(self.network) + .derive_priv(&secp, path) + .map_err(|e| e.to_string())?; + Ok(crate::bip32::ExtendedPubKey::from_priv(&secp, &xpriv)) + } } #[tokio::test] @@ -537,6 +550,12 @@ mod tests { async fn public_key(&self, _: &DerivationPath) -> Result { unreachable!() } + async fn extended_public_key( + &self, + _: &DerivationPath, + ) -> Result { + unreachable!() + } } let (wallet, mut info) = test_wallet_and_info(); diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index 492b35dda..c553c59dc 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -384,6 +384,19 @@ mod tests { .map_err(|e| e.to_string())?; Ok(secp256k1::PublicKey::from_secret_key(&secp, &xpriv.private_key)) } + + async fn extended_public_key( + &self, + path: &DerivationPath, + ) -> Result { + let secp = secp256k1::Secp256k1::new(); + let xpriv = self + .root + .to_extended_priv_key(self.network) + .derive_priv(&secp, path) + .map_err(|e| e.to_string())?; + Ok(crate::bip32::ExtendedPubKey::from_priv(&secp, &xpriv)) + } } fn root_from(wallet: &Wallet) -> crate::wallet::root_extended_keys::RootExtendedPrivKey { @@ -396,6 +409,27 @@ mod tests { } } + #[tokio::test] + async fn in_memory_signer_extended_public_key_matches_wallet_derivation() { + use crate::bip32::DerivationPath; + use std::str::FromStr; + let (wallet, _info) = test_wallet_and_info(); + let signer = InMemorySigner { + root: root_from(&wallet), + network: Network::Testnet, + }; + // A hardened path — only derivable with the private key, which is the + // whole point of exposing extended-pubkey export on the signer. + let path = DerivationPath::from_str("m/9'/1'/15'/0'").expect("valid path"); + let from_signer = + signer.extended_public_key(&path).await.expect("signer extended_public_key"); + let from_wallet = wallet.derive_extended_public_key(&path).expect("wallet extended pubkey"); + assert_eq!( + from_signer, from_wallet, + "signer xpub at a hardened path must equal the wallet's own derivation" + ); + } + fn dest_outputs(amount: u64) -> Vec<(Address, u64)> { let dest = Address::from_str("yTb47qEBpNmgXvYYsHEN4nh8yJwa5iC4Cs").unwrap(); vec![(dest, amount)] @@ -443,6 +477,12 @@ mod tests { async fn public_key(&self, _: &DerivationPath) -> Result { unreachable!() } + async fn extended_public_key( + &self, + _: &DerivationPath, + ) -> Result { + unreachable!() + } } let (wallet, mut info) = test_wallet_and_info(); From d9fe0a3169cfd0c87a6cb3734d51a727cbfbde74 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 24 Jun 2026 16:53:23 +1000 Subject: [PATCH 006/149] chore(dash-spv): refresh masternode seed files (#805) Co-authored-by: QuantumExplorer <11468583+QuantumExplorer@users.noreply.github.com> --- dash-network-seeds/seeds/mainnet.txt | 209 ++++++++++++++------------- dash-network-seeds/seeds/testnet.txt | 19 +-- 2 files changed, 117 insertions(+), 111 deletions(-) diff --git a/dash-network-seeds/seeds/mainnet.txt b/dash-network-seeds/seeds/mainnet.txt index 4f5ee7c72..af4faac77 100644 --- a/dash-network-seeds/seeds/mainnet.txt +++ b/dash-network-seeds/seeds/mainnet.txt @@ -1,19 +1,16 @@ -# Auto-generated by masternode-seeds-fetcher on 2026-05-18T07:36:21Z for mainnet +# Auto-generated by masternode-seeds-fetcher on 2026-06-22T08:21:33Z for mainnet # Source: Dash P2P network (mnlistdiff) -# Primary peer: 168.119.87.131:9999 -# Tip block hash: 0000000000000013670d971ce450b3774deb714558cd52edcb2c8c486a86b048 -# Tip block height: 2473351 -# 2054 seeds (1697 regular + 357 evo) of 2943 total masternodes, valid-only -# Probe summary: core_reachable=2036/2054 platform_reachable=354/357 ssl_valid=296/357 +# Primary peer: 142.93.157.112:9999 +# Tip block hash: 0000000000000016a36e221c6615eed1fe3791deb6018837de7633c85ddf3b60 +# Tip block height: 2492563 +# 2063 seeds (1718 regular + 345 evo) of 2952 total masternodes, valid-only +# Probe summary: core_reachable=2036/2063 platform_reachable=340/345 ssl_valid=286/345 # Columns: # Values: core_reach/plat_reach=ok|timeout|refused|error|?, core_sync=sync|-N|+N|?, plat_live=ok|none|?|-, plat_ssl=valid|expired|self-signed|untrusted|no-handshake|?|- # Do not edit manually — refreshed weekly by .github/workflows/update-masternode-seeds.yml -regular 2.56.213.218:9999 - ok sync - - - -regular 2.56.213.221:9999 - ok sync - - - +regular 1.117.75.220:9999 - ok sync - - - evo 2.59.156.192:9999 443 ok sync ok ok valid regular 2.59.219.54:9999 - ok sync - - - -regular 2.233.120.35:9999 - ok sync - - - -regular 5.2.67.190:9999 - ok sync - - - regular 5.2.73.58:9999 - ok sync - - - regular 5.9.237.32:9999 - ok sync - - - regular 5.9.237.37:9999 - ok sync - - - @@ -23,11 +20,11 @@ regular 5.35.103.26:9999 - ok sync - - - regular 5.35.103.123:9999 - ok sync - - - regular 5.39.10.23:9999 - ok sync - - - regular 5.39.11.80:9999 - ok sync - - - -evo 5.39.27.224:9999 443 ok sync ok ok valid +evo 5.39.27.224:9999 443 ok sync refused none no-handshake regular 5.53.124.120:9999 - ok sync - - - evo 5.75.133.148:9999 443 ok sync ok ok valid regular 5.75.251.102:9999 - ok sync - - - -regular 5.135.14.86:9999 - timeout ? - - - +regular 5.135.14.86:9999 - ok sync - - - regular 5.161.110.79:9999 - ok sync - - - regular 5.175.162.96:9999 - ok sync - - - regular 5.175.162.97:9999 - ok sync - - - @@ -57,6 +54,7 @@ regular 5.175.162.159:9999 - ok sync - - - regular 5.175.162.160:9999 - ok sync - - - regular 5.175.162.161:9999 - ok sync - - - regular 5.175.162.162:9999 - ok sync - - - +regular 5.180.234.247:9999 - ok sync - - - evo 5.189.145.80:9999 443 ok sync ok ok valid evo 5.189.151.7:9999 443 ok sync ok ok valid evo 5.189.164.253:9999 443 ok sync ok ok valid @@ -195,9 +193,16 @@ regular 8.222.149.162:9999 - ok sync - - - regular 8.222.149.195:9999 - ok sync - - - regular 8.222.150.74:9999 - ok sync - - - regular 8.222.151.173:9999 - ok sync - - - +regular 13.140.137.75:9999 - ok sync - - - +regular 13.140.137.151:9999 - ok sync - - - +regular 13.140.137.152:9999 - ok sync - - - +regular 13.140.137.153:9999 - ok sync - - - +regular 13.140.137.154:9999 - ok sync - - - +regular 13.140.144.201:9999 - ok sync - - - +regular 13.140.144.202:9999 - ok sync - - - regular 15.188.53.12:9999 - ok sync - - - -evo 15.235.102.215:9999 443 ok sync ok ok expired -evo 15.235.102.216:9999 443 ok sync ok ok expired +evo 15.235.102.215:9999 443 ok sync ok ok valid +evo 15.235.102.216:9999 443 ok sync ok ok valid regular 23.88.22.66:9999 - ok sync - - - regular 23.88.22.67:9999 - ok sync - - - regular 23.88.22.68:9999 - ok sync - - - @@ -206,6 +211,7 @@ regular 23.95.186.248:9999 - ok sync - - - regular 23.163.0.174:9999 - ok sync - - - regular 31.10.97.36:9999 - ok sync - - - regular 31.56.38.0:9999 - ok sync - - - +regular 31.56.38.1:9999 - ok sync - - - regular 31.56.38.185:9999 - ok sync - - - regular 31.58.170.16:9999 - ok sync - - - regular 31.58.170.130:9999 - ok sync - - - @@ -214,11 +220,12 @@ regular 31.58.170.139:9999 - ok sync - - - regular 31.58.170.180:9999 - ok sync - - - regular 31.58.170.181:9999 - ok sync - - - regular 31.58.170.213:9999 - ok sync - - - +regular 31.58.170.222:9999 - ok sync - - - regular 31.58.170.230:9999 - ok sync - - - +regular 31.77.108.136:9999 - ok sync - - - regular 31.148.99.148:9999 - ok sync - - - evo 31.220.77.84:9999 443 ok sync ok ok valid evo 31.220.84.93:9999 443 ok sync ok ok valid -regular 31.220.87.198:9999 - ok sync - - - evo 31.220.91.43:9999 443 ok sync ok ok valid evo 31.220.91.60:9999 443 ok sync ok ok valid regular 34.209.37.222:9999 - ok sync - - - @@ -239,19 +246,17 @@ evo 37.60.235.24:9999 443 ok sync ok ok valid evo 37.60.235.39:9999 443 ok sync ok ok valid evo 37.60.235.89:9999 443 ok sync ok ok valid evo 37.60.235.205:9999 443 ok sync ok ok valid -evo 37.60.235.218:9999 443 ok sync ok ok valid evo 37.60.235.224:9999 443 ok sync ok ok valid -evo 37.60.236.41:9999 443 ok sync ok ok valid evo 37.60.236.230:9999 443 ok sync ok ok valid evo 37.60.243.59:9999 443 ok sync ok ok valid evo 37.60.244.35:9999 443 ok sync ok ok valid evo 37.60.244.87:9999 443 ok sync ok ok valid evo 37.60.244.253:9999 443 ok sync ok ok valid evo 37.60.246.228:9999 443 ok sync ok ok valid +evo 37.60.251.14:9999 443 ok sync ok ok valid evo 37.60.254.201:9999 443 ok sync ok ok valid evo 37.60.254.202:9999 443 ok sync ok ok valid evo 37.60.254.205:9999 443 ok sync ok ok valid -evo 37.60.254.211:9999 443 ok sync ok ok valid evo 37.60.254.213:9999 443 ok sync ok ok valid evo 37.60.255.160:9999 443 ok sync ok ok valid regular 37.97.227.21:9999 - ok sync - - - @@ -262,7 +267,6 @@ regular 37.139.11.82:9999 - ok sync - - - regular 37.139.29.66:9999 - ok sync - - - regular 38.54.12.111:9999 - ok sync - - - regular 38.102.125.108:9999 - ok sync - - - -evo 38.242.145.178:9999 443 ok sync ok ok valid evo 38.242.197.189:9999 443 ok sync ok ok valid evo 38.242.200.227:9999 443 ok sync ok ok valid evo 38.242.206.56:9999 443 ok sync ok ok valid @@ -275,7 +279,6 @@ evo 43.167.239.145:9999 443 ok sync ok ok valid evo 43.167.244.109:9999 443 ok sync ok ok valid regular 43.224.35.163:9999 - error ? - - - regular 43.229.77.46:9999 - ok sync - - - -evo 44.240.99.214:9999 443 ok sync ok ok valid regular 45.32.35.170:9999 - ok sync - - - evo 45.32.70.131:9999 443 ok sync ok ok valid regular 45.32.74.179:9999 - ok sync - - - @@ -284,6 +287,7 @@ regular 45.32.115.119:9999 - error ? - - - regular 45.32.156.167:9999 - ok sync - - - regular 45.32.199.30:9999 - ok sync - - - regular 45.32.207.17:9999 - ok sync - - - +regular 45.32.222.119:9999 - error ? - - - regular 45.32.241.67:9999 - ok sync - - - regular 45.58.52.179:9999 - ok sync - - - regular 45.58.52.232:9999 - ok sync - - - @@ -341,6 +345,7 @@ regular 46.4.162.116:9999 - ok sync - - - regular 46.4.162.117:9999 - ok sync - - - regular 46.4.162.118:9999 - ok sync - - - regular 46.4.162.119:9999 - ok sync - - - +regular 46.4.162.121:9999 - ok sync - - - regular 46.4.162.122:9999 - ok sync - - - regular 46.4.162.124:9999 - ok sync - - - regular 46.4.162.125:9999 - ok sync - - - @@ -359,11 +364,11 @@ regular 46.4.217.242:9999 - ok sync - - - regular 46.4.217.244:9999 - ok sync - - - regular 46.4.217.245:9999 - ok sync - - - regular 46.4.217.246:9999 - ok sync - - - +regular 46.4.217.249:9999 - ok sync - - - regular 46.4.217.250:9999 - ok sync - - - regular 46.4.217.251:9999 - ok sync - - - regular 46.4.217.252:9999 - ok sync - - - regular 46.4.217.254:9999 - ok sync - - - -regular 46.10.241.191:9999 - error ? - - - regular 46.30.189.187:9999 - ok sync - - - regular 46.30.189.213:9999 - ok sync - - - regular 46.30.189.251:9999 - ok sync - - - @@ -378,10 +383,10 @@ regular 46.101.108.68:9999 - ok sync - - - regular 46.101.110.82:9999 - ok sync - - - regular 46.101.118.96:9999 - ok sync - - - regular 46.101.129.63:9999 - ok sync - - - -regular 46.101.195.230:9999 - ok sync - - - regular 46.101.240.34:9999 - ok sync - - - regular 46.148.231.121:9999 - ok sync - - - regular 46.224.50.151:9999 - ok sync - - - +regular 46.225.120.51:9999 - ok sync - - - regular 46.250.254.10:9999 - ok sync - - - regular 49.12.1.250:9999 - ok sync - - - evo 49.12.102.105:9999 443 ok sync ok ok valid @@ -397,18 +402,18 @@ regular 49.13.209.157:9999 - ok sync - - - regular 49.13.212.202:9999 - ok sync - - - regular 49.13.212.210:9999 - ok sync - - - evo 49.13.237.193:9999 443 ok sync ok ok valid +evo 50.116.28.103:9999 443 ok sync ok ok expired regular 51.38.27.202:9999 - ok sync - - - -evo 51.68.3.29:9999 443 ok sync ok ok valid +evo 51.68.3.29:9999 443 ok sync refused none no-handshake evo 51.68.35.98:9999 443 ok sync ok ok valid evo 51.77.216.185:9999 443 ok sync ok ok expired -evo 51.79.109.182:9999 443 ok sync ok ok expired +evo 51.79.109.182:9999 443 ok sync ok ok valid regular 51.158.167.31:9999 - ok sync - - - evo 51.158.169.237:9999 443 ok sync ok ok valid regular 51.158.188.88:9999 - ok sync - - - evo 51.161.34.134:9999 443 ok sync ok ok expired evo 51.195.235.166:9999 443 ok sync ok ok valid evo 52.10.213.198:9999 443 ok sync ok ok valid -evo 52.33.9.172:9999 443 ok sync ok ok valid evo 52.36.102.91:9999 443 ok sync ok ok valid regular 54.37.38.226:9999 - ok sync - - - evo 54.69.95.118:9999 443 ok sync ok ok valid @@ -427,12 +432,9 @@ regular 62.171.149.86:9999 - ok sync - - - evo 62.171.150.24:9999 443 ok sync ok ok valid evo 62.171.168.4:9999 443 ok sync ok ok valid evo 62.171.168.44:9999 443 ok sync ok ok valid -evo 62.171.170.14:9999 443 ok sync ok ok valid -evo 62.171.170.222:9999 443 ok sync ok ok valid evo 62.171.171.253:9999 443 ok sync ok ok valid evo 62.171.174.123:9999 443 ok sync ok ok valid evo 62.171.176.79:9999 443 ok sync ok ok valid -regular 62.171.177.236:9999 - ok sync - - - evo 62.171.178.71:9999 443 ok sync ok ok valid evo 62.171.189.0:9999 443 ok sync ok ok valid regular 62.171.189.217:9999 - ok sync - - - @@ -442,10 +444,8 @@ regular 64.225.96.193:9999 - ok sync - - - regular 64.226.66.91:9999 - ok sync - - - regular 64.226.81.185:9999 - ok sync - - - regular 64.227.32.83:9999 - ok sync - - - -regular 64.227.122.247:9999 - ok sync - - - -regular 64.227.139.86:9999 - ok sync - - - +regular 64.227.35.91:9999 - ok sync - - - regular 64.227.165.75:9999 - ok sync - - - -regular 65.20.101.115:9999 - ok sync - - - regular 65.21.183.4:9999 - ok sync - - - evo 65.21.237.222:9999 443 ok sync ok ok expired evo 65.21.237.223:9999 443 ok sync ok ok expired @@ -466,8 +466,6 @@ evo 65.108.74.78:9999 443 ok sync refused none no-handshake evo 65.108.74.79:9999 443 ok sync ok ok expired evo 65.108.74.95:9999 443 ok sync ok ok expired evo 65.108.74.109:9999 443 ok sync ok ok expired -regular 65.108.145.8:9999 - ok sync - - - -regular 65.108.150.87:9999 - ok sync - - - regular 65.108.204.190:9999 - ok sync - - - regular 65.108.221.24:9999 - ok sync - - - evo 65.108.246.145:9999 443 ok sync ok ok valid @@ -501,7 +499,6 @@ regular 68.183.92.68:9999 - ok sync - - - regular 68.183.188.215:9999 - ok sync - - - regular 68.183.192.117:9999 - ok sync - - - regular 68.183.199.77:9999 - ok sync - - - -regular 68.183.201.176:9999 - ok sync - - - regular 68.183.207.246:9999 - ok sync - - - regular 68.183.208.90:9999 - ok sync - - - regular 68.183.235.156:9999 - ok sync - - - @@ -513,49 +510,55 @@ evo 72.60.38.160:9999 443 ok sync ok ok valid evo 72.62.58.108:9999 443 ok sync ok ok valid regular 74.50.90.113:9999 - ok sync - - - evo 75.119.128.71:9999 443 ok sync ok ok valid -regular 75.119.134.20:9999 - ok sync - - - evo 75.119.137.66:9999 443 ok sync ok ok valid evo 75.119.138.9:9999 443 ok sync ok ok valid evo 75.119.152.219:9999 443 ok sync ok ok valid evo 75.119.153.10:9999 443 ok sync ok ok valid evo 75.119.156.254:9999 443 ok sync ok ok valid +evo 77.20.160.6:9999 443 ok sync ok ok valid regular 77.42.16.46:9999 - ok sync - - - regular 77.42.65.17:9999 - ok sync - - - evo 77.42.74.68:9999 443 ok sync ok ok valid regular 77.42.91.72:9999 - ok sync - - - +regular 77.105.61.79:9999 - ok sync - - - +regular 77.105.61.188:9999 - ok sync - - - +regular 77.105.61.212:9999 - ok sync - - - evo 77.221.148.204:9999 443 ok sync ok ok valid regular 77.223.99.4:9999 - ok sync - - - regular 77.237.235.0:9999 - ok sync - - - regular 77.237.236.66:9999 - ok sync - - - regular 77.237.236.214:9999 - ok sync - - - -regular 77.237.244.217:9999 - ok sync - - - -regular 77.237.245.19:9999 - ok sync - - - +evo 77.237.245.19:9999 443 ok sync ok ok valid regular 78.47.119.20:9999 - ok sync - - - -regular 78.83.19.0:9999 - ok sync - - - +regular 78.83.19.0:9999 - error ? - - - evo 78.141.225.100:9999 443 ok sync ok ok valid regular 78.141.240.214:9999 - ok sync - - - regular 79.143.181.108:9999 - ok sync - - - -regular 80.208.230.144:9999 - ok sync - - - -regular 80.209.233.182:9999 - ok sync - - - -regular 80.209.239.129:9999 - ok sync - - - +regular 80.208.230.144:9999 - error ? - - - +regular 80.209.233.182:9999 - error ? - - - +regular 80.209.239.129:9999 - error ? - - - evo 80.240.19.200:9999 443 ok sync ok ok valid regular 80.240.132.231:9999 - ok sync - - - regular 80.240.135.83:9999 - ok sync - - - regular 80.249.147.8:9999 - ok sync - - - +evo 81.14.228.87:9999 443 ok sync ok ok valid regular 81.227.250.51:9999 - ok sync - - - regular 82.180.146.192:9999 - ok sync - - - regular 82.196.1.10:9999 - ok sync - - - regular 82.202.230.83:9999 - ok sync - - - evo 82.208.20.153:9999 443 ok sync ok ok valid +regular 82.223.81.127:9999 - ok sync - - - +regular 83.149.100.183:9999 - ok sync - - - +regular 83.149.100.184:9999 - ok sync - - - +regular 83.149.100.185:9999 - error ? - - - +regular 83.149.100.186:9999 - ok sync - - - regular 83.239.99.40:9999 - ok sync - - - regular 84.9.50.17:9999 - ok sync - - - regular 84.98.12.196:9999 - ok sync - - - regular 84.242.179.204:9999 - ok sync - - - -regular 84.247.132.230:9999 - error ? - - - regular 84.247.137.20:9999 - ok sync - - - evo 84.247.180.190:9999 443 ok sync ok ok valid evo 84.247.180.198:9999 443 ok sync ok ok valid -evo 84.247.180.200:9999 443 ok sync ok ok valid evo 84.247.180.201:9999 443 ok sync ok ok valid evo 84.247.180.205:9999 443 ok sync ok ok valid evo 84.247.187.76:9999 443 ok sync ok ok valid @@ -637,6 +640,8 @@ regular 85.209.241.146:9999 - ok sync - - - regular 85.209.241.150:9999 - ok sync - - - regular 85.209.241.156:9999 - ok sync - - - regular 85.209.241.164:9999 - ok sync - - - +regular 85.209.241.165:9999 - ok sync - - - +regular 85.209.241.166:9999 - ok sync - - - regular 85.209.241.167:9999 - ok sync - - - regular 85.209.241.172:9999 - ok sync - - - regular 85.209.241.177:9999 - ok sync - - - @@ -658,6 +663,7 @@ regular 85.209.241.218:9999 - ok sync - - - regular 85.209.241.219:9999 - ok sync - - - regular 85.209.241.220:9999 - ok sync - - - regular 85.209.241.221:9999 - ok sync - - - +regular 85.209.241.222:9999 - ok sync - - - regular 85.209.241.223:9999 - ok sync - - - regular 85.209.241.224:9999 - ok sync - - - regular 85.209.241.225:9999 - ok sync - - - @@ -681,6 +687,8 @@ regular 85.209.241.247:9999 - ok sync - - - regular 85.209.241.248:9999 - ok sync - - - regular 85.209.241.249:9999 - ok sync - - - regular 85.209.242.250:9999 - ok sync - - - +regular 86.107.168.40:9999 - ok sync - - - +evo 86.107.168.78:9999 443 ok sync ok ok valid evo 87.228.24.64:9999 443 ok sync ok ok valid regular 87.229.22.23:9999 - ok sync - - - regular 87.229.22.31:9999 - ok sync - - - @@ -712,15 +720,14 @@ regular 88.99.38.175:9999 - ok sync - - - evo 89.35.131.23:9999 443 ok sync ok ok valid evo 89.35.131.39:9999 443 ok sync ok ok valid regular 89.35.131.46:9999 - ok sync - - - -evo 89.35.131.61:9999 443 ok sync ok ok expired evo 89.35.131.149:9999 443 ok sync ok ok valid evo 89.35.131.158:9999 443 ok sync ok ok valid -evo 89.35.131.218:9999 443 error ? ok ok valid +evo 89.35.131.218:9999 443 ok sync ok ok valid evo 89.35.131.219:9999 443 ok sync ok ok valid -regular 89.40.15.222:9999 - ok sync - - - +regular 89.40.15.222:9999 - error ? - - - regular 89.46.42.216:9999 - ok sync - - - regular 89.46.42.218:9999 - ok sync - - - -regular 89.47.167.131:9999 - ok sync - - - +regular 89.47.167.131:9999 - error ? - - - regular 89.117.3.242:9999 - ok sync - - - evo 89.117.57.27:9999 443 ok sync ok ok expired evo 89.125.50.14:9999 443 ok sync ok ok expired @@ -739,8 +746,12 @@ evo 89.125.209.234:9999 443 ok sync ok ok valid regular 89.179.73.96:9999 - ok sync - - - regular 91.98.160.22:9999 - ok sync - - - regular 91.99.169.186:9999 - ok sync - - - +regular 91.184.249.209:9999 - ok sync - - - evo 91.198.108.35:9999 443 ok sync ok ok valid evo 91.198.108.36:9999 443 ok sync ok ok valid +evo 91.198.108.37:9999 443 ok sync ok ok valid +evo 91.198.108.38:9999 443 ok sync ok ok valid +regular 91.198.108.39:9999 - ok sync - - - regular 91.198.108.40:9999 - ok sync - - - evo 91.199.149.177:9999 443 ok sync ok ok expired evo 93.115.172.36:9999 443 ok sync ok ok valid @@ -762,7 +773,6 @@ regular 95.85.48.49:9999 - ok sync - - - regular 95.85.54.64:9999 - ok sync - - - evo 95.111.232.18:9999 443 ok sync ok ok valid evo 95.111.233.139:9999 443 ok sync ok ok valid -evo 95.111.239.54:9999 443 ok sync ok ok valid evo 95.111.241.20:9999 443 ok sync ok ok valid evo 95.111.242.220:9999 443 ok sync ok ok valid evo 95.179.139.125:9999 443 ok sync ok ok expired @@ -771,7 +781,6 @@ evo 95.179.159.65:9999 443 ok sync ok ok valid regular 95.179.213.67:9999 - ok sync - - - regular 95.179.240.114:9999 - ok sync - - - evo 95.179.241.182:9999 443 ok sync ok ok valid -regular 95.179.243.76:9999 - ok sync - - - regular 95.216.79.224:9999 - ok sync - - - regular 95.216.79.228:9999 - ok sync - - - regular 95.216.84.33:9999 - ok sync - - - @@ -784,7 +793,6 @@ regular 95.216.84.43:9999 - ok sync - - - regular 95.216.84.45:9999 - ok sync - - - regular 95.216.109.130:9999 - ok sync - - - regular 95.216.109.131:9999 - ok sync - - - -regular 95.216.109.133:9999 - ok sync - - - regular 95.216.109.134:9999 - ok sync - - - regular 95.216.109.135:9999 - ok sync - - - regular 95.216.109.136:9999 - ok sync - - - @@ -821,9 +829,6 @@ regular 95.217.71.207:9999 - ok sync - - - regular 95.217.71.209:9999 - ok sync - - - regular 95.217.71.210:9999 - ok sync - - - regular 95.217.71.211:9999 - ok sync - - - -evo 95.217.83.2:9999 443 ok sync ok ok valid -regular 95.217.83.5:9999 - ok sync - - - -evo 95.217.83.14:9999 443 ok sync ok ok valid regular 95.217.125.97:9999 - ok sync - - - regular 95.217.125.98:9999 - ok sync - - - regular 95.217.125.101:9999 - ok sync - - - @@ -890,26 +895,20 @@ regular 108.160.131.192:9999 - ok sync - - - regular 108.160.134.116:9999 - ok sync - - - regular 108.160.140.103:9999 - ok sync - - - evo 109.123.244.131:9999 443 ok sync ok ok valid -regular 109.173.240.233:9999 - ok sync - - - -regular 109.199.96.61:9999 - ok sync - - - evo 109.199.97.144:9999 443 ok sync ok ok valid evo 109.199.105.14:9999 443 ok sync ok ok valid regular 109.199.105.203:9999 - ok sync - - - evo 109.199.107.155:9999 443 ok sync ok ok valid -evo 109.199.108.223:9999 443 ok sync ok ok valid evo 109.199.108.234:9999 443 ok sync ok ok valid evo 109.199.109.39:9999 443 ok sync ok ok valid -evo 109.199.119.192:9999 443 ok sync ok ok valid evo 109.199.120.79:9999 443 ok sync ok ok valid evo 109.199.124.30:9999 443 ok sync ok ok valid evo 109.199.124.135:9999 443 ok sync ok ok valid -regular 109.235.67.246:9999 - ok sync - - - +regular 109.235.67.246:9999 - error ? - - - evo 114.132.172.215:9999 443 ok sync ok ok valid -regular 115.120.238.240:9999 - error ? - - - -regular 115.190.248.57:9999 - error ? - - - +regular 115.190.248.57:9999 - ok sync - - - regular 116.203.244.222:9999 - ok sync - - - -regular 123.193.1.112:9999 - ok sync - - - -regular 128.140.7.186:9999 - ok sync - - - +regular 123.193.64.166:9999 - ok sync - - - regular 128.140.58.30:9999 - ok sync - - - regular 128.199.17.79:9999 - ok sync - - - regular 128.199.22.78:9999 - ok sync - - - @@ -1159,11 +1158,13 @@ evo 135.181.110.216:9999 443 ok sync ok ok valid regular 135.181.174.222:9999 - ok sync - - - regular 135.181.177.53:9999 - ok sync - - - evo 135.181.229.49:9999 443 ok sync ok ok valid +regular 136.243.29.192:9999 - ok sync - - - regular 136.243.29.193:9999 - ok sync - - - regular 136.243.29.194:9999 - ok sync - - - regular 136.243.29.195:9999 - ok sync - - - regular 136.243.29.196:9999 - ok sync - - - regular 136.243.29.197:9999 - ok sync - - - +regular 136.243.29.198:9999 - ok sync - - - regular 136.243.29.199:9999 - ok sync - - - regular 136.243.29.201:9999 - ok sync - - - regular 136.243.29.205:9999 - ok sync - - - @@ -1184,7 +1185,6 @@ regular 136.243.115.139:9999 - ok sync - - - regular 136.243.115.140:9999 - ok sync - - - regular 136.243.115.141:9999 - ok sync - - - regular 136.243.115.142:9999 - ok sync - - - -regular 136.243.138.106:9999 - ok sync - - - regular 136.243.142.32:9999 - ok sync - - - regular 136.243.142.33:9999 - ok sync - - - regular 136.243.142.36:9999 - ok sync - - - @@ -1195,14 +1195,14 @@ regular 137.184.5.78:9999 - ok sync - - - regular 137.184.175.173:9999 - ok sync - - - regular 137.220.38.255:9999 - error ? - - - regular 138.68.68.160:9999 - ok sync - - - +regular 138.68.146.83:9999 - ok sync - - - regular 138.68.149.239:9999 - ok sync - - - +regular 138.68.155.23:9999 - ok sync - - - regular 138.68.157.15:9999 - ok sync - - - regular 138.197.128.77:9999 - ok sync - - - -regular 138.197.130.116:9999 - ok sync - - - regular 138.197.131.126:9999 - ok sync - - - regular 138.197.147.28:9999 - ok sync - - - regular 138.197.161.208:9999 - ok sync - - - -regular 139.28.97.2:9999 - ok sync - - - regular 139.59.0.167:9999 - ok sync - - - regular 139.59.3.197:9999 - ok sync - - - regular 139.59.4.172:9999 - ok sync - - - @@ -1218,9 +1218,9 @@ regular 139.59.138.149:9999 - ok sync - - - regular 139.59.144.169:9999 - ok sync - - - regular 139.59.156.27:9999 - ok sync - - - regular 139.59.178.169:9999 - ok sync - - - -regular 139.59.179.170:9999 - ok sync - - - regular 139.59.231.57:9999 - ok sync - - - regular 139.59.243.139:9999 - ok sync - - - +regular 139.59.243.194:9999 - ok sync - - - evo 139.84.137.143:9999 443 ok sync ok ok valid evo 139.84.170.10:9999 443 ok sync ok ok valid evo 139.84.231.221:9999 443 ok sync ok ok valid @@ -1252,13 +1252,10 @@ evo 143.198.145.184:9999 443 ok sync ok ok valid regular 143.244.131.179:9999 - ok sync - - - regular 144.24.238.178:9999 - ok sync - - - regular 144.31.188.42:9999 - ok sync - - - -evo 144.91.87.82:9999 443 ok sync ok ok valid evo 144.91.99.105:9999 443 ok sync ok ok valid evo 144.91.105.209:9999 443 ok sync ok ok valid -evo 144.91.106.188:9999 443 ok sync ok ok valid evo 144.91.121.81:9999 443 ok sync ok ok valid evo 144.126.141.62:9999 443 ok sync ok ok valid -regular 144.126.143.197:9999 - ok sync - - - regular 144.126.203.253:9999 - ok sync - - - regular 144.202.19.42:9999 - ok sync - - - regular 144.202.37.1:9999 - error ? - - - @@ -1275,15 +1272,15 @@ regular 146.185.180.40:9999 - ok sync - - - regular 146.185.182.189:9999 - ok sync - - - regular 146.190.33.135:9999 - ok sync - - - regular 147.45.41.210:9999 - ok sync - - - -regular 147.93.131.52:9999 - ok sync - - - -regular 147.93.132.12:9999 - ok sync - - - regular 147.182.146.51:9999 - ok sync - - - +regular 147.182.146.128:9999 - ok sync - - - regular 148.113.202.169:9999 - ok sync - - - regular 148.113.209.97:9999 - ok sync - - - regular 148.251.73.224:9999 - ok sync - - - regular 148.251.73.232:9999 - ok sync - - - regular 148.251.73.234:9999 - ok sync - - - regular 148.251.73.235:9999 - ok sync - - - +regular 148.251.73.237:9999 - ok sync - - - regular 148.251.73.238:9999 - ok sync - - - regular 149.28.85.41:9999 - ok sync - - - regular 149.28.112.102:9999 - error ? - - - @@ -1305,6 +1302,7 @@ regular 151.244.85.9:9999 - ok sync - - - regular 151.244.85.15:9999 - ok sync - - - regular 151.244.85.31:9999 - ok sync - - - regular 151.244.85.36:9999 - ok sync - - - +regular 151.244.85.43:9999 - ok sync - - - regular 151.244.85.45:9999 - ok sync - - - regular 151.244.85.47:9999 - ok sync - - - regular 151.244.85.48:9999 - ok sync - - - @@ -1317,11 +1315,14 @@ regular 151.244.85.70:9999 - ok sync - - - regular 151.244.85.76:9999 - ok sync - - - regular 151.244.85.80:9999 - ok sync - - - regular 151.244.85.81:9999 - ok sync - - - +regular 151.244.85.82:9999 - ok sync - - - regular 151.244.85.88:9999 - ok sync - - - regular 151.244.85.96:9999 - ok sync - - - regular 151.244.85.98:9999 - ok sync - - - regular 151.244.85.109:9999 - ok sync - - - +regular 151.244.85.110:9999 - ok sync - - - regular 151.244.85.111:9999 - ok sync - - - +regular 151.244.85.121:9999 - ok sync - - - regular 151.244.85.123:9999 - ok sync - - - regular 151.244.85.131:9999 - ok sync - - - regular 151.244.85.134:9999 - ok sync - - - @@ -1329,10 +1330,13 @@ regular 151.244.85.135:9999 - ok sync - - - regular 151.244.85.141:9999 - ok sync - - - regular 151.244.85.143:9999 - ok sync - - - regular 151.244.85.147:9999 - ok sync - - - +regular 151.244.85.155:9999 - ok sync - - - +regular 151.244.85.162:9999 - ok sync - - - regular 151.244.85.166:9999 - ok sync - - - regular 151.244.85.178:9999 - ok sync - - - regular 151.244.85.179:9999 - ok sync - - - regular 151.244.85.181:9999 - ok sync - - - +regular 151.244.85.186:9999 - ok sync - - - regular 151.244.85.187:9999 - ok sync - - - regular 151.244.85.190:9999 - ok sync - - - regular 151.244.85.195:9999 - ok sync - - - @@ -1349,6 +1353,7 @@ regular 151.244.85.220:9999 - ok sync - - - regular 151.244.85.228:9999 - ok sync - - - regular 151.244.85.229:9999 - ok sync - - - regular 151.244.85.236:9999 - ok sync - - - +regular 151.244.85.243:9999 - ok sync - - - regular 151.244.85.244:9999 - ok sync - - - regular 151.244.85.245:9999 - ok sync - - - regular 151.244.85.251:9999 - ok sync - - - @@ -1358,6 +1363,7 @@ regular 152.67.95.184:9999 - ok sync - - - regular 154.16.63.112:9999 - ok sync - - - evo 155.133.26.122:9999 443 ok sync ok ok valid regular 155.138.154.140:9999 - error ? - - - +evo 155.138.159.73:9999 443 ok sync ok ok valid evo 156.67.29.45:9999 443 ok sync ok ok valid regular 157.90.150.75:9999 - ok sync - - - evo 157.90.238.161:9999 443 ok sync ok ok valid @@ -1366,6 +1372,7 @@ evo 157.173.122.21:9999 443 ok sync ok ok valid evo 157.173.122.25:9999 443 ok sync ok ok valid evo 157.173.122.26:9999 443 ok sync ok ok valid regular 157.180.91.78:9999 - ok sync - - - +regular 157.230.22.29:9999 - ok sync - - - regular 157.230.24.176:9999 - ok sync - - - regular 157.230.113.158:9999 - ok sync - - - regular 157.230.119.39:9999 - ok sync - - - @@ -1399,6 +1406,7 @@ regular 159.203.33.3:9999 - ok sync - - - regular 159.203.33.254:9999 - ok sync - - - regular 159.203.39.201:9999 - ok sync - - - regular 159.203.61.192:9999 - ok sync - - - +regular 159.223.12.172:9999 - ok sync - - - regular 159.223.60.121:9999 - ok sync - - - regular 159.223.218.14:9999 - ok sync - - - regular 159.223.218.50:9999 - ok sync - - - @@ -1408,15 +1416,12 @@ regular 161.35.69.134:9999 - ok sync - - - regular 161.35.94.177:9999 - ok sync - - - regular 161.35.150.124:9999 - ok sync - - - regular 161.35.168.34:9999 - ok sync - - - -regular 161.35.207.170:9999 - ok sync - - - regular 161.35.213.210:9999 - ok sync - - - regular 161.35.247.75:9999 - ok sync - - - evo 161.97.66.31:9999 443 ok sync ok ok valid -regular 161.97.69.219:9999 - ok sync - - - evo 161.97.70.66:9999 443 ok sync ok ok valid evo 161.97.74.173:9999 443 ok sync ok ok valid evo 161.97.75.36:9999 443 ok sync ok ok valid -evo 161.97.83.102:9999 443 ok sync ok ok valid evo 161.97.85.159:9999 443 ok sync ok ok valid evo 161.97.85.182:9999 443 ok sync ok ok valid evo 161.97.88.199:9999 443 ok sync ok ok valid @@ -1430,7 +1435,6 @@ evo 161.97.106.14:9999 443 ok sync ok ok valid evo 161.97.106.164:9999 443 ok sync ok ok valid evo 161.97.117.125:9999 443 ok sync ok ok valid evo 161.97.120.156:9999 443 ok sync ok ok valid -regular 161.97.124.111:9999 - ok sync - - - evo 161.97.142.210:9999 443 ok sync ok ok valid evo 161.97.153.122:9999 443 ok sync ok ok valid evo 161.97.159.172:9999 443 ok sync ok ok valid @@ -1440,7 +1444,6 @@ evo 161.97.166.49:9999 443 ok sync ok ok valid evo 161.97.173.67:9999 443 ok sync ok ok valid evo 161.97.175.233:9999 443 ok sync ok ok valid evo 161.97.176.38:9999 443 ok sync ok ok valid -evo 161.97.179.214:9999 443 ok sync ok ok valid evo 161.97.180.105:9999 443 ok sync ok ok valid evo 161.97.180.182:9999 443 ok sync ok ok valid regular 162.212.35.99:9999 - ok sync - - - @@ -1467,12 +1470,11 @@ regular 164.92.160.130:9999 - ok sync - - - regular 164.92.164.29:9999 - ok sync - - - regular 164.92.170.229:9999 - ok sync - - - regular 164.92.175.210:9999 - ok sync - - - -regular 164.92.204.214:9999 - ok sync - - - -regular 164.92.249.64:9999 - ok sync - - - +regular 165.22.20.20:9999 - ok sync - - - regular 165.22.30.195:9999 - ok sync - - - regular 165.22.62.28:9999 - ok sync - - - +regular 165.22.62.245:9999 - ok sync - - - regular 165.22.65.68:9999 - ok sync - - - -regular 165.22.107.61:9999 - ok sync - - - regular 165.22.194.80:9999 - ok sync - - - regular 165.22.208.146:9999 - ok sync - - - regular 165.22.221.97:9999 - ok sync - - - @@ -1483,10 +1485,12 @@ regular 165.227.35.129:9999 - ok sync - - - regular 165.227.43.54:9999 - ok sync - - - regular 165.227.46.145:9999 - ok sync - - - regular 165.227.166.196:9999 - ok sync - - - +regular 165.227.238.23:9999 - ok sync - - - regular 165.232.33.54:9999 - ok sync - - - regular 165.232.95.72:9999 - ok sync - - - regular 165.232.114.131:9999 - ok sync - - - regular 165.232.187.22:9999 - ok sync - - - +regular 167.71.40.198:9999 - ok sync - - - regular 167.71.75.173:9999 - ok sync - - - regular 167.71.76.44:9999 - ok sync - - - regular 167.71.133.157:9999 - ok sync - - - @@ -1498,6 +1502,7 @@ evo 167.86.77.58:9999 443 ok sync ok ok valid evo 167.86.93.21:9999 443 ok sync ok ok valid evo 167.86.94.138:9999 443 ok sync ok ok valid evo 167.88.169.16:9999 443 ok sync ok ok valid +regular 167.99.65.122:9999 - ok sync - - - regular 167.99.73.247:9999 - ok sync - - - regular 167.99.74.32:9999 - ok sync - - - regular 167.99.177.115:9999 - ok sync - - - @@ -1506,15 +1511,15 @@ regular 167.99.182.250:9999 - ok sync - - - regular 167.99.185.82:9999 - ok sync - - - regular 167.99.199.59:9999 - ok sync - - - regular 167.99.205.145:9999 - ok sync - - - +evo 167.104.216.28:9999 443 ok sync ok ok valid regular 167.114.185.96:9999 - ok sync - - - -regular 167.172.45.235:9999 - ok sync - - - regular 167.172.65.155:9999 - ok sync - - - regular 167.172.67.55:9999 - ok sync - - - regular 167.172.70.187:9999 - ok sync - - - regular 167.172.87.128:9999 - ok sync - - - regular 167.172.94.148:9999 - ok sync - - - regular 167.172.183.190:9999 - ok sync - - - -evo 167.235.102.194:9999 443 ok sync ok ok valid +evo 167.233.98.163:9999 443 ok sync ok ok valid regular 167.235.132.126:9999 - ok sync - - - regular 168.119.57.12:9999 - ok sync - - - regular 168.119.80.0:9999 - ok sync - - - @@ -1525,10 +1530,13 @@ regular 168.119.80.5:9999 - ok sync - - - regular 168.119.80.6:9999 - ok sync - - - regular 168.119.80.7:9999 - ok sync - - - regular 168.119.80.10:9999 - ok sync - - - +regular 168.119.80.11:9999 - ok sync - - - regular 168.119.80.12:9999 - ok sync - - - regular 168.119.80.13:9999 - ok sync - - - regular 168.119.80.15:9999 - ok sync - - - +regular 168.119.80.18:9999 - ok sync - - - regular 168.119.83.0:9999 - ok sync - - - +regular 168.119.83.1:9999 - ok sync - - - regular 168.119.83.2:9999 - ok sync - - - regular 168.119.83.3:9999 - ok sync - - - regular 168.119.83.6:9999 - ok sync - - - @@ -1586,6 +1594,7 @@ regular 173.249.33.131:9999 - ok sync - - - regular 173.249.35.151:9999 - ok sync - - - regular 173.249.40.168:9999 - ok sync - - - regular 174.138.11.164:9999 - ok sync - - - +regular 174.138.14.99:9999 - ok sync - - - regular 174.138.24.25:9999 - ok sync - - - regular 176.9.145.240:9999 - ok sync - - - regular 176.9.210.0:9999 - ok sync - - - @@ -1601,7 +1610,7 @@ regular 176.9.210.21:9999 - ok sync - - - regular 176.9.210.22:9999 - ok sync - - - regular 176.9.210.23:9999 - ok sync - - - regular 176.102.65.145:9999 - ok sync - - - -evo 177.93.141.60:9999 443 ok sync ok ok expired +evo 177.93.141.60:9999 443 ok sync ok ok valid regular 177.93.141.86:9999 - ok sync - - - evo 178.18.244.255:9999 443 ok sync ok ok valid evo 178.18.254.136:9999 443 ok sync ok ok valid @@ -1644,7 +1653,6 @@ regular 178.62.192.42:9999 - ok sync - - - regular 178.62.196.119:9999 - ok sync - - - regular 178.62.200.153:9999 - ok sync - - - regular 178.62.207.128:9999 - ok sync - - - -regular 178.62.209.128:9999 - ok sync - - - regular 178.62.215.116:9999 - ok sync - - - regular 178.62.220.199:9999 - ok sync - - - regular 178.62.230.209:9999 - ok sync - - - @@ -1662,6 +1670,7 @@ regular 178.63.121.136:9999 - ok sync - - - regular 178.63.121.137:9999 - ok sync - - - regular 178.63.121.140:9999 - ok sync - - - regular 178.63.121.141:9999 - ok sync - - - +regular 178.63.121.144:9999 - ok sync - - - regular 178.63.121.147:9999 - ok sync - - - regular 178.63.235.192:9999 - ok sync - - - regular 178.63.235.193:9999 - ok sync - - - @@ -1679,6 +1688,7 @@ regular 178.63.236.103:9999 - ok sync - - - regular 178.63.236.104:9999 - ok sync - - - regular 178.63.236.105:9999 - ok sync - - - regular 178.63.236.107:9999 - ok sync - - - +regular 178.63.236.108:9999 - ok sync - - - regular 178.63.236.109:9999 - ok sync - - - regular 178.63.236.110:9999 - ok sync - - - regular 178.63.236.112:9999 - ok sync - - - @@ -1735,7 +1745,6 @@ regular 185.164.163.218:9999 - ok sync - - - regular 185.164.163.220:9999 - ok sync - - - regular 185.165.171.117:9999 - ok sync - - - evo 185.166.217.154:9999 443 ok sync ok ok valid -regular 185.175.56.212:9999 - ok sync - - - regular 185.185.40.14:9999 - ok sync - - - regular 185.185.40.104:9999 - ok sync - - - regular 185.185.40.115:9999 - ok sync - - - @@ -1785,6 +1794,7 @@ regular 188.40.163.8:9999 - ok sync - - - regular 188.40.163.9:9999 - ok sync - - - regular 188.40.163.12:9999 - ok sync - - - regular 188.40.163.15:9999 - ok sync - - - +regular 188.40.163.16:9999 - ok sync - - - regular 188.40.163.18:9999 - ok sync - - - regular 188.40.163.21:9999 - ok sync - - - regular 188.40.163.24:9999 - ok sync - - - @@ -1794,14 +1804,15 @@ regular 188.40.163.27:9999 - ok sync - - - regular 188.40.163.28:9999 - ok sync - - - regular 188.40.163.30:9999 - ok sync - - - regular 188.40.163.31:9999 - ok sync - - - -regular 188.40.169.39:9999 - ok sync - - - regular 188.40.169.42:9999 - ok sync - - - regular 188.40.169.44:9999 - ok sync - - - regular 188.40.175.65:9999 - ok sync - - - regular 188.40.175.67:9999 - ok sync - - - regular 188.40.175.68:9999 - ok sync - - - regular 188.40.175.69:9999 - ok sync - - - +regular 188.40.175.70:9999 - ok sync - - - regular 188.40.175.71:9999 - ok sync - - - +regular 188.40.175.73:9999 - ok sync - - - regular 188.40.178.64:9999 - ok sync - - - regular 188.40.178.66:9999 - ok sync - - - regular 188.40.178.67:9999 - ok sync - - - @@ -1841,6 +1852,7 @@ regular 188.40.182.222:9999 - ok sync - - - regular 188.40.184.65:9999 - ok sync - - - regular 188.40.184.67:9999 - ok sync - - - regular 188.40.184.68:9999 - ok sync - - - +regular 188.40.184.69:9999 - ok sync - - - regular 188.40.184.72:9999 - ok sync - - - regular 188.40.185.128:9999 - ok sync - - - regular 188.40.185.129:9999 - ok sync - - - @@ -1928,7 +1940,6 @@ regular 188.166.5.202:9999 - ok sync - - - regular 188.166.8.181:9999 - ok sync - - - regular 188.166.21.185:9999 - ok sync - - - regular 188.166.30.248:9999 - ok sync - - - -regular 188.166.33.180:9999 - ok sync - - - regular 188.166.54.247:9999 - ok sync - - - regular 188.166.56.220:9999 - ok sync - - - regular 188.166.60.137:9999 - ok sync - - - @@ -1968,10 +1979,9 @@ regular 193.160.119.129:9999 - ok sync - - - regular 194.5.157.214:9999 - ok sync - - - regular 194.58.47.132:9999 - ok sync - - - regular 194.58.47.228:9999 - ok sync - - - -regular 194.135.80.33:9999 - ok sync - - - -regular 194.135.84.100:9999 - ok sync - - - -regular 194.135.94.228:9999 - ok sync - - - -evo 194.163.156.190:9999 443 ok sync ok ok valid +regular 194.135.80.33:9999 - error ? - - - +regular 194.135.84.100:9999 - error ? - - - +regular 194.135.94.228:9999 - error ? - - - evo 194.163.166.76:9999 443 ok sync ok ok valid evo 194.163.172.206:9999 443 ok sync ok ok valid regular 194.163.183.132:9999 - ok sync - - - @@ -1985,6 +1995,10 @@ regular 198.199.119.50:9999 - ok sync - - - regular 198.199.124.50:9999 - ok sync - - - regular 198.211.119.126:9999 - ok sync - - - regular 198.251.79.146:9999 - ok sync - - - +regular 199.85.8.8:9999 - ok sync - - - +regular 199.85.8.16:9999 - ok sync - - - +regular 199.85.8.190:9999 - ok sync - - - +regular 199.85.8.191:9999 - ok sync - - - regular 202.5.16.33:9999 - ok sync - - - evo 202.71.14.79:9999 443 ok sync ok ok valid regular 202.71.14.128:9999 - ok sync - - - @@ -1993,11 +2007,11 @@ regular 202.182.115.161:9999 - ok sync - - - regular 205.206.173.159:9999 - ok sync - - - regular 205.206.254.21:9999 - ok sync - - - regular 205.206.254.210:9999 - ok sync - - - +regular 206.189.20.59:9999 - ok sync - - - regular 206.189.21.120:9999 - ok sync - - - regular 206.189.28.109:9999 - ok sync - - - regular 206.189.112.246:9999 - ok sync - - - regular 206.189.134.126:9999 - ok sync - - - -regular 206.189.136.218:9999 - ok sync - - - regular 206.189.143.107:9999 - ok sync - - - evo 206.245.167.74:9999 443 ok sync ok ok valid evo 206.245.167.76:9999 443 ok sync ok ok valid @@ -2008,16 +2022,12 @@ regular 207.154.207.223:9999 - ok sync - - - regular 207.154.214.156:9999 - ok sync - - - regular 207.154.226.228:9999 - ok sync - - - regular 207.154.229.81:9999 - ok sync - - - -regular 207.154.235.44:9999 - ok sync - - - evo 207.180.213.141:9999 443 ok sync ok ok valid -evo 207.180.218.245:9999 443 ok sync ok ok valid -regular 207.180.224.34:9999 - ok sync - - - evo 207.180.224.96:9999 443 ok sync ok ok valid regular 207.180.227.146:9999 - ok sync - - - evo 207.180.233.6:9999 443 ok sync ok ok valid evo 207.180.233.246:9999 443 ok sync ok ok valid regular 207.180.237.66:9999 - ok sync - - - -evo 207.180.241.242:9999 443 ok sync ok ok valid evo 207.244.247.40:9999 443 ok sync ok ok valid regular 208.87.98.19:9999 - ok sync - - - regular 208.87.98.21:9999 - ok sync - - - @@ -2032,10 +2042,10 @@ regular 209.38.226.76:9999 - ok sync - - - regular 209.97.142.65:9999 - ok sync - - - regular 209.141.36.206:9999 - ok sync - - - regular 209.250.240.73:9999 - ok sync - - - -regular 212.24.104.225:9999 - ok sync - - - -regular 212.24.109.253:9999 - ok sync - - - +regular 212.24.104.225:9999 - error ? - - - +regular 212.24.109.253:9999 - error ? - - - +evo 212.132.105.190:9999 443 ok sync ok ok valid regular 213.136.91.123:9999 - ok sync - - - -evo 213.159.77.221:9999 443 ok sync ok ok valid evo 213.199.44.112:9999 443 ok sync ok ok valid evo 213.199.53.161:9999 443 ok sync ok ok valid evo 213.199.54.34:9999 443 ok sync ok ok valid @@ -2053,12 +2063,11 @@ regular 216.189.157.22:9999 - ok sync - - - regular 216.189.157.131:9999 - ok sync - - - regular 216.189.157.210:9999 - ok sync - - - regular 216.230.232.125:9999 - ok sync - - - -regular 216.238.73.67:9999 - ok sync - - - evo 216.238.75.46:9999 443 ok sync ok ok valid regular 216.238.82.102:9999 - ok sync - - - -regular 216.238.82.108:9999 - ok sync - - - regular 216.238.85.93:9999 - ok sync - - - regular 216.238.85.238:9999 - ok sync - - - evo 216.238.99.9:9999 443 ok sync ok ok valid regular 217.69.5.141:9999 - ok sync - - - evo 217.76.54.175:9999 443 ok sync ok ok valid +regular 217.154.231.20:9999 - error ? - - - diff --git a/dash-network-seeds/seeds/testnet.txt b/dash-network-seeds/seeds/testnet.txt index f9426f81b..976fc979f 100644 --- a/dash-network-seeds/seeds/testnet.txt +++ b/dash-network-seeds/seeds/testnet.txt @@ -1,14 +1,13 @@ -# Auto-generated by masternode-seeds-fetcher on 2026-05-18T07:39:39Z for testnet +# Auto-generated by masternode-seeds-fetcher on 2026-06-22T08:25:06Z for testnet # Source: Dash P2P network (mnlistdiff) -# Primary peer: 68.67.122.32:19999 -# Tip block hash: 0000007ca418eb769dee64c6ad8e31a3b38702bfb33b2b304b124306f00e2025 -# Tip block height: 1478969 -# 87 seeds (56 regular + 31 evo) of 542 total masternodes, valid-only -# Probe summary: core_reachable=87/87 platform_reachable=31/31 ssl_valid=29/31 +# Primary peer: 68.67.122.37:19999 +# Tip block hash: 00000016d1295feff912992e68e83b864fcfa5ec90ed178cd3bb2b5168aba790 +# Tip block height: 1500630 +# 84 seeds (54 regular + 30 evo) of 540 total masternodes, valid-only +# Probe summary: core_reachable=84/84 platform_reachable=30/30 ssl_valid=30/30 # Columns: # Values: core_reach/plat_reach=ok|timeout|refused|error|?, core_sync=sync|-N|+N|?, plat_live=ok|none|?|-, plat_ssl=valid|expired|self-signed|untrusted|no-handshake|?|- # Do not edit manually — refreshed weekly by .github/workflows/update-masternode-seeds.yml -regular 62.60.244.174:19999 - ok sync - - - evo 68.67.122.1:19999 1443 ok sync ok ok valid evo 68.67.122.2:19999 1443 ok sync ok ok valid evo 68.67.122.3:19999 1443 ok sync ok ok valid @@ -91,7 +90,5 @@ regular 68.67.122.79:19999 - ok sync - - - regular 68.67.122.80:19999 - ok sync - - - regular 68.67.122.81:19999 - ok sync - - - regular 68.67.122.82:19999 - ok sync - - - -regular 147.93.132.9:19999 - ok sync - - - -regular 185.187.169.193:19999 - ok sync - - - -evo 206.245.167.63:19999 1443 ok sync ok ok self-signed -evo 212.47.68.199:19999 1443 ok sync ok ok expired +regular 78.17.70.175:19999 - ok sync - - - +evo 206.245.167.63:19999 1443 ok sync ok ok valid From 72621254555c5f5e12ed2bc222b31209fc126192 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Thu, 25 Jun 2026 06:02:11 -0700 Subject: [PATCH 007/149] feat(key-wallet): address pools gap limit can be externally modified runtime (#822) --- .../src/managed_account/address_pool.rs | 43 +++++++++++++++++++ .../managed_account/managed_account_trait.rs | 33 ++++++++++++++ 2 files changed, 76 insertions(+) diff --git a/key-wallet/src/managed_account/address_pool.rs b/key-wallet/src/managed_account/address_pool.rs index a6059667d..045098aa1 100644 --- a/key-wallet/src/managed_account/address_pool.rs +++ b/key-wallet/src/managed_account/address_pool.rs @@ -882,6 +882,19 @@ impl AddressPool { unused_count < self.gap_limit } + /// Raise the gap limit and generate the addresses the wider limit now requires. + /// + /// The new limit is capped at [`crate::gap_limit::MAX_GAP_LIMIT`]. Returns the freshly + /// generated [`AddressInfo`] entries + pub fn set_gap_limit( + &mut self, + gap_limit: u32, + key_source: &KeySource, + ) -> Result> { + self.gap_limit = gap_limit.clamp(1, crate::gap_limit::MAX_GAP_LIMIT); + self.maintain_gap_limit(key_source) + } + /// Generate addresses to maintain the gap limit. /// /// Returns the freshly generated [`AddressInfo`] entries (in derivation @@ -1218,6 +1231,36 @@ mod tests { assert_ne!(addr1, addr3); // Should return different address after marking used } + #[test] + fn test_set_gap_limit_widens_and_generates() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let key_source = test_key_source(); + + // Start with a small gap limit (5 pre-generated addresses). + let mut pool = AddressPool::new( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + &key_source, + ) + .unwrap(); + assert_eq!(pool.gap_limit, 5); + assert_eq!(pool.addresses.len(), 5); + + // Widening generates the extra addresses up front and returns them. + let new = pool.set_gap_limit(20, &key_source).unwrap(); + assert_eq!(pool.gap_limit, 20); + assert_eq!(new.len(), 15); + assert_eq!(pool.addresses.len(), 20); + assert_eq!(pool.highest_generated, Some(19)); + + // The new limit is capped at MAX_GAP_LIMIT. + pool.set_gap_limit(crate::gap_limit::MAX_GAP_LIMIT + 500, &key_source).unwrap(); + assert_eq!(pool.gap_limit, crate::gap_limit::MAX_GAP_LIMIT); + assert_eq!(pool.addresses.len(), crate::gap_limit::MAX_GAP_LIMIT as usize); + } + #[test] fn test_gap_limit_maintenance() { let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); diff --git a/key-wallet/src/managed_account/managed_account_trait.rs b/key-wallet/src/managed_account/managed_account_trait.rs index dfbb46b28..2c04d821e 100644 --- a/key-wallet/src/managed_account/managed_account_trait.rs +++ b/key-wallet/src/managed_account/managed_account_trait.rs @@ -552,6 +552,39 @@ pub trait ManagedAccountTrait { .sum() } + /// Widen the gap limit for every address pool of this account. + /// Returns every freshly generated address (across all pools) + fn set_gap_limit( + &mut self, + gap_limit: u32, + key_source: &address_pool::KeySource, + ) -> crate::error::Result> { + let mut generated = Vec::new(); + let mut first_error = None; + + for pool in self.managed_account_type_mut().address_pools_mut() { + match pool.set_gap_limit(gap_limit, key_source) { + Ok(new) => generated.extend(new), + Err(e) => { + first_error = Some(e); + break; + } + } + } + + // Bump the monitor revision if any pool actually generated addresses, even when a later + // pool failed — otherwise the revision would stay stale despite changed monitored state. + if !generated.is_empty() { + self.bump_monitor_revision(); + } + + if let Some(e) = first_error { + return Err(e); + } + + Ok(generated) + } + /// Get the gap limit for non-standard (single-pool) accounts fn gap_limit(&self) -> Option { match self.managed_account_type() { From 8c7b501e5a367b0978332345883e985c01f59395 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Thu, 25 Jun 2026 22:55:56 -0700 Subject: [PATCH 008/149] feat(key-wallet): track internal address pools for CoinJoin accounts (#821) --- key-wallet-ffi/src/managed_account.rs | 84 ++++++++++--------- key-wallet-ffi/src/utxo_tests.rs | 14 +++- .../managed_account_collection.rs | 26 ++++-- .../managed_account/managed_account_trait.rs | 8 +- .../managed_account/managed_account_type.rs | 62 +++++++++----- key-wallet/src/tests/account_tests.rs | 4 +- .../transaction_checking/account_checker.rs | 18 ++-- .../transaction_router/tests/conversions.rs | 6 +- .../transaction_router/tests/routing.rs | 4 +- .../transaction_checking/wallet_checker.rs | 8 +- 10 files changed, 139 insertions(+), 95 deletions(-) diff --git a/key-wallet-ffi/src/managed_account.rs b/key-wallet-ffi/src/managed_account.rs index bf8e3e8c0..b835d4dd9 100644 --- a/key-wallet-ffi/src/managed_account.rs +++ b/key-wallet-ffi/src/managed_account.rs @@ -25,7 +25,7 @@ use key_wallet::managed_account::address_pool::AddressPool; use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; use key_wallet::managed_account::managed_platform_account::ManagedPlatformAccount; use key_wallet::managed_account::{ManagedCoreFundsAccount, ManagedCoreKeysAccount}; -use key_wallet::AccountType; +use key_wallet::{AccountType, ManagedAccountType}; /// Internal handle variant: a funds-bearing account or a keys-only account. pub(crate) enum FFIManagedCoreAccountInner { @@ -1285,9 +1285,13 @@ pub unsafe extern "C" fn managed_core_account_get_external_address_pool( let account = &*account; let managed_account = account.keys_account(); - // Get external address pool if this is a standard account + // External pool exists for the dual-pool account types (Standard and CoinJoin). match managed_account.managed_account_type() { - key_wallet::managed_account::managed_account_type::ManagedAccountType::Standard { + ManagedAccountType::Standard { + external_addresses, + .. + } + | ManagedAccountType::CoinJoin { external_addresses, .. } => { @@ -1321,9 +1325,14 @@ pub unsafe extern "C" fn managed_core_account_get_internal_address_pool( let account = &*account; let managed_account = account.keys_account(); - // Get internal address pool if this is a standard account + // Internal pool exists for the dual-pool account types (Standard and CoinJoin). + use key_wallet::managed_account::managed_account_type::ManagedAccountType; match managed_account.managed_account_type() { - key_wallet::managed_account::managed_account_type::ManagedAccountType::Standard { + ManagedAccountType::Standard { + internal_addresses, + .. + } + | ManagedAccountType::CoinJoin { internal_addresses, .. } => { @@ -1363,21 +1372,19 @@ pub unsafe extern "C" fn managed_core_account_get_address_pool( use key_wallet::managed_account::managed_account_type::ManagedAccountType; - match pool_type { + let pool = match pool_type { FFIAddressPoolType::External => { // Only standard accounts have external pools match managed_account.managed_account_type() { ManagedAccountType::Standard { external_addresses, .. - } => { - let ffi_pool = FFIAddressPool { - pool: external_addresses as *const AddressPool as *mut AddressPool, - pool_type: FFIAddressPoolType::External, - }; - Box::into_raw(Box::new(ffi_pool)) - } - _ => std::ptr::null_mut(), + } => external_addresses, + ManagedAccountType::CoinJoin { + external_addresses, + .. + } => external_addresses, + _ => return std::ptr::null_mut(), } } FFIAddressPoolType::Internal => { @@ -1386,19 +1393,17 @@ pub unsafe extern "C" fn managed_core_account_get_address_pool( ManagedAccountType::Standard { internal_addresses, .. - } => { - let ffi_pool = FFIAddressPool { - pool: internal_addresses as *const AddressPool as *mut AddressPool, - pool_type: FFIAddressPoolType::Internal, - }; - Box::into_raw(Box::new(ffi_pool)) - } - _ => std::ptr::null_mut(), + } => internal_addresses, + ManagedAccountType::CoinJoin { + internal_addresses, + .. + } => internal_addresses, + _ => return std::ptr::null_mut(), } } FFIAddressPoolType::Single => { // Get the single address pool for non-standard accounts - let pool_ref = match managed_account.managed_account_type() { + match managed_account.managed_account_type() { ManagedAccountType::Standard { .. } => { @@ -1406,9 +1411,11 @@ pub unsafe extern "C" fn managed_core_account_get_address_pool( return std::ptr::null_mut(); } ManagedAccountType::CoinJoin { - addresses, .. - } => addresses, + } => { + // Conjoin accounts don't have a "single" pool + return std::ptr::null_mut(); + } ManagedAccountType::IdentityRegistration { addresses, } => addresses, @@ -1452,15 +1459,15 @@ pub unsafe extern "C" fn managed_core_account_get_address_pool( addresses, .. } => addresses, - }; - - let ffi_pool = FFIAddressPool { - pool: pool_ref as *const AddressPool as *mut AddressPool, - pool_type: FFIAddressPoolType::Single, - }; - Box::into_raw(Box::new(ffi_pool)) + } } - } + }; + + let ffi_pool = FFIAddressPool { + pool: pool as *const AddressPool as *mut AddressPool, + pool_type, + }; + Box::into_raw(Box::new(ffi_pool)) } // ==================== Platform Payment Account Functions ==================== @@ -2256,20 +2263,19 @@ mod tests { let cj_account = cj_result.account; - // Test that external/internal return null for CoinJoin account let cj_external = managed_core_account_get_external_address_pool(cj_account); - assert!(cj_external.is_null()); + assert!(!cj_external.is_null()); let cj_internal = managed_core_account_get_internal_address_pool(cj_account); - assert!(cj_internal.is_null()); + assert!(!cj_internal.is_null()); - // Test that Single pool works for CoinJoin account let cj_single = managed_core_account_get_address_pool(cj_account, FFIAddressPoolType::Single); - assert!(!cj_single.is_null()); + assert!(cj_single.is_null()); // Clean up - address_pool_free(cj_single); + address_pool_free(cj_external); + address_pool_free(cj_internal); managed_core_account_free(cj_account); wallet_manager_free_wallet_ids(wallet_ids_out, count_out); wallet_manager_free(manager); diff --git a/key-wallet-ffi/src/utxo_tests.rs b/key-wallet-ffi/src/utxo_tests.rs index 74c0dddca..81fe55977 100644 --- a/key-wallet-ffi/src/utxo_tests.rs +++ b/key-wallet-ffi/src/utxo_tests.rs @@ -344,10 +344,16 @@ mod utxo_tests { let mut coinjoin_account = ManagedCoreFundsAccount::new( ManagedAccountType::CoinJoin { index: 0, - addresses: key_wallet::managed_account::address_pool::AddressPoolBuilder::default() - .base_path(key_wallet::DerivationPath::from(vec![])) - .build() - .unwrap(), + external_addresses: + key_wallet::managed_account::address_pool::AddressPoolBuilder::default() + .base_path(key_wallet::DerivationPath::from(vec![])) + .build() + .unwrap(), + internal_addresses: + key_wallet::managed_account::address_pool::AddressPoolBuilder::default() + .base_path(key_wallet::DerivationPath::from(vec![])) + .build() + .unwrap(), }, Network::Testnet, ); diff --git a/key-wallet/src/managed_account/managed_account_collection.rs b/key-wallet/src/managed_account/managed_account_collection.rs index 8eaaac6e7..4c6c85ce1 100644 --- a/key-wallet/src/managed_account/managed_account_collection.rs +++ b/key-wallet/src/managed_account/managed_account_collection.rs @@ -585,20 +585,32 @@ impl ManagedAccountCollection { AccountType::CoinJoin { index, } => { - // CoinJoin addresses live on the external branch (m/9'/coin'/4'/account'/0/index) - // to match Dash Core, so derive through the `External` pool which uses [0, index]. - let mut coinjoin_path = base_path; - coinjoin_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); - let addresses = AddressPool::new( - coinjoin_path, + // Dual-pool: external (.../0/index) for Dash Core mixed coins, internal + // (.../1/index) for DashSync mixing-change. Watch both so no funds are missed. + let mut external_path = base_path.clone(); + external_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); + let external_addresses = AddressPool::new( + external_path, AddressPoolType::External, DEFAULT_COINJOIN_GAP_LIMIT, network, key_source, )?; + + let mut internal_path = base_path; + internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1)?); + let internal_addresses = AddressPool::new( + internal_path, + AddressPoolType::Internal, + DEFAULT_COINJOIN_GAP_LIMIT, + network, + key_source, + )?; + ManagedAccountType::CoinJoin { index, - addresses, + external_addresses, + internal_addresses, } } AccountType::IdentityRegistration => { diff --git a/key-wallet/src/managed_account/managed_account_trait.rs b/key-wallet/src/managed_account/managed_account_trait.rs index 2c04d821e..ff66eead9 100644 --- a/key-wallet/src/managed_account/managed_account_trait.rs +++ b/key-wallet/src/managed_account/managed_account_trait.rs @@ -133,7 +133,7 @@ pub trait ManagedAccountTrait { .. } => self.get_next_receive_address_index(), ManagedAccountType::CoinJoin { - addresses, + external_addresses: addresses, .. } | ManagedAccountType::IdentityRegistration { @@ -237,7 +237,7 @@ pub trait ManagedAccountTrait { .. } => Err("Standard accounts must use next_receive_address or next_change_address"), ManagedAccountType::CoinJoin { - addresses, + external_addresses: addresses, .. } | ManagedAccountType::IdentityRegistration { @@ -320,7 +320,7 @@ pub trait ManagedAccountTrait { "Standard accounts must use next_receive_address_with_info or next_change_address_with_info", ), ManagedAccountType::CoinJoin { - addresses, + external_addresses: addresses, .. } | ManagedAccountType::IdentityRegistration { @@ -592,7 +592,7 @@ pub trait ManagedAccountTrait { .. } => None, ManagedAccountType::CoinJoin { - addresses, + external_addresses: addresses, .. } | ManagedAccountType::IdentityRegistration { diff --git a/key-wallet/src/managed_account/managed_account_type.rs b/key-wallet/src/managed_account/managed_account_type.rs index d0f0492a4..40295e800 100644 --- a/key-wallet/src/managed_account/managed_account_type.rs +++ b/key-wallet/src/managed_account/managed_account_type.rs @@ -29,12 +29,18 @@ pub enum ManagedAccountType { /// Internal (change) address pool internal_addresses: AddressPool, }, - /// CoinJoin account for private transactions + /// CoinJoin account for private transactions. + /// + /// Dual-pool like `Standard`: Dash Core receives mixed coins on the external branch + /// (m/9'/coin'/4'/account'/0/index), while DashSync also uses the internal branch + /// (.../1/index) for mixing-change, so both chains must be watched to see all funds. CoinJoin { /// Account index index: u32, - /// CoinJoin address pool - addresses: AddressPool, + /// External (mixed-coin receive) address pool — .../0/index + external_addresses: AddressPool, + /// Internal (mixing-change) address pool — .../1/index + internal_addresses: AddressPool, }, /// Identity registration funding IdentityRegistration { @@ -211,14 +217,15 @@ impl ManagedAccountType { external_addresses, internal_addresses, .. - } => { - vec![external_addresses, internal_addresses] } - Self::CoinJoin { - addresses, + | Self::CoinJoin { + external_addresses, + internal_addresses, .. + } => { + vec![external_addresses, internal_addresses] } - | Self::IdentityRegistration { + Self::IdentityRegistration { addresses, .. } @@ -282,14 +289,15 @@ impl ManagedAccountType { external_addresses, internal_addresses, .. - } => { - vec![external_addresses, internal_addresses] } - Self::CoinJoin { - addresses, + | Self::CoinJoin { + external_addresses, + internal_addresses, .. + } => { + vec![external_addresses, internal_addresses] } - | Self::IdentityRegistration { + Self::IdentityRegistration { addresses, .. } @@ -529,23 +537,37 @@ impl ManagedAccountType { AccountType::CoinJoin { index, } => { - // CoinJoin addresses live on the external branch (m/9'/coin'/4'/account'/0/index) - // to match Dash Core, so derive through the `External` pool which uses [0, index]. - let mut path = account_type + // Dual-pool: Dash Core receives mixed coins on the external branch + // (m/9'/coin'/4'/account'/0/index); DashSync also uses the internal branch + // (.../1/index) for mixing-change. Watch both so no funds are missed. + let base_path = account_type .derivation_path(network) .unwrap_or_else(|_| DerivationPath::master()); - path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); - let pool = AddressPool::new( - path, + + let mut external_path = base_path.clone(); + external_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); + let external_pool = AddressPool::new( + external_path, AddressPoolType::External, DEFAULT_COINJOIN_GAP_LIMIT, network, key_source, )?; + let mut internal_path = base_path; + internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1)?); + let internal_pool = AddressPool::new( + internal_path, + AddressPoolType::Internal, + DEFAULT_COINJOIN_GAP_LIMIT, + network, + key_source, + )?; + Ok(Self::CoinJoin { index, - addresses: pool, + external_addresses: external_pool, + internal_addresses: internal_pool, }) } AccountType::IdentityRegistration => { diff --git a/key-wallet/src/tests/account_tests.rs b/key-wallet/src/tests/account_tests.rs index 1732823db..f788c46ec 100644 --- a/key-wallet/src/tests/account_tests.rs +++ b/key-wallet/src/tests/account_tests.rs @@ -142,9 +142,9 @@ fn test_coinjoin_account_creation() { .unwrap(); let pool = match &managed_type { ManagedAccountType::CoinJoin { - addresses, + external_addresses, .. - } => addresses, + } => external_addresses, _ => panic!("Expected CoinJoin managed account type"), }; let first_address = diff --git a/key-wallet/src/transaction_checking/account_checker.rs b/key-wallet/src/transaction_checking/account_checker.rs index 3ff50840c..bbff4ba8f 100644 --- a/key-wallet/src/transaction_checking/account_checker.rs +++ b/key-wallet/src/transaction_checking/account_checker.rs @@ -556,6 +556,11 @@ impl ManagedCoreFundsAccount { external_addresses, internal_addresses, .. + } + | ManagedAccountType::CoinJoin { + external_addresses, + internal_addresses, + .. } => { if external_addresses.contains_address(address) { AddressClassification::External @@ -565,16 +570,6 @@ impl ManagedCoreFundsAccount { AddressClassification::Other } } - ManagedAccountType::CoinJoin { - addresses, - .. - } => { - if addresses.contains_address(address) { - AddressClassification::External - } else { - AddressClassification::Other - } - } _ => { // For non-standard accounts, all addresses are "Other" AddressClassification::Other @@ -725,10 +720,9 @@ impl ManagedCoreFundsAccount { .. } => CoreAccountTypeMatch::CoinJoin { account_index: index.unwrap_or(0), - // For CoinJoin, use both receive addresses and other addresses - // since CoinJoin addresses can be classified as either involved_addresses: { let mut all_addresses = involved_receive_addresses.clone(); + all_addresses.extend(involved_change_addresses.clone()); all_addresses.extend(involved_other_addresses.clone()); all_addresses }, diff --git a/key-wallet/src/transaction_checking/transaction_router/tests/conversions.rs b/key-wallet/src/transaction_checking/transaction_router/tests/conversions.rs index 1620447ca..309805bcc 100644 --- a/key-wallet/src/transaction_checking/transaction_router/tests/conversions.rs +++ b/key-wallet/src/transaction_checking/transaction_router/tests/conversions.rs @@ -52,7 +52,8 @@ fn test_try_from_standard_bip32_account() { fn test_try_from_coinjoin_account() { let managed_account = ManagedAccountType::CoinJoin { index: 0, - addresses: test_single_pool(), + external_addresses: test_single_pool(), + internal_addresses: test_single_pool(), }; let check_type: AccountTypeToCheck = managed_account.try_into().unwrap(); @@ -181,7 +182,8 @@ fn test_try_from_ref_all_account_types() { ( ManagedAccountType::CoinJoin { index: 0, - addresses: test_single_pool(), + external_addresses: test_single_pool(), + internal_addresses: test_single_pool(), }, AccountTypeToCheck::CoinJoin, ), diff --git a/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs b/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs index b5a49a1d5..76568c45f 100644 --- a/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs +++ b/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs @@ -180,13 +180,13 @@ async fn test_transaction_routing_to_coinjoin_account() { .first_coinjoin_managed_account_mut() .expect("Failed to get first CoinJoin managed account"); let ManagedAccountType::CoinJoin { - addresses, + external_addresses, .. } = managed_account.managed_account_type_mut() else { panic!("Expected CoinJoin account type"); }; - addresses + external_addresses .next_unused(&KeySource::Public(xpub), true) .expect("Failed to derive CoinJoin address") }; diff --git a/key-wallet/src/transaction_checking/wallet_checker.rs b/key-wallet/src/transaction_checking/wallet_checker.rs index 9a5f803bf..b62def243 100644 --- a/key-wallet/src/transaction_checking/wallet_checker.rs +++ b/key-wallet/src/transaction_checking/wallet_checker.rs @@ -1645,13 +1645,15 @@ mod tests { let managed_account = managed_wallet.first_coinjoin_managed_account_mut().expect("managed coinjoin"); - // Get an address from the CoinJoin pool + // Get an address from the CoinJoin external (receive) pool let coinjoin_address = if let ManagedAccountType::CoinJoin { - addresses, + external_addresses, .. } = managed_account.managed_account_type_mut() { - addresses.next_unused(&KeySource::Public(xpub), true).expect("coinjoin address") + external_addresses + .next_unused(&KeySource::Public(xpub), true) + .expect("coinjoin address") } else { panic!("Expected CoinJoin account type"); }; From e6b8b8720d2ffa1fe139a1c97bf214d66c7c4ac5 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Fri, 26 Jun 2026 18:08:31 +1000 Subject: [PATCH 009/149] fix: resolve just-retired quorum by walking earlier masternode lists (#801) * fix: resolve just-retired quorum by walking earlier masternode lists `get_quorum_at_height` and the FFI `ffi_dash_spv_get_quorum_public_key` resolved a quorum only through the single nearest masternode list at or below the lookup height. A Platform signing quorum is selected at a lagged height, so by the proof's `core_chain_locked_height` it can already have retired out of the active set. `apply_diff` drops a retired quorum from every later list, so the nearest list misses and the lookup fails with `Quorum not found`, surfacing downstream as `InvalidQuorum`. The failure is intermittent, firing only at the retirement edge. The full entry is not actually gone: `apply_diff` clones the base list without mutating earlier ones, and old per-height lists are retained, so the quorum still lives in every list from its mint height up to retirement. Add `MasternodeListEngine::quorum_entry_for_hash_at_or_before_height`, which walks `masternode_lists` backward from the lookup height and returns the first list still holding the quorum, skipping `Invalid` entries. Both consumer sites route through it, so the real full `QualifiedQuorumEntry` is returned with no synthesized data, no new storage, and no API change. The walk is floored at a few active windows below the height (derived from the type's `signing_active_quorum_count` and DKG interval): a legitimately referenced signing quorum cannot be older than that, so a miss scans a bounded span rather than every accumulated list. Closes #800. * fix: emit `tracing::warn!` message as format string in `dash-spv` quorum lookup Addresses CodeRabbit review comment on PR #801 https://github.com/dashpay/rust-dashcore/pull/801#discussion_r3343112113 * refactor: drop redundant `QuorumHash` import in `helpers` test module Addresses CodeRabbit review comment on PR #801 https://github.com/dashpay/rust-dashcore/pull/801#discussion_r3343112141 * docs: avoid private intra-doc link in public quorum walk-back docs The public `quorum_entry_for_hash_at_or_before_height` doc linked to the private `QUORUM_WALK_BACK_ACTIVE_WINDOWS` constant, which rustdoc rejects under `-D rustdoc::private-intra-doc-links`. Demote the link to a plain code span so the Documentation CI job passes. --- dash-spv-ffi/src/platform_integration.rs | 58 ++--- dash-spv/src/client/queries.rs | 73 ++---- .../src/sml/masternode_list_engine/helpers.rs | 208 ++++++++++++++++++ 3 files changed, 247 insertions(+), 92 deletions(-) diff --git a/dash-spv-ffi/src/platform_integration.rs b/dash-spv-ffi/src/platform_integration.rs index ce1609788..bb323c3bf 100644 --- a/dash-spv-ffi/src/platform_integration.rs +++ b/dash-spv-ffi/src/platform_integration.rs @@ -101,53 +101,25 @@ pub unsafe extern "C" fn ffi_dash_spv_get_quorum_public_key( }; let engine_guard = engine.blocking_read(); - let (before, _after) = engine_guard.masternode_lists_around_height(core_chain_locked_height); - let ml = match before { - Some(ml) => ml, - None => { - return FFIResult::error( - FFIErrorCode::ValidationError, - &format!( - "No masternode list found at or before height {}", - core_chain_locked_height - ), - ); - } - }; - - let list_height = ml.known_height; - match ml.quorums.get(&llmq_type) { - Some(quorums) => match quorums.get(&quorum_hash) { - Some(quorum) => { - let pubkey_bytes: &[u8; 48] = quorum.quorum_entry.quorum_public_key.as_ref(); - std::ptr::copy_nonoverlapping( - pubkey_bytes.as_ptr(), - out_pubkey, - QUORUM_PUBKEY_SIZE, - ); - - FFIResult { - error_code: 0, - error_message: ptr::null(), - } + match engine_guard.quorum_entry_for_hash_at_or_before_height( + llmq_type, + quorum_hash, + core_chain_locked_height, + ) { + Some((_list_height, quorum)) => { + let pubkey_bytes: &[u8; 48] = quorum.quorum_entry.quorum_public_key.as_ref(); + std::ptr::copy_nonoverlapping(pubkey_bytes.as_ptr(), out_pubkey, QUORUM_PUBKEY_SIZE); + + FFIResult { + error_code: 0, + error_message: ptr::null(), } - None => FFIResult::error( - FFIErrorCode::ValidationError, - &format!( - "Quorum not found: type {} at list height {} (requested {}) with hash {:x} (masternode list exists with {} quorums of this type)", - quorum_type, - list_height, - core_chain_locked_height, - quorum_hash, - quorums.len() - ), - ), - }, + } None => FFIResult::error( FFIErrorCode::ValidationError, &format!( - "No quorums of type {} found at list height {} (requested {})", - quorum_type, list_height, core_chain_locked_height + "Quorum not found: type {} at or before height {} with hash {:x}", + quorum_type, core_chain_locked_height, quorum_hash ), ), } diff --git a/dash-spv/src/client/queries.rs b/dash-spv/src/client/queries.rs index 7e215cc86..00836b220 100644 --- a/dash-spv/src/client/queries.rs +++ b/dash-spv/src/client/queries.rs @@ -53,56 +53,31 @@ impl DashSpvClient Result { let masternode_engine = self.masternode_list_engine()?; let masternode_engine_guard = masternode_engine.read().await; - let (before, _after) = masternode_engine_guard.masternode_lists_around_height(height); - if let Some(ml) = before { - let list_height = ml.known_height; - match ml.quorums.get(&quorum_type) { - Some(quorums) => match quorums.get(&quorum_hash) { - Some(quorum) => { - tracing::debug!( - "Found quorum type {} at list height {} (requested {}) with hash {}", - quorum_type, - list_height, - height, - hex::encode(quorum_hash) - ); - return Ok(quorum.clone()); - } - None => { - let message = format!( - "Quorum not found: type {} at list height {} (requested {}) with hash {} (masternode list exists with {} quorums of this type)", - quorum_type, - list_height, - height, - hex::encode(quorum_hash), - quorums.len() - ); - tracing::warn!(message); - return Err(SpvError::QuorumLookupError(message)); - } - }, - None => { - tracing::warn!( - "No quorums of type {} found at list height {} (requested {}) (masternode list exists)", - quorum_type, - list_height, - height - ); - return Err(SpvError::QuorumLookupError(format!( - "No quorums of type {} found at list height {} (requested {})", - quorum_type, list_height, height - ))); - } + match masternode_engine_guard.quorum_entry_for_hash_at_or_before_height( + quorum_type, + quorum_hash, + height, + ) { + Some((list_height, quorum)) => { + tracing::debug!( + "Found quorum type {} at list height {} (requested {}) with hash {}", + quorum_type, + list_height, + height, + hex::encode(quorum_hash) + ); + Ok(quorum.clone()) + } + None => { + let message = format!( + "Quorum not found: type {} at or before height {} with hash {}", + quorum_type, + height, + hex::encode(quorum_hash) + ); + tracing::warn!("{}", message); + Err(SpvError::QuorumLookupError(message)) } } - - tracing::warn!( - "No masternode list found at or before height {} - cannot retrieve quorum", - height - ); - Err(SpvError::QuorumLookupError(format!( - "No masternode list found at or before height {}", - height - ))) } } diff --git a/dash/src/sml/masternode_list_engine/helpers.rs b/dash/src/sml/masternode_list_engine/helpers.rs index 3fa11bb15..9dfbaeccc 100644 --- a/dash/src/sml/masternode_list_engine/helpers.rs +++ b/dash/src/sml/masternode_list_engine/helpers.rs @@ -1,6 +1,17 @@ +use crate::QuorumHash; use crate::prelude::CoreBlockHeight; +use crate::sml::llmq_entry_verification::LLMQEntryVerificationStatus; +use crate::sml::llmq_type::LLMQType; use crate::sml::masternode_list::MasternodeList; use crate::sml::masternode_list_engine::MasternodeListEngine; +use crate::sml::quorum_entry::qualified_quorum_entry::QualifiedQuorumEntry; + +/// How many active windows below the lookup height [`MasternodeListEngine::quorum_entry_for_hash_at_or_before_height`] +/// searches before giving up. A signing quorum referenced by a proof was selected at a lagged +/// height that can exceed one active window (Platform selects roughly 4.5 DKG intervals back), so a +/// single window is too tight. Four windows covers that lag with wide margin while still bounding a +/// miss to a fixed span of lists rather than every list the engine has accumulated. +const QUORUM_WALK_BACK_ACTIVE_WINDOWS: u32 = 4; impl MasternodeListEngine { /// Retrieves the closest masternode lists before and after a given core block height. @@ -38,4 +49,201 @@ impl MasternodeListEngine { (lower, upper) } + + /// Resolves a quorum entry by type and hash, searching masternode lists at or below + /// `height` from the nearest downward and returning the first one that still holds it. + /// + /// The nearest list at or below `height` may no longer contain the quorum: once a quorum + /// retires out of the active set, `apply_diff` drops it from every list built from that + /// point on. A signing quorum selected at a lagged height can therefore be absent from the + /// nearest list yet still present in an earlier, retained one. Walking backward returns that + /// earlier full entry rather than failing the lookup. Entries marked `Invalid` are skipped. + /// + /// The returned `CoreBlockHeight` is the height of the list the entry was resolved from. The + /// first match is the highest list still holding the quorum, so a hit stops a few cycles back at + /// most. The walk is floored at `QUORUM_WALK_BACK_ACTIVE_WINDOWS` active windows below `height` + /// (derived from the type's DKG interval and active quorum count): a legitimately referenced + /// signing quorum cannot be older than that, so flooring it bounds a miss to a fixed span of + /// lists rather than scanning every list the engine has accumulated. + pub fn quorum_entry_for_hash_at_or_before_height( + &self, + llmq_type: LLMQType, + quorum_hash: QuorumHash, + height: CoreBlockHeight, + ) -> Option<(CoreBlockHeight, &QualifiedQuorumEntry)> { + let params = llmq_type.params(); + let active_window = + params.signing_active_quorum_count.saturating_mul(params.dkg_params.interval); + let floor = + height.saturating_sub(active_window.saturating_mul(QUORUM_WALK_BACK_ACTIVE_WINDOWS)); + + self.masternode_lists.range(floor..=height).rev().find_map(|(_, list)| { + list.quorum_entry_of_type_for_quorum_hash(llmq_type, quorum_hash) + .filter(|quorum| { + !matches!(quorum.verified, LLMQEntryVerificationStatus::Invalid(_)) + }) + .map(|quorum| (list.known_height, quorum)) + }) + } +} + +#[cfg(test)] +mod tests { + use std::slice; + + use hashes::Hash; + + use super::*; + use crate::BlockHash; + use crate::bls_sig_utils::{BLSPublicKey, BLSSignature}; + use crate::hash_types::QuorumVVecHash; + use crate::sml::quorum_validation_error::QuorumValidationError; + use crate::transaction::special_transaction::quorum_commitment::QuorumEntry; + + const PLATFORM_TYPE: LLMQType = LLMQType::LlmqtypeDevnetPlatform; + + fn quorum_entry(quorum_hash: QuorumHash, pubkey: u8) -> QualifiedQuorumEntry { + let mut entry: QualifiedQuorumEntry = QuorumEntry { + version: 2, + llmq_type: PLATFORM_TYPE, + quorum_hash, + quorum_index: Some(0), + signers: vec![true; 4], + valid_members: vec![true; 4], + quorum_public_key: BLSPublicKey::from([pubkey; 48]), + quorum_vvec_hash: QuorumVVecHash::all_zeros(), + threshold_sig: BLSSignature::from([1; 96]), + all_commitment_aggregated_signature: BLSSignature::from([1; 96]), + } + .into(); + entry.verified = LLMQEntryVerificationStatus::Verified; + entry + } + + fn list_with_quorums(height: u32, quorums: &[QualifiedQuorumEntry]) -> MasternodeList { + let mut list = + MasternodeList::empty(BlockHash::from_byte_array([height as u8; 32]), height); + let by_hash = list.quorums.entry(PLATFORM_TYPE).or_default(); + for quorum in quorums { + by_hash.insert(quorum.quorum_entry.quorum_hash, quorum.clone()); + } + list + } + + /// A quorum retired out of the active set is dropped from the nearest list at or below the + /// lookup height, but the backward walk resolves it from the earlier list that still holds it. + #[test] + fn resolves_retired_quorum_from_earlier_list() { + let retired_hash = QuorumHash::from_byte_array([0xAB; 32]); + let active_hash = QuorumHash::from_byte_array([0xCD; 32]); + let retired = quorum_entry(retired_hash, 7); + + let mut engine = MasternodeListEngine::default(); + // Pre-retirement list still holds the retired quorum. + engine.masternode_lists.insert(148, list_with_quorums(148, slice::from_ref(&retired))); + // Post-retirement list holds only the then-active quorum, not the retired one. + engine + .masternode_lists + .insert(208, list_with_quorums(208, &[quorum_entry(active_hash, 9)])); + + // The nearest list at or below the lookup height no longer holds the retired quorum. + let nearest = engine.masternode_lists_around_height(208).0.unwrap(); + assert!( + nearest.quorum_entry_of_type_for_quorum_hash(PLATFORM_TYPE, retired_hash).is_none() + ); + + // The walk resolves it from the earlier retained list, returning that list's height. + let (resolved_height, resolved) = engine + .quorum_entry_for_hash_at_or_before_height(PLATFORM_TYPE, retired_hash, 208) + .expect("retired quorum resolves from earlier list"); + assert_eq!(resolved_height, 148); + assert_eq!(resolved.quorum_entry.quorum_public_key, retired.quorum_entry.quorum_public_key); + } + + /// While still in the active set the quorum resolves from the nearest list directly. + #[test] + fn resolves_active_quorum_from_nearest_list() { + let hash = QuorumHash::from_byte_array([0xAB; 32]); + let mut engine = MasternodeListEngine::default(); + engine.masternode_lists.insert(148, list_with_quorums(148, &[quorum_entry(hash, 7)])); + + let (resolved_height, _) = engine + .quorum_entry_for_hash_at_or_before_height(PLATFORM_TYPE, hash, 148) + .expect("active quorum resolves"); + assert_eq!(resolved_height, 148); + } + + /// A lookup below every retained list, or for an unknown hash, finds nothing. + #[test] + fn returns_none_when_not_present() { + let hash = QuorumHash::from_byte_array([0xAB; 32]); + let mut engine = MasternodeListEngine::default(); + engine.masternode_lists.insert(148, list_with_quorums(148, &[quorum_entry(hash, 7)])); + + assert!( + engine.quorum_entry_for_hash_at_or_before_height(PLATFORM_TYPE, hash, 100).is_none() + ); + assert!( + engine + .quorum_entry_for_hash_at_or_before_height( + PLATFORM_TYPE, + QuorumHash::from_byte_array([0xEE; 32]), + 208 + ) + .is_none() + ); + } + + /// An `Invalid` entry is skipped, even when it is the only list holding the hash. + #[test] + fn skips_invalid_entries() { + let hash = QuorumHash::from_byte_array([0xAB; 32]); + let mut invalid = quorum_entry(hash, 7); + invalid.verified = + LLMQEntryVerificationStatus::Invalid(QuorumValidationError::InvalidQuorumPublicKey); + + let mut engine = MasternodeListEngine::default(); + engine.masternode_lists.insert(148, list_with_quorums(148, &[invalid])); + + assert!( + engine.quorum_entry_for_hash_at_or_before_height(PLATFORM_TYPE, hash, 208).is_none() + ); + } + + /// The walk is floored at a few active windows below the lookup height: a quorum that only + /// survives in a list older than the floor is treated as not found, while one within the window + /// still resolves. This bounds a miss instead of scanning every accumulated list. + #[test] + fn does_not_walk_below_active_window_floor() { + let params = PLATFORM_TYPE.params(); + let span = params.signing_active_quorum_count + * params.dkg_params.interval + * QUORUM_WALK_BACK_ACTIVE_WINDOWS; + let height = span + 5_000; + let floor = height - span; + + let within_hash = QuorumHash::from_byte_array([0x11; 32]); + let below_hash = QuorumHash::from_byte_array([0x22; 32]); + + let mut engine = MasternodeListEngine::default(); + // One list just above the floor and one well below it. + engine + .masternode_lists + .insert(floor + 100, list_with_quorums(floor + 100, &[quorum_entry(within_hash, 7)])); + engine + .masternode_lists + .insert(floor - 100, list_with_quorums(floor - 100, &[quorum_entry(below_hash, 9)])); + + let (resolved_height, _) = engine + .quorum_entry_for_hash_at_or_before_height(PLATFORM_TYPE, within_hash, height) + .expect("quorum within the window resolves"); + assert_eq!(resolved_height, floor + 100); + + assert!( + engine + .quorum_entry_for_hash_at_or_before_height(PLATFORM_TYPE, below_hash, height) + .is_none(), + "quorum below the floor must not be walked to" + ); + } } From 1b53458107024e4a937b86137f1f6c4837d75530 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Fri, 26 Jun 2026 19:21:25 +1000 Subject: [PATCH 010/149] fix(dash-spv): reprocess blocks on rescan against newly derived scripts (#820) * fix(dash-spv): reprocess blocks on rescan against newly derived scripts The commit-time batch rescan re-matches scripts derived during block processing against the active batches, but `BlockMatchTracker::track` subtracted wallets that already had a block applied, so a block processed before those scripts existed was silently skipped. Scripts derived from processing a block could therefore never trigger a re-examination of any earlier block in the same batch, and partially recognized blocks stayed partial. Add `track_for_new_scripts`, which keeps the in-flight accounting of `track` but ignores processed records, and use it in `rescan_batch`: every match there is by construction against scripts that did not exist at first processing. Re-application is safe because wallet-side block processing is idempotent (transaction dedup, outpoint-keyed UTXO insertion, balance recomputed from the UTXO set). Termination is preserved since the rescan only queries newly collected scripts, and re-processing a block whose outputs are already recognized derives nothing further. The plain `scan_batch` path keeps the processed-record skip, late-added wallets still get their residual pass through plain `track`. * test(dash-spv): cover a burst payment beyond the gap window on cold sync One transaction pays a run of consecutive fresh addresses reaching past the gap window, mined before the client ever starts. The historical scan must climb the burst via fixpoint re-scanning and credit every output. Adds `DashCoreNode::send_many` so a regtest wallet can pay many addresses in a single transaction. --- .../src/sync/filters/block_match_tracker.rs | 39 +++++++++ dash-spv/src/sync/filters/manager.rs | 52 +++++++++++- dash-spv/src/test_utils/node.rs | 18 ++++- .../tests/dashd_sync/tests_transaction.rs | 81 ++++++++++++++++++- 4 files changed, 186 insertions(+), 4 deletions(-) diff --git a/dash-spv/src/sync/filters/block_match_tracker.rs b/dash-spv/src/sync/filters/block_match_tracker.rs index 6f9301491..e0d911b47 100644 --- a/dash-spv/src/sync/filters/block_match_tracker.rs +++ b/dash-spv/src/sync/filters/block_match_tracker.rs @@ -92,6 +92,28 @@ impl BlockMatchTracker { } } + /// Like [`Self::track`], but for matches driven by newly derived + /// scripts: the prior processing of this block predates those scripts, + /// so processed records do not cover them and the block must be + /// re-applied to the full candidate set. Never returns + /// `AlreadyProcessed`. + pub(super) fn track_for_new_scripts( + &mut self, + key: &FilterMatchKey, + batch_start: u32, + candidate_wallets: BTreeSet, + ) -> BlockTrackResult { + if self.blocks_remaining.contains_key(key.hash()) { + return BlockTrackResult::InFlight { + wallets: candidate_wallets, + }; + } + self.blocks_remaining.insert(*key.hash(), (key.height(), batch_start)); + BlockTrackResult::NewlyTracked { + wallets: candidate_wallets, + } + } + /// Record that `wallets` have had the block at `(height, hash)` applied /// to their state. Idempotent: existing entries merge, never shrink. pub(super) fn record_processed( @@ -210,6 +232,23 @@ mod tests { tracker.track(&key, 5000, BTreeSet::from([wallet_a, wallet_b])), BlockTrackResult::AlreadyProcessed ); + + // A match on newly derived scripts re-queues the fully-processed + // block anyway: the prior processing predates those scripts. + assert_eq!( + tracker.track_for_new_scripts(&key, 5000, BTreeSet::from([wallet_a])), + BlockTrackResult::NewlyTracked { + wallets: BTreeSet::from([wallet_a]) + } + ); + // While re-queued the block reports InFlight, preserving the + // one-increment-one-decrement pending accounting. + assert_eq!( + tracker.track_for_new_scripts(&key, 5000, BTreeSet::from([wallet_a])), + BlockTrackResult::InFlight { + wallets: BTreeSet::from([wallet_a]) + } + ); } /// `prune_at_or_below` drops every entry at or below the given height diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index 4128333f1..d2e45e9a2 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -671,7 +671,11 @@ impl { @@ -685,6 +689,7 @@ impl {} } } @@ -936,7 +941,7 @@ mod tests { }; use crate::sync::{ManagerIdentifier, SyncManagerProgress}; use dashcore::bip158::BlockFilter; - use dashcore::Header; + use dashcore::{Address, Header}; use dashcore::{Block, Network, Transaction}; use dashcore_hashes::Hash; use key_wallet_manager::test_utils::{ @@ -1347,6 +1352,49 @@ mod tests { assert!(!attr_b.contains(&wallet_a)); } + /// A block that was already processed for a wallet is re-queued by + /// `rescan_batch` when newly derived scripts match it: the prior + /// processing predates those scripts, so the block must be re-applied + /// against the extended pools. Plain scans keep skipping processed + /// blocks. + #[tokio::test] + async fn test_rescan_batch_reprocesses_already_processed_block() { + let address = Address::dummy(Network::Regtest, 33); + let mut manager = create_test_manager().await; + manager.set_state(SyncState::Syncing); + + let (key, filter) = filter_for_address(20, &address); + let mut filters: HashMap = HashMap::new(); + filters.insert(key.clone(), filter); + let mut batch = FiltersBatch::new(0, 99, filters); + batch.mark_verified(); + manager.active_batches.insert(0, batch); + + // The block was downloaded and processed for the wallet before the + // rescan script existed. + manager.tracker.record_processed(20, *key.hash(), &BTreeSet::from([MOCK_WALLET_ID])); + + let mut new_scripts: HashMap> = HashMap::new(); + new_scripts.insert(MOCK_WALLET_ID, HashSet::from([address.script_pubkey()])); + + let events = manager.rescan_batch(0, &new_scripts).await.unwrap(); + + let blocks = events + .iter() + .find_map(|e| match e { + SyncEvent::BlocksNeeded { + blocks, + } => Some(blocks), + _ => None, + }) + .expect("BlocksNeeded event for the re-queued block"); + assert!(blocks.get(&key).expect("re-queued block entry").contains(&MOCK_WALLET_ID)); + + // The re-queued block is accounted exactly once so the batch cannot + // commit before its re-processing completes. + assert_eq!(manager.active_batches.get(&0).unwrap().pending_blocks(), 1); + } + /// `rescan_batch` honours each wallet's own `synced_height`: a new /// address belonging to a wallet that has already advanced past a height /// must not produce a `BlocksNeeded` for that height, even when the diff --git a/dash-spv/src/test_utils/node.rs b/dash-spv/src/test_utils/node.rs index aaf46574b..41dcd9b71 100644 --- a/dash-spv/src/test_utils/node.rs +++ b/dash-spv/src/test_utils/node.rs @@ -6,7 +6,7 @@ use dashcore::{Address, Amount, BlockHash, Transaction, Txid}; use dashcore_rpc::json as rpc_json; use dashcore_rpc::{Auth, Client, RpcApi}; use serde::Deserialize; -use serde_json::Value; +use serde_json::{Map, Value}; use std::collections::HashMap; use std::fs; use std::net::SocketAddr; @@ -316,6 +316,22 @@ impl DashCoreNode { txid } + /// Send DASH to many addresses in a single transaction from the primary + /// wallet, so one transaction carries one output per `(address, amount)` + /// pair. + pub fn send_many(&self, payments: &[(Address, Amount)]) -> Txid { + let client = self.rpc_client(); + let amounts: Map = payments + .iter() + .map(|(address, amount)| (address.to_string(), serde_json::json!(amount.to_dash()))) + .collect(); + let txid: Txid = client + .call("sendmany", &[serde_json::json!(""), Value::Object(amounts)]) + .expect("failed to sendmany"); + tracing::info!("Sent {} outputs in one transaction, txid: {}", payments.len(), txid); + txid + } + /// Send DASH to an address from a specific wallet. pub fn send_to_address_from_wallet( &self, diff --git a/dash-spv/tests/dashd_sync/tests_transaction.rs b/dash-spv/tests/dashd_sync/tests_transaction.rs index ec3b66957..0b99f8a32 100644 --- a/dash-spv/tests/dashd_sync/tests_transaction.rs +++ b/dash-spv/tests/dashd_sync/tests_transaction.rs @@ -5,12 +5,18 @@ use std::time::Duration; use tokio::sync::RwLock; use super::helpers::{ - wait_for_mempool_tx, wait_for_sync, wait_for_wallet_synced, EMPTY_MNEMONIC, SECONDARY_MNEMONIC, + count_wallet_transactions, get_spendable_balance, wait_for_mempool_tx, wait_for_sync, + wait_for_wallet_synced, EMPTY_MNEMONIC, SECONDARY_MNEMONIC, }; use super::setup::{create_and_start_client, TestContext}; use dash_spv::test_utils::{create_test_wallet, TestChain}; use dashcore::address::NetworkUnchecked; +use dashcore::secp256k1::Secp256k1; +use dashcore::PublicKey; use key_wallet::account::ManagedAccountTrait; +use key_wallet::bip32::{ChildNumber, ExtendedPrivKey}; +use key_wallet::gap_limit::DEFAULT_EXTERNAL_GAP_LIMIT; +use key_wallet::mnemonic::{Language, Mnemonic}; use key_wallet::wallet::managed_wallet_info::transaction_builder::{ BuilderError, TransactionBuilder, }; @@ -247,6 +253,79 @@ async fn test_multiple_transactions_across_blocks() { const MEMPOOL_TIMEOUT: Duration = Duration::from_secs(30); +/// Derive the first `count` BIP44 external addresses of `mnemonic` directly, +/// independently of any wallet state, so a test can pay addresses far beyond +/// the pre-generated pool window. +fn derive_external_addresses(mnemonic: &str, count: u32) -> Vec
{ + let mnemonic = Mnemonic::from_phrase(mnemonic, Language::English).expect("mnemonic"); + let seed = mnemonic.to_seed(""); + let secp = Secp256k1::new(); + let master = ExtendedPrivKey::new_master(Network::Regtest, &seed).expect("master key"); + let chain = [ + ChildNumber::from_hardened_idx(44).expect("purpose"), + ChildNumber::from_hardened_idx(1).expect("coin type"), + ChildNumber::from_hardened_idx(0).expect("account"), + ChildNumber::from_normal_idx(0).expect("external branch"), + ]; + (0..count) + .map(|index| { + let mut path = chain.to_vec(); + path.push(ChildNumber::from_normal_idx(index).expect("index")); + let xprv = master.derive_priv(&secp, &path).expect("derive"); + let pk = PublicKey::new(xprv.private_key.public_key(&secp)); + Address::p2pkh(&pk, Network::Regtest) + }) + .collect() +} + +/// A single transaction paying a run of consecutive fresh addresses reaching +/// far beyond the gap window (the shape of a CreateDenominations burst), +/// mined before the client ever starts. A sync from scratch must recognize +/// every output: the block is scanned to fixpoint against the extending +/// pool, so outputs past the initial look-ahead are still credited. +#[tokio::test] +async fn test_burst_payment_beyond_gap_window_synced_from_scratch() { + let Some(ctx) = TestContext::new(TestChain::Minimal).await else { + return; + }; + if !ctx.dashd.supports_mining { + eprintln!("Skipping test (dashd RPC miner not available)"); + return; + } + + let burst = DEFAULT_EXTERNAL_GAP_LIMIT + 21; + let addresses = derive_external_addresses(EMPTY_MNEMONIC, burst); + let per_output = Amount::from_sat(100_000); + let payments: Vec<(Address, Amount)> = + addresses.into_iter().map(|address| (address, per_output)).collect(); + let burst_txid = ctx.dashd.node.send_many(&payments); + + let miner_address = ctx.dashd.node.get_new_address_from_wallet("default"); + ctx.dashd.node.generate_blocks(1, &miner_address); + let funded_height = ctx.dashd.initial_height + 1; + + // Only now create the wallet and start the client, so discovery has to + // climb the whole burst during the historical scan. + let (wallet, wallet_id) = create_test_wallet(EMPTY_MNEMONIC, Network::Regtest); + let mut client_handle = create_and_start_client(&ctx.client_config, Arc::clone(&wallet)).await; + wait_for_sync(&mut client_handle.progress_receiver, funded_height).await; + wait_for_wallet_synced(&wallet, &wallet_id, funded_height).await; + + assert_eq!( + count_wallet_transactions(&wallet, &wallet_id).await, + 1, + "burst tx {} must be discovered", + burst_txid + ); + assert_eq!( + get_spendable_balance(&wallet, &wallet_id).await, + burst as u64 * per_output.to_sat(), + "every burst output must be credited, not only those inside the initial gap window" + ); + + client_handle.stop().await; +} + async fn reserve_first_address(mnemonic: &str) -> Address { let (temp_mgr, temp_id) = create_test_wallet(mnemonic, Network::Regtest); From e9f381642a3dabc668b5795e2b6b9c428aed4d00 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Fri, 26 Jun 2026 19:28:04 +1000 Subject: [PATCH 011/149] fix: compute SML entry hash with `SER_GETHASH` semantics in `dash` (#798) `MasternodeListEntry::calculate_entry_hash` hashed the full network consensus serialization, including the leading `version`. Dash Core's `CSimplifiedMNListEntry::CalcHash` uses `CHashWriter(SER_GETHASH, ...)`, and `SER_GETHASH` does not set `SER_NETWORK`, so the `SER_NETWORK`-gated leading `nVersion` is omitted from the hash pre-image. The two contexts therefore diverged for every entry. Extract the post-`version` body of the wire encoder into a shared `consensus_encode_body` helper. `consensus_encode` writes `version` then the body (byte-identical to before), while `calculate_entry_hash` hashes the body alone, matching Core's `CalcHash`. Verified against Core for both a `version` 1 and a `version` 2 Evo entry. Also fix `SocketAddr` decoding to use `to_ipv4_mapped` instead of `to_ipv4`, so an all-zero (`::`) service address round-trips to the same 16 bytes instead of acquiring an `::ffff:` mapped prefix. The prefix corruption changed the wire bytes and thus the entry hash for masternodes with an unset service. --- dash/src/sml/address.rs | 16 ++++++- dash/src/sml/masternode_list_entry/hash.rs | 51 ++++++++++++++++++++-- dash/src/sml/masternode_list_entry/mod.rs | 22 ++++++++-- 3 files changed, 82 insertions(+), 7 deletions(-) diff --git a/dash/src/sml/address.rs b/dash/src/sml/address.rs index 5eafec8e8..8ec1757ce 100644 --- a/dash/src/sml/address.rs +++ b/dash/src/sml/address.rs @@ -40,7 +40,7 @@ impl Decodable for SocketAddr { let ipv6 = Ipv6Addr::from_bits(ip); - if let Some(ipv4) = ipv6.to_ipv4() { + if let Some(ipv4) = ipv6.to_ipv4_mapped() { Ok(SocketAddr::V4(SocketAddrV4::new(ipv4, port))) } else { Ok(SocketAddr::V6(SocketAddrV6::new(ipv6, port, 0, 0))) @@ -86,6 +86,20 @@ mod tests { assert_eq!(writer, decoded_writer); } + #[test] + fn encode_decode_unspecified_preserves_bytes() { + // An all-zero (`::`) address must round-trip to the same 16 zero bytes. Decoding it as + // IPv4 `0.0.0.0` would re-encode with the `::ffff:` mapped prefix and corrupt the bytes, + // which in turn breaks the masternode entry hash for entries with an unset service. + let original = [0u8; 18]; + let mut reader = &original[..]; + let decoded = SocketAddr::consensus_decode(&mut reader).unwrap(); + + let mut writer = Vec::new(); + decoded.consensus_encode(&mut writer).unwrap(); + assert_eq!(writer, original); + } + #[test] fn encode_decode_ipv6() { let address = SocketAddr::V6(SocketAddrV6::new( diff --git a/dash/src/sml/masternode_list_entry/hash.rs b/dash/src/sml/masternode_list_entry/hash.rs index 51f5932cf..dae58fd15 100644 --- a/dash/src/sml/masternode_list_entry/hash.rs +++ b/dash/src/sml/masternode_list_entry/hash.rs @@ -1,13 +1,58 @@ use hashes::{Hash, sha256d}; -use crate::consensus::Encodable; use crate::sml::masternode_list_entry::MasternodeListEntry; impl MasternodeListEntry { pub fn calculate_entry_hash(&self) -> sha256d::Hash { let mut writer = Vec::new(); - - self.consensus_encode(&mut writer).expect("encoding failed"); + self.consensus_encode_body(&mut writer).expect("encoding failed"); sha256d::Hash::hash(&writer) } } + +#[cfg(test)] +mod tests { + use hashes::Hash; + + use crate::consensus::deserialize; + use crate::network::message_sml::MnListDiff; + + // Ground-truth entry hashes produced by Dash Core's `CSimplifiedMNListEntry::CalcHash` + // (`CHashWriter(SER_GETHASH, ...)`) for the matching entries in this fixture. `SER_GETHASH` + // omits the `SER_NETWORK`-gated leading `version`, so the pre-image is the wire body without + // that field. Hashing the full wire (with `version`) yields different values and fails here. + // The first case is a `version` 1 entry, the second a `version` 2 Evo entry, exercising both + // the legacy path and the `nType`/platform fields. + #[test] + fn entry_hash_matches_core_calc_hash() { + let bytes: &[u8] = + include_bytes!("../../../tests/data/test_DML_diffs/mn_list_diff_0_2227096.bin"); + let diff: MnListDiff = deserialize(bytes).expect("expected to deserialize"); + + let cases = [ + ( + "0008858d870b0aa7967c39a551fc953e4e7fa602f19ba1fc805c218f87f41cb6", + "759c929f9d225554a09a8ad817bfaf555847547097495e08d3ba316529b65426", + ), + ( + "000c898c950a9c4a4d1eb3c227ab6d65ab652b44010e25f6dbe7a673e4bb52de", + "045c5f8ae528d32d0e694ddb9d652794d41b89db5f7eaee703beef62b35e4903", + ), + ]; + + for (pro_reg_tx_hash_hex, expected_entry_hash_hex) in cases { + let entry = diff + .new_masternodes + .iter() + .find(|e| hex::encode(e.pro_reg_tx_hash.to_byte_array()) == pro_reg_tx_hash_hex) + .expect("expected entry present in fixture"); + + assert_eq!( + hex::encode(entry.calculate_entry_hash().to_byte_array()), + expected_entry_hash_hex, + "entry hash for {} must match Dash Core's CalcHash", + pro_reg_tx_hash_hex + ); + } + } +} diff --git a/dash/src/sml/masternode_list_entry/mod.rs b/dash/src/sml/masternode_list_entry/mod.rs index 494205230..e26197fe6 100644 --- a/dash/src/sml/masternode_list_entry/mod.rs +++ b/dash/src/sml/masternode_list_entry/mod.rs @@ -153,15 +153,22 @@ impl PartialOrd for MasternodeListEntry { } } -impl Encodable for MasternodeListEntry { - fn consensus_encode(&self, writer: &mut W) -> Result { +impl MasternodeListEntry { + /// Encodes everything after the leading `version`, shared by the wire format and the hash + /// pre-image. Core's `CSimplifiedMNListEntry::CalcHash` uses `CHashWriter(SER_GETHASH, ...)`, + /// and `SER_GETHASH` does not set `SER_NETWORK`, so the leading `version` (which is + /// `SER_NETWORK`-gated in Core's `SERIALIZE_METHODS`) is excluded from the hash but present on + /// the wire. Every remaining field keys off the `version` member, identical in both contexts. + fn consensus_encode_body( + &self, + writer: &mut W, + ) -> Result { debug_assert_eq!( matches!(self.service_address, MasternodeNetInfo::Legacy(_)), self.version < 3, "Legacy service address must be used iff version < 3" ); let mut len = 0; - len += self.version.consensus_encode(writer)?; len += self.pro_reg_tx_hash.consensus_encode(writer)?; if let Some(confirmed_hash) = self.confirmed_hash { len += confirmed_hash.consensus_encode(writer)?; @@ -196,6 +203,15 @@ impl Encodable for MasternodeListEntry { } } +impl Encodable for MasternodeListEntry { + fn consensus_encode(&self, writer: &mut W) -> Result { + let mut len = 0; + len += self.version.consensus_encode(writer)?; + len += self.consensus_encode_body(writer)?; + Ok(len) + } +} + impl Decodable for MasternodeListEntry { fn consensus_decode(reader: &mut R) -> Result { let version: u16 = Decodable::consensus_decode(reader)?; From ddb59d91eb2a4af3abfc3f3960c69bb4d345bad6 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Fri, 26 Jun 2026 05:56:32 -0700 Subject: [PATCH 012/149] fix: release builds warning removed (#825) --- contrib/run_clippy.py | 16 ++++++++++++---- dash/src/consensus/serde.rs | 10 ++++++---- dash/src/sml/quorum_entry/validation.rs | 5 ++--- 3 files changed, 20 insertions(+), 11 deletions(-) diff --git a/contrib/run_clippy.py b/contrib/run_clippy.py index 054a61098..29b0b50c9 100755 --- a/contrib/run_clippy.py +++ b/contrib/run_clippy.py @@ -4,7 +4,7 @@ import subprocess import sys -cmd = [ +base = [ "cargo", "clippy", "--workspace", "--all-features", @@ -14,8 +14,16 @@ # Exclude dash-fuzz on Windows (honggfuzz doesn't support Windows) if sys.platform == "win32": - cmd.extend(["--exclude", "dash-fuzz"]) + base.extend(["--exclude", "dash-fuzz"]) -cmd.extend(["--", "-D", "warnings"]) +profiles = [ + ("debug", []), + ("release", ["--release"]), +] -sys.exit(subprocess.call(cmd)) +for label, extra in profiles: + cmd = base + extra + ["--", "-D", "warnings"] + rc = subprocess.call(cmd) + if rc != 0: + print(f"Clippy failed ({label}): {' '.join(cmd)}", file=sys.stderr) + sys.exit(rc) diff --git a/dash/src/consensus/serde.rs b/dash/src/consensus/serde.rs index 0f02d93f0..d1a1bc694 100644 --- a/dash/src/consensus/serde.rs +++ b/dash/src/consensus/serde.rs @@ -10,13 +10,12 @@ use core::fmt; use core::marker::PhantomData; -use serde::de::{SeqAccess, Unexpected, Visitor}; -use serde::ser::SerializeSeq; -use serde::{Deserializer, Serializer}; - use super::encode::Error as ConsensusError; use super::{Decodable, Encodable}; use crate::io; +use serde::de::{SeqAccess, Unexpected, Visitor}; +use serde::ser::SerializeSeq; +use serde::{Deserializer, Serializer}; /// Hex-encoding strategy pub struct Hex(PhantomData) @@ -194,6 +193,7 @@ impl<'a, T: 'a + Encodable, E: ByteEncoder> fmt::Display for DisplayWrapper<'a, ); } } + let _ = error; fmt::Error })?; let result = writer.actually_flush(); @@ -220,6 +220,7 @@ impl ErrorTrackingWriter { } #[track_caller] + #[cfg_attr(not(debug_assertions), allow(unused_variables))] fn assert_no_error(&self, fun: &str) { #[cfg(debug_assertions)] { @@ -238,6 +239,7 @@ impl ErrorTrackingWriter { } } + #[cfg_attr(not(debug_assertions), allow(unused_variables))] fn set_error(&mut self, was: bool) { #[cfg(debug_assertions)] { diff --git a/dash/src/sml/quorum_entry/validation.rs b/dash/src/sml/quorum_entry/validation.rs index aca96adfa..df8979e61 100644 --- a/dash/src/sml/quorum_entry/validation.rs +++ b/dash/src/sml/quorum_entry/validation.rs @@ -3,6 +3,7 @@ use crate::sml::quorum_entry::qualified_quorum_entry::QualifiedQuorumEntry; use crate::sml::quorum_validation_error::QuorumValidationError; use blsful::{Bls12381G2Impl, PublicKey, SerializationFormat, Signature}; use hashes::Hash; +use tracing::error; impl QualifiedQuorumEntry { /// Verifies the aggregated commitment signature for the quorum. @@ -50,9 +51,7 @@ impl QualifiedQuorumEntry { match result { Ok(public_key) => Some(public_key), Err(e) => { - // Log error in debug builds - #[cfg(debug_assertions)] - eprintln!("Failed to deserialize operator key: {:?}", e); + error!("Failed to deserialize operator key: {}", e); None } } From 03096dd03460784d4c3ee72e674127cbf32a6c3b Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Mon, 29 Jun 2026 03:24:42 -0700 Subject: [PATCH 013/149] feat(key-wallet): support for more account types + strategy to consume all utxo available (#823) --- .../tests/dashd_sync/tests_transaction.rs | 107 ++++++++++++++ key-wallet-ffi/src/transaction.rs | 6 +- key-wallet-manager/src/lib.rs | 7 +- .../managed_wallet_info/coin_selection.rs | 61 ++++++++ .../src/wallet/managed_wallet_info/mod.rs | 8 ++ .../transaction_builder.rs | 47 +++++- .../transaction_building.rs | 136 ++++++++++++++---- .../wallet_info_interface.rs | 25 ++++ 8 files changed, 363 insertions(+), 34 deletions(-) diff --git a/dash-spv/tests/dashd_sync/tests_transaction.rs b/dash-spv/tests/dashd_sync/tests_transaction.rs index 0b99f8a32..92f4931ff 100644 --- a/dash-spv/tests/dashd_sync/tests_transaction.rs +++ b/dash-spv/tests/dashd_sync/tests_transaction.rs @@ -17,13 +17,18 @@ use key_wallet::account::ManagedAccountTrait; use key_wallet::bip32::{ChildNumber, ExtendedPrivKey}; use key_wallet::gap_limit::DEFAULT_EXTERNAL_GAP_LIMIT; use key_wallet::mnemonic::{Language, Mnemonic}; +use key_wallet::wallet::initialization::WalletAccountCreationOptions; +use key_wallet::wallet::managed_wallet_info::coin_selection::SelectionStrategy; +use key_wallet::wallet::managed_wallet_info::fee::FeeRate; use key_wallet::wallet::managed_wallet_info::transaction_builder::{ BuilderError, TransactionBuilder, }; +use key_wallet::wallet::managed_wallet_info::transaction_building::AccountTypePreference; use key_wallet::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; use key_wallet::wallet::ManagedWalletInfo; use key_wallet::ManagedAccountType; use key_wallet_manager::{WalletId, WalletManager}; +use std::collections::BTreeSet; /// Verify incremental sync works by generating blocks after initial sync. /// @@ -464,3 +469,105 @@ async fn test_spend_incoming_balance() { client_handle.stop().await; } + +/// Drain every UTXO of one account into another account of the same wallet. +/// +/// Funds a BIP32 account with 3 UTXOs, then drains it into a BIP44 receive address with +/// `SelectionStrategy::All` (spend every UTXO; single output = total - fee, no change) +#[tokio::test] +async fn test_drain_account_into_another() { + let Some(ctx) = TestContext::new(TestChain::Minimal).await else { + return; + }; + if !ctx.dashd.supports_mining { + eprintln!("Skipping test (dashd RPC miner not available)"); + return; + } + + const FUNDING: u64 = 50_000_000; + const FEE: u64 = 488; + + // Fresh wallet with a BIP32 account 0 (drain source) and a BIP44 account 0 (destination). + // The default test options only create the BIP44 account. + let (wallet, wallet_id) = { + let mut manager = WalletManager::::new(Network::Regtest); + let id = manager + .create_wallet_from_mnemonic( + EMPTY_MNEMONIC, + 0, + WalletAccountCreationOptions::SpecificAccounts( + BTreeSet::from([0]), // BIP44 account 0 + BTreeSet::from([0]), // BIP32 account 0 + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + None, + ), + ) + .expect("create wallet with bip32 + bip44 accounts"); + (Arc::new(RwLock::new(manager)), id) + }; + + let mut client_handle = create_and_start_client(&ctx.client_config, Arc::clone(&wallet)).await; + wait_for_sync(&mut client_handle.progress_receiver, ctx.dashd.initial_height).await; + + let (fund_addresses, dest) = { + let mut lock = wallet.write().await; + let (w, info) = lock.get_wallet_and_info_mut(&wallet_id).expect("wallet"); + let fund: Vec
= (0..3) + .map(|_| info.next_receive_address(w, 0, AccountTypePreference::BIP32, true).unwrap()) + .collect(); + let dest = info.next_receive_address(w, 0, AccountTypePreference::BIP44, true).unwrap(); + (fund, dest) + }; + + let miner = ctx.dashd.node.get_new_address_from_wallet("default"); + for address in &fund_addresses { + ctx.dashd.node.send_to_address(address, Amount::from_sat(FUNDING)); + } + ctx.dashd.node.generate_blocks(1, &miner); + + let funded_height = ctx.dashd.initial_height + 1; + wait_for_sync(&mut client_handle.progress_receiver, funded_height).await; + wait_for_wallet_synced(&wallet, &wallet_id, funded_height).await; + + // Drain the BIP32 account into the BIP44 destination (amount ignored under `All`). + let (tx, _) = { + let mut lock = wallet.write().await; + + let (w, info) = lock.get_wallet_and_info_mut(&wallet_id).expect("wallet"); + info.build_and_sign_transaction( + w, + AccountTypePreference::BIP32, + 0, + vec![(dest.as_unchecked().clone(), 0)], + FeeRate::normal(), + SelectionStrategy::All, + ) + .await + .expect("drain") + }; + + client_handle.client.broadcast_transaction(&tx).await.expect("broadcast"); + wait_for_mempool_tx(&mut client_handle.wallet_event_receiver, MEMPOOL_TIMEOUT) + .await + .expect("mempool"); + ctx.dashd.node.generate_blocks(1, &miner); + + let drained_height = funded_height + 1; + wait_for_sync(&mut client_handle.progress_receiver, drained_height).await; + wait_for_wallet_synced(&wallet, &wallet_id, drained_height).await; + + // Final state: BIP32 emptied, BIP44 holds the single drained UTXO. + let reader = wallet.read().await; + let info = reader.get_wallet_info(&wallet_id).expect("wallet"); + let balance = |pref| info.account_balance(pref, 0).unwrap().total(); + let utxos = |pref| info.funds_account(pref, 0).unwrap().utxos.len(); + + assert_eq!(balance(AccountTypePreference::BIP32), 0); + assert_eq!(utxos(AccountTypePreference::BIP32), 0); + assert_eq!(balance(AccountTypePreference::BIP44), FUNDING * 3 - FEE); + assert_eq!(utxos(AccountTypePreference::BIP44), 1); + + client_handle.stop().await; +} diff --git a/key-wallet-ffi/src/transaction.rs b/key-wallet-ffi/src/transaction.rs index 03b0f3bc4..7878506bd 100644 --- a/key-wallet-ffi/src/transaction.rs +++ b/key-wallet-ffi/src/transaction.rs @@ -14,7 +14,9 @@ use dashcore::{ use key_wallet::wallet::managed_wallet_info::asset_lock_builder::{ AssetLockFundingType, CreditOutputFunding, }; +use key_wallet::wallet::managed_wallet_info::coin_selection::SelectionStrategy::BranchAndBound; use key_wallet::wallet::managed_wallet_info::fee::FeeRate; +use key_wallet::wallet::managed_wallet_info::transaction_building::AccountTypePreference; use secp256k1::{Message, Secp256k1, SecretKey}; use std::ffi::{CStr, CString}; use std::os::raw::c_char; @@ -135,9 +137,11 @@ pub unsafe extern "C" fn wallet_build_and_sign_transaction( manager .build_and_sign_transaction( &wallet_id, + AccountTypePreference::BIP44, account_index, outputs, - FeeRate::new(fee_per_kb) + FeeRate::new(fee_per_kb), + BranchAndBound ) .await, error diff --git a/key-wallet-manager/src/lib.rs b/key-wallet-manager/src/lib.rs index 3d0df8aad..874d39020 100644 --- a/key-wallet-manager/src/lib.rs +++ b/key-wallet-manager/src/lib.rs @@ -29,6 +29,7 @@ use dashcore::prelude::CoreBlockHeight; use key_wallet::account::AccountCollection; use key_wallet::managed_account::transaction_record::TransactionRecord; use key_wallet::transaction_checking::{DerivedAddressInfo, TransactionContext}; +use key_wallet::wallet::managed_wallet_info::coin_selection::SelectionStrategy; use key_wallet::wallet::managed_wallet_info::transaction_building::AccountTypePreference; use key_wallet::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; @@ -578,9 +579,11 @@ impl WalletManager { pub async fn build_and_sign_transaction( &mut self, wallet_id: &WalletId, - account_index: u32, + source: AccountTypePreference, + source_index: u32, outputs: Vec<(Address, u64)>, fee_rate: FeeRate, + strategy: SelectionStrategy, ) -> Result<(Transaction, u64), WalletError> { // Get the managed account for UTXOs and signing data let (wallet, managed_wallet) = self @@ -588,7 +591,7 @@ impl WalletManager { .ok_or(WalletError::WalletNotFound(*wallet_id))?; managed_wallet - .build_and_sign_transaction(wallet, account_index, outputs, fee_rate) + .build_and_sign_transaction(wallet, source, source_index, outputs, fee_rate, strategy) .await .map_err(|e| WalletError::TransactionBuild(e.to_string())) } diff --git a/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs b/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs index 523d1eb49..5bbd7d53a 100644 --- a/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs +++ b/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs @@ -36,6 +36,9 @@ pub enum SelectionStrategy { OptimalConsolidation, /// Random selection for privacy Random, + /// Select EVERY spendable UTXO (drain / sweep an account empty). There is no change: the + /// single deliverable output is worth the total minus the fee to spend them all + All, } /// Result of UTXO selection @@ -249,6 +252,38 @@ impl CoinSelector { input_size, ) } + SelectionStrategy::All => { + let selected: Vec = + utxos.into_iter().filter(|u| u.is_spendable(current_height)).cloned().collect(); + + if selected.is_empty() { + return Err(SelectionError::NoUtxosAvailable); + } + + let total_value: u64 = selected.iter().map(|u| u.value()).sum(); + let estimated_size = base_size + selected.len() * input_size; + let estimated_fee = fee_rate.calculate_fee(estimated_size); + + // The caller's `target_amount` is ignored: a drain spends everything, there is no + // target to satisfy + let deliverable = total_value + .checked_sub(estimated_fee) + .filter(|d| *d > self.dust_threshold) + .ok_or(SelectionError::InsufficientFunds { + available: total_value, + required: estimated_fee + self.dust_threshold + 1, + })?; + + Ok(SelectionResult { + selected, + total_value, + target_amount: deliverable, + change_amount: 0, + estimated_size, + estimated_fee, + exact_match: true, + }) + } } } @@ -660,6 +695,32 @@ impl std::error::Error for SelectionError {} mod tests { use super::*; + #[test] + fn test_select_all_drains_everything() { + let utxos = vec![ + Utxo::dummy(0, 10000, 100, false, true), + Utxo::dummy(0, 20000, 100, false, true), + Utxo::dummy(0, 30000, 100, false, true), + ]; + let selector = CoinSelector::new(SelectionStrategy::All); + // Pass a non-zero target to prove it is IGNORED: a drain takes everything regardless. + let result = selector.select_coins(&utxos, 12_345, FeeRate::new(1000), 200).unwrap(); + + assert_eq!(result.selected.len(), 3, "All selects every spendable UTXO"); + assert_eq!(result.total_value, 60000); + assert!(result.estimated_fee > 0); + assert_eq!(result.target_amount, 60000 - result.estimated_fee, "deliverable = total - fee"); + assert_eq!(result.change_amount, 0, "a drain leaves no change"); + assert!(result.exact_match, "no change output for a drain"); + } + + #[test] + fn test_select_all_empty_is_error() { + let selector = CoinSelector::new(SelectionStrategy::All); + let result = selector.select_coins(&[], 0, FeeRate::new(1000), 200); + assert!(matches!(result, Err(SelectionError::NoUtxosAvailable))); + } + #[test] fn test_smallest_first_selection() { let utxos = vec![ diff --git a/key-wallet/src/wallet/managed_wallet_info/mod.rs b/key-wallet/src/wallet/managed_wallet_info/mod.rs index d062160b7..62b3d9e71 100644 --- a/key-wallet/src/wallet/managed_wallet_info/mod.rs +++ b/key-wallet/src/wallet/managed_wallet_info/mod.rs @@ -174,6 +174,10 @@ impl ManagedWalletInfo { address } + AccountTypePreference::CoinJoin => { + debug_assert!(false, "CoinJoin accounts are spend-only in our current use cases"); + None + } }; address @@ -217,6 +221,10 @@ impl ManagedWalletInfo { address } + AccountTypePreference::CoinJoin => { + debug_assert!(false, "CoinJoin accounts are spend-only in our current use cases"); + None + } }; address diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index b87a0a913..d86ea7649 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -20,6 +20,10 @@ use secp256k1::ecdsa::Signature; use secp256k1::{Message, PublicKey, Secp256k1}; use std::cmp::Ordering; +/// A transaction with more inputs would exceed the relay standard-size cap (~100 KB at ~148 +/// bytes/signed input) and be rejected by the network +const MAX_STANDARD_TX_INPUTS: usize = 500; + /// Calculate varint size for a given number fn varint_size(n: usize) -> usize { match n { @@ -240,7 +244,7 @@ impl TransactionBuilder { size } - fn assemble_unsigned(self) -> Result<(Transaction, Vec), BuilderError> { + fn assemble_unsigned(mut self) -> Result<(Transaction, Vec), BuilderError> { if let Some(TransactionPayload::AssetLockPayloadType(p)) = &self.special_payload { if p.credit_outputs.is_empty() { return Err(BuilderError::NoOutputs); @@ -249,6 +253,12 @@ impl TransactionBuilder { return Err(BuilderError::NoOutputs); } + // A drain (`All`) never emits change; drop the change address before sizing so the fee + // estimate doesn't include a phantom (~34-byte) change output. + if self.selection_strategy == SelectionStrategy::All { + self.change_addr = None; + } + // For AssetLock the on-chain spend equals the OP_RETURN burn, which // mirrors the sum of the credit_outputs carried in the payload. For // every other tx type, it's just the sum of user-provided outputs. @@ -271,6 +281,14 @@ impl TransactionBuilder { .map_err(BuilderError::CoinSelection)?; let mut selected_inputs = selection.selected; + + if selected_inputs.len() > MAX_STANDARD_TX_INPUTS { + return Err(BuilderError::TooManyInputs { + count: selected_inputs.len(), + max: MAX_STANDARD_TX_INPUTS, + }); + } + let total_input: u64 = selected_inputs.iter().map(|u| u.value()).sum(); if total_input < total_output + selection.estimated_fee { @@ -290,8 +308,17 @@ impl TransactionBuilder { _ => self.outputs, }; - // Add change output if above dust threshold - if change_amount > 546 { + if self.selection_strategy == SelectionStrategy::All { + // Drain: the single output takes the whole balance minus fee (the caller's amount is + // ignored); no change. + let [out] = tx_outputs.as_mut_slice() else { + return Err(BuilderError::InvalidData( + "SelectionStrategy::All requires exactly one output (the destination)".into(), + )); + }; + out.value = total_input.saturating_sub(selection.estimated_fee); + } else if change_amount > 546 { + // Add change output if above dust threshold let Some(change_addr) = self.change_addr else { return Err(BuilderError::NoChangeAddress); }; @@ -506,6 +533,8 @@ pub enum BuilderError { NoOutputs, /// No change address provided NoChangeAddress, + /// The requested funding account does not exist + AccountNotFound(String), /// Insufficient funds InsufficientFunds { available: u64, @@ -521,6 +550,11 @@ pub enum BuilderError { CoinSelection(crate::wallet::managed_wallet_info::coin_selection::SelectionError), /// Signing was attempted with a watch-only wallet WatchOnlyWallet, + /// More inputs than fit in a single standard transaction + TooManyInputs { + count: usize, + max: usize, + }, } impl fmt::Display for BuilderError { @@ -529,6 +563,7 @@ impl fmt::Display for BuilderError { Self::NoInputs => write!(f, "No inputs provided"), Self::NoOutputs => write!(f, "No outputs provided"), Self::NoChangeAddress => write!(f, "No change address provided"), + Self::AccountNotFound(msg) => write!(f, "Account not found: {msg}"), Self::InsufficientFunds { available, required, @@ -540,6 +575,12 @@ impl fmt::Display for BuilderError { Self::SigningFailed(msg) => write!(f, "Signing failed: {}", msg), Self::CoinSelection(err) => write!(f, "Coin selection error: {}", err), Self::WatchOnlyWallet => write!(f, "Cannot sign with a watch-only wallet"), + Self::TooManyInputs { + count, + max, + } => { + write!(f, "Too many inputs for a standard transaction: {count} (max {max})") + } } } } diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index c553c59dc..1b1833b05 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -2,6 +2,7 @@ use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::signer::Signer; +use crate::wallet::managed_wallet_info::coin_selection::SelectionStrategy; use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::wallet::managed_wallet_info::transaction_builder::{BuilderError, TransactionBuilder}; use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; @@ -15,32 +16,47 @@ use dashcore::{Address, Transaction}; pub enum AccountTypePreference { BIP44, BIP32, + CoinJoin, } impl ManagedWalletInfo { pub async fn build_and_sign_transaction( &mut self, wallet: &Wallet, - account_index: u32, + source: AccountTypePreference, + source_index: u32, outputs: Vec<(Address, u64)>, fee_rate: FeeRate, + strategy: SelectionStrategy, ) -> Result<(Transaction, u64), BuilderError> { let height = self.last_processed_height(); - let managed_account = self - .accounts - .standard_bip44_accounts - .get_mut(&account_index) - .ok_or(BuilderError::NoChangeAddress)?; - - let account = wallet - .accounts - .standard_bip44_accounts - .get(&account_index) - .ok_or(BuilderError::NoChangeAddress)?; + let managed_account = match source { + AccountTypePreference::BIP44 => { + self.accounts.standard_bip44_accounts.get_mut(&source_index) + } + AccountTypePreference::BIP32 => { + self.accounts.standard_bip32_accounts.get_mut(&source_index) + } + AccountTypePreference::CoinJoin => { + self.accounts.coinjoin_accounts.get_mut(&source_index) + } + } + .ok_or_else(|| { + BuilderError::AccountNotFound(format!("managed account {source:?} #{source_index}")) + })?; + let account = match source { + AccountTypePreference::BIP44 => wallet.get_bip44_account(source_index), + AccountTypePreference::BIP32 => wallet.get_bip32_account(source_index), + AccountTypePreference::CoinJoin => wallet.get_coinjoin_account(source_index), + } + .ok_or_else(|| { + BuilderError::AccountNotFound(format!("wallet account {source:?} #{source_index}")) + })?; let mut tx_builder = TransactionBuilder::new() .set_fee_rate(fee_rate) + .set_selection_strategy(strategy) .set_current_height(height) .set_funding(managed_account, account); @@ -54,30 +70,45 @@ impl ManagedWalletInfo { tx_builder.build_signed(wallet, |addr| managed_account.address_derivation_path(&addr)).await } + #[allow(clippy::too_many_arguments)] pub async fn build_and_sign_transaction_with_signer( &mut self, wallet: &Wallet, - account_index: u32, + source: AccountTypePreference, + source_index: u32, outputs: Vec<(Address, u64)>, fee_rate: FeeRate, + strategy: SelectionStrategy, signer: &S, ) -> Result<(Transaction, u64), BuilderError> { let height = self.last_processed_height(); - let managed_account = self - .accounts - .standard_bip44_accounts - .get_mut(&account_index) - .ok_or(BuilderError::NoChangeAddress)?; - - let account = wallet - .accounts - .standard_bip44_accounts - .get(&account_index) - .ok_or(BuilderError::NoChangeAddress)?; + let managed_account = match source { + AccountTypePreference::BIP44 => { + self.accounts.standard_bip44_accounts.get_mut(&source_index) + } + AccountTypePreference::BIP32 => { + self.accounts.standard_bip32_accounts.get_mut(&source_index) + } + AccountTypePreference::CoinJoin => { + self.accounts.coinjoin_accounts.get_mut(&source_index) + } + } + .ok_or_else(|| { + BuilderError::AccountNotFound(format!("managed account {source:?} #{source_index}")) + })?; + let account = match source { + AccountTypePreference::BIP44 => wallet.get_bip44_account(source_index), + AccountTypePreference::BIP32 => wallet.get_bip32_account(source_index), + AccountTypePreference::CoinJoin => wallet.get_coinjoin_account(source_index), + } + .ok_or_else(|| { + BuilderError::AccountNotFound(format!("wallet account {source:?} #{source_index}")) + })?; let mut tx_builder = TransactionBuilder::new() .set_fee_rate(fee_rate) + .set_selection_strategy(strategy) .set_current_height(height) .set_funding(managed_account, account); @@ -93,6 +124,7 @@ impl ManagedWalletInfo { } #[cfg(test)] mod tests { + use super::AccountTypePreference; use crate::wallet::managed_wallet_info::coin_selection::SelectionStrategy; use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::wallet::managed_wallet_info::transaction_builder::TransactionBuilder; @@ -143,6 +175,37 @@ mod tests { assert!(change_output.is_some(), "Should have change output"); } + #[test] + fn test_sweep_builder_drains_to_single_output() { + let utxos = vec![ + Utxo::dummy(0, 100000, 100, false, true), + Utxo::dummy(0, 200000, 100, false, true), + Utxo::dummy(0, 300000, 100, false, true), + ]; + let dest = Address::from_str("yTb47qEBpNmgXvYYsHEN4nh8yJwa5iC4Cs") + .unwrap() + .require_network(Network::Testnet) + .unwrap(); + + // `All` selects every input and pays total - fee to `dest` as one output, no change + let total = 600_000u64; + let fee = FeeRate::normal().calculate_fee(8 + 1 + 1 + 34 + 3 * 148); + let deliverable = total - fee; + let (tx, _fee) = TransactionBuilder::new() + .set_fee_rate(FeeRate::normal()) + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::All) + .add_inputs(utxos) + .add_output(&dest, deliverable) + .build_unsigned() + .unwrap(); + + assert_eq!(tx.input.len(), 3, "sweep spends every input"); + assert_eq!(tx.output.len(), 1, "sweep has one real output and no change"); + assert_eq!(tx.output[0].value, deliverable); + assert_eq!(tx.output[0].script_pubkey, dest.script_pubkey()); + } + #[test] fn test_asset_lock_transaction() { // Test based on DSTransactionTests.m testAssetLockTx1 @@ -437,8 +500,8 @@ mod tests { #[tokio::test] async fn test_signer_invalid_account_index() { - // No BIP44 account 99 exists, so next_change_address returns None - // and we surface NoChangeAddress before any signing happens. + // No BIP44 account 99 exists, so account resolution fails with AccountNotFound + // before any funding or signing happens. let (wallet, mut info) = test_wallet_and_info(); let signer = InMemorySigner { root: root_from(&wallet), @@ -447,13 +510,16 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, + AccountTypePreference::BIP44, 99, dest_outputs(100_000), FeeRate::normal(), + SelectionStrategy::BranchAndBound, &signer, ) .await; - assert!(matches!(result, Err(BuilderError::NoChangeAddress))); + + assert!(matches!(result, Err(BuilderError::AccountNotFound(_)))); } #[tokio::test] @@ -489,9 +555,11 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, + AccountTypePreference::BIP44, 0, dest_outputs(100_000), FeeRate::normal(), + SelectionStrategy::BranchAndBound, &NoDigestSigner, ) .await; @@ -553,9 +621,11 @@ mod tests { let (tx, fee) = info .build_and_sign_transaction_with_signer( &wallet, + AccountTypePreference::BIP44, 0, dest_outputs(send_amount), FeeRate::normal(), + SelectionStrategy::BranchAndBound, &signer, ) .await @@ -592,9 +662,11 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, + AccountTypePreference::BIP44, 0, dest_outputs(500_000), FeeRate::normal(), + SelectionStrategy::BranchAndBound, &signer, ) .await; @@ -639,7 +711,15 @@ mod tests { let outputs = vec![(mainnet_dest, 100_000u64)]; let result = info - .build_and_sign_transaction_with_signer(&wallet, 0, outputs, FeeRate::normal(), &signer) + .build_and_sign_transaction_with_signer( + &wallet, + AccountTypePreference::BIP44, + 0, + outputs, + FeeRate::normal(), + SelectionStrategy::BranchAndBound, + &signer, + ) .await; assert!( matches!(result, Err(BuilderError::InvalidData(_))), diff --git a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs index da54ea329..2b3ea3fbe 100644 --- a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs +++ b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs @@ -8,8 +8,10 @@ use super::managed_account_operations::ManagedAccountOperations; use crate::account::{AccountType, ManagedAccountTrait}; use crate::managed_account::managed_account_collection::ManagedAccountCollection; use crate::managed_account::managed_account_ref::ManagedAccountRefMut; +use crate::managed_account::ManagedCoreFundsAccount; use crate::transaction_checking::TransactionContext; use crate::transaction_checking::WalletTransactionChecker; +use crate::wallet::managed_wallet_info::transaction_building::AccountTypePreference; use crate::wallet::managed_wallet_info::TransactionRecord; use crate::wallet::ManagedWalletInfo; use crate::{Network, Utxo, Wallet, WalletCoreBalance}; @@ -112,6 +114,29 @@ pub trait WalletInfoInterface: Sized + WalletTransactionChecker + ManagedAccount .collect() } + /// The funds-bearing account selected by `(preference, index)`, if it exists. + fn funds_account( + &self, + preference: AccountTypePreference, + index: u32, + ) -> Option<&ManagedCoreFundsAccount> { + let accounts = self.accounts(); + match preference { + AccountTypePreference::BIP44 => accounts.standard_bip44_accounts.get(&index), + AccountTypePreference::BIP32 => accounts.standard_bip32_accounts.get(&index), + AccountTypePreference::CoinJoin => accounts.coinjoin_accounts.get(&index), + } + } + + /// Balance of the funds-bearing account selected by `(preference, index)`, if it exists. + fn account_balance( + &self, + preference: AccountTypePreference, + index: u32, + ) -> Option { + self.funds_account(preference, index).map(|account| account.balance) + } + /// Get transaction history fn transaction_history(&self) -> Vec<&TransactionRecord>; From 6adc753c8c84d596eedccdca30fe67f755382d05 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Wed, 1 Jul 2026 02:05:01 +1000 Subject: [PATCH 014/149] docs: add `0.43.0` and `0.44.0` changelog entries (#830) * docs: add `0.43.0` changelog entry * docs: add `0.44.0` changelog entry --- CHANGELOG.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f15ea0e00..a9dd317ca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,43 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +## 0.44.0 - 2026-07-01 + +### Added + +- Storage truncation primitives for reorg support in `dash-spv` (#790) @xdustinface +- Decode SML ProTx v3 entries (#797) @xdustinface +- `normalize_phrase` and `is_word_in_language` helpers in `key-wallet` (#806) @llbartekll + +### Changed + +- **Breaking:** remove `Address::network()` and replace `Network` with `AddressPrefix` for address prefix handling (#802) @ZocoLini + +### Fixed + +- Write the compact-size count in `QuorumSnapshot` encode (#796) @xdustinface +- Correct CoinJoin discovery in `key-wallet` (#804) @xdustinface + +## 0.43.0 - 2026-06-01 + +### Added + +- Devnet support for `dash-spv` (#784) @xdustinface +- Consolidate devnet config into a `DevnetConfig` struct (#788) @xdustinface +- `downgrade_to_external_signable` method on wallets (#782) @ZocoLini + +### Changed + +- **Breaking:** network-scoped wallet-id derivation, the same seed now yields different wallet IDs per network (#793) @QuantumExplorer +- Derive `Zeroize` on `WalletType` and wipe key material on `drop` (#780) @ZocoLini +- Avoid rebuilding `script_pubkey`s for BIP158 filter matching (#781) @xdustinface +- Migrate from `vX.Y-dev` branching model to a single `dev` branch (#778) @xdustinface +- Bump workspace version to `0.43.0` (#779) @xdustinface + +### Fixed + +- Handle buffered sync events during `WaitingForConnections` in `dash-spv` (#768) @xdustinface + ## 0.42.0 - 2026-05-15 ### Highlights From 78d10022c92384affe9d71be3266ac7bb4710291 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Wed, 1 Jul 2026 02:05:20 +1000 Subject: [PATCH 015/149] chore: bump version to `0.45.0` (#829) --- Cargo.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Cargo.toml b/Cargo.toml index 8677d9cad..f885763f1 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ members = ["dash", "dash-network", "hashes", "internals", "fuzz", "rpc-client", resolver = "2" [workspace.package] -version = "0.43.0" +version = "0.45.0" [patch.crates-io] dashcore_hashes = { path = "hashes" } From d7caf4ef1a289cd9ef187d1dc02c22ad3fee5ac7 Mon Sep 17 00:00:00 2001 From: lklimek <842586+lklimek@users.noreply.github.com> Date: Thu, 2 Jul 2026 03:13:08 +0200 Subject: [PATCH 016/149] feat(key-wallet): impl Zeroize for ExtendedPrivKey (#833) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet): impl Zeroize for ExtendedPrivKey ExtendedPrivKey held its secp256k1::SecretKey (and ChainCode) with no Zeroize impl, so downstream code could not wrap it in zeroize::Zeroizing and had to scrub the scalar manually with SecretKey::non_secure_erase() on every path (error-prone; the scalar leaks un-wiped on `?`-early-return and panic-unwind paths). Add a hand-written `impl Zeroize for ExtendedPrivKey` mirroring the existing `RootExtendedPrivKey` and `ChainCode` impls. It is hand-written rather than `#[derive(Zeroize)]` because secp256k1::SecretKey exposes `non_secure_erase()` instead of a `Zeroize` impl, so a derive does not compile. The type stays `Copy` (no `Drop`), so this is additive and non-breaking; callers opt in via `zeroize::Zeroizing`. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01MoY6vhmqZuHzNsMfJ8wakQ 🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent * feat(key-wallet): zeroize all ExtendedPrivKey fields; add Default to Network Expand `impl Zeroize for ExtendedPrivKey` to wipe the whole value rather than only the secret material. In addition to `private_key` (non_secure_erase) and `chain_code`, it now clears `depth` and `parent_fingerprint`, resets `child_number` to `Normal { index: 0 }`, and resets `network` to its default — matching how `RootExtendedPrivKey` zeroes all of its own fields. `dashcore::Network` had no `Default`, so add `#[derive(Default)]` to it with `Mainnet` (the `#[repr(u8)]` 0 discriminant) as the default. Additive and non-breaking. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01MoY6vhmqZuHzNsMfJ8wakQ 🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent * refactor(key-wallet): reset network via Network::Mainnet in zeroize; drop Network Default Revert the `#[derive(Default)]` added to `dashcore::Network` and instead reset `network` to `Network::Mainnet` directly in `ExtendedPrivKey::zeroize` — Mainnet is the `#[repr(u8)]` 0 discriminant, i.e. the natural "zero" value. Keeps the change confined to key-wallet with no cross-crate Default impl. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01MoY6vhmqZuHzNsMfJ8wakQ 🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent * fix(key-wallet)!: drop Copy from ExtendedPrivKey, zeroize on Drop Removing Copy lets ExtendedPrivKey implement Drop, so its secret material is wiped automatically on scope exit instead of relying on callers to remember to zeroize. Mirrors the RootExtendedPrivKey convention. BREAKING CHANGE: ExtendedPrivKey no longer implements Copy; downstream code relying on implicit copies must clone or borrow. Fallout fixed: - bip32::derive_priv clones self instead of *self. - taproot-psbt example takes &ExtendedPrivKey (borrow, no clone). - key-wallet tests reorder/capture-or-clone where the value is reused after move. Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Claude Opus 4.8 --- dash/examples/taproot-psbt.rs | 6 +++--- key-wallet/src/bip32.rs | 30 +++++++++++++++++++++++++-- key-wallet/src/tests/account_tests.rs | 5 +++-- key-wallet/src/tests/wallet_tests.rs | 3 ++- 4 files changed, 36 insertions(+), 8 deletions(-) diff --git a/dash/examples/taproot-psbt.rs b/dash/examples/taproot-psbt.rs index f94c5ac83..6485fd5bb 100644 --- a/dash/examples/taproot-psbt.rs +++ b/dash/examples/taproot-psbt.rs @@ -118,7 +118,7 @@ fn main() -> Result<(), Box> { let tx_hex_string = encode::serialize_hex(&generate_bip86_key_spend_tx( &secp, // The master extended private key from the descriptor in step 4 - ExtendedPrivKey::from_str(BENEFACTOR_XPRIV_STR)?, + &ExtendedPrivKey::from_str(BENEFACTOR_XPRIV_STR)?, // Set these fields with valid data for the UTXO from step 5 above UTXO_1, vec![ @@ -231,7 +231,7 @@ struct P2trUtxo<'a> { fn generate_bip86_key_spend_tx( secp: &secp256k1::Secp256k1, - master_xpriv: ExtendedPrivKey, + master_xpriv: &ExtendedPrivKey, input_utxo: P2trUtxo, outputs: Vec, ) -> Result> { @@ -430,7 +430,7 @@ impl BenefactorWallet { // Spend a normal BIP86-like output as an input in our inheritance funding transaction let tx = generate_bip86_key_spend_tx( &self.secp, - self.master_xpriv, + &self.master_xpriv, input_utxo, vec![TxOut { script_pubkey: script_pubkey.clone(), diff --git a/key-wallet/src/bip32.rs b/key-wallet/src/bip32.rs index 8933b19e4..041794a1d 100644 --- a/key-wallet/src/bip32.rs +++ b/key-wallet/src/bip32.rs @@ -45,6 +45,7 @@ use base58ck; #[cfg(feature = "bincode")] use bincode_derive::{Decode, Encode}; use dashcore::Network; +use zeroize::Zeroize; /// XpubIdentifier as a hash160 result type XpubIdentifier = hash160::Hash; @@ -340,7 +341,7 @@ impl<'de> serde::Deserialize<'de> for Fingerprint { } /// Extended private key -#[derive(Copy, Clone, PartialEq, Eq)] +#[derive(Clone, PartialEq, Eq)] pub struct ExtendedPrivKey { /// The network this key is to be used on pub network: Network, @@ -356,6 +357,31 @@ pub struct ExtendedPrivKey { pub chain_code: ChainCode, } +// Hand-written (not `#[derive(Zeroize)]`): `secp256k1::SecretKey` has no +// `Zeroize` impl, only `non_secure_erase()`, so a derive would not compile. +// `Drop` (below) calls this, so the value is wiped automatically on scope exit +// with no caller action required. Cf. `RootExtendedPrivKey`. +impl zeroize::Zeroize for ExtendedPrivKey { + fn zeroize(&mut self) { + // Secret key material. + self.private_key.non_secure_erase(); + self.chain_code.zeroize(); + // Derivation metadata — cleared too so the whole value is wiped. + self.depth.zeroize(); + self.parent_fingerprint.0.zeroize(); + self.child_number = ChildNumber::Normal { + index: 0, + }; + self.network = Network::Mainnet; // repr(u8)=0 discriminant, the "zero" value + } +} + +impl Drop for ExtendedPrivKey { + fn drop(&mut self) { + self.zeroize(); + } +} + #[cfg(feature = "bincode")] impl bincode::Encode for ExtendedPrivKey { fn encode( @@ -1522,7 +1548,7 @@ impl ExtendedPrivKey { secp: &Secp256k1, path: &P, ) -> Result { - let mut sk: ExtendedPrivKey = *self; + let mut sk: ExtendedPrivKey = self.clone(); for cnum in path.as_ref() { sk = sk.ckd_priv(secp, *cnum)?; } diff --git a/key-wallet/src/tests/account_tests.rs b/key-wallet/src/tests/account_tests.rs index f788c46ec..585e35fb0 100644 --- a/key-wallet/src/tests/account_tests.rs +++ b/key-wallet/src/tests/account_tests.rs @@ -342,11 +342,11 @@ fn test_account_extended_key_generation() { let derivation_path = account_type.derivation_path(network).unwrap(); let account_key = master.derive_priv(&secp, &derivation_path).unwrap(); + let expected_xpub = ExtendedPubKey::from_priv(&secp, &account_key); let account = Account::from_xpriv(Some([0u8; 32]), account_type, account_key, network).unwrap(); // Verify extended public key can be derived let xpub = account.extended_public_key(); - let expected_xpub = ExtendedPubKey::from_priv(&secp, &account_key); assert_eq!(xpub, expected_xpub); // Verify the account can be created as watch-only @@ -400,7 +400,8 @@ fn test_account_network_consistency() { let derivation_path = account_type.derivation_path(network).unwrap(); let account_key = master.derive_priv(&secp, &derivation_path).unwrap(); - let account = Account::from_xpriv(Some([0u8; 32]), account_type, account_key, network).unwrap(); + let account = + Account::from_xpriv(Some([0u8; 32]), account_type, account_key.clone(), network).unwrap(); // Verify account stores the correct network assert_eq!(account.network, network); diff --git a/key-wallet/src/tests/wallet_tests.rs b/key-wallet/src/tests/wallet_tests.rs index 0b525aebd..420f1500d 100644 --- a/key-wallet/src/tests/wallet_tests.rs +++ b/key-wallet/src/tests/wallet_tests.rs @@ -97,6 +97,7 @@ fn test_wallet_creation_from_extended_key() { let seed = mnemonic.to_seed(""); let root_key = RootExtendedPrivKey::new_master(&seed).unwrap(); let master_key = root_key.to_extended_priv_key(Network::Testnet); + let master_private_key = master_key.private_key; let wallet = Wallet::from_extended_key( master_key, @@ -113,7 +114,7 @@ fn test_wallet_creation_from_extended_key() { // Verify extended key is stored match &wallet.wallet_type { WalletType::ExtendedPrivKey(wallet_key) => { - assert_eq!(wallet_key.root_private_key, master_key.private_key); + assert_eq!(wallet_key.root_private_key, master_private_key); } _ => panic!("Expected extended private key wallet type"), } From 9cb803742c9a9fc3bc1b2551b209edb14fafb39d Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Wed, 1 Jul 2026 19:12:54 -0700 Subject: [PATCH 017/149] refactor(key-wallet): ManagedAccountType::from_account_type refactor (#828) * refactor(key-wallet): ManagedAccountType::from_account_type refactor * refactor(key-wallet): ManagedAccountType::from_account_type refactor * refactor(key-wallet): ManagedAccountType::from_account_type refactor --- .../managed_account_collection.rs | 305 ++---------------- .../managed_account/managed_account_type.rs | 203 +++++------- 2 files changed, 114 insertions(+), 394 deletions(-) diff --git a/key-wallet/src/managed_account/managed_account_collection.rs b/key-wallet/src/managed_account/managed_account_collection.rs index 4c6c85ce1..b79107c31 100644 --- a/key-wallet/src/managed_account/managed_account_collection.rs +++ b/key-wallet/src/managed_account/managed_account_collection.rs @@ -6,12 +6,8 @@ use std::collections::BTreeMap; use crate::account::account_collection::{DashpayAccountKey, PlatformPaymentAccountKey}; -use crate::account::account_type::AccountType; -use crate::gap_limit::{ - DEFAULT_COINJOIN_GAP_LIMIT, DEFAULT_EXTERNAL_GAP_LIMIT, DEFAULT_INTERNAL_GAP_LIMIT, - DEFAULT_SPECIAL_GAP_LIMIT, DIP17_GAP_LIMIT, -}; -use crate::managed_account::address_pool::{AddressPool, AddressPoolType}; +use crate::gap_limit::DIP17_GAP_LIMIT; +use crate::managed_account::address_pool::AddressPoolType; use crate::managed_account::managed_account_ref::{ ManagedAccountRef, ManagedAccountRefMut, OwnedManagedCoreAccount, }; @@ -20,8 +16,8 @@ use crate::managed_account::managed_account_type::ManagedAccountType; use crate::managed_account::managed_platform_account::ManagedPlatformAccount; use crate::managed_account::{ManagedCoreFundsAccount, ManagedCoreKeysAccount}; use crate::transaction_checking::account_checker::CoreAccountTypeMatch; +use crate::KeySource; use crate::{Account, AccountCollection}; -use crate::{KeySource, Network}; #[cfg(feature = "serde")] use serde::{Deserialize, Serialize}; @@ -489,8 +485,11 @@ impl ManagedAccountCollection { account: &Account, ) -> Result { let key_source = KeySource::Public(account.account_xpub); - let managed_type = - Self::build_managed_account_type(account.account_type, account.network, &key_source)?; + let managed_type = ManagedAccountType::from_account_type( + account.account_type, + account.network, + &key_source, + )?; Ok(ManagedCoreFundsAccount::new(managed_type, account.network)) } @@ -499,8 +498,11 @@ impl ManagedAccountCollection { account: &Account, ) -> Result { let key_source = KeySource::Public(account.account_xpub); - let managed_type = - Self::build_managed_account_type(account.account_type, account.network, &key_source)?; + let managed_type = ManagedAccountType::from_account_type( + account.account_type, + account.network, + &key_source, + )?; Ok(ManagedCoreKeysAccount::new(managed_type, account.network)) } @@ -511,8 +513,11 @@ impl ManagedAccountCollection { account: &crate::account::BLSAccount, ) -> Result { let key_source = KeySource::BLSPublic(account.bls_public_key.clone()); - let managed_type = - Self::build_managed_account_type(account.account_type, account.network, &key_source)?; + let managed_type = ManagedAccountType::from_account_type( + account.account_type, + account.network, + &key_source, + )?; Ok(ManagedCoreKeysAccount::new(managed_type, account.network)) } @@ -527,266 +532,14 @@ impl ManagedAccountCollection { Some(priv_key) => KeySource::EdDSAPrivate(priv_key), None => KeySource::NoKeySource, }; - let managed_type = - Self::build_managed_account_type(account.account_type, account.network, &key_source)?; + let managed_type = ManagedAccountType::from_account_type( + account.account_type, + account.network, + &key_source, + )?; Ok(ManagedCoreKeysAccount::new(managed_type, account.network)) } - /// Build the [`ManagedAccountType`] (address pools + variant data) for an - /// account type and network. Shared between the funds-bearing and - /// keys-only construction helpers above — the wrap into one variant or - /// the other happens in the caller. - fn build_managed_account_type( - account_type: AccountType, - network: Network, - key_source: &KeySource, - ) -> Result { - // Get the derivation path for this account type - let base_path = account_type - .derivation_path(network) - .unwrap_or_else(|_| crate::bip32::DerivationPath::master()); - - // Create the appropriate ManagedAccountType with address pools - let managed_type = match account_type { - AccountType::Standard { - index, - standard_account_type, - } => { - // For standard accounts, add the receive/change branch to the path - let mut external_path = base_path.clone(); - external_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); // 0 for external - let external_pool = AddressPool::new( - external_path, - AddressPoolType::External, - DEFAULT_EXTERNAL_GAP_LIMIT, - network, - key_source, - )?; - - let mut internal_path = base_path; - internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1)?); // 1 for internal - let internal_pool = AddressPool::new( - internal_path, - AddressPoolType::Internal, - DEFAULT_INTERNAL_GAP_LIMIT, - network, - key_source, - )?; - - let managed_standard_type = standard_account_type; - - ManagedAccountType::Standard { - index, - standard_account_type: managed_standard_type, - external_addresses: external_pool, - internal_addresses: internal_pool, - } - } - AccountType::CoinJoin { - index, - } => { - // Dual-pool: external (.../0/index) for Dash Core mixed coins, internal - // (.../1/index) for DashSync mixing-change. Watch both so no funds are missed. - let mut external_path = base_path.clone(); - external_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); - let external_addresses = AddressPool::new( - external_path, - AddressPoolType::External, - DEFAULT_COINJOIN_GAP_LIMIT, - network, - key_source, - )?; - - let mut internal_path = base_path; - internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1)?); - let internal_addresses = AddressPool::new( - internal_path, - AddressPoolType::Internal, - DEFAULT_COINJOIN_GAP_LIMIT, - network, - key_source, - )?; - - ManagedAccountType::CoinJoin { - index, - external_addresses, - internal_addresses, - } - } - AccountType::IdentityRegistration => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::IdentityRegistration { - addresses, - } - } - AccountType::IdentityTopUp { - registration_index, - } => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::IdentityTopUp { - registration_index, - addresses, - } - } - AccountType::IdentityTopUpNotBoundToIdentity => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::IdentityTopUpNotBoundToIdentity { - addresses, - } - } - AccountType::IdentityInvitation => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::IdentityInvitation { - addresses, - } - } - AccountType::AssetLockAddressTopUp => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::AssetLockAddressTopUp { - addresses, - } - } - AccountType::AssetLockShieldedAddressTopUp => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::AssetLockShieldedAddressTopUp { - addresses, - } - } - AccountType::ProviderVotingKeys => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::ProviderVotingKeys { - addresses, - } - } - AccountType::ProviderOwnerKeys => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::ProviderOwnerKeys { - addresses, - } - } - AccountType::ProviderOperatorKeys => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::ProviderOperatorKeys { - addresses, - } - } - AccountType::ProviderPlatformKeys => { - let addresses = AddressPool::new( - base_path, - AddressPoolType::AbsentHardened, - DEFAULT_SPECIAL_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::ProviderPlatformKeys { - addresses, - } - } - AccountType::DashpayReceivingFunds { - index, - user_identity_id, - friend_identity_id, - } => { - let addresses = - AddressPool::new(base_path, AddressPoolType::Absent, 20, network, key_source)?; - ManagedAccountType::DashpayReceivingFunds { - index, - user_identity_id, - friend_identity_id, - addresses, - } - } - AccountType::DashpayExternalAccount { - index, - user_identity_id, - friend_identity_id, - } => { - let addresses = - AddressPool::new(base_path, AddressPoolType::Absent, 20, network, key_source)?; - ManagedAccountType::DashpayExternalAccount { - index, - user_identity_id, - friend_identity_id, - addresses, - } - } - AccountType::PlatformPayment { - account, - key_class, - } => { - // DIP-17/DIP-18: Platform Payment addresses - let addresses = AddressPool::new( - base_path, - AddressPoolType::Absent, - DIP17_GAP_LIMIT, - network, - key_source, - )?; - ManagedAccountType::PlatformPayment { - account, - key_class, - addresses, - } - } - }; - - Ok(managed_type) - } - /// Create a ManagedPlatformAccount from an Account for Platform Payment accounts fn create_managed_platform_account_from_account( account: &Account, @@ -795,15 +548,9 @@ impl ManagedAccountCollection { // Use the account's existing public key let key_source = KeySource::Public(account.account_xpub); - // Get the derivation path for this account type - let base_path = account - .account_type - .derivation_path(account.network) - .unwrap_or_else(|_| crate::bip32::DerivationPath::master()); - - // Create address pool for DIP-17 Platform Payment addresses - let addresses = AddressPool::new( - base_path, + // DIP-17 Platform Payment addresses: single pool on the account's path. + let addresses = ManagedAccountType::single_pool( + account.account_type, AddressPoolType::Absent, DIP17_GAP_LIMIT, account.network, diff --git a/key-wallet/src/managed_account/managed_account_type.rs b/key-wallet/src/managed_account/managed_account_type.rs index 40295e800..cb91b9655 100644 --- a/key-wallet/src/managed_account/managed_account_type.rs +++ b/key-wallet/src/managed_account/managed_account_type.rs @@ -5,6 +5,7 @@ use crate::gap_limit::{ DEFAULT_SPECIAL_GAP_LIMIT, DIP17_GAP_LIMIT, }; +use crate::managed_account::address_pool::AddressPoolType; use crate::{AccountType, AddressPool, DerivationPath}; #[cfg(feature = "bincode")] use bincode_derive::{Decode, Encode}; @@ -488,50 +489,79 @@ impl ManagedAccountType { } } + /// Build a single address pool for `account_type` (single-pool account types). The pool path + /// is the account's derivation path + pub(crate) fn single_pool( + account_type: AccountType, + pool_type: AddressPoolType, + gap_limit: u32, + network: crate::Network, + key_source: &crate::KeySource, + ) -> Result { + let path = + account_type.derivation_path(network).unwrap_or_else(|_| DerivationPath::master()); + + AddressPool::new(path, pool_type, gap_limit, network, key_source) + } + + /// Build the external (`.../0/index`) and internal (`.../1/index`) pools for a dual-pool + /// account type. Both branches derive from the account's path. + pub(crate) fn dual_pools( + account_type: AccountType, + external_gap_limit: u32, + internal_gap_limit: u32, + network: crate::Network, + key_source: &crate::KeySource, + ) -> Result<(AddressPool, AddressPool), crate::error::Error> { + let base_path = + account_type.derivation_path(network).unwrap_or_else(|_| DerivationPath::master()); + + let mut external_path = base_path.clone(); + external_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); + let external = AddressPool::new( + external_path, + AddressPoolType::External, + external_gap_limit, + network, + key_source, + )?; + + let mut internal_path = base_path; + internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1)?); + let internal = AddressPool::new( + internal_path, + AddressPoolType::Internal, + internal_gap_limit, + network, + key_source, + )?; + + Ok((external, internal)) + } + /// Create a ManagedAccountType from an AccountType with address pools pub fn from_account_type( account_type: AccountType, network: crate::Network, key_source: &crate::KeySource, ) -> Result { - use crate::bip32::DerivationPath; - use crate::managed_account::address_pool::{AddressPool, AddressPoolType}; - match account_type { AccountType::Standard { index, standard_account_type, } => { - // Create external and internal address pools for standard accounts - let base_path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - - let mut external_path = base_path.clone(); - external_path.push(crate::bip32::ChildNumber::from_normal_idx(0).unwrap()); - let external_pool = AddressPool::new( - external_path, - AddressPoolType::External, + let (external_addresses, internal_addresses) = Self::dual_pools( + account_type, DEFAULT_EXTERNAL_GAP_LIMIT, - network, - key_source, - )?; - - let mut internal_path = base_path; - internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1).unwrap()); - let internal_pool = AddressPool::new( - internal_path, - AddressPoolType::Internal, DEFAULT_INTERNAL_GAP_LIMIT, network, key_source, )?; - Ok(Self::Standard { index, standard_account_type, - external_addresses: external_pool, - internal_addresses: internal_pool, + external_addresses, + internal_addresses, }) } AccountType::CoinJoin { @@ -540,42 +570,22 @@ impl ManagedAccountType { // Dual-pool: Dash Core receives mixed coins on the external branch // (m/9'/coin'/4'/account'/0/index); DashSync also uses the internal branch // (.../1/index) for mixing-change. Watch both so no funds are missed. - let base_path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - - let mut external_path = base_path.clone(); - external_path.push(crate::bip32::ChildNumber::from_normal_idx(0)?); - let external_pool = AddressPool::new( - external_path, - AddressPoolType::External, + let (external_addresses, internal_addresses) = Self::dual_pools( + account_type, DEFAULT_COINJOIN_GAP_LIMIT, - network, - key_source, - )?; - - let mut internal_path = base_path; - internal_path.push(crate::bip32::ChildNumber::from_normal_idx(1)?); - let internal_pool = AddressPool::new( - internal_path, - AddressPoolType::Internal, DEFAULT_COINJOIN_GAP_LIMIT, network, key_source, )?; - Ok(Self::CoinJoin { index, - external_addresses: external_pool, - internal_addresses: internal_pool, + external_addresses, + internal_addresses, }) } AccountType::IdentityRegistration => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -589,11 +599,8 @@ impl ManagedAccountType { AccountType::IdentityTopUp { registration_index, } => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -606,11 +613,8 @@ impl ManagedAccountType { }) } AccountType::IdentityTopUpNotBoundToIdentity => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -622,11 +626,8 @@ impl ManagedAccountType { }) } AccountType::IdentityInvitation => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -638,11 +639,8 @@ impl ManagedAccountType { }) } AccountType::AssetLockAddressTopUp => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -654,11 +652,8 @@ impl ManagedAccountType { }) } AccountType::AssetLockShieldedAddressTopUp => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -670,11 +665,8 @@ impl ManagedAccountType { }) } AccountType::ProviderVotingKeys => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -686,11 +678,8 @@ impl ManagedAccountType { }) } AccountType::ProviderOwnerKeys => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -702,11 +691,8 @@ impl ManagedAccountType { }) } AccountType::ProviderOperatorKeys => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::Absent, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -718,11 +704,8 @@ impl ManagedAccountType { }) } AccountType::ProviderPlatformKeys => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, + let pool = Self::single_pool( + account_type, AddressPoolType::AbsentHardened, DEFAULT_SPECIAL_GAP_LIMIT, network, @@ -738,12 +721,9 @@ impl ManagedAccountType { user_identity_id, friend_identity_id, } => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, - crate::managed_account::address_pool::AddressPoolType::Absent, + let pool = Self::single_pool( + account_type, + AddressPoolType::Absent, 20, network, key_source, @@ -760,12 +740,9 @@ impl ManagedAccountType { user_identity_id, friend_identity_id, } => { - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, - crate::managed_account::address_pool::AddressPoolType::Absent, + let pool = Self::single_pool( + account_type, + AddressPoolType::Absent, 20, network, key_source, @@ -782,13 +759,9 @@ impl ManagedAccountType { key_class, } => { // DIP-17: m/9'/coin_type'/17'/account'/key_class'/index - // The leaf index is non-hardened - let path = account_type - .derivation_path(network) - .unwrap_or_else(|_| DerivationPath::master()); - let pool = AddressPool::new( - path, - crate::managed_account::address_pool::AddressPoolType::Absent, + let pool = Self::single_pool( + account_type, + AddressPoolType::Absent, DIP17_GAP_LIMIT, network, key_source, From a8a096838b829cf5bec3c2374a23511640a0c35c Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Wed, 1 Jul 2026 19:13:14 -0700 Subject: [PATCH 018/149] perf: reduced amount of block hashes computed (#834) --- dash-spv/benches/storage.rs | 7 +- dash-spv/src/client/lifecycle.rs | 10 ++- dash-spv/src/storage/block_headers.rs | 74 +++++----------- dash-spv/src/storage/mod.rs | 55 +++++------- dash-spv/src/sync/block_headers/manager.rs | 9 +- dash-spv/src/sync/blocks/manager.rs | 24 ++++-- dash-spv/src/sync/blocks/pipeline.rs | 63 +++++++------- dash-spv/src/sync/blocks/sync_manager.rs | 8 +- dash-spv/src/sync/chainlock/manager.rs | 4 +- dash-spv/src/sync/filter_headers/pipeline.rs | 18 ++-- dash-spv/src/sync/filters/manager.rs | 85 +++++++++++++++++-- dash-spv/src/sync/filters/pipeline.rs | 83 +++++++++++++++--- dash-spv/src/sync/masternodes/manager.rs | 12 ++- dash-spv/src/sync/masternodes/sync_manager.rs | 21 ++--- key-wallet-manager/src/event_tests.rs | 42 ++++++--- key-wallet-manager/src/process_block.rs | 17 ++-- .../src/test_utils/mock_wallet.rs | 3 + key-wallet-manager/src/wallet_interface.rs | 3 +- .../tests/spv_integration_tests.rs | 2 +- 19 files changed, 331 insertions(+), 209 deletions(-) diff --git a/dash-spv/benches/storage.rs b/dash-spv/benches/storage.rs index 44452c303..8466d6b76 100644 --- a/dash-spv/benches/storage.rs +++ b/dash-spv/benches/storage.rs @@ -28,7 +28,10 @@ fn bench_disk_storage(c: &mut Criterion) { let rt = Builder::new_multi_thread().worker_threads(4).enable_all().build().unwrap(); - let headers = (0..NUM_ELEMENTS).map(create_test_header).collect::>(); + let headers = (0..NUM_ELEMENTS) + .map(create_test_header) + .map(dash_spv::types::HashedBlockHeader::from) + .collect::>(); let mut rng = StdRng::seed_from_u64(SEED); c.bench_function("storage/disk/store", |b| { @@ -75,7 +78,7 @@ fn bench_disk_storage(c: &mut Criterion) { b.to_async(&rt).iter_batched( || { let height = rand::random::() % NUM_ELEMENTS; - headers[height as usize].block_hash() + *headers[height as usize].hash() }, async |hash| { let _ = storage.get_header_height_by_hash(&hash).await.unwrap(); diff --git a/dash-spv/src/client/lifecycle.rs b/dash-spv/src/client/lifecycle.rs index 8c7d4f29f..d844ceec7 100644 --- a/dash-spv/src/client/lifecycle.rs +++ b/dash-spv/src/client/lifecycle.rs @@ -291,7 +291,10 @@ impl DashSpvClient DashSpvClient StorageResult<()>; + async fn store_headers(&mut self, headers: &[HashedBlockHeader]) -> StorageResult<()>; async fn store_headers_at_height( - &mut self, - headers: &[BlockHeader], - height: u32, - ) -> StorageResult<()>; - - //TODO - change API of the BlockHeaderStorage trait to accept (store) and return (load) - // HashedBlockHeaders instead of BlockHeaders to avoid unnecessary hashing and remove - // the two store_hashed_headers methods below. - async fn store_hashed_headers(&mut self, headers: &[HashedBlockHeader]) -> StorageResult<()>; - - async fn store_hashed_headers_at_height( &mut self, headers: &[HashedBlockHeader], height: u32, @@ -66,9 +55,9 @@ pub trait BlockHeaderStorage: Send + Sync + 'static { /// Returns `StorageError::InvalidArgument` when the range extends into a /// segment queued for deletion by a prior `truncate_above` (before the next /// `persist`). Callers must clamp the range to at most `get_tip_height`. - async fn load_headers(&self, range: Range) -> StorageResult>; + async fn load_headers(&self, range: Range) -> StorageResult>; - async fn get_header(&self, height: u32) -> StorageResult> { + async fn get_header(&self, height: u32) -> StorageResult> { if let Some(tip_height) = self.get_tip_height().await { if height > tip_height { return Ok(None); @@ -85,7 +74,7 @@ pub trait BlockHeaderStorage: Send + Sync + 'static { return Ok(None); } - Ok(self.load_headers(height..height + 1).await?.first().copied()) + Ok(self.load_headers(height..height + 1).await?.into_iter().next()) } async fn get_tip_height(&self) -> Option; @@ -161,27 +150,12 @@ impl PersistentStorage for PersistentBlockHeaderStorage { #[async_trait] impl BlockHeaderStorage for PersistentBlockHeaderStorage { - async fn store_headers(&mut self, headers: &[BlockHeader]) -> StorageResult<()> { + async fn store_headers(&mut self, headers: &[HashedBlockHeader]) -> StorageResult<()> { let height = self.block_headers.read().await.next_height(); self.store_headers_at_height(headers, height).await } async fn store_headers_at_height( - &mut self, - headers: &[BlockHeader], - height: u32, - ) -> StorageResult<()> { - let headers = - headers.iter().map(HashedBlockHeader::from).collect::>(); - self.store_hashed_headers_at_height(&headers, height).await - } - - async fn store_hashed_headers(&mut self, headers: &[HashedBlockHeader]) -> StorageResult<()> { - let height = self.block_headers.read().await.next_height(); - self.store_hashed_headers_at_height(headers, height).await - } - - async fn store_hashed_headers_at_height( &mut self, headers: &[HashedBlockHeader], height: u32, @@ -198,16 +172,9 @@ impl BlockHeaderStorage for PersistentBlockHeaderStorage { Ok(()) } - async fn load_headers(&self, range: Range) -> StorageResult> { - Ok(self - .block_headers - .write() - .await - .get_items(range) - .await? - .into_iter() - .map(|cached| *cached.header()) - .collect()) + // Returns the cached `HashedBlockHeader`s straight from the segment cache — no recomputation. + async fn load_headers(&self, range: Range) -> StorageResult> { + self.block_headers.write().await.get_items(range).await } async fn get_tip_height(&self) -> Option { @@ -268,9 +235,13 @@ mod tests { use super::*; use tempfile::TempDir; + fn hashed_batch(r: std::ops::Range) -> Vec { + BlockHeader::dummy_batch(r).into_iter().map(HashedBlockHeader::from).collect() + } + #[tokio::test] async fn test_get_tip() { - let headers = BlockHeader::dummy_batch(0..5); + let headers = hashed_batch(0..5); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); // Tip should be none before storing headers @@ -278,13 +249,13 @@ mod tests { // Add one header and validate tip storage.store_headers(&headers[0..1]).await.unwrap(); let tip = storage.get_tip().await.unwrap(); - let expected_tip = BlockHeaderTip::new(0, HashedBlockHeader::from(headers[0])); + let expected_tip = BlockHeaderTip::new(0, headers[0].clone()); assert_eq!(tip, expected_tip); assert_eq!(storage.get_tip_height().await, Some(0)); // Add multiple headers and validate tip storage.store_headers(&headers[1..]).await.unwrap(); let tip = storage.get_tip().await.unwrap(); - let expected_tip = BlockHeaderTip::new(4, HashedBlockHeader::from(headers[4])); + let expected_tip = BlockHeaderTip::new(4, headers[4].clone()); assert_eq!(tip, expected_tip); assert_eq!(storage.get_tip_height().await, Some(4)); } @@ -294,35 +265,32 @@ mod tests { let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - let headers = BlockHeader::dummy_batch(0..10); + let headers = hashed_batch(0..10); storage.store_headers(&headers).await.unwrap(); - let orphaned_hash = headers[7].block_hash(); + let orphaned_hash = *headers[7].hash(); assert_eq!(storage.get_header_height_by_hash(&orphaned_hash).await.unwrap(), Some(7)); storage.truncate_above(100).await.unwrap(); assert_eq!(storage.get_tip_height().await, Some(9)); - assert_eq!( - storage.get_header_height_by_hash(&headers[4].block_hash()).await.unwrap(), - Some(4) - ); + assert_eq!(storage.get_header_height_by_hash(headers[4].hash()).await.unwrap(), Some(4)); storage.truncate_above(5).await.unwrap(); assert_eq!(storage.get_tip_height().await, Some(5)); assert_eq!(storage.get_header_height_by_hash(&orphaned_hash).await.unwrap(), None); - let kept_hash = headers[3].block_hash(); + let kept_hash = *headers[3].hash(); assert_eq!(storage.get_header_height_by_hash(&kept_hash).await.unwrap(), Some(3)); - let replacement = BlockHeader::dummy_batch(100..105); + let replacement = hashed_batch(100..105); storage.store_headers_at_height(&replacement, 6).await.unwrap(); assert_eq!(storage.get_tip_height().await, Some(10)); let reloaded = storage.load_headers(6..11).await.unwrap(); assert_eq!(reloaded, replacement); - let new_hash = replacement[0].block_hash(); + let new_hash = *replacement[0].hash(); assert_eq!(storage.get_header_height_by_hash(&new_hash).await.unwrap(), Some(6)); // Exercise the durability contract: persist, drop, reopen, and verify diff --git a/dash-spv/src/storage/mod.rs b/dash-spv/src/storage/mod.rs index 57e4fb736..453b60037 100644 --- a/dash-spv/src/storage/mod.rs +++ b/dash-spv/src/storage/mod.rs @@ -19,7 +19,6 @@ use crate::ClientConfig; use async_trait::async_trait; use dashcore::hash_types::FilterHeader; use dashcore::prelude::CoreBlockHeight; -use dashcore::Header as BlockHeader; use std::ops::Range; use std::path::PathBuf; use std::sync::Arc; @@ -269,31 +268,19 @@ impl StorageManager for DiskStorageManager { #[async_trait] impl BlockHeaderStorage for DiskStorageManager { - async fn store_headers(&mut self, headers: &[BlockHeader]) -> StorageResult<()> { + async fn store_headers(&mut self, headers: &[HashedBlockHeader]) -> StorageResult<()> { self.block_headers.write().await.store_headers(headers).await } async fn store_headers_at_height( - &mut self, - headers: &[BlockHeader], - height: u32, - ) -> StorageResult<()> { - self.block_headers.write().await.store_headers_at_height(headers, height).await - } - - async fn store_hashed_headers(&mut self, headers: &[HashedBlockHeader]) -> StorageResult<()> { - self.block_headers.write().await.store_hashed_headers(headers).await - } - - async fn store_hashed_headers_at_height( &mut self, headers: &[HashedBlockHeader], height: u32, ) -> StorageResult<()> { - self.block_headers.write().await.store_hashed_headers_at_height(headers, height).await + self.block_headers.write().await.store_headers_at_height(headers, height).await } - async fn load_headers(&self, range: Range) -> StorageResult> { + async fn load_headers(&self, range: Range) -> StorageResult> { self.block_headers.read().await.load_headers(range).await } @@ -426,6 +413,10 @@ mod tests { use dashcore::Header as BlockHeader; use tempfile::TempDir; + fn hashed_batch(r: std::ops::Range) -> Vec { + BlockHeader::dummy_batch(r).into_iter().map(HashedBlockHeader::from).collect() + } + #[tokio::test] async fn test_store_load_headers() -> Result<(), Box> { // Create a temporary directory for the test @@ -433,7 +424,7 @@ mod tests { let config = ClientConfig::testnet().with_storage_path(temp_dir.path()); let mut storage = DiskStorageManager::new(&config).await.expect("Unable to create storage"); - let headers = BlockHeader::dummy_batch(0..60_000); + let headers = hashed_batch(0..60_000); storage.store_headers(&headers[0..0]).await.expect("Should handle empty header batch"); assert_eq!(storage.get_tip_height().await, None); @@ -478,8 +469,8 @@ mod tests { // Create test headers starting from checkpoint height const CHECKPOINT_HEIGHT: u32 = 1_100_000; - let headers: Vec = - BlockHeader::dummy_batch(CHECKPOINT_HEIGHT..CHECKPOINT_HEIGHT + 100); + let headers: Vec = + hashed_batch(CHECKPOINT_HEIGHT..CHECKPOINT_HEIGHT + 100); storage.store_headers_at_height(&headers, CHECKPOINT_HEIGHT).await?; @@ -496,18 +487,18 @@ mod tests { async fn check_storage( storage: &DiskStorageManager, - headers: &[BlockHeader], + headers: &[HashedBlockHeader], ) -> StorageResult<()> { assert_eq!(storage.get_stored_headers_len().await, headers.len() as u32); let header_at_base = storage.get_header(CHECKPOINT_HEIGHT).await?; - assert_eq!(header_at_base, Some(headers[0])); + assert_eq!(header_at_base, Some(headers[0].clone())); let header_at_ending = storage.get_header(CHECKPOINT_HEIGHT + 99).await?; - assert_eq!(header_at_ending, Some(headers[99])); + assert_eq!(header_at_ending, Some(headers[99].clone())); // Test the reverse index (hash -> blockchain height) - let hash_0 = headers[0].block_hash(); + let hash_0 = *headers[0].hash(); let height_0 = storage.get_header_height_by_hash(&hash_0).await?; assert_eq!( height_0, @@ -515,7 +506,7 @@ mod tests { "Hash should map to blockchain height 1,100,000" ); - let hash_99 = headers[99].block_hash(); + let hash_99 = *headers[99].hash(); let height_99 = storage.get_header_height_by_hash(&hash_99).await?; assert_eq!( height_99, @@ -536,13 +527,13 @@ mod tests { let mut storage = DiskStorageManager::new(&config).await.unwrap(); // Create and store headers - let headers = BlockHeader::dummy_batch(0..10); + let headers = hashed_batch(0..10); storage.store_headers(&headers).await.unwrap(); // Test reverse lookups for (i, header) in headers.iter().enumerate() { - let hash = header.block_hash(); + let hash = *header.hash(); let height = storage.get_header_height_by_hash(&hash).await.unwrap(); assert_eq!(height, Some(i as u32), "Height mismatch for header {}", i); } @@ -558,7 +549,7 @@ mod tests { // We need to get the actual headers that were stored to test properly for i in 0..10 { let stored_header = storage.get_header(i).await.unwrap().unwrap(); - let hash = stored_header.block_hash(); + let hash = *stored_header.hash(); let height = storage.get_header_height_by_hash(&hash).await.unwrap(); assert_eq!(height, Some(i), "Height mismatch after reload for header {}", i); } @@ -571,10 +562,10 @@ mod tests { DiskStorageManager::with_temp_dir().await.expect("Failed to create tmp storage"); // Store some headers - let header = BlockHeader::dummy_batch(0..1); + let header = hashed_batch(0..1); storage.store_headers(&header).await.unwrap(); - let hash = header[0].block_hash(); + let hash = *header[0].hash(); assert!(storage.get_header_height_by_hash(&hash).await.unwrap().is_some()); // Clear storage @@ -590,10 +581,10 @@ mod tests { let config = ClientConfig::regtest().with_storage_path(temp_dir.path()); let mut mgr = DiskStorageManager::new(&config).await.unwrap(); - let headers = BlockHeader::dummy_batch(0..10); + let headers = hashed_batch(0..10); mgr.store_headers(&headers).await.unwrap(); - let orphaned_hash = headers[7].block_hash(); + let orphaned_hash = *headers[7].hash(); assert_eq!(mgr.get_header_height_by_hash(&orphaned_hash).await.unwrap(), Some(7)); ::truncate_above(&mut mgr, 5).await.unwrap(); @@ -601,7 +592,7 @@ mod tests { assert_eq!(mgr.get_tip_height().await, Some(5)); assert_eq!(mgr.get_header_height_by_hash(&orphaned_hash).await.unwrap(), None); - let kept_hash = headers[3].block_hash(); + let kept_hash = *headers[3].hash(); assert_eq!(mgr.get_header_height_by_hash(&kept_hash).await.unwrap(), Some(3)); } diff --git a/dash-spv/src/sync/block_headers/manager.rs b/dash-spv/src/sync/block_headers/manager.rs index 713699f7c..b3699e979 100644 --- a/dash-spv/src/sync/block_headers/manager.rs +++ b/dash-spv/src/sync/block_headers/manager.rs @@ -108,7 +108,7 @@ impl BlockHeadersManager { BlockHeaderValidator::new().validate(headers)?; // Store headers - self.header_storage.write().await.store_hashed_headers(headers).await?; + self.header_storage.write().await.store_headers(headers).await?; let tip = self.tip().await?; @@ -277,7 +277,12 @@ mod tests { let mut storage = DiskStorageManager::with_temp_dir().await.unwrap(); // Store a genesis header so the manager can initialize let genesis = Header::dummy_batch(0..1); - storage.store_headers(&genesis).await.unwrap(); + storage + .store_headers( + &genesis.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let checkpoint_manager = create_test_checkpoint_manager(); BlockHeadersManager::new(storage.block_headers(), storage.metadata(), checkpoint_manager) .await diff --git a/dash-spv/src/sync/blocks/manager.rs b/dash-spv/src/sync/blocks/manager.rs index da9e01220..70df8624b 100644 --- a/dash-spv/src/sync/blocks/manager.rs +++ b/dash-spv/src/sync/blocks/manager.rs @@ -80,12 +80,13 @@ impl BlocksManager, /// Heights queued or in-flight (waiting for download). pending_heights: BTreeSet, - /// Downloaded blocks ready to process (height -> Block). - downloaded: BTreeMap, + /// Downloaded blocks ready to process (height -> block, with its cached hash). + downloaded: BTreeMap, /// Map hash -> height for looking up height when block arrives. hash_to_height: HashMap, /// Per-block interested wallets, populated when the block is queued. @@ -138,8 +138,8 @@ impl BlocksPipeline { /// Looks up the height from the internal hash_to_height map and stores /// the block in the downloaded buffer for height-ordered processing. /// Returns `true` if this was a tracked block, `false` if unrequested. - pub(super) fn receive_block(&mut self, block: &Block) -> bool { - let hash = block.block_hash(); + pub(super) fn receive_block(&mut self, block: &HashedBlock) -> bool { + let hash = *block.hash(); if !self.coordinator.receive(&hash) { tracing::debug!("Ignoring unrequested block: {}", hash); return false; @@ -158,7 +158,9 @@ impl BlocksPipeline { /// Returns None if: /// - No downloaded blocks available, or /// - Waiting for a lower-height block still pending - pub(super) fn take_next_ordered_block(&mut self) -> Option<(Block, u32, BTreeSet)> { + pub(super) fn take_next_ordered_block( + &mut self, + ) -> Option<(HashedBlock, u32, BTreeSet)> { let lowest_downloaded = *self.downloaded.keys().next()?; // Check if any pending blocks have lower heights @@ -169,7 +171,7 @@ impl BlocksPipeline { } let block = self.downloaded.remove(&lowest_downloaded).unwrap(); - let wallets = self.hash_to_wallets.remove(&block.block_hash()).unwrap_or_default(); + let wallets = self.hash_to_wallets.remove(block.hash()).unwrap_or_default(); Some((block, lowest_downloaded, wallets)) } @@ -178,11 +180,11 @@ impl BlocksPipeline { /// Used when blocks are already persisted from a previous sync. pub(super) fn add_from_storage( &mut self, - block: Block, + block: HashedBlock, height: u32, wallets: BTreeSet, ) { - let hash = block.block_hash(); + let hash = *block.hash(); self.hash_to_wallets.entry(hash).or_default().extend(wallets); self.downloaded.insert(height, block); } @@ -204,6 +206,7 @@ impl BlocksPipeline { #[cfg(test)] mod tests { + use dashcore::blockdata::block::Block; use dashcore_hashes::Hash; use super::*; @@ -281,11 +284,11 @@ mod tests { assert_eq!(pipeline.coordinator.active_count(), 1); // Receive block - assert!(pipeline.receive_block(&block)); + assert!(pipeline.receive_block(&HashedBlock::from(&block))); assert_eq!(pipeline.coordinator.active_count(), 0); assert_eq!(pipeline.downloaded.len(), 1); assert!(pipeline.pending_heights.is_empty()); - assert_eq!(pipeline.downloaded.get(&100).unwrap().block_hash(), hash); + assert_eq!(*pipeline.downloaded.get(&100).unwrap().hash(), hash); } #[test] @@ -293,7 +296,7 @@ mod tests { let mut pipeline = BlocksPipeline::new(); let block = make_test_block(1); - assert!(!pipeline.receive_block(&block)); + assert!(!pipeline.receive_block(&HashedBlock::from(&block))); assert!(pipeline.downloaded.is_empty()); } @@ -332,7 +335,7 @@ mod tests { assert_eq!(pipeline.coordinator.active_count(), 1); // B: already received, sitting in `downloaded` — must survive requeue. - pipeline.add_from_storage(block_b.clone(), 200, BTreeSet::from([[2u8; 32]])); + pipeline.add_from_storage(HashedBlock::from(&block_b), 200, BTreeSet::from([[2u8; 32]])); pipeline.requeue_in_flight(); @@ -385,7 +388,7 @@ mod tests { // Use add_from_storage to test ordering logic without network // Add block 2 first (out of order) - pipeline.add_from_storage(block2.clone(), 101, BTreeSet::new()); + pipeline.add_from_storage(HashedBlock::from(&block2), 101, BTreeSet::new()); // Also track height 100 as pending to simulate waiting pipeline.pending_heights.insert(100); @@ -394,17 +397,17 @@ mod tests { // Add block 1 pipeline.pending_heights.remove(&100); - pipeline.add_from_storage(block1.clone(), 100, BTreeSet::new()); + pipeline.add_from_storage(HashedBlock::from(&block1), 100, BTreeSet::new()); // Now block 1 is ready (lowest height) let (block, height, _) = pipeline.take_next_ordered_block().unwrap(); assert_eq!(height, 100); - assert_eq!(block.block_hash(), hash1); + assert_eq!(*block.hash(), hash1); // Block 2 is now ready let (block, height, _) = pipeline.take_next_ordered_block().unwrap(); assert_eq!(height, 101); - assert_eq!(block.block_hash(), hash2); + assert_eq!(*block.hash(), hash2); // No more blocks assert!(pipeline.take_next_ordered_block().is_none()); @@ -417,7 +420,7 @@ mod tests { // Add block at height 101, but height 100 is still pending pipeline.pending_heights.insert(100); - pipeline.add_from_storage(block2.clone(), 101, BTreeSet::new()); + pipeline.add_from_storage(HashedBlock::from(&block2), 101, BTreeSet::new()); // Cannot take block 2 - block at height 100 is still pending assert!(pipeline.take_next_ordered_block().is_none()); @@ -436,13 +439,13 @@ mod tests { let block = make_test_block(1); let hash = block.block_hash(); - pipeline.add_from_storage(block.clone(), 100, BTreeSet::new()); + pipeline.add_from_storage(HashedBlock::from(&block), 100, BTreeSet::new()); assert_eq!(pipeline.downloaded.len(), 1); let (taken_block, height, _) = pipeline.take_next_ordered_block().unwrap(); assert_eq!(height, 100); - assert_eq!(taken_block.block_hash(), hash); + assert_eq!(*taken_block.hash(), hash); } #[test] @@ -452,7 +455,7 @@ mod tests { // Adding to downloaded makes it incomplete let block = make_test_block(1); - pipeline.add_from_storage(block, 100, BTreeSet::new()); + pipeline.add_from_storage(HashedBlock::from(&block), 100, BTreeSet::new()); assert!(!pipeline.is_complete()); // Take the block @@ -487,10 +490,10 @@ mod tests { // Drive the block through receive_block to land it in `downloaded`. let hashes = pipeline.coordinator.take_pending(1); pipeline.coordinator.mark_sent(&hashes); - assert!(pipeline.receive_block(&block)); + assert!(pipeline.receive_block(&HashedBlock::from(&block))); let (taken_block, height, taken_wallets) = pipeline.take_next_ordered_block().unwrap(); - assert_eq!(taken_block.block_hash(), hash); + assert_eq!(*taken_block.hash(), hash); assert_eq!(height, 100); assert_eq!(taken_wallets, wallets); } @@ -516,7 +519,7 @@ mod tests { let hashes = pipeline.coordinator.take_pending(1); assert_eq!(hashes.len(), 1); pipeline.coordinator.mark_sent(&hashes); - assert!(pipeline.receive_block(&block)); + assert!(pipeline.receive_block(&HashedBlock::from(&block))); let (_, _, taken_wallets) = pipeline.take_next_ordered_block().unwrap(); let mut expected = wallets_a; @@ -550,7 +553,7 @@ mod tests { assert_eq!(pipeline.coordinator.active_count(), 1); // Late wallet ids are still merged for when the block arrives. - assert!(pipeline.receive_block(&block)); + assert!(pipeline.receive_block(&HashedBlock::from(&block))); let (_, _, taken_wallets) = pipeline.take_next_ordered_block().unwrap(); let mut expected = wallets_a; expected.extend(wallets_b); @@ -571,7 +574,7 @@ mod tests { pipeline.queue([(FilterMatchKey::new(100, hash), wallets_a.clone())]); let hashes = pipeline.coordinator.take_pending(1); pipeline.coordinator.mark_sent(&hashes); - assert!(pipeline.receive_block(&block)); + assert!(pipeline.receive_block(&HashedBlock::from(&block))); assert_eq!(pipeline.downloaded.len(), 1); assert_eq!(pipeline.coordinator.pending_count(), 0); assert_eq!(pipeline.coordinator.active_count(), 0); @@ -598,8 +601,8 @@ mod tests { let wallets_a: BTreeSet = BTreeSet::from([[1u8; 32]]); let wallets_b: BTreeSet = BTreeSet::from([[2u8; 32]]); - pipeline.add_from_storage(block.clone(), 100, wallets_a.clone()); - pipeline.add_from_storage(block.clone(), 100, wallets_b.clone()); + pipeline.add_from_storage(HashedBlock::from(&block), 100, wallets_a.clone()); + pipeline.add_from_storage(HashedBlock::from(&block), 100, wallets_b.clone()); let (_, _, taken_wallets) = pipeline.take_next_ordered_block().unwrap(); let mut expected = wallets_a; @@ -618,12 +621,12 @@ mod tests { pipeline.coordinator.mark_sent(&hashes); // First receive - let result = pipeline.receive_block(&block); + let result = pipeline.receive_block(&HashedBlock::from(&block)); assert!(result); assert_eq!(pipeline.downloaded.len(), 1); // Duplicate receive (not tracked anymore since already completed) - let result = pipeline.receive_block(&block); + let result = pipeline.receive_block(&HashedBlock::from(&block)); assert!(!result); assert_eq!(pipeline.downloaded.len(), 1); } diff --git a/dash-spv/src/sync/blocks/sync_manager.rs b/dash-spv/src/sync/blocks/sync_manager.rs index 871d78f23..e7ecbc68a 100644 --- a/dash-spv/src/sync/blocks/sync_manager.rs +++ b/dash-spv/src/sync/blocks/sync_manager.rs @@ -74,7 +74,7 @@ impl SyncM let hashed_block = HashedBlock::from(block); // Check if this is a block we requested (pipeline handles buffering with height) - if !self.pipeline.receive_block(block) { + if !self.pipeline.receive_block(&hashed_block) { tracing::debug!("Received unrequested block {}", hashed_block.hash()); return Ok(vec![]); } @@ -146,11 +146,7 @@ impl SyncM ))); } // Block loaded from storage, add to pipeline for processing - self.pipeline.add_from_storage( - hashed_block.block().clone(), - key.height(), - wallets.clone(), - ); + self.pipeline.add_from_storage(hashed_block, key.height(), wallets.clone()); self.progress.add_from_storage(1); continue; } diff --git a/dash-spv/src/sync/chainlock/manager.rs b/dash-spv/src/sync/chainlock/manager.rs index 4b55d840f..c211919df 100644 --- a/dash-spv/src/sync/chainlock/manager.rs +++ b/dash-spv/src/sync/chainlock/manager.rs @@ -227,7 +227,7 @@ impl ChainLockManager { async fn verify_block_hash(&self, chainlock: &ChainLock) -> bool { let storage = self.header_storage.read().await; match storage.get_header(chainlock.block_height).await { - Ok(Some(header)) => header.block_hash() == chainlock.block_hash, + Ok(Some(header)) => *header.hash() == chainlock.block_hash, Ok(None) => { // Don't reject if we don't have the header yet true @@ -412,7 +412,7 @@ mod tests { .block_headers() .write() .await - .store_headers_at_height(&[header], 100) + .store_headers_at_height(&[crate::types::HashedBlockHeader::from(header)], 100) .await .expect("store header at 100"); diff --git a/dash-spv/src/sync/filter_headers/pipeline.rs b/dash-spv/src/sync/filter_headers/pipeline.rs index 0640f4d53..309b28ca0 100644 --- a/dash-spv/src/sync/filter_headers/pipeline.rs +++ b/dash-spv/src/sync/filter_headers/pipeline.rs @@ -87,13 +87,10 @@ impl FilterHeadersPipeline { let batch_end = (current + FILTER_HEADERS_BATCH_SIZE - 1).min(new_target); // Get stop hash for this batch - let stop_hash = storage - .get_header(batch_end) - .await? - .ok_or_else(|| { + let stop_hash = + storage.get_header(batch_end).await?.map(|h| *h.hash()).ok_or_else(|| { SyncError::Storage(format!("Missing header at height {}", batch_end)) - })? - .block_hash(); + })?; self.coordinator.enqueue([stop_hash]); self.batch_starts.insert(stop_hash, current); @@ -145,13 +142,10 @@ impl FilterHeadersPipeline { let batch_end = (current + FILTER_HEADERS_BATCH_SIZE - 1).min(target_height); // Get stop hash for this batch - let stop_hash = storage - .get_header(batch_end) - .await? - .ok_or_else(|| { + let stop_hash = + storage.get_header(batch_end).await?.map(|h| *h.hash()).ok_or_else(|| { SyncError::Storage(format!("Missing header at height {}", batch_end)) - })? - .block_hash(); + })?; self.coordinator.enqueue([stop_hash]); self.batch_starts.insert(stop_hash, current); diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index d2e45e9a2..d1e961a71 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -154,7 +154,7 @@ impl>(), + ) + .await + .unwrap(); // Pre-populate filter header storage with headers at heights 1..=200 let filter_headers = storage.filter_headers(); @@ -1737,7 +1748,15 @@ mod tests { // Headers must exist in storage so start_download can resolve them. let headers = dashcore::block::Header::dummy_batch(0..301); - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let (tx, _rx) = unbounded_channel(); let _ = manager.tick(&RequestSender::new(tx)).await.unwrap(); @@ -2002,7 +2021,15 @@ mod tests { // Store headers so send_pending can resolve stop hashes let headers = dashcore::block::Header::dummy_batch(0..101); - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); // Filter headers available up to 100, wallet at genesis (scan_start = 0) manager.progress.update_filter_header_tip_height(100); @@ -2053,7 +2080,15 @@ mod tests { // Store block headers so start_download can resolve heights let headers = dashcore::block::Header::dummy_batch(0..101); - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); // Simulate restart where everything is already synced but state is WaitForEvents. // committed == stored == filter_header_tip — start_download detects synced state. @@ -2096,7 +2131,15 @@ mod tests { // intentionally left empty: with the old behavior the lookahead batch // reads from height 0 and trips the empty-segment guard. let headers = dashcore::block::Header::dummy_batch(0..102); - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); manager.set_state(SyncState::WaitForEvents); manager.wallet.write().await.update_wallet_synced_height(&MOCK_WALLET_ID, 100); @@ -2171,7 +2214,15 @@ mod tests { // header; the rest are dummies and never get matched against. let mut headers: Vec = dashcore::block::Header::dummy_batch(0..201); headers[50] = block_at_50.header; - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); // Persist a filter at every height in 0..=100 so `load_filters` over // the initial batch range succeeds. Non-matching heights get a @@ -2373,7 +2424,15 @@ mod tests { let mut manager = create_test_manager().await; let headers = dashcore::block::Header::dummy_batch(0..101); - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); // Populate filter storage so load_filters succeeds during lookahead { @@ -2434,7 +2493,15 @@ mod tests { let mut manager = create_test_manager().await; let headers = dashcore::block::Header::dummy_batch(0..101); - manager.header_storage.write().await.store_headers(&headers).await.unwrap(); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); { let dummy_filter = BlockFilter::new(&[0u8; 32]); let mut fs = manager.filter_storage.write().await; diff --git a/dash-spv/src/sync/filters/pipeline.rs b/dash-spv/src/sync/filters/pipeline.rs index 3f2c2a4d5..acfbd3906 100644 --- a/dash-spv/src/sync/filters/pipeline.rs +++ b/dash-spv/src/sync/filters/pipeline.rs @@ -155,8 +155,8 @@ impl FiltersPipeline { // Get stop hash for this batch. If the header isn't available yet, // re-queue for the next tick instead of losing the batch permanently. - let header = match storage.get_header(batch_end).await { - Ok(Some(h)) => h, + let stop_hash = match storage.get_header(batch_end).await { + Ok(Some(h)) => *h.hash(), Ok(None) => { tracing::debug!( "Header at height {} not yet available, re-queuing filter batch {}", @@ -178,7 +178,7 @@ impl FiltersPipeline { } }; - requests.request_filters(start_height, header.block_hash())?; + requests.request_filters(start_height, stop_hash)?; self.coordinator.mark_sent(&[start_height]); @@ -431,7 +431,12 @@ mod tests { let headers = Header::dummy_batch(0..6000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 3500); @@ -829,7 +834,12 @@ mod tests { let headers = Header::dummy_batch(0..1000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 999); @@ -863,7 +873,12 @@ mod tests { let headers = Header::dummy_batch(0..25000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 24999); @@ -884,7 +899,12 @@ mod tests { let headers = Header::dummy_batch(0..1500); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); // Target is 1200, so second batch ends at 1200 not 1999 @@ -910,7 +930,12 @@ mod tests { let headers = Header::dummy_batch(0..3000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 2500); @@ -930,7 +955,12 @@ mod tests { let headers = Header::dummy_batch(0..100); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 50); @@ -955,7 +985,12 @@ mod tests { let headers = Header::dummy_batch(0..100); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 99); @@ -990,7 +1025,12 @@ mod tests { let headers = Header::dummy_batch(0..1000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = create_pipeline_with_short_timeout(); pipeline.init(0, 999); @@ -1053,7 +1093,12 @@ mod tests { let headers = Header::dummy_batch(0..5000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = create_pipeline_with_low_concurrency(); pipeline.init(0, 2500); @@ -1089,7 +1134,12 @@ mod tests { let headers = Header::dummy_batch(0..1000); let tmp_dir = TempDir::new().unwrap(); let mut storage = PersistentBlockHeaderStorage::open(tmp_dir.path()).await.unwrap(); - storage.store_headers(&headers).await.unwrap(); + storage + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let mut pipeline = FiltersPipeline::new(); pipeline.init(0, 1999); @@ -1114,7 +1164,12 @@ mod tests { // Store the missing headers and retry let more_headers = Header::dummy_batch(1000..2000); - storage.store_headers(&more_headers).await.unwrap(); + storage + .store_headers( + &more_headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); let sent = pipeline.send_pending(&sender, &storage).await.unwrap(); assert_eq!(sent, 1); diff --git a/dash-spv/src/sync/masternodes/manager.rs b/dash-spv/src/sync/masternodes/manager.rs index a4e042274..b31101586 100644 --- a/dash-spv/src/sync/masternodes/manager.rs +++ b/dash-spv/src/sync/masternodes/manager.rs @@ -654,7 +654,17 @@ mod tests { ) -> (TestMasternodesManager, RequestSender, mpsc::UnboundedReceiver) { let storage = DiskStorageManager::with_temp_dir().await.unwrap(); let block_headers = storage.block_headers(); - block_headers.write().await.store_headers(&Header::dummy_batch(0..tip + 1)).await.unwrap(); + block_headers + .write() + .await + .store_headers( + &Header::dummy_batch(0..tip + 1) + .iter() + .map(crate::types::HashedBlockHeader::from) + .collect::>(), + ) + .await + .unwrap(); let engine = engine_with_lists(&[(tip, 1)]); let mut manager = MasternodesManager::new( block_headers, diff --git a/dash-spv/src/sync/masternodes/sync_manager.rs b/dash-spv/src/sync/masternodes/sync_manager.rs index 120bb0d78..e949c0d2d 100644 --- a/dash-spv/src/sync/masternodes/sync_manager.rs +++ b/dash-spv/src/sync/masternodes/sync_manager.rs @@ -89,7 +89,7 @@ pub(super) async fn build_mnlistdiff_request_pairs( let base_hash = if let Some(height) = base_height { match storage.get_header(height).await { - Ok(Some(h)) => h.block_hash(), + Ok(Some(h)) => *h.hash(), Ok(None) => { tracing::warn!("Base header not found at height {}, using all-zeros", height); BlockHash::all_zeros() @@ -109,7 +109,7 @@ pub(super) async fn build_mnlistdiff_request_pairs( }; let target_hash = match storage.get_header(validation_height).await { - Ok(Some(h)) => h.block_hash(), + Ok(Some(h)) => *h.hash(), Ok(None) => { tracing::warn!("Target header not found at height {}, skipping", validation_height); continue; @@ -187,7 +187,7 @@ pub(super) async fn feed_qrinfo_heights_to_engine( let cycle_boundary_height = work_block_height + WORK_DIFF_DEPTH; if let Ok(Some(cycle_boundary_header)) = storage.get_header(cycle_boundary_height).await { - let cycle_boundary_hash = cycle_boundary_header.block_hash(); + let cycle_boundary_hash = *cycle_boundary_header.hash(); engine.feed_block_height(cycle_boundary_height, cycle_boundary_hash); fed_count += 1; tracing::debug!( @@ -654,7 +654,6 @@ mod tests { use crate::storage::{BlockHeaderStorage, BlockHeaderTip}; use crate::types::HashedBlockHeader; use async_trait::async_trait; - use dashcore::block::Header as BlockHeader; use dashcore::bls_sig_utils::{BLSPublicKey, BLSSignature}; use dashcore::hash_types::QuorumVVecHash; use dashcore::network::message_qrinfo::{MNSkipListMode, QRInfo, QuorumSnapshot}; @@ -672,27 +671,17 @@ mod tests { #[async_trait] impl BlockHeaderStorage for MockHeaderStorage { - async fn store_headers(&mut self, _: &[BlockHeader]) -> StorageResult<()> { + async fn store_headers(&mut self, _: &[HashedBlockHeader]) -> StorageResult<()> { Ok(()) } async fn store_headers_at_height( - &mut self, - _: &[BlockHeader], - _: u32, - ) -> StorageResult<()> { - Ok(()) - } - async fn store_hashed_headers(&mut self, _: &[HashedBlockHeader]) -> StorageResult<()> { - Ok(()) - } - async fn store_hashed_headers_at_height( &mut self, _: &[HashedBlockHeader], _: u32, ) -> StorageResult<()> { Ok(()) } - async fn load_headers(&self, _: Range) -> StorageResult> { + async fn load_headers(&self, _: Range) -> StorageResult> { Ok(vec![]) } async fn get_tip_height(&self) -> Option { diff --git a/key-wallet-manager/src/event_tests.rs b/key-wallet-manager/src/event_tests.rs index 516c5356f..286896921 100644 --- a/key-wallet-manager/src/event_tests.rs +++ b/key-wallet-manager/src/event_tests.rs @@ -283,7 +283,7 @@ async fn test_late_instant_send_lock_after_block_confirmation_emits_event() { // Confirm the transaction in a block first. let block = make_block(vec![tx.clone()], 0xe3, 4000); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 300, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 300, &wallets).await; let mut rx = manager.subscribe_events(); let lock = InstantLock { @@ -332,7 +332,7 @@ async fn test_block_with_new_tx_emits_inserted_record() { let block = make_block(vec![tx.clone()], 0xcc, 1000); let wallets = BTreeSet::from([wallet_id]); - let result = manager.process_block_for_wallets(&block, 100, &wallets).await; + let result = manager.process_block_for_wallets(&block, block.block_hash(), 100, &wallets).await; assert_eq!(result.new_txids.len(), 1); let events = drain_events(&mut rx); @@ -399,7 +399,7 @@ async fn test_block_confirming_known_mempool_tx_emits_updated_record() { let mut rx = manager.subscribe_events(); let block = make_block(vec![tx.clone()], 0xdd, 2000); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 200, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 200, &wallets).await; let events = drain_events(&mut rx); assert_eq!(events.len(), 1, "one BlockProcessed expected, got {:?}", events); @@ -502,7 +502,7 @@ async fn test_block_with_index_less_account_tx_carries_account_type() { let mut rx = manager.subscribe_events(); let block = make_block(vec![tx.clone()], 0xee, 9999); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 9000, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 9000, &wallets).await; let events = drain_events(&mut rx); let block_event = events @@ -540,7 +540,7 @@ async fn test_empty_block_for_idle_wallet_emits_nothing() { let block = make_block(Vec::new(), 0x55, 3000); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 50, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 50, &wallets).await; assert_no_events(&mut rx); } @@ -555,13 +555,27 @@ async fn test_block_processed_carries_matured_coinbase_record() { let coinbase_height = 100; let coinbase_block = make_block(vec![coinbase_tx.clone()], 0xc0, 4000); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&coinbase_block, coinbase_height, &wallets).await; + manager + .process_block_for_wallets( + &coinbase_block, + coinbase_block.block_hash(), + coinbase_height, + &wallets, + ) + .await; // Advance to maturity height. With coinbase_height = 100, maturity is at // height 200. Processing block 200 must surface the matured record. let mut rx = manager.subscribe_events(); let mature_block = make_block(Vec::new(), 0xc1, 5000); - manager.process_block_for_wallets(&mature_block, coinbase_height + 100, &wallets).await; + manager + .process_block_for_wallets( + &mature_block, + mature_block.block_hash(), + coinbase_height + 100, + &wallets, + ) + .await; let events = drain_events(&mut rx); let block_event = events @@ -903,7 +917,7 @@ async fn test_block_with_external_and_internal_high_index_extends_both_pools() { let mut rx = manager.subscribe_events(); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 700, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 700, &wallets).await; let events = drain_events(&mut rx); let block_event = events @@ -967,7 +981,7 @@ async fn test_block_with_two_records_pushing_external_boundary_dedupes() { let mut rx = manager.subscribe_events(); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 800, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 800, &wallets).await; let events = drain_events(&mut rx); let block_event = events @@ -1054,7 +1068,7 @@ async fn test_apply_chain_lock_promotes_in_block_record_and_emits_event() { let tx = create_tx_paying_to(&addr, 0xa1); let block = make_block(vec![tx.clone()], 0xa1, 1000); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 100, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 100, &wallets).await; let mut rx = manager.subscribe_events(); manager.apply_chain_lock(ChainLock::dummy(100)); @@ -1125,7 +1139,7 @@ async fn test_apply_chain_lock_with_no_records_emits_chain_lock_processed_and_ad let tx = create_tx_paying_to(&addr, 0xa2); let block = make_block(vec![tx.clone()], 0xa2, 1100); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 100, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 100, &wallets).await; let events = drain_events(&mut rx); let bp = events @@ -1161,7 +1175,7 @@ async fn test_apply_chain_lock_is_idempotent_on_already_finalized() { let tx = create_tx_paying_to(&addr, 0xa3); let block = make_block(vec![tx.clone()], 0xa3, 1200); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 50, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 50, &wallets).await; let mut rx = manager.subscribe_events(); manager.apply_chain_lock(ChainLock::dummy(50)); @@ -1228,7 +1242,7 @@ async fn test_block_processed_chainlocked_flag_matches_record_context() { let block_below = make_block(vec![tx_below.clone()], 0xa4, 1300); let wallets = BTreeSet::from([wallet_id]); let mut rx = manager.subscribe_events(); - manager.process_block_for_wallets(&block_below, 500, &wallets).await; + manager.process_block_for_wallets(&block_below, block_below.block_hash(), 500, &wallets).await; let events_below = drain_events(&mut rx); let bp_below = events_below @@ -1252,7 +1266,7 @@ async fn test_block_processed_chainlocked_flag_matches_record_context() { .expect("address generation"); let tx_above = create_tx_paying_to(&addr2, 0xa5); let block_above = make_block(vec![tx_above.clone()], 0xa5, 1400); - manager.process_block_for_wallets(&block_above, 2000, &wallets).await; + manager.process_block_for_wallets(&block_above, block_above.block_hash(), 2000, &wallets).await; let events_above = drain_events(&mut rx); let bp_above = events_above diff --git a/key-wallet-manager/src/process_block.rs b/key-wallet-manager/src/process_block.rs index 27ea4cba5..c340693ca 100644 --- a/key-wallet-manager/src/process_block.rs +++ b/key-wallet-manager/src/process_block.rs @@ -6,7 +6,7 @@ use core::fmt::Write as _; use dashcore::ephemerealdata::chain_lock::ChainLock; use dashcore::ephemerealdata::instant_lock::InstantLock; use dashcore::prelude::CoreBlockHeight; -use dashcore::{Address, Block, ScriptBuf, Transaction}; +use dashcore::{Address, Block, BlockHash, ScriptBuf, Transaction}; use key_wallet::account::AccountType; use key_wallet::managed_account::transaction_record::TransactionRecord; use key_wallet::transaction_checking::{BlockInfo, DerivedAddressInfo, TransactionContext}; @@ -20,6 +20,7 @@ impl WalletInterface for WalletM async fn process_block_for_wallets( &mut self, block: &Block, + block_hash: BlockHash, height: CoreBlockHeight, wallets: &BTreeSet, ) -> BlockProcessingResult { @@ -27,7 +28,7 @@ impl WalletInterface for WalletM if wallets.is_empty() { return result; } - let info = BlockInfo::new(height, block.block_hash(), block.header.time); + let info = BlockInfo::new(height, block_hash, block.header.time); // Late-block: when every wallet in scope already has its // finality boundary at or above this height, the block is @@ -587,7 +588,7 @@ mod tests { let mut rx = manager.subscribe_events(); let wallets = BTreeSet::from([wallet_id]); - manager.process_block_for_wallets(&block, 100, &wallets).await; + manager.process_block_for_wallets(&block, block.block_hash(), 100, &wallets).await; let mut found = false; while let Ok(event) = rx.try_recv() { @@ -646,18 +647,18 @@ mod tests { let block = make_block(vec![]); let only_w1 = BTreeSet::from([wallet_id1]); - manager.process_block_for_wallets(&block, 200, &only_w1).await; + manager.process_block_for_wallets(&block, block.block_hash(), 200, &only_w1).await; assert_eq!(manager.get_wallet_info(&wallet_id1).unwrap().last_processed_height(), 200); assert_eq!(manager.get_wallet_info(&wallet_id2).unwrap().last_processed_height(), 0); let only_w2 = BTreeSet::from([wallet_id2]); - manager.process_block_for_wallets(&block, 300, &only_w2).await; + manager.process_block_for_wallets(&block, block.block_hash(), 300, &only_w2).await; assert_eq!(manager.get_wallet_info(&wallet_id1).unwrap().last_processed_height(), 200); assert_eq!(manager.get_wallet_info(&wallet_id2).unwrap().last_processed_height(), 300); // Empty wallet set is a no-op even though the height is past both wallets. let none = BTreeSet::new(); - manager.process_block_for_wallets(&block, 1000, &none).await; + manager.process_block_for_wallets(&block, block.block_hash(), 1000, &none).await; assert_eq!(manager.get_wallet_info(&wallet_id1).unwrap().last_processed_height(), 200); assert_eq!(manager.get_wallet_info(&wallet_id2).unwrap().last_processed_height(), 300); } @@ -773,7 +774,9 @@ mod tests { let tx2 = create_tx_paying_to(&addr, 0xd1); let block = make_block(vec![tx2]); let block_wallets = BTreeSet::from([wallet_id]); - let _result = manager.process_block_for_wallets(&block, 100, &block_wallets).await; + let _result = manager + .process_block_for_wallets(&block, block.block_hash(), 100, &block_wallets) + .await; assert!( manager.monitor_revision() > rev_before_block, "block with tx paying to our address should bump revision (UTXO added)" diff --git a/key-wallet-manager/src/test_utils/mock_wallet.rs b/key-wallet-manager/src/test_utils/mock_wallet.rs index 2e9118625..21ee92d0c 100644 --- a/key-wallet-manager/src/test_utils/mock_wallet.rs +++ b/key-wallet-manager/src/test_utils/mock_wallet.rs @@ -128,6 +128,7 @@ impl WalletInterface for MockWallet { async fn process_block_for_wallets( &mut self, block: &Block, + _block_hash: dashcore::BlockHash, height: u32, wallets: &BTreeSet, ) -> BlockProcessingResult { @@ -290,6 +291,7 @@ impl WalletInterface for NonMatchingMockWallet { async fn process_block_for_wallets( &mut self, _block: &Block, + _block_hash: dashcore::BlockHash, height: u32, wallets: &BTreeSet, ) -> BlockProcessingResult { @@ -425,6 +427,7 @@ impl WalletInterface for MultiMockWallet { async fn process_block_for_wallets( &mut self, block: &Block, + _block_hash: dashcore::BlockHash, height: CoreBlockHeight, wallets: &BTreeSet, ) -> BlockProcessingResult { diff --git a/key-wallet-manager/src/wallet_interface.rs b/key-wallet-manager/src/wallet_interface.rs index 4974f8a75..99772b096 100644 --- a/key-wallet-manager/src/wallet_interface.rs +++ b/key-wallet-manager/src/wallet_interface.rs @@ -7,7 +7,7 @@ use async_trait::async_trait; use dashcore::ephemerealdata::chain_lock::ChainLock; use dashcore::ephemerealdata::instant_lock::InstantLock; use dashcore::prelude::CoreBlockHeight; -use dashcore::{Address, Block, OutPoint, ScriptBuf, Transaction, Txid}; +use dashcore::{Address, Block, BlockHash, OutPoint, ScriptBuf, Transaction, Txid}; use std::collections::{BTreeMap, BTreeSet}; use tokio::sync::broadcast; @@ -68,6 +68,7 @@ pub trait WalletInterface: Send + Sync + 'static { async fn process_block_for_wallets( &mut self, block: &Block, + block_hash: BlockHash, height: CoreBlockHeight, wallets: &BTreeSet, ) -> BlockProcessingResult; diff --git a/key-wallet-manager/tests/spv_integration_tests.rs b/key-wallet-manager/tests/spv_integration_tests.rs index fe26aa8a9..46ada40f3 100644 --- a/key-wallet-manager/tests/spv_integration_tests.rs +++ b/key-wallet-manager/tests/spv_integration_tests.rs @@ -17,7 +17,7 @@ async fn process_block_all_wallets( height: u32, ) -> BlockProcessingResult { let wallet_ids: BTreeSet = manager.list_wallets().into_iter().copied().collect(); - manager.process_block_for_wallets(block, height, &wallet_ids).await + manager.process_block_for_wallets(block, block.block_hash(), height, &wallet_ids).await } #[tokio::test] From 788c19d3f8d15ef9287a4b7b6cb1ef2e43645992 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Thu, 2 Jul 2026 21:37:28 +1000 Subject: [PATCH 019/149] fix(key-wallet): reserve selected UTXOs to prevent double-spend (#812) * fix(key-wallet): reserve selected UTXOs to prevent concurrent double-spend A freshly built transaction is not reflected in the UTXO set until its broadcast is processed back into the wallet, so a second build in that window can select the same coins and produce a double-spend the network rejects. - Add an ephemeral `ReservationSet` on `ManagedCoreFundsAccount`: `set_funding` skips reserved outpoints during coin selection and `assemble_unsigned` reserves the inputs it selects. - Release the reservation when the spend is processed, and on a failed sign, so coin selection is unaffected by a build that never reaches the network. - Reclaim stale reservations via a roughly one-hour (24-block) TTL backstop, skipping the sweep at height 0 where elapsed time is unknown. - Share the set via `Arc>` so a reservation outlives the wallet lock, and never persist it: after a restart, chain and mempool sync are the source of truth. - Cover the read, write, release, TTL, and height-0 paths with unit tests, plus a `dashd` integration test asserting two pre-broadcast builds select disjoint inputs. * docs: trim and clarify `ReservationSet` doc comments Drop the data-shape restatement from the `ReservationSet` struct doc and lead the `lock` method doc with its rationale rather than the method name. Make the height-0 sweep comment explicit that a height-0 entry relies on a later non-zero height to ever be reclaimed. * test: cover reservation release on the confirmed spend path `processing_a_spend_releases_its_reservation` only exercised the `Mempool` context. Extend it so a second funded outpoint is reserved and then spent under `TransactionContext::InBlock`, asserting the reservation is released on the confirmed path as well. * fix(key-wallet): release reservations when fee encoding fails `build_unsigned` reserved the selected inputs in `assemble_unsigned` and then encoded the transaction with `unwrap`, leaking the reservation until the TTL backstop if the encode ever failed. Route both builders through a fallible `encoded_size` helper that returns a `BuilderError` instead of panicking, and release the reserved inputs on that failure path so `build_unsigned` matches the existing release behavior in `build_signed`. * docs: note the builder must not be held across an await in `set_funding` Strengthen the `set_funding` lock invariant to state that the builder must not be held across an `await` before `build_signed` or `assemble_unsigned`, since suspending there reopens the read-then-reserve window for a concurrent build. * feat(key-wallet): release UTXO reservations for abandoned builds Add `ManagedCoreFundsAccount::release_reservation` so a caller can release the reservations held for a built transaction's inputs when that transaction will not be broadcast (e.g. the user cancelled). Without this the inputs stay reserved until the TTL backstop reclaims them, making the funds appear unspendable for about an hour. Broadcast transactions still release on their own once the spend is processed back into the wallet, so this is only for abandoned builds. --- .../tests/dashd_sync/tests_transaction.rs | 67 +++++++ .../managed_core_funds_account.rs | 37 +++- key-wallet/src/managed_account/mod.rs | 1 + key-wallet/src/managed_account/reservation.rs | 178 ++++++++++++++++++ key-wallet/src/tests/spent_outpoints_tests.rs | 63 ++++++- .../transaction_builder.rs | 175 +++++++++++++++-- 6 files changed, 506 insertions(+), 15 deletions(-) create mode 100644 key-wallet/src/managed_account/reservation.rs diff --git a/dash-spv/tests/dashd_sync/tests_transaction.rs b/dash-spv/tests/dashd_sync/tests_transaction.rs index 92f4931ff..0dfb5d7aa 100644 --- a/dash-spv/tests/dashd_sync/tests_transaction.rs +++ b/dash-spv/tests/dashd_sync/tests_transaction.rs @@ -1,5 +1,6 @@ use dash_spv::sync::ProgressPercentage; use dashcore::{Address, Amount, Network}; +use std::collections::HashSet; use std::sync::Arc; use std::time::Duration; use tokio::sync::RwLock; @@ -429,6 +430,72 @@ async fn test_spend_change_balance() { client_handle.stop().await; } +/// Two transactions built from the same wallet before either is broadcast must +/// not select the same UTXO. The first build reserves its inputs so the second +/// build's coin selection skips them, preventing a double-spend the network +/// would reject. Without the reservation both builds pick the same UTXO and the +/// `tx_a`/`tx_b` inputs overlap, so the disjoint-input assertion below fails. +/// +/// The two builds run sequentially: this guards that the first build's +/// reservation outlives its wallet lock and is observed by the second build, +/// not that the two builds are scheduled concurrently. +#[tokio::test] +async fn test_concurrent_builds_do_not_double_spend() { + let Some(ctx) = TestContext::new(TestChain::Minimal).await else { + return; + }; + if !ctx.dashd.supports_mining { + eprintln!("Skipping test (dashd RPC miner not available)"); + return; + } + + let (wallet, wallet_id) = create_test_wallet(EMPTY_MNEMONIC, Network::Regtest); + let mut client_handle = create_and_start_client(&ctx.client_config, Arc::clone(&wallet)).await; + wait_for_sync(&mut client_handle.progress_receiver, ctx.dashd.initial_height).await; + + // Fund the wallet with two separate, equal UTXOs at the same receive + // address. Either one alone covers a single send, so naive coin selection + // would pick the same UTXO for both builds. + let receive_address = reserve_first_address(EMPTY_MNEMONIC).await; + let utxo_amount = Amount::from_sat(200_000_000); + ctx.dashd.node.send_to_address(&receive_address, utxo_amount); + ctx.dashd.node.send_to_address(&receive_address, utxo_amount); + + let miner_address = ctx.dashd.node.get_new_address_from_wallet("default"); + ctx.dashd.node.generate_blocks(1, &miner_address); + let funded_height = ctx.dashd.initial_height + 1; + wait_for_sync(&mut client_handle.progress_receiver, funded_height).await; + wait_for_wallet_synced(&wallet, &wallet_id, funded_height).await; + + // Build both transactions before broadcasting either, so neither spend has + // been processed back into the wallet when the second build selects inputs. + let dest_a = Address::dummy(Network::Regtest, 1); + let dest_b = Address::dummy(Network::Regtest, 2); + let (tx_a, _) = + build_and_sign(&wallet, &wallet_id, &dest_a, 50_000_000).await.expect("build tx_a"); + let (tx_b, _) = + build_and_sign(&wallet, &wallet_id, &dest_b, 50_000_000).await.expect("build tx_b"); + + let inputs_a: HashSet<_> = tx_a.input.iter().map(|i| i.previous_output).collect(); + let overlap = tx_b.input.iter().any(|i| inputs_a.contains(&i.previous_output)); + assert!(!overlap, "tx_a and tx_b must not share an input (double-spend)"); + + // Both broadcasts succeed and both transactions reach the mempool: a + // double-spend would have the second rejected by the network. + client_handle.client.broadcast_transaction(&tx_a).await.expect("broadcast tx_a"); + let detected_a = wait_for_mempool_tx(&mut client_handle.wallet_event_receiver, MEMPOOL_TIMEOUT) + .await + .expect("detect tx_a"); + assert_eq!(detected_a, tx_a.txid()); + client_handle.client.broadcast_transaction(&tx_b).await.expect("broadcast tx_b"); + let detected_b = wait_for_mempool_tx(&mut client_handle.wallet_event_receiver, MEMPOOL_TIMEOUT) + .await + .expect("detect tx_b"); + assert_eq!(detected_b, tx_b.txid()); + + client_handle.stop().await; +} + /// Spend an incoming mempool UTXO (we own the output, none of the inputs) /// before it confirms. #[tokio::test] diff --git a/key-wallet/src/managed_account/managed_core_funds_account.rs b/key-wallet/src/managed_account/managed_core_funds_account.rs index cf95d78d1..eb4f9dd98 100644 --- a/key-wallet/src/managed_account/managed_core_funds_account.rs +++ b/key-wallet/src/managed_account/managed_core_funds_account.rs @@ -19,6 +19,7 @@ use crate::managed_account::address_pool; use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::managed_account::managed_account_type::ManagedAccountType; use crate::managed_account::managed_core_keys_account::ManagedCoreKeysAccount; +use crate::managed_account::reservation::ReservationSet; use crate::managed_account::transaction_record::{ InputDetail, OutputDetail, OutputRole, TransactionDirection, }; @@ -61,6 +62,12 @@ pub struct ManagedCoreFundsAccount { /// Rebuilt from `transactions` during deserialization. #[cfg_attr(feature = "serde", serde(skip_serializing))] spent_outpoints: HashSet, + /// Outpoints reserved by in-flight transaction builds so concurrent builds + /// do not select the same UTXO before the first build's transaction is + /// processed. Empty after a restart, where chain and mempool sync + /// re-establish which coins are spent. + #[cfg_attr(feature = "serde", serde(skip))] + reservations: ReservationSet, } impl ManagedCoreFundsAccount { @@ -71,6 +78,7 @@ impl ManagedCoreFundsAccount { balance: WalletCoreBalance::default(), utxos: BTreeMap::new(), spent_outpoints: HashSet::new(), + reservations: ReservationSet::default(), } } @@ -97,9 +105,31 @@ impl ManagedCoreFundsAccount { balance: WalletCoreBalance::default(), utxos: BTreeMap::new(), spent_outpoints: HashSet::new(), + reservations: ReservationSet::default(), } } + /// Reservation set tracking outpoints chosen by in-flight transaction + /// builds, consulted by coin selection so concurrent builds do not pick the + /// same UTXO. + /// + /// The reservation is only effective when each build runs `set_funding` + /// through `assemble_unsigned` under a single uninterrupted hold of the + /// wallet lock. If two builds interleave between observing the UTXO set and + /// reserving their inputs, both can see the same UTXO as free and select it. + pub(crate) fn reservations(&self) -> &ReservationSet { + &self.reservations + } + + /// Release the reservations held for `tx`'s inputs. Call this when a built + /// transaction will not be broadcast, e.g. the user cancelled, so its inputs + /// become selectable again immediately instead of waiting out the TTL + /// backstop. Broadcast transactions release on their own once the spend is + /// processed back into the wallet, so this is only for abandoned builds. + pub fn release_reservation(&self, tx: &Transaction) { + self.reservations.release(tx.input.iter().map(|input| &input.previous_output)); + } + /// Get a reference to the inner keys-account state. pub fn keys(&self) -> &ManagedCoreKeysAccount { &self.keys @@ -208,7 +238,11 @@ impl ManagedCoreFundsAccount { } } - // Remove UTXOs spent by this transaction and track spent outpoints + // Remove UTXOs spent by this transaction and track spent outpoints. + // Processing the spend also hands the inputs off from the + // ephemeral build reservation to the durable spent set, so the + // reservation taken when this transaction was built is released. + self.reservations.release(tx.input.iter().map(|input| &input.previous_output)); for input in &tx.input { self.spent_outpoints.insert(input.previous_output); @@ -726,6 +760,7 @@ impl<'de> Deserialize<'de> for ManagedCoreFundsAccount { balance: helper.balance, utxos: helper.utxos, spent_outpoints, + reservations: ReservationSet::default(), }) } } diff --git a/key-wallet/src/managed_account/mod.rs b/key-wallet/src/managed_account/mod.rs index 25b483624..5d1766c5f 100644 --- a/key-wallet/src/managed_account/mod.rs +++ b/key-wallet/src/managed_account/mod.rs @@ -21,6 +21,7 @@ pub mod managed_core_funds_account; pub mod managed_core_keys_account; pub mod managed_platform_account; pub mod platform_address; +pub(crate) mod reservation; pub mod transaction_record; pub use managed_account_ref::{ManagedAccountRef, ManagedAccountRefMut, OwnedManagedCoreAccount}; diff --git a/key-wallet/src/managed_account/reservation.rs b/key-wallet/src/managed_account/reservation.rs new file mode 100644 index 000000000..db6fd9adf --- /dev/null +++ b/key-wallet/src/managed_account/reservation.rs @@ -0,0 +1,178 @@ +//! Ephemeral reservation of UTXOs selected by in-flight transaction builds. +//! +//! Coin selection reads the UTXO set, but a freshly built transaction is not +//! reflected in that set until the broadcast transaction is processed back into +//! the wallet (its inputs leave `utxos`). In the window between selecting inputs +//! and that processing, a second build sees the same UTXOs and can pick them +//! again, producing a double-spend the network rejects. +//! +//! A [`ReservationSet`] bridges exactly that window: selected inputs are +//! reserved at build time and skipped by subsequent coin selection. The set is +//! shared via `Arc>` so a reservation taken while the wallet lock is +//! held survives the lock being released and is observed by the next build. It +//! is never persisted: after a restart it is empty, where chain and mempool +//! sync are the source of truth for which coins are already spent. + +use std::collections::{HashMap, HashSet}; +use std::sync::{Arc, Mutex, MutexGuard}; + +use dashcore::blockdata::transaction::OutPoint; + +/// Number of blocks after which a stale reservation is reclaimed, about one +/// hour at the mainnet 2.5-minute block target. +/// +/// A reservation is normally released the moment its broadcast transaction is +/// processed (the input leaves the UTXO set), and a restart drops all +/// reservations, so this backstop only matters for a reservation whose +/// transaction was built but never processed in a long-running process, e.g. a +/// silently failed broadcast. The value is kept comfortably above the real +/// selection-to-processing latency (seconds): reclaiming a still-in-flight +/// reservation too early would let coin selection re-pick the input and rebuild +/// the very double-spend this guards against. +const RESERVATION_TTL_BLOCKS: u32 = 24; + +/// Ephemeral, in-memory set of reserved outpoints. Cloning shares the +/// underlying state, which is what lets a build's reservation outlive the +/// wallet lock and be seen by the next build. +#[derive(Debug, Clone, Default)] +pub(crate) struct ReservationSet { + inner: Arc>>, +} + +impl ReservationSet { + /// Recovers from a poisoned mutex rather than panicking: the guarded data is + /// a plain `HashMap` with no invariant a partial write could break, and + /// panicking here would strand all later coin selection in a long-running + /// node. + fn lock(&self) -> MutexGuard<'_, HashMap> { + self.inner.lock().unwrap_or_else(|poisoned| poisoned.into_inner()) + } + + fn sweep(reserved: &mut HashMap, current_height: u32) { + // Height 0 means the wallet has no processed height yet, so the elapsed + // span is unknown and no entry can be reliably judged stale. An entry + // stamped at height 0 therefore relies on a later non-zero height being + // presented before it can ever be reclaimed by the TTL backstop. + if current_height == 0 { + return; + } + reserved.retain(|_, reserved_at| { + current_height.saturating_sub(*reserved_at) < RESERVATION_TTL_BLOCKS + }); + } + + /// Reserve `outpoints` as of `current_height`, dropping expired entries + /// first. Re-reserving an outpoint refreshes its height. + pub(crate) fn reserve(&self, outpoints: &[OutPoint], current_height: u32) { + let mut reserved = self.lock(); + Self::sweep(&mut reserved, current_height); + for outpoint in outpoints { + reserved.insert(*outpoint, current_height); + } + } + + /// Return the currently reserved outpoints, dropping expired entries first. + pub(crate) fn reserved(&self, current_height: u32) -> HashSet { + let mut reserved = self.lock(); + Self::sweep(&mut reserved, current_height); + reserved.keys().copied().collect() + } + + /// Release the given outpoints. Idempotent: releasing an outpoint that is + /// not reserved does nothing. + pub(crate) fn release<'a>(&self, outpoints: impl IntoIterator) { + let mut reserved = self.lock(); + for outpoint in outpoints { + reserved.remove(outpoint); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use dashcore::Txid; + + fn outpoint(byte: u8, vout: u32) -> OutPoint { + OutPoint::new(Txid::from([byte; 32]), vout) + } + + #[test] + fn reserve_then_skip_then_release() { + let set = ReservationSet::default(); + let a = outpoint(0x01, 0); + let b = outpoint(0x02, 1); + + set.reserve(&[a], 100); + assert!(set.reserved(100).contains(&a)); + assert!(!set.reserved(100).contains(&b)); + assert_eq!(set.reserved(100), HashSet::from([a])); + + set.release([&a]); + assert!(set.reserved(100).is_empty()); + } + + #[test] + fn release_is_idempotent() { + let set = ReservationSet::default(); + let a = outpoint(0x03, 0); + // Releasing an unreserved outpoint is a no-op. + set.release([&a]); + set.reserve(&[a], 10); + set.release([&a]); + set.release([&a]); + assert!(!set.reserved(10).contains(&a)); + } + + #[test] + fn ttl_reclaims_stale_reservation_by_height() { + let set = ReservationSet::default(); + let a = outpoint(0x04, 0); + set.reserve(&[a], 100); + + // The boundary is exclusive, so the entry survives until exactly + // `reserved_at + RESERVATION_TTL_BLOCKS`. + assert_eq!(set.reserved(100 + RESERVATION_TTL_BLOCKS - 1), HashSet::from([a])); + // At that height the sweep's strict less-than drops the entry. + assert!(set.reserved(100 + RESERVATION_TTL_BLOCKS).is_empty()); + } + + #[test] + fn zero_height_disables_sweep() { + let set = ReservationSet::default(); + let a = outpoint(0x07, 0); + set.reserve(&[a], 0); + + // Height 0 means the wallet has no processed height yet, so the elapsed + // span is unknown and the sweep is suppressed: the reservation stands. + assert!(set.reserved(0).contains(&a)); + + // Once a real height is known the TTL backstop applies again. + set.reserve(&[a], 1); + assert!(set.reserved(1 + RESERVATION_TTL_BLOCKS).is_empty()); + } + + #[test] + fn re_reserving_refreshes_the_ttl_height() { + let set = ReservationSet::default(); + let a = outpoint(0x06, 0); + set.reserve(&[a], 100); + // A later reserve replaces the stored height, so the TTL is measured + // from the most recent reservation, not the first. + set.reserve(&[a], 150); + + assert_eq!(set.reserved(150 + RESERVATION_TTL_BLOCKS - 1), HashSet::from([a])); + assert!(set.reserved(150 + RESERVATION_TTL_BLOCKS).is_empty()); + } + + #[test] + fn clone_shares_state() { + let set = ReservationSet::default(); + let clone = set.clone(); + let a = outpoint(0x05, 0); + // A reservation taken on one handle is visible through the other, which + // is what lets a build's reservation outlive the wallet lock. + set.reserve(&[a], 1); + assert!(clone.reserved(1).contains(&a)); + } +} diff --git a/key-wallet/src/tests/spent_outpoints_tests.rs b/key-wallet/src/tests/spent_outpoints_tests.rs index cf8215f58..6e80f18fd 100644 --- a/key-wallet/src/tests/spent_outpoints_tests.rs +++ b/key-wallet/src/tests/spent_outpoints_tests.rs @@ -1,13 +1,15 @@ //! Tests for spent_outpoints deserialization and tracking. use dashcore::blockdata::transaction::{OutPoint, Transaction}; -use dashcore::{TxIn, Txid}; +use dashcore::hashes::Hash; +use dashcore::{BlockHash, TxIn, Txid}; use crate::account::{AccountType, StandardAccountType, TransactionRecord}; use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::managed_account::transaction_record::TransactionDirection; use crate::managed_account::ManagedCoreFundsAccount; -use crate::transaction_checking::{TransactionContext, TransactionType}; +use crate::test_utils::TestWalletContext; +use crate::transaction_checking::{BlockInfo, TransactionContext, TransactionType}; /// Create a transaction that spends the given outpoints. fn spending_tx(spent: &[OutPoint]) -> Transaction { @@ -70,6 +72,63 @@ fn fresh_account_has_empty_spent_outpoints() { let _ = probe; // used only for clarity of intent } +#[test] +fn reservations_are_not_persisted() { + let account = ManagedCoreFundsAccount::dummy_bip44(); + let outpoint = OutPoint::new(Txid::from([0x42; 32]), 0); + + account.reservations().reserve(&[outpoint], 0); + assert!(account.reservations().reserved(0).contains(&outpoint)); + + let json = serde_json::to_string(&account).unwrap(); + assert!(!json.contains("reservations")); + + // After a round-trip the reservation set is empty: it is ephemeral, and on + // restart chain/mempool sync is the source of truth for spent coins. + let deserialized: ManagedCoreFundsAccount = serde_json::from_str(&json).unwrap(); + assert!(!deserialized.reservations().reserved(0).contains(&outpoint)); +} + +#[tokio::test] +async fn processing_a_spend_releases_its_reservation() { + let (mut ctx, funding_tx) = TestWalletContext::new_random().with_mempool_funding(150_000).await; + let funded = OutPoint::new(funding_tx.txid(), 0); + + let account = ctx.managed_wallet.first_bip44_managed_account_mut().expect("BIP44 account"); + assert!(account.utxos.contains_key(&funded)); + account.reservations().reserve(&[funded], 0); + assert!(account.reservations().reserved(0).contains(&funded)); + + let spend = spending_tx(&[funded]); + ctx.check_transaction(&spend, TransactionContext::Mempool).await; + + let account = ctx.managed_wallet.first_bip44_managed_account().expect("BIP44 account"); + assert!(!account.reservations().reserved(0).contains(&funded)); + + // The confirmed path releases reservations too: a separate funding tx with + // a distinct input range yields a second outpoint that is reserved and then + // spent in a block. + let second_funding = Transaction::dummy(&ctx.receive_address, 1..2, &[120_000]); + ctx.check_transaction(&second_funding, TransactionContext::Mempool).await; + let second_funded = OutPoint::new(second_funding.txid(), 0); + + let account = ctx.managed_wallet.first_bip44_managed_account_mut().expect("BIP44 account"); + assert!(account.utxos.contains_key(&second_funded)); + account.reservations().reserve(&[second_funded], 0); + assert!(account.reservations().reserved(0).contains(&second_funded)); + + let block_hash = BlockHash::from_slice(&[7u8; 32]).expect("hash"); + let confirmed_spend = spending_tx(&[second_funded]); + ctx.check_transaction( + &confirmed_spend, + TransactionContext::InBlock(BlockInfo::new(100, block_hash, 1_700_000_000)), + ) + .await; + + let account = ctx.managed_wallet.first_bip44_managed_account().expect("BIP44 account"); + assert!(!account.reservations().reserved(0).contains(&second_funded)); +} + #[test] fn serde_round_trip_rebuilds_spent_outpoints() { let mut account = ManagedCoreFundsAccount::dummy_bip44(); diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index d86ea7649..65d628e85 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -3,6 +3,7 @@ //! This module provides high-level transaction building functionality //! using types from the dashcore crate. +use crate::managed_account::reservation::ReservationSet; use crate::managed_account::ManagedCoreFundsAccount; use crate::wallet::managed_wallet_info::coin_selection::{CoinSelector, SelectionStrategy}; use crate::wallet::managed_wallet_info::fee::FeeRate; @@ -10,7 +11,7 @@ use crate::{Account, DerivationPath, Signer, Utxo, Wallet}; use core::fmt; use dashcore::blockdata::script::{Builder, PushBytes, ScriptBuf}; use dashcore::blockdata::transaction::special_transaction::TransactionPayload; -use dashcore::blockdata::transaction::Transaction; +use dashcore::blockdata::transaction::{OutPoint, Transaction}; use dashcore::consensus::Encodable; use dashcore::sighash::{EcdsaSighashType, LegacySighash, SighashCache}; use dashcore::Address; @@ -24,6 +25,15 @@ use std::cmp::Ordering; /// bytes/signed input) and be rejected by the network const MAX_STANDARD_TX_INPUTS: usize = 500; +/// Consensus-encoded byte length of `tx`, used to compute the fee from the real +/// serialized size. Surfaces an encode error instead of panicking so the caller +/// can release any reservations it took rather than stranding the inputs. +fn encoded_size(tx: &Transaction) -> Result { + let mut bytes = Vec::new(); + tx.consensus_encode(&mut bytes) + .map_err(|err| BuilderError::InvalidData(format!("failed to encode transaction: {}", err))) +} + /// Calculate varint size for a given number fn varint_size(n: usize) -> usize { match n { @@ -48,6 +58,10 @@ pub struct TransactionBuilder { selection_strategy: SelectionStrategy, /// Special transaction payload for Dash-specific transactions special_payload: Option, + /// Reservation set of the funding account, captured by `set_funding`. The + /// inputs chosen during assembly are reserved here so a concurrent build + /// skips them until the broadcast transaction is processed. + reservations: Option, } impl Default for TransactionBuilder { @@ -67,6 +81,7 @@ impl TransactionBuilder { current_height: 0, selection_strategy: SelectionStrategy::BranchAndBound, special_payload: None, + reservations: None, } } @@ -80,8 +95,25 @@ impl TransactionBuilder { self } + /// Seed the builder with the funding account's spendable UTXOs, skipping any + /// already reserved by another in-flight build. + /// + /// This call and the later `assemble_unsigned` that reserves the chosen + /// inputs must run under one uninterrupted hold of the wallet lock. If two + /// builds interleave between here and their reservation, both can observe the + /// same UTXO as free and select it, defeating the reservation. The builder + /// must therefore not be held across an `await` between `set_funding` and + /// `build_signed` or `assemble_unsigned`, since suspending there reopens the + /// read-then-reserve window for a concurrent build. pub fn set_funding(mut self, funds_acc: &mut ManagedCoreFundsAccount, acc: &Account) -> Self { - self.inputs = funds_acc.utxos.values().cloned().collect(); + let reserved = funds_acc.reservations().reserved(self.current_height); + self.inputs = funds_acc + .utxos + .values() + .filter(|utxo| !reserved.contains(&utxo.outpoint)) + .cloned() + .collect(); + self.reservations = Some(funds_acc.reservations().clone()); self.change_addr = funds_acc.next_change_address(Some(&acc.account_xpub), true).ok(); self } @@ -351,6 +383,15 @@ impl TransactionBuilder { special_transaction_payload: self.special_payload, }; + // Reserve the chosen inputs so a concurrent build skips them until the + // broadcast transaction is processed back into the wallet (which + // releases the reservation) or the TTL backstop reclaims it. + if let Some(reservations) = &self.reservations { + let outpoints: Vec = + selected_inputs.iter().map(|utxo| utxo.outpoint).collect(); + reservations.reserve(&outpoints, self.current_height); + } + return Ok((transaction, selected_inputs)); // BIP-69: Sort outputs by amount first, then by scriptPubKey @@ -376,13 +417,19 @@ impl TransactionBuilder { pub fn build_unsigned(self) -> Result<(Transaction, u64), BuilderError> { let fee_rate = self.fee_rate; + let reservations = self.reservations.clone(); - let (tx, _) = self.assemble_unsigned()?; - - let mut tx_bytes = Vec::new(); - tx.consensus_encode(&mut tx_bytes).unwrap(); + let (tx, inputs) = self.assemble_unsigned()?; - let fee = fee_rate.calculate_fee(tx_bytes.len()); + let fee = match encoded_size(&tx) { + Ok(size) => fee_rate.calculate_fee(size), + Err(err) => { + if let Some(reservations) = &reservations { + reservations.release(inputs.iter().map(|utxo| &utxo.outpoint)); + } + return Err(err); + } + }; Ok((tx, fee)) } @@ -400,14 +447,33 @@ impl TransactionBuilder { P: Fn(Address) -> Option + Send, { let fee_rate = self.fee_rate; + let reservations = self.reservations.clone(); let (tx, inputs) = self.assemble_unsigned()?; - let tx = signer.sign_tx(tx, inputs, path_resolver).await?; - - let mut tx_bytes = Vec::new(); - tx.consensus_encode(&mut tx_bytes).unwrap(); + // Signing never reaches the network for a local key, but an external + // signer can fail. A failed sign means the reserved inputs are still + // spendable, so release them now instead of stranding the funds until + // the TTL backstop reclaims them. + let reserved: Vec = inputs.iter().map(|utxo| utxo.outpoint).collect(); + let tx = match signer.sign_tx(tx, inputs, path_resolver).await { + Ok(tx) => tx, + Err(err) => { + if let Some(reservations) = &reservations { + reservations.release(reserved.iter()); + } + return Err(err); + } + }; - let fee = fee_rate.calculate_fee(tx_bytes.len()); + let fee = match encoded_size(&tx) { + Ok(size) => fee_rate.calculate_fee(size), + Err(err) => { + if let Some(reservations) = &reservations { + reservations.release(reserved.iter()); + } + return Err(err); + } + }; Ok((tx, fee)) } @@ -590,6 +656,7 @@ impl std::error::Error for BuilderError {} #[cfg(test)] mod tests { use super::*; + use crate::test_utils::TestWalletContext; use crate::Network; use dashcore::blockdata::transaction::special_transaction::asset_lock::AssetLockPayload; use dashcore::{OutPoint, Txid}; @@ -1006,4 +1073,88 @@ mod tests { "Should select multiple inputs to cover fees for special payload" ); } + + #[test] + fn set_funding_skips_reserved_utxos() { + let ctx = TestWalletContext::new_random(); + let account = + ctx.wallet.accounts.standard_bip44_accounts.get(&0).expect("BIP44 account").clone(); + + let mut funds = ManagedCoreFundsAccount::dummy_bip44(); + let reserved = Utxo::dummy(0x01, 100_000, 100, false, true); + let free = Utxo::dummy(0x02, 200_000, 100, false, true); + funds.utxos.insert(reserved.outpoint, reserved.clone()); + funds.utxos.insert(free.outpoint, free.clone()); + + funds.reservations().reserve(&[reserved.outpoint], 200); + + let builder = + TransactionBuilder::new().set_current_height(200).set_funding(&mut funds, &account); + + let candidates: Vec = builder.inputs.iter().map(|utxo| utxo.outpoint).collect(); + assert!(!candidates.contains(&reserved.outpoint)); + assert!(candidates.contains(&free.outpoint)); + } + + #[tokio::test] + async fn build_signed_releases_reservation_on_signing_failure() { + let ctx = TestWalletContext::new_random(); + let account = + ctx.wallet.accounts.standard_bip44_accounts.get(&0).expect("BIP44 account").clone(); + + let mut funds = ManagedCoreFundsAccount::dummy_bip44(); + let utxo = Utxo::dummy(0x01, 1_000_000, 100, false, true); + funds.utxos.insert(utxo.outpoint, utxo.clone()); + + let destination = Address::dummy(Network::Testnet, 0); + let builder = TransactionBuilder::new() + .set_current_height(200) + .set_fee_rate(FeeRate::normal()) + .set_funding(&mut funds, &account) + .set_change_address(Address::dummy(Network::Testnet, 1)) + .add_output(&destination, 500_000); + + // A resolver that never finds a derivation path forces signing to fail + // after the inputs have already been reserved by `assemble_unsigned`. + let result = builder.build_signed(&ctx.wallet, |_addr| None).await; + assert!(result.is_err()); + + // The failed sign must leave no reservation behind, so the UTXO stays + // selectable instead of being stranded until the TTL backstop. + assert!(funds.reservations().reserved(200).is_empty()); + } + + #[test] + fn build_unsigned_reserves_selected_inputs() { + let ctx = TestWalletContext::new_random(); + let account = + ctx.wallet.accounts.standard_bip44_accounts.get(&0).expect("BIP44 account").clone(); + + let mut funds = ManagedCoreFundsAccount::dummy_bip44(); + let utxo = Utxo::dummy(0x01, 1_000_000, 100, false, true); + funds.utxos.insert(utxo.outpoint, utxo.clone()); + + let destination = Address::dummy(Network::Testnet, 0); + let (tx, _) = TransactionBuilder::new() + .set_current_height(200) + .set_fee_rate(FeeRate::normal()) + .set_funding(&mut funds, &account) + .set_change_address(Address::dummy(Network::Testnet, 1)) + .add_output(&destination, 500_000) + .build_unsigned() + .expect("build unsigned"); + + // Every input the build selected is reserved, so a later build observes + // them as taken and skips them. + let reserved = funds.reservations().reserved(200); + assert!(!reserved.is_empty()); + for input in &tx.input { + assert!(reserved.contains(&input.previous_output)); + } + + // Abandoning the build releases its inputs immediately rather than + // stranding them until the TTL backstop reclaims them. + funds.release_reservation(&tx); + assert!(funds.reservations().reserved(200).is_empty()); + } } From afcff1566c81abb0a9cf64b99b0ee675169fddfc Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Thu, 2 Jul 2026 17:24:55 +0400 Subject: [PATCH 020/149] refactor(key-wallet): move extended_public_key out of Signer into ExtendedPubKeySigner subtrait (#839) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * refactor(key-wallet): move extended_public_key out of Signer into ExtendedPubKeySigner Signer::extended_public_key (added in #817) forced every signer to implement xpub export, but exporting a chain code is a key-export capability, not a signing one: pure signing backends (HSMs, remote signing services whose policy forbids chain-code export) could not implement Signer without stubbing the method with an error. Move the method unchanged into a new ExtendedPubKeySigner: Signer subtrait. Callers that need offline descendant derivation (e.g. DashPay contact xpubs) bound on the subtrait; everything else keeps the minimal Signer. Reusing the Signer supertrait keeps a single associated Error type. No production code called the method, so only the two InMemorySigner test signers move their impl to the new trait, and the unreachable NoDigestSigner stubs are deleted outright — evidence the requirement never belonged on the base trait. Co-Authored-By: Claude Fable 5 * test(key-wallet): pin asset-lock InMemorySigner xpub export against wallet derivation The asset_lock_builder test module duplicates the InMemorySigner ExtendedPubKeySigner impl but had no correctness check, unlike its twin in transaction_building.rs. Mirror the hardened-path assertion so both copies stay verified against Wallet::derive_extended_public_key. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- key-wallet/src/lib.rs | 2 +- key-wallet/src/signer.rs | 17 +++++++-- .../managed_wallet_info/asset_lock_builder.rs | 38 +++++++++++++++---- .../transaction_building.rs | 11 ++---- 4 files changed, 50 insertions(+), 18 deletions(-) diff --git a/key-wallet/src/lib.rs b/key-wallet/src/lib.rs index a9cb892b3..89b64b949 100644 --- a/key-wallet/src/lib.rs +++ b/key-wallet/src/lib.rs @@ -62,7 +62,7 @@ pub use managed_account::managed_platform_account::ManagedPlatformAccount; pub use managed_account::platform_address::PlatformP2PKHAddress; pub use mnemonic::{Language, Mnemonic}; pub use seed::Seed; -pub use signer::{Signer, SignerMethod, TransactionCategory}; +pub use signer::{ExtendedPubKeySigner, Signer, SignerMethod, TransactionCategory}; pub use utxo::Utxo; pub use wallet::{balance::WalletCoreBalance, Wallet}; diff --git a/key-wallet/src/signer.rs b/key-wallet/src/signer.rs index 3435289aa..05ca2b920 100644 --- a/key-wallet/src/signer.rs +++ b/key-wallet/src/signer.rs @@ -125,7 +125,18 @@ pub trait Signer: Send + Sync { /// keys) that the caller later references when signing Platform state /// transitions. async fn public_key(&self, path: &DerivationPath) -> Result; +} +/// A [`Signer`] that can additionally export BIP-32 extended public keys. +/// +/// Kept separate from [`Signer`] because exporting an extended public key +/// is a key-export capability, not a signing one: a pure signing backend +/// (an HSM or remote signing service whose policy forbids exporting chain +/// codes, since an xpub plus any descendant non-hardened private key +/// compromises the whole subtree) can implement [`Signer`] alone, while +/// callers that need offline descendant derivation bound on this trait. +#[async_trait] +pub trait ExtendedPubKeySigner: Signer { /// Return the BIP-32 extended public key at `path` — the public point /// plus the chain code and parent fingerprint, so the caller can /// non-hardened-derive a whole range of descendants from a single @@ -133,9 +144,9 @@ pub trait Signer: Send + Sync { /// (e.g. DashPay contact payment addresses under /// `m/9'/coin'/15'/account'/sender/recipient`). /// - /// Distinct from [`Self::public_key`], which returns only the leaf point - /// (no chain code). A signer that cannot export an extended public key at - /// a hardened path should return an error rather than panic. + /// Distinct from [`Signer::public_key`], which returns only the leaf + /// point (no chain code). A signer that cannot export an extended public + /// key at a hardened path should return an error rather than panic. async fn extended_public_key( &self, path: &DerivationPath, diff --git a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs index 56c9dad4c..05753d336 100644 --- a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs @@ -344,7 +344,7 @@ impl ManagedWalletInfo { #[cfg(test)] mod tests { use super::*; - use crate::signer::SignerMethod; + use crate::signer::{ExtendedPubKeySigner, SignerMethod}; use crate::wallet::initialization::WalletAccountCreationOptions; use crate::Network; use dashcore::ScriptBuf; @@ -469,7 +469,10 @@ mod tests { .map_err(|e| e.to_string())?; Ok(secp256k1::PublicKey::from_secret_key(&secp, &xpriv.private_key)) } + } + #[async_trait::async_trait] + impl ExtendedPubKeySigner for InMemorySigner { async fn extended_public_key( &self, path: &DerivationPath, @@ -484,6 +487,33 @@ mod tests { } } + #[tokio::test] + async fn in_memory_signer_extended_public_key_matches_wallet_derivation() { + use std::str::FromStr; + let (wallet, _info) = test_wallet_and_info(); + let root = match &wallet.wallet_type { + crate::wallet::WalletType::Mnemonic { + root_extended_private_key, + .. + } => root_extended_private_key.clone(), + _ => unreachable!("test_wallet_and_info produces a mnemonic wallet"), + }; + let signer = InMemorySigner { + root, + network: Network::Testnet, + }; + // A hardened path — only derivable with the private key, which is the + // whole point of exposing extended-pubkey export on the signer. + let path = DerivationPath::from_str("m/9'/1'/15'/0'").expect("valid path"); + let from_signer = + signer.extended_public_key(&path).await.expect("signer extended_public_key"); + let from_wallet = wallet.derive_extended_public_key(&path).expect("wallet extended pubkey"); + assert_eq!( + from_signer, from_wallet, + "signer xpub at a hardened path must equal the wallet's own derivation" + ); + } + #[tokio::test] async fn test_signer_empty_credit_outputs_rejected() { let (wallet, mut info) = test_wallet_and_info(); @@ -550,12 +580,6 @@ mod tests { async fn public_key(&self, _: &DerivationPath) -> Result { unreachable!() } - async fn extended_public_key( - &self, - _: &DerivationPath, - ) -> Result { - unreachable!() - } } let (wallet, mut info) = test_wallet_and_info(); diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index 1b1833b05..1a9cb8c4c 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -387,7 +387,7 @@ mod tests { use super::super::transaction_builder::BuilderError; use super::super::wallet_info_interface::WalletInfoInterface; - use crate::signer::{Signer, SignerMethod}; + use crate::signer::{ExtendedPubKeySigner, Signer, SignerMethod}; use crate::wallet::initialization::WalletAccountCreationOptions; use crate::wallet::ManagedWalletInfo; use crate::DerivationPath; @@ -447,7 +447,10 @@ mod tests { .map_err(|e| e.to_string())?; Ok(secp256k1::PublicKey::from_secret_key(&secp, &xpriv.private_key)) } + } + #[async_trait::async_trait] + impl ExtendedPubKeySigner for InMemorySigner { async fn extended_public_key( &self, path: &DerivationPath, @@ -543,12 +546,6 @@ mod tests { async fn public_key(&self, _: &DerivationPath) -> Result { unreachable!() } - async fn extended_public_key( - &self, - _: &DerivationPath, - ) -> Result { - unreachable!() - } } let (wallet, mut info) = test_wallet_and_info(); From b0a909c76af7e7c13320a6c13acfe3bb89b9fba9 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Fri, 3 Jul 2026 00:07:05 +1000 Subject: [PATCH 021/149] ci: fix `ready-for-review` label automation for fork pull requests (#838) * ci: fix `ready-for-review` label automation for fork pull requests Fork pull requests run the `pull_request` and `pull_request_review` events with a read-only `GITHUB_TOKEN` and no access to secrets, so the `evaluate` job could not mutate labels and failed with "Resource not accessible by integration" when it tried to add `ready-for-review` after CodeRabbit approved. The privileged `workflow_run` path that should have covered this was a silent no-op for forks because `github.event.workflow_run.pull_requests` is always empty for cross-repository PRs. Restructure the automation around the `workflow_run` relay pattern. A new lightweight `Ready for Review Trigger` workflow runs on `pull_request` and `pull_request_review`, does nothing but complete, and thereby fires a `workflow_run` event. The `Ready for Review Label` workflow now runs only on `workflow_run`, where it holds a write-capable token even for forks, resolves the PR by matching the head sha against the open pull requests (the commit-to-PR endpoint does not index fork heads), and applies or removes the label. No new secrets are required. Fold the dismissal, changes-requested and merge-conflict handling into the single re-evaluation path and exclude the relay job from the CI gate. * ci: grant `contents: read` to ready-for-review label workflow Setting a workflow-level `permissions` block drops the default `contents: read`, which both `actions/checkout@v4` steps need. Restore it alongside the existing scopes. Addresses CodeRabbit review comment on PR #838 https://github.com/dashpay/rust-dashcore/pull/838#discussion_r_a1eee44ea981beeca42d21f6 * ci: clear `ready-for-review` label when a PR is converted to draft The trigger workflow did not relay `converted_to_draft`, so converting a labeled PR back to draft fired no `workflow_run` and the label survived. Relay that event and make the draft branch remove the label instead of skipping, matching the defensive `|| true` used by the sibling merge-conflict and CodeRabbit branches so a `gh` hiccup on one PR does not abort a multi-PR batch. Addresses CodeRabbit review comment on PR #838 https://github.com/dashpay/rust-dashcore/pull/838#discussion_r_9be318a0d2ccc17bb43f870a --- .github/scripts/check_ci_status.sh | 2 +- .../workflows/ready-for-review-trigger.yml | 25 +++++ .github/workflows/ready-for-review.yml | 96 ++++++++----------- 3 files changed, 66 insertions(+), 57 deletions(-) create mode 100644 .github/workflows/ready-for-review-trigger.yml diff --git a/.github/scripts/check_ci_status.sh b/.github/scripts/check_ci_status.sh index e7dcdeb2c..66f1d8f45 100755 --- a/.github/scripts/check_ci_status.sh +++ b/.github/scripts/check_ci_status.sh @@ -9,7 +9,7 @@ PR_NUMBER="$1" REPO="$2" gh pr checks "$PR_NUMBER" --repo "$REPO" --json name,bucket --jq ' - [.[] | select(.name | test("^(validate-triggers|evaluate)$") | not)] | + [.[] | select(.name | test("^(validate-triggers|evaluate|ready-for-review-trigger)$") | not)] | if length == 0 then "no_checks" elif all(.bucket == "pass" or .bucket == "skipping") then "all_passed" elif any(.bucket == "fail") then "has_failures" diff --git a/.github/workflows/ready-for-review-trigger.yml b/.github/workflows/ready-for-review-trigger.yml new file mode 100644 index 000000000..3529c82ad --- /dev/null +++ b/.github/workflows/ready-for-review-trigger.yml @@ -0,0 +1,25 @@ +name: Ready for Review Trigger + +# Fork pull requests run pull_request and pull_request_review events with a +# read-only token and no secrets, so they cannot add or remove labels. This +# workflow does nothing but complete, which fires a workflow_run event that the +# privileged "Ready for Review Label" workflow reacts to with a write token. +on: + pull_request: + types: [ready_for_review, labeled, converted_to_draft] + pull_request_review: + types: [submitted, dismissed] + +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number }} + cancel-in-progress: true + +permissions: {} + +jobs: + relay: + name: ready-for-review-trigger + runs-on: ubuntu-latest + steps: + - name: Relay to applier + run: echo "Relaying pull request event to the ready-for-review applier via workflow_run." diff --git a/.github/workflows/ready-for-review.yml b/.github/workflows/ready-for-review.yml index cc02490e2..6f23183a9 100644 --- a/.github/workflows/ready-for-review.yml +++ b/.github/workflows/ready-for-review.yml @@ -1,27 +1,33 @@ name: Ready for Review Label +# Runs only on workflow_run so it holds a write-capable token even for pull +# requests opened from forks. Events tied to the fork head (pull_request, +# pull_request_review) get a read-only token and no secrets, so they cannot +# mutate labels themselves. The companion "Ready for Review Trigger" workflow +# relays those events here by completing and firing this workflow_run. on: - pull_request: - types: [ready_for_review, labeled] - pull_request_review: - types: [submitted, dismissed] workflow_run: - workflows: ["Continuous integration", "Pre-commit Checks", "Sanitizer"] + workflows: + - "Continuous integration" + - "Pre-commit Checks" + - "Sanitizer" + - "Ready for Review Trigger" types: [completed] concurrency: - group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.workflow_run.head_branch }} + group: ${{ github.workflow }}-${{ github.event.workflow_run.head_sha }} cancel-in-progress: true permissions: + contents: read pull-requests: write checks: read + issues: write jobs: # Ensure all pull_request-triggered workflows are monitored by workflow_run validate-triggers: runs-on: ubuntu-latest - if: github.event_name == 'workflow_run' steps: - uses: actions/checkout@v4 with: @@ -34,10 +40,7 @@ jobs: SELF=".github/workflows/ready-for-review.yml" # Extract workflow names from the workflow_run trigger list in this file - MONITORED=$(grep -A1 'workflows:' "$SELF" \ - | grep '\[' \ - | tr ',' '\n' \ - | sed 's/.*"\([^"]*\)".*/\1/') + MONITORED=$(grep -E '^\s+- "' "$SELF" | sed 's/.*"\([^"]*\)".*/\1/') MISSING="" for f in .github/workflows/*.yml; do @@ -73,14 +76,6 @@ jobs: evaluate: runs-on: ubuntu-latest - permissions: - pull-requests: write - checks: read - issues: write - # Skip non-CodeRabbit reviews - if: >- - github.event_name != 'pull_request_review' || - github.event.review.user.login == 'coderabbitai[bot]' steps: - uses: actions/checkout@v4 with: @@ -90,54 +85,42 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} REPO: ${{ github.repository }} - EVENT_NAME: ${{ github.event_name }} - EVENT_ACTION: ${{ github.event.action }} - REVIEW_STATE: ${{ github.event.review.state }} - PR_NUMBER_DIRECT: ${{ github.event.pull_request.number }} - LABEL_NAME: ${{ github.event.label.name }} - PULL_REQUESTS_JSON: ${{ toJson(github.event.workflow_run.pull_requests) }} + HEAD_SHA: ${{ github.event.workflow_run.head_sha }} run: | - # On review dismissal or changes requested, remove label and exit - if [ "$EVENT_NAME" = "pull_request_review" ]; then - if [ "$EVENT_ACTION" = "dismissed" ] || [ "$REVIEW_STATE" = "changes_requested" ]; then - echo "CodeRabbit review dismissed or changes requested, removing label." - gh pr edit "$PR_NUMBER_DIRECT" --repo "$REPO" --remove-label "ready-for-review" || true - exit 0 - fi - fi - - # On merge-conflict label, remove ready-for-review and exit - if [ "$EVENT_NAME" = "pull_request" ] && [ "$EVENT_ACTION" = "labeled" ]; then - if [ "$LABEL_NAME" = "merge-conflict" ]; then - echo "Merge conflict detected, removing ready-for-review label." - gh pr edit "$PR_NUMBER_DIRECT" --repo "$REPO" --remove-label "ready-for-review" || true - fi - exit 0 - fi - - # Collect PR numbers depending on the event type - if [ "$EVENT_NAME" = "workflow_run" ]; then - PR_NUMBERS=$(echo "$PULL_REQUESTS_JSON" | jq -r '.[].number') - else - PR_NUMBERS="$PR_NUMBER_DIRECT" - fi + # Resolve the PR(s) for the head commit. github.event.workflow_run's + # pull_requests array is empty for fork PRs, and the commit-to-PR + # endpoint does not index fork heads, so match the head sha against + # the list of open PRs, which works regardless of where the branch + # lives. Matching the exact sha also ignores stale runs whose commit + # the PR has already moved past. + PR_NUMBERS=$(gh api --paginate "repos/$REPO/pulls?state=open&per_page=100" \ + --jq ".[] | select(.head.sha == \"$HEAD_SHA\") | .number") if [ -z "$PR_NUMBERS" ]; then - echo "No associated pull requests found, skipping." + echo "No open pull request found for $HEAD_SHA, skipping." exit 0 fi for PR_NUMBER in $PR_NUMBERS; do echo "=== Checking PR #$PR_NUMBER ===" - # Skip draft PRs - IS_DRAFT=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json isDraft --jq '.isDraft') - if [ "$IS_DRAFT" = "true" ]; then - echo "PR is a draft. Skipping." + PR_JSON=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json isDraft,labels) + + if [ "$(echo "$PR_JSON" | jq -r '.isDraft')" = "true" ]; then + echo "PR is a draft. Removing ready-for-review label if present." + gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "ready-for-review" || true continue fi - # Check if CodeRabbit has approved + # A merge conflict disqualifies the PR regardless of the other gates. + if [ "$(echo "$PR_JSON" | jq -r '[.labels[].name] | index("merge-conflict") != null')" = "true" ]; then + echo "Merge conflict present. Removing ready-for-review label if present." + gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "ready-for-review" || true + continue + fi + + # Check if CodeRabbit has approved. A dismissed or changes-requested + # review leaves the latest state != APPROVED and removes the label. CODERABBIT_STATE=$(gh api --paginate "repos/$REPO/pulls/$PR_NUMBER/reviews" | jq -rs ' add | map(select(.user.login == "coderabbitai[bot]")) @@ -148,7 +131,8 @@ jobs: echo "CodeRabbit review state: $CODERABBIT_STATE" if [ "$CODERABBIT_STATE" != "APPROVED" ]; then - echo "CodeRabbit has not approved. Skipping." + echo "CodeRabbit has not approved. Removing label if present." + gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "ready-for-review" || true continue fi From 95d08135ccb0cb834b165ffb606d8fae4d1e566e Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Sun, 5 Jul 2026 20:30:41 +0700 Subject: [PATCH 022/149] fix(dash-spv): degrade gracefully when advisory file locking is unsupported (#841) `LockFile::new()` treated any `TryLockError::Error` from `File::try_lock()` as fatal. On Android (`target_os = "android"`) the Rust standard library does not implement `flock`-based advisory locking and returns `ErrorKind::Unsupported` at compile time, so SPV storage init always failed with `Write failed: Failed to acquire lock: try_lock() not supported`, blocking the entire Core/SPV layer on Android. The lock file is only a best-effort guard against multiple processes sharing one data directory. On single-process platforms that lack advisory locking, proceeding without it is safe. Treat `ErrorKind::Unsupported` as non-fatal: skip the lock with a warning and continue. Genuine I/O errors and the already-locked (`WouldBlock`) case remain fatal, and the fast path on desktop/iOS is unchanged. Extract the classification into a `classify_lock_result` helper so the degradation path can be unit-tested with a synthesized `Unsupported` error (which never occurs naturally on Linux/macOS CI). Fixes #840 Co-authored-by: Claude Opus 4.8 --- dash-spv/src/storage/lockfile.rs | 92 ++++++++++++++++++++++++++++---- 1 file changed, 81 insertions(+), 11 deletions(-) diff --git a/dash-spv/src/storage/lockfile.rs b/dash-spv/src/storage/lockfile.rs index 03d3a3141..6a85b1cc3 100644 --- a/dash-spv/src/storage/lockfile.rs +++ b/dash-spv/src/storage/lockfile.rs @@ -22,18 +22,10 @@ impl LockFile { .open(&path) .map_err(|e| StorageError::WriteFailed(format!("Failed to create lock file: {}", e)))?; - file.try_lock().map_err(|e| match e { - std::fs::TryLockError::WouldBlock => StorageError::DirectoryLocked(format!( - "Data directory '{}' is already in use by another process", - path.parent().map(|p| p.display().to_string()).unwrap_or_default() - )), - std::fs::TryLockError::Error(io_err) => { - StorageError::WriteFailed(format!("Failed to acquire lock: {}", io_err)) + if classify_lock_result(file.try_lock(), &path)? { + if let Err(e) = writeln!(file, "{}", std::process::id()) { + tracing::warn!("Failed to write PID to lock file: {}", e); } - })?; - - if let Err(e) = writeln!(file, "{}", std::process::id()) { - tracing::warn!("Failed to write PID to lock file: {}", e); } Ok(Self { @@ -43,6 +35,42 @@ impl LockFile { } } +/// Interprets the result of [`File::try_lock`] for an advisory data-directory lock. +/// +/// Returns `Ok(true)` when the lock was acquired, `Ok(false)` when advisory +/// locking is unsupported on the platform (so the caller should proceed without +/// it), and `Err(..)` when the directory is already locked or a genuine I/O +/// error occurred. +fn classify_lock_result( + result: Result<(), std::fs::TryLockError>, + path: &std::path::Path, +) -> StorageResult { + match result { + Ok(()) => Ok(true), + Err(std::fs::TryLockError::WouldBlock) => Err(StorageError::DirectoryLocked(format!( + "Data directory '{}' is already in use by another process", + path.parent().map(|p| p.display().to_string()).unwrap_or_default() + ))), + // Some platforms (e.g. Android) do not implement advisory file locking + // and return `ErrorKind::Unsupported` at the std layer. The lock is only + // a best-effort guard against multiple processes sharing one data + // directory; on such single-process platforms its absence is safe, so + // degrade gracefully instead of failing init. + Err(std::fs::TryLockError::Error(io_err)) + if io_err.kind() == std::io::ErrorKind::Unsupported => + { + tracing::warn!( + "advisory file lock not supported on this platform ({io_err}); \ + continuing without multi-process protection" + ); + Ok(false) + } + Err(std::fs::TryLockError::Error(io_err)) => { + Err(StorageError::WriteFailed(format!("Failed to acquire lock: {io_err}"))) + } + } +} + impl Drop for LockFile { fn drop(&mut self) { if let Err(e) = std::fs::remove_file(&self.path) { @@ -123,6 +151,48 @@ mod tests { assert!(lock2.is_ok(), "Should acquire lock after previous one dropped"); } + #[test] + fn test_classify_lock_result_acquired() { + let path = PathBuf::from("/tmp/data/.lock"); + assert!(classify_lock_result(Ok(()), &path).expect("acquired lock should be Ok")); + } + + #[test] + fn test_classify_lock_result_would_block() { + let path = PathBuf::from("/tmp/data/.lock"); + let result = classify_lock_result(Err(std::fs::TryLockError::WouldBlock), &path); + match result { + Err(StorageError::DirectoryLocked(msg)) => { + assert!(msg.contains("already in use"), "unexpected message: {msg}"); + } + other => panic!("Expected DirectoryLocked error, got: {:?}", other), + } + } + + #[test] + fn test_classify_lock_result_unsupported_is_non_fatal() { + // Platforms such as Android return `ErrorKind::Unsupported` from + // `try_lock`; this must degrade gracefully rather than fail init. + let path = PathBuf::from("/tmp/data/.lock"); + let io_err = + std::io::Error::new(std::io::ErrorKind::Unsupported, "try_lock() not supported"); + let result = classify_lock_result(Err(std::fs::TryLockError::Error(io_err)), &path); + assert!(!result.expect("unsupported lock should be Ok(false)")); + } + + #[test] + fn test_classify_lock_result_genuine_io_error_is_fatal() { + let path = PathBuf::from("/tmp/data/.lock"); + let io_err = std::io::Error::new(std::io::ErrorKind::PermissionDenied, "denied"); + let result = classify_lock_result(Err(std::fs::TryLockError::Error(io_err)), &path); + match result { + Err(StorageError::WriteFailed(msg)) => { + assert!(msg.contains("Failed to acquire lock"), "unexpected message: {msg}"); + } + other => panic!("Expected WriteFailed error, got: {:?}", other), + } + } + #[test] fn test_lock_file_removed_on_drop() { let temp_dir = TempDir::new().expect("Failed to create temp directory"); From a1b98489b91e871d05a69989011ae7b61618a1a9 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Sun, 5 Jul 2026 06:35:08 -0700 Subject: [PATCH 023/149] fix(dash-spv): replaced TARGET_PEERS with already existing max_peers config field (#827) --- dash-spv/src/client/config.rs | 2 +- dash-spv/src/client/config_test.rs | 2 +- dash-spv/src/network/constants.rs | 3 --- dash-spv/src/network/manager.rs | 20 +++++++++++++------- dash-spv/src/network/pool.rs | 26 ++++++++++++++++---------- dash-spv/src/network/tests.rs | 4 ++-- dash-spv/tests/peer_test.rs | 2 +- 7 files changed, 34 insertions(+), 25 deletions(-) diff --git a/dash-spv/src/client/config.rs b/dash-spv/src/client/config.rs index ef84a75ac..d14aa095f 100644 --- a/dash-spv/src/client/config.rs +++ b/dash-spv/src/client/config.rs @@ -86,7 +86,7 @@ impl Default for ClientConfig { validation_mode: ValidationMode::Full, enable_filters: true, enable_masternodes: true, - max_peers: 8, + max_peers: 3, user_agent: None, // Mempool defaults enable_mempool_tracking: true, diff --git a/dash-spv/src/client/config_test.rs b/dash-spv/src/client/config_test.rs index dd1a763d5..c47dbba61 100644 --- a/dash-spv/src/client/config_test.rs +++ b/dash-spv/src/client/config_test.rs @@ -23,7 +23,7 @@ mod tests { assert_eq!(config.validation_mode, ValidationMode::Full); assert!(config.enable_filters); assert!(config.enable_masternodes); - assert_eq!(config.max_peers, 8); + assert_eq!(config.max_peers, 3); // Mempool defaults assert!(config.enable_mempool_tracking); diff --git a/dash-spv/src/network/constants.rs b/dash-spv/src/network/constants.rs index e1278f0f6..30928f70e 100644 --- a/dash-spv/src/network/constants.rs +++ b/dash-spv/src/network/constants.rs @@ -2,9 +2,6 @@ use std::time::Duration; -// Connection limits -pub const TARGET_PEERS: usize = 3; - // Timeouts pub const CONNECTION_TIMEOUT: Duration = Duration::from_secs(30); pub const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(10); diff --git a/dash-spv/src/network/manager.rs b/dash-spv/src/network/manager.rs index e1984c0af..130eb715c 100644 --- a/dash-spv/src/network/manager.rs +++ b/dash-spv/src/network/manager.rs @@ -40,6 +40,7 @@ const DEFAULT_NETWORK_EVENT_CAPACITY: usize = 10000; pub struct PeerNetworkManager { /// Peer pool pool: Arc, + max_peers: usize, /// DNS discovery discovery: Arc, /// AddrV2 handler @@ -117,8 +118,11 @@ impl PeerNetworkManager { // Create request queue for outgoing messages let (request_tx, request_rx) = unbounded_channel(); + let max_peers = config.max_peers.max(1) as usize; + Ok(Self { - pool: Arc::new(PeerPool::new()), + pool: Arc::new(PeerPool::new(max_peers)), + max_peers, discovery: Arc::new(discovery), addrv2_handler: Arc::new(AddrV2Handler::new()), peer_store: Arc::new(peer_store), @@ -194,7 +198,7 @@ impl PeerNetworkManager { peer_addresses.extend( dns_peers .into_iter() - .take(TARGET_PEERS) + .take(self.max_peers) .map(|addr| AddrV2Message::new(addr, ServiceFlags::NETWORK)), ); tracing::info!( @@ -974,10 +978,10 @@ impl PeerNetworkManager { self.evict_mismatched_peers().await; // Re-read count after potential churn so top-up sees the current pool size. let count = self.pool.peer_count().await; - if count < TARGET_PEERS { + if count < self.max_peers { // Try known addresses first, sorted by reputation let known = self.addrv2_handler.get_known_addresses().await; - let needed = TARGET_PEERS.saturating_sub(count); + let needed = self.max_peers.saturating_sub(count); // Select best peers based on reputation let best_peers = self.reputation_manager.select_best_peers(known, needed * 2).await; let mut attempted = 0; @@ -1039,7 +1043,7 @@ impl PeerNetworkManager { async fn dns_fallback_tick(&self) { let count = self.pool.peer_count().await; - if count >= TARGET_PEERS { + if count >= self.max_peers { return; } let dns_peers = tokio::select! { @@ -1049,7 +1053,7 @@ impl PeerNetworkManager { return } }; - let needed = TARGET_PEERS.saturating_sub(count); + let needed = self.max_peers.saturating_sub(count); tracing::debug!("DNS fallback tick found {} addresses. Needed {}", dns_peers.len(), needed); let mut dns_attempted = 0; for addr in dns_peers.iter() { @@ -1399,6 +1403,7 @@ impl Clone for PeerNetworkManager { fn clone(&self) -> Self { Self { pool: self.pool.clone(), + max_peers: self.max_peers, discovery: self.discovery.clone(), addrv2_handler: self.addrv2_handler.clone(), peer_store: self.peer_store.clone(), @@ -1518,7 +1523,8 @@ impl PeerNetworkManager { let discovery = DnsDiscovery::new(); let (request_tx, request_rx) = unbounded_channel(); Self { - pool: Arc::new(PeerPool::new()), + pool: Arc::new(PeerPool::new(8)), + max_peers: 8, discovery: Arc::new(discovery), addrv2_handler: Arc::new(AddrV2Handler::new()), peer_store: Arc::new(peer_store), diff --git a/dash-spv/src/network/pool.rs b/dash-spv/src/network/pool.rs index e1982bf46..1a2e68b21 100644 --- a/dash-spv/src/network/pool.rs +++ b/dash-spv/src/network/pool.rs @@ -1,7 +1,6 @@ //! Peer pool for managing multiple peer connections use crate::error::{NetworkError, SpvError as Error}; -use crate::network::constants::TARGET_PEERS; use crate::network::peer::Peer; use dashcore::network::constants::ServiceFlags; use dashcore::prelude::CoreBlockHeight; @@ -16,14 +15,21 @@ pub struct PeerPool { peers: Arc>>>>, /// Addresses currently being connected to connecting: Arc>>, + /// Maximum number of simultaneous peer connections (from `ClientConfig::max_peers`). + max_peers: usize, } impl PeerPool { - /// Create a new peer pool - pub fn new() -> Self { + /// Create a new peer pool with a connection cap. + pub fn new(max_peers: usize) -> Self { + // Assert peers are greater than 0. We may change this + // so 0 means 'connect to as many peers as you can' + debug_assert!(max_peers > 0, "max_peers must be greater than 0 for the spv client to sync"); + Self { peers: Arc::new(RwLock::new(HashMap::new())), connecting: Arc::new(RwLock::new(HashSet::new())), + max_peers, } } @@ -42,10 +48,10 @@ impl PeerPool { connecting.remove(&addr); // Check if we're at capacity - if peers.len() >= TARGET_PEERS { + if peers.len() >= self.max_peers { return Err(Error::Network(NetworkError::ConnectionFailed(format!( "Maximum peers ({}) reached", - TARGET_PEERS + self.max_peers )))); } @@ -192,12 +198,12 @@ impl PeerPool { /// Check if we need more peers pub async fn needs_more_peers(&self) -> bool { - self.peer_count().await < TARGET_PEERS + self.peer_count().await < self.max_peers } /// Check if we can accept more peers pub async fn can_accept_peers(&self) -> bool { - self.peer_count().await < TARGET_PEERS + self.peer_count().await < self.max_peers } /// Remove unhealthy peers and return their addresses so the caller can @@ -230,7 +236,7 @@ impl PeerPool { impl Default for PeerPool { fn default() -> Self { - Self::new() + Self::new(8) } } @@ -249,7 +255,7 @@ mod tests { #[tokio::test] async fn test_peer_pool_basic() { - let pool = PeerPool::new(); + let pool = PeerPool::new(8); // Initial state assert_eq!(pool.peer_count().await, 0); @@ -265,7 +271,7 @@ mod tests { #[tokio::test] async fn test_service_lookup() { - let pool = PeerPool::new(); + let pool = PeerPool::new(8); let compact_filters = ServiceFlags::COMPACT_FILTERS; let combined = compact_filters | ServiceFlags::NODE_HEADERS_COMPRESSED; diff --git a/dash-spv/src/network/tests.rs b/dash-spv/src/network/tests.rs index 0aa9f5390..4bb3447e0 100644 --- a/dash-spv/src/network/tests.rs +++ b/dash-spv/src/network/tests.rs @@ -29,12 +29,12 @@ mod pool_tests { #[tokio::test] async fn test_pool_limits() { - let pool = PeerPool::new(); + let pool = PeerPool::new(8); // Test needs_more_peers logic assert!(pool.needs_more_peers().await); - // Can accept up to TARGET_PEERS + // Can accept up to 8 peers assert!(pool.can_accept_peers().await); // Test peer count diff --git a/dash-spv/tests/peer_test.rs b/dash-spv/tests/peer_test.rs index b62cf222e..8634d4413 100644 --- a/dash-spv/tests/peer_test.rs +++ b/dash-spv/tests/peer_test.rs @@ -177,7 +177,7 @@ mod unit_tests { #[tokio::test] async fn test_connection_pool_limits() { - let pool = PeerPool::new(); + let pool = PeerPool::new(8); // Should start empty assert_eq!(pool.peer_count().await, 0); From c88264e77f0af05c7a7f45efe72c87572e80a67a Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Sun, 5 Jul 2026 23:35:43 +1000 Subject: [PATCH 024/149] fix(dash-spv): avoid filter sync getting stuck on the first new block after a restart (#835) Fixes #824. A fully synced client that restarted, or reconnected after losing all peers, came back with its filter download state uninitialized. The next new block then re-requested every filter from height 1 and wedged the in-order store loop, so `committed_height` froze one block below the tip and the client never reported synced again. `start_download` now anchors `next_batch_to_store` and `processing_height` and parks the pipeline at the download frontier before its already-synced early return. The already-synced reconnect branch of `start_sync` delegates to `start_download` instead of duplicating that logic, which also downloads the boundary filter body when the restored filter header tip is ahead of the stored bodies. `handle_new_filter_headers` now gives a new boundary block a processing batch even while a historical rescan is still draining `active_batches`. Covered by unit regression tests for both restart paths, a reconnect test with filter headers ahead of stored filter bodies, and a dashd integration test that mines a block right after a fully synced restart. Co-authored-by: lklimek <842586+lklimek@users.noreply.github.com> --- dash-spv/src/sync/filters/manager.rs | 353 +++++++++++++++++++-- dash-spv/src/sync/filters/sync_manager.rs | 24 +- dash-spv/tests/dashd_sync/tests_restart.rs | 20 ++ 3 files changed, 357 insertions(+), 40 deletions(-) diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index d1e961a71..dbf53695a 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -191,15 +191,28 @@ impl 0 { + (stored_filters_tip + 1).max(header_start_height) + } else { + scan_start + }; + + // Anchor the scan and store frontiers even when we return early below. + // A wallet that boots already synced takes the early return, and a + // later new block would otherwise create a lookahead batch from a + // stale `processing_height` of 0 and wedge the in-order store loop on + // a `next_batch_to_store` of 0. self.processing_height = scan_start; + self.next_batch_to_store = download_start; // Check if already at target (nothing to download) if scan_start > self.progress.filter_header_tip_height() { + // Park the idle pipeline at the download frontier so a later + // `extend_target` from a new block queues from here instead of + // re-requesting every filter from an uninitialized position. + self.filter_pipeline.init(download_start, download_start.saturating_sub(1)); // Only emit FiltersSyncComplete if we've also reached the chain tip // This prevents premature sync complete while filter headers are still syncing if self.progress.committed_height() >= self.progress.target_height() { @@ -216,16 +229,6 @@ impl 0 { - (stored_filters_tip + 1).max(header_start_height) - } else { - scan_start - }; - - self.next_batch_to_store = download_start; - tracing::info!( "Starting filter download (scan_start={}, download_start={}, stored_filters_tip={}, target={})", scan_start, @@ -902,10 +905,13 @@ impl { // In-progress work preserved across a disconnect cycle. Don't @@ -934,19 +940,23 @@ impl (TestFiltersManager, Vec
, BlockFilter) { + let mut manager = create_test_manager().await; + + let headers = Header::dummy_batch(0..102); + manager + .header_storage + .write() + .await + .store_headers(&headers.iter().map(HashedBlockHeader::from).collect::>()) + .await + .unwrap(); + + let boundary_filter = BlockFilter::new(&[0u8; 32]); + { + let mut fs = manager.filter_storage.write().await; + for h in 0..=100 { + fs.store_filter(h, &boundary_filter.content).await.unwrap(); + } + } + + // The prev header is non-zero because the segment store treats an + // all-zero filter header as an empty (sentinel) slot. + let prev_filter_header = FilterHeader::from_byte_array([1u8; 32]); + manager + .filter_header_storage + .write() + .await + .store_filter_headers_at_height( + &[prev_filter_header, boundary_filter.filter_header(&prev_filter_header)], + 100, + ) + .await + .unwrap(); + + manager.wallet.write().await.update_wallet_synced_height(&MOCK_WALLET_ID, 100); + manager.progress.update_committed_height(100); + manager.progress.update_stored_height(100); + manager.progress.update_filter_header_tip_height(100); + manager.progress.update_target_height(100); + + (manager, headers, boundary_filter) + } + /// Build a real `BlockFilter` for a single-output block paying `address`. fn filter_for_address( height: u32, @@ -2060,7 +2118,9 @@ mod tests { // Pipeline target at 150 with no pending batches, so extend_target(150) // is a no-op and send_pending returns immediately (no headers needed) manager.filter_pipeline.init(151, 150); - // Active batch prevents try_create_lookahead_batches from running + // The active batch ends at 200, so try_create_lookahead_batches chains + // its next start off 201, past the filter header tip 150, and the bound + // stops it before creating any batch or emitting an event. manager.active_batches.insert(101, FiltersBatch::new(101, 200, HashMap::new())); let (tx, _rx) = unbounded_channel(); @@ -2170,6 +2230,206 @@ mod tests { assert_eq!(manager.active_batches.keys().next(), Some(&101)); } + /// A fully synced node that reconnects and then sees one new block must + /// commit it. `start_sync` takes the `stored == committed == tip` branch, + /// reports `Synced`, and anchors the store and processing cursors at the + /// frontier; the boundary block's filter header, body, and commit then flow + /// through to the new tip. If those cursors were left at 0, the boundary + /// block would create a batch from height 0 and wedge the in-order store + /// loop so `committed_height` freezes one below tip. + #[tokio::test] + async fn test_start_sync_synced_restart_then_boundary_block_commits() { + let (mut manager, headers, boundary_filter) = setup_synced_manager_at_tip().await; + + // Fully-synced restart: `start_sync` requires `WaitingForConnections`. + manager.set_state(SyncState::WaitingForConnections); + + let (tx, _rx) = unbounded_channel(); + let requests = RequestSender::new(tx); + + // Reconnect: the already-synced branch reports completion and anchors the + // store/processing cursors at the frontier. + let events = manager.start_sync(&requests).await.unwrap(); + assert_eq!(manager.state(), SyncState::Synced); + assert!(events.iter().any(|e| matches!( + e, + SyncEvent::FiltersSyncComplete { + tip_height: 100 + } + ))); + + // Block 101's filter header lands: the manager queues its body download + // and a processing batch starting at the frontier (not at height 0). + manager + .handle_sync_event( + &SyncEvent::FilterHeadersStored { + start_height: 101, + end_height: 101, + tip_height: 101, + }, + &requests, + ) + .await + .unwrap(); + + // The peer answers with the boundary filter body over the real path. + let peer: SocketAddr = "127.0.0.1:19999".parse().unwrap(); + let cfilter = CFilter { + filter_type: 0, + block_hash: headers[101].block_hash(), + filter: boundary_filter.content.clone(), + }; + manager + .handle_message(Message::new(peer, NetworkMessage::CFilter(cfilter)), &requests) + .await + .unwrap(); + + // A trailing tick drives any residual processing to completion. + manager.tick(&requests).await.unwrap(); + + assert_eq!(manager.progress.committed_height(), 101); + assert_eq!(manager.state(), SyncState::Synced); + assert!(manager.active_batches.is_empty()); + assert!(manager.next_batch_to_store > 100); + } + + /// A node that boots already synced (default `WaitForEvents` state, which + /// `start_sync` never runs from) must commit a new block that lands after + /// the initial `FilterHeadersSyncComplete`. That event drives + /// `start_download`'s early return, which must anchor the store cursor and + /// park the pipeline at the frontier. Left at their defaults, the next + /// block makes `extend_target` re-request every filter from height 1 and + /// wedges the in-order store loop on a `next_batch_to_store` of 0, freezing + /// `committed_height` one below tip. + #[tokio::test] + async fn test_synced_boot_then_boundary_block_commits() { + let (mut manager, headers, boundary_filter) = setup_synced_manager_at_tip().await; + + // Fully-synced boot leaves the manager in its default state. + assert_eq!(manager.state(), SyncState::WaitForEvents); + + let (tx, mut rx) = unbounded_channel(); + let requests = RequestSender::new(tx); + + // Filter header sync completing at the stored tip reports Synced via + // `start_download`'s early return, anchoring the frontier cursors. + let events = manager + .handle_sync_event( + &SyncEvent::FilterHeadersSyncComplete { + tip_height: 100, + }, + &requests, + ) + .await + .unwrap(); + assert_eq!(manager.state(), SyncState::Synced); + assert!(events.iter().any(|e| matches!( + e, + SyncEvent::FiltersSyncComplete { + tip_height: 100 + } + ))); + assert_eq!(manager.next_batch_to_store, 101); + + // Block 101's filter header lands. + manager + .handle_sync_event( + &SyncEvent::FilterHeadersStored { + start_height: 101, + end_height: 101, + tip_height: 101, + }, + &requests, + ) + .await + .unwrap(); + + // Only the boundary body may be requested: a pipeline left unparked + // would re-request every filter from height 1 here. + while let Ok(request) = rx.try_recv() { + let (NetworkRequest::SendMessage(msg) + | NetworkRequest::SendMessageToPeer(msg, _) + | NetworkRequest::BroadcastMessage(msg)) = request; + if let NetworkMessage::GetCFilters(get) = msg { + assert_eq!(get.start_height, 101, "unexpected filter re-download"); + } + } + + // The peer answers with the boundary filter body over the real path. + let peer: SocketAddr = "127.0.0.1:19999".parse().unwrap(); + let cfilter = CFilter { + filter_type: 0, + block_hash: headers[101].block_hash(), + filter: boundary_filter.content.clone(), + }; + manager + .handle_message(Message::new(peer, NetworkMessage::CFilter(cfilter)), &requests) + .await + .unwrap(); + + // A trailing tick drives any residual processing to completion. + manager.tick(&requests).await.unwrap(); + + assert_eq!(manager.progress.committed_height(), 101); + assert_eq!(manager.state(), SyncState::Synced); + assert!(manager.active_batches.is_empty()); + } + + /// A node that restores its filter header tip ahead of its stored filter + /// bodies (headers at 101, bodies and commit at 100) must download and + /// commit the boundary body on reconnect. `start_sync` delegates to the + /// download path, which sees the scan frontier at or below the restored + /// tip and requests the boundary body rather than parking in + /// `WaitForEvents` and never asking for it. + #[tokio::test] + async fn test_start_sync_filter_headers_ahead_of_bodies_commits_boundary() { + let (mut manager, headers, boundary_filter) = setup_synced_manager_at_tip().await; + + // Filter headers restored ahead of the stored filter bodies. + manager.progress.update_filter_header_tip_height(101); + manager.set_state(SyncState::WaitingForConnections); + + let (tx, mut rx) = unbounded_channel(); + let requests = RequestSender::new(tx); + + // Reconnect delegates to the download path, which must request the + // boundary body at 101 rather than parking without asking for it. + let events = manager.start_sync(&requests).await.unwrap(); + assert_eq!(manager.state(), SyncState::Syncing); + assert!(!events.iter().any(|e| matches!(e, SyncEvent::SyncStart { .. }))); + assert!(manager.active_batches.contains_key(&101)); + + let mut requested_boundary = false; + while let Ok(request) = rx.try_recv() { + let (NetworkRequest::SendMessage(msg) + | NetworkRequest::SendMessageToPeer(msg, _) + | NetworkRequest::BroadcastMessage(msg)) = request; + if let NetworkMessage::GetCFilters(get) = msg { + assert_eq!(get.start_height, 101, "unexpected filter re-download"); + requested_boundary = true; + } + } + assert!(requested_boundary, "boundary filter body 101 must be requested"); + + // The peer answers with the boundary filter body over the real path. + let peer: SocketAddr = "127.0.0.1:19999".parse().unwrap(); + let cfilter = CFilter { + filter_type: 0, + block_hash: headers[101].block_hash(), + filter: boundary_filter.content.clone(), + }; + manager + .handle_message(Message::new(peer, NetworkMessage::CFilter(cfilter)), &requests) + .await + .unwrap(); + + manager.tick(&requests).await.unwrap(); + + assert_eq!(manager.progress.committed_height(), 101); + assert_eq!(manager.state(), SyncState::Synced); + assert!(manager.active_batches.is_empty()); + } + #[tokio::test] async fn test_handle_new_filter_headers_stays_synced_when_already_synced() { let mut manager = create_test_manager().await; @@ -2466,6 +2726,57 @@ mod tests { ); } + /// When the filter header tip grows while a historical rescan is still + /// draining `active_batches`, the boundary block must still get a + /// processing batch. `handle_new_filter_headers` must create the lookahead + /// batch even with a non-empty active set, or a block landing during a + /// restart reinit window gets its body downloaded but never a processing + /// batch, freezing `committed_height` one below tip. Asserted directly + /// after the call (no `tick`) so the deterministic path is verified in + /// isolation. + #[tokio::test] + async fn test_handle_new_filter_headers_extends_boundary_during_rescan() { + let mut manager = create_test_manager().await; + + let headers = Header::dummy_batch(0..102); + manager + .header_storage + .write() + .await + .store_headers(&headers.iter().map(HashedBlockHeader::from).collect::>()) + .await + .unwrap(); + { + let dummy_filter = BlockFilter::new(&[0u8; 32]); + let mut fs = manager.filter_storage.write().await; + for h in 0..=101 { + fs.store_filter(h, &dummy_filter.content).await.unwrap(); + } + } + + // Historical rescan in progress: an active batch ends at the old tip + // 100, filters stored through 101, committed still climbing. + manager.set_state(SyncState::Syncing); + manager.progress.update_committed_height(0); + manager.progress.update_stored_height(101); + manager.progress.update_filter_header_tip_height(100); + manager.progress.update_target_height(101); + manager.active_batches.insert(0, FiltersBatch::new(0, 100, HashMap::new())); + + let (tx, _rx) = unbounded_channel(); + let requests = RequestSender::new(tx); + + // New block 101 arrives: tip grows past the in-flight rescan boundary. + manager.handle_new_filter_headers(101, &requests).await.unwrap(); + + assert!( + manager.active_batches.contains_key(&101), + "boundary block 101 must get a processing batch even while a rescan \ + is draining active_batches, got keys: {:?}", + manager.active_batches.keys().collect::>() + ); + } + #[tokio::test] async fn test_handle_new_filter_headers_skips_when_fully_committed() { let mut manager = create_test_manager().await; diff --git a/dash-spv/src/sync/filters/sync_manager.rs b/dash-spv/src/sync/filters/sync_manager.rs index 3e080880a..ecce171fa 100644 --- a/dash-spv/src/sync/filters/sync_manager.rs +++ b/dash-spv/src/sync/filters/sync_manager.rs @@ -1,7 +1,6 @@ use crate::error::{SyncError, SyncResult}; use crate::network::{Message, MessageType, RequestSender}; use crate::storage::{BlockHeaderStorage, FilterHeaderStorage, FilterStorage}; -use crate::sync::progress::ProgressPercentage; use crate::sync::sync_manager::ensure_not_started; use crate::sync::{ FiltersManager, ManagerIdentifier, SyncEvent, SyncManager, SyncManagerProgress, SyncState, @@ -81,26 +80,13 @@ impl< return Ok(events); } - // Already at or beyond stored filters tip - check if fully synced + // Already at or beyond stored filters tip - delegate to start_download, + // whose early-return path anchors the store and processing cursors at + // the frontier and parks the idle pipeline there. Unlike the branch + // above this must not emit a SyncStart. if stored_filters_tip > 0 && stored_filters_tip == self.progress.committed_height() { self.progress.update_filter_header_tip_height(stored_filters_tip); - // Initialize the pipeline at the current tip. On full disconnect in-flight state gets - // reset, so we need to initialize the pipeline otherwise it would re-queue from height 1. - self.filter_pipeline.init(stored_filters_tip + 1, stored_filters_tip); - // Only emit SyncComplete if we've also reached the chain tip - if self.progress.committed_height() >= self.progress.target_height() { - self.set_state(SyncState::Synced); - tracing::info!( - "FiltersManager: already synced at height {}", - self.progress.committed_height() - ); - return Ok(vec![SyncEvent::FiltersSyncComplete { - tip_height: stored_filters_tip, - }]); - } - // Caught up to stored filters but chain tip not reached yet - self.set_state(SyncState::WaitForEvents); - return Ok(vec![]); + return self.start_download(requests).await; } // No stored filters to process - wait for FilterHeadersSyncComplete events diff --git a/dash-spv/tests/dashd_sync/tests_restart.rs b/dash-spv/tests/dashd_sync/tests_restart.rs index 9fe6c63f7..89bce2421 100644 --- a/dash-spv/tests/dashd_sync/tests_restart.rs +++ b/dash-spv/tests/dashd_sync/tests_restart.rs @@ -47,6 +47,26 @@ async fn test_sync_restart_consistency() { assert_eq!(first_balance, second_balance, "Balance mismatch after restart"); assert_eq!(first_tx_count, second_tx_count, "Transaction count mismatch after restart"); tracing::info!("State consistent after restart"); + + // A block mined right after a fully-synced restart must commit through the + // Filters phase instead of freezing `committed_height` one below the new + // tip. `wait_for_sync` checks the filters committed height, so the wedge + // surfaces as a timeout here. + if ctx.dashd.supports_mining { + drop(client_handle); + tokio::time::sleep(Duration::from_millis(100)).await; + + tracing::info!("Restarting fully synced, then mining a boundary block"); + let mut client_handle = ctx.spawn_new_client().await; + wait_for_sync(&mut client_handle.progress_receiver, ctx.dashd.initial_height).await; + + let miner_address = ctx.dashd.node.get_new_address_from_wallet("default"); + ctx.dashd.node.generate_blocks(1, &miner_address); + wait_for_sync(&mut client_handle.progress_receiver, ctx.dashd.initial_height + 1).await; + client_handle.stop().await; + } else { + eprintln!("Skipping boundary-block restart check (dashd RPC miner not available)"); + } } /// Verify correct rescan behavior when restarting with a fresh wallet but existing storage. From 417d61da85d2b62b8a92c31cb42fe07d0bdd47fd Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Mon, 6 Jul 2026 12:40:40 +1000 Subject: [PATCH 025/149] fix(key-wallet): don't build asset locks on unconfirmed funds (#836) * fix(key-wallet): restrict asset-lock funding to final inputs Asset-lock builders selected funding UTXOs filtered only by `is_spendable`, which deliberately admits unconfirmed mempool outputs. A non-final input is not InstantSend-eligible per DIP-0010, so the resulting funding transaction could never receive the InstantSend lock or ChainLock Platform requires, and identity registration or top-up died on a 300s `AssetLockFinalityTimeout`. Add an opt-in `TransactionBuilder::require_final_inputs` that restricts coin selection to `is_confirmed || is_instantlocked` UTXOs and enable it in `build_asset_lock` and `build_asset_lock_with_signer`. Ordinary spends keep admitting unconfirmed inputs, now pinned by a regression test. Part of #832 (U2). * fix(key-wallet): require final parents for trusted self-send change `Utxo::is_trusted` was set from a flat `has_owned_input && change-addr` check, so change of a self-send spending a still-unconfirmed parent was credited to the confirmed balance bucket. `Utxo::is_trusted` documents Bitcoin Core `CWalletTx::IsTrusted` parity, which is recursive: a zero-conf output is trusted only if every parent is itself trusted. Set `is_trusted` only when every input spends one of our own UTXOs that is confirmed, InstantSend-locked, or itself trusted. The stored parent flag carries the recursion transitively, and the spent parents are still present in the UTXO map at that point because insertion runs before spent-outpoint removal. Unknown or non-final parents deny trust, so funds the network may still drop never surface as confirmed. Part of #832 (U1). --------- Co-authored-by: lklimek <842586+lklimek@users.noreply.github.com> --- .../managed_core_funds_account.rs | 30 +++-- .../transaction_checking/wallet_checker.rs | 80 ++++++++++++ .../managed_wallet_info/asset_lock_builder.rs | 123 ++++++++++++------ .../transaction_builder.rs | 17 +++ .../transaction_building.rs | 26 ++++ 5 files changed, 225 insertions(+), 51 deletions(-) diff --git a/key-wallet/src/managed_account/managed_core_funds_account.rs b/key-wallet/src/managed_account/managed_core_funds_account.rs index eb4f9dd98..474b725aa 100644 --- a/key-wallet/src/managed_account/managed_core_funds_account.rs +++ b/key-wallet/src/managed_account/managed_core_funds_account.rs @@ -179,11 +179,20 @@ impl ManagedCoreFundsAccount { .map(|info| info.address.clone()) .collect(); - // Detect a self-send: this account owns at least one input being - // spent. `account_match.sent` is computed by matching inputs against - // this account's UTXO set, so a non-zero value means we owned at - // least one of the spent outpoints. - let has_owned_input = account_match.sent > 0; + // Detect a trusted self-send, mirroring Bitcoin Core's + // `CWalletTx::IsTrusted`: every input must spend one of our own + // UTXOs that is itself final (confirmed, InstantSend-locked, or + // trusted). Parent trust already carries the recursion, so one + // level of lookup is transitive over the whole ancestry. The + // spent parents are still present in `self.utxos` here because + // they are only removed after the insert loop below. An unknown + // or non-final parent denies trust, so funds that the network + // may still drop never surface as confirmed. + let all_inputs_final_and_ours = tx.input.iter().all(|input| { + self.utxos.get(&input.previous_output).is_some_and(|parent| { + parent.is_confirmed || parent.is_instantlocked || parent.is_trusted + }) + }); let txid = tx.txid(); let mut utxos_changed = false; @@ -215,12 +224,13 @@ impl ManagedCoreFundsAccount { } // Flag outputs from a "trusted" mempool transaction we created — - // one that spends at least one of our own UTXOs and pays this - // output back to one of our internal (change) addresses. Such - // an output is just our previously-tracked funds returning, so - // `update_balance` credits it to the confirmed bucket even + // one whose inputs all spend our own final UTXOs and which pays + // this output back to one of our internal (change) addresses. + // Such an output is just our previously-tracked funds returning, + // so `update_balance` credits it to the confirmed bucket even // before the parent transaction settles. - let is_trusted_output = has_owned_input && change_addrs.contains(&addr); + let is_trusted_output = + all_inputs_final_and_ours && change_addrs.contains(&addr); let txout = dashcore::TxOut { value: output.value, script_pubkey: output.script_pubkey.clone(), diff --git a/key-wallet/src/transaction_checking/wallet_checker.rs b/key-wallet/src/transaction_checking/wallet_checker.rs index b62def243..209930457 100644 --- a/key-wallet/src/transaction_checking/wallet_checker.rs +++ b/key-wallet/src/transaction_checking/wallet_checker.rs @@ -1814,6 +1814,86 @@ mod tests { assert_eq!(ctx.managed_wallet.balance.spendable(), change_amount); } + /// Sibling of `test_self_send_change_in_mempool_lands_in_confirmed_balance`: + /// a self-send change output is only trusted when the spent parent is + /// itself final. `Utxo::is_trusted` mirrors Bitcoin Core's + /// `CWalletTx::IsTrusted`, which is recursive: a 0-conf output is trusted + /// only if every parent resolves to confirmed, InstantSend-locked, or + /// trusted. Change spending an unconfirmed external parent must therefore + /// stay in the unconfirmed bucket, otherwise non-final funds surface as + /// confirmed/spendable and downstream asset locks get built on them. + #[tokio::test] + async fn test_self_send_change_with_unconfirmed_parent_is_not_trusted() { + let mut ctx = TestWalletContext::new_random(); + let external_address = Address::p2pkh( + &dashcore::PublicKey::from_slice(&[0x02; 33]).expect("pubkey"), + Network::Testnet, + ); + + // Unconfirmed external funding UTXO: the parent stays in the mempool. + let funding_value = 1_000_000u64; + let funding_tx = Transaction::dummy(&ctx.receive_address, 0..1, &[funding_value]); + ctx.check_transaction(&funding_tx, TransactionContext::Mempool).await; + assert_eq!(ctx.managed_wallet.balance.confirmed(), 0); + assert_eq!(ctx.managed_wallet.balance.unconfirmed(), funding_value); + + let change_address = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("account") + .next_change_address(Some(&ctx.xpub), true) + .expect("change address"); + + // Spend the still-unconfirmed funding UTXO: some out, the rest back + // to ourselves as change, broadcast into the mempool. + let send_amount = 600_000u64; + let fee = 1_000u64; + let change_amount = funding_value - send_amount - fee; + let spend_tx = Transaction { + version: 2, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint { + txid: funding_tx.txid(), + vout: 0, + }, + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: dashcore::Witness::new(), + }], + output: vec![ + TxOut { + value: send_amount, + script_pubkey: external_address.script_pubkey(), + }, + TxOut { + value: change_amount, + script_pubkey: change_address.script_pubkey(), + }, + ], + special_transaction_payload: None, + }; + ctx.check_transaction(&spend_tx, TransactionContext::Mempool).await; + + let change_outpoint = OutPoint { + txid: spend_tx.txid(), + vout: 1, + }; + let change_utxo = + ctx.bip44_account().utxos.get(&change_outpoint).expect("change UTXO recorded"); + + assert!( + !change_utxo.is_trusted, + "change spending an unconfirmed parent must not be trusted" + ); + assert_eq!( + ctx.managed_wallet.balance.confirmed(), + 0, + "non-final funds must not be counted as confirmed" + ); + assert_eq!(ctx.managed_wallet.balance.unconfirmed(), change_amount); + } + /// Sibling of `test_self_send_change_in_mempool_lands_in_confirmed_balance`: /// when the wallet receives a mempool payment but does not own any of the /// inputs, an output that happens to land on one of our addresses must diff --git a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs index 05753d336..4542ace06 100644 --- a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs @@ -213,6 +213,7 @@ impl ManagedWalletInfo { credit_outputs, ))) .set_funding(funds_acc, acc) + .require_final_inputs() .build_signed(wallet, |addr| funds_acc.address_derivation_path(&addr)) .await?; @@ -283,6 +284,7 @@ impl ManagedWalletInfo { credit_outputs, ))) .set_funding(funds_acc, &acc) + .require_final_inputs() .build_signed(signer, |addr| funds_acc.address_derivation_path(&addr)) .await?; @@ -346,8 +348,9 @@ mod tests { use super::*; use crate::signer::{ExtendedPubKeySigner, SignerMethod}; use crate::wallet::initialization::WalletAccountCreationOptions; - use crate::Network; - use dashcore::ScriptBuf; + use crate::{Network, Utxo}; + use dashcore::{OutPoint, ScriptBuf, Txid}; + use dashcore_hashes::Hash; fn test_credit_outputs(amounts: &[u64]) -> Vec { amounts @@ -374,6 +377,40 @@ mod tests { (wallet, info) } + /// Fund account 0 with a UTXO at a fresh receive address and return its + /// outpoint. + fn insert_funded_utxo( + info: &mut ManagedWalletInfo, + wallet: &Wallet, + txid_byte: u8, + value: u64, + is_confirmed: bool, + ) -> OutPoint { + let account_xpub = wallet.get_bip44_account(0).unwrap().account_xpub; + let account = info.accounts.standard_bip44_accounts.get_mut(&0).unwrap(); + let funding_address = account.next_receive_address(Some(&account_xpub), true).unwrap(); + let outpoint = OutPoint { + txid: Txid::from_byte_array([txid_byte; 32]), + vout: 0, + }; + let utxo = Utxo { + outpoint, + txout: TxOut { + value, + script_pubkey: funding_address.script_pubkey(), + }, + address: funding_address, + height: 1000, + is_coinbase: false, + is_confirmed, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }; + account.utxos.insert(outpoint, utxo); + outpoint + } + // -- Error type tests -- #[test] @@ -422,6 +459,46 @@ mod tests { ); } + /// An account whose only funds are an unconfirmed mempool UTXO must not + /// produce an asset lock: a non-final input is not InstantSend-eligible + /// per DIP-0010, so the funding transaction could never receive the lock + /// Platform requires. + #[tokio::test] + async fn test_rejects_non_final_funding() { + let (wallet, mut info) = test_wallet_and_info(); + insert_funded_utxo(&mut info, &wallet, 0x11, 1_000_000, false); + info.update_last_processed_height(1100); + + let result = info.build_asset_lock(&wallet, 0, test_credit_outputs(&[200_000]), 1000).await; + assert!( + matches!(result, Err(AssetLockError::Builder(_))), + "asset lock must not be built on unconfirmed funds, got: {:?}", + result.err() + ); + } + + /// With a mix of confirmed and unconfirmed funds, coin selection must only + /// spend the confirmed UTXO, even though the unconfirmed one is larger. + #[tokio::test] + async fn test_selects_only_final_funding() { + let (wallet, mut info) = test_wallet_and_info(); + let confirmed_outpoint = insert_funded_utxo(&mut info, &wallet, 0x22, 1_000_000, true); + insert_funded_utxo(&mut info, &wallet, 0x33, 5_000_000, false); + info.update_last_processed_height(1100); + + let result = info + .build_asset_lock(&wallet, 0, test_credit_outputs(&[200_000]), 1000) + .await + .expect("confirmed funds should cover the asset lock"); + assert!(!result.transaction.input.is_empty()); + for txin in &result.transaction.input { + assert_eq!( + txin.previous_output, confirmed_outpoint, + "asset lock spent a non-final input" + ); + } + } + // -- Signer-variant tests -- /// Signer implementation backed by a real [`RootExtendedPrivKey`]. Models @@ -599,10 +676,6 @@ mod tests { #[tokio::test] async fn test_signer_happy_path_end_to_end() { - use crate::Utxo; - use dashcore::{OutPoint, TxOut, Txid}; - use dashcore_hashes::Hash; - let (wallet, mut info) = test_wallet_and_info(); let root = match &wallet.wallet_type { crate::wallet::WalletType::Mnemonic { @@ -612,41 +685,9 @@ mod tests { _ => unreachable!(), }; - // Generate a receive address on account 0 and fund it with a - // real UTXO at that address — coin selection needs a confirmed, - // spendable output the signer can sign for. - let account_xpub = wallet.get_bip44_account(0).unwrap().account_xpub; - let funding_address = info - .accounts - .standard_bip44_accounts - .get_mut(&0) - .unwrap() - .next_receive_address(Some(&account_xpub), true) - .unwrap(); - - let utxo = Utxo { - outpoint: OutPoint { - txid: Txid::from_byte_array([0x11; 32]), - vout: 0, - }, - txout: TxOut { - value: 1_000_000, - script_pubkey: funding_address.script_pubkey(), - }, - address: funding_address, - height: 1000, - is_coinbase: false, - is_confirmed: true, - is_instantlocked: false, - is_locked: false, - is_trusted: false, - }; - info.accounts - .standard_bip44_accounts - .get_mut(&0) - .unwrap() - .utxos - .insert(utxo.outpoint, utxo); + // Coin selection needs a confirmed, spendable output the signer can + // sign for. + insert_funded_utxo(&mut info, &wallet, 0x11, 1_000_000, true); info.update_last_processed_height(1100); let signer = InMemorySigner { diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 65d628e85..2ed56efb3 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -56,6 +56,7 @@ pub struct TransactionBuilder { fee_rate: FeeRate, current_height: u32, selection_strategy: SelectionStrategy, + require_final_inputs: bool, /// Special transaction payload for Dash-specific transactions special_payload: Option, /// Reservation set of the funding account, captured by `set_funding`. The @@ -80,11 +81,23 @@ impl TransactionBuilder { fee_rate: FeeRate::normal(), current_height: 0, selection_strategy: SelectionStrategy::BranchAndBound, + require_final_inputs: false, special_payload: None, reservations: None, } } + /// Restrict coin selection to final inputs: confirmed or + /// InstantSend-locked UTXOs. Per DIP-0010 only such inputs are + /// InstantSend-eligible, so transactions that must receive an + /// InstantSend lock themselves (e.g. asset locks funding Platform + /// credits) must never spend other mempool outputs, including our + /// own trusted change. + pub fn require_final_inputs(mut self) -> Self { + self.require_final_inputs = true; + self + } + pub fn set_current_height(mut self, current_height: u32) -> Self { self.current_height = current_height; self @@ -301,6 +314,10 @@ impl TransactionBuilder { _ => self.outputs.iter().map(|o| o.value).sum(), }; + if self.require_final_inputs { + self.inputs.retain(|utxo| utxo.is_confirmed || utxo.is_instantlocked); + } + let selection = CoinSelector::new(self.selection_strategy) .select_coins_with_size( self.inputs.iter(), diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index 1a9cb8c4c..029ab4782 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -175,6 +175,32 @@ mod tests { assert!(change_output.is_some(), "Should have change output"); } + /// Ordinary spends deliberately admit unconfirmed inputs: only builders + /// that opt in via `require_final_inputs` (e.g. asset locks) restrict + /// selection to confirmed or InstantSend-locked UTXOs. + #[test] + fn test_ordinary_spend_admits_unconfirmed_inputs() { + let utxos = vec![Utxo::dummy(0, 300000, 100, false, false)]; + let recipient_address = Address::from_str("yTb47qEBpNmgXvYYsHEN4nh8yJwa5iC4Cs") + .unwrap() + .require_network(Network::Testnet) + .unwrap(); + let change_address = Address::from_str("yXfXh3jFYHHxnJZVsXnPcktCENqPaAhcX1") + .unwrap() + .require_network(Network::Testnet) + .unwrap(); + + let (tx, _fee) = TransactionBuilder::new() + .set_fee_rate(FeeRate::normal()) + .set_current_height(200) + .set_change_address(change_address) + .add_output(&recipient_address, 150000) + .add_inputs(utxos) + .build_unsigned() + .expect("ordinary spend of an unconfirmed UTXO must succeed"); + assert!(!tx.input.is_empty()); + } + #[test] fn test_sweep_builder_drains_to_single_output() { let utxos = vec![ From 647fa9820f3614090e4e5f5f2b709961d68e538b Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Mon, 6 Jul 2026 17:42:48 +1000 Subject: [PATCH 026/149] ci: stop `evaluate` failing when a PR's CI is pending or red (#844) * ci: stop `evaluate` failing when a PR's CI is pending or red `check_ci_status.sh` runs `gh pr checks` under `set -euo pipefail`. That command exits `8` when any check is still pending and non-zero when any check fails, even with `--json`, so the script aborted before echoing its status whenever the PR's own CI was not fully green. The caller assigns its output with `CI_STATUS=$(check_ci_status.sh ...)`, also under `set -e`, so the failed command substitution took the whole `evaluate` job down with "exit code 1". This surfaced constantly after the `workflow_run` relay landed, because `evaluate` now re-runs on every review and label event, routinely while CI is still pending or red, which is exactly when `gh pr checks` returns non-zero. Capture the JSON with `|| true` and treat empty output as `no_checks`, so the `jq` rollup classifies `pending`/`has_failures`/`all_passed` instead of aborting the caller. * ci: surface unexpected `gh pr checks` failures instead of hiding them The previous `2>/dev/null || true` collapsed every `gh pr checks` failure into empty output, which `check_ci_status.sh` then reported as `no_checks`, stripping the `ready-for-review` label and turning the job green on a broken state with the error hidden. `gh pr checks --json` exits 0 when checks passed or failed (the failure exit code is suppressed by `--json`) and 8 when they are pending. Any other exit code means gh itself failed (auth, rate limit, network, unknown PR). Tolerate only the expected 0 and 8, propagate anything else with stderr intact, and treat empty output as `no_checks` only when gh actually succeeded. Addresses CodeRabbit review comment on PR #844 https://github.com/dashpay/rust-dashcore/pull/844#discussion_r3526546863 --- .github/scripts/check_ci_status.sh | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/.github/scripts/check_ci_status.sh b/.github/scripts/check_ci_status.sh index 66f1d8f45..47a94ab21 100755 --- a/.github/scripts/check_ci_status.sh +++ b/.github/scripts/check_ci_status.sh @@ -8,7 +8,26 @@ set -euo pipefail PR_NUMBER="$1" REPO="$2" -gh pr checks "$PR_NUMBER" --repo "$REPO" --json name,bucket --jq ' +# gh pr checks signals check state through its exit code: 0 when every check +# passed (and, with --json, also when some failed), 8 when checks are still +# pending. Any other code is gh itself failing (auth, rate limit, network, +# unknown PR); surface that instead of misreading it as an empty check set, +# which would silently strip the label and hide the error. Keep stderr visible. +set +e +CHECKS=$(gh pr checks "$PR_NUMBER" --repo "$REPO" --json name,bucket) +EXIT=$? +set -e + +if [ "$EXIT" -ne 0 ] && [ "$EXIT" -ne 8 ]; then + exit "$EXIT" +fi + +if [ -z "$CHECKS" ]; then + echo "no_checks" + exit 0 +fi + +echo "$CHECKS" | jq -r ' [.[] | select(.name | test("^(validate-triggers|evaluate|ready-for-review-trigger)$") | not)] | if length == 0 then "no_checks" elif all(.bucket == "pass" or .bucket == "skipping") then "all_passed" From 3170ad32416c10ebcb5f578568d5b40e9b062460 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Tue, 7 Jul 2026 02:48:59 +1000 Subject: [PATCH 027/149] fix(key-wallet): preserve UTXO flags across reprocessing (#837) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `update_utxos` rebuilds each output with a fresh `Utxo::new` on every reprocess, so a mempool→block confirmation silently reset `is_instantlocked` (and `is_trusted` and any coin reservation) back to false. An InstantSend lock is permanent for a txid per DIP-0010 and trust only ever settles, so both now latch monotonically when an entry for the outpoint already exists, and the coin-reservation flag is carried through unchanged. `is_confirmed` stays freshly derived so a reorg can still downgrade it. Addresses CodeRabbit review comment on PR #836 https://github.com/dashpay/rust-dashcore/pull/836#discussion_r3512506949 --- .../managed_account/managed_core_funds_account.rs | 12 ++++++++++++ .../src/transaction_checking/wallet_checker.rs | 2 ++ 2 files changed, 14 insertions(+) diff --git a/key-wallet/src/managed_account/managed_core_funds_account.rs b/key-wallet/src/managed_account/managed_core_funds_account.rs index 474b725aa..da4c2c13a 100644 --- a/key-wallet/src/managed_account/managed_core_funds_account.rs +++ b/key-wallet/src/managed_account/managed_core_funds_account.rs @@ -242,6 +242,18 @@ impl ManagedCoreFundsAccount { utxo.is_instantlocked = matches!(context, TransactionContext::InstantSend(_)); utxo.is_trusted = is_trusted_output; + // Reprocessing (e.g. mempool→block) rebuilds this UTXO from + // scratch, so carry forward flags that must not regress. An + // InstantSend lock is permanent for a txid (DIP-0010) and + // trust only ever settles, so both latch monotonically. A + // coin reservation is orthogonal to chain context and is kept + // as-is. `is_confirmed` stays freshly derived so a reorg can + // still downgrade it. + if let Some(prior) = self.utxos.get(&outpoint) { + utxo.is_instantlocked |= prior.is_instantlocked; + utxo.is_trusted |= prior.is_trusted; + utxo.is_locked = prior.is_locked; + } self.utxos.insert(outpoint, utxo); utxos_changed = true; } diff --git a/key-wallet/src/transaction_checking/wallet_checker.rs b/key-wallet/src/transaction_checking/wallet_checker.rs index 209930457..65469b561 100644 --- a/key-wallet/src/transaction_checking/wallet_checker.rs +++ b/key-wallet/src/transaction_checking/wallet_checker.rs @@ -958,6 +958,8 @@ mod tests { assert!(ctx.transaction(&txid).is_confirmed()); assert_eq!(ctx.transaction(&txid).height(), Some(1000)); assert!(ctx.first_utxo().is_confirmed); + // The earlier IS lock must survive the mempool→block reprocess. + assert!(ctx.first_utxo().is_instantlocked, "IS-lock flag must not be lost on confirmation"); assert_eq!(ctx.managed_wallet.balance.spendable(), 200_000); // Stage 4: chain-locked block (rescan with stronger context) From 49c243acbe08b9f059f7ff0dea24063099753788 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Tue, 7 Jul 2026 22:28:19 +1000 Subject: [PATCH 028/149] feat(key-wallet): seed sync checkpoint from `birth_height` (#693) Move sync checkpoint seeding into `ManagedWalletInfo` construction so it can't be forgotten, and let callers control the scan range when importing. - `WalletInfoInterface::from_wallet` now takes `birth_height` and seeds `synced_height` and `last_processed_height` to `birth_height.saturating_sub(1)`. Previously every wallet-add path had to remember to call `set_birth_height` separately, and forgetting it dragged `WalletManager::synced_height` (a min across wallets) back to genesis on every add. - `WalletManager::import_wallet_from_extended_priv_key`, `import_wallet_from_xpub`, and `import_wallet_from_bytes` (plus the matching FFI) gain a `birth_height` parameter so previously-used keys can rescan from a chosen height instead of being silently anchored at the chain tip. - Unused `ManagedWalletInfo::with_birth_height` removed. --- dash-spv-ffi/tests/test_wallet_manager.rs | 1 + key-wallet-ffi/FFI_API.md | 2 +- key-wallet-ffi/src/wallet_manager.rs | 5 ++++- .../src/wallet_manager_serialization_tests.rs | 1 + key-wallet-ffi/src/wallet_manager_tests.rs | 2 ++ key-wallet-ffi/tests/test_import_wallet.rs | 1 + key-wallet-manager/src/lib.rs | 21 +++++++++++++------ .../tests/test_serialized_wallets.rs | 11 +++++++--- 8 files changed, 33 insertions(+), 11 deletions(-) diff --git a/dash-spv-ffi/tests/test_wallet_manager.rs b/dash-spv-ffi/tests/test_wallet_manager.rs index b6ecb0d7a..9e04908ff 100644 --- a/dash-spv-ffi/tests/test_wallet_manager.rs +++ b/dash-spv-ffi/tests/test_wallet_manager.rs @@ -93,6 +93,7 @@ mod tests { wallet_manager_ptr, serialized_wallet.as_ptr(), serialized_wallet.len(), + 0, imported_wallet_id.as_mut_ptr(), &mut error as *mut FFIError, ); diff --git a/key-wallet-ffi/FFI_API.md b/key-wallet-ffi/FFI_API.md index 60018fdfd..4cd96c44a 100644 --- a/key-wallet-ffi/FFI_API.md +++ b/key-wallet-ffi/FFI_API.md @@ -678,7 +678,7 @@ Get wallet IDs # Safety - `manager` must be a valid pointer to an FFIWalletMan #### `wallet_manager_import_wallet_from_bytes` ```c -wallet_manager_import_wallet_from_bytes(manager: *mut FFIWalletManager, wallet_bytes: *const u8, wallet_bytes_len: usize, wallet_id_out: *mut u8, error: *mut FFIError,) -> bool +wallet_manager_import_wallet_from_bytes(manager: *mut FFIWalletManager, wallet_bytes: *const u8, wallet_bytes_len: usize, birth_height: u32, wallet_id_out: *mut u8, error: *mut FFIError,) -> bool ``` **Module:** `wallet_manager` diff --git a/key-wallet-ffi/src/wallet_manager.rs b/key-wallet-ffi/src/wallet_manager.rs index e2736b26b..2cb96add8 100644 --- a/key-wallet-ffi/src/wallet_manager.rs +++ b/key-wallet-ffi/src/wallet_manager.rs @@ -278,6 +278,8 @@ pub unsafe extern "C" fn wallet_manager_free_wallet_bytes(wallet_bytes: *mut u8, /// - `manager` must be a valid pointer to an FFIWalletManager instance /// - `wallet_bytes` must be a valid pointer to bincode-serialized wallet bytes /// - `wallet_bytes_len` must be the exact length of the wallet bytes +/// - `birth_height` is the birth height to seed the wallet's sync checkpoint +/// (0 to rescan from genesis) /// - `wallet_id_out` must be a valid pointer to a 32-byte array that will receive the wallet ID /// - `error` must be a valid pointer to an FFIError structure /// - The caller must ensure all pointers remain valid for the duration of this call @@ -287,6 +289,7 @@ pub unsafe extern "C" fn wallet_manager_import_wallet_from_bytes( manager: *mut FFIWalletManager, wallet_bytes: *const u8, wallet_bytes_len: usize, + birth_height: u32, wallet_id_out: *mut u8, error: *mut FFIError, ) -> bool { @@ -299,7 +302,7 @@ pub unsafe extern "C" fn wallet_manager_import_wallet_from_bytes( // Import the wallet using async runtime let result = manager_ref.runtime.block_on(async { let mut manager_guard = manager_ref.manager.write().await; - manager_guard.import_wallet_from_bytes(wallet_bytes_slice) + manager_guard.import_wallet_from_bytes(wallet_bytes_slice, birth_height) }); let wallet_id = unwrap_or_return!(result, error); diff --git a/key-wallet-ffi/src/wallet_manager_serialization_tests.rs b/key-wallet-ffi/src/wallet_manager_serialization_tests.rs index df78f06a6..7f0ac983d 100644 --- a/key-wallet-ffi/src/wallet_manager_serialization_tests.rs +++ b/key-wallet-ffi/src/wallet_manager_serialization_tests.rs @@ -196,6 +196,7 @@ mod tests { manager2, wallet_bytes_out, wallet_bytes_len_out, + 0, imported_wallet_id.as_mut_ptr(), error, ) diff --git a/key-wallet-ffi/src/wallet_manager_tests.rs b/key-wallet-ffi/src/wallet_manager_tests.rs index 76abf7e48..7a8239582 100644 --- a/key-wallet-ffi/src/wallet_manager_tests.rs +++ b/key-wallet-ffi/src/wallet_manager_tests.rs @@ -925,6 +925,7 @@ mod tests { manager3, wallet_bytes_slice.as_ptr(), wallet_bytes_slice.len(), + 0, import_wallet_id_out.as_mut_ptr(), error, ) @@ -1081,6 +1082,7 @@ mod tests { manager2, wallet_bytes_copy.as_ptr(), wallet_bytes_copy.len(), + 0, import_wallet_id_out.as_mut_ptr(), error, ) diff --git a/key-wallet-ffi/tests/test_import_wallet.rs b/key-wallet-ffi/tests/test_import_wallet.rs index 33a4c2103..ee5fc3ecf 100644 --- a/key-wallet-ffi/tests/test_import_wallet.rs +++ b/key-wallet-ffi/tests/test_import_wallet.rs @@ -58,6 +58,7 @@ mod tests { manager2, ptr::null(), 0, + 0, imported_wallet_id.as_mut_ptr(), &mut error, ); diff --git a/key-wallet-manager/src/lib.rs b/key-wallet-manager/src/lib.rs index 874d39020..facf471d6 100644 --- a/key-wallet-manager/src/lib.rs +++ b/key-wallet-manager/src/lib.rs @@ -307,6 +307,7 @@ impl WalletManager { /// /// # Arguments /// * `xprv` - The extended private key string (base58check encoded) + /// * `birth_height` - Birth height for wallet scanning (0 to sync from genesis) /// * `account_creation_options` - Specifies which accounts to create during initialization /// /// # Returns @@ -315,6 +316,7 @@ impl WalletManager { pub fn import_wallet_from_extended_priv_key( &mut self, xprv: &str, + birth_height: CoreBlockHeight, account_creation_options: key_wallet::wallet::initialization::WalletAccountCreationOptions, ) -> Result { // Parse the extended private key @@ -334,7 +336,7 @@ impl WalletManager { } // Create managed wallet info - let managed_info = T::from_wallet(&wallet, self.last_processed_height()); + let managed_info = T::from_wallet(&wallet, birth_height); self.wallets.insert(wallet_id, wallet); self.wallet_infos.insert(wallet_id, managed_info); @@ -349,6 +351,7 @@ impl WalletManager { /// /// # Arguments /// * `xpub` - The extended public key string (base58check encoded) + /// * `birth_height` - Birth height for wallet scanning (0 to sync from genesis) /// * `can_sign_externally` - If true, creates an externally signable wallet (e.g., for hardware wallets). /// If false, creates a pure watch-only wallet. /// @@ -358,6 +361,7 @@ impl WalletManager { pub fn import_wallet_from_xpub( &mut self, xpub: &str, + birth_height: CoreBlockHeight, can_sign_externally: bool, ) -> Result { // Parse the extended public key @@ -380,7 +384,7 @@ impl WalletManager { } // Create managed wallet info - let managed_info = T::from_wallet(&wallet, self.last_processed_height()); + let managed_info = T::from_wallet(&wallet, birth_height); self.wallets.insert(wallet_id, wallet); self.wallet_infos.insert(wallet_id, managed_info); @@ -394,8 +398,14 @@ impl WalletManager { /// This is useful for restoring wallets from backups or transferring wallets /// between systems. /// + /// The serialized form contains only the immutable `Wallet`, not the mutable + /// `ManagedWalletInfo`, so the caller must supply a `birth_height` to seed the + /// new wallet's sync checkpoint. Pass 0 to rescan from genesis, or the chain + /// tip if no historical scan is desired. + /// /// # Arguments /// * `wallet_bytes` - The bincode-serialized wallet bytes + /// * `birth_height` - Birth height for wallet scanning (0 to sync from genesis) /// /// # Returns /// * `Ok(WalletId)` - The computed wallet ID of the imported wallet @@ -404,6 +414,7 @@ impl WalletManager { pub fn import_wallet_from_bytes( &mut self, wallet_bytes: &[u8], + birth_height: CoreBlockHeight, ) -> Result { // Deserialize the wallet from bincode let wallet: Wallet = bincode::decode_from_slice(wallet_bytes, bincode::config::standard()) @@ -420,10 +431,8 @@ impl WalletManager { return Err(WalletError::WalletExists(wallet_id)); } - // Create managed wallet info from the imported wallet, using the manager's - // current aggregated last-processed height as the fallback birth height - // since the serialized form does not preserve it. - let managed_info = T::from_wallet(&wallet, self.last_processed_height()); + // Create managed wallet info from the imported wallet + let managed_info = T::from_wallet(&wallet, birth_height); self.wallets.insert(wallet_id, wallet); self.wallet_infos.insert(wallet_id, managed_info); diff --git a/key-wallet-manager/tests/test_serialized_wallets.rs b/key-wallet-manager/tests/test_serialized_wallets.rs index edcdab694..1bc7742b9 100644 --- a/key-wallet-manager/tests/test_serialized_wallets.rs +++ b/key-wallet-manager/tests/test_serialized_wallets.rs @@ -64,10 +64,15 @@ mod tests { assert_eq!(wallet_id, wallet_id3); println!("Externally signable wallet ID: {}", hex::encode(wallet_id3)); - // Test 4: Import the serialized wallet back + // Test 4: Import the serialized wallet back with a specific birth height let mut manager4 = WalletManager::::new(Network::Testnet); - let import_result = manager4.import_wallet_from_bytes(&bytes); + let import_result = manager4.import_wallet_from_bytes(&bytes, 50_000); assert!(import_result.is_ok()); - assert_eq!(import_result.unwrap(), wallet_id); + let imported_id = import_result.unwrap(); + assert_eq!(imported_id, wallet_id); + let imported = manager4.get_wallet_info(&imported_id).unwrap(); + assert_eq!(imported.birth_height(), 50_000); + assert_eq!(imported.synced_height(), 49_999); + assert_eq!(imported.last_processed_height(), 49_999); } } From 832bab2e175b5cbd2d66d2f2a0996a68a5540c61 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Wed, 8 Jul 2026 06:37:15 -0700 Subject: [PATCH 029/149] perf(hashes): SIMD-batched SipHash-2-4 for BIP158 filter matching (#845) --- dash-spv/src/sync/filters/manager.rs | 16 +- dash/src/bip152.rs | 4 +- dash/src/bip158.rs | 237 +++++++----- hashes/Cargo.toml | 5 + hashes/benches/hashes.rs | 21 +- hashes/benches/siphash.rs | 42 ++ hashes/src/siphash24.rs | 557 +++++++++++++++++++++++++-- key-wallet-manager/src/matching.rs | 6 +- 8 files changed, 740 insertions(+), 148 deletions(-) create mode 100644 hashes/benches/siphash.rs diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index dbf53695a..9ccbc3c5e 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -7,7 +7,7 @@ use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::sync::Arc; -use dashcore::bip158::BlockFilter; +use dashcore::bip158::{BlockFilter, FilterQuery}; use dashcore::ScriptBuf; use super::batch::FiltersBatch; @@ -783,6 +783,14 @@ impl = wallet_states + .iter() + .map(|(id, synced, scripts)| { + (*id, *synced, scripts.iter().map(|s| s.as_bytes()).collect()) + }) + .collect(); + let block_to_wallets = { let Some(batch) = self.active_batches.get(&batch_start) else { return Ok(events); @@ -802,13 +810,11 @@ impl matched, Err(e) => { tracing::warn!( diff --git a/dash/src/bip152.rs b/dash/src/bip152.rs index 872dd29b5..50cf991e9 100644 --- a/dash/src/bip152.rs +++ b/dash/src/bip152.rs @@ -123,11 +123,11 @@ impl ShortId { pub fn with_siphash_keys>(txid: &T, siphash_keys: (u64, u64)) -> ShortId { // 2. Running SipHash-2-4 with the input being the transaction ID and the keys (k0/k1) // set to the first two little-endian 64-bit integers from the above hash, respectively. - let hash = siphash24::Hash::hash_with_keys(siphash_keys.0, siphash_keys.1, txid.as_ref()); + let hash = siphash24::siphash(siphash_keys.0, siphash_keys.1, txid.as_ref()); // 3. Dropping the 2 most significant bytes from the SipHash output to make it 6 bytes. let mut id = ShortId([0; 6]); - id.0.copy_from_slice(&hash[0..6]); + id.0.copy_from_slice(&hash.to_le_bytes()[0..6]); id } } diff --git a/dash/src/bip158.rs b/dash/src/bip158.rs index 772a7c81a..46df3c61b 100644 --- a/dash/src/bip158.rs +++ b/dash/src/bip158.rs @@ -146,22 +146,14 @@ impl BlockFilter { filter_hash.filter_header(previous_filter_header) } - /// Returns true if any query matches against this [`BlockFilter`]. - pub fn match_any(&self, block_hash: &BlockHash, query: I) -> Result - where - I: Iterator, - I::Item: Borrow<[u8]>, - { + /// Returns true if any element of `query` matches against this [`BlockFilter`]. + pub fn match_any(&self, block_hash: &BlockHash, query: &FilterQuery) -> Result { let filter_reader = BlockFilterReader::new(block_hash); filter_reader.match_any(&mut self.content.as_slice(), query) } - /// Returns true if all queries match against this [`BlockFilter`]. - pub fn match_all(&self, block_hash: &BlockHash, query: I) -> Result - where - I: Iterator, - I::Item: Borrow<[u8]>, - { + /// Returns true if all elements of `query` match against this [`BlockFilter`]. + pub fn match_all(&self, block_hash: &BlockHash, query: &FilterQuery) -> Result { let filter_reader = BlockFilterReader::new(block_hash); filter_reader.match_all(&mut self.content.as_slice(), query) } @@ -246,21 +238,17 @@ impl BlockFilterReader { } } - /// Returns true if any query matches against this [`BlockFilterReader`]. - pub fn match_any(&self, reader: &mut R, query: I) -> Result + /// Returns true if any element of `query` matches against this [`BlockFilterReader`]. + pub fn match_any(&self, reader: &mut R, query: &FilterQuery) -> Result where - I: Iterator, - I::Item: Borrow<[u8]>, R: io::Read + ?Sized, { self.reader.match_any(reader, query) } - /// Returns true if all queries match against this [`BlockFilterReader`]. - pub fn match_all(&self, reader: &mut R, query: I) -> Result + /// Returns true if all elements of `query` match against this [`BlockFilterReader`]. + pub fn match_all(&self, reader: &mut R, query: &FilterQuery) -> Result where - I: Iterator, - I::Item: Borrow<[u8]>, R: io::Read + ?Sized, { self.reader.match_all(reader, query) @@ -282,11 +270,9 @@ impl GcsFilterReader { } } - /// Returns true if any query matches against this [`GcsFilterReader`]. - pub fn match_any(&self, reader: &mut R, query: I) -> Result + /// Returns true if any element of `query` matches against this [`GcsFilterReader`]. + pub fn match_any(&self, reader: &mut R, query: &FilterQuery) -> Result where - I: Iterator, - I::Item: Borrow<[u8]>, R: io::Read + ?Sized, { let mut decoder = reader; @@ -294,8 +280,7 @@ impl GcsFilterReader { let reader = &mut decoder; // map hashes to [0, n_elements << grp] let nm = n_elements.0 * self.m; - let mut mapped = - query.map(|e| map_to_range(self.filter.hash(e.borrow()), nm)).collect::>(); + let mut mapped = self.filter.hash_query(query, nm); // For an empty query set, "any" should be false (no items to match). if mapped.is_empty() { return Ok(false); @@ -329,11 +314,9 @@ impl GcsFilterReader { Ok(false) } - /// Returns true if all queries match against this [`GcsFilterReader`]. - pub fn match_all(&self, reader: &mut R, query: I) -> Result + /// Returns true if all elements of `query` match against this [`GcsFilterReader`]. + pub fn match_all(&self, reader: &mut R, query: &FilterQuery) -> Result where - I: Iterator, - I::Item: Borrow<[u8]>, R: io::Read + ?Sized, { let mut decoder = reader; @@ -341,8 +324,7 @@ impl GcsFilterReader { let reader = &mut decoder; // map hashes to [0, n_elements << grp] let nm = n_elements.0 * self.m; - let mut mapped = - query.map(|e| map_to_range(self.filter.hash(e.borrow()), nm)).collect::>(); + let mut mapped = self.filter.hash_query(query, nm); if mapped.is_empty() { return Ok(true); } @@ -382,11 +364,83 @@ fn map_to_range(hash: u64, nm: u64) -> u64 { ((hash as u128 * nm as u128) >> 64) as u64 } +/// Byte length of a standard P2PKH scriptPubKey +const P2PKH_SCRIPT_LEN: usize = 25; +/// Byte length of a standard P2SH scriptPubKey +const P2SH_SCRIPT_LEN: usize = 23; + +/// A set of elements (scriptPubKeys) to match against GCS filters, bucketed by length +/// so each bucket hashes through the fixed-length SIMD [`siphash24::siphash_batch`]. The +/// two standard Dash sizes (P2PKH = 25, P2SH = 23) get their own buckets; the rest go to +/// `other`. Build once and reuse across filters to avoid re-bucketing per match. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct FilterQuery { + p2pkh: Vec<[u8; P2PKH_SCRIPT_LEN]>, + p2sh: Vec<[u8; P2SH_SCRIPT_LEN]>, + other: Vec>, +} + +impl FilterQuery { + /// Creates an empty query. + pub fn new() -> FilterQuery { + FilterQuery::default() + } + + /// Adds one element (e.g. a scriptPubKey), routing it to its length bucket. + pub fn push(&mut self, element: &[u8]) { + // The length guards make `copy_from_slice` into the fixed-size array infallible. + match element.len() { + P2PKH_SCRIPT_LEN => { + let mut buf = [0u8; P2PKH_SCRIPT_LEN]; + buf.copy_from_slice(element); + self.p2pkh.push(buf); + } + P2SH_SCRIPT_LEN => { + let mut buf = [0u8; P2SH_SCRIPT_LEN]; + buf.copy_from_slice(element); + self.p2sh.push(buf); + } + _ => self.other.push(element.to_vec()), + } + } + + /// Total number of elements across all buckets. + pub fn len(&self) -> usize { + self.p2pkh.len() + self.p2sh.len() + self.other.len() + } + + /// Whether the query has no elements. + pub fn is_empty(&self) -> bool { + self.p2pkh.is_empty() && self.p2sh.is_empty() && self.other.is_empty() + } + + /// Deduplicates each length bucket (sorting as a side effect). Used by the writer, since + /// BIP158 hashes each distinct element once; the matcher tolerates duplicates. + fn dedup(&mut self) { + self.p2pkh.sort_unstable(); + self.p2pkh.dedup(); + self.p2sh.sort_unstable(); + self.p2sh.dedup(); + self.other.sort_unstable(); + self.other.dedup(); + } +} + +impl> FromIterator for FilterQuery { + fn from_iter>(iter: I) -> FilterQuery { + let mut query = FilterQuery::new(); + for element in iter { + query.push(element.as_ref()); + } + query + } +} + /// Golomb-Rice encoded filter writer. pub struct GcsFilterWriter<'a, W> { filter: GcsFilter, writer: &'a mut W, - elements: BTreeSet>, + elements: FilterQuery, m: u64, } @@ -396,7 +450,7 @@ impl<'a, W: io::Write> GcsFilterWriter<'a, W> { GcsFilterWriter { filter: GcsFilter::new(k0, k1, p), writer, - elements: BTreeSet::new(), + elements: FilterQuery::new(), m, } } @@ -404,20 +458,17 @@ impl<'a, W: io::Write> GcsFilterWriter<'a, W> { /// Adds data to the filter. pub fn add_element(&mut self, element: &[u8]) { if !element.is_empty() { - self.elements.insert(element.to_vec()); + self.elements.push(element); } } /// Writes the filter to the wrapped writer. pub fn finish(&mut self) -> Result { + // BIP158 counts distinct elements; the query keeps duplicates, so fold them here. + self.elements.dedup(); let nm = self.elements.len() as u64 * self.m; - // map hashes to [0, n_elements * M) - let mut mapped: Vec<_> = self - .elements - .iter() - .map(|e| map_to_range(self.filter.hash(e.as_slice()), nm)) - .collect(); + let mut mapped = self.filter.hash_query(&self.elements, nm); mapped.sort_unstable(); // write number of elements as varint @@ -486,9 +537,42 @@ impl GcsFilter { Ok((q << self.p) + r) } - /// Hashes an arbitrary slice with siphash using parameters of this filter. - fn hash(&self, element: &[u8]) -> u64 { - siphash24::Hash::hash_to_u64_with_keys(self.k0, self.k1, element) + /// Hashes each length bucket of `query` with this filter's keys (fixed-length buckets via + /// the SIMD [`siphash24::siphash_batch`], `other` via the scalar one-shot) and maps the + /// results into `[0, nm)`, returned unsorted. + fn hash_query(&self, query: &FilterQuery, nm: u64) -> Vec { + let mut mapped = vec![0u64; query.len()]; + let mut off = 0; + + if !query.p2pkh.is_empty() { + let end = off + query.p2pkh.len(); + siphash24::siphash_batch::( + self.k0, + self.k1, + &query.p2pkh, + &mut mapped[off..end], + ); + off = end; + } + if !query.p2sh.is_empty() { + let end = off + query.p2sh.len(); + siphash24::siphash_batch::( + self.k0, + self.k1, + &query.p2sh, + &mut mapped[off..end], + ); + off = end; + } + for e in &query.other { + mapped[off] = siphash24::siphash(self.k0, self.k1, e); + off += 1; + } + + for m in &mut mapped { + *m = map_to_range(*m, nm); + } + mapped } } @@ -642,27 +726,14 @@ mod test { assert_eq!(test_filter.content, filter.content); let block_hash = &block.block_hash(); - assert!( - filter - .match_all( - block_hash, - &mut txmap.values().filter_map(|s| if !s.is_empty() { - Some(s.as_bytes()) - } else { - None - }) - ) - .unwrap() - ); + let all: FilterQuery = + txmap.values().filter(|s| !s.is_empty()).map(|s| s.as_bytes()).collect(); + assert!(filter.match_all(block_hash, &all).unwrap()); for script in txmap.values() { - let query = [script]; if !script.is_empty() { - assert!( - filter - .match_any(block_hash, &mut query.iter().map(|s| s.as_bytes())) - .unwrap() - ); + let query: FilterQuery = core::iter::once(script.as_bytes()).collect(); + assert!(filter.match_any(block_hash, &query).unwrap()); } } @@ -703,47 +774,27 @@ mod test { let bytes = out; { - let query = [hex!("abcdef"), hex!("eeeeee")]; + let query: FilterQuery = + [hex!("abcdef"), hex!("eeeeee")].iter().map(|v| v.as_slice()).collect(); let reader = GcsFilterReader::new(0, 0, M, P); - assert!( - reader - .match_any(&mut bytes.as_slice(), &mut query.iter().map(|v| v.as_slice())) - .unwrap() - ); + assert!(reader.match_any(&mut bytes.as_slice(), &query).unwrap()); } { - let query = [hex!("abcdef"), hex!("123456")]; + let query: FilterQuery = + [hex!("abcdef"), hex!("123456")].iter().map(|v| v.as_slice()).collect(); let reader = GcsFilterReader::new(0, 0, M, P); - assert!( - !reader - .match_any(&mut bytes.as_slice(), &mut query.iter().map(|v| v.as_slice())) - .unwrap() - ); + assert!(!reader.match_any(&mut bytes.as_slice(), &query).unwrap()); } { let reader = GcsFilterReader::new(0, 0, M, P); - let mut query = Vec::new(); - for p in &patterns { - query.push(p.clone()); - } - assert!( - reader - .match_all(&mut bytes.as_slice(), &mut query.iter().map(|v| v.as_slice())) - .unwrap() - ); + let query: FilterQuery = patterns.iter().map(|v| v.as_slice()).collect(); + assert!(reader.match_all(&mut bytes.as_slice(), &query).unwrap()); } { let reader = GcsFilterReader::new(0, 0, M, P); - let mut query = Vec::new(); - for p in &patterns { - query.push(p.clone()); - } - query.push(hex!("abcdef")); - assert!( - !reader - .match_all(&mut bytes.as_slice(), &mut query.iter().map(|v| v.as_slice())) - .unwrap() - ); + let mut query: FilterQuery = patterns.iter().map(|v| v.as_slice()).collect(); + query.push(&hex!("abcdef")); + assert!(!reader.match_all(&mut bytes.as_slice(), &query).unwrap()); } } diff --git a/hashes/Cargo.toml b/hashes/Cargo.toml index c3279111f..927b6fa1c 100644 --- a/hashes/Cargo.toml +++ b/hashes/Cargo.toml @@ -36,6 +36,7 @@ bincode = { version = "2.0.1", optional = true } [dev-dependencies] serde_test = "1.0" criterion = "0.5" +test-case = "3.3" [target.wasm32-unknown-unknown.dev-dependencies] wasm-bindgen-test = "0.3" @@ -44,5 +45,9 @@ wasm-bindgen-test = "0.3" name = "hashes" harness = false +[[bench]] +name = "siphash" +harness = false + [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(bench)', 'cfg(fuzzing)'] } diff --git a/hashes/benches/hashes.rs b/hashes/benches/hashes.rs index 2e4c16da9..53eab8436 100644 --- a/hashes/benches/hashes.rs +++ b/hashes/benches/hashes.rs @@ -1,6 +1,6 @@ use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput}; use dashcore_hashes::{hash160, ripemd160, sha1, sha256, sha256d, sha512, sha512_256}; -use dashcore_hashes::{hmac, siphash24, Hash, HashEngine}; +use dashcore_hashes::{hmac, Hash, HashEngine}; fn bench_sha256(c: &mut Criterion) { let mut group = c.benchmark_group("sha256"); @@ -121,24 +121,6 @@ fn bench_hash160(c: &mut Criterion) { group.finish(); } -fn bench_siphash24(c: &mut Criterion) { - let mut group = c.benchmark_group("siphash24"); - - // Standard siphash benchmarks - for (size, label) in [(1024, "1k"), (65536, "64k"), (1048576, "1m")] { - group.throughput(Throughput::Bytes(size as u64)); - group.bench_with_input(BenchmarkId::from_parameter(label), &size, |b, &size| { - let bytes = vec![1u8; size]; - b.iter(|| { - let hash = - siphash24::Hash::hash_with_keys(0x0706050403020100, 0x0f0e0d0c0b0a0908, &bytes); - black_box(hash); - }); - }); - } - group.finish(); -} - fn bench_hmac_sha256(c: &mut Criterion) { let mut group = c.benchmark_group("hmac_sha256"); @@ -250,7 +232,6 @@ criterion_group!( bench_sha1, bench_ripemd160, bench_hash160, - bench_siphash24, bench_hmac_sha256, bench_hmac_sha512, bench_constant_time_comparisons, diff --git a/hashes/benches/siphash.rs b/hashes/benches/siphash.rs new file mode 100644 index 000000000..ddd8aa449 --- /dev/null +++ b/hashes/benches/siphash.rs @@ -0,0 +1,42 @@ +use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput}; +use dashcore_hashes::siphash24::{siphash, siphash_batch}; + +const K0: u64 = 0x0706050403020100; +const K1: u64 = 0x0f0e0d0c0b0a0908; + +const LEN: usize = 25; + +const COUNTS: [usize; 3] = [1_024, 16_384, 262_144]; + +fn make_inputs(count: usize) -> Vec<[u8; LEN]> { + (0..count).map(|i| [(i & 0xff) as u8; LEN]).collect() +} + +fn bench_siphash(c: &mut Criterion) { + let mut group = c.benchmark_group("siphash24"); + + for count in COUNTS { + let inputs = make_inputs(count); + let mut out = vec![0u64; count]; + group.throughput(Throughput::Elements(count as u64)); + + group.bench_with_input(BenchmarkId::new("single", count), &inputs, |b, inputs| { + b.iter(|| { + for e in inputs { + black_box(siphash(K0, K1, e)); + } + }); + }); + + group.bench_with_input(BenchmarkId::new("batch", count), &inputs, |b, inputs| { + b.iter(|| { + siphash_batch::(K0, K1, black_box(inputs), &mut out); + black_box(&out); + }); + }); + } + group.finish(); +} + +criterion_group!(siphash_benches, bench_siphash); +criterion_main!(siphash_benches); diff --git a/hashes/src/siphash24.rs b/hashes/src/siphash24.rs index 9e7f41e07..72c6c9340 100644 --- a/hashes/src/siphash24.rs +++ b/hashes/src/siphash24.rs @@ -26,6 +26,12 @@ use core::{cmp, mem, ptr, str}; use crate::{Error, Hash as _, HashEngine as _}; +// Lane-vector types for the per-arch SIMD batchers, so helper signatures stay readable. +#[cfg(target_arch = "aarch64")] +use core::arch::aarch64::uint64x2_t; +#[cfg(target_arch = "x86_64")] +use core::arch::x86_64::__m256i; + crate::internal_macros::hash_type! { 64, false, @@ -44,26 +50,25 @@ fn from_engine(e: HashEngine) -> Hash { Hash::from_u64(state.v0 ^ state.v1 ^ state.v2 ^ state.v3) } -macro_rules! compress { - ($state:expr) => {{ - compress!($state.v0, $state.v1, $state.v2, $state.v3) - }}; - ($v0:expr, $v1:expr, $v2:expr, $v3:expr) => {{ - $v0 = $v0.wrapping_add($v1); - $v1 = $v1.rotate_left(13); - $v1 ^= $v0; - $v0 = $v0.rotate_left(32); - $v2 = $v2.wrapping_add($v3); - $v3 = $v3.rotate_left(16); - $v3 ^= $v2; - $v0 = $v0.wrapping_add($v3); - $v3 = $v3.rotate_left(21); - $v3 ^= $v0; - $v2 = $v2.wrapping_add($v1); - $v1 = $v1.rotate_left(17); - $v1 ^= $v2; - $v2 = $v2.rotate_left(32); - }}; +/// One scalar SipRound over the four state words. The reference the SIMD kernels +/// ([`sip_round_avx2`], [`sip_round_neon`]) reproduce lane-for-lane. +#[inline] +fn sip_round_scalar(mut v0: u64, mut v1: u64, mut v2: u64, mut v3: u64) -> (u64, u64, u64, u64) { + v0 = v0.wrapping_add(v1); + v1 = v1.rotate_left(13); + v1 ^= v0; + v0 = v0.rotate_left(32); + v2 = v2.wrapping_add(v3); + v3 = v3.rotate_left(16); + v3 ^= v2; + v0 = v0.wrapping_add(v3); + v3 = v3.rotate_left(21); + v3 ^= v0; + v2 = v2.wrapping_add(v1); + v1 = v1.rotate_left(17); + v1 ^= v2; + v2 = v2.rotate_left(32); + (v0, v1, v2, v3) } /// Load an integer of the desired type from a byte stream, in LE order. Uses @@ -138,18 +143,23 @@ impl HashEngine { (self.k0, self.k1) } + #[inline] + fn rounds(state: &mut State, n: usize) { + let (mut v0, mut v1, mut v2, mut v3) = (state.v0, state.v1, state.v2, state.v3); + for _ in 0..n { + (v0, v1, v2, v3) = sip_round_scalar(v0, v1, v2, v3); + } + (state.v0, state.v1, state.v2, state.v3) = (v0, v1, v2, v3); + } + #[inline] fn c_rounds(state: &mut State) { - compress!(state); - compress!(state); + HashEngine::rounds(state, 2); } #[inline] fn d_rounds(state: &mut State) { - compress!(state); - compress!(state); - compress!(state); - compress!(state); + HashEngine::rounds(state, 4); } } @@ -280,8 +290,418 @@ unsafe fn u8to64_le(buf: &[u8], start: usize, len: usize) -> u64 { out } +/// Scalar SipHash-2-4 one-shot: hashes `data` under keys `(k0, k1)` to a `u64`. +/// The reference [`siphash_batch`] must match bit-for-bit. +pub fn siphash(k0: u64, k1: u64, data: &[u8]) -> u64 { + let mut v0 = k0 ^ 0x736f6d6570736575; + let mut v1 = k1 ^ 0x646f72616e646f6d; + let mut v2 = k0 ^ 0x6c7967656e657261; + let mut v3 = k1 ^ 0x7465646279746573; + + let len = data.len(); + let nfull = len / 8; + for blk in 0..nfull { + let m = load_int_le!(data, blk * 8, u64); + v3 ^= m; + (v0, v1, v2, v3) = sip_round_scalar(v0, v1, v2, v3); + (v0, v1, v2, v3) = sip_round_scalar(v0, v1, v2, v3); + v0 ^= m; + } + + let left = len & 7; + let b = ((len as u64 & 0xff) << 56) | unsafe { u8to64_le(data, nfull * 8, left) }; + v3 ^= b; + (v0, v1, v2, v3) = sip_round_scalar(v0, v1, v2, v3); + (v0, v1, v2, v3) = sip_round_scalar(v0, v1, v2, v3); + v0 ^= b; + + v2 ^= 0xff; + for _ in 0..4 { + (v0, v1, v2, v3) = sip_round_scalar(v0, v1, v2, v3); + } + + v0 ^ v1 ^ v2 ^ v3 +} + +/// Batch SipHash-2-4 over `inputs` that are all exactly `LEN` bytes, writing +/// `out[i] = siphash(k0, k1, &inputs[i])`. Uses the AVX2 (x86_64) or NEON (aarch64) batch +/// kernel when available, else the scalar one-shot; bit-identical to per-input [`siphash`]. +pub fn siphash_batch(k0: u64, k1: u64, inputs: &[[u8; LEN]], out: &mut [u64]) { + assert!(out.len() >= inputs.len(), "output buffer too small"); + + #[cfg(target_arch = "x86_64")] + { + if std::is_x86_feature_detected!("avx2") { + unsafe { hash_many_wide_avx2::(k0, k1, inputs, out) }; + } else { + siphash_each(k0, k1, inputs, out); + } + } + + #[cfg(target_arch = "aarch64")] + unsafe { + hash_many_wide_neon::(k0, k1, inputs, out) + }; + + #[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))] + siphash_each(k0, k1, inputs, out); +} + +/// Scalar fallback for [`siphash_batch`] where no SIMD kernel applies (x86 without AVX2, and +/// targets with no vectorised path). aarch64 always batches via NEON, so it never uses this. +#[cfg(not(target_arch = "aarch64"))] +#[inline] +fn siphash_each(k0: u64, k1: u64, inputs: &[[u8; LEN]], out: &mut [u64]) { + for (i, inp) in inputs.iter().enumerate() { + out[i] = siphash(k0, k1, inp); + } +} + +/// Rotate-left of each 64-bit lane by `L` (`R = 64 - L`): shift + shift + or. Lane-vector +/// (`uint64x2_t` = 2 messages) analogue of [`rotl_avx2`]. +#[cfg(target_arch = "aarch64")] +#[inline] +unsafe fn sip_rotl(x: uint64x2_t) -> uint64x2_t { + use core::arch::aarch64::*; + vorrq_u64(vshlq_n_u64::(x), vshrq_n_u64::(x)) +} + +/// SipRound on a lane-vector (`uint64x2_t` = 2 messages), shared by the NEON kernels. +#[cfg(target_arch = "aarch64")] +#[inline] +unsafe fn sip_round_neon( + mut v0: uint64x2_t, + mut v1: uint64x2_t, + mut v2: uint64x2_t, + mut v3: uint64x2_t, +) -> (uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t) { + use core::arch::aarch64::*; + v0 = vaddq_u64(v0, v1); + v1 = sip_rotl::<13, 51>(v1); + v1 = veorq_u64(v1, v0); + v0 = sip_rotl::<32, 32>(v0); + v2 = vaddq_u64(v2, v3); + v3 = sip_rotl::<16, 48>(v3); + v3 = veorq_u64(v3, v2); + v0 = vaddq_u64(v0, v3); + v3 = sip_rotl::<21, 43>(v3); + v3 = veorq_u64(v3, v0); + v2 = vaddq_u64(v2, v1); + v1 = sip_rotl::<17, 47>(v1); + v1 = veorq_u64(v1, v2); + v2 = sip_rotl::<32, 32>(v2); + (v0, v1, v2, v3) +} + +/// ILP knob: independent `uint64x2_t` register-sets the NEON batcher keeps in flight (2 +/// messages each, so batch width `2 * NEON_R` = 12). More sets hide SipHash's serial +/// per-message latency; `6` is the widest that fits aarch64's 32 registers without spilling. +#[cfg(target_arch = "aarch64")] +const NEON_R: usize = 6; + +/// Hashes exactly `2 * R` equal-length inputs, keeping `R` independent `uint64x2_t` +/// register-sets in flight (2 messages each) to overlap the per-message SipHash dependency +/// chains. Writes `out[0..2*R]`. NEON analogue of [`siphash_block_avx2`] (4 lanes there). +/// +/// # Safety +/// NEON is baseline on aarch64. `inputs`/`out` must have at least `2 * R` elements. +#[cfg(target_arch = "aarch64")] +#[inline] +// The `for r in 0..R` loops step the register-sets in lockstep by design; index-based. +#[allow(clippy::needless_range_loop)] +unsafe fn siphash_block_neon( + k0: u64, + k1: u64, + inputs: &[[u8; LEN]], + out: &mut [u64], +) { + use core::arch::aarch64::*; + + // Build a lane-vector {x, y} from two scalar words. `vcombine`/`vcreate` moves the GPRs + // straight into the SIMD lanes (lane0 = x, lane1 = y), avoiding the stack round-trip a + // `vld1q_u64([x, y])` load would incur. + let pack = |x: u64, y: u64| -> uint64x2_t { vcombine_u64(vcreate_u64(x), vcreate_u64(y)) }; + + let mut v0 = [vdupq_n_u64(k0 ^ 0x736f6d6570736575); R]; + let mut v1 = [vdupq_n_u64(k1 ^ 0x646f72616e646f6d); R]; + let mut v2 = [vdupq_n_u64(k0 ^ 0x6c7967656e657261); R]; + let mut v3 = [vdupq_n_u64(k1 ^ 0x7465646279746573); R]; + + // One `uint64x2_t` per register holding lane `k`'s next 8-byte word `| extra`; + // register `r` carries inputs `2r..2r+2`, lane order low..high = those two. + macro_rules! words { + ($word:expr) => {{ + let mut m = [vdupq_n_u64(0); R]; + for r in 0..R { + let b = r * 2; + m[r] = pack($word(&inputs[b]), $word(&inputs[b + 1])); + } + m + }}; + } + macro_rules! xor_all { + ($dst:ident, $src:expr) => {{ + let src = $src; + for r in 0..R { + $dst[r] = veorq_u64($dst[r], src[r]); + } + }}; + } + macro_rules! rounds { + ($n:literal) => { + for r in 0..R { + let (mut a0, mut a1, mut a2, mut a3) = (v0[r], v1[r], v2[r], v3[r]); + for _ in 0..$n { + (a0, a1, a2, a3) = sip_round_neon(a0, a1, a2, a3); + } + v0[r] = a0; + v1[r] = a1; + v2[r] = a2; + v3[r] = a3; + } + }; + } + + // Full 8-byte message blocks. + for blk in 0..LEN / 8 { + let off = blk * 8; + let m = words!(|inp: &[u8; LEN]| load_int_le!(inp, off, u64)); + xor_all!(v3, m); + rounds!(2); + xor_all!(v0, m); + } + + // Final block: the leftover tail bytes with the length in the top byte. + let lb = (LEN as u64 & 0xff) << 56; + let off = LEN / 8 * 8; + let left = LEN % 8; + let b = words!(|inp: &[u8; LEN]| lb | u8to64_le(inp, off, left)); + xor_all!(v3, b); + rounds!(2); + xor_all!(v0, b); + + let ff = vdupq_n_u64(0xff); + for r in 0..R { + v2[r] = veorq_u64(v2[r], ff); + } + rounds!(4); + + // out lane = v0 ^ v1 ^ v2 ^ v3, written as one 128-bit store per register (lane order + // low..high maps to out[b..b+2], matching the scalar reference). + for r in 0..R { + let h = veorq_u64(veorq_u64(v0[r], v1[r]), veorq_u64(v2[r], v3[r])); + let b = r * 2; + vst1q_u64(out[b..].as_mut_ptr(), h); + } +} + +/// Driver over the `R`-register NEON block (width `2 * R`): the tail after the wide loop is +/// drained with 2-lane (`R = 1`) blocks, then a final scalar one-shot. +/// +/// # Safety +/// NEON is baseline on aarch64. +#[cfg(target_arch = "aarch64")] +unsafe fn hash_many_wide_neon( + k0: u64, + k1: u64, + inputs: &[[u8; LEN]], + out: &mut [u64], +) { + let w = 2 * R; + let n = inputs.len(); + let mut i = 0; + while i + w <= n { + siphash_block_neon::(k0, k1, &inputs[i..], &mut out[i..]); + i += w; + } + // Drain remaining pairs through the 2-lane kernel (the same block with R = 1). + while i + 2 <= n { + siphash_block_neon::(k0, k1, &inputs[i..], &mut out[i..]); + i += 2; + } + if i < n { + out[i] = siphash(k0, k1, &inputs[i]); + } +} + +/// ILP knob: independent `__m256i` register-sets the AVX2 batcher keeps in flight (4 +/// messages each, so batch width `4 * AVX2_R` = 12). More sets hide SipHash's serial +/// per-message latency; `3` is the widest that fits x86_64's 16 YMMs without spilling +/// (`4` spills and regresses). +#[cfg(target_arch = "x86_64")] +const AVX2_R: usize = 3; + +/// Rotate-left of each 64-bit lane by `L` (`R = 64 - L`), the generic case: shift + shift + or. +#[cfg(target_arch = "x86_64")] +#[target_feature(enable = "avx2")] +#[inline] +unsafe fn rotl_avx2(x: __m256i) -> __m256i { + use core::arch::x86_64::*; + _mm256_or_si256(_mm256_slli_epi64::(x), _mm256_srli_epi64::(x)) +} + +/// SipRound on a `__m256i` holding 4 lanes (4 messages). +#[cfg(target_arch = "x86_64")] +#[target_feature(enable = "avx2")] +#[inline] +unsafe fn sip_round_avx2( + mut v0: __m256i, + mut v1: __m256i, + mut v2: __m256i, + mut v3: __m256i, +) -> (__m256i, __m256i, __m256i, __m256i) { + use core::arch::x86_64::*; + v0 = _mm256_add_epi64(v0, v1); + v1 = rotl_avx2::<13, 51>(v1); + v1 = _mm256_xor_si256(v1, v0); + v0 = rotl_avx2::<32, 32>(v0); + v2 = _mm256_add_epi64(v2, v3); + v3 = rotl_avx2::<16, 48>(v3); + v3 = _mm256_xor_si256(v3, v2); + v0 = _mm256_add_epi64(v0, v3); + v3 = rotl_avx2::<21, 43>(v3); + v3 = _mm256_xor_si256(v3, v0); + v2 = _mm256_add_epi64(v2, v1); + v1 = rotl_avx2::<17, 47>(v1); + v1 = _mm256_xor_si256(v1, v2); + v2 = rotl_avx2::<32, 32>(v2); + (v0, v1, v2, v3) +} + +/// Hashes exactly `4 * R` equal-length inputs, keeping `R` independent `__m256i` +/// register sets in flight (4 messages each) to overlap the per-message SipHash dependency +/// chains. Writes `out[0..4*R]`. +/// +/// # Safety +/// Requires AVX2. `inputs`/`out` must have at least `4 * R` elements. +#[cfg(target_arch = "x86_64")] +#[target_feature(enable = "avx2")] +// The `for r in 0..R` loops step the register sets in lockstep by design; index-based. +#[allow(clippy::needless_range_loop)] +unsafe fn siphash_block_avx2( + k0: u64, + k1: u64, + inputs: &[[u8; LEN]], + out: &mut [u64], +) { + use core::arch::x86_64::*; + + let mut v0 = [_mm256_set1_epi64x((k0 ^ 0x736f6d6570736575) as i64); R]; + let mut v1 = [_mm256_set1_epi64x((k1 ^ 0x646f72616e646f6d) as i64); R]; + let mut v2 = [_mm256_set1_epi64x((k0 ^ 0x6c7967656e657261) as i64); R]; + let mut v3 = [_mm256_set1_epi64x((k1 ^ 0x7465646279746573) as i64); R]; + + // These lockstep helpers stay macros on purpose: as functions taking `&mut [__m256i; R]` + // the compiler spills the state arrays to the stack instead of keeping them in YMM + // registers (measured ~4% slower), whereas the macros keep every lane as an SSA value. + // One `__m256i` per register holds lane `k`'s next 8-byte word `| extra`; + // register `r` carries inputs `4r..4r+4`, lane order low..high = those four. + macro_rules! words { + ($word:expr) => {{ + let mut m = [_mm256_setzero_si256(); R]; + for r in 0..R { + let b = r * 4; + m[r] = _mm256_set_epi64x( + $word(&inputs[b + 3]) as i64, + $word(&inputs[b + 2]) as i64, + $word(&inputs[b + 1]) as i64, + $word(&inputs[b]) as i64, + ); + } + m + }}; + } + macro_rules! xor_all { + ($dst:ident, $src:expr) => {{ + let src = $src; + for r in 0..R { + $dst[r] = _mm256_xor_si256($dst[r], src[r]); + } + }}; + } + macro_rules! rounds { + ($n:literal) => { + for r in 0..R { + let (mut a0, mut a1, mut a2, mut a3) = (v0[r], v1[r], v2[r], v3[r]); + for _ in 0..$n { + (a0, a1, a2, a3) = sip_round_avx2(a0, a1, a2, a3); + } + v0[r] = a0; + v1[r] = a1; + v2[r] = a2; + v3[r] = a3; + } + }; + } + + // Full 8-byte message blocks. + for blk in 0..LEN / 8 { + let off = blk * 8; + let m = words!(|inp: &[u8; LEN]| load_int_le!(inp, off, u64)); + xor_all!(v3, m); + rounds!(2); + xor_all!(v0, m); + } + + // Final block: the leftover tail bytes with the length in the top byte. + let lb = (LEN as u64 & 0xff) << 56; + let off = LEN / 8 * 8; + let left = LEN % 8; + let b = words!(|inp: &[u8; LEN]| lb | unsafe { u8to64_le(inp, off, left) }); + xor_all!(v3, b); + rounds!(2); + xor_all!(v0, b); + + for r in 0..R { + v2[r] = _mm256_xor_si256(v2[r], _mm256_set1_epi64x(0xff)); + } + rounds!(4); + + // out lane = v0 ^ v1 ^ v2 ^ v3, written as one 256-bit store per register (lane order + // low..high maps to out[b..b+4], matching the scalar reference). + for r in 0..R { + let h = _mm256_xor_si256(_mm256_xor_si256(v0[r], v1[r]), _mm256_xor_si256(v2[r], v3[r])); + let b = r * 4; + _mm256_storeu_si256(out[b..].as_mut_ptr() as *mut __m256i, h); + } +} + +/// Driver over the `R`-register AVX2 block (width `4 * R`): the tail after the wide loop is +/// drained with 4-lane (`R = 1`) blocks, then a final scalar one-shot. Mirrors +/// [`hash_many_wide_neon`]. +/// +/// # Safety +/// Requires AVX2. +#[cfg(target_arch = "x86_64")] +#[target_feature(enable = "avx2")] +unsafe fn hash_many_wide_avx2( + k0: u64, + k1: u64, + inputs: &[[u8; LEN]], + out: &mut [u64], +) { + let w = 4 * R; + let n = inputs.len(); + let mut i = 0; + while i + w <= n { + siphash_block_avx2::(k0, k1, &inputs[i..], &mut out[i..]); + i += w; + } + // Drain remaining quads through the 4-lane kernel (the same block with R = 1). + while i + 4 <= n { + siphash_block_avx2::(k0, k1, &inputs[i..], &mut out[i..]); + i += 4; + } + while i < n { + out[i] = siphash(k0, k1, &inputs[i]); + i += 1; + } +} + #[cfg(test)] mod tests { + use test_case::test_case; + use super::*; #[test] @@ -365,9 +785,94 @@ mod tests { let out = Hash::hash_with_keys(k0, k1, &vin[0..i]); assert_eq!(vec, out, "vec #{}", i); + // The standalone one-shot must match the official vector too. + assert_eq!(siphash(k0, k1, &vin[0..i]), u64::from_le_bytes(vecs[i]), "siphash #{}", i); + let inc = Hash::from_engine(state_inc.clone()); assert_eq!(vec, inc, "vec #{}", i); state_inc.input(&[i as u8]); } + + // Same 64 vectors, but each input (length `L`, bytes `0..L`) hashed in BULK: + // a batch of identical inputs through `siphash_batch::` must match the + // official vector on every lane. `L` must be a constant, so we unroll 0..64. + // A batch of 20 exercises the wide SIMD path (>8) plus the scalar leftovers. + fn check_bulk(k0: u64, k1: u64, expected: [u8; 8]) { + let input: [u8; L] = core::array::from_fn(|b| b as u8); + let batch = [input; 20]; + let mut out = [0u64; 20]; + siphash_batch::(k0, k1, &batch, &mut out); + let want = u64::from_le_bytes(expected); + for (j, &got) in out.iter().enumerate() { + assert_eq!(got, want, "bulk len={L} idx={j}"); + } + } + macro_rules! check_all_lens { + ($($l:literal),* $(,)?) => {$( check_bulk::<$l>(k0, k1, vecs[$l]); )*}; + } + check_all_lens!( + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, + 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, + 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, + ); + } + + // The compile-time-length batch must be bit-identical to the scalar one-shot. + // `test_siphash_2_4` already checks every length at a single count, so the job here + // is COUNT coverage — one case per batch size, sweeping zero/one/several wide (8-lane) + // blocks and every leftover size 0..7, plus two large counts for many blocks. For each + // count, every fixed LEN below (tail sizes 0..7) is hashed with pseudo-random content. + #[test_case(0)] + #[test_case(1)] + #[test_case(2)] + #[test_case(3)] + #[test_case(4)] + #[test_case(5)] + #[test_case(6)] + #[test_case(7)] + #[test_case(8)] + #[test_case(9)] + #[test_case(15)] + #[test_case(16)] + #[test_case(17)] + #[test_case(100)] + #[test_case(257)] + fn siphash_batch_matches_scalar(count: usize) { + fn check(count: usize) { + let mut seed: u64 = 0x1234_5678_9abc_def0 + ^ (LEN as u64).wrapping_mul(0x9e37_79b9_7f4a_7c15) + ^ (count as u64).wrapping_mul(0xd6e8_feb8_6659_fd93); + let mut next = || { + seed ^= seed << 13; + seed ^= seed >> 7; + seed ^= seed << 17; + seed + }; + let k0 = 0x0706_0504_0302_0100u64; + let k1 = 0x0f0e_0d0c_0b0a_0908u64; + let inputs: Vec<[u8; LEN]> = + (0..count).map(|_| core::array::from_fn(|_| next() as u8)).collect(); + let mut got = vec![0u64; count]; + siphash_batch::(k0, k1, &inputs, &mut got); + for (idx, inp) in inputs.iter().enumerate() { + assert_eq!(got[idx], siphash(k0, k1, inp), "LEN={LEN} count={count} idx={idx}"); + } + } + + check::<0>(count); + check::<1>(count); + check::<2>(count); + check::<3>(count); + check::<4>(count); + check::<5>(count); + check::<6>(count); + check::<7>(count); + check::<8>(count); + check::<9>(count); + check::<16>(count); + check::<23>(count); + check::<25>(count); + check::<32>(count); + check::<40>(count); } } diff --git a/key-wallet-manager/src/matching.rs b/key-wallet-manager/src/matching.rs index 84a052439..debafb7d9 100644 --- a/key-wallet-manager/src/matching.rs +++ b/key-wallet-manager/src/matching.rs @@ -1,4 +1,4 @@ -use dashcore::bip158::BlockFilter; +use dashcore::bip158::{BlockFilter, FilterQuery}; use dashcore::prelude::CoreBlockHeight; use dashcore::{BlockHash, ScriptBuf}; #[cfg(feature = "parallel-filters")] @@ -36,11 +36,13 @@ pub fn check_compact_filters_for_script_pubkeys( script_pubkeys: &[ScriptBuf], min_height: CoreBlockHeight, ) -> BTreeSet { + // Group the scripts by length once; the same query is reused across every filter. + let query: FilterQuery = script_pubkeys.iter().map(|s| s.as_bytes()).collect(); let match_filter = |(key, filter): (&FilterMatchKey, &BlockFilter)| { if key.height() <= min_height { return None; } - match filter.match_any(key.hash(), script_pubkeys.iter().map(|s| s.as_bytes())) { + match filter.match_any(key.hash(), &query) { Ok(true) => Some(key.clone()), Ok(false) => None, Err(e) => { From 91bb72e14f9fd22a8d62e1f80d40743d943a75f0 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Thu, 9 Jul 2026 02:55:21 -0700 Subject: [PATCH 030/149] peer reputation module cleanup (#328) Co-authored-by: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> --- dash-spv/src/network/manager.rs | 44 +---- dash-spv/src/network/mod.rs | 3 +- dash-spv/src/network/reputation.rs | 203 +++++------------------ dash-spv/src/network/reputation_tests.rs | 55 +++--- dash-spv/src/storage/peers.rs | 7 +- 5 files changed, 83 insertions(+), 229 deletions(-) diff --git a/dash-spv/src/network/manager.rs b/dash-spv/src/network/manager.rs index 130eb715c..680eedf4d 100644 --- a/dash-spv/src/network/manager.rs +++ b/dash-spv/src/network/manager.rs @@ -16,9 +16,7 @@ use crate::network::addrv2::AddrV2Handler; use crate::network::constants::*; use crate::network::discovery::DnsDiscovery; use crate::network::pool::PeerPool; -use crate::network::reputation::{ - misbehavior_scores, positive_scores, PeerReputationManager, ReputationAware, -}; +use crate::network::reputation::{ChangeReason, PeerReputationManager, ReputationAware}; use crate::network::{ HandshakeManager, Message, MessageDispatcher, MessageType, NetworkEvent, NetworkManager, NetworkRequest, Peer, RequestSender, @@ -360,11 +358,7 @@ impl PeerNetworkManager { pool.remove_peer(&addr).await; // Update reputation for handshake failure reputation_manager - .update_reputation( - addr, - misbehavior_scores::INVALID_MESSAGE, - "Handshake failed", - ) + .update_reputation(addr, ChangeReason::HandshakeFailed) .await; // For handshake failures, try again later tokio::time::sleep(RECONNECT_DELAY).await; @@ -377,11 +371,7 @@ impl PeerNetworkManager { pool.remove_peer(&addr).await; // Minor reputation penalty for connection failure reputation_manager - .update_reputation( - addr, - misbehavior_scores::TIMEOUT / 2, - "Connection failed", - ) + .update_reputation(addr, ChangeReason::ConnectionFailed) .await; } } @@ -638,8 +628,7 @@ impl PeerNetworkManager { reputation_manager .update_reputation( addr, - misbehavior_scores::INVALID_MESSAGE, - "Headers2 decompression failed", + ChangeReason::Headers2DecompressionFailed, ) .await; continue; // Don't forward corrupted message @@ -698,11 +687,7 @@ impl PeerNetworkManager { tracing::debug!("Timeout reading from {}, continuing...", addr); // Minor reputation penalty for timeout reputation_manager - .update_reputation( - addr, - misbehavior_scores::TIMEOUT, - "Read timeout", - ) + .update_reputation(addr, ChangeReason::ReadTimeout) .await; continue; } @@ -722,8 +707,7 @@ impl PeerNetworkManager { reputation_manager .update_reputation( addr, - misbehavior_scores::INVALID_TRANSACTION, - "Invalid transaction type in block", + ChangeReason::InvalidTransactionInBlock, ) .await; } else if error_msg @@ -784,9 +768,7 @@ impl PeerNetworkManager { let conn_duration = Duration::from_secs(60 * loop_iteration); // Rough estimate if conn_duration > Duration::from_secs(3600) { // 1 hour - reputation_manager - .update_reputation(addr, positive_scores::LONG_UPTIME, "Long connection uptime") - .await; + reputation_manager.update_reputation(addr, ChangeReason::LongUptime).await; } }); } @@ -1013,9 +995,7 @@ impl PeerNetworkManager { if let Err(e) = peer_guard.send_ping().await { tracing::error!("Failed to ping {}: {}", addr, e); // Update reputation for ping failure - self.reputation_manager - .update_reputation(addr, misbehavior_scores::TIMEOUT, "Ping failed") - .await; + self.reputation_manager.update_reputation(addr, ChangeReason::PingFailed).await; } } let has_expired = peer_guard.remove_expired_pings(); @@ -1316,13 +1296,7 @@ impl PeerNetworkManager { self.disconnect_peer(addr, reason).await?; // Update reputation to trigger ban - self.reputation_manager - .update_reputation( - *addr, - misbehavior_scores::INVALID_HEADER * 2, // Severe penalty - reason, - ) - .await; + self.reputation_manager.update_reputation(*addr, ChangeReason::ManuallyBanned).await; Ok(()) } diff --git a/dash-spv/src/network/mod.rs b/dash-spv/src/network/mod.rs index a428ddde1..017af47d9 100644 --- a/dash-spv/src/network/mod.rs +++ b/dash-spv/src/network/mod.rs @@ -9,7 +9,7 @@ pub mod manager; mod message_dispatcher; pub mod peer; pub mod pool; -pub mod reputation; +mod reputation; mod message_type; #[cfg(test)] @@ -35,6 +35,7 @@ pub use manager::PeerNetworkManager; pub use message_dispatcher::{Message, MessageDispatcher}; pub use message_type::MessageType; pub use peer::Peer; +pub(crate) use reputation::PeerReputation; use std::net::SocketAddr; use tokio::sync::mpsc::UnboundedReceiver; diff --git a/dash-spv/src/network/reputation.rs b/dash-spv/src/network/reputation.rs index 5c65ce32b..f90656584 100644 --- a/dash-spv/src/network/reputation.rs +++ b/dash-spv/src/network/reputation.rs @@ -14,58 +14,51 @@ use std::sync::Arc; use std::time::{Duration, Instant}; use tokio::sync::RwLock; -/// Misbehavior score thresholds for different violations -pub mod misbehavior_scores { - /// Invalid message format or protocol violation - pub const INVALID_MESSAGE: i32 = 10; - - /// Invalid block header - pub const INVALID_HEADER: i32 = 50; - - /// Invalid compact filter - pub const INVALID_FILTER: i32 = 25; - - /// Timeout or slow response - pub const TIMEOUT: i32 = 5; - - /// Sending unsolicited data - pub const UNSOLICITED_DATA: i32 = 15; - - /// Invalid transaction - pub const INVALID_TRANSACTION: i32 = 20; - - /// Invalid masternode list diff - pub const INVALID_MASTERNODE_DIFF: i32 = 30; - - /// Invalid ChainLock - pub const INVALID_CHAINLOCK: i32 = 40; - - /// Invalid InstantLock - pub const INVALID_INSTANTLOCK: i32 = 35; - - /// Duplicate message - pub const DUPLICATE_MESSAGE: i32 = 5; - - /// Connection flood attempt - pub const CONNECTION_FLOOD: i32 = 20; +/// Reason for a peer reputation change. Each reason owns its score delta +/// (positive = penalty, negative = reward) and a human-readable label. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ChangeReason { + HandshakeFailed, + ConnectionFailed, + Headers2DecompressionFailed, + ReadTimeout, + PingFailed, + InvalidTransactionInBlock, + ManuallyBanned, + LongUptime, } -/// Positive behavior scores -pub mod positive_scores { - /// Successfully provided valid headers - pub const VALID_HEADERS: i32 = -5; - - /// Successfully provided valid filters - pub const VALID_FILTERS: i32 = -3; - - /// Successfully provided valid block - pub const VALID_BLOCK: i32 = -10; - - /// Fast response time - pub const FAST_RESPONSE: i32 = -2; +impl ChangeReason { + /// Score delta for this reason: positive for misbehavior (penalty), + /// negative for good behavior (reward). + pub fn score(&self) -> i32 { + match self { + ChangeReason::HandshakeFailed => 10, + ChangeReason::ConnectionFailed => 2, + ChangeReason::Headers2DecompressionFailed => 10, + ChangeReason::ReadTimeout => 5, + ChangeReason::PingFailed => 5, + ChangeReason::InvalidTransactionInBlock => 20, + ChangeReason::ManuallyBanned => 100, + ChangeReason::LongUptime => -5, + } + } +} - /// Long uptime connection - pub const LONG_UPTIME: i32 = -5; +impl std::fmt::Display for ChangeReason { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let label = match self { + ChangeReason::HandshakeFailed => "Handshake failed", + ChangeReason::ConnectionFailed => "Connection failed", + ChangeReason::Headers2DecompressionFailed => "Headers2 decompression failed", + ChangeReason::ReadTimeout => "Read timeout", + ChangeReason::PingFailed => "Ping failed", + ChangeReason::InvalidTransactionInBlock => "Invalid transaction type in block", + ChangeReason::ManuallyBanned => "Manually banned", + ChangeReason::LongUptime => "Long connection uptime", + }; + f.write_str(label) + } } /// Ban duration for misbehaving peers @@ -223,25 +216,10 @@ impl PeerReputation { } } -/// Reputation change event -#[derive(Debug, Clone)] -pub struct ReputationEvent { - pub peer: SocketAddr, - pub change: i32, - pub reason: String, - pub timestamp: Instant, -} - /// Peer reputation manager pub struct PeerReputationManager { /// Reputation data for each peer reputations: Arc>>, - - /// Recent reputation events for monitoring - recent_events: Arc>>, - - /// Maximum number of events to keep - max_events: usize, } impl Default for PeerReputationManager { @@ -255,18 +233,13 @@ impl PeerReputationManager { pub fn new() -> Self { Self { reputations: Arc::new(RwLock::new(HashMap::new())), - recent_events: Arc::new(RwLock::new(Vec::new())), - max_events: 1000, } } - /// Update peer reputation - pub async fn update_reputation( - &self, - peer: SocketAddr, - score_change: i32, - reason: &str, - ) -> bool { + /// Update peer reputation by the score delta of `reason`. + pub async fn update_reputation(&self, peer: SocketAddr, reason: ChangeReason) -> bool { + let score_change = reason.score(); + let mut reputations = self.reputations.write().await; let reputation = reputations.entry(peer).or_default(); @@ -311,32 +284,9 @@ impl PeerReputationManager { ); } - // Record event - let event = ReputationEvent { - peer, - change: score_change, - reason: reason.to_string(), - timestamp: Instant::now(), - }; - - drop(reputations); // Release lock before recording event - self.record_event(event).await; - should_ban } - /// Record a reputation event - async fn record_event(&self, event: ReputationEvent) { - let mut events = self.recent_events.write().await; - events.push(event); - - // Keep only recent events - if events.len() > self.max_events { - let drain_count = events.len() - self.max_events; - events.drain(0..drain_count); - } - } - /// Check if a peer is banned pub async fn is_banned(&self, peer: &SocketAddr) -> bool { let mut reputations = self.reputations.write().await; @@ -348,35 +298,6 @@ impl PeerReputationManager { } } - /// Get peer reputation score - pub async fn get_score(&self, peer: &SocketAddr) -> i32 { - let mut reputations = self.reputations.write().await; - if let Some(reputation) = reputations.get_mut(peer) { - reputation.apply_decay(); - reputation.score - } else { - 0 - } - } - - /// Temporarily ban a peer for a specified duration, regardless of score. - /// This can be used for critical protocol violations (e.g., invalid ChainLocks). - pub async fn temporary_ban_peer(&self, peer: SocketAddr, duration: Duration, reason: &str) { - let mut reputations = self.reputations.write().await; - let reputation = reputations.entry(peer).or_default(); - - reputation.banned_until = Some(Instant::now() + duration); - reputation.ban_count += 1; - - tracing::warn!( - "Peer {} temporarily banned for {:?} (ban #{}, reason: {})", - peer, - duration, - reputation.ban_count, - reason - ); - } - /// Record a connection attempt pub async fn record_connection_attempt(&self, peer: SocketAddr) { let mut reputations = self.reputations.write().await; @@ -404,11 +325,6 @@ impl PeerReputationManager { reputations.clone() } - /// Get recent reputation events - pub async fn get_recent_events(&self) -> Vec { - self.recent_events.read().await.clone() - } - /// Clear banned status for a peer (admin function) pub async fn unban_peer(&self, peer: &SocketAddr) { let mut reputations = self.reputations.write().await; @@ -419,33 +335,6 @@ impl PeerReputationManager { } } - /// Reset reputation for a peer - pub async fn reset_reputation(&self, peer: &SocketAddr) { - let mut reputations = self.reputations.write().await; - reputations.remove(peer); - tracing::info!("Reset reputation for peer {}", peer); - } - - /// Get peers sorted by reputation (best first) - pub async fn get_peers_by_reputation(&self) -> Vec<(SocketAddr, i32)> { - let mut reputations = self.reputations.write().await; - - // Apply decay and collect scores - let mut peer_scores: Vec<(SocketAddr, i32)> = reputations - .iter_mut() - .map(|(addr, rep)| { - rep.apply_decay(); - (*addr, rep.score) - }) - .filter(|(_, score)| *score < MAX_MISBEHAVIOR_SCORE) // Exclude banned peers - .collect(); - - // Sort by score (lower is better) - peer_scores.sort_by_key(|(_, score)| *score); - - peer_scores - } - /// Save reputation data to persistent storage pub async fn save_to_storage(&self, storage: &impl PeerStorage) -> std::io::Result<()> { let reputations = self.reputations.read().await; diff --git a/dash-spv/src/network/reputation_tests.rs b/dash-spv/src/network/reputation_tests.rs index b8b140057..68b74e13b 100644 --- a/dash-spv/src/network/reputation_tests.rs +++ b/dash-spv/src/network/reputation_tests.rs @@ -7,23 +7,22 @@ mod tests { use super::super::*; use std::net::SocketAddr; + async fn score(manager: &PeerReputationManager, peer: &SocketAddr) -> i32 { + manager.get_all_reputations().await.get(peer).map_or(0, |rep| rep.score) + } + #[tokio::test] async fn test_basic_reputation_operations() { let manager = PeerReputationManager::new(); let peer: SocketAddr = "127.0.0.1:8333".parse().unwrap(); - // Initial score should be 0 - assert_eq!(manager.get_score(&peer).await, 0); + assert_eq!(score(&manager, &peer).await, 0); - // Test misbehavior - manager - .update_reputation(peer, misbehavior_scores::INVALID_MESSAGE, "Test invalid message") - .await; - assert_eq!(manager.get_score(&peer).await, 10); + manager.update_reputation(peer, ChangeReason::HandshakeFailed).await; + assert_eq!(score(&manager, &peer).await, 10); - // Test positive behavior - manager.update_reputation(peer, positive_scores::VALID_HEADERS, "Test valid headers").await; - assert_eq!(manager.get_score(&peer).await, 5); + manager.update_reputation(peer, ChangeReason::LongUptime).await; + assert_eq!(score(&manager, &peer).await, 5); } #[tokio::test] @@ -31,17 +30,9 @@ mod tests { let manager = PeerReputationManager::new(); let peer: SocketAddr = "192.168.1.1:8333".parse().unwrap(); - // Accumulate misbehavior + // Banned on the 10th violation (10 * 10 = 100). for i in 0..10 { - let banned = manager - .update_reputation( - peer, - misbehavior_scores::INVALID_MESSAGE, - &format!("Violation {}", i), - ) - .await; - - // Should be banned on the 10th violation (total score = 100) + let banned = manager.update_reputation(peer, ChangeReason::HandshakeFailed).await; if i == 9 { assert!(banned); } else { @@ -58,11 +49,10 @@ mod tests { let peer1: SocketAddr = "10.0.0.1:8333".parse().unwrap(); let peer2: SocketAddr = "10.0.0.2:8333".parse().unwrap(); - // Set reputations - manager.update_reputation(peer1, -10, "Good peer").await; - manager.update_reputation(peer2, 50, "Bad peer").await; + manager.update_reputation(peer1, ChangeReason::LongUptime).await; + manager.update_reputation(peer1, ChangeReason::LongUptime).await; + manager.update_reputation(peer2, ChangeReason::InvalidTransactionInBlock).await; - // Save and load let temp_dir = tempfile::TempDir::new().unwrap(); let peer_storage = PersistentPeerStorage::open(temp_dir.path()) .await @@ -72,9 +62,8 @@ mod tests { let new_manager = PeerReputationManager::new(); new_manager.load_from_storage(&peer_storage).await.unwrap(); - // Verify scores were preserved - assert_eq!(new_manager.get_score(&peer1).await, -10); - assert_eq!(new_manager.get_score(&peer2).await, 50); + assert_eq!(score(&new_manager, &peer1).await, -10); + assert_eq!(score(&new_manager, &peer2).await, 20); } #[tokio::test] @@ -85,15 +74,17 @@ mod tests { let neutral_peer = AddrV2Message::dummy(0, "2.2.2.2".parse().unwrap(), 8333); let bad_peer = AddrV2Message::dummy(0, "3.3.3.3".parse().unwrap(), 8333); - // Set different reputations - manager.update_reputation(good_peer.socket_addr().unwrap(), -20, "Very good").await; - manager.update_reputation(bad_peer.socket_addr().unwrap(), 80, "Very bad").await; - // neutral_peer has default score of 0 + manager.update_reputation(good_peer.socket_addr().unwrap(), ChangeReason::LongUptime).await; + manager + .update_reputation( + bad_peer.socket_addr().unwrap(), + ChangeReason::InvalidTransactionInBlock, + ) + .await; let all_peers = vec![good_peer.clone(), neutral_peer.clone(), bad_peer.clone()]; let selected = manager.select_best_peers(all_peers, 2).await; - // Should select good_peer first, then neutral_peer assert_eq!(selected.len(), 2); assert_eq!(selected[0], good_peer.socket_addr().unwrap()); assert_eq!(selected[1], neutral_peer.socket_addr().unwrap()); diff --git a/dash-spv/src/storage/peers.rs b/dash-spv/src/storage/peers.rs index 69e075ab9..360e83650 100644 --- a/dash-spv/src/storage/peers.rs +++ b/dash-spv/src/storage/peers.rs @@ -10,7 +10,7 @@ use dashcore::{ use crate::{ error::StorageResult, - network::reputation::PeerReputation, + network::PeerReputation, storage::{io::atomic_write, PersistentStorage}, StorageError, }; @@ -97,12 +97,12 @@ impl PeerStorage for PersistentPeerStorage { return Ok(Vec::new()); }; - let mut peers = Vec::new(); - let peers = tokio::task::spawn_blocking(move || { let file = File::open(&peers_file)?; let mut reader = BufReader::new(file); + let mut peers = Vec::new(); + loop { match AddrV2Message::consensus_decode(&mut reader) { Ok(peer) => peers.push(peer), @@ -116,7 +116,6 @@ impl PeerStorage for PersistentPeerStorage { } } } - Ok(peers) }) .await From e4f11a18ee95e0c4cd85efb6347fc27e4521aa7f Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Thu, 9 Jul 2026 23:13:47 +1000 Subject: [PATCH 031/149] feat(dash-spv): anchor sync at wallet birth height checkpoint (#848) When no explicit `start_from_height` is configured, `DashSpvClient::new` now derives the sync anchor from the wallet's `earliest_required_height` and starts from the nearest checkpoint at or before it. Previously the wallet birth height only short-circuited the compact-filter download, while block headers and filter headers always synced from genesis. Explicit `start_from_height` still takes precedence. This also fixes the checkpoint anchoring itself, which never worked. `initialize_genesis_block` rebuilt the checkpoint header with a hardcoded version `0x20000000` and verified the recomputed hash against the stored `block_hash`. Old blocks use version `2`, so the hash never matched, the code logged a warning and silently fell back to a full genesis sync, defeating `start_from_height` entirely. The `Checkpoint` does not store the block version, so the exact header bytes cannot be reconstructed. Since the checkpoint `block_hash` is a trusted constant and chain linkage only ever compares against the stored hash (never a recomputed one), we now store the anchor via the new `HashedBlockHeader::with_trusted_hash`, pairing the reconstructed header with the trusted hash. `time` and `bits` still come from the checkpoint for difficulty checks of later headers. --- dash-spv/src/chain/checkpoints.rs | 14 ++++- dash-spv/src/client/lifecycle.rs | 98 ++++++++++++++----------------- dash-spv/src/client/mod.rs | 61 +++++++++++++++++++ dash-spv/src/types.rs | 12 ++++ 4 files changed, 131 insertions(+), 54 deletions(-) diff --git a/dash-spv/src/chain/checkpoints.rs b/dash-spv/src/chain/checkpoints.rs index f78621b54..6bb53ba5c 100644 --- a/dash-spv/src/chain/checkpoints.rs +++ b/dash-spv/src/chain/checkpoints.rs @@ -6,7 +6,7 @@ //! - Protect against deep reorganizations //! - Bootstrap masternode lists at specific heights -use dashcore::{BlockHash, CompactTarget, Target}; +use dashcore::{BlockHash, CompactTarget, Network, Target}; use dashcore_hashes::{hex, Hash}; use serde::{Deserialize, Serialize}; use std::collections::HashMap; @@ -66,6 +66,18 @@ pub struct CheckpointManager { } impl CheckpointManager { + /// Create a checkpoint manager seeded with the hardcoded checkpoints for `network`. + /// + /// Networks without bundled checkpoints (devnet, regtest) yield an empty manager. + pub fn for_network(network: Network) -> Self { + let checkpoints = match network { + Network::Mainnet => mainnet_checkpoints(), + Network::Testnet => testnet_checkpoints(), + _ => vec![], + }; + Self::new(checkpoints) + } + /// Create a new checkpoint manager from a list of checkpoints pub fn new(checkpoints: Vec) -> Self { let mut checkpoint_map = HashMap::new(); diff --git a/dash-spv/src/client/lifecycle.rs b/dash-spv/src/client/lifecycle.rs index d844ceec7..2f789bdf8 100644 --- a/dash-spv/src/client/lifecycle.rs +++ b/dash-spv/src/client/lifecycle.rs @@ -9,7 +9,7 @@ //! - Wallet data loading use super::{ClientConfig, DashSpvClient, EventHandler}; -use crate::chain::checkpoints::{mainnet_checkpoints, testnet_checkpoints, CheckpointManager}; +use crate::chain::checkpoints::CheckpointManager; use crate::error::{Result, SpvError}; use crate::network::NetworkManager; use crate::storage::{ @@ -20,8 +20,12 @@ use crate::sync::{ BlockHeadersManager, BlocksManager, ChainLockManager, FilterHeadersManager, FiltersManager, InstantSendManager, Managers, MasternodesManager, MempoolManager, SyncCoordinator, }; +use crate::types::HashedBlockHeader; +use dashcore::block::{Header as BlockHeader, Version}; use dashcore::network::constants::NetworkExt; +use dashcore::pow::CompactTarget; use dashcore::sml::masternode_list_engine::MasternodeListEngine; +use dashcore::TxMerkleNode; use dashcore_hashes::Hash; use key_wallet_manager::WalletInterface; use std::sync::Arc; @@ -42,9 +46,20 @@ impl DashSpvClient Some(height), + None => { + let birth_height = wallet.read().await.earliest_required_height().await; + (birth_height > 0).then_some(birth_height) + } + }; + // Initialize genesis block or checkpoint before creating managers, // so they can read the tip from storage during construction. - Self::initialize_genesis_block(&config, &mut storage).await?; + Self::initialize_genesis_block(&config, start_from_height, &mut storage).await?; let masternode_engine = { if config.enable_masternodes { @@ -65,12 +80,7 @@ impl DashSpvClient = Managers::default(); - let checkpoints = match config.network { - dashcore::Network::Mainnet => mainnet_checkpoints(), - dashcore::Network::Testnet => testnet_checkpoints(), - _ => Vec::new(), - }; - let checkpoint_manager = Arc::new(CheckpointManager::new(checkpoints)); + let checkpoint_manager = Arc::new(CheckpointManager::for_network(config.network)); managers.block_headers = Some( BlockHeadersManager::new( storage.block_headers(), @@ -228,7 +238,11 @@ impl DashSpvClient Result<()> { + async fn initialize_genesis_block( + config: &ClientConfig, + start_from_height: Option, + storage: &mut S, + ) -> Result<()> { // Check if we already have any headers in storage let current_tip = storage.get_tip_height().await; @@ -239,16 +253,8 @@ impl DashSpvClient crate::chain::checkpoints::mainnet_checkpoints(), - dashcore::Network::Testnet => crate::chain::checkpoints::testnet_checkpoints(), - _ => vec![], - }; - - // Create checkpoint manager - let checkpoint_manager = crate::chain::checkpoints::CheckpointManager::new(checkpoints); + if let Some(start_height) = start_from_height { + let checkpoint_manager = CheckpointManager::for_network(config.network); // Find the best checkpoint at or before the requested height if let Some(checkpoint) = checkpoint_manager.last_checkpoint_before_height(start_height) @@ -260,51 +266,37 @@ impl DashSpvClient, + ) -> (u32, Option) { + let mut wallet_manager = WalletManager::::new(Network::Mainnet); + wallet_manager + .create_wallet_from_mnemonic( + TEST_MNEMONIC, + birth_height, + WalletAccountCreationOptions::Default, + ) + .expect("wallet creation must succeed"); + let wallet = Arc::new(RwLock::new(wallet_manager)); + + let temp_dir = TempDir::new().unwrap(); + let mut config = ClientConfig::mainnet() + .without_filters() + .without_masternodes() + .with_storage_path(temp_dir.path()); + config.start_from_height = start_from_height; + + let storage = DiskStorageManager::new(&config).await.expect("Failed to create storage"); + let client = DashSpvClient::new( + config, + MockNetworkManager::new(), + storage, + wallet, + vec![Arc::new(())], + ) + .await + .expect("client construction must succeed"); + (client.tip_height().await, client.tip_hash().await) + } + + #[tokio::test] + async fn birth_height_anchors_chain_to_nearest_checkpoint() { + // A wallet born at 120_000 anchors at the 100_000 checkpoint, not genesis, and + // the stored tip carries the trusted checkpoint hash (what the next header's + // `prev_blockhash` is validated against), not a hash recomputed from a header. + let (height, hash) = anchored_tip(120_000, None).await; + assert_eq!(height, 100_000); + let expected: dashcore::BlockHash = + "00000000000fd08c2fb661d2fcb0d49abb3a91e5f27082ce64feed3b4dede2e2".parse().unwrap(); + assert_eq!(hash, Some(expected)); + + // Birth height 0 keeps syncing from genesis. + assert_eq!(anchored_tip(0, None).await.0, 0); + + // Explicit `start_from_height` wins over the wallet birth height, even when + // the birth height would resolve to a higher checkpoint. + assert_eq!(anchored_tip(120_000, Some(60_000)).await.0, 50_000); + } + #[tokio::test] async fn client_exposes_shared_wallet_manager() { let config = ClientConfig::mainnet() diff --git a/dash-spv/src/types.rs b/dash-spv/src/types.rs index dc21b4732..ff14a9449 100644 --- a/dash-spv/src/types.rs +++ b/dash-spv/src/types.rs @@ -27,6 +27,18 @@ pub struct HashedBlockHeader { } impl HashedBlockHeader { + /// Pair a header with a known, trusted hash without recomputing it from the header. + /// + /// The caller guarantees the hash is correct. This is used to anchor the chain at a + /// checkpoint whose hash is trusted but whose exact header bytes (the block version) + /// are not stored, so hashing the reconstructed header would not reproduce it. + pub(crate) fn with_trusted_hash(header: BlockHeader, hash: BlockHash) -> Self { + Self { + header, + hash, + } + } + pub fn header(&self) -> &BlockHeader { &self.header } From 116421f116cae5bd73a0557f87c97e391bf0198e Mon Sep 17 00:00:00 2001 From: lklimek <842586+lklimek@users.noreply.github.com> Date: Thu, 9 Jul 2026 15:22:50 +0200 Subject: [PATCH 032/149] fix(key-wallet): actionable error when add_account(_, None) hits a keyless wallet (#847) * fix(key-wallet): make add_account(_, None) fail actionably on keyless wallets Deriving an account from an in-wallet private key on a watch-only or external-signable wallet used to bubble up a cryptic InvalidParameter("External signable wallet has no private key"), which got misdiagnosed as a production bug three times instead of being read as a usage error. Add a typed, pattern-matchable Error::KeylessWalletRequiresAccountKey { account_type, required_key } and map ONLY the keyless-wallet failure of root_extended_priv_key() to it in the derive-from-root branch of add_account / add_bls_account / add_eddsa_account. Its Display names its own remedy: supply the account's xpub/seed via the Some(..) argument. The None hot-wallet convenience path, the Some(..) branch, and every other error path are left unchanged. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01BqMDY7ZZNwSvdPgBrJQQNq * fix(key-wallet-ffi): handle KeylessWalletRequiresAccountKey in FFIError conversion New key_wallet::Error variant broke the exhaustive match in From for FFIError, failing CI (clippy, docs, MSRV, ASan, ffi builds on all platforms) with E0004 non-exhaustive patterns. Co-Authored-By: Claude Sonnet 5 --------- Co-authored-by: Claude Opus 4.8 --- key-wallet-ffi/src/error.rs | 9 ++- key-wallet/src/error.rs | 20 ++++++ .../src/tests/unit_variant_wallet_tests.rs | 69 +++++++++++++++++++ key-wallet/src/wallet/accounts.rs | 46 +++++++++---- 4 files changed, 129 insertions(+), 15 deletions(-) diff --git a/key-wallet-ffi/src/error.rs b/key-wallet-ffi/src/error.rs index 683d8e900..bc2fddf58 100644 --- a/key-wallet-ffi/src/error.rs +++ b/key-wallet-ffi/src/error.rs @@ -250,9 +250,12 @@ impl From for FFIError { Error::InvalidNetwork => FFIErrorCode::InvalidNetwork, Error::InvalidAddress(_) => FFIErrorCode::InvalidAddress, Error::Serialization(_) => FFIErrorCode::SerializationError, - Error::WatchOnly | Error::CoinJoinNotEnabled | Error::NoKeySource => { - FFIErrorCode::InvalidState - } + Error::WatchOnly + | Error::CoinJoinNotEnabled + | Error::NoKeySource + | Error::KeylessWalletRequiresAccountKey { + .. + } => FFIErrorCode::InvalidState, Error::Bip32(_) | Error::Slip10(_) | Error::BLS(_) diff --git a/key-wallet/src/error.rs b/key-wallet/src/error.rs index 5e3105e7d..dd11fb10d 100644 --- a/key-wallet/src/error.rs +++ b/key-wallet/src/error.rs @@ -41,6 +41,16 @@ pub enum Error { WatchOnly, /// No key source available for address derivation NoKeySource, + /// A keyless wallet (watch-only or external-signable) was asked to derive an + /// account from an in-wallet private key it does not hold. The caller must + /// instead supply the account's key material via the `Some(..)` argument of + /// the relevant `add_*account` method. + KeylessWalletRequiresAccountKey { + /// The account type the caller was trying to add. + account_type: crate::account::AccountType, + /// What must be supplied instead, e.g. `"extended public key (xpub)"`. + required_key: &'static str, + }, } impl fmt::Display for Error { @@ -63,6 +73,16 @@ impl fmt::Display for Error { Error::InvalidParameter(s) => write!(f, "Invalid parameter: {}", s), Error::WatchOnly => write!(f, "Watch-only wallet: private keys not available"), Error::NoKeySource => write!(f, "No key source available for address derivation"), + Error::KeylessWalletRequiresAccountKey { + account_type, + required_key, + } => write!( + f, + "Wallet has no in-wallet private key (watch-only / external-signable): cannot \ + derive the {} account from a root key. Supply the account's {} via the \ + Some(..) argument of the add_*account method.", + account_type, required_key + ), } } } diff --git a/key-wallet/src/tests/unit_variant_wallet_tests.rs b/key-wallet/src/tests/unit_variant_wallet_tests.rs index 9234695cf..67b5ceeab 100644 --- a/key-wallet/src/tests/unit_variant_wallet_tests.rs +++ b/key-wallet/src/tests/unit_variant_wallet_tests.rs @@ -6,6 +6,8 @@ //! the host to track addresses or route signing requests to an external device. use crate::account::account_collection::AccountCollection; +use crate::account::{AccountType, StandardAccountType}; +use crate::error::Error; use crate::mnemonic::{Language, Mnemonic}; use crate::wallet::initialization::WalletAccountCreationOptions; use crate::wallet::{Wallet, WalletType}; @@ -161,6 +163,73 @@ fn derive_extended_public_key_error_points_to_per_account_xpub() { assert!(watch.derive_extended_public_key(&hardened).is_err()); } +// -------- add_*account(_, None) is actionable on keyless wallets ------- + +#[test] +fn add_account_none_on_keyless_wallet_returns_typed_error() { + let full = built_full_wallet(); + let account_type = AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account, + }; + + for mut wallet in [ + Wallet::new_watch_only(Network::Testnet, full.wallet_id, AccountCollection::new()), + Wallet::new_external_signable(Network::Testnet, full.wallet_id, AccountCollection::new()), + ] { + let err = wallet.add_account(account_type, None).unwrap_err(); + assert!( + matches!( + err, + Error::KeylessWalletRequiresAccountKey { account_type: at, .. } if at == account_type + ), + "expected KeylessWalletRequiresAccountKey, got: {err:?}" + ); + // The message must name its own remedy: the Some(..) argument. + let msg = err.to_string(); + assert!( + msg.contains("Some(") && msg.contains("private key"), + "error should name the Some(..) remedy; got: {msg}" + ); + } +} + +#[cfg(feature = "bls")] +#[test] +fn add_bls_account_none_on_keyless_wallet_returns_typed_error() { + let full = built_full_wallet(); + let account_type = AccountType::ProviderOperatorKeys; + let mut wallet = + Wallet::new_external_signable(Network::Testnet, full.wallet_id, AccountCollection::new()); + + let err = wallet.add_bls_account(account_type, None).unwrap_err(); + assert!( + matches!( + err, + Error::KeylessWalletRequiresAccountKey { account_type: at, .. } if at == account_type + ), + "expected KeylessWalletRequiresAccountKey, got: {err:?}" + ); +} + +#[cfg(feature = "eddsa")] +#[test] +fn add_eddsa_account_none_on_keyless_wallet_returns_typed_error() { + let full = built_full_wallet(); + let account_type = AccountType::ProviderPlatformKeys; + let mut wallet = + Wallet::new_watch_only(Network::Testnet, full.wallet_id, AccountCollection::new()); + + let err = wallet.add_eddsa_account(account_type, None).unwrap_err(); + assert!( + matches!( + err, + Error::KeylessWalletRequiresAccountKey { account_type: at, .. } if at == account_type + ), + "expected KeylessWalletRequiresAccountKey, got: {err:?}" + ); +} + // -------- bincode round-trip ------------------------------------------- #[test] diff --git a/key-wallet/src/wallet/accounts.rs b/key-wallet/src/wallet/accounts.rs index 2ff00d229..776065c54 100644 --- a/key-wallet/src/wallet/accounts.rs +++ b/key-wallet/src/wallet/accounts.rs @@ -18,13 +18,15 @@ impl Wallet { /// # Arguments /// * `account_type` - The type of account to create /// * `account_xpub` - Optional extended public key for the account. If not provided, - /// the account will be derived from the wallet's private key. - /// This will fail if the wallet doesn't have a private key - /// (watch-only wallets or externally managed wallets where - /// the private key is stored securely outside of the SDK). + /// the account is derived from the wallet's in-wallet private key. /// /// # Returns /// Ok(()) if the account was successfully added + /// + /// # Errors + /// Returns [`Error::KeylessWalletRequiresAccountKey`] when `account_xpub` is + /// `None` on a keyless wallet (watch-only / external-signable): such wallets + /// hold no in-wallet private key, so pass the account's xpub via `Some(..)`. pub fn add_account( &mut self, account_type: AccountType, @@ -41,8 +43,12 @@ impl Wallet { // Derive from wallet's private key let derivation_path = account_type.derivation_path(self.network)?; - // This will fail if the wallet doesn't have a private key (watch-only or externally managed) - let root_key = self.root_extended_priv_key()?; + let root_key = self.root_extended_priv_key().map_err(|_| { + Error::KeylessWalletRequiresAccountKey { + account_type, + required_key: "extended public key (xpub)", + } + })?; let master_key = root_key.to_extended_priv_key(self.network); let secp = Secp256k1::new(); let account_xpriv = @@ -70,10 +76,14 @@ impl Wallet { /// # Arguments /// * `account_type` - The type of account (must be ProviderOperatorKeys) /// * `bls_seed` - Optional 32-byte seed for BLS key generation. If not provided, - /// the account will be derived from the wallet's private key. + /// the account is derived from the wallet's in-wallet private key. /// /// # Returns /// Ok(()) if the account was successfully added + /// + /// # Errors + /// Returns [`Error::KeylessWalletRequiresAccountKey`] when `bls_seed` is `None` + /// on a keyless wallet (watch-only / external-signable): pass the seed via `Some(..)`. #[cfg(feature = "bls")] pub fn add_bls_account( &mut self, @@ -98,8 +108,12 @@ impl Wallet { // Derive from wallet's private key let derivation_path = account_type.derivation_path(self.network)?; - // This will fail if the wallet doesn't have a private key - let root_key = self.root_extended_priv_key()?; + let root_key = self.root_extended_priv_key().map_err(|_| { + Error::KeylessWalletRequiresAccountKey { + account_type, + required_key: "32-byte BLS seed", + } + })?; let master_key = root_key.to_extended_priv_key(self.network); let secp = Secp256k1::new(); let account_xpriv = @@ -131,10 +145,14 @@ impl Wallet { /// # Arguments /// * `account_type` - The type of account (must be ProviderPlatformKeys) /// * `ed25519_seed` - Optional 32-byte seed for Ed25519 key generation. If not provided, - /// the account will be derived from the wallet's private key. + /// the account is derived from the wallet's in-wallet private key. /// /// # Returns /// Ok(()) if the account was successfully added + /// + /// # Errors + /// Returns [`Error::KeylessWalletRequiresAccountKey`] when `ed25519_seed` is `None` + /// on a keyless wallet (watch-only / external-signable): pass the seed via `Some(..)`. #[cfg(feature = "eddsa")] pub fn add_eddsa_account( &mut self, @@ -159,8 +177,12 @@ impl Wallet { // Derive from wallet's private key let derivation_path = account_type.derivation_path(self.network)?; - // This will fail if the wallet doesn't have a private key - let root_key = self.root_extended_priv_key()?; + let root_key = self.root_extended_priv_key().map_err(|_| { + Error::KeylessWalletRequiresAccountKey { + account_type, + required_key: "32-byte Ed25519 seed", + } + })?; let master_key = root_key.to_extended_priv_key(self.network); let secp = Secp256k1::new(); let account_xpriv = From 1860089eddea27f66e5b3e637d18a73ae138478c Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Fri, 10 Jul 2026 01:28:12 +1000 Subject: [PATCH 033/149] chore(dash-spv): add recent mainnet/testnet checkpoints (#857) Extend `mainnet_checkpoints()` up to height 2,500,000 (previously 2,300,000) and `testnet_checkpoints()` up to height 1,500,000 (previously 1,400,000), keeping the existing 50k-block cadence. Mainnet entries were sourced from Blockchair and cross-verified against cryptoid so every block hash was confirmed by two independent providers, and Blockchair's raw-header fields were checked against its own dashboard `chainwork`. Testnet entries came straight from a local `dashd` via `getblockheader`. New entries carry `masternode_list_name: None` since that field has no consumer in `dash-spv`. Extend `test_mainnet_checkpoint_consistency` and `test_testnet_checkpoint_consistency` with a monotonic `chain_work` check, which confirms each new checkpoint chains onto the previous cumulative work. --- dash-spv/src/chain/checkpoints.rs | 72 +++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/dash-spv/src/chain/checkpoints.rs b/dash-spv/src/chain/checkpoints.rs index 6bb53ba5c..2eec44d34 100644 --- a/dash-spv/src/chain/checkpoints.rs +++ b/dash-spv/src/chain/checkpoints.rs @@ -733,6 +733,54 @@ pub fn mainnet_checkpoints() -> Vec { 972444458, Some("ML2300000__70232"), ), + // Height 2350000 + create_checkpoint( + 2350000, + "00000000000000258216a62e8c7170be1207335474ddfa667092a71e3d4162d2", + "000000000000002702e07f9f3402026e4cd5707961ce54af22c873331a329708", + 1759648416, + 0x192f2bfb, + "0x00000000000000000000000000000000000000000000ada31a5dd056c969c842", + "3e77d2ed0c24aab79096d700cdc4fe3ea9502fcec38ee114c81c9063842a6725", + 3635235496, + None, + ), + // Height 2400000 + create_checkpoint( + 2400000, + "000000000000002883c9151a37092287c1773c5a4d175a8f688373daa631f934", + "000000000000001d1d2f9edbd2792ebbfa001e42ff47e6ef81efeaef68df4fe7", + 1767528049, + 0x19377eb0, + "0x00000000000000000000000000000000000000000000b2955e92bc38ce3a61ed", + "9247dc06cf2e952b6a97d7c0057946933c1b5699ce6b76b86583775c26ae3b01", + 3220391604, + None, + ), + // Height 2450000 + create_checkpoint( + 2450000, + "00000000000000074832a3ff13968b2aa0c841fcdfced9c3619257242470f25f", + "0000000000000023c3bd45c16e76cfe33e5cac0e3c4c392b61ee8f4e9d9e345a", + 1775408495, + 0x192e7792, + "0x00000000000000000000000000000000000000000000b716ba49112c2a375b0f", + "c6b4c57362b96e300b1b3ad20aa73e3fb0c40fe67049a51bd1f0b2d034bd74dc", + 1523946232, + None, + ), + // Height 2500000 + create_checkpoint( + 2500000, + "000000000000000ac721a208c5efba67e76cb08796324de3519fefa32a5b2f44", + "000000000000003908fc91c23e32714e3d4e07254f8e2e9a0a803c12c3f11083", + 1783288591, + 0x193d61e0, + "0x00000000000000000000000000000000000000000000bb84d5645e0656eac7db", + "053838492199315a53ada9322184cef33c08ea4cbdc9e0a5c30a5a8857cd469e", + 3924859976, + None, + ), ] } @@ -1087,6 +1135,30 @@ pub fn testnet_checkpoints() -> Vec { 516400, None, ), + // Height 1450000 + create_checkpoint( + 1450000, + "000000fa4225fa022d4216eabb176848d61d03027df12e0685f82d8b13c60f03", + "000000f16d17ad5b97a6c7e1c431953cc589155a6f24b981bc542f68759d48f8", + 1775019403, + 0x1e01222f, + "0x00000000000000000000000000000000000000000000000003eb87177577500b", + "914a43b78cd4e12cac031e44f8cb0c63f8a3bfd3a238d08ebf83af61962cbb49", + 61968, + None, + ), + // Height 1500000 + create_checkpoint( + 1500000, + "000000b625f73fab6cb338c31cb656680ccfb3b574b9225022c8e90e293a0c12", + "000000376fa86df5684a69d8055daa1ddaf349844075d55e9a190f2b1899f4f0", + 1782027920, + 0x1e00ca28, + "0x00000000000000000000000000000000000000000000000003ec5aff7410d4a1", + "7ad0f11d8ed65954ce5f42e4aebbe10fd45cac880c31dee76b4b9dcc08382945", + 461766, + None, + ), ] } From 078681011e0fade0d19137f5b2fdad2c14d38601 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Fri, 10 Jul 2026 18:39:57 +0700 Subject: [PATCH 034/149] fix(dash-spv): derive storage lockfile path from full directory name (#862) DiskStorageManager::new used Path::set_extension("lock") to derive the data-directory lockfile. For directory names containing dots (e.g. probes/95.183.53.19-9999), set_extension replaces everything after the last dot, so distinct sibling directories collided on the same lockfile and spuriously failed with "Data directory ... is already in use by another process". Append ".lock" to the full final path component instead. Dot-free paths are unchanged (discovery -> discovery.lock). Co-authored-by: Claude Fable 5 --- dash-spv/src/storage/mod.rs | 65 +++++++++++++++++++++++++++++++++---- 1 file changed, 59 insertions(+), 6 deletions(-) diff --git a/dash-spv/src/storage/mod.rs b/dash-spv/src/storage/mod.rs index 453b60037..73135493b 100644 --- a/dash-spv/src/storage/mod.rs +++ b/dash-spv/src/storage/mod.rs @@ -20,7 +20,7 @@ use async_trait::async_trait; use dashcore::hash_types::FilterHeader; use dashcore::prelude::CoreBlockHeight; use std::ops::Range; -use std::path::PathBuf; +use std::path::{Path, PathBuf}; use std::sync::Arc; use std::time::Duration; use tokio::sync::RwLock; @@ -99,16 +99,34 @@ pub struct DiskStorageManager { _lock_file: LockFile, } +/// Derives the lockfile path for a storage directory by appending `.lock` to +/// the directory's full name. `Path::set_extension` is unsuitable here: it +/// replaces everything after the last dot, so directory names containing dots +/// (e.g. `95.183.53.19-9999` and `95.183.53.20-9999`) would collide on the +/// same lockfile. +fn lock_file_path(storage_path: &Path) -> PathBuf { + match storage_path.file_name() { + Some(name) => { + let mut lock_name = name.to_os_string(); + lock_name.push(".lock"); + storage_path.with_file_name(lock_name) + } + // Paths without a final component (e.g. `/` or `..`); keep the + // previous `set_extension` behavior. + None => { + let mut lock_file = storage_path.to_path_buf(); + lock_file.set_extension("lock"); + lock_file + } + } +} + impl DiskStorageManager { pub async fn new(config: &ClientConfig) -> StorageResult { use std::fs; let storage_path = config.storage_path.clone(); - let lock_file = { - let mut lock_file = storage_path.clone(); - lock_file.set_extension("lock"); - lock_file - }; + let lock_file = lock_file_path(&storage_path); fs::create_dir_all(&storage_path)?; @@ -417,6 +435,41 @@ mod tests { BlockHeader::dummy_batch(r).into_iter().map(HashedBlockHeader::from).collect() } + #[test] + fn test_lock_file_path_appends_to_full_name() { + // Dot-free directory names keep the historical behavior. + assert_eq!( + lock_file_path(Path::new("data/discovery")), + PathBuf::from("data/discovery.lock") + ); + + // Dotted directory names must not have their "extension" replaced: + // distinct sibling directories need distinct lockfiles. + let a = lock_file_path(Path::new("probes/95.183.53.19-9999")); + let b = lock_file_path(Path::new("probes/95.183.53.20-9999")); + assert_eq!(a, PathBuf::from("probes/95.183.53.19-9999.lock")); + assert_eq!(b, PathBuf::from("probes/95.183.53.20-9999.lock")); + assert_ne!(a, b); + } + + #[tokio::test] + async fn test_dotted_sibling_dirs_do_not_share_lock() -> Result<(), Box> + { + let temp_dir = TempDir::new()?; + let path_a = temp_dir.path().join("95.183.53.19-9999"); + let path_b = temp_dir.path().join("95.183.53.20-9999"); + + let config_a = ClientConfig::testnet().with_storage_path(&path_a); + let config_b = ClientConfig::testnet().with_storage_path(&path_b); + + let _storage_a = DiskStorageManager::new(&config_a).await?; + // Before the fix this failed with "Data directory ... is already in + // use by another process" because both paths derived the same lockfile. + let _storage_b = DiskStorageManager::new(&config_b).await?; + + Ok(()) + } + #[tokio::test] async fn test_store_load_headers() -> Result<(), Box> { // Create a temporary directory for the test From 66f08730bec0a6fedda43be1842b115baa98c1db Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Fri, 10 Jul 2026 18:53:28 +0700 Subject: [PATCH 035/149] fix(key-wallet): zeroize ExtendedBLSPrivKey and ExtendedEd25519PrivKey on drop (#859) The BLS and Ed25519 extended private key types held raw secret material but had no Zeroize/Drop implementations, so their secret bytes were left in freed heap memory when dropped. Their secp256k1 siblings (ExtendedPrivKey, RootExtendedPrivKey) already wipe themselves on drop. Mirror the secp256k1 pattern for both types: a hand-written Zeroize that wipes the secret key, chain code, and derivation metadata, plus a Drop impl that calls it. Add a Zeroize impl for Fingerprint so modules outside bip32 can wipe it too. Also replace the derived PartialEq on ExtendedEd25519PrivKey, which short-circuited over the raw 32-byte secret, with a constant-time comparison of the secret bytes. Fixes #858 Co-authored-by: Claude Fable 5 --- key-wallet/src/bip32.rs | 9 +++ key-wallet/src/derivation_bls_bip32.rs | 44 ++++++++++++++ key-wallet/src/derivation_slip10.rs | 81 +++++++++++++++++++++++++- 3 files changed, 133 insertions(+), 1 deletion(-) diff --git a/key-wallet/src/bip32.rs b/key-wallet/src/bip32.rs index 041794a1d..f48bcaee7 100644 --- a/key-wallet/src/bip32.rs +++ b/key-wallet/src/bip32.rs @@ -120,6 +120,15 @@ impl zeroize::Zeroize for ChainCode { } } +// Manual implementation of Zeroize for Fingerprint, so extended-key types in +// other modules (BLS, Ed25519) can wipe their derivation metadata too. +// Note: Fingerprint is Copy, so this won't prevent copies in registers/stack +impl zeroize::Zeroize for Fingerprint { + fn zeroize(&mut self) { + zeroize::Zeroize::zeroize(&mut self.0); + } +} + impl fmt::LowerHex for ChainCode { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { for &byte in &self.0 { diff --git a/key-wallet/src/derivation_bls_bip32.rs b/key-wallet/src/derivation_bls_bip32.rs index 7c76caca1..ec85f0333 100644 --- a/key-wallet/src/derivation_bls_bip32.rs +++ b/key-wallet/src/derivation_bls_bip32.rs @@ -78,6 +78,33 @@ pub struct ExtendedBLSPrivKey { pub chain_code: ChainCode, } +// Hand-written (not `#[derive(Zeroize)]`): `BlsSecretKey` has no `Zeroize` +// impl of its own, but its inner scalar (public field `0`) does, so we wipe +// the value field by field. `Drop` (below) calls this, so the key is wiped +// automatically on scope exit with no caller action required. +// Cf. `ExtendedPrivKey` in `bip32`. +impl zeroize::Zeroize for ExtendedBLSPrivKey { + fn zeroize(&mut self) { + // Secret key material. + self.private_key.0.zeroize(); + self.chain_code.zeroize(); + // Derivation metadata — cleared too so the whole value is wiped. + self.depth.zeroize(); + self.parent_fingerprint.zeroize(); + self.child_number = ChildNumber::Normal { + index: 0, + }; + self.network = Network::Mainnet; // repr(u8)=0 discriminant, the "zero" value + } +} + +impl Drop for ExtendedBLSPrivKey { + fn drop(&mut self) { + use zeroize::Zeroize; + self.zeroize(); + } +} + impl ExtendedBLSPrivKey { /// Create a new master key from a seed pub fn new_master(network: Network, seed: &[u8]) -> Result { @@ -1213,4 +1240,21 @@ mod tests { assert_eq!(child_hardened.parent_fingerprint, master.fingerprint()); assert_eq!(child_unhardened.parent_fingerprint, master.fingerprint()); } + + #[test] + fn test_zeroize_clears_key_material() { + use zeroize::Zeroize; + + let seed = [42u8; 32]; + let mut key = ExtendedBLSPrivKey::new_master(Network::Testnet, &seed).unwrap(); + assert_ne!(key.private_key.to_be_bytes(), [0u8; 32]); + assert_ne!(key.chain_code.as_ref(), &[0u8; 32]); + + key.zeroize(); + + assert_eq!(key.private_key.to_be_bytes(), [0u8; 32]); + assert_eq!(key.chain_code.as_ref(), &[0u8; 32]); + assert_eq!(key.depth, 0); + assert_eq!(key.parent_fingerprint, Fingerprint::default()); + } } diff --git a/key-wallet/src/derivation_slip10.rs b/key-wallet/src/derivation_slip10.rs index b24f1c91f..804154003 100644 --- a/key-wallet/src/derivation_slip10.rs +++ b/key-wallet/src/derivation_slip10.rs @@ -26,7 +26,7 @@ pub use dashcore::ed25519_dalek::VerifyingKey as Ed25519PublicKey; pub use crate::bip32::DerivationPath; /// Extended Ed25519 private key for SLIP-0010 -#[derive(Clone, PartialEq, Eq)] +#[derive(Clone)] pub struct ExtendedEd25519PrivKey { /// Network this key is for pub network: Network, @@ -42,6 +42,52 @@ pub struct ExtendedEd25519PrivKey { pub chain_code: ChainCode, } +// Hand-written instead of derived so the raw secret bytes are compared in +// constant time (a derived `==` on `[u8; 32]` short-circuits on the first +// differing byte, leaking how much of the secret matches). +impl PartialEq for ExtendedEd25519PrivKey { + fn eq(&self, other: &Self) -> bool { + let mut diff = 0u8; + for (a, b) in self.private_key.iter().zip(other.private_key.iter()) { + diff |= a ^ b; + } + // black_box keeps the compiler from turning the fold back into an + // early-exit comparison. + core::hint::black_box(diff) == 0 + && self.network == other.network + && self.depth == other.depth + && self.parent_fingerprint == other.parent_fingerprint + && self.child_number == other.child_number + && self.chain_code == other.chain_code + } +} + +impl Eq for ExtendedEd25519PrivKey {} + +// `Drop` (below) calls this, so the key is wiped automatically on scope exit +// with no caller action required. Cf. `ExtendedPrivKey` in `bip32`. +impl zeroize::Zeroize for ExtendedEd25519PrivKey { + fn zeroize(&mut self) { + // Secret key material. + self.private_key.zeroize(); + self.chain_code.zeroize(); + // Derivation metadata — cleared too so the whole value is wiped. + self.depth.zeroize(); + self.parent_fingerprint.zeroize(); + self.child_number = ChildNumber::Normal { + index: 0, + }; + self.network = Network::Mainnet; // repr(u8)=0 discriminant, the "zero" value + } +} + +impl Drop for ExtendedEd25519PrivKey { + fn drop(&mut self) { + use zeroize::Zeroize; + self.zeroize(); + } +} + impl ExtendedEd25519PrivKey { /// Create a new master key from seed pub fn new_master(network: Network, seed: &[u8]) -> Result { @@ -741,4 +787,37 @@ mod test { hex::encode(private_key) } + + #[test] + fn test_zeroize_clears_key_material() { + use zeroize::Zeroize; + + let seed = hex::decode(CASE_1_SEED).unwrap(); + let mut key = ExtendedEd25519PrivKey::new_master(Network::Mainnet, &seed).unwrap(); + assert_ne!(key.private_key, [0u8; 32]); + assert_ne!(key.chain_code.as_ref(), &[0u8; 32]); + + key.zeroize(); + + assert_eq!(key.private_key, [0u8; 32]); + assert_eq!(key.chain_code.as_ref(), &[0u8; 32]); + assert_eq!(key.depth, 0); + assert_eq!(key.parent_fingerprint, Fingerprint::default()); + } + + #[test] + fn test_partial_eq_matches_field_equality() { + let seed = hex::decode(CASE_1_SEED).unwrap(); + let master = ExtendedEd25519PrivKey::new_master(Network::Mainnet, &seed).unwrap(); + + assert_eq!(master, master.clone()); + + let child = master.ckd_priv(ChildNumber::from_hardened_idx(0).unwrap()).unwrap(); + assert_ne!(master, child); + + // Same secret but different metadata must not compare equal + let mut other_network = master.clone(); + other_network.network = Network::Testnet; + assert_ne!(master, other_network); + } } From 845ee9e30212ee5bcddc0f78be03611940b235a3 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Fri, 10 Jul 2026 23:47:39 +1000 Subject: [PATCH 036/149] fix(spv): match masternode owner/voting key hashes in compact filters (#863) During SPV historical sync, `dash-spv` chose which blocks to download by matching the wallet's watched scriptPubKeys against BIP157/158 compact filters. The matcher built its `FilterQuery` from scriptPubKeys only. Dash Core, however, also inserts a `ProRegTx`'s owner (`keyIDOwner`) and voting (`keyIDVoting`) key hashes into the block's compact filter as bare 20-byte `hash160` elements, not as P2PKH scripts (see `ExtractSpecialTxFilterElements`). A masternode owner who controls only the owner/voting keys, with no wallet-owned input or output in the `ProRegTx`, therefore never matched, the block was never downloaded, and the payload-aware wallet scan never ran on it. This adds each provider owner and voting key hash (bare 20-byte `hash160`) to the compact-filter query. `key-wallet` surfaces them via `WalletInfoInterface::monitored_filter_elements`, extracting the same bytes (same order) the BIP37 bloom path already inserts via `address_payload_bytes`. `key-wallet-manager` exposes `WalletInterface::monitored_filter_elements_for` and folds the extra elements into the shared `FilterQuery` (its `other` length bucket) in `check_compact_filters_for_elements` (renamed from `check_compact_filters_for_script_pubkeys`). `dash-spv`'s filter manager now passes each wallet's elements alongside its scripts in both the batch scan and the new-script rescan, including the per-wallet attribution queries. Step 1 of #861: owner/voting only. Collateral outpoint and `proTxHash` follow in stacked branches. --- dash-spv/src/sync/filters/manager.rs | 85 ++++++++++++++----- key-wallet-manager/src/lib.rs | 2 +- key-wallet-manager/src/matching.rs | 85 ++++++++++++++++--- key-wallet-manager/src/process_block.rs | 7 ++ key-wallet-manager/src/wallet_interface.rs | 12 +++ .../wallet_info_interface.rs | 37 ++++++++ 6 files changed, 196 insertions(+), 32 deletions(-) diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index 9ccbc3c5e..58192af13 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -23,12 +23,25 @@ use crate::validation::{FilterValidationInput, FilterValidator, Validator}; use crate::sync::progress::ProgressPercentage; use dashcore::hash_types::FilterHeader; use key_wallet_manager::WalletInterface; -use key_wallet_manager::{check_compact_filters_for_script_pubkeys, FilterMatchKey, WalletId}; +use key_wallet_manager::{check_compact_filters_for_elements, FilterMatchKey, WalletId}; use tokio::sync::RwLock; /// Batch size for processing filters. const BATCH_PROCESSING_SIZE: u32 = 5000; +/// Snapshot of a behind wallet's compact-filter query inputs for a batch scan. +struct WalletScanState { + /// The wallet these inputs belong to. + id: WalletId, + /// The wallet's committed sync checkpoint; heights at or below it are skipped. + synced: u32, + /// Monitored scriptPubKeys. + scripts: Vec, + /// Bare `hash160` filter elements (owner/voting key hashes) a compact + /// filter carries beyond the scriptPubKeys. + elements: Vec>, +} + /// Maximum number of batches to scan ahead while waiting for blocks. const MAX_LOOKAHEAD_BATCHES: usize = 3; @@ -646,21 +659,33 @@ impl = { + // own progress are skipped during the rescan, plus the bare + // owner/voting filter elements a compact filter carries beyond the + // wallet's scriptPubKeys. + let mut synced_heights: HashMap = HashMap::new(); + let mut filter_elements: HashMap>> = HashMap::new(); + { let wallet = self.wallet.read().await; - new_scripts.keys().map(|id| (*id, wallet.wallet_synced_height(id))).collect() - }; + for id in new_scripts.keys() { + synced_heights.insert(*id, wallet.wallet_synced_height(id)); + filter_elements.insert(*id, wallet.monitored_filter_elements_for(id)); + } + } let mut block_to_wallets: BTreeMap> = BTreeMap::new(); for (wallet_id, scripts) in new_scripts { - if scripts.is_empty() { + let elements = filter_elements.get(wallet_id).map(Vec::as_slice).unwrap_or(&[]); + if scripts.is_empty() && elements.is_empty() { continue; } let scripts_vec: Vec = scripts.iter().cloned().collect(); let min_synced = synced_heights.get(wallet_id).copied().unwrap_or(0); - let matches = - check_compact_filters_for_script_pubkeys(batch_filters, &scripts_vec, min_synced); + let matches = check_compact_filters_for_elements( + batch_filters, + &scripts_vec, + elements, + min_synced, + ); for key in matches { block_to_wallets.entry(key).or_default().insert(*wallet_id); } @@ -742,12 +767,20 @@ impl)> = Vec::new(); + let mut wallet_states: Vec = Vec::new(); for wallet_id in &behind { let synced = wallet.wallet_synced_height(wallet_id); let scripts = wallet.monitored_script_pubkeys_for(wallet_id); - if !scripts.is_empty() { - wallet_states.push((*wallet_id, synced, scripts)); + // Bare owner/voting key hashes a compact filter carries beyond the + // wallet's scriptPubKeys. + let elements = wallet.monitored_filter_elements_for(wallet_id); + if !scripts.is_empty() || !elements.is_empty() { + wallet_states.push(WalletScanState { + id: *wallet_id, + synced, + scripts, + elements, + }); } } drop(wallet); @@ -777,17 +810,25 @@ impl = - wallet_states.iter().flat_map(|(_, _, scripts)| scripts.iter().cloned()).collect(); - let min_synced = wallet_states.iter().map(|(_, synced, _)| *synced).min().unwrap_or(0); + wallet_states.iter().flat_map(|s| s.scripts.iter().cloned()).collect(); + let union_elements: Vec> = + wallet_states.iter().flat_map(|s| s.elements.iter().cloned()).collect(); + let min_synced = wallet_states.iter().map(|s| s.synced).min().unwrap_or(0); - // Pre-group each wallet's scripts by length once; reused across every matched filter. + // Pre-group each wallet's scripts and bare elements by length once; + // reused across every matched filter. let wallet_queries: Vec<(WalletId, u32, FilterQuery)> = wallet_states .iter() - .map(|(id, synced, scripts)| { - (*id, *synced, scripts.iter().map(|s| s.as_bytes()).collect()) + .map(|s| { + let mut query: FilterQuery = s.scripts.iter().map(|sp| sp.as_bytes()).collect(); + for element in &s.elements { + query.push(element); + } + (s.id, s.synced, query) }) .collect(); @@ -797,8 +838,12 @@ impl> = BTreeMap::new(); for key in matches { diff --git a/key-wallet-manager/src/lib.rs b/key-wallet-manager/src/lib.rs index facf471d6..b3337e7a3 100644 --- a/key-wallet-manager/src/lib.rs +++ b/key-wallet-manager/src/lib.rs @@ -21,7 +21,7 @@ mod wallet_interface; pub use error::WalletError; pub use events::{DerivedAddress, WalletEvent}; -pub use matching::{check_compact_filters_for_script_pubkeys, FilterMatchKey}; +pub use matching::{check_compact_filters_for_elements, FilterMatchKey}; pub use wallet_interface::{BlockProcessingResult, MempoolTransactionResult, WalletInterface}; use dashcore::blockdata::transaction::Transaction; diff --git a/key-wallet-manager/src/matching.rs b/key-wallet-manager/src/matching.rs index debafb7d9..18c8bc19c 100644 --- a/key-wallet-manager/src/matching.rs +++ b/key-wallet-manager/src/matching.rs @@ -26,18 +26,28 @@ impl FilterMatchKey { } } -/// Check compact filters against a set of scriptPubKeys and return the keys -/// that matched. +/// Check compact filters against a set of scriptPubKeys plus any bare +/// filter elements, returning the keys that matched. +/// +/// `extra_elements` carries arbitrary-length elements a peer inserts into the +/// compact filter that are not scriptPubKeys, notably the provider owner/voting +/// key hashes (bare 20-byte `hash160`) from a `ProRegTx`. They fold into the +/// same [`FilterQuery`] as the scripts, landing in its `other` length bucket. /// /// Entries with `key.height() <= min_height` are skipped. Pass `0` to test /// every filter in the input. -pub fn check_compact_filters_for_script_pubkeys( +pub fn check_compact_filters_for_elements( input: &HashMap, script_pubkeys: &[ScriptBuf], + extra_elements: &[Vec], min_height: CoreBlockHeight, ) -> BTreeSet { - // Group the scripts by length once; the same query is reused across every filter. - let query: FilterQuery = script_pubkeys.iter().map(|s| s.as_bytes()).collect(); + // Group the scripts and bare elements by length once; the same query is + // reused across every filter. + let mut query: FilterQuery = script_pubkeys.iter().map(|s| s.as_bytes()).collect(); + for element in extra_elements { + query.push(element); + } let match_filter = |(key, filter): (&FilterMatchKey, &BlockFilter)| { if key.height() <= min_height { return None; @@ -70,6 +80,8 @@ pub fn check_compact_filters_for_script_pubkeys( #[cfg(test)] mod tests { use super::*; + use dashcore::address::Payload; + use dashcore::bip158::BlockFilterWriter; use dashcore::{Address, Block, Transaction}; use key_wallet::Network; @@ -77,9 +89,17 @@ mod tests { addresses.iter().map(|a| a.script_pubkey()).collect() } + /// The bare 20-byte `hash160` a peer inserts for a P2PKH address. + fn hash160_of(address: &Address) -> Vec { + match address.payload() { + Payload::PubkeyHash(hash) => <[u8; 20]>::from(*hash).to_vec(), + other => panic!("expected P2PKH dummy address, got {other:?}"), + } + } + #[test] fn test_empty_input_returns_empty() { - let result = check_compact_filters_for_script_pubkeys(&HashMap::new(), &[], 0); + let result = check_compact_filters_for_elements(&HashMap::new(), &[], &[], 0); assert!(result.is_empty()); } @@ -94,7 +114,7 @@ mod tests { let mut input = HashMap::new(); input.insert(key.clone(), filter); - let output = check_compact_filters_for_script_pubkeys(&input, &[], 0); + let output = check_compact_filters_for_elements(&input, &[], &[], 0); assert!(!output.contains(&key)); } @@ -110,7 +130,7 @@ mod tests { input.insert(key.clone(), filter); let scripts = scripts_for(&[address]); - let output = check_compact_filters_for_script_pubkeys(&input, &scripts, 0); + let output = check_compact_filters_for_elements(&input, &scripts, &[], 0); assert!(output.contains(&key)); } @@ -128,7 +148,7 @@ mod tests { input.insert(key.clone(), filter); let scripts = scripts_for(&[address]); - let output = check_compact_filters_for_script_pubkeys(&input, &scripts, 0); + let output = check_compact_filters_for_elements(&input, &scripts, &[], 0); assert!(!output.contains(&key)); } @@ -159,7 +179,7 @@ mod tests { input.insert(key_3.clone(), filter_3); let scripts = scripts_for(&[address_1, address_2]); - let output = check_compact_filters_for_script_pubkeys(&input, &scripts, 0); + let output = check_compact_filters_for_elements(&input, &scripts, &[], 0); assert_eq!(output.len(), 2); assert!(output.contains(&key_1)); assert!(output.contains(&key_2)); @@ -183,7 +203,7 @@ mod tests { } let scripts = scripts_for(&[address]); - let output = check_compact_filters_for_script_pubkeys(&input, &scripts, 0); + let output = check_compact_filters_for_elements(&input, &scripts, &[], 0); // Verify output is sorted by height (ascending) let heights_out: Vec = output.iter().map(|k| k.height()).collect(); @@ -193,4 +213,47 @@ mod tests { assert_eq!(heights_out, sorted_heights); assert_eq!(heights_out, vec![100, 200, 300, 400, 500]); } + + /// A masternode owner watching only the owner/voting key sees a compact + /// filter that carries that key as a bare 20-byte `hash160` (the way Dash + /// Core's `ExtractSpecialTxFilterElements` inserts it), not as a P2PKH + /// script. The scripts-only query must miss it and the bare-element query + /// must hit it, which is exactly the gap this change closes. + #[test] + fn test_bare_owner_key_hash_requires_extra_element() { + // The owner key hash a peer inserts as a bare element. + let owner_address = Address::dummy(Network::Regtest, 7); + let owner_hash = hash160_of(&owner_address); + assert_eq!(owner_hash.len(), 20, "provider owner key is a 20-byte hash160"); + + // An unrelated output so the filter is realistic. + let unrelated = Address::dummy(Network::Regtest, 99); + let tx = Transaction::dummy(&unrelated, 0..0, &[1]); + let block = Block::dummy(100, vec![tx]); + + // Build the filter like a Dash Core peer: block output scripts plus the + // owner key hash as a bare element. + let mut content = Vec::new(); + { + let mut writer = BlockFilterWriter::new(&mut content, &block); + writer.add_output_scripts(); + writer.add_element(&owner_hash); + writer.finish().expect("finish filter"); + } + let filter = BlockFilter::new(&content); + let key = FilterMatchKey::new(100, block.block_hash()); + + let mut input = HashMap::new(); + input.insert(key.clone(), filter); + + // The P2PKH scriptPubKey form of the owner address does not match the + // bare 20-byte element. + let owner_script = scripts_for(&[owner_address]); + let scripts_only = check_compact_filters_for_elements(&input, &owner_script, &[], 0); + assert!(!scripts_only.contains(&key), "scripts-only query must miss the bare hash"); + + // Carrying the bare hash in `extra_elements` matches. + let with_element = check_compact_filters_for_elements(&input, &[], &[owner_hash], 0); + assert!(with_element.contains(&key), "bare-element query must match"); + } } diff --git a/key-wallet-manager/src/process_block.rs b/key-wallet-manager/src/process_block.rs index c340693ca..1e2d511a7 100644 --- a/key-wallet-manager/src/process_block.rs +++ b/key-wallet-manager/src/process_block.rs @@ -228,6 +228,13 @@ impl WalletInterface for WalletM .unwrap_or_default() } + fn monitored_filter_elements_for(&self, wallet_id: &WalletId) -> Vec> { + self.wallet_infos + .get(wallet_id) + .map(|info| info.monitored_filter_elements()) + .unwrap_or_default() + } + fn watched_outpoints(&self) -> Vec { self.watched_outpoints() } diff --git a/key-wallet-manager/src/wallet_interface.rs b/key-wallet-manager/src/wallet_interface.rs index 99772b096..60ade9b29 100644 --- a/key-wallet-manager/src/wallet_interface.rs +++ b/key-wallet-manager/src/wallet_interface.rs @@ -88,6 +88,18 @@ pub trait WalletInterface: Send + Sync + 'static { /// Get cached scriptPubKeys for every address monitored by `wallet_id`. fn monitored_script_pubkeys_for(&self, wallet_id: &WalletId) -> Vec; + /// Get the bare `hash160` compact-filter elements monitored by `wallet_id` + /// that are not covered by its scriptPubKeys. + /// + /// These are the provider owner/voting key hashes a Dash Core peer inserts + /// into a block's BIP158 compact filter as bare 20-byte elements, so a + /// masternode owner watching only those keys can match the block. The + /// default returns an empty set for implementations that monitor scripts + /// only. + fn monitored_filter_elements_for(&self, _wallet_id: &WalletId) -> Vec> { + Vec::new() + } + /// Get all outpoints the wallet is watching (unspent outputs). /// Used for bloom filter construction to detect spends of our UTXOs. fn watched_outpoints(&self) -> Vec; diff --git a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs index 2b3ea3fbe..6e0b931c0 100644 --- a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs +++ b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs @@ -8,6 +8,7 @@ use super::managed_account_operations::ManagedAccountOperations; use crate::account::{AccountType, ManagedAccountTrait}; use crate::managed_account::managed_account_collection::ManagedAccountCollection; use crate::managed_account::managed_account_ref::ManagedAccountRefMut; +use crate::managed_account::managed_account_type::ManagedAccountType; use crate::managed_account::ManagedCoreFundsAccount; use crate::transaction_checking::TransactionContext; use crate::transaction_checking::WalletTransactionChecker; @@ -15,6 +16,7 @@ use crate::wallet::managed_wallet_info::transaction_building::AccountTypePrefere use crate::wallet::managed_wallet_info::TransactionRecord; use crate::wallet::ManagedWalletInfo; use crate::{Network, Utxo, Wallet, WalletCoreBalance}; +use dashcore::address::Payload; use dashcore::ephemerealdata::chain_lock::ChainLock; use dashcore::ephemerealdata::instant_lock::InstantLock; use dashcore::prelude::CoreBlockHeight; @@ -88,6 +90,19 @@ pub trait WalletInfoInterface: Sized + WalletTransactionChecker + ManagedAccount /// Get cached scriptPubKeys for every monitored address. fn monitored_script_pubkeys(&self) -> Vec; + /// Get bare `hash160` filter elements that a compact filter carries in + /// addition to scriptPubKeys. + /// + /// Dash Core inserts a `ProRegTx`'s owner (`keyIDOwner`) and voting + /// (`keyIDVoting`) key hashes into the block's BIP158 compact filter as + /// bare 20-byte `hash160` elements, not as P2PKH scripts (see Dash Core's + /// `ExtractSpecialTxFilterElements`). A masternode owner watching only + /// those keys therefore never matches on the scriptPubKey path, so the + /// query must also carry the same bare hashes. This returns each provider + /// owner/voting address's 20-byte `hash160`; every other account type is + /// omitted because its scriptPubKeys already cover the query. + fn monitored_filter_elements(&self) -> Vec>; + /// Get all UTXOs for the wallet fn utxos(&self) -> BTreeSet<&Utxo>; @@ -340,6 +355,28 @@ impl WalletInfoInterface for ManagedWalletInfo { scripts } + fn monitored_filter_elements(&self) -> Vec> { + let mut elements = Vec::new(); + for account in self.accounts.all_accounts() { + if matches!( + account.managed_account_type(), + ManagedAccountType::ProviderOwnerKeys { .. } + | ManagedAccountType::ProviderVotingKeys { .. } + ) { + // Extract the bare 20-byte `hash160` a peer inserted into the + // compact filter, matching the byte order the BIP37 bloom path + // uses in `dash-spv`'s `address_payload_bytes`. Provider + // owner/voting keys are P2PKH, so this is the pubkey hash. + elements.extend(account.all_addresses().iter().filter_map(|a| match a.payload() { + Payload::PubkeyHash(hash) => Some(<[u8; 20]>::from(*hash).to_vec()), + Payload::ScriptHash(hash) => Some(<[u8; 20]>::from(*hash).to_vec()), + _ => None, + })); + } + } + elements + } + fn utxos(&self) -> BTreeSet<&Utxo> { let mut utxos = BTreeSet::new(); for account in self.accounts.all_funding_accounts() { From 1ee1c94840f14c63fa34e9d19f6eaaa1d29847c3 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Sat, 11 Jul 2026 20:45:06 +1000 Subject: [PATCH 037/149] fix(spv): match masternode collateral outpoints in compact filters (#864) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dash Core inserts a `ProRegTx`'s `collateralOutpoint` into the block's BIP158 compact filter serialized the consensus way — 32-byte txid followed by the 4-byte little-endian vout, 36 bytes total (see `ExtractSpecialTxFilterElements`). A wallet that owns the 1000-DASH collateral output but holds none of the masternode's owner/voting keys — for example a third party registered the masternode against the wallet's collateral — never matches that `ProRegTx` on the scriptPubKey path, so the block is skipped during compact-filter sync. This extends `ManagedWalletInfo::monitored_filter_elements` to append each watched UTXO's consensus-serialized outpoint after the existing owner/voting hashes. The matcher and manager already fold `monitored_filter_elements` into the shared `FilterQuery`, so no change is needed there. The outpoint elements are gated on the wallet holding provider accounts (`ManagedAccountCollection::has_provider_accounts`). The collateral element only helps a masternode owner — it discovers a `ProRegTx` that references one of the wallet's UTXOs as collateral — and the common paths are already covered without it (an owner-created `ProRegTx` is matched by its fee input's script, a key-holding owner by the owner/voting hashes). A funding-only wallet can never hit that case, so it skips these elements rather than pay the per-UTXO false-positive cost on every compact-filter query. Within a provider wallet every UTXO is watched, to avoid hardcoding network-specific collateral amounts. Step 2 of #861: collateral outpoint. `proTxHash` follows in the next stacked branch. --- key-wallet-manager/src/matching.rs | 47 +++++++++++- .../managed_account_collection.rs | 12 +++ key-wallet/src/test_utils/wallet.rs | 10 ++- .../wallet_info_interface.rs | 76 +++++++++++++++++++ 4 files changed, 142 insertions(+), 3 deletions(-) diff --git a/key-wallet-manager/src/matching.rs b/key-wallet-manager/src/matching.rs index 18c8bc19c..68502ad0e 100644 --- a/key-wallet-manager/src/matching.rs +++ b/key-wallet-manager/src/matching.rs @@ -82,7 +82,7 @@ mod tests { use super::*; use dashcore::address::Payload; use dashcore::bip158::BlockFilterWriter; - use dashcore::{Address, Block, Transaction}; + use dashcore::{Address, Block, OutPoint, Transaction, Txid}; use key_wallet::Network; fn scripts_for(addresses: &[Address]) -> Vec { @@ -256,4 +256,49 @@ mod tests { let with_element = check_compact_filters_for_elements(&input, &[], &[owner_hash], 0); assert!(with_element.contains(&key), "bare-element query must match"); } + + /// A wallet that owns a masternode's 1000-DASH collateral output but not + /// its owner/voting keys sees a compact filter that carries the + /// `ProRegTx`'s `collateralOutpoint` as a bare 36-byte consensus-serialized + /// element (the way Dash Core's `ExtractSpecialTxFilterElements` inserts + /// it), not as one of the block's scriptPubKeys. The scripts-only query + /// must miss it and the query carrying the serialized outpoint must hit it. + #[test] + fn test_collateral_outpoint_requires_extra_element() { + // The collateral outpoint a peer inserts as a bare element. + let outpoint = OutPoint::new(Txid::from([7u8; 32]), 1); + let serialized = dashcore::consensus::encode::serialize(&outpoint); + assert_eq!(serialized.len(), 36, "outpoint serializes to txid ++ le-vout"); + + // An unrelated output so the filter is realistic. + let unrelated = Address::dummy(Network::Regtest, 99); + let tx = Transaction::dummy(&unrelated, 0..0, &[1]); + let block = Block::dummy(100, vec![tx]); + + // Build the filter like a Dash Core peer: block output scripts plus the + // collateral outpoint as a bare element. + let mut content = Vec::new(); + { + let mut writer = BlockFilterWriter::new(&mut content, &block); + writer.add_output_scripts(); + writer.add_element(&serialized); + writer.finish().expect("finish filter"); + } + let filter = BlockFilter::new(&content); + let key = FilterMatchKey::new(100, block.block_hash()); + + let mut input = HashMap::new(); + input.insert(key.clone(), filter); + + // A wallet watching only its own addresses (none of them in this block) + // does not carry the bare 36-byte outpoint element and misses. + let watched = Address::dummy(Network::Regtest, 7); + let scripts_only = + check_compact_filters_for_elements(&input, &scripts_for(&[watched]), &[], 0); + assert!(!scripts_only.contains(&key), "scripts-only query must miss the outpoint"); + + // Carrying the serialized outpoint in `extra_elements` matches. + let with_element = check_compact_filters_for_elements(&input, &[], &[serialized], 0); + assert!(with_element.contains(&key), "serialized-outpoint query must match"); + } } diff --git a/key-wallet/src/managed_account/managed_account_collection.rs b/key-wallet/src/managed_account/managed_account_collection.rs index b79107c31..4cf5f459f 100644 --- a/key-wallet/src/managed_account/managed_account_collection.rs +++ b/key-wallet/src/managed_account/managed_account_collection.rs @@ -96,6 +96,18 @@ impl ManagedAccountCollection { } /// Check if a managed account type exists in the collection + /// Whether the wallet holds any masternode provider account (owner, + /// voting, operator, or platform keys). + /// + /// A wallet with none of these can never own or operate a masternode, so it + /// has no reason to watch for masternode special transactions. + pub(crate) fn has_provider_accounts(&self) -> bool { + self.provider_owner_keys.is_some() + || self.provider_voting_keys.is_some() + || self.provider_operator_keys.is_some() + || self.provider_platform_keys.is_some() + } + pub fn contains_managed_account_type(&self, managed_type: &ManagedAccountType) -> bool { use crate::account::StandardAccountType; diff --git a/key-wallet/src/test_utils/wallet.rs b/key-wallet/src/test_utils/wallet.rs index 2a4756dd9..7f7859a5a 100644 --- a/key-wallet/src/test_utils/wallet.rs +++ b/key-wallet/src/test_utils/wallet.rs @@ -29,8 +29,14 @@ impl TestWalletContext { /// Creates a new random testnet wallet with a BIP44 account and one /// pre-derived receive address. pub fn new_random() -> Self { - let wallet = Wallet::new_random(Network::Testnet, WalletAccountCreationOptions::Default) - .expect("Should create wallet"); + Self::new_random_with_options(WalletAccountCreationOptions::Default) + } + + /// Like [`new_random`](Self::new_random) but with explicit account-creation + /// options, so a test can build e.g. a funding-only wallet with no provider + /// accounts. + pub fn new_random_with_options(options: WalletAccountCreationOptions) -> Self { + let wallet = Wallet::new_random(Network::Testnet, options).expect("Should create wallet"); let mut managed_wallet = ManagedWalletInfo::from_wallet_with_name(&wallet, "Test".to_string(), 0); diff --git a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs index 6e0b931c0..ca7aca184 100644 --- a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs +++ b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs @@ -374,6 +374,31 @@ impl WalletInfoInterface for ManagedWalletInfo { })); } } + // Dash Core inserts a `ProRegTx`'s `collateralOutpoint` into the block's + // compact filter serialized the consensus way — 32-byte txid followed by + // the 4-byte little-endian vout, 36 bytes total (see + // `ExtractSpecialTxFilterElements`). A wallet that owns the collateral + // output but not the masternode's owner/voting keys would otherwise never + // match that `ProRegTx` on the scriptPubKey path, so watch each UTXO's + // outpoint directly. + // + // This is gated on the wallet holding provider accounts because the + // element is only ever useful to a masternode owner: it discovers a + // `ProRegTx` that *references* one of the wallet's UTXOs as collateral. + // The common paths are already covered without it — an owner-created + // `ProRegTx` is matched by its fee input's script, and a key-holding + // owner by the owner/voting hashes above — so the collateral element + // only adds the narrow case where the wallet owns the collateral but + // holds neither the keys nor the funding inputs. A funding-only wallet + // can never hit that case, so it skips these elements rather than pay + // the per-UTXO false-positive cost on every compact-filter query. Within + // a provider wallet every UTXO is watched (not just collateral-sized + // ones) to avoid hardcoding network-specific collateral amounts. + if self.accounts.has_provider_accounts() { + for utxo in self.utxos() { + elements.push(dashcore::consensus::encode::serialize(&utxo.outpoint)); + } + } elements } @@ -505,3 +530,54 @@ impl WalletInfoInterface for ManagedWalletInfo { self.accounts.all_accounts().iter().map(|a| a.monitor_revision()).sum() } } + +#[cfg(test)] +mod tests { + use super::*; + use crate::test_utils::TestWalletContext; + use crate::wallet::initialization::WalletAccountCreationOptions; + + /// A wallet that owns a UTXO must surface that UTXO's outpoint as a bare + /// filter element, consensus-serialized to the 36-byte form Dash Core + /// inserts for a `ProRegTx`'s `collateralOutpoint`, so a compact-filter + /// scan matches a masternode registration against the wallet's collateral + /// even when the wallet holds none of the masternode's keys. + #[tokio::test] + async fn test_watched_utxo_outpoint_is_filter_element() { + let (ctx, _tx) = TestWalletContext::new_random().with_mempool_funding(200_000).await; + let outpoint = ctx.first_utxo().outpoint; + let serialized = dashcore::consensus::encode::serialize(&outpoint); + assert_eq!(serialized.len(), 36, "outpoint serializes to txid ++ le-vout"); + + let elements = ctx.managed_wallet.monitored_filter_elements(); + assert!( + elements.contains(&serialized), + "monitored_filter_elements must carry the watched UTXO's serialized outpoint" + ); + } + + /// A funding-only wallet (no provider accounts) must not surface its UTXO + /// outpoints as filter elements. The collateral outpoint only helps a + /// masternode owner discover a `ProRegTx` referencing its collateral, so a + /// plain payment wallet skips that per-UTXO cost entirely. + #[tokio::test] + async fn test_funding_only_wallet_omits_collateral_outpoints() { + let (ctx, _tx) = TestWalletContext::new_random_with_options( + WalletAccountCreationOptions::BIP44AccountsOnly([0].into()), + ) + .with_mempool_funding(200_000) + .await; + + // The wallet is funded, so it owns a UTXO... + let serialized = dashcore::consensus::encode::serialize(&ctx.first_utxo().outpoint); + assert_eq!(serialized.len(), 36); + + // ...but with no provider accounts it emits no filter elements at all, + // in particular none of its UTXO outpoints. + let elements = ctx.managed_wallet.monitored_filter_elements(); + assert!( + elements.is_empty(), + "funding-only wallet must not emit collateral outpoint elements" + ); + } +} From d4692bb29c626942ddf51204d50d90206c1a5b0c Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Sun, 12 Jul 2026 15:04:25 +0700 Subject: [PATCH 038/149] feat(key-wallet-ffi): one-shot transaction decode to (address, amount) structs (#870) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet-ffi): one-shot transaction decode to (address, amount) structs Port the transaction-decode FFI from dashpay/platform#3981 (rs-platform-wallet-ffi/src/tx_decode.rs) into key-wallet-ffi, so wallet-less Core-chain decoding lives next to the library that owns the types. `transaction_decode` consensus-decodes raw tx bytes into `DecodedTransactionFFI` — per-input `(prev_txid, prev_vout, best-effort sender address)` and per-output `(address, value_duffs, scriptPubKey)` — in one struct-out call; `decoded_transaction_free` releases it. Adapted from the platform version to this crate's conventions: bool return with an `FFIError` out-parameter instead of a result struct, and `dash_network::ffi::FFINetwork`. Struct names are kept verbatim so the Swift wrapper migration stays a re-point. Closes #869 Co-Authored-By: Claude Fable 5 * chore(key-wallet-ffi): regenerate FFI docs and apply typos fix Pre-commit fixes: regenerate FFI_API.md for the new transaction_decode / decoded_transaction_free entries, and "unparseable" -> "unparsable" per the typos hook. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- key-wallet-ffi/Cargo.toml | 1 + key-wallet-ffi/FFI_API.md | 38 ++- key-wallet-ffi/src/lib.rs | 1 + key-wallet-ffi/src/tx_decode.rs | 562 ++++++++++++++++++++++++++++++++ 4 files changed, 600 insertions(+), 2 deletions(-) create mode 100644 key-wallet-ffi/src/tx_decode.rs diff --git a/key-wallet-ffi/Cargo.toml b/key-wallet-ffi/Cargo.toml index 203ec8f65..25c50843d 100644 --- a/key-wallet-ffi/Cargo.toml +++ b/key-wallet-ffi/Cargo.toml @@ -43,6 +43,7 @@ hex = "0.4" cbindgen = "0.29" [dev-dependencies] +dashcore = { path = "../dash", features = ["test-utils"] } key-wallet = { path = "../key-wallet", features = ["test-utils"] } key-wallet-manager = { path = "../key-wallet-manager", features = ["test-utils"] } hex = "0.4" diff --git a/key-wallet-ffi/FFI_API.md b/key-wallet-ffi/FFI_API.md index 4cd96c44a..8e8528272 100644 --- a/key-wallet-ffi/FFI_API.md +++ b/key-wallet-ffi/FFI_API.md @@ -4,7 +4,7 @@ This document provides a comprehensive reference for all FFI (Foreign Function I **Auto-generated**: This documentation is automatically generated from the source code. Do not edit manually. -**Total Functions**: 258 +**Total Functions**: 260 ## Table of Contents @@ -272,16 +272,18 @@ Functions: 10 ### Transaction Management -Functions: 14 +Functions: 16 | Function | Description | Module | |----------|-------------|--------| +| `decoded_transaction_free` | Release a `DecodedTransactionFFI` returned by `transaction_decode` | tx_decode | | `transaction_add_input` | Add an input to a transaction # Safety - `tx` must be a valid pointer to an... | transaction | | `transaction_add_output` | Add an output to a transaction # Safety - `tx` must be a valid pointer to... | transaction | | `transaction_bytes_free` | Free transaction bytes # Safety - `tx_bytes` must be a valid pointer... | transaction | | `transaction_check_result_free` | Free a transaction check result # Safety - `result` must be a valid... | transaction_checking | | `transaction_classify` | Get the transaction classification for routing Returns a string describing... | transaction_checking | | `transaction_create` | Create a new empty transaction # Returns - Pointer to FFITransaction on... | transaction | +| `transaction_decode` | Consensus-decode `tx_bytes` into a `DecodedTransactionFFI` | tx_decode | | `transaction_deserialize` | Deserialize a transaction # Safety - `data` must be a valid pointer to... | transaction | | `transaction_destroy` | Destroy a transaction # Safety - `tx` must be a valid pointer to an... | transaction | | `transaction_get_txid` | Get the transaction ID # Safety - `tx` must be a valid pointer to an... | transaction | @@ -3557,6 +3559,22 @@ Validate an address # Safety - `address` must be a valid null-terminated C str ### Transaction Management - Detailed +#### `decoded_transaction_free` + +```c +decoded_transaction_free(decoded: *mut DecodedTransactionFFI) -> () +``` + +**Description:** +Release a `DecodedTransactionFFI` returned by `transaction_decode`. Safe to call with null. # Safety `decoded` must be null or a pointer previously returned by `transaction_decode` that has not been freed yet. + +**Safety:** +`decoded` must be null or a pointer previously returned by `transaction_decode` that has not been freed yet. + +**Module:** `tx_decode` + +--- + #### `transaction_add_input` ```c @@ -3650,6 +3668,22 @@ Create a new empty transaction # Returns - Pointer to FFITransaction on success --- +#### `transaction_decode` + +```c +transaction_decode(tx_bytes: *const u8, tx_bytes_len: usize, network: FFINetwork, out_decoded: *mut *mut DecodedTransactionFFI, error: *mut FFIError,) -> bool +``` + +**Description:** +Consensus-decode `tx_bytes` into a `DecodedTransactionFFI`. Rejects trailing bytes after a valid transaction (uses [`deserialize`], not a streaming decode), so a valid prefix followed by garbage is an error rather than a silent trim. # Safety - `tx_bytes` must point to `tx_bytes_len` readable bytes. - `out_decoded` must be a valid pointer; it is nulled on entry and, on success, receives an owned `*mut DecodedTransactionFFI` that must be released with `decoded_transaction_free`. - `error` must be a valid pointer to an FFIError. + +**Safety:** +- `tx_bytes` must point to `tx_bytes_len` readable bytes. - `out_decoded` must be a valid pointer; it is nulled on entry and, on success, receives an owned `*mut DecodedTransactionFFI` that must be released with `decoded_transaction_free`. - `error` must be a valid pointer to an FFIError. + +**Module:** `tx_decode` + +--- + #### `transaction_deserialize` ```c diff --git a/key-wallet-ffi/src/lib.rs b/key-wallet-ffi/src/lib.rs index ec406b33d..dbe36f76b 100644 --- a/key-wallet-ffi/src/lib.rs +++ b/key-wallet-ffi/src/lib.rs @@ -18,6 +18,7 @@ pub mod managed_wallet; pub mod mnemonic; pub mod transaction; pub mod transaction_checking; +pub mod tx_decode; pub mod types; pub mod utils; pub mod utxo; diff --git a/key-wallet-ffi/src/tx_decode.rs b/key-wallet-ffi/src/tx_decode.rs new file mode 100644 index 000000000..2ff7c265a --- /dev/null +++ b/key-wallet-ffi/src/tx_decode.rs @@ -0,0 +1,562 @@ +//! C-ABI transaction decoding — consensus-decode raw Dash transaction bytes +//! into per-input/per-output structures with derived addresses, in one call. +//! +//! `transaction_deserialize` returns an opaque [`crate::transaction::FFITransaction`] +//! handle with no read accessors, so an FFI consumer that needs to match +//! arbitrary transactions against external addresses (e.g. the CrowdNode +//! on-chain API: "did a tx pay amount X to address Y?") needs per-output +//! `(address, amount)` reconstructed from the raw bytes. This entry point +//! decodes straight into [`Transaction`] and marshals that into the C +//! structures below — no intermediate representation. +//! +//! - Output `address` comes from [`Address::from_script`]; null for +//! non-standard scripts (OP_RETURN, bare multisig, …). +//! - Input `address` is recovered only for P2PKH-shaped scriptSigs and is +//! **unauthenticated** — see [`DecodedTxInputFFI::address`]. +//! - `txid` / `prev_txid` are in consensus (internal) byte order — reverse +//! for explorer-style display. + +use std::ffi::CString; +use std::os::raw::c_char; + +use crate::error::{FFIError, FFIErrorCode}; +use crate::{check_ptr, unwrap_or_return}; +use dash_network::ffi::FFINetwork; +use dashcore::blockdata::script::Instruction; +use dashcore::consensus::deserialize; +use dashcore::hashes::Hash; +use dashcore::{Address, Network, PublicKey, ScriptBuf, Transaction}; + +/// One decoded transaction input. +#[repr(C)] +pub struct DecodedTxInputFFI { + /// Previous output's txid in consensus (internal) byte order. + pub prev_txid: [u8; 32], + /// Previous output's index. + pub prev_vout: u32, + /// Sender address recovered from a P2PKH-shaped scriptSig (exactly two + /// pushes: DER signature, then public key), or null when the input is + /// coinbase / non-P2PKH / unparsable. **Unauthenticated**: the value is + /// derived from a script push the spender fully controls, with no + /// signature verification against the spent UTXO — use it for display or + /// as a matching hint only, never for authentication or authorization. + /// The attacker-resistant matching primitive is the per-output + /// `(address, value_duffs)` pair. Owned by the parent structure — freed + /// by `decoded_transaction_free`. + pub address: *mut c_char, +} + +/// One decoded transaction output. +#[repr(C)] +pub struct DecodedTxOutputFFI { + /// Destination address for standard scripts (P2PKH / P2SH), or null for + /// non-standard scripts. Owned by the parent structure. + pub address: *mut c_char, + /// Output value in duffs. + pub value_duffs: u64, + /// Raw scriptPubKey bytes (owned by the parent structure). + pub script_pubkey: *mut u8, + pub script_pubkey_len: usize, +} + +/// A consensus-decoded transaction. Allocated by `transaction_decode`; +/// release with `decoded_transaction_free`. +#[repr(C)] +pub struct DecodedTransactionFFI { + /// Transaction id in consensus (internal) byte order. + pub txid: [u8; 32], + pub inputs: *mut DecodedTxInputFFI, + pub inputs_count: usize, + pub outputs: *mut DecodedTxOutputFFI, + pub outputs_count: usize, +} + +/// Boxed-slice allocator for arrays owned by `DecodedTransactionFFI`. +fn vec_to_ptr(v: Vec) -> *mut T { + if v.is_empty() { + std::ptr::null_mut() + } else { + Box::into_raw(v.into_boxed_slice()) as *mut T + } +} + +/// Render an optional address to an owned C string, or null. +fn addr_to_cstr(address: Option
) -> *mut c_char { + address + .and_then(|a| CString::new(a.to_string()).ok()) + .map(CString::into_raw) + .unwrap_or(std::ptr::null_mut()) +} + +/// Best-effort P2PKH sender address from a scriptSig. Requires the exact +/// P2PKH spend shape — two data pushes and nothing else: a DER-shaped ECDSA +/// signature (`0x30` tag, ≤ 73 bytes including the sighash byte) followed by +/// a 33/65-byte public key — so a P2SH redeem script or any other +/// last-push-happens-to-parse collision returns `None` rather than an +/// unrelated address. Nothing here verifies the signature against the spent +/// UTXO, so even a `Some` result is only the *claimed* sender — see +/// [`DecodedTxInputFFI::address`]. +fn input_address_from_script_sig(script_sig: &ScriptBuf, network: Network) -> Option
{ + let mut sig: Option<&[u8]> = None; + let mut pubkey_bytes: Option<&[u8]> = None; + for instruction in script_sig.instructions() { + match instruction { + Ok(Instruction::PushBytes(bytes)) => match (sig, pubkey_bytes) { + (None, None) => sig = Some(bytes.as_bytes()), + (Some(_), None) => pubkey_bytes = Some(bytes.as_bytes()), + _ => return None, // more than two pushes + }, + _ => return None, // non-push opcode or unparsable script + } + } + let (sig, pubkey_bytes) = (sig?, pubkey_bytes?); + if sig.is_empty() || sig[0] != 0x30 || sig.len() > 73 { + return None; + } + if pubkey_bytes.len() != 33 && pubkey_bytes.len() != 65 { + return None; + } + let pubkey = PublicKey::from_slice(pubkey_bytes).ok()?; + Some(Address::p2pkh(&pubkey, network)) +} + +/// Consensus-decode `tx_bytes` into a `DecodedTransactionFFI`. Rejects +/// trailing bytes after a valid transaction (uses [`deserialize`], not a +/// streaming decode), so a valid prefix followed by garbage is an error +/// rather than a silent trim. +/// +/// # Safety +/// - `tx_bytes` must point to `tx_bytes_len` readable bytes. +/// - `out_decoded` must be a valid pointer; it is nulled on entry and, on +/// success, receives an owned `*mut DecodedTransactionFFI` that must be +/// released with `decoded_transaction_free`. +/// - `error` must be a valid pointer to an FFIError. +#[no_mangle] +pub unsafe extern "C" fn transaction_decode( + tx_bytes: *const u8, + tx_bytes_len: usize, + network: FFINetwork, + out_decoded: *mut *mut DecodedTransactionFFI, + error: *mut FFIError, +) -> bool { + check_ptr!(out_decoded, error); + *out_decoded = std::ptr::null_mut(); + check_ptr!(tx_bytes, error); + if tx_bytes_len == 0 { + (*error).set(FFIErrorCode::InvalidInput, "tx_bytes_len is zero"); + return false; + } + + let network: Network = network.into(); + let slice = std::slice::from_raw_parts(tx_bytes, tx_bytes_len); + let tx: Transaction = unwrap_or_return!(deserialize(slice), error); + let txid = tx.txid(); // before the input/output vectors are consumed + + let inputs: Vec = tx + .input + .into_iter() + .map(|txin| { + let address = if txin.previous_output.is_null() { + None // coinbase + } else { + input_address_from_script_sig(&txin.script_sig, network) + }; + DecodedTxInputFFI { + prev_txid: txin.previous_output.txid.to_byte_array(), + prev_vout: txin.previous_output.vout, + address: addr_to_cstr(address), + } + }) + .collect(); + + let outputs: Vec = tx + .output + .into_iter() + .map(|txout| { + let address = Address::from_script(&txout.script_pubkey, network).ok(); + let script = txout.script_pubkey.into_bytes(); + let script_len = script.len(); + DecodedTxOutputFFI { + address: addr_to_cstr(address), + value_duffs: txout.value, + script_pubkey: vec_to_ptr(script), + script_pubkey_len: script_len, + } + }) + .collect(); + + let ffi = DecodedTransactionFFI { + txid: txid.to_byte_array(), + inputs_count: inputs.len(), + inputs: vec_to_ptr(inputs), + outputs_count: outputs.len(), + outputs: vec_to_ptr(outputs), + }; + + *out_decoded = Box::into_raw(Box::new(ffi)); + (*error).clean(); + true +} + +/// Release a `DecodedTransactionFFI` returned by `transaction_decode`. +/// Safe to call with null. +/// +/// # Safety +/// `decoded` must be null or a pointer previously returned by +/// `transaction_decode` that has not been freed yet. +#[no_mangle] +pub unsafe extern "C" fn decoded_transaction_free(decoded: *mut DecodedTransactionFFI) { + if decoded.is_null() { + return; + } + let decoded = Box::from_raw(decoded); + + if !decoded.inputs.is_null() { + let inputs = + Vec::from_raw_parts(decoded.inputs, decoded.inputs_count, decoded.inputs_count); + for input in &inputs { + if !input.address.is_null() { + drop(CString::from_raw(input.address)); + } + } + drop(inputs); + } + + if !decoded.outputs.is_null() { + let outputs = + Vec::from_raw_parts(decoded.outputs, decoded.outputs_count, decoded.outputs_count); + for output in &outputs { + if !output.address.is_null() { + drop(CString::from_raw(output.address)); + } + if !output.script_pubkey.is_null() { + drop(Vec::from_raw_parts( + output.script_pubkey, + output.script_pubkey_len, + output.script_pubkey_len, + )); + } + } + drop(outputs); + } +} + +#[cfg(test)] +mod tests { + //! Everything funnels through the public FFI entry point: fixture → + //! serialize → `transaction_decode` → plain-Rust copy → assert. + //! `decode_ok` owns the pointer walking and frees exactly once. + //! Fixtures use `dashcore::test_utils` (`Address::dummy`, + //! `Transaction::dummy`, `Transaction::dummy_coinbase`) where they fit; + //! test_utils has no builder for a P2PKH-shaped scriptSig + //! (signature + pubkey pushes), so that fixture stays hand-built. + use super::*; + use dashcore::consensus::serialize; + use dashcore::secp256k1::{Secp256k1, SecretKey}; + use dashcore::{OutPoint, TxIn, TxOut, Txid, Witness}; + use std::ffi::CStr; + + /// FFI result copied into plain Rust so assertions read naturally. + struct Decoded { + txid: [u8; 32], + /// `(prev_txid, prev_vout, sender address)` + inputs: Vec<([u8; 32], u32, Option)>, + /// `(address, value_duffs, script_pubkey)` + outputs: Vec<(Option, u64, Vec)>, + } + + /// Copy an optional C string into owned Rust. + unsafe fn cstr_opt(ptr: *mut c_char) -> Option { + if ptr.is_null() { + None + } else { + Some(CStr::from_ptr(ptr).to_str().unwrap().to_owned()) + } + } + + /// Decode `tx` through the FFI boundary, copy the result out, and free + /// the FFI allocation exactly once. + fn decode_ok(tx: &Transaction, network: FFINetwork) -> Decoded { + let bytes = serialize(tx); + unsafe { + let mut error = FFIError::default(); + let mut out: *mut DecodedTransactionFFI = std::ptr::null_mut(); + let ok = transaction_decode(bytes.as_ptr(), bytes.len(), network, &mut out, &mut error); + assert!(ok); + assert_eq!(error.code, FFIErrorCode::Success); + assert!(!out.is_null()); + + let inputs = if (*out).inputs.is_null() { + Vec::new() + } else { + std::slice::from_raw_parts((*out).inputs, (*out).inputs_count) + .iter() + .map(|i| (i.prev_txid, i.prev_vout, cstr_opt(i.address))) + .collect() + }; + let outputs = if (*out).outputs.is_null() { + Vec::new() + } else { + std::slice::from_raw_parts((*out).outputs, (*out).outputs_count) + .iter() + .map(|o| { + let script = if o.script_pubkey.is_null() { + Vec::new() + } else { + std::slice::from_raw_parts(o.script_pubkey, o.script_pubkey_len) + .to_vec() + }; + (cstr_opt(o.address), o.value_duffs, script) + }) + .collect() + }; + let copied = Decoded { + txid: (*out).txid, + inputs, + outputs, + }; + decoded_transaction_free(out); + copied + } + } + + fn test_pubkey() -> PublicKey { + let secp = Secp256k1::new(); + let sk = SecretKey::from_slice(&[0x42u8; 32]).expect("valid secret key"); + PublicKey::new(sk.public_key(&secp)) + } + + /// One P2PKH-shaped input (spending 11..11:3), one P2PKH output of + /// 151072 duffs, one OP_RETURN output. The 151072 amount mirrors a + /// CrowdNode signUp signal so the fixture doubles as documentation. + fn p2pkh_spend_tx(network: Network) -> (Transaction, Address) { + let pubkey = test_pubkey(); + let addr = Address::p2pkh(&pubkey, network); + let script_sig = dashcore::blockdata::script::Builder::new() + .push_slice([0x30u8; 71]) // DER-shaped: 0x30 tag, ≤ 73 bytes + .push_key(&pubkey) + .into_script(); + let tx = Transaction { + version: 1, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint { + txid: Txid::from_byte_array([0x11u8; 32]), + vout: 3, + }, + script_sig, + sequence: 0xffffffff, + witness: Witness::default(), + }], + output: vec![ + TxOut { + value: 151_072, + script_pubkey: addr.script_pubkey(), + }, + TxOut { + value: 0, + script_pubkey: ScriptBuf::new_op_return(&[0xAAu8; 4]), + }, + ], + special_transaction_payload: None, + }; + (tx, addr) + } + + /// A one-input `Transaction::dummy` with its scriptSig replaced. + fn tx_with_script_sig(script_sig: ScriptBuf) -> Transaction { + let addr = Address::dummy(Network::Testnet, 1); + let mut tx = Transaction::dummy(&addr, 1..2, &[9_000]); + tx.input[0].script_sig = script_sig; + tx + } + + #[test] + fn decodes_outputs_with_addresses_and_values() { + let addr = Address::dummy(Network::Testnet, 7); + let tx = Transaction::dummy(&addr, 1..2, &[151_072, 20_002]); + let decoded = decode_ok(&tx, FFINetwork::Testnet); + + assert_eq!(decoded.txid, tx.txid().to_byte_array()); + assert_eq!(decoded.outputs.len(), 2); + assert_eq!(decoded.outputs[0].0, Some(addr.to_string())); + assert_eq!(decoded.outputs[0].1, 151_072); + assert_eq!(decoded.outputs[0].2, addr.script_pubkey().into_bytes()); + assert_eq!(decoded.outputs[1].0, Some(addr.to_string())); + assert_eq!(decoded.outputs[1].1, 20_002); + assert!(addr.to_string().starts_with('y'), "testnet P2PKH starts with 'y'"); + } + + #[test] + fn op_return_output_has_no_address() { + let (tx, _) = p2pkh_spend_tx(Network::Testnet); + let decoded = decode_ok(&tx, FFINetwork::Testnet); + assert!(decoded.outputs[1].0.is_none()); + assert!(!decoded.outputs[1].2.is_empty(), "script bytes still present"); + } + + #[test] + fn recovers_p2pkh_input_address_from_script_sig() { + let (tx, addr) = p2pkh_spend_tx(Network::Testnet); + let decoded = decode_ok(&tx, FFINetwork::Testnet); + + assert_eq!(decoded.inputs.len(), 1); + assert_eq!(decoded.inputs[0].0, [0x11u8; 32]); + assert_eq!(decoded.inputs[0].1, 3); + assert_eq!(decoded.inputs[0].2, Some(addr.to_string())); + } + + #[test] + fn network_changes_rendered_addresses() { + let addr = Address::dummy(Network::Testnet, 7); + let tx = Transaction::dummy(&addr, 1..2, &[151_072]); + let decoded = decode_ok(&tx, FFINetwork::Mainnet); + let rendered = decoded.outputs[0].0.clone().unwrap(); + assert_ne!(rendered, addr.to_string()); + assert!(rendered.starts_with('X'), "mainnet P2PKH starts with 'X'"); + } + + #[test] + fn coinbase_input_has_no_address() { + let addr = Address::dummy(Network::Testnet, 3); + let tx = Transaction::dummy_coinbase(&addr, 50_000); + let decoded = decode_ok(&tx, FFINetwork::Testnet); + assert!(decoded.inputs[0].2.is_none()); + } + + #[test] + fn non_p2pkh_script_sig_yields_no_input_address() { + // Transaction::dummy fills script_sig with a *lock* script + // (OP_DUP OP_HASH160 <20 B> OP_EQUALVERIFY OP_CHECKSIG): opcodes + // present, last push 20 bytes — must not produce an address. + let addr = Address::dummy(Network::Testnet, 9); + let tx = Transaction::dummy(&addr, 1..2, &[1_000]); + let decoded = decode_ok(&tx, FFINetwork::Testnet); + assert!(decoded.inputs[0].2.is_none()); + } + + #[test] + fn redeem_script_collision_yields_no_input_address() { + // Three pushes ending in a valid 33-byte pubkey — the P2SH + // redeem-script shape the exactly-two-pushes rule exists to reject. + let script_sig = dashcore::blockdata::script::Builder::new() + .push_slice([0x30u8; 71]) + .push_slice([0x01u8; 20]) + .push_key(&test_pubkey()) + .into_script(); + let decoded = decode_ok(&tx_with_script_sig(script_sig), FFINetwork::Testnet); + assert!(decoded.inputs[0].2.is_none()); + } + + #[test] + fn non_signature_first_push_yields_no_input_address() { + // Two pushes, but the first is not DER-shaped (no 0x30 tag). + let script_sig = dashcore::blockdata::script::Builder::new() + .push_slice([0xAAu8; 10]) + .push_key(&test_pubkey()) + .into_script(); + let decoded = decode_ok(&tx_with_script_sig(script_sig), FFINetwork::Testnet); + assert!(decoded.inputs[0].2.is_none()); + } + + #[test] + fn trailing_garbage_is_a_deserialization_error() { + let (tx, _) = p2pkh_spend_tx(Network::Testnet); + let mut bytes = serialize(&tx); + bytes.extend_from_slice(&[0xDE, 0xAD]); + unsafe { + let mut error = FFIError::default(); + // Pre-poison the out param to pin that failures null it. + let mut out: *mut DecodedTransactionFFI = std::ptr::NonNull::dangling().as_ptr(); + let ok = transaction_decode( + bytes.as_ptr(), + bytes.len(), + FFINetwork::Testnet, + &mut out, + &mut error, + ); + assert!(!ok); + assert_eq!(error.code, FFIErrorCode::InvalidInput); + assert!(out.is_null(), "out param is nulled on failure"); + } + } + + #[test] + fn garbage_bytes_are_a_deserialization_error() { + let bytes = [0xFFu8; 16]; + unsafe { + let mut error = FFIError::default(); + let mut out: *mut DecodedTransactionFFI = std::ptr::NonNull::dangling().as_ptr(); + let ok = transaction_decode( + bytes.as_ptr(), + bytes.len(), + FFINetwork::Testnet, + &mut out, + &mut error, + ); + assert!(!ok); + assert_eq!(error.code, FFIErrorCode::InvalidInput); + assert!(out.is_null(), "out param is nulled on failure"); + } + } + + #[test] + fn marshals_decoded_transaction_across_the_boundary() { + // Raw pointer walk (no helper) — pins the C-side memory layout. + let (tx, _) = p2pkh_spend_tx(Network::Testnet); + let bytes = serialize(&tx); + unsafe { + let mut error = FFIError::default(); + let mut out: *mut DecodedTransactionFFI = std::ptr::null_mut(); + let ok = transaction_decode( + bytes.as_ptr(), + bytes.len(), + FFINetwork::Testnet, + &mut out, + &mut error, + ); + assert!(ok); + assert!(!out.is_null()); + + let inputs = std::slice::from_raw_parts((*out).inputs, (*out).inputs_count); + let outputs = std::slice::from_raw_parts((*out).outputs, (*out).outputs_count); + assert_eq!((*out).inputs_count, 1); + assert_eq!((*out).outputs_count, 2); + assert_eq!(inputs[0].prev_vout, 3); + assert!(!inputs[0].address.is_null()); + assert_eq!(outputs[0].value_duffs, 151_072); + let addr = CStr::from_ptr(outputs[0].address).to_str().unwrap(); + assert!(addr.starts_with('y')); + assert!(outputs[0].script_pubkey_len > 0); + + decoded_transaction_free(out); + } + } + + #[test] + fn rejects_null_and_empty_input() { + unsafe { + let mut error = FFIError::default(); + // Pre-poison the out param to pin that failures null it. + let mut out: *mut DecodedTransactionFFI = std::ptr::NonNull::dangling().as_ptr(); + let ok = + transaction_decode(std::ptr::null(), 4, FFINetwork::Testnet, &mut out, &mut error); + assert!(!ok); + assert_eq!(error.code, FFIErrorCode::InvalidInput); + assert!(out.is_null(), "out param is nulled on failure"); + + let bytes = [0u8; 4]; + out = std::ptr::NonNull::dangling().as_ptr(); + let ok = + transaction_decode(bytes.as_ptr(), 0, FFINetwork::Testnet, &mut out, &mut error); + assert!(!ok); + assert_eq!(error.code, FFIErrorCode::InvalidInput); + assert!(out.is_null(), "out param is nulled on failure"); + } + } + + #[test] + fn free_null_is_safe() { + unsafe { decoded_transaction_free(std::ptr::null_mut()) } + } +} From 234e3c4333f3be6f7a1d01e5af90805ae49783c7 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Mon, 13 Jul 2026 00:31:37 +0700 Subject: [PATCH 039/149] fix(key-wallet): return the fee the tx actually pays from build/build_signed (#872) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(key-wallet): return the fee the tx actually pays from build/build_signed TransactionBuilder::build_unsigned()/build_signed() returned a fee computed from the encoded size of the built transaction. When the builder drops a dust change remainder (0 < change <= 546 duffs), those duffs go to miners, so the real fee is the size fee plus the dust — the returned value under-reported by up to 546 duffs exactly when the fee got larger. Compute the returned fee as sum(selected input values) - sum(output values) instead, which is what the transaction actually pays in every case (normal change, exact change, dust drop, and drain). Remove the now-unused encoded_size helper and its error paths, and update the doc comments. Fixes #871 Co-Authored-By: Claude Fable 5 * test(key-wallet): pin asset-lock fee semantics — locked credits are not fee An asset lock burns the locked amount into an on-chain OP_RETURN output mirroring the payload's credit outputs, so it is part of the output sum and inputs - outputs yields the miner fee only. Add a regression test proving the returned fee excludes the locked credits. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- .../transaction_builder.rs | 112 ++++++++++++------ 1 file changed, 77 insertions(+), 35 deletions(-) diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 2ed56efb3..b2afbefe7 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -12,7 +12,6 @@ use core::fmt; use dashcore::blockdata::script::{Builder, PushBytes, ScriptBuf}; use dashcore::blockdata::transaction::special_transaction::TransactionPayload; use dashcore::blockdata::transaction::{OutPoint, Transaction}; -use dashcore::consensus::Encodable; use dashcore::sighash::{EcdsaSighashType, LegacySighash, SighashCache}; use dashcore::Address; use dashcore::{TxIn, TxOut}; @@ -25,15 +24,6 @@ use std::cmp::Ordering; /// bytes/signed input) and be rejected by the network const MAX_STANDARD_TX_INPUTS: usize = 500; -/// Consensus-encoded byte length of `tx`, used to compute the fee from the real -/// serialized size. Surfaces an encode error instead of panicking so the caller -/// can release any reservations it took rather than stranding the inputs. -fn encoded_size(tx: &Transaction) -> Result { - let mut bytes = Vec::new(); - tx.consensus_encode(&mut bytes) - .map_err(|err| BuilderError::InvalidData(format!("failed to encode transaction: {}", err))) -} - /// Calculate varint size for a given number fn varint_size(n: usize) -> usize { match n { @@ -432,28 +422,25 @@ impl TransactionBuilder { } } + /// Build the unsigned transaction. The returned fee is the fee the + /// transaction actually pays: Σ(selected input values) − Σ(output values). + /// This can exceed the size-based fee target when a dust change remainder + /// (≤ 546 duffs) is dropped and left to miners. pub fn build_unsigned(self) -> Result<(Transaction, u64), BuilderError> { - let fee_rate = self.fee_rate; - let reservations = self.reservations.clone(); - let (tx, inputs) = self.assemble_unsigned()?; - let fee = match encoded_size(&tx) { - Ok(size) => fee_rate.calculate_fee(size), - Err(err) => { - if let Some(reservations) = &reservations { - reservations.release(inputs.iter().map(|utxo| &utxo.outpoint)); - } - return Err(err); - } - }; + let total_input: u64 = inputs.iter().map(|utxo| utxo.value()).sum(); + let total_output: u64 = tx.output.iter().map(|out| out.value).sum(); - Ok((tx, fee)) + Ok((tx, total_input.saturating_sub(total_output))) } /// Build and sign the transaction. The `path_resolver` maps each input /// address to the derivation path the signer should use for that input. - /// The returned fee is computed from the encoded size of the signed tx. + /// The returned fee is the fee the transaction actually pays: + /// Σ(selected input values) − Σ(output values). This can exceed the + /// size-based fee target when a dust change remainder (≤ 546 duffs) is + /// dropped and left to miners. pub async fn build_signed( self, signer: &S, @@ -463,10 +450,10 @@ impl TransactionBuilder { S: TransactionSigner + ?Sized + Sync, P: Fn(Address) -> Option + Send, { - let fee_rate = self.fee_rate; let reservations = self.reservations.clone(); let (tx, inputs) = self.assemble_unsigned()?; + let total_input: u64 = inputs.iter().map(|utxo| utxo.value()).sum(); // Signing never reaches the network for a local key, but an external // signer can fail. A failed sign means the reserved inputs are still // spendable, so release them now instead of stranding the funds until @@ -482,17 +469,9 @@ impl TransactionBuilder { } }; - let fee = match encoded_size(&tx) { - Ok(size) => fee_rate.calculate_fee(size), - Err(err) => { - if let Some(reservations) = &reservations { - reservations.release(reserved.iter()); - } - return Err(err); - } - }; + let total_output: u64 = tx.output.iter().map(|out| out.value).sum(); - Ok((tx, fee)) + Ok((tx, total_input.saturating_sub(total_output))) } } @@ -852,6 +831,69 @@ mod tests { assert_eq!(tx.output[0].value, 150000); } + /// When the change remainder is dust (≤ 546 duffs) the builder drops it and + /// those duffs go to miners. The returned fee must be what the transaction + /// actually pays (Σ inputs − Σ outputs), not the smaller size-based target. + #[test] + fn test_dropped_dust_change_counts_toward_returned_fee() { + // 150000 to recipient + 226 size fee + 300 dust remainder + let utxos = vec![Utxo::dummy(0, 150526, 100, false, true)]; + + let recipient_address = Address::dummy(Network::Testnet, 0); + let change_address = Address::dummy(Network::Testnet, 0); + + let (tx, fee) = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::SmallestFirst) + .set_fee_rate(FeeRate::normal()) + .set_change_address(change_address) + .add_inputs(utxos) + .add_output(&recipient_address, 150000) + .build_unsigned() + .unwrap(); + + assert_eq!(tx.output.len(), 1, "dust change must be dropped"); + let total_output: u64 = tx.output.iter().map(|o| o.value).sum(); + assert_eq!(fee, 150526 - total_output, "fee must equal inputs minus outputs"); + assert_eq!(fee, 526, "fee must include the 300-duff dropped dust remainder"); + } + + /// An asset lock burns the locked amount into an on-chain OP_RETURN output + /// mirroring the payload's credit outputs. That output is part of + /// Σ(outputs), so the returned fee must be the miner fee only — the locked + /// credits must never be counted as fee. + #[test] + fn test_asset_lock_fee_excludes_locked_credits() { + let utxos = vec![Utxo::dummy(0, 1_000_000, 100, false, true)]; + let change_address = Address::dummy(Network::Testnet, 0); + + let asset_lock_payload = AssetLockPayload { + version: 1, + credit_outputs: vec![TxOut { + value: 100_000, + script_pubkey: ScriptBuf::new(), + }], + }; + + let (tx, fee) = TransactionBuilder::new() + .set_current_height(200) + .set_fee_rate(FeeRate::normal()) + .set_change_address(change_address) + .set_special_payload(TransactionPayload::AssetLockPayloadType(asset_lock_payload)) + .add_inputs(utxos) + .build_unsigned() + .unwrap(); + + assert_eq!(tx.output.len(), 2, "OP_RETURN burn output + change"); + let total_output: u64 = tx.output.iter().map(|o| o.value).sum(); + assert_eq!(fee, 1_000_000 - total_output, "fee must equal inputs minus outputs"); + assert!( + fee < 1_000, + "fee must be the miner fee only, not include the 100k locked credits, got {}", + fee + ); + } + #[test] fn test_special_payload_size_calculations() { // Test that special payload sizes are calculated correctly From 93b7477c85b61d4824d7a7d0a574f219233694cc Mon Sep 17 00:00:00 2001 From: bfoss765 <38437574+bfoss765@users.noreply.github.com> Date: Sun, 12 Jul 2026 22:39:52 -0400 Subject: [PATCH 040/149] fix(key-wallet): cover CoinJoin/DashPay accounts for AssetLock routing (#867) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(key-wallet): cover CoinJoin/DashPay accounts for AssetLock routing `TransactionRouter::get_relevant_account_types(AssetLock)` returned only the BIP44/BIP32 standard accounts plus the identity/asset-lock accounts, omitting the CoinJoin and DashPay fund-bearing accounts. Only `check_core_transaction` marks a UTXO spent, and it scopes spend detection to the account types this router returns. So an asset lock funded from a CoinJoin or DashPay UTXO never had its input debited: the spent UTXO kept counting toward the balance indefinitely while the (BIP44) change output was still credited, inflating the reported balance by exactly the spent amount — on both relay and rescan. Field evidence (Android SDK wallet, dashpay/dash-wallet#1507): outpoint-diff reconciliation against the dashj oracle showed the SDK balance high by exactly the value of asset-lock-spent CoinJoin inputs that were never debited. Fix: the AssetLock arm now consults the full `fund_bearing_account_types()` set (BIP44, BIP32, CoinJoin, DashpayReceivingFunds, DashpayExternalAccount) before the identity/asset-lock accounts. Discovery is membership-based like Dash Core's `IsMine`, so consulting extra accounts never yields false positives. Adds an end-to-end regression test that funds a CoinJoin UTXO, spends it via an asset-lock transaction through `check_core_transaction`, and asserts the CoinJoin UTXO is debited and removed (fails on the old routing: total_sent 0). Also extends the AssetLock routing unit test. Refs: dashpay/platform#4073, dashpay/platform#4074, dashpay/dash-wallet#1507 Co-Authored-By: Claude Fable 5 * test(key-wallet): assert aggregate wallet balance in the asset-lock CoinJoin-debit test Addresses CodeRabbit's nitpick on dashpay/rust-dashcore#867. The regression the test guards against is balance inflation, but it previously asserted only the per-transaction totals and the CoinJoin UTXO removal. Add confirmed/spendable/ total balance assertions after the spend: a non-debited CoinJoin coin would report funding_value + change_value; a correct debit leaves only the confirmed BIP44 change (the asset-lock credit output is locked into Platform, never a spendable wallet UTXO). Existing assertions preserved. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- .../transaction_router/mod.rs | 32 +++- .../transaction_router/tests/routing.rs | 8 +- .../transaction_checking/wallet_checker.rs | 180 ++++++++++++++++++ 3 files changed, 209 insertions(+), 11 deletions(-) diff --git a/key-wallet/src/transaction_checking/transaction_router/mod.rs b/key-wallet/src/transaction_checking/transaction_router/mod.rs index b7d74b290..51f35acbe 100644 --- a/key-wallet/src/transaction_checking/transaction_router/mod.rs +++ b/key-wallet/src/transaction_checking/transaction_router/mod.rs @@ -134,16 +134,28 @@ impl TransactionRouter { AccountTypeToCheck::StandardBIP32, AccountTypeToCheck::CoinJoin, ], - TransactionType::AssetLock => vec![ - AccountTypeToCheck::StandardBIP44, - AccountTypeToCheck::StandardBIP32, - AccountTypeToCheck::IdentityRegistration, - AccountTypeToCheck::IdentityTopUp, - AccountTypeToCheck::IdentityTopUpNotBound, - AccountTypeToCheck::IdentityInvitation, - AccountTypeToCheck::AssetLockAddressTopUp, - AccountTypeToCheck::AssetLockShieldedAddressTopUp, - ], + TransactionType::AssetLock => { + // An asset lock can be funded from any fund-bearing account, and only + // `check_core_transaction` debits a spent UTXO — scoped to the accounts + // returned here. Omitting CoinJoin / DashPay meant asset locks funded from + // those accounts never had their inputs debited, so the spent UTXOs kept + // counting toward the balance indefinitely while the (BIP44) change output + // was still credited — the balance ended up high by exactly the + // CoinJoin/DashPay amount spent, both on relay and on rescan + // (dashpay/platform#4073, dashpay/platform#4074, dashpay/dash-wallet#1507). + // Discovery is membership-based like Dash Core's `IsMine`, so consulting the + // full fund-bearing set never yields false positives. + let mut accounts = Self::fund_bearing_account_types(); + accounts.extend([ + AccountTypeToCheck::IdentityRegistration, + AccountTypeToCheck::IdentityTopUp, + AccountTypeToCheck::IdentityTopUpNotBound, + AccountTypeToCheck::IdentityInvitation, + AccountTypeToCheck::AssetLockAddressTopUp, + AccountTypeToCheck::AssetLockShieldedAddressTopUp, + ]); + accounts + } TransactionType::AssetUnlock => { vec![AccountTypeToCheck::StandardBIP44, AccountTypeToCheck::StandardBIP32] } diff --git a/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs b/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs index 76568c45f..65d2d487b 100644 --- a/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs +++ b/key-wallet/src/transaction_checking/transaction_router/tests/routing.rs @@ -432,9 +432,15 @@ fn test_asset_lock_transaction_routing() { let tx_type = TransactionType::AssetLock; let accounts = TransactionRouter::get_relevant_account_types(&tx_type); - // Should check standard accounts and all identity accounts + // Should check all fund-bearing accounts (an asset lock can be funded from any of them) + // and all identity accounts. assert!(accounts.contains(&AccountTypeToCheck::StandardBIP44)); assert!(accounts.contains(&AccountTypeToCheck::StandardBIP32)); + // Fund-bearing accounts an asset lock can spend from. Their absence let asset-lock + // spends of CoinJoin / DashPay UTXOs go undebited (dashpay/platform#4073, #4074). + assert!(accounts.contains(&AccountTypeToCheck::CoinJoin)); + assert!(accounts.contains(&AccountTypeToCheck::DashpayReceivingFunds)); + assert!(accounts.contains(&AccountTypeToCheck::DashpayExternalAccount)); assert!(accounts.contains(&AccountTypeToCheck::IdentityRegistration)); assert!(accounts.contains(&AccountTypeToCheck::IdentityTopUp)); assert!(accounts.contains(&AccountTypeToCheck::IdentityTopUpNotBound)); diff --git a/key-wallet/src/transaction_checking/wallet_checker.rs b/key-wallet/src/transaction_checking/wallet_checker.rs index 65469b561..356ce45da 100644 --- a/key-wallet/src/transaction_checking/wallet_checker.rs +++ b/key-wallet/src/transaction_checking/wallet_checker.rs @@ -565,6 +565,186 @@ mod tests { assert_eq!(record.net_amount, -(funding_value as i64)); } + /// Regression: an asset-lock transaction that spends a CoinJoin UTXO must + /// debit that UTXO. + /// + /// `check_core_transaction` only marks a UTXO spent for the account types + /// returned by `TransactionRouter::get_relevant_account_types`. Before the + /// fix, the `AssetLock` arm omitted `CoinJoin` (and the DashPay accounts), + /// so an asset lock funded from a CoinJoin UTXO never had that input + /// debited: the spent UTXO kept counting toward the balance while the BIP44 + /// change output was still credited, inflating the balance by exactly the + /// spent amount — on both relay and rescan (dashpay/platform#4073, #4074, + /// dashpay/dash-wallet#1507). + #[tokio::test] + async fn test_asset_lock_spending_coinjoin_utxo_is_debited() { + use crate::account::AccountType; + use crate::managed_account::managed_account_type::ManagedAccountType; + use dashcore::blockdata::transaction::special_transaction::asset_lock::AssetLockPayload; + use dashcore::blockdata::transaction::special_transaction::TransactionPayload; + + let network = Network::Testnet; + + // Wallet with a BIP44 account (asset-lock change lands here) and a + // CoinJoin account (funds the asset lock). + let mut wallet = Wallet::new_random(network, WalletAccountCreationOptions::None) + .expect("Should create wallet"); + wallet + .add_account( + AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account, + }, + None, + ) + .expect("Should add BIP44 account"); + wallet + .add_account( + AccountType::CoinJoin { + index: 0, + }, + None, + ) + .expect("Should add CoinJoin account"); + + let mut managed_wallet = + ManagedWalletInfo::from_wallet_with_name(&wallet, "Test".to_string(), 0); + + // Derive a CoinJoin external address and a BIP44 change address. + let coinjoin_xpub = + wallet.accounts.coinjoin_accounts.get(&0).expect("coinjoin account").account_xpub; + let coinjoin_address = { + let managed_account = + managed_wallet.first_coinjoin_managed_account_mut().expect("managed coinjoin"); + if let ManagedAccountType::CoinJoin { + external_addresses, + .. + } = managed_account.managed_account_type_mut() + { + external_addresses + .next_unused(&KeySource::Public(coinjoin_xpub), true) + .expect("coinjoin address") + } else { + panic!("Expected CoinJoin account type"); + } + }; + let bip44_xpub = + wallet.accounts.standard_bip44_accounts.get(&0).expect("bip44 account").account_xpub; + let change_address = managed_wallet + .first_bip44_managed_account_mut() + .expect("managed bip44") + .next_change_address(Some(&bip44_xpub), true) + .expect("bip44 change address"); + + // Fund the CoinJoin address, creating a CoinJoin UTXO. + let funding_value = 100_000_000u64; + let funding_tx = Transaction::dummy(&coinjoin_address, 0..1, &[funding_value]); + let funding_context = TransactionContext::InChainLockedBlock(BlockInfo::new( + 1, + BlockHash::from_slice(&[7u8; 32]).expect("Should create block hash"), + 1_650_000_000, + )); + let funding_result = managed_wallet + .check_core_transaction(&funding_tx, funding_context, &mut wallet, true, true) + .await; + assert!(funding_result.is_relevant, "CoinJoin funding must be relevant"); + assert_eq!(funding_result.total_received, funding_value); + assert_eq!( + managed_wallet.first_coinjoin_managed_account().expect("coinjoin account").utxos.len(), + 1, + "CoinJoin funding must create a UTXO" + ); + + // Build an asset-lock tx spending the CoinJoin UTXO, with BIP44 change. + let credit_value = 40_000_000u64; + let change_value = funding_value - credit_value - 1_000; // small fee + let asset_lock_tx = Transaction { + version: 3, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint { + txid: funding_tx.txid(), + vout: 0, + }, + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: dashcore::Witness::new(), + }], + output: vec![TxOut { + value: change_value, + script_pubkey: change_address.script_pubkey(), + }], + special_transaction_payload: Some(TransactionPayload::AssetLockPayloadType( + AssetLockPayload { + version: 1, + credit_outputs: vec![TxOut { + value: credit_value, + script_pubkey: change_address.script_pubkey(), + }], + }, + )), + }; + assert_eq!( + TransactionRouter::classify_transaction(&asset_lock_tx), + TransactionType::AssetLock, + "tx must classify as AssetLock so it routes through the AssetLock arm" + ); + + let spend_context = TransactionContext::InChainLockedBlock(BlockInfo::new( + 2, + BlockHash::from_slice(&[8u8; 32]).expect("Should create block hash"), + 1_650_000_100, + )); + let spend_result = managed_wallet + .check_core_transaction(&asset_lock_tx, spend_context, &mut wallet, true, true) + .await; + + assert!(spend_result.is_relevant, "asset lock spending our UTXO must be relevant"); + // The load-bearing assertion: the CoinJoin input must be debited. + assert_eq!( + spend_result.total_sent, funding_value, + "asset lock must debit the CoinJoin UTXO it spends" + ); + assert_eq!( + spend_result.total_received, change_value, + "BIP44 change output must be credited" + ); + assert!( + managed_wallet + .first_coinjoin_managed_account() + .expect("coinjoin account") + .utxos + .is_empty(), + "spent CoinJoin UTXO must be removed" + ); + + // Aggregate wallet balance — the actual regression is balance inflation. + // `check_core_transaction` was called with `update_balance = true`, so + // the cached balance now reflects the spend. If the spent CoinJoin coin + // were NOT debited, the wallet would still count its `funding_value` + // (100_000_000) on top of the new BIP44 change, reporting + // `funding_value + change_value`. After a correct debit only the + // confirmed BIP44 change UTXO remains (the asset-lock credit output is + // locked into Platform, never counted as a spendable wallet UTXO — see + // the `total_received == change_value` assertion above). + assert_eq!( + managed_wallet.balance.confirmed(), + change_value, + "confirmed balance must be only the BIP44 change; the spent CoinJoin \ + coin must not inflate it" + ); + assert_eq!( + managed_wallet.balance.spendable(), + change_value, + "spendable balance must fall from funding_value to change_value after the spend" + ); + assert_eq!( + managed_wallet.balance.total(), + change_value, + "total balance must not double-count the spent CoinJoin UTXO" + ); + } + /// Test the full coinbase maturity flow - immature to mature transition #[tokio::test] async fn test_wallet_checker_immature_transaction_flow() { From b53617961e1e987cb6d267e11dff2aaade442c3a Mon Sep 17 00:00:00 2001 From: bfoss765 <38437574+bfoss765@users.noreply.github.com> Date: Sun, 12 Jul 2026 22:47:34 -0400 Subject: [PATCH 041/149] feat(key-wallet): widen DEFAULT_COINJOIN_GAP_LIMIT 30 -> 100 for dashj parity (#868) Proposal: align the CoinJoin address discovery window with dashj's `DeterministicKeyChain` lookahead (`DEFAULT_LOOKAHEAD_SIZE = 100`), the effective window dashj-core watches for CoinJoin keychains. This is window dimensioning, not a scan-algorithm fix. The gap limit only bounds the largest run of consecutive UNUSED CoinJoin addresses the wallet can bridge before discovery stalls; the progressive rescan-to-quiescence algorithm in dash-spv + key-wallet is already correct at gap 30 for any spray whose unused runs are <= 30. See PR description for the two-sided evidence and the reference to the ongoing team discussion. Refs: dashpay/platform#4074, dashpay/dash-wallet#1507 Co-authored-by: Claude Fable 5 --- key-wallet/src/gap_limit.rs | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/key-wallet/src/gap_limit.rs b/key-wallet/src/gap_limit.rs index 9d4d5b088..7eaefc99c 100644 --- a/key-wallet/src/gap_limit.rs +++ b/key-wallet/src/gap_limit.rs @@ -16,8 +16,31 @@ pub const DEFAULT_EXTERNAL_GAP_LIMIT: u32 = 30; /// Standard gap limit for internal (change) addresses pub const DEFAULT_INTERNAL_GAP_LIMIT: u32 = 30; -/// Standard gap limit for CoinJoin addresses -pub const DEFAULT_COINJOIN_GAP_LIMIT: u32 = 30; +/// Standard gap limit for CoinJoin addresses. +/// +/// Set to match dashj's `DeterministicKeyChain` lookahead +/// (`DEFAULT_LOOKAHEAD_SIZE = 100`), which is the effective discovery window +/// dashj-core (the reference wallet, `org.dashj:dashj-core:22.x`) watches for +/// CoinJoin keychains. Address discovery here is dynamic — the pool watches +/// `highest_used + gap_limit` addresses — so this bounds the largest run of +/// UNUSED CoinJoin addresses the wallet can bridge during a filter/rescan +/// before discovery stalls. +/// +/// The previous value of 30 starved heavy mixers: DIP-9 CoinJoin mixing +/// sprays denominations across many sequential addresses, and a run wider +/// than 30 consecutive unused addresses caused the SDK to stop discovering — +/// it then missed both the txs that CREATED far-index UTXOs and the txs that +/// SPENT nearer ones, leaving a persistent balance mismatch vs dashj that +/// survived a clean re-creation + full rescan (dashpay/platform#4074, +/// dashpay/dash-wallet#1507). 100 restores parity with dashj's window so the +/// SDK discovers every CoinJoin tx dashj does. +/// +/// Cost is modest: the CoinJoin account watches two pools (external + +/// internal), so this pre-derives and filter-matches ~200 CoinJoin addresses +/// per account instead of ~60. BIP158 filter matching is a set intersection, +/// so the extra scripts add negligible per-block work; the derivation is a +/// one-time keychain expansion. +pub const DEFAULT_COINJOIN_GAP_LIMIT: u32 = 100; /// Standard gap limit for special purpose keys (identity, provider keys) pub const DEFAULT_SPECIAL_GAP_LIMIT: u32 = 5; From 337f6ccc7f45a5f7a50abf0d6fecb1ef0b786e7b Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Mon, 13 Jul 2026 20:00:18 +0700 Subject: [PATCH 042/149] feat(key-wallet-ffi): typed special-transaction payload on FFITransactionRecord (expose ProRegTx service IP) (#876) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet-ffi): typed special-transaction payload on FFITransactionRecord Expose DIP-2/DIP-3 special payload data — most importantly a ProRegTx's masternode service IP:port — as a typed, FFI-safe struct instead of forcing every binding to consensus-decode the raw tx_data bytes. - New `special_payload` module: `FFISpecialTransactionPayload` tagged by the on-wire DIP-2 tx type, with typed structs for ProviderRegistration (service address string + raw ip/port, collateral outpoint, owner/ voting key hashes, operator BLS key, payout script, platform fields), ProviderUpdateService, ProviderUpdateRegistrar and ProviderUpdateRevocation. Other payload kinds surface the discriminant only, extensible later. - `FFITransactionRecord` gains an owned `special_transaction_payload` pointer, populated from the stored record's transaction and freed by the record's Drop impl. - `ManagedCoreKeysAccount::drop_finalized_transaction` (feature `keep-finalized-transactions` OFF) now retains provider-payload records on provider-key accounts, so a chainlocked-at-first-sight historical ProRegTx stays queryable. - New ungated `managed_core_account_get_special_transactions` FFI query (paired with the now-ungated `managed_core_account_free_transactions`) returns the payload-carrying records in every feature configuration. Closes #875 Co-Authored-By: Claude Fable 5 * docs(key-wallet-ffi): fix broken intra-doc link in special_payload module docs Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- key-wallet-ffi/FFI_API.md | 27 +- key-wallet-ffi/src/lib.rs | 1 + key-wallet-ffi/src/managed_account.rs | 258 ++++++++- key-wallet-ffi/src/special_payload.rs | 548 ++++++++++++++++++ .../managed_core_keys_account.rs | 56 +- .../keep_finalized_transactions_tests.rs | 197 +++++++ 6 files changed, 1068 insertions(+), 19 deletions(-) create mode 100644 key-wallet-ffi/src/special_payload.rs diff --git a/key-wallet-ffi/FFI_API.md b/key-wallet-ffi/FFI_API.md index 8e8528272..566d3aa19 100644 --- a/key-wallet-ffi/FFI_API.md +++ b/key-wallet-ffi/FFI_API.md @@ -4,7 +4,7 @@ This document provides a comprehensive reference for all FFI (Foreign Function I **Auto-generated**: This documentation is automatically generated from the source code. Do not edit manually. -**Total Functions**: 260 +**Total Functions**: 261 ## Table of Contents @@ -140,7 +140,7 @@ Functions: 63 ### Account Management -Functions: 108 +Functions: 109 | Function | Description | Module | |----------|-------------|--------| @@ -231,7 +231,7 @@ Functions: 108 | `managed_account_collection_summary_data` | Get structured account collection summary data for managed collection ... | managed_account_collection | | `managed_account_collection_summary_free` | Free a managed account collection summary and all its allocated memory #... | managed_account_collection | | `managed_core_account_free` | Free a managed account handle # Safety - `account` must be a valid pointer... | managed_account | -| `managed_core_account_free_transactions` | Free transactions array returned by managed_core_account_get_transactions ... | managed_account | +| `managed_core_account_free_transactions` | Free a transactions array returned by `managed_core_account_get_transactions`... | managed_account | | `managed_core_account_get_account_type` | Get the account type of a managed account # Safety - `account` must be a... | managed_account | | `managed_core_account_get_address_pool` | Get an address pool from a managed account by type This function returns... | managed_account | | `managed_core_account_get_balance` | Get the balance of a managed account | managed_account | @@ -239,6 +239,7 @@ Functions: 108 | `managed_core_account_get_index` | Get the account index from a managed account Returns the primary account... | managed_account | | `managed_core_account_get_internal_address_pool` | Get the internal address pool from a managed account This function returns... | managed_account | | `managed_core_account_get_network` | Get the network of a managed account # Safety - `account` must be a valid... | managed_account | +| `managed_core_account_get_special_transactions` | Get the transactions of a managed account that carry a DIP-2 special payload... | managed_account | | `managed_core_account_get_transaction_count` | Get the number of transactions in a managed account Only available with the... | managed_account | | `managed_core_account_get_transactions` | Get all transactions from a managed account Returns an array of... | managed_account | | `managed_core_account_get_utxo_count` | Get the number of UTXOs in a managed account | managed_account | @@ -3066,10 +3067,10 @@ managed_core_account_free_transactions(transactions: *mut FFITransactionRecord, ``` **Description:** -Free transactions array returned by managed_core_account_get_transactions Only available with the `keep-finalized-transactions` Cargo feature, in which configuration `managed_core_account_get_transactions` is also available — the two functions are paired. # Safety - `transactions` must be a pointer returned by `managed_core_account_get_transactions` - `count` must be the count returned by `managed_core_account_get_transactions` - This function must only be called once per allocation +Free a transactions array returned by `managed_core_account_get_transactions` or `managed_core_account_get_special_transactions`. # Safety - `transactions` must be a pointer returned by one of the paired getters - `count` must be the count returned alongside it - This function must only be called once per allocation **Safety:** -- `transactions` must be a pointer returned by `managed_core_account_get_transactions` - `count` must be the count returned by `managed_core_account_get_transactions` - This function must only be called once per allocation +- `transactions` must be a pointer returned by one of the paired getters - `count` must be the count returned alongside it - This function must only be called once per allocation **Module:** `managed_account` @@ -3187,6 +3188,22 @@ Get the network of a managed account # Safety - `account` must be a valid poin --- +#### `managed_core_account_get_special_transactions` + +```c +managed_core_account_get_special_transactions(account: *const FFIManagedCoreAccount, transactions_out: *mut *mut FFITransactionRecord, count_out: *mut usize,) -> bool +``` + +**Description:** +Get the transactions of a managed account that carry a DIP-2 special payload (provider registrations / updates, asset locks, …). Unlike `managed_core_account_get_transactions` this is available in every feature configuration: provider-relevant records are retained in memory even after their block is chainlocked (see `ManagedCoreKeysAccount::drop_finalized_transaction` in `key-wallet`), so a provider-key account's registration history — including each masternode's service IP — remains queryable without the `keep-finalized-transactions` feature. Returns records in txid order. Each record's `special_transaction_payload` field is non-null. # Safety - `account` must be a valid pointer to an FFIManagedCoreAccount instance - `transactions_out` must be a valid pointer to receive the transactions array pointer - `count_out` must be a valid pointer to receive the count - The caller must free the returned array using `managed_core_account_free_transactions` + +**Safety:** +- `account` must be a valid pointer to an FFIManagedCoreAccount instance - `transactions_out` must be a valid pointer to receive the transactions array pointer - `count_out` must be a valid pointer to receive the count - The caller must free the returned array using `managed_core_account_free_transactions` + +**Module:** `managed_account` + +--- + #### `managed_core_account_get_transaction_count` ```c diff --git a/key-wallet-ffi/src/lib.rs b/key-wallet-ffi/src/lib.rs index dbe36f76b..755d67f67 100644 --- a/key-wallet-ffi/src/lib.rs +++ b/key-wallet-ffi/src/lib.rs @@ -16,6 +16,7 @@ pub mod managed_account; pub mod managed_account_collection; pub mod managed_wallet; pub mod mnemonic; +pub mod special_payload; pub mod transaction; pub mod transaction_checking; pub mod tx_decode; diff --git a/key-wallet-ffi/src/managed_account.rs b/key-wallet-ffi/src/managed_account.rs index b835d4dd9..84f9a99c2 100644 --- a/key-wallet-ffi/src/managed_account.rs +++ b/key-wallet-ffi/src/managed_account.rs @@ -7,12 +7,12 @@ use dash_network::ffi::FFINetwork; use dashcore::hashes::Hash; use std::os::raw::{c_char, c_uint}; -#[cfg(feature = "keep-finalized-transactions")] use std::ptr::slice_from_raw_parts_mut; use std::sync::Arc; use crate::address_pool::{FFIAddressPool, FFIAddressPoolType}; use crate::error::{FFIError, FFIErrorCode}; +use crate::special_payload::FFISpecialTransactionPayload; use crate::types::{ FFIAccountKind, FFIInputDetail, FFIOutputDetail, FFITransactionContext, FFITransactionDirection, FFITransactionType, @@ -894,6 +894,9 @@ pub struct FFITransactionRecord { pub tx_len: usize, /// Optional label (null if not set) pub label: *mut c_char, + /// Typed DIP-2 special-transaction payload (null for classical + /// transactions). Owned by this record and freed with it. + pub special_transaction_payload: *mut FFISpecialTransactionPayload, } impl From<&TransactionRecord> for FFITransactionRecord { @@ -943,6 +946,12 @@ impl From<&TransactionRecord> for FFITransactionRecord { Box::into_raw(output_slice) as *mut FFIOutputDetail }; + // Typed special-transaction payload (ProRegTx service IP etc.) + let special_transaction_payload = match &value.transaction.special_transaction_payload { + Some(payload) => Box::into_raw(Box::new(FFISpecialTransactionPayload::from(payload))), + None => std::ptr::null_mut(), + }; + FFITransactionRecord { txid, net_amount, @@ -958,6 +967,7 @@ impl From<&TransactionRecord> for FFITransactionRecord { tx_data, tx_len, label, + special_transaction_payload, } } } @@ -995,6 +1005,12 @@ impl Drop for FFITransactionRecord { self.label = std::ptr::null_mut(); } + + if !self.special_transaction_payload.is_null() { + let _ = unsafe { Box::from_raw(self.special_transaction_payload) }; + + self.special_transaction_payload = std::ptr::null_mut(); + } } } @@ -1041,17 +1057,64 @@ pub unsafe extern "C" fn managed_core_account_get_transactions( true } -#[cfg(feature = "keep-finalized-transactions")] -/// Free transactions array returned by managed_core_account_get_transactions +/// Get the transactions of a managed account that carry a DIP-2 special +/// payload (provider registrations / updates, asset locks, …). +/// +/// Unlike `managed_core_account_get_transactions` this is available in +/// every feature configuration: provider-relevant records are retained +/// in memory even after their block is chainlocked (see +/// `ManagedCoreKeysAccount::drop_finalized_transaction` in `key-wallet`), +/// so a provider-key account's registration history — including each +/// masternode's service IP — remains queryable without the +/// `keep-finalized-transactions` feature. +/// +/// Returns records in txid order. Each record's +/// `special_transaction_payload` field is non-null. /// -/// Only available with the `keep-finalized-transactions` Cargo feature, in -/// which configuration `managed_core_account_get_transactions` is also -/// available — the two functions are paired. +/// # Safety +/// +/// - `account` must be a valid pointer to an FFIManagedCoreAccount instance +/// - `transactions_out` must be a valid pointer to receive the transactions array pointer +/// - `count_out` must be a valid pointer to receive the count +/// - The caller must free the returned array using `managed_core_account_free_transactions` +#[no_mangle] +pub unsafe extern "C" fn managed_core_account_get_special_transactions( + account: *const FFIManagedCoreAccount, + transactions_out: *mut *mut FFITransactionRecord, + count_out: *mut usize, +) -> bool { + if account.is_null() || transactions_out.is_null() || count_out.is_null() { + return false; + } + + let account = &*account; + let ffi_tx = account + .keys_account() + .transactions() + .values() + .filter(|record| record.transaction.special_transaction_payload.is_some()) + .map(FFITransactionRecord::from) + .collect::>(); + + if ffi_tx.is_empty() { + *transactions_out = std::ptr::null_mut(); + *count_out = 0; + return true; + } + + *count_out = ffi_tx.len(); + *transactions_out = Box::into_raw(ffi_tx.into_boxed_slice()) as *mut FFITransactionRecord; + true +} + +/// Free a transactions array returned by +/// `managed_core_account_get_transactions` or +/// `managed_core_account_get_special_transactions`. /// /// # Safety /// -/// - `transactions` must be a pointer returned by `managed_core_account_get_transactions` -/// - `count` must be the count returned by `managed_core_account_get_transactions` +/// - `transactions` must be a pointer returned by one of the paired getters +/// - `count` must be the count returned alongside it /// - This function must only be called once per allocation #[no_mangle] pub unsafe extern "C" fn managed_core_account_free_transactions( @@ -1768,8 +1831,10 @@ mod tests { }; use dash_network::ffi::FFINetwork; use dashcore::Transaction; + #[cfg(feature = "keep-finalized-transactions")] + use key_wallet::managed_account::transaction_record::{OutputDetail, OutputRole}; use key_wallet::managed_account::transaction_record::{ - OutputDetail, OutputRole, TransactionDirection, TransactionRecord, + TransactionDirection, TransactionRecord, }; use key_wallet::transaction_checking::transaction_context::TransactionContext; use key_wallet::transaction_checking::transaction_router::TransactionType; @@ -2638,6 +2703,7 @@ mod tests { // Create label label: CString::new("Payment for coffee").unwrap().into_raw(), + special_transaction_payload: std::ptr::null_mut(), }; // Second record: empty sub-arrays @@ -2668,6 +2734,7 @@ mod tests { tx_data: std::ptr::null_mut(), tx_len: 0, label: std::ptr::null_mut(), + special_transaction_payload: std::ptr::null_mut(), }; records.push(r0); @@ -2682,4 +2749,177 @@ mod tests { managed_core_account_free_transactions(records_ptr, count); } } + + /// Masternode-registration transaction wrapping the mainnet-shape + /// ProRegTx payload fixture (service `54.148.58.128:9999`). + fn proreg_transaction() -> Transaction { + use dashcore::blockdata::transaction::special_transaction::TransactionPayload; + + Transaction { + version: 3, + lock_time: 0, + input: vec![], + output: vec![], + special_transaction_payload: Some(TransactionPayload::ProviderRegistrationPayloadType( + crate::special_payload::mainnet_shape_proreg_payload(), + )), + } + } + + /// Acceptance round-trip for issue #875: a stored `ProRegTx` record + /// exposes its typed payload — service address included — without the + /// consumer decoding `tx_data`. + #[test] + fn test_transaction_record_exposes_typed_proreg_payload() { + use dashcore::hashes::hex::FromHex; + + let tx = proreg_transaction(); + let record = TransactionRecord::new( + tx, + key_wallet::AccountType::ProviderOwnerKeys, + TransactionContext::Mempool, + TransactionType::ProviderRegistration, + TransactionDirection::Internal, + vec![], + vec![], + 0, + ); + + let ffi = FFITransactionRecord::from(&record); + assert!(!ffi.special_transaction_payload.is_null()); + let payload = unsafe { &*ffi.special_transaction_payload }; + assert_eq!(payload.payload_type, 1, "DIP-2 type 1 = ProRegTx"); + assert!(!payload.provider_registration.is_null()); + + let reg = unsafe { &*payload.provider_registration }; + let service = + unsafe { std::ffi::CStr::from_ptr(reg.service_address) }.to_str().expect("utf8"); + assert_eq!(service, "54.148.58.128:9999"); + assert_eq!(reg.service_port, 9999); + let expected_hash = + <[u8; 20]>::from_hex("70993555a01f7e8d6179d6135b5c56809d2d1d36").expect("hash160"); + assert_eq!(reg.owner_key_hash, expected_hash); + assert_eq!(reg.voting_key_hash, expected_hash); + + // The raw bytes escape hatch still round-trips alongside the + // typed payload. + let tx_bytes = unsafe { std::slice::from_raw_parts(ffi.tx_data, ffi.tx_len) }; + let decoded: Transaction = + dashcore::consensus::deserialize(tx_bytes).expect("tx_data must stay decodable"); + assert_eq!(decoded.txid().to_byte_array(), ffi.txid); + } + + /// `managed_core_account_get_special_transactions` surfaces + /// payload-carrying records from a provider keys account in every + /// feature configuration (this is how retained `ProRegTx` records + /// stay queryable without `keep-finalized-transactions`). + #[test] + fn test_get_special_transactions_returns_proreg_record() { + use key_wallet::transaction_checking::BlockInfo; + + unsafe { + let mut error = FFIError::default(); + let manager = wallet_manager_create(FFINetwork::Testnet, &mut error); + assert!(!manager.is_null()); + + let mnemonic = CString::new(TEST_MNEMONIC).unwrap(); + assert!(wallet_manager_add_wallet_from_mnemonic_with_options( + manager, + mnemonic.as_ptr(), + ptr::null(), + &mut error, + )); + + let mut wallet_ids_out: *mut u8 = ptr::null_mut(); + let mut count_out: usize = 0; + assert!(wallet_manager_get_wallet_ids( + manager, + &mut wallet_ids_out, + &mut count_out, + &mut error, + )); + assert_eq!(count_out, 1); + + // Insert a chainlocked ProRegTx record into the provider + // owner-keys account plus a payload-less control record. + let proreg_tx = proreg_transaction(); + let proreg_txid = proreg_tx.txid(); + let plain_tx = Transaction { + version: 2, + lock_time: 0, + input: vec![], + output: vec![], + special_transaction_payload: None, + }; + let account_type_rust = key_wallet::AccountType::ProviderOwnerKeys; + let block_hash = dashcore::BlockHash::from_byte_array([9u8; 32]); + + let mut wallet_id_array = [0u8; 32]; + ptr::copy_nonoverlapping(wallet_ids_out, wallet_id_array.as_mut_ptr(), 32); + + let manager_ref = &*manager; + manager_ref.runtime.block_on(async { + let mut manager_guard = manager_ref.manager.write().await; + let wallet_info = + manager_guard.get_wallet_info_mut(&wallet_id_array).expect("wallet"); + let mut account = crate::address_pool::get_managed_account_by_type_mut( + &mut wallet_info.accounts, + &account_type_rust, + ) + .expect("provider owner keys account"); + for (tx, tx_type) in [ + (proreg_tx.clone(), TransactionType::ProviderRegistration), + (plain_tx.clone(), TransactionType::Standard), + ] { + let txid = tx.txid(); + let record = TransactionRecord::new( + tx, + account_type_rust, + TransactionContext::InChainLockedBlock(BlockInfo::new( + 100, + block_hash, + 1_700_000_000, + )), + tx_type, + TransactionDirection::Internal, + vec![], + vec![], + 0, + ); + account.transactions_mut().insert(txid, record); + } + }); + + let result = managed_wallet_get_account( + manager, + wallet_ids_out, + 0, + FFIAccountKind::ProviderOwnerKeys, + ); + assert!(!result.account.is_null()); + + let mut txs_out: *mut FFITransactionRecord = ptr::null_mut(); + let mut txs_count: usize = 0; + assert!(managed_core_account_get_special_transactions( + result.account, + &mut txs_out, + &mut txs_count, + )); + assert_eq!(txs_count, 1, "only the payload-carrying record is returned"); + + let record = &*txs_out; + assert_eq!(record.txid, proreg_txid.to_byte_array()); + assert!(!record.special_transaction_payload.is_null()); + let payload = &*record.special_transaction_payload; + assert_eq!(payload.payload_type, 1); + let reg = &*payload.provider_registration; + let service = std::ffi::CStr::from_ptr(reg.service_address).to_str().expect("utf8"); + assert_eq!(service, "54.148.58.128:9999"); + + managed_core_account_free_transactions(txs_out, txs_count); + managed_core_account_free(result.account); + wallet_manager_free_wallet_ids(wallet_ids_out, count_out); + wallet_manager_free(manager); + } + } } diff --git a/key-wallet-ffi/src/special_payload.rs b/key-wallet-ffi/src/special_payload.rs new file mode 100644 index 000000000..33cfc81aa --- /dev/null +++ b/key-wallet-ffi/src/special_payload.rs @@ -0,0 +1,548 @@ +//! FFI-safe typed views of DIP-2 special-transaction payloads. +//! +//! [`FFISpecialTransactionPayload`] is a tagged representation of +//! `dashcore`'s `TransactionPayload`: `payload_type` carries the on-wire +//! DIP-2 transaction type (`1` = ProRegTx … `9` = AssetUnlockTx), and for +//! the four masternode provider payload kinds a matching typed struct +//! pointer is non-null. Other payload kinds (coinbase, quorum commitment, +//! asset lock/unlock, …) currently expose only the discriminant; typed +//! structs for them can be added later without breaking the layout +//! contract (consumers must key off `payload_type`). +//! +//! All heap-allocated fields are owned by the structs and freed by their +//! `Drop` impls, which run when the owning +//! [`FFITransactionRecord`](crate::managed_account::FFITransactionRecord) +//! is freed. + +use std::net::{IpAddr, Ipv6Addr, SocketAddr}; +use std::os::raw::c_char; + +use dashcore::blockdata::transaction::special_transaction::provider_registration::ProviderRegistrationPayload; +use dashcore::blockdata::transaction::special_transaction::provider_update_registrar::ProviderUpdateRegistrarPayload; +use dashcore::blockdata::transaction::special_transaction::provider_update_revocation::ProviderUpdateRevocationPayload; +use dashcore::blockdata::transaction::special_transaction::provider_update_service::ProviderUpdateServicePayload; +use dashcore::blockdata::transaction::special_transaction::TransactionPayload; +use dashcore::bls_sig_utils::BLSPublicKey; +use dashcore::hashes::Hash; +use dashcore::ScriptBuf; + +/// Box `script`'s bytes into a raw `(ptr, len)` pair; `(null, 0)` for an +/// empty script. +fn script_to_raw(script: &ScriptBuf) -> (*mut u8, usize) { + let bytes = script.as_bytes().to_vec().into_boxed_slice(); + let len = bytes.len(); + if len == 0 { + (std::ptr::null_mut(), 0) + } else { + (Box::into_raw(bytes) as *mut u8, len) + } +} + +/// Free a `(ptr, len)` pair produced by [`script_to_raw`]. +/// +/// # Safety +/// +/// `ptr`/`len` must come from `script_to_raw` and not have been freed yet. +unsafe fn free_raw_script(ptr: &mut *mut u8, len: &mut usize) { + if !ptr.is_null() && *len > 0 { + let slice_ptr = std::ptr::slice_from_raw_parts_mut(*ptr, *len); + let _ = Box::from_raw(slice_ptr); + } + *ptr = std::ptr::null_mut(); + *len = 0; +} + +/// Render `addr` as its canonical display string (`ip:port`), boxed as a +/// C string. +fn socket_addr_to_cstring(addr: &SocketAddr) -> *mut c_char { + std::ffi::CString::new(addr.to_string()).unwrap_or_default().into_raw() +} + +/// Normalize `addr`'s IP to 16 IPv6 octets (IPv4 addresses are mapped), +/// matching the on-wire DIP-3 service field encoding. +fn socket_addr_to_ipv6_octets(addr: &SocketAddr) -> [u8; 16] { + match addr.ip() { + IpAddr::V4(v4) => v4.to_ipv6_mapped().octets(), + IpAddr::V6(v6) => v6.octets(), + } +} + +/// Copy a BLS public key's 48 raw bytes. +fn bls_public_key_bytes(key: &BLSPublicKey) -> [u8; 48] { + *AsRef::<[u8; 48]>::as_ref(key) +} + +/// Reassemble a `SocketAddr` from 16 IPv6 octets + port, un-mapping +/// v4-mapped addresses so the display string reads `54.148.58.128:9999` +/// rather than `[::ffff:54.148.58.128]:9999`. +fn socket_addr_from_ipv6_octets(octets: [u8; 16], port: u16) -> SocketAddr { + let v6 = Ipv6Addr::from(octets); + match v6.to_ipv4_mapped() { + Some(v4) => SocketAddr::new(IpAddr::V4(v4), port), + None => SocketAddr::new(IpAddr::V6(v6), port), + } +} + +/// Typed view of a DIP-3 `ProRegTx` (provider registration) payload. +/// +/// Byte-order notes: `collateral_txid` and hash fields use the same byte +/// order as the rest of this FFI surface (`to_byte_array`, i.e. the +/// internal little-endian hash representation). `service_ip` is the raw +/// 16-byte on-wire IPv6 form with IPv4 addresses v4-mapped +/// (`::ffff:a.b.c.d`); `service_address` is the human-readable +/// `ip:port` rendering of the same data. +#[repr(C)] +pub struct FFIProviderRegistrationPayload { + /// ProRegTx payload version. + pub version: u16, + /// Masternode type: `0` = regular, `1` = high-performance (Evo). + pub masternode_type: u16, + /// Masternode operating mode (`0` currently). + pub masternode_mode: u16, + /// Collateral outpoint txid (32 bytes, `to_byte_array` order). All + /// zeros when the collateral is an output of this transaction itself. + pub collateral_txid: [u8; 32], + /// Collateral outpoint output index. + pub collateral_vout: u32, + /// Masternode service endpoint as a display string, e.g. + /// `"54.148.58.128:9999"`. Owned by this struct. + pub service_address: *mut c_char, + /// Raw service IP: 16 IPv6 octets, IPv4 addresses v4-mapped. + pub service_ip: [u8; 16], + /// Service port (host byte order). + pub service_port: u16, + /// Owner key hash160 (20 bytes). + pub owner_key_hash: [u8; 20], + /// Voting key hash160 (20 bytes). + pub voting_key_hash: [u8; 20], + /// Operator BLS public key (48 bytes). + pub operator_public_key: [u8; 48], + /// Operator reward in basis points (0–10000). + pub operator_reward: u16, + /// Payout script bytes (owned by this struct, null when empty). + pub script_payout: *mut u8, + /// Length of `script_payout`. + pub script_payout_len: usize, + /// `true` when the payload carries platform (Evo) fields; the three + /// fields below are only meaningful in that case. + pub has_platform_fields: bool, + /// Platform node ID (20 bytes, zeroed when `has_platform_fields` is false). + pub platform_node_id: [u8; 20], + /// Platform P2P port, `-1` when absent. + pub platform_p2p_port: i32, + /// Platform HTTP port, `-1` when absent. + pub platform_http_port: i32, +} + +impl From<&ProviderRegistrationPayload> for FFIProviderRegistrationPayload { + fn from(p: &ProviderRegistrationPayload) -> Self { + let (script_payout, script_payout_len) = script_to_raw(&p.script_payout); + FFIProviderRegistrationPayload { + version: p.version, + masternode_type: p.masternode_type as u16, + masternode_mode: p.masternode_mode, + collateral_txid: p.collateral_outpoint.txid.to_byte_array(), + collateral_vout: p.collateral_outpoint.vout, + service_address: socket_addr_to_cstring(&p.service_address), + service_ip: socket_addr_to_ipv6_octets(&p.service_address), + service_port: p.service_address.port(), + owner_key_hash: p.owner_key_hash.to_byte_array(), + voting_key_hash: p.voting_key_hash.to_byte_array(), + operator_public_key: bls_public_key_bytes(&p.operator_public_key), + operator_reward: p.operator_reward, + script_payout, + script_payout_len, + has_platform_fields: p.platform_node_id.is_some(), + platform_node_id: p.platform_node_id.map(|h| h.to_byte_array()).unwrap_or([0; 20]), + platform_p2p_port: p.platform_p2p_port.map_or(-1, i32::from), + platform_http_port: p.platform_http_port.map_or(-1, i32::from), + } + } +} + +impl Drop for FFIProviderRegistrationPayload { + fn drop(&mut self) { + if !self.service_address.is_null() { + let _ = unsafe { std::ffi::CString::from_raw(self.service_address) }; + self.service_address = std::ptr::null_mut(); + } + unsafe { free_raw_script(&mut self.script_payout, &mut self.script_payout_len) }; + } +} + +/// Typed view of a DIP-3 `ProUpServTx` (provider update service) payload. +/// +/// Same byte-order conventions as [`FFIProviderRegistrationPayload`]. +#[repr(C)] +pub struct FFIProviderUpdateServicePayload { + /// ProUpServTx payload version. + pub version: u16, + /// Masternode type (`0` regular / `1` Evo), `-1` when the payload + /// version predates the field. + pub masternode_type: i32, + /// ProRegTx hash of the masternode being updated (32 bytes, + /// `to_byte_array` order). + pub pro_tx_hash: [u8; 32], + /// New masternode service endpoint as a display string. Owned by + /// this struct. + pub service_address: *mut c_char, + /// Raw new service IP: 16 IPv6 octets, IPv4 addresses v4-mapped. + pub service_ip: [u8; 16], + /// New service port (host byte order). + pub service_port: u16, + /// Operator payout script bytes (owned by this struct, null when empty). + pub script_payout: *mut u8, + /// Length of `script_payout`. + pub script_payout_len: usize, + /// `true` when the payload carries platform (Evo) fields; the three + /// fields below are only meaningful in that case. + pub has_platform_fields: bool, + /// Platform node ID (20 bytes, zeroed when `has_platform_fields` is false). + pub platform_node_id: [u8; 20], + /// Platform P2P port, `-1` when absent. + pub platform_p2p_port: i32, + /// Platform HTTP port, `-1` when absent. + pub platform_http_port: i32, +} + +impl From<&ProviderUpdateServicePayload> for FFIProviderUpdateServicePayload { + fn from(p: &ProviderUpdateServicePayload) -> Self { + // The wire format stores the IP as a u128 whose little-endian + // byte view is the 16 IPv6 octets in network order (see the + // `ProviderUpdateServicePayload` consensus round-trip tests). + let service_ip = p.ip_address.to_le_bytes(); + let service = socket_addr_from_ipv6_octets(service_ip, p.port); + let (script_payout, script_payout_len) = script_to_raw(&p.script_payout); + FFIProviderUpdateServicePayload { + version: p.version, + masternode_type: p.mn_type.map_or(-1, i32::from), + pro_tx_hash: p.pro_tx_hash.to_byte_array(), + service_address: socket_addr_to_cstring(&service), + service_ip, + service_port: p.port, + script_payout, + script_payout_len, + has_platform_fields: p.platform_node_id.is_some(), + platform_node_id: p.platform_node_id.unwrap_or([0; 20]), + platform_p2p_port: p.platform_p2p_port.map_or(-1, i32::from), + platform_http_port: p.platform_http_port.map_or(-1, i32::from), + } + } +} + +impl Drop for FFIProviderUpdateServicePayload { + fn drop(&mut self) { + if !self.service_address.is_null() { + let _ = unsafe { std::ffi::CString::from_raw(self.service_address) }; + self.service_address = std::ptr::null_mut(); + } + unsafe { free_raw_script(&mut self.script_payout, &mut self.script_payout_len) }; + } +} + +/// Typed view of a DIP-3 `ProUpRegTx` (provider update registrar) payload. +#[repr(C)] +pub struct FFIProviderUpdateRegistrarPayload { + /// ProUpRegTx payload version. + pub version: u16, + /// ProRegTx hash of the masternode being updated (32 bytes, + /// `to_byte_array` order). + pub pro_tx_hash: [u8; 32], + /// Masternode operating mode (`0` currently). + pub provider_mode: u16, + /// New operator BLS public key (48 bytes). + pub operator_public_key: [u8; 48], + /// New voting key hash160 (20 bytes). + pub voting_key_hash: [u8; 20], + /// New payout script bytes (owned by this struct, null when empty). + pub script_payout: *mut u8, + /// Length of `script_payout`. + pub script_payout_len: usize, +} + +impl From<&ProviderUpdateRegistrarPayload> for FFIProviderUpdateRegistrarPayload { + fn from(p: &ProviderUpdateRegistrarPayload) -> Self { + let (script_payout, script_payout_len) = script_to_raw(&p.script_payout); + FFIProviderUpdateRegistrarPayload { + version: p.version, + pro_tx_hash: p.pro_tx_hash.to_byte_array(), + provider_mode: p.provider_mode, + operator_public_key: bls_public_key_bytes(&p.operator_public_key), + voting_key_hash: p.voting_key_hash.to_byte_array(), + script_payout, + script_payout_len, + } + } +} + +impl Drop for FFIProviderUpdateRegistrarPayload { + fn drop(&mut self) { + unsafe { free_raw_script(&mut self.script_payout, &mut self.script_payout_len) }; + } +} + +/// Typed view of a DIP-3 `ProUpRevTx` (provider update revocation) payload. +#[repr(C)] +pub struct FFIProviderUpdateRevocationPayload { + /// ProUpRevTx payload version. + pub version: u16, + /// ProRegTx hash of the masternode being revoked (32 bytes, + /// `to_byte_array` order). + pub pro_tx_hash: [u8; 32], + /// Revocation reason (`0` not specified, `1` termination of service, + /// `2` compromised keys, `3` change of keys). + pub reason: u16, +} + +impl From<&ProviderUpdateRevocationPayload> for FFIProviderUpdateRevocationPayload { + fn from(p: &ProviderUpdateRevocationPayload) -> Self { + FFIProviderUpdateRevocationPayload { + version: p.version, + pro_tx_hash: p.pro_tx_hash.to_byte_array(), + reason: p.reason, + } + } +} + +/// Tagged FFI view of a transaction's DIP-2 special payload. +/// +/// `payload_type` is the on-wire DIP-2 transaction type (`1` ProRegTx, +/// `2` ProUpServTx, `3` ProUpRegTx, `4` ProUpRevTx, `5` CbTx, `6` QcTx, +/// `7` MnHfTx, `8` AssetLockTx, `9` AssetUnlockTx; pre-DIP-0002 +/// transactions with non-standard type bytes surface their raw value). +/// For the four provider payload kinds the matching struct pointer is +/// non-null; exactly one pointer is ever non-null. All pointers are owned +/// by this struct and freed with it. +#[repr(C)] +pub struct FFISpecialTransactionPayload { + /// On-wire DIP-2 transaction type discriminant. + pub payload_type: u16, + /// Non-null iff `payload_type == 1` (ProRegTx). + pub provider_registration: *mut FFIProviderRegistrationPayload, + /// Non-null iff `payload_type == 2` (ProUpServTx). + pub provider_update_service: *mut FFIProviderUpdateServicePayload, + /// Non-null iff `payload_type == 3` (ProUpRegTx). + pub provider_update_registrar: *mut FFIProviderUpdateRegistrarPayload, + /// Non-null iff `payload_type == 4` (ProUpRevTx). + pub provider_update_revocation: *mut FFIProviderUpdateRevocationPayload, +} + +impl From<&TransactionPayload> for FFISpecialTransactionPayload { + fn from(payload: &TransactionPayload) -> Self { + let mut ffi = FFISpecialTransactionPayload { + payload_type: payload.get_type().to_u16(), + provider_registration: std::ptr::null_mut(), + provider_update_service: std::ptr::null_mut(), + provider_update_registrar: std::ptr::null_mut(), + provider_update_revocation: std::ptr::null_mut(), + }; + match payload { + TransactionPayload::ProviderRegistrationPayloadType(p) => { + ffi.provider_registration = Box::into_raw(Box::new(p.into())); + } + TransactionPayload::ProviderUpdateServicePayloadType(p) => { + ffi.provider_update_service = Box::into_raw(Box::new(p.into())); + } + TransactionPayload::ProviderUpdateRegistrarPayloadType(p) => { + ffi.provider_update_registrar = Box::into_raw(Box::new(p.into())); + } + TransactionPayload::ProviderUpdateRevocationPayloadType(p) => { + ffi.provider_update_revocation = Box::into_raw(Box::new(p.into())); + } + // Only the discriminant is exposed for the remaining payload + // kinds; typed structs can be added later. + _ => {} + } + ffi + } +} + +impl Drop for FFISpecialTransactionPayload { + fn drop(&mut self) { + if !self.provider_registration.is_null() { + let _ = unsafe { Box::from_raw(self.provider_registration) }; + self.provider_registration = std::ptr::null_mut(); + } + if !self.provider_update_service.is_null() { + let _ = unsafe { Box::from_raw(self.provider_update_service) }; + self.provider_update_service = std::ptr::null_mut(); + } + if !self.provider_update_registrar.is_null() { + let _ = unsafe { Box::from_raw(self.provider_update_registrar) }; + self.provider_update_registrar = std::ptr::null_mut(); + } + if !self.provider_update_revocation.is_null() { + let _ = unsafe { Box::from_raw(self.provider_update_revocation) }; + self.provider_update_revocation = std::ptr::null_mut(); + } + } +} + +/// Mainnet-shape ProRegTx payload for tests: owner/voting hash160 and +/// service endpoint from the masternode registration referenced in +/// issue #875. +#[cfg(test)] +pub(crate) fn mainnet_shape_proreg_payload() -> ProviderRegistrationPayload { + use dashcore::blockdata::transaction::special_transaction::provider_registration::ProviderMasternodeType; + use std::str::FromStr; + + let key_hash = dashcore::PubkeyHash::from_hex("70993555a01f7e8d6179d6135b5c56809d2d1d36") + .expect("hash160"); + ProviderRegistrationPayload { + version: 1, + masternode_type: ProviderMasternodeType::Regular, + masternode_mode: 0, + collateral_outpoint: dashcore::OutPoint { + txid: dashcore::Txid::from_hex( + "e0ab1fc3cbe921a2a026e5c9c17d1e5b1b1c460ac4b60964d97b0e6ba9a5f719", + ) + .expect("txid"), + vout: 1, + }, + service_address: SocketAddr::from_str("54.148.58.128:9999").expect("socket addr"), + owner_key_hash: key_hash, + operator_public_key: BLSPublicKey::from([0x11; 48]), + voting_key_hash: key_hash, + operator_reward: 250, + script_payout: ScriptBuf::from_hex("76a914fef33f56f709ba6b08d073932f925afedb606d0288ac") + .expect("script"), + inputs_hash: dashcore::hash_types::InputsHash::from_slice(&[0x22; 32]) + .expect("inputs hash"), + signature: vec![0x33; 65], + platform_node_id: None, + platform_p2p_port: None, + platform_http_port: None, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use dashcore::bls_sig_utils::BLSSignature; + use dashcore::hash_types::InputsHash; + use dashcore::hashes::hex::FromHex; + use dashcore::Txid; + use std::ffi::CStr; + + #[test] + fn proreg_payload_exposes_service_and_keys() { + let payload = + TransactionPayload::ProviderRegistrationPayloadType(mainnet_shape_proreg_payload()); + let ffi = FFISpecialTransactionPayload::from(&payload); + + assert_eq!(ffi.payload_type, 1); + assert!(ffi.provider_update_service.is_null()); + assert!(ffi.provider_update_registrar.is_null()); + assert!(ffi.provider_update_revocation.is_null()); + + let reg = unsafe { &*ffi.provider_registration }; + let service = unsafe { CStr::from_ptr(reg.service_address) }.to_str().expect("utf8"); + assert_eq!(service, "54.148.58.128:9999"); + assert_eq!(reg.service_port, 9999); + let mut expected_ip = [0u8; 16]; + expected_ip[10] = 0xff; + expected_ip[11] = 0xff; + expected_ip[12..].copy_from_slice(&[54, 148, 58, 128]); + assert_eq!(reg.service_ip, expected_ip); + + let expected_hash = + <[u8; 20]>::from_hex("70993555a01f7e8d6179d6135b5c56809d2d1d36").expect("hash160"); + assert_eq!(reg.owner_key_hash, expected_hash); + assert_eq!(reg.voting_key_hash, expected_hash); + assert_eq!(reg.operator_public_key, [0x11; 48]); + assert_eq!(reg.operator_reward, 250); + assert_eq!(reg.masternode_type, 0); + assert_eq!(reg.collateral_vout, 1); + assert!(!reg.has_platform_fields); + assert_eq!(reg.platform_p2p_port, -1); + + let script = + unsafe { std::slice::from_raw_parts(reg.script_payout, reg.script_payout_len) }; + assert_eq!( + script, + Vec::::from_hex("76a914fef33f56f709ba6b08d073932f925afedb606d0288ac") + .expect("script hex") + .as_slice() + ); + } + + #[test] + fn proupserv_payload_exposes_new_service_address() { + // 54.148.58.128 as a v4-mapped IPv6, little-endian u128 view. + let mut octets = [0u8; 16]; + octets[10] = 0xff; + octets[11] = 0xff; + octets[12..].copy_from_slice(&[54, 148, 58, 128]); + + let payload = + TransactionPayload::ProviderUpdateServicePayloadType(ProviderUpdateServicePayload { + version: 1, + mn_type: None, + pro_tx_hash: Txid::from_slice(&[0x44; 32]).expect("txid"), + ip_address: u128::from_le_bytes(octets), + port: 19999, + script_payout: dashcore::ScriptBuf::new(), + inputs_hash: InputsHash::from_slice(&[0x55; 32]).expect("inputs hash"), + platform_node_id: None, + platform_p2p_port: None, + platform_http_port: None, + payload_sig: BLSSignature::from([0x66; 96]), + }); + let ffi = FFISpecialTransactionPayload::from(&payload); + + assert_eq!(ffi.payload_type, 2); + assert!(ffi.provider_registration.is_null()); + let upserv = unsafe { &*ffi.provider_update_service }; + let service = unsafe { CStr::from_ptr(upserv.service_address) }.to_str().expect("utf8"); + assert_eq!(service, "54.148.58.128:19999"); + assert_eq!(upserv.service_ip, octets); + assert_eq!(upserv.service_port, 19999); + assert_eq!(upserv.pro_tx_hash, [0x44; 32]); + assert_eq!(upserv.masternode_type, -1); + assert!(upserv.script_payout.is_null(), "empty payout script must be null"); + assert_eq!(upserv.script_payout_len, 0); + } + + #[test] + fn prouprev_payload_exposes_reason() { + let payload = TransactionPayload::ProviderUpdateRevocationPayloadType( + ProviderUpdateRevocationPayload { + version: 1, + pro_tx_hash: Txid::from_slice(&[0x77; 32]).expect("txid"), + reason: 2, + inputs_hash: InputsHash::from_slice(&[0x88; 32]).expect("inputs hash"), + payload_sig: BLSSignature::from([0x99; 96]), + }, + ); + let ffi = FFISpecialTransactionPayload::from(&payload); + + assert_eq!(ffi.payload_type, 4); + let rev = unsafe { &*ffi.provider_update_revocation }; + assert_eq!(rev.pro_tx_hash, [0x77; 32]); + assert_eq!(rev.reason, 2); + } + + #[test] + fn unsupported_payload_kind_exposes_discriminant_only() { + use dashcore::blockdata::transaction::special_transaction::coinbase::CoinbasePayload; + use dashcore::hash_types::{MerkleRootMasternodeList, MerkleRootQuorums}; + + let payload = TransactionPayload::CoinbasePayloadType(CoinbasePayload { + version: 2, + height: 100, + merkle_root_masternode_list: MerkleRootMasternodeList::from_slice(&[0; 32]) + .expect("hash"), + merkle_root_quorums: MerkleRootQuorums::from_slice(&[0; 32]).expect("hash"), + best_cl_height: None, + best_cl_signature: None, + asset_locked_amount: None, + }); + let ffi = FFISpecialTransactionPayload::from(&payload); + + assert_eq!(ffi.payload_type, 5); + assert!(ffi.provider_registration.is_null()); + assert!(ffi.provider_update_service.is_null()); + assert!(ffi.provider_update_registrar.is_null()); + assert!(ffi.provider_update_revocation.is_null()); + } +} diff --git a/key-wallet/src/managed_account/managed_core_keys_account.rs b/key-wallet/src/managed_account/managed_core_keys_account.rs index 62fa7a977..ecb974a61 100644 --- a/key-wallet/src/managed_account/managed_core_keys_account.rs +++ b/key-wallet/src/managed_account/managed_core_keys_account.rs @@ -51,7 +51,9 @@ pub struct ManagedCoreKeysAccount { /// including ones that have been chainlocked. With the feature OFF /// (the default), records of chainlocked transactions are dropped /// from this map and only their txids are retained in - /// `finalized_txids` to bound memory growth. + /// `finalized_txids` to bound memory growth — except provider-payload + /// records on provider-key accounts, which are always retained (see + /// [`Self::drop_finalized_transaction`]). transactions: BTreeMap, /// Txids of transactions that have been finalized in a chainlocked /// block and whose full records have been dropped from @@ -98,15 +100,59 @@ impl ManagedCoreKeysAccount { /// [`ManagedAccountTrait::transaction_is_finalized`] keeps /// returning `true`. /// + /// Exception: on masternode provider-key accounts, records whose + /// transaction carries a DIP-3 provider payload (`ProRegTx`, + /// `ProUpServTx`, `ProUpRegTx`, `ProUpRevTx`) are retained. Their + /// payload data — most importantly the masternode's service IP:port + /// and `proTxHash` linkage — is what the account exists to surface, + /// and it would otherwise become unreachable the moment the + /// registration is chainlocked. These records are rare (one per + /// masternode lifecycle event matching our keys), so retaining them + /// does not meaningfully grow memory. + /// /// With the feature on the full record stays in `transactions` /// indefinitely, so there's nothing to do — the function does not /// exist in that mode. #[cfg(not(feature = "keep-finalized-transactions"))] pub(crate) fn drop_finalized_transaction(&mut self, txid: &Txid) { self.finalized_txids.insert(*txid); + if self.retains_provider_payload_record(txid) { + return; + } self.transactions.remove(txid); } + /// Whether the record for `txid` must survive finalization: true only + /// on provider-key accounts for transactions carrying a DIP-3 + /// provider payload. See [`Self::drop_finalized_transaction`]. + #[cfg(not(feature = "keep-finalized-transactions"))] + fn retains_provider_payload_record(&self, txid: &Txid) -> bool { + use dashcore::blockdata::transaction::special_transaction::TransactionPayload; + + let is_provider_keys_account = matches!( + self.managed_account_type, + ManagedAccountType::ProviderVotingKeys { .. } + | ManagedAccountType::ProviderOwnerKeys { .. } + | ManagedAccountType::ProviderOperatorKeys { .. } + | ManagedAccountType::ProviderPlatformKeys { .. } + ); + if !is_provider_keys_account { + return false; + } + + self.transactions.get(txid).is_some_and(|record| { + matches!( + record.transaction.special_transaction_payload, + Some( + TransactionPayload::ProviderRegistrationPayloadType(_) + | TransactionPayload::ProviderUpdateServicePayloadType(_) + | TransactionPayload::ProviderUpdateRegistrarPayloadType(_) + | TransactionPayload::ProviderUpdateRevocationPayloadType(_) + ) + ) + }) + } + /// Promote any `InBlock` records at height `<= cl_height` to /// [`TransactionContext::InChainLockedBlock`] and return their txids. /// @@ -367,10 +413,10 @@ impl ManagedAccountTrait for ManagedCoreKeysAccount { } /// With the feature OFF, chainlocked records are dropped from - /// `transactions` and only their txids are retained in - /// `finalized_txids`. A live record can never satisfy this check - /// (it would have been pruned at the chainlock event), so the only - /// `true` answer comes from the txid set. + /// `transactions` and their txids are retained in `finalized_txids`. + /// The txid set is authoritative: even the provider-payload records + /// that [`Self::drop_finalized_transaction`] keeps alive in + /// `transactions` have their txid added to the set at finalization. #[cfg(not(feature = "keep-finalized-transactions"))] fn transaction_is_finalized(&self, txid: &Txid) -> bool { self.finalized_txids.contains(txid) diff --git a/key-wallet/src/tests/keep_finalized_transactions_tests.rs b/key-wallet/src/tests/keep_finalized_transactions_tests.rs index dcfef377a..d5100cee8 100644 --- a/key-wallet/src/tests/keep_finalized_transactions_tests.rs +++ b/key-wallet/src/tests/keep_finalized_transactions_tests.rs @@ -22,6 +22,15 @@ use crate::{ transaction_checking::{BlockInfo, TransactionContext}, wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface, }; +#[cfg(not(feature = "keep-finalized-transactions"))] +use crate::{ + managed_account::{ + address_pool::KeySource, managed_account_type::ManagedAccountType, ManagedCoreKeysAccount, + }, + transaction_checking::account_checker::{AccountMatch, CoreAccountTypeMatch}, + transaction_checking::transaction_router::TransactionType as RoutedTransactionType, + Network, +}; use dashcore::ephemerealdata::chain_lock::ChainLock; #[cfg(not(feature = "keep-finalized-transactions"))] use dashcore::ephemerealdata::instant_lock::InstantLock; @@ -213,6 +222,194 @@ async fn test_apply_chain_lock_skips_unmined_and_above_height() { assert!(!ctx.bip44_account().transaction_is_finalized(&block_txid)); } +/// Build a masternode-registration transaction whose DIP-3 payload carries a +/// service endpoint, plus the [`AccountMatch`] a provider owner-keys account +/// would report for it. +#[cfg(not(feature = "keep-finalized-transactions"))] +fn proreg_tx_and_owner_match() -> (Transaction, AccountMatch) { + use dashcore::blockdata::transaction::special_transaction::provider_registration::{ + ProviderMasternodeType, ProviderRegistrationPayload, + }; + use dashcore::blockdata::transaction::special_transaction::TransactionPayload; + use dashcore::bls_sig_utils::BLSPublicKey; + use dashcore::hash_types::InputsHash; + use dashcore::{OutPoint, PubkeyHash, ScriptBuf, Txid}; + use std::net::SocketAddr; + use std::str::FromStr; + + let key_hash = PubkeyHash::from_slice(&[0x70; 20]).expect("hash160"); + let payload = ProviderRegistrationPayload { + version: 1, + masternode_type: ProviderMasternodeType::Regular, + masternode_mode: 0, + collateral_outpoint: OutPoint { + txid: Txid::from_slice(&[0x19; 32]).expect("txid"), + vout: 1, + }, + service_address: SocketAddr::from_str("54.148.58.128:9999").expect("socket addr"), + owner_key_hash: key_hash, + operator_public_key: BLSPublicKey::from([0x11; 48]), + voting_key_hash: key_hash, + operator_reward: 0, + script_payout: ScriptBuf::new(), + inputs_hash: InputsHash::from_slice(&[0x22; 32]).expect("inputs hash"), + signature: vec![0x33; 65], + platform_node_id: None, + platform_p2p_port: None, + platform_http_port: None, + }; + let tx = Transaction { + version: 3, + lock_time: 0, + input: vec![], + output: vec![], + special_transaction_payload: Some(TransactionPayload::ProviderRegistrationPayloadType( + payload, + )), + }; + let account_match = AccountMatch { + account_type_match: CoreAccountTypeMatch::ProviderOwnerKeys { + involved_addresses: vec![], + }, + received: 0, + sent: 0, + received_for_credit_conversion: 0, + }; + (tx, account_match) +} + +/// Build a keys account of the given type without key material (the +/// retention decision only looks at the account type and the payload). +#[cfg(not(feature = "keep-finalized-transactions"))] +fn keys_account(account_type: AccountType) -> ManagedCoreKeysAccount { + let managed_type = ManagedAccountType::from_account_type( + account_type, + Network::Testnet, + &KeySource::NoKeySource, + ) + .expect("managed account type"); + ManagedCoreKeysAccount::new(managed_type, Network::Testnet) +} + +/// A `ProRegTx` first seen already chainlocked (the historical-rescan case +/// from masternode-key discovery) must keep its full record on a provider +/// owner-keys account even with the feature OFF: the payload's service +/// IP / proTxHash data is what the account exists to surface. +#[cfg(not(feature = "keep-finalized-transactions"))] +#[test] +fn test_provider_payload_record_retained_on_provider_account() { + let mut account = keys_account(AccountType::ProviderOwnerKeys); + let (tx, account_match) = proreg_tx_and_owner_match(); + let txid = tx.txid(); + + let block_hash = BlockHash::from_slice(&[5u8; 32]).expect("hash"); + let _ = account.record_transaction( + &tx, + &account_match, + TransactionContext::InChainLockedBlock(BlockInfo::new(100, block_hash, 1_700_000_000)), + RoutedTransactionType::ProviderRegistration, + ); + + assert!(account.transaction_is_finalized(&txid)); + assert!(account.has_transaction(&txid)); + let record = account + .transactions() + .get(&txid) + .expect("provider payload record must be retained past finalization"); + assert!(record.transaction.special_transaction_payload.is_some()); +} + +/// The retention exception is payload-driven: a payload-less transaction +/// on the same provider account still drops at finalization. +#[cfg(not(feature = "keep-finalized-transactions"))] +#[test] +fn test_plain_record_still_dropped_on_provider_account() { + let mut account = keys_account(AccountType::ProviderOwnerKeys); + let (_, account_match) = proreg_tx_and_owner_match(); + let tx = Transaction { + version: 2, + lock_time: 0, + input: vec![], + output: vec![], + special_transaction_payload: None, + }; + let txid = tx.txid(); + + let block_hash = BlockHash::from_slice(&[6u8; 32]).expect("hash"); + let _ = account.record_transaction( + &tx, + &account_match, + TransactionContext::InChainLockedBlock(BlockInfo::new(100, block_hash, 1_700_000_000)), + RoutedTransactionType::Standard, + ); + + assert!(account.transaction_is_finalized(&txid)); + assert!(account.has_transaction(&txid)); + assert!( + !account.transactions().contains_key(&txid), + "payload-less records must still be dropped at finalization" + ); +} + +/// The retention exception is also account-driven: the same `ProRegTx` +/// record drops at finalization on a non-provider keys account. +#[cfg(not(feature = "keep-finalized-transactions"))] +#[test] +fn test_provider_payload_record_dropped_on_non_provider_account() { + let mut account = keys_account(AccountType::IdentityRegistration); + let (tx, _) = proreg_tx_and_owner_match(); + let account_match = AccountMatch { + account_type_match: CoreAccountTypeMatch::IdentityRegistration { + involved_addresses: vec![], + }, + received: 0, + sent: 0, + received_for_credit_conversion: 0, + }; + let txid = tx.txid(); + + let block_hash = BlockHash::from_slice(&[7u8; 32]).expect("hash"); + let _ = account.record_transaction( + &tx, + &account_match, + TransactionContext::InChainLockedBlock(BlockInfo::new(100, block_hash, 1_700_000_000)), + RoutedTransactionType::ProviderRegistration, + ); + + assert!(account.transaction_is_finalized(&txid)); + assert!(!account.transactions().contains_key(&txid)); +} + +/// A retained provider record must also survive the deferred-chainlock +/// path (`apply_chain_lock` promoting an `InBlock` record) with its +/// context updated, and repeated chainlocks must be idempotent. +#[cfg(not(feature = "keep-finalized-transactions"))] +#[test] +fn test_provider_payload_record_retained_through_apply_chain_lock() { + let mut account = keys_account(AccountType::ProviderOwnerKeys); + let (tx, account_match) = proreg_tx_and_owner_match(); + let txid = tx.txid(); + + let block_hash = BlockHash::from_slice(&[8u8; 32]).expect("hash"); + let _ = account.record_transaction( + &tx, + &account_match, + TransactionContext::InBlock(BlockInfo::new(50, block_hash, 1_700_000_000)), + RoutedTransactionType::ProviderRegistration, + ); + + let promoted = account.apply_chain_lock(60); + assert_eq!(promoted, vec![txid]); + assert!(account.transaction_is_finalized(&txid)); + let record = account.transactions().get(&txid).expect("record retained"); + assert!(matches!(record.context, TransactionContext::InChainLockedBlock(_))); + + // A later chainlock must not re-promote (or drop) the retained record. + let promoted_again = account.apply_chain_lock(70); + assert!(promoted_again.is_empty()); + assert!(account.transactions().contains_key(&txid)); +} + /// IS-lock first, then a chainlocked block: the record must drop only at /// the chainlock step. We also assert that the chainlock event still /// "lands" — `transaction_is_finalized` must report `true` when asked From 35684e14618f046be10a9aaa40da1ef86cfe0860 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 23:12:47 +1000 Subject: [PATCH 043/149] chore(dash-spv): refresh masternode seed files (#874) Co-authored-by: QuantumExplorer <11468583+QuantumExplorer@users.noreply.github.com> --- dash-network-seeds/seeds/mainnet.txt | 113 ++++++++++++--------------- dash-network-seeds/seeds/testnet.txt | 13 +-- 2 files changed, 59 insertions(+), 67 deletions(-) diff --git a/dash-network-seeds/seeds/mainnet.txt b/dash-network-seeds/seeds/mainnet.txt index af4faac77..2acbec4da 100644 --- a/dash-network-seeds/seeds/mainnet.txt +++ b/dash-network-seeds/seeds/mainnet.txt @@ -1,16 +1,19 @@ -# Auto-generated by masternode-seeds-fetcher on 2026-06-22T08:21:33Z for mainnet +# Auto-generated by masternode-seeds-fetcher on 2026-07-13T07:19:17Z for mainnet # Source: Dash P2P network (mnlistdiff) -# Primary peer: 142.93.157.112:9999 -# Tip block hash: 0000000000000016a36e221c6615eed1fe3791deb6018837de7633c85ddf3b60 -# Tip block height: 2492563 -# 2063 seeds (1718 regular + 345 evo) of 2952 total masternodes, valid-only -# Probe summary: core_reachable=2036/2063 platform_reachable=340/345 ssl_valid=286/345 +# Primary peer: 8.219.54.127:9999 +# Tip block hash: 0000000000000005234788c6d97c81614eec529819dedbb5c364b7b99cab6d03 +# Tip block height: 2504059 +# 2054 seeds (1714 regular + 340 evo) of 2956 total masternodes, valid-only +# Probe summary: core_reachable=2040/2054 platform_reachable=335/340 ssl_valid=277/340 # Columns: # Values: core_reach/plat_reach=ok|timeout|refused|error|?, core_sync=sync|-N|+N|?, plat_live=ok|none|?|-, plat_ssl=valid|expired|self-signed|untrusted|no-handshake|?|- # Do not edit manually — refreshed weekly by .github/workflows/update-masternode-seeds.yml regular 1.117.75.220:9999 - ok sync - - - +regular 2.25.80.112:9999 - ok sync - - - evo 2.59.156.192:9999 443 ok sync ok ok valid regular 2.59.219.54:9999 - ok sync - - - +regular 2.233.120.35:9999 - ok sync - - - +regular 5.2.67.190:9999 - ok sync - - - regular 5.2.73.58:9999 - ok sync - - - regular 5.9.237.32:9999 - ok sync - - - regular 5.9.237.37:9999 - ok sync - - - @@ -21,7 +24,6 @@ regular 5.35.103.123:9999 - ok sync - - - regular 5.39.10.23:9999 - ok sync - - - regular 5.39.11.80:9999 - ok sync - - - evo 5.39.27.224:9999 443 ok sync refused none no-handshake -regular 5.53.124.120:9999 - ok sync - - - evo 5.75.133.148:9999 443 ok sync ok ok valid regular 5.75.251.102:9999 - ok sync - - - regular 5.135.14.86:9999 - ok sync - - - @@ -55,7 +57,6 @@ regular 5.175.162.160:9999 - ok sync - - - regular 5.175.162.161:9999 - ok sync - - - regular 5.175.162.162:9999 - ok sync - - - regular 5.180.234.247:9999 - ok sync - - - -evo 5.189.145.80:9999 443 ok sync ok ok valid evo 5.189.151.7:9999 443 ok sync ok ok valid evo 5.189.164.253:9999 443 ok sync ok ok valid regular 8.219.0.187:9999 - ok sync - - - @@ -201,6 +202,7 @@ regular 13.140.137.154:9999 - ok sync - - - regular 13.140.144.201:9999 - ok sync - - - regular 13.140.144.202:9999 - ok sync - - - regular 15.188.53.12:9999 - ok sync - - - +regular 15.204.112.226:9999 - ok sync - - - evo 15.235.102.215:9999 443 ok sync ok ok valid evo 15.235.102.216:9999 443 ok sync ok ok valid regular 23.88.22.66:9999 - ok sync - - - @@ -208,7 +210,6 @@ regular 23.88.22.67:9999 - ok sync - - - regular 23.88.22.68:9999 - ok sync - - - evo 23.88.63.58:9999 443 ok sync ok ok valid regular 23.95.186.248:9999 - ok sync - - - -regular 23.163.0.174:9999 - ok sync - - - regular 31.10.97.36:9999 - ok sync - - - regular 31.56.38.0:9999 - ok sync - - - regular 31.56.38.1:9999 - ok sync - - - @@ -224,11 +225,11 @@ regular 31.58.170.222:9999 - ok sync - - - regular 31.58.170.230:9999 - ok sync - - - regular 31.77.108.136:9999 - ok sync - - - regular 31.148.99.148:9999 - ok sync - - - -evo 31.220.77.84:9999 443 ok sync ok ok valid evo 31.220.84.93:9999 443 ok sync ok ok valid evo 31.220.91.43:9999 443 ok sync ok ok valid evo 31.220.91.60:9999 443 ok sync ok ok valid regular 34.209.37.222:9999 - ok sync - - - +evo 34.210.238.67:9999 443 ok sync ok ok valid regular 34.246.176.25:9999 - ok sync - - - regular 35.174.217.98:9999 - ok sync - - - regular 35.180.128.14:9999 - ok sync - - - @@ -250,7 +251,7 @@ evo 37.60.235.224:9999 443 ok sync ok ok valid evo 37.60.236.230:9999 443 ok sync ok ok valid evo 37.60.243.59:9999 443 ok sync ok ok valid evo 37.60.244.35:9999 443 ok sync ok ok valid -evo 37.60.244.87:9999 443 ok sync ok ok valid +evo 37.60.244.87:9999 443 ok sync ok ok expired evo 37.60.244.253:9999 443 ok sync ok ok valid evo 37.60.246.228:9999 443 ok sync ok ok valid evo 37.60.251.14:9999 443 ok sync ok ok valid @@ -279,6 +280,7 @@ evo 43.167.239.145:9999 443 ok sync ok ok valid evo 43.167.244.109:9999 443 ok sync ok ok valid regular 43.224.35.163:9999 - error ? - - - regular 43.229.77.46:9999 - ok sync - - - +evo 44.240.99.214:9999 443 ok sync ok ok valid regular 45.32.35.170:9999 - ok sync - - - evo 45.32.70.131:9999 443 ok sync ok ok valid regular 45.32.74.179:9999 - ok sync - - - @@ -319,7 +321,7 @@ evo 45.94.58.58:9999 443 ok sync ok ok valid regular 45.128.156.30:9999 - ok sync - - - regular 45.135.180.68:9999 - ok sync - - - evo 45.135.180.70:9999 443 ok sync ok ok valid -evo 45.135.180.79:9999 443 ok sync ok ok valid +evo 45.135.180.79:9999 443 ok sync ok ok expired evo 45.135.180.114:9999 443 ok sync ok ok valid evo 45.135.180.130:9999 443 ok sync ok ok valid regular 45.135.180.221:9999 - ok sync - - - @@ -369,8 +371,6 @@ regular 46.4.217.250:9999 - ok sync - - - regular 46.4.217.251:9999 - ok sync - - - regular 46.4.217.252:9999 - ok sync - - - regular 46.4.217.254:9999 - ok sync - - - -regular 46.30.189.187:9999 - ok sync - - - -regular 46.30.189.213:9999 - ok sync - - - regular 46.30.189.251:9999 - ok sync - - - regular 46.36.40.242:9999 - ok sync - - - regular 46.62.132.144:9999 - ok sync - - - @@ -380,7 +380,6 @@ regular 46.101.37.234:9999 - ok sync - - - regular 46.101.60.13:9999 - ok sync - - - regular 46.101.84.163:9999 - ok sync - - - regular 46.101.108.68:9999 - ok sync - - - -regular 46.101.110.82:9999 - ok sync - - - regular 46.101.118.96:9999 - ok sync - - - regular 46.101.129.63:9999 - ok sync - - - regular 46.101.240.34:9999 - ok sync - - - @@ -392,16 +391,14 @@ regular 49.12.1.250:9999 - ok sync - - - evo 49.12.102.105:9999 443 ok sync ok ok valid regular 49.12.189.12:9999 - ok sync - - - regular 49.12.198.203:9999 - ok sync - - - -regular 49.13.27.107:9999 - ok sync - - - evo 49.13.28.255:9999 443 ok sync ok ok valid regular 49.13.140.167:9999 - ok sync - - - -evo 49.13.154.121:9999 443 ok sync ok ok expired evo 49.13.193.251:9999 443 ok sync ok ok valid regular 49.13.197.215:9999 - ok sync - - - regular 49.13.209.157:9999 - ok sync - - - regular 49.13.212.202:9999 - ok sync - - - regular 49.13.212.210:9999 - ok sync - - - -evo 49.13.237.193:9999 443 ok sync ok ok valid +evo 49.13.237.193:9999 443 ok sync ok ok expired evo 50.116.28.103:9999 443 ok sync ok ok expired regular 51.38.27.202:9999 - ok sync - - - evo 51.68.3.29:9999 443 ok sync refused none no-handshake @@ -414,6 +411,7 @@ regular 51.158.188.88:9999 - ok sync - - - evo 51.161.34.134:9999 443 ok sync ok ok expired evo 51.195.235.166:9999 443 ok sync ok ok valid evo 52.10.213.198:9999 443 ok sync ok ok valid +evo 52.33.9.172:9999 443 ok sync ok ok valid evo 52.36.102.91:9999 443 ok sync ok ok valid regular 54.37.38.226:9999 - ok sync - - - evo 54.69.95.118:9999 443 ok sync ok ok valid @@ -467,9 +465,9 @@ evo 65.108.74.79:9999 443 ok sync ok ok expired evo 65.108.74.95:9999 443 ok sync ok ok expired evo 65.108.74.109:9999 443 ok sync ok ok expired regular 65.108.204.190:9999 - ok sync - - - -regular 65.108.221.24:9999 - ok sync - - - evo 65.108.246.145:9999 443 ok sync ok ok valid evo 65.109.84.201:9999 443 ok sync ok ok valid +evo 65.109.84.202:9999 443 ok sync ok ok valid evo 65.109.84.203:9999 443 ok sync ok ok valid evo 65.109.84.204:9999 443 ok sync ok ok expired regular 65.109.93.110:9999 - ok sync - - - @@ -505,10 +503,8 @@ regular 68.183.235.156:9999 - ok sync - - - regular 69.61.107.215:9999 - ok sync - - - regular 70.34.198.26:9999 - ok sync - - - evo 70.34.206.123:9999 443 ok sync ok ok valid -regular 70.34.210.105:9999 - ok sync - - - evo 72.60.38.160:9999 443 ok sync ok ok valid evo 72.62.58.108:9999 443 ok sync ok ok valid -regular 74.50.90.113:9999 - ok sync - - - evo 75.119.128.71:9999 443 ok sync ok ok valid evo 75.119.137.66:9999 443 ok sync ok ok valid evo 75.119.138.9:9999 443 ok sync ok ok valid @@ -518,29 +514,26 @@ evo 75.119.156.254:9999 443 ok sync ok ok valid evo 77.20.160.6:9999 443 ok sync ok ok valid regular 77.42.16.46:9999 - ok sync - - - regular 77.42.65.17:9999 - ok sync - - - -evo 77.42.74.68:9999 443 ok sync ok ok valid +evo 77.42.74.68:9999 443 ok sync ok ok expired regular 77.42.91.72:9999 - ok sync - - - regular 77.105.61.79:9999 - ok sync - - - regular 77.105.61.188:9999 - ok sync - - - regular 77.105.61.212:9999 - ok sync - - - evo 77.221.148.204:9999 443 ok sync ok ok valid -regular 77.223.99.4:9999 - ok sync - - - regular 77.237.235.0:9999 - ok sync - - - regular 77.237.236.66:9999 - ok sync - - - regular 77.237.236.214:9999 - ok sync - - - evo 77.237.245.19:9999 443 ok sync ok ok valid regular 78.47.119.20:9999 - ok sync - - - -regular 78.83.19.0:9999 - error ? - - - +regular 78.83.19.0:9999 - ok sync - - - evo 78.141.225.100:9999 443 ok sync ok ok valid regular 78.141.240.214:9999 - ok sync - - - regular 79.143.181.108:9999 - ok sync - - - -regular 80.208.230.144:9999 - error ? - - - -regular 80.209.233.182:9999 - error ? - - - -regular 80.209.239.129:9999 - error ? - - - +regular 80.208.230.144:9999 - ok sync - - - +regular 80.209.239.129:9999 - ok sync - - - evo 80.240.19.200:9999 443 ok sync ok ok valid regular 80.240.132.231:9999 - ok sync - - - regular 80.240.135.83:9999 - ok sync - - - -regular 80.249.147.8:9999 - ok sync - - - evo 81.14.228.87:9999 443 ok sync ok ok valid regular 81.227.250.51:9999 - ok sync - - - regular 82.180.146.192:9999 - ok sync - - - @@ -548,9 +541,8 @@ regular 82.196.1.10:9999 - ok sync - - - regular 82.202.230.83:9999 - ok sync - - - evo 82.208.20.153:9999 443 ok sync ok ok valid regular 82.223.81.127:9999 - ok sync - - - -regular 83.149.100.183:9999 - ok sync - - - +regular 83.149.100.183:9999 - error ? - - - regular 83.149.100.184:9999 - ok sync - - - -regular 83.149.100.185:9999 - error ? - - - regular 83.149.100.186:9999 - ok sync - - - regular 83.239.99.40:9999 - ok sync - - - regular 84.9.50.17:9999 - ok sync - - - @@ -624,8 +616,6 @@ regular 85.209.241.84:9999 - ok sync - - - regular 85.209.241.85:9999 - ok sync - - - regular 85.209.241.86:9999 - ok sync - - - regular 85.209.241.87:9999 - ok sync - - - -regular 85.209.241.88:9999 - ok sync - - - -regular 85.209.241.89:9999 - ok sync - - - regular 85.209.241.91:9999 - ok sync - - - regular 85.209.241.92:9999 - ok sync - - - regular 85.209.241.93:9999 - ok sync - - - @@ -633,7 +623,8 @@ regular 85.209.241.94:9999 - ok sync - - - regular 85.209.241.102:9999 - ok sync - - - regular 85.209.241.103:9999 - ok sync - - - regular 85.209.241.104:9999 - ok sync - - - -regular 85.209.241.105:9999 - ok sync - - - +regular 85.209.241.106:9999 - ok sync - - - +regular 85.209.241.107:9999 - ok sync - - - regular 85.209.241.121:9999 - ok sync - - - regular 85.209.241.127:9999 - ok sync - - - regular 85.209.241.146:9999 - ok sync - - - @@ -670,7 +661,9 @@ regular 85.209.241.225:9999 - ok sync - - - regular 85.209.241.226:9999 - ok sync - - - regular 85.209.241.228:9999 - ok sync - - - regular 85.209.241.229:9999 - ok sync - - - +regular 85.209.241.230:9999 - ok sync - - - regular 85.209.241.231:9999 - ok sync - - - +regular 85.209.241.232:9999 - ok sync - - - regular 85.209.241.233:9999 - ok sync - - - regular 85.209.241.234:9999 - ok sync - - - regular 85.209.241.235:9999 - ok sync - - - @@ -720,17 +713,16 @@ regular 88.99.38.175:9999 - ok sync - - - evo 89.35.131.23:9999 443 ok sync ok ok valid evo 89.35.131.39:9999 443 ok sync ok ok valid regular 89.35.131.46:9999 - ok sync - - - -evo 89.35.131.149:9999 443 ok sync ok ok valid +evo 89.35.131.149:9999 443 ok sync ok ok expired evo 89.35.131.158:9999 443 ok sync ok ok valid evo 89.35.131.218:9999 443 ok sync ok ok valid evo 89.35.131.219:9999 443 ok sync ok ok valid -regular 89.40.15.222:9999 - error ? - - - +regular 89.40.15.222:9999 - ok sync - - - regular 89.46.42.216:9999 - ok sync - - - regular 89.46.42.218:9999 - ok sync - - - -regular 89.47.167.131:9999 - error ? - - - regular 89.117.3.242:9999 - ok sync - - - evo 89.117.57.27:9999 443 ok sync ok ok expired -evo 89.125.50.14:9999 443 ok sync ok ok expired +evo 89.125.50.14:9999 443 ok sync ok ok valid evo 89.125.50.206:9999 443 ok sync ok ok valid evo 89.125.209.27:9999 443 ok sync ok ok valid evo 89.125.209.69:9999 443 ok sync ok ok valid @@ -741,10 +733,9 @@ evo 89.125.209.133:9999 443 ok sync ok ok valid evo 89.125.209.152:9999 443 ok sync ok ok valid evo 89.125.209.156:9999 443 ok sync ok ok valid evo 89.125.209.178:9999 443 ok sync ok ok valid -evo 89.125.209.195:9999 443 ok sync ok ok valid +evo 89.125.209.193:9999 443 ok sync ok ok valid evo 89.125.209.234:9999 443 ok sync ok ok valid regular 89.179.73.96:9999 - ok sync - - - -regular 91.98.160.22:9999 - ok sync - - - regular 91.99.169.186:9999 - ok sync - - - regular 91.184.249.209:9999 - ok sync - - - evo 91.198.108.35:9999 443 ok sync ok ok valid @@ -778,7 +769,6 @@ evo 95.111.242.220:9999 443 ok sync ok ok valid evo 95.179.139.125:9999 443 ok sync ok ok expired regular 95.179.152.242:9999 - ok sync - - - evo 95.179.159.65:9999 443 ok sync ok ok valid -regular 95.179.213.67:9999 - ok sync - - - regular 95.179.240.114:9999 - ok sync - - - evo 95.179.241.182:9999 443 ok sync ok ok valid regular 95.216.79.224:9999 - ok sync - - - @@ -803,6 +793,7 @@ regular 95.216.126.39:9999 - ok sync - - - regular 95.216.126.45:9999 - ok sync - - - regular 95.216.126.46:9999 - ok sync - - - evo 95.216.146.18:9999 443 ok sync ok ok valid +regular 95.216.223.223:9999 - ok sync - - - regular 95.216.230.97:9999 - ok sync - - - regular 95.216.230.98:9999 - ok sync - - - regular 95.216.230.101:9999 - ok sync - - - @@ -854,7 +845,6 @@ regular 104.168.46.54:9999 - ok sync - - - regular 104.207.130.246:9999 - ok sync - - - regular 104.207.130.255:9999 - ok sync - - - regular 104.207.131.17:9999 - error ? - - - -regular 104.234.167.115:9999 - ok sync - - - regular 104.236.58.131:9999 - ok sync - - - regular 104.236.183.29:9999 - ok sync - - - regular 104.238.159.13:9999 - ok sync - - - @@ -898,13 +888,15 @@ evo 109.123.244.131:9999 443 ok sync ok ok valid evo 109.199.97.144:9999 443 ok sync ok ok valid evo 109.199.105.14:9999 443 ok sync ok ok valid regular 109.199.105.203:9999 - ok sync - - - +regular 109.199.106.117:9999 - ok sync - - - evo 109.199.107.155:9999 443 ok sync ok ok valid evo 109.199.108.234:9999 443 ok sync ok ok valid evo 109.199.109.39:9999 443 ok sync ok ok valid +regular 109.199.111.47:9999 - ok sync - - - evo 109.199.120.79:9999 443 ok sync ok ok valid evo 109.199.124.30:9999 443 ok sync ok ok valid evo 109.199.124.135:9999 443 ok sync ok ok valid -regular 109.235.67.246:9999 - error ? - - - +regular 109.235.67.246:9999 - ok sync - - - evo 114.132.172.215:9999 443 ok sync ok ok valid regular 115.190.248.57:9999 - ok sync - - - regular 116.203.244.222:9999 - ok sync - - - @@ -923,7 +915,6 @@ regular 129.212.233.191:9999 - ok sync - - - evo 130.162.233.186:9999 443 ok sync ok ok valid evo 130.185.118.3:9999 443 ok sync ok ok valid regular 130.255.155.163:9999 - ok sync - - - -regular 132.243.197.2:9999 - ok sync - - - regular 132.243.197.3:9999 - ok sync - - - regular 132.243.197.4:9999 - ok sync - - - regular 132.243.197.6:9999 - ok sync - - - @@ -1157,7 +1148,7 @@ regular 135.181.54.104:9999 - ok sync - - - evo 135.181.110.216:9999 443 ok sync ok ok valid regular 135.181.174.222:9999 - ok sync - - - regular 135.181.177.53:9999 - ok sync - - - -evo 135.181.229.49:9999 443 ok sync ok ok valid +evo 135.181.229.49:9999 443 ok sync ok ok expired regular 136.243.29.192:9999 - ok sync - - - regular 136.243.29.193:9999 - ok sync - - - regular 136.243.29.194:9999 - ok sync - - - @@ -1188,6 +1179,7 @@ regular 136.243.115.142:9999 - ok sync - - - regular 136.243.142.32:9999 - ok sync - - - regular 136.243.142.33:9999 - ok sync - - - regular 136.243.142.36:9999 - ok sync - - - +regular 136.243.142.37:9999 - ok sync - - - regular 136.243.142.41:9999 - ok sync - - - evo 136.244.99.17:9999 443 ok sync ok ok valid regular 136.244.103.110:9999 - ok sync - - - @@ -1201,8 +1193,10 @@ regular 138.68.155.23:9999 - ok sync - - - regular 138.68.157.15:9999 - ok sync - - - regular 138.197.128.77:9999 - ok sync - - - regular 138.197.131.126:9999 - ok sync - - - +regular 138.197.141.70:9999 - ok sync - - - regular 138.197.147.28:9999 - ok sync - - - regular 138.197.161.208:9999 - ok sync - - - +regular 139.28.97.2:9999 - ok sync - - - regular 139.59.0.167:9999 - ok sync - - - regular 139.59.3.197:9999 - ok sync - - - regular 139.59.4.172:9999 - ok sync - - - @@ -1273,9 +1267,8 @@ regular 146.185.182.189:9999 - ok sync - - - regular 146.190.33.135:9999 - ok sync - - - regular 147.45.41.210:9999 - ok sync - - - regular 147.182.146.51:9999 - ok sync - - - -regular 147.182.146.128:9999 - ok sync - - - +regular 147.182.157.129:9999 - ok sync - - - regular 148.113.202.169:9999 - ok sync - - - -regular 148.113.209.97:9999 - ok sync - - - regular 148.251.73.224:9999 - ok sync - - - regular 148.251.73.232:9999 - ok sync - - - regular 148.251.73.234:9999 - ok sync - - - @@ -1291,7 +1284,6 @@ evo 149.28.241.190:9999 443 ok sync ok ok expired evo 149.28.247.165:9999 443 ok sync ok ok expired evo 149.102.140.101:9999 443 ok sync ok ok valid regular 149.104.110.119:9999 - ok sync - - - -regular 149.248.3.156:9999 - ok sync - - - regular 149.248.53.48:9999 - ok sync - - - regular 151.244.85.0:9999 - ok sync - - - regular 151.244.85.1:9999 - ok sync - - - @@ -1300,7 +1292,9 @@ regular 151.244.85.7:9999 - ok sync - - - regular 151.244.85.8:9999 - ok sync - - - regular 151.244.85.9:9999 - ok sync - - - regular 151.244.85.15:9999 - ok sync - - - +regular 151.244.85.25:9999 - ok sync - - - regular 151.244.85.31:9999 - ok sync - - - +regular 151.244.85.34:9999 - ok sync - - - regular 151.244.85.36:9999 - ok sync - - - regular 151.244.85.43:9999 - ok sync - - - regular 151.244.85.45:9999 - ok sync - - - @@ -1319,6 +1313,7 @@ regular 151.244.85.82:9999 - ok sync - - - regular 151.244.85.88:9999 - ok sync - - - regular 151.244.85.96:9999 - ok sync - - - regular 151.244.85.98:9999 - ok sync - - - +regular 151.244.85.106:9999 - ok sync - - - regular 151.244.85.109:9999 - ok sync - - - regular 151.244.85.110:9999 - ok sync - - - regular 151.244.85.111:9999 - ok sync - - - @@ -1353,6 +1348,7 @@ regular 151.244.85.220:9999 - ok sync - - - regular 151.244.85.228:9999 - ok sync - - - regular 151.244.85.229:9999 - ok sync - - - regular 151.244.85.236:9999 - ok sync - - - +regular 151.244.85.239:9999 - ok sync - - - regular 151.244.85.243:9999 - ok sync - - - regular 151.244.85.244:9999 - ok sync - - - regular 151.244.85.245:9999 - ok sync - - - @@ -1430,10 +1426,7 @@ evo 161.97.91.68:9999 443 ok sync ok ok valid evo 161.97.91.217:9999 443 ok sync ok ok valid evo 161.97.96.120:9999 443 ok sync ok ok valid evo 161.97.102.156:9999 443 ok sync ok ok valid -evo 161.97.104.37:9999 443 ok sync ok ok valid evo 161.97.106.14:9999 443 ok sync ok ok valid -evo 161.97.106.164:9999 443 ok sync ok ok valid -evo 161.97.117.125:9999 443 ok sync ok ok valid evo 161.97.120.156:9999 443 ok sync ok ok valid evo 161.97.142.210:9999 443 ok sync ok ok valid evo 161.97.153.122:9999 443 ok sync ok ok valid @@ -1444,7 +1437,6 @@ evo 161.97.166.49:9999 443 ok sync ok ok valid evo 161.97.173.67:9999 443 ok sync ok ok valid evo 161.97.175.233:9999 443 ok sync ok ok valid evo 161.97.176.38:9999 443 ok sync ok ok valid -evo 161.97.180.105:9999 443 ok sync ok ok valid evo 161.97.180.182:9999 443 ok sync ok ok valid regular 162.212.35.99:9999 - ok sync - - - evo 162.212.35.100:9999 443 ok sync ok ok valid @@ -1463,7 +1455,6 @@ regular 162.243.136.66:9999 - ok sync - - - regular 162.243.205.212:9999 - ok sync - - - regular 162.243.219.25:9999 - ok sync - - - regular 163.172.66.41:9999 - ok sync - - - -evo 164.68.114.36:9999 443 ok sync ok ok valid evo 164.68.118.37:9999 443 ok sync ok ok valid regular 164.90.221.233:9999 - ok sync - - - regular 164.92.160.130:9999 - ok sync - - - @@ -1477,7 +1468,6 @@ regular 165.22.62.245:9999 - ok sync - - - regular 165.22.65.68:9999 - ok sync - - - regular 165.22.194.80:9999 - ok sync - - - regular 165.22.208.146:9999 - ok sync - - - -regular 165.22.221.97:9999 - ok sync - - - regular 165.22.227.208:9999 - ok sync - - - regular 165.22.234.135:9999 - ok sync - - - regular 165.22.236.90:9999 - ok sync - - - @@ -1511,7 +1501,6 @@ regular 167.99.182.250:9999 - ok sync - - - regular 167.99.185.82:9999 - ok sync - - - regular 167.99.199.59:9999 - ok sync - - - regular 167.99.205.145:9999 - ok sync - - - -evo 167.104.216.28:9999 443 ok sync ok ok valid regular 167.114.185.96:9999 - ok sync - - - regular 167.172.65.155:9999 - ok sync - - - regular 167.172.67.55:9999 - ok sync - - - @@ -1520,6 +1509,7 @@ regular 167.172.87.128:9999 - ok sync - - - regular 167.172.94.148:9999 - ok sync - - - regular 167.172.183.190:9999 - ok sync - - - evo 167.233.98.163:9999 443 ok sync ok ok valid +evo 167.235.102.194:9999 443 ok sync ok ok valid regular 167.235.132.126:9999 - ok sync - - - regular 168.119.57.12:9999 - ok sync - - - regular 168.119.80.0:9999 - ok sync - - - @@ -1652,6 +1642,7 @@ regular 178.62.186.39:9999 - ok sync - - - regular 178.62.192.42:9999 - ok sync - - - regular 178.62.196.119:9999 - ok sync - - - regular 178.62.200.153:9999 - ok sync - - - +regular 178.62.204.56:9999 - ok sync - - - regular 178.62.207.128:9999 - ok sync - - - regular 178.62.215.116:9999 - ok sync - - - regular 178.62.220.199:9999 - ok sync - - - @@ -1804,6 +1795,7 @@ regular 188.40.163.27:9999 - ok sync - - - regular 188.40.163.28:9999 - ok sync - - - regular 188.40.163.30:9999 - ok sync - - - regular 188.40.163.31:9999 - ok sync - - - +regular 188.40.169.35:9999 - ok sync - - - regular 188.40.169.42:9999 - ok sync - - - regular 188.40.169.44:9999 - ok sync - - - regular 188.40.175.65:9999 - ok sync - - - @@ -1979,11 +1971,10 @@ regular 193.160.119.129:9999 - ok sync - - - regular 194.5.157.214:9999 - ok sync - - - regular 194.58.47.132:9999 - ok sync - - - regular 194.58.47.228:9999 - ok sync - - - -regular 194.135.80.33:9999 - error ? - - - -regular 194.135.84.100:9999 - error ? - - - -regular 194.135.94.228:9999 - error ? - - - +regular 194.135.80.33:9999 - ok sync - - - +regular 194.135.84.100:9999 - ok sync - - - +regular 194.135.94.228:9999 - ok sync - - - evo 194.163.166.76:9999 443 ok sync ok ok valid -evo 194.163.172.206:9999 443 ok sync ok ok valid regular 194.163.183.132:9999 - ok sync - - - regular 194.238.25.199:9999 - ok sync - - - evo 195.26.254.228:9999 443 ok sync ok ok valid @@ -2042,8 +2033,8 @@ regular 209.38.226.76:9999 - ok sync - - - regular 209.97.142.65:9999 - ok sync - - - regular 209.141.36.206:9999 - ok sync - - - regular 209.250.240.73:9999 - ok sync - - - -regular 212.24.104.225:9999 - error ? - - - -regular 212.24.109.253:9999 - error ? - - - +regular 212.24.104.225:9999 - ok sync - - - +regular 212.24.109.253:9999 - ok sync - - - evo 212.132.105.190:9999 443 ok sync ok ok valid regular 213.136.91.123:9999 - ok sync - - - evo 213.199.44.112:9999 443 ok sync ok ok valid @@ -2070,4 +2061,4 @@ regular 216.238.85.238:9999 - ok sync - - - evo 216.238.99.9:9999 443 ok sync ok ok valid regular 217.69.5.141:9999 - ok sync - - - evo 217.76.54.175:9999 443 ok sync ok ok valid -regular 217.154.231.20:9999 - error ? - - - +regular 217.154.231.20:9999 - ok sync - - - diff --git a/dash-network-seeds/seeds/testnet.txt b/dash-network-seeds/seeds/testnet.txt index 976fc979f..259fa1eff 100644 --- a/dash-network-seeds/seeds/testnet.txt +++ b/dash-network-seeds/seeds/testnet.txt @@ -1,13 +1,14 @@ -# Auto-generated by masternode-seeds-fetcher on 2026-06-22T08:25:06Z for testnet +# Auto-generated by masternode-seeds-fetcher on 2026-07-13T07:19:27Z for testnet # Source: Dash P2P network (mnlistdiff) -# Primary peer: 68.67.122.37:19999 -# Tip block hash: 00000016d1295feff912992e68e83b864fcfa5ec90ed178cd3bb2b5168aba790 -# Tip block height: 1500630 -# 84 seeds (54 regular + 30 evo) of 540 total masternodes, valid-only -# Probe summary: core_reachable=84/84 platform_reachable=30/30 ssl_valid=30/30 +# Primary peer: 68.67.122.48:19999 +# Tip block hash: 00000077d1156178d7977a520b67efb521ee2c66a08c80d8118cf25d6f081a50 +# Tip block height: 1513736 +# 85 seeds (55 regular + 30 evo) of 375 total masternodes, valid-only +# Probe summary: core_reachable=85/85 platform_reachable=30/30 ssl_valid=30/30 # Columns: # Values: core_reach/plat_reach=ok|timeout|refused|error|?, core_sync=sync|-N|+N|?, plat_live=ok|none|?|-, plat_ssl=valid|expired|self-signed|untrusted|no-handshake|?|- # Do not edit manually — refreshed weekly by .github/workflows/update-masternode-seeds.yml +regular 62.60.244.174:19999 - ok sync - - - evo 68.67.122.1:19999 1443 ok sync ok ok valid evo 68.67.122.2:19999 1443 ok sync ok ok valid evo 68.67.122.3:19999 1443 ok sync ok ok valid From 0ae8df445b992dd2d0b9bb8e5f5f41031298ab1d Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Mon, 13 Jul 2026 23:22:48 +1000 Subject: [PATCH 044/149] feat(dash-spv): floor mainnet sync at HD/BIP39 activation height (#877) Floor mainnet sync at block 227121, the height from which HD/BIP39 wallets could first exist on chain. No address any HD wallet derives can appear before it, so syncing the pre-HD-wallet chain is wasted work. This matches the entry in DashSync's `mainnet_checkpoint_array` commented "first sync time (aka BIP39 creation time)", the value the official Dash mobile SPV client uses. Add `MAINNET_HD_ACTIVATION_HEIGHT` and a `NetworkExt::hd_wallet_sync_floor()` method to the `dash` crate, alongside `known_genesis_block_hash` where the other per-network chain constants live (mainnet returns the floor, other networks return `0`, meaning sync from genesis). When the caller sets no explicit `start_from_height`, `DashSpvClient::new` floors the wallet-derived birth height via `birth_height.max(config.network.hd_wallet_sync_floor())`, so a low or zero birth height can no longer drag mainnet sync back to genesis. Anchoring reuses the existing checkpoint machinery and snaps to the nearest checkpoint at or before the floor (the 200000 checkpoint), a safe undershoot that never skips a block that could hold an HD wallet transaction. An explicit `start_from_height` is still honored downward, and testnet/regtest/devnet are unaffected. --- dash-spv/src/client/lifecycle.rs | 6 +++++- dash-spv/src/client/mod.rs | 24 +++++++++++++++--------- dash/src/network/constants.rs | 31 ++++++++++++++++++++++++++++++- 3 files changed, 50 insertions(+), 11 deletions(-) diff --git a/dash-spv/src/client/lifecycle.rs b/dash-spv/src/client/lifecycle.rs index 2f789bdf8..4ce3d0c47 100644 --- a/dash-spv/src/client/lifecycle.rs +++ b/dash-spv/src/client/lifecycle.rs @@ -49,11 +49,15 @@ impl DashSpvClient Some(height), None => { let birth_height = wallet.read().await.earliest_required_height().await; - (birth_height > 0).then_some(birth_height) + let start = birth_height.max(config.network.hd_wallet_sync_floor()); + (start > 0).then_some(start) } }; diff --git a/dash-spv/src/client/mod.rs b/dash-spv/src/client/mod.rs index 541a71794..6a7c8bda5 100644 --- a/dash-spv/src/client/mod.rs +++ b/dash-spv/src/client/mod.rs @@ -92,20 +92,26 @@ mod tests { #[tokio::test] async fn birth_height_anchors_chain_to_nearest_checkpoint() { - // A wallet born at 120_000 anchors at the 100_000 checkpoint, not genesis, and - // the stored tip carries the trusted checkpoint hash (what the next header's - // `prev_blockhash` is validated against), not a hash recomputed from a header. + // A wallet birth height below mainnet's HD/BIP39 activation floor is clamped up to + // the floor, which anchors at the nearest checkpoint at or before it: the 200_000 + // checkpoint. The stored tip carries the trusted checkpoint hash (what the next + // header's `prev_blockhash` is validated against), not a hash recomputed from a header. + // 120_000 is below mainnet's HD/BIP39 sync floor, so it is clamped up. let (height, hash) = anchored_tip(120_000, None).await; - assert_eq!(height, 100_000); + assert_eq!(height, 200_000); let expected: dashcore::BlockHash = - "00000000000fd08c2fb661d2fcb0d49abb3a91e5f27082ce64feed3b4dede2e2".parse().unwrap(); + "000000000004d0615ff622ec78457ca211dc63fc9c62cca9d9d9af7206be721b".parse().unwrap(); assert_eq!(hash, Some(expected)); - // Birth height 0 keeps syncing from genesis. - assert_eq!(anchored_tip(0, None).await.0, 0); + // Birth height 0 no longer drags mainnet sync to genesis; it floors at HD activation. + assert_eq!(anchored_tip(0, None).await.0, 200_000); - // Explicit `start_from_height` wins over the wallet birth height, even when - // the birth height would resolve to a higher checkpoint. + // A birth height above the floor is unaffected and anchors at the nearest + // checkpoint at or below it. + assert_eq!(anchored_tip(560_000, None).await.0, 550_000); + + // Explicit `start_from_height` wins over the wallet birth height and is honored + // even below the floor. assert_eq!(anchored_tip(120_000, Some(60_000)).await.0, 50_000); } diff --git a/dash/src/network/constants.rs b/dash/src/network/constants.rs index 6fc4268bc..57c8cddec 100644 --- a/dash/src/network/constants.rs +++ b/dash/src/network/constants.rs @@ -50,9 +50,22 @@ pub const NODE_HEADERS_COMPRESSED: ServiceFlags = ServiceFlags::NODE_HEADERS_COM /// 60001 - Support `pong` message and nonce in `ping` message pub const PROTOCOL_VERSION: u32 = 70237; +/// Mainnet height from which Dash HD/BIP39 wallets could first exist on chain. +/// +/// No address any HD wallet derives can appear earlier, so an SPV client scanning for HD +/// wallet activity has no reason to sync mainnet below this height. Matches the entry in +/// DashSync's `mainnet_checkpoint_array` commented "first sync time (aka BIP39 creation +/// time)", the value the official Dash mobile SPV client uses as its floor. +pub const MAINNET_HD_ACTIVATION_HEIGHT: u32 = 227121; + pub trait NetworkExt { /// Returns the known genesis block hash for `network`, if one is hardcoded. fn known_genesis_block_hash(&self) -> Option; + + /// Earliest block height worth syncing for HD/BIP39 wallets, or `0` when the network + /// has no floor (sync from genesis). Mainnet is floored at + /// [`MAINNET_HD_ACTIVATION_HEIGHT`]; other networks return `0`. + fn hd_wallet_sync_floor(&self) -> u32; } impl NetworkExt for Network { @@ -88,6 +101,13 @@ impl NetworkExt for Network { } } } + + fn hd_wallet_sync_floor(&self) -> u32 { + match self { + Network::Mainnet => MAINNET_HD_ACTIVATION_HEIGHT, + _ => 0, + } + } } /// Flags to indicate which network services a node supports. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] @@ -265,10 +285,19 @@ impl Decodable for ServiceFlags { #[cfg(test)] mod tests { - use super::ServiceFlags; + use super::{MAINNET_HD_ACTIVATION_HEIGHT, NetworkExt, ServiceFlags}; use crate::Network; use crate::consensus::encode::{deserialize, serialize}; + #[test] + fn test_hd_wallet_sync_floor() { + // Only mainnet is floored; every other network syncs from genesis (0). + assert_eq!(Network::Mainnet.hd_wallet_sync_floor(), MAINNET_HD_ACTIVATION_HEIGHT); + assert_eq!(Network::Testnet.hd_wallet_sync_floor(), 0); + assert_eq!(Network::Devnet.hd_wallet_sync_floor(), 0); + assert_eq!(Network::Regtest.hd_wallet_sync_floor(), 0); + } + #[test] fn test_network_magic() { assert_eq!(Network::Mainnet.magic(), 0xBD6B0CBF); From 56a407883fc49a08cdb3b72845fdfc77983af92b Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 05:07:27 +0700 Subject: [PATCH 045/149] fix(key-wallet): derive BLS operator & Ed25519 platform-node keys per DashSync/dashbls (#879) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(key-wallet): derive BLS operator & Ed25519 platform-node keys per DashSync/dashbls Fixes three independent bugs that made masternode operator (BLS) and platform node (Ed25519) keys diverge from DashSync/dashwallet-ios for the same mnemonic (#878): 1. Hardened BLS child derivation prepended a spurious 0x00 to the HMAC input (secp256k1 BIP32 convention). dashbls uses sk(32) || i(4) || {0,1} with no leading zero, so every hardened level diverged. 2. Non-hardened BLS derivation fed the parent public key into the HMAC in modern/IETF serialization. dashbls serializes it with the legacy flag (fLegacy=true) throughout the HD chain. ExtendedBLSPrivKey/ ExtendedBLSPubKey now use legacy bytes in the HMAC input and expose public_key_bytes_legacy()/to_bytes_legacy(). 3. The default wallet flow derived a secp256k1 BIP32 key at the provider path, reused its secret bytes as the BLS/SLIP-0010 master seed, and applied the provider path again on top. BLSAccount::from_seed / EdDSAAccount::from_seed now take the wallet seed (e.g. the 64-byte BIP39 seed), feed it directly into the target-curve master and apply the account path once; add_bls_account/add_eddsa_account use the wallet's own seed. Seedless wallets (extended-priv-key based, watch- only, external-signable) skip auto-creation of these accounts and require an explicit seed. Also fixes BLSAccount::has_internal_and_external() to false — operator keys live in a single pool (account/i), so seed-based signing derivation (derive_from_seed_private_key_at) now works and yields m/9'/coin'/3'/3'/i. Derivation is pinned by test vectors generated with dashbls (dashpay/bls-signatures @ 0842b17, ExtendedPrivateKey with fLegacy=true) from the standard BIP39 test mnemonic, and SLIP-0010 vectors verified with an independent implementation. Keys previously persisted from these accounts change after this fix — the old ones never corresponded to anything on-chain. Co-Authored-By: Claude Fable 5 * refactor(key-wallet): parameterize BLS HD serialization mode (modern default, legacy explicit) Mirror dashbls's fLegacy parameter instead of hard-coding legacy in the HD chain: derive_priv/derive_pub/derive_path use the modern (IETF) serialization like dashbls PrivateChild/PublicChild with the default fLegacy=false, and *_legacy / *_with_mode variants select the legacy Dash serialization. The provider-key account layer (BLSAccount, BLS address pools) derives with the legacy variants, which is what dashbls/DashSync use for masternode operator keys. Hardened derivation never serializes the public key, so both modes agree there; only non-hardened children differ. Modern-mode derivation is pinned by dashbls-generated vectors alongside the legacy ones. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- .../src/account_derivation_tests.rs | 80 ++++ key-wallet/examples/account_types.rs | 8 +- .../src/account/account_collection_test.rs | 8 +- key-wallet/src/account/bls_account.rs | 43 +- key-wallet/src/account/eddsa_account.rs | 21 +- key-wallet/src/derivation_bls_bip32.rs | 399 +++++++++++++++++- .../src/managed_account/address_pool.rs | 6 +- key-wallet/src/tests/mod.rs | 2 + .../tests/provider_key_derivation_tests.rs | 162 +++++++ key-wallet/src/wallet/accounts.rs | 78 ++-- key-wallet/src/wallet/helper.rs | 38 +- 11 files changed, 746 insertions(+), 99 deletions(-) create mode 100644 key-wallet/src/tests/provider_key_derivation_tests.rs diff --git a/key-wallet-ffi/src/account_derivation_tests.rs b/key-wallet-ffi/src/account_derivation_tests.rs index 688560d3a..e0cd2b243 100644 --- a/key-wallet-ffi/src/account_derivation_tests.rs +++ b/key-wallet-ffi/src/account_derivation_tests.rs @@ -246,4 +246,84 @@ mod tests { wallet::wallet_free(wallet); } } + + /// Provider operator (BLS) and platform node (Ed25519) keys derived from a + /// seed through the FFI must match the DashSync/dashbls reference vectors + /// (see rust-dashcore issue #878 and + /// key-wallet/src/tests/provider_key_derivation_tests.rs). + #[test] + #[cfg(all(feature = "bls", feature = "eddsa"))] + fn test_provider_key_derivation_matches_dashsync_reference() { + let mut error = FFIError::default(); + + let mnemonic = std::ffi::CString::new(MNEMONIC).unwrap(); + let passphrase = std::ffi::CString::new("").unwrap(); + + let wallet = unsafe { + wallet::wallet_create_from_mnemonic(mnemonic.as_ptr(), FFINetwork::Mainnet, &mut error) + }; + assert!(!wallet.is_null()); + + let mut seed = [0u8; 64]; + let ok = unsafe { + crate::mnemonic::mnemonic_to_seed( + mnemonic.as_ptr(), + passphrase.as_ptr(), + seed.as_mut_ptr(), + &mut (seed.len()), + &mut error, + ) + }; + assert!(ok); + + let collection = + unsafe { crate::account_collection::wallet_get_account_collection(wallet, &mut error) }; + assert!(!collection.is_null()); + + unsafe { + // BLS operator key 0 at m/9'/5'/3'/3'/0 (legacy HD chain). + let operator_account = + crate::account_collection::account_collection_get_provider_operator_keys(collection) + as *mut crate::account::FFIBLSAccount; + assert!(!operator_account.is_null()); + + let sk0_hex = super::super::bls_account_derive_private_key_from_seed( + operator_account, + seed.as_ptr(), + seed.len(), + 0, + &mut error, + ); + assert!(!sk0_hex.is_null(), "BLS derivation failed: {:?}", error.code); + let sk0 = std::ffi::CStr::from_ptr(sk0_hex).to_str().unwrap(); + assert_eq!(sk0, "11122e1ad656d0610ce0f80d40da874d67ea656a3e66ed371c915ec3a488a43a"); + crate::utils::string_free(sk0_hex); + crate::account::bls_account_free(operator_account); + + // Ed25519 platform node key 0 at m/9'/5'/3'/4'/0' (SLIP-0010). + let platform_account = + crate::account_collection::account_collection_get_provider_platform_keys(collection) + as *mut crate::account::FFIEdDSAAccount; + assert!(!platform_account.is_null()); + + let node_sk0_hex = super::super::eddsa_account_derive_private_key_from_seed( + platform_account, + seed.as_ptr(), + seed.len(), + 0, + &mut error, + ); + assert!(!node_sk0_hex.is_null(), "Ed25519 derivation failed: {:?}", error.code); + let node_sk0 = std::ffi::CStr::from_ptr(node_sk0_hex).to_str().unwrap(); + assert_eq!( + node_sk0, + "5fa238b12be77347abf9b5957bd902d16c6aaca28d25c4267ffacbd7458dceb1" + ); + crate::utils::string_free(node_sk0_hex); + crate::account::eddsa_account_free(platform_account); + + crate::account_collection::account_collection_free(collection); + wallet::wallet_free(wallet); + } + } } diff --git a/key-wallet/examples/account_types.rs b/key-wallet/examples/account_types.rs index 5220624e9..7ecc65b08 100644 --- a/key-wallet/examples/account_types.rs +++ b/key-wallet/examples/account_types.rs @@ -68,7 +68,7 @@ fn main() -> Result<(), Box> { let bls_account = BLSAccount::from_seed( None, AccountType::ProviderVotingKeys, - bls_seed, + &bls_seed, Network::Testnet, )?; @@ -110,7 +110,7 @@ fn main() -> Result<(), Box> { let eddsa_account = EdDSAAccount::from_seed( None, AccountType::IdentityRegistration, - ed25519_seed, + &ed25519_seed, Network::Testnet, )?; @@ -164,7 +164,7 @@ fn main() -> Result<(), Box> { let bls_account = BLSAccount::from_seed( None, AccountType::ProviderVotingKeys, - bls_seed, + &bls_seed, Network::Testnet, )?; let watch_only_bls = bls_account.to_watch_only(); @@ -177,7 +177,7 @@ fn main() -> Result<(), Box> { let eddsa_account = EdDSAAccount::from_seed( None, AccountType::IdentityRegistration, - ed25519_seed, + &ed25519_seed, Network::Testnet, )?; let watch_only_eddsa = eddsa_account.to_watch_only(); diff --git a/key-wallet/src/account/account_collection_test.rs b/key-wallet/src/account/account_collection_test.rs index dead953e0..e6ee1eabc 100644 --- a/key-wallet/src/account/account_collection_test.rs +++ b/key-wallet/src/account/account_collection_test.rs @@ -58,7 +58,7 @@ mod tests { let bls_account = BLSAccount::from_seed( None, AccountType::ProviderOperatorKeys, - [42u8; 32], + &[42u8; 32], Network::Testnet, ) .unwrap(); @@ -70,7 +70,7 @@ mod tests { let eddsa_account = EdDSAAccount::from_seed( None, AccountType::ProviderPlatformKeys, - [99u8; 32], + &[99u8; 32], Network::Testnet, ) .unwrap(); @@ -118,7 +118,7 @@ mod tests { let bls_account = BLSAccount::from_seed( None, AccountType::ProviderVotingKeys, // Wrong! Should be ProviderOperatorKeys - [42u8; 32], + &[42u8; 32], Network::Testnet, ) .unwrap(); @@ -136,7 +136,7 @@ mod tests { let eddsa_account = EdDSAAccount::from_seed( None, AccountType::IdentityRegistration, // Wrong! Should be ProviderPlatformKeys - [99u8; 32], + &[99u8; 32], Network::Testnet, ) .unwrap(); diff --git a/key-wallet/src/account/bls_account.rs b/key-wallet/src/account/bls_account.rs index 59324c0be..5f8601eb5 100644 --- a/key-wallet/src/account/bls_account.rs +++ b/key-wallet/src/account/bls_account.rs @@ -109,15 +109,23 @@ impl BLSAccount { }) } - /// Create a BLS account from raw private key bytes (seed) + /// Create a BLS account from a wallet seed (e.g. the 64-byte BIP39 seed). + /// + /// The seed is fed directly into the BLS HD master key (dashbls + /// `ExtendedPrivateKey::FromSeed`) and the account's derivation path + /// (e.g. `m/9'/5'/3'/3'` for mainnet operator keys) is applied in the BLS + /// scheme, matching DashSync. The stored extended public key is the + /// account-level key, so key `i` is its `i`th child. pub fn from_seed( parent_wallet_id: Option>, account_type: AccountType, - seed: [u8; 32], + seed: &[u8], network: Network, ) -> Result { - let bls_private_key = ExtendedBLSPrivKey::new_master(network, &seed)?; - let bls_public_key = ExtendedBLSPubKey::from_private_key(&bls_private_key); + let master = ExtendedBLSPrivKey::new_master(network, seed)?; + let path = account_type.derivation_path(network)?; + let account_xpriv = master.derive_path_legacy(&path)?; + let bls_public_key = ExtendedBLSPubKey::from_private_key(&account_xpriv); Ok(Self { parent_wallet_id, @@ -139,7 +147,7 @@ impl BLSAccount { return Err(Error::WatchOnly); } let child_num = ChildNumber::from_normal_idx(index)?; - current_key = current_key.ckd_pub(child_num)?; + current_key = current_key.derive_pub_legacy(child_num)?; } Ok(current_key) @@ -237,7 +245,11 @@ impl } fn has_internal_and_external(&self) -> bool { - true + // Provider operator keys live in a single pool (`account/i`, matching + // DashSync) — there are no separate external/internal chains. This also + // keeps the chain-agnostic seed derivation helpers usable, so operator + // key `i` derived from the seed is `m/9'/coin'/3'/3'/i`. + false } fn has_intermediate_derivation(&self) -> Option { @@ -266,9 +278,10 @@ impl // Get the derivation path for this account type let path = self.account_type.derivation_path(self.network)?; - // Derive the account private key from master + // Derive the account private key from master (legacy mode, matching + // dashbls/DashSync for provider operator keys) master_xpriv - .derive_path(&path) + .derive_path_legacy(&path) .map_err(|e| Error::InvalidParameter(format!("BLS derivation error: {}", e))) } @@ -284,9 +297,9 @@ impl return Err(Error::WatchOnly); } - // Derive the child private key from account private key + // Derive the child private key from account private key (legacy mode) account_xpriv - .derive_path(child_path) + .derive_path_legacy(child_path) .map_err(|e| Error::InvalidParameter(format!("BLS child derivation error: {}", e))) } @@ -303,9 +316,9 @@ impl } } - // Derive the child public key from account public key + // Derive the child public key from account public key (legacy mode) self.bls_public_key - .derive_path(child_path) + .derive_path_legacy(child_path) .map_err(|e| Error::InvalidParameter(format!("BLS public key derivation error: {}", e))) } @@ -456,7 +469,7 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, - seed, + &seed, Network::Testnet, ) .expect("Failed to create BLS account from seed"); @@ -473,7 +486,7 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, - seed, + &seed, Network::Testnet, ) .expect("Failed to create BLS account from seed"); @@ -492,7 +505,7 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, - seed, + &seed, Network::Testnet, ) .expect("Failed to create BLS account from seed"); diff --git a/key-wallet/src/account/eddsa_account.rs b/key-wallet/src/account/eddsa_account.rs index 497fd4c37..b9c3a5cd1 100644 --- a/key-wallet/src/account/eddsa_account.rs +++ b/key-wallet/src/account/eddsa_account.rs @@ -84,15 +84,22 @@ impl EdDSAAccount { }) } - /// Create an EdDSA account from a private key (seed) + /// Create an EdDSA account from a wallet seed (e.g. the 64-byte BIP39 seed). + /// + /// The seed is fed directly into the SLIP-0010 Ed25519 master key and the + /// account's derivation path (e.g. `m/9'/5'/3'/4'` for mainnet platform + /// node keys) is applied in the Ed25519 scheme, matching DashSync. The + /// stored extended public key is the account-level key. pub fn from_seed( parent_wallet_id: Option>, account_type: AccountType, - ed25519_seed: [u8; 32], + seed: &[u8], network: Network, ) -> Result { - let ed25519_private_key = ExtendedEd25519PrivKey::new_master(network, &ed25519_seed)?; - let ed25519_public_key = ExtendedEd25519PubKey::from_priv(&ed25519_private_key)?; + let master = ExtendedEd25519PrivKey::new_master(network, seed)?; + let path = account_type.derivation_path(network)?; + let account_xpriv = master.derive_priv(&path)?; + let ed25519_public_key = ExtendedEd25519PubKey::from_priv(&account_xpriv)?; Ok(Self { parent_wallet_id, @@ -454,7 +461,7 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, - seed, + &seed, Network::Testnet, ) .expect("Failed to create EdDSA account from seed"); @@ -471,7 +478,7 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, - seed, + &seed, Network::Testnet, ) .expect("Failed to create EdDSA account from seed"); @@ -516,7 +523,7 @@ mod tests { index: 0, standard_account_type: StandardAccountType::BIP44Account, }, - seed, + &seed, Network::Testnet, ) .expect("Failed to create EdDSA account from seed"); diff --git a/key-wallet/src/derivation_bls_bip32.rs b/key-wallet/src/derivation_bls_bip32.rs index ec85f0333..44e6fc5f7 100644 --- a/key-wallet/src/derivation_bls_bip32.rs +++ b/key-wallet/src/derivation_bls_bip32.rs @@ -1,20 +1,44 @@ //! BIP32-like implementation for BLS12-381. //! //! Implementation of hierarchical deterministic wallets for BLS12-381, -//! inspired by BIP32 and adapted for BLS signatures. +//! matching the dashbls (`bls-signatures`) `ExtendedPrivateKey` / +//! `ExtendedPublicKey` scheme used by Dash Core and DashSync for masternode +//! operator keys (DIP-3 `m/9'/coin'/3'/3'`). //! //! Key differences from standard BIP32: //! - Uses BLS12-381 curve instead of secp256k1 //! - Keys are 32 bytes (private) and 48 bytes (public) -//! - Uses "BLS12381 seed" as the HMAC key for master key generation +//! - Uses "BLS HD seed" as the HMAC key for master key generation //! - Supports both hardened and non-hardened derivation +//! - Hardened child HMAC input is `sk(32) || index(4 BE) || {0,1}` — unlike +//! secp256k1 BIP32 there is **no** leading `0x00` byte +//! +//! # Serialization modes +//! +//! Non-hardened derivation feeds the parent public key into the HMAC, so the +//! G1 serialization format is part of the derivation itself — dashbls +//! parameterizes it as `fLegacy`. Both modes are supported, mirroring dashbls: +//! +//! - [`ExtendedBLSPrivKey::derive_priv`] / [`ExtendedBLSPubKey::derive_pub`] +//! use the **modern** (IETF/basic-scheme) serialization, like dashbls +//! `PrivateChild(i)` with the default `fLegacy = false`. +//! - [`ExtendedBLSPrivKey::derive_priv_legacy`] / +//! [`ExtendedBLSPubKey::derive_pub_legacy`] use the **legacy** Dash +//! serialization (`fLegacy = true`). This is what dashbls/DashSync use for +//! masternode operator keys (DIP-3 `m/9'/coin'/3'/3'`), so the provider-key +//! account layer derives with these. +//! - The `*_with_mode` variants take an explicit [`SerializationFormat`]. +//! +//! Hardened derivation never serializes the public key, so the mode only +//! matters for non-hardened children. Output serialization is likewise +//! available in both formats ([`ExtendedBLSPubKey::to_bytes`] / +//! [`ExtendedBLSPubKey::to_bytes_legacy`]). use core::fmt; use dashcore_hashes::{sha256, Hash, HashEngine, Hmac, HmacEngine}; use std::error; // NOTE: We use Bls12381G2Impl for BLS keys (48-byte public keys) -#[cfg(any(feature = "serde", feature = "bincode"))] use dashcore::blsful::SerializationFormat; use dashcore::blsful::{Bls12381G2Impl, PublicKey as BlsPublicKey, SecretKey as BlsSecretKey}; @@ -168,19 +192,47 @@ impl ExtendedBLSPrivKey { }) } - /// Derive a child private key + /// Derive a child private key using the modern (IETF) public key + /// serialization for non-hardened children. + /// + /// Equivalent to dashbls `PrivateChild(i)` with the default + /// `fLegacy = false`. For Dash masternode operator keys use + /// [`Self::derive_priv_legacy`], which matches DashSync. pub fn derive_priv(&self, child: ChildNumber) -> Result { - // Build the input data for HMAC + self.derive_priv_with_mode(child, SerializationFormat::Modern) + } + + /// Derive a child private key using the legacy Dash public key + /// serialization for non-hardened children. + /// + /// Equivalent to dashbls `PrivateChild(i, fLegacy = true)` — the mode + /// dashbls/DashSync use for masternode operator keys. + pub fn derive_priv_legacy(&self, child: ChildNumber) -> Result { + self.derive_priv_with_mode(child, SerializationFormat::Legacy) + } + + /// Derive a child private key with an explicit serialization mode. + /// + /// The mode selects the G1 serialization of the parent public key in the + /// HMAC input for non-hardened children; hardened derivation never + /// serializes the public key, so both modes agree there. + pub fn derive_priv_with_mode( + &self, + child: ChildNumber, + format: SerializationFormat, + ) -> Result { + // Build the input data for HMAC, following dashbls + // `ExtendedPrivateKey::PrivateChild` (extendedprivatekey.cpp) let mut input_data = Vec::new(); if child.is_hardened() { - // Hardened derivation: 0x00 || private_key || index - input_data.push(0x00); + // Hardened derivation: private_key || index + // (no leading 0x00 — that prefix belongs to secp256k1 BIP32, + // where it pads the 33-byte pubkey slot; dashbls doesn't use it) input_data.extend_from_slice(&self.private_key.to_be_bytes()); } else { // Non-hardened derivation: public_key || index - let public_key_bytes = self.public_key_bytes(); - input_data.extend_from_slice(&public_key_bytes); + input_data.extend_from_slice(&self.public_key().to_bytes_with_mode(format)); } let child_bytes = u32::from(child).to_be_bytes(); input_data.extend_from_slice(&child_bytes); @@ -234,7 +286,7 @@ impl ExtendedBLSPrivKey { BlsPublicKey::from(&self.private_key) } - /// Get the public key bytes + /// Get the public key bytes (modern/IETF serialization) pub fn public_key_bytes(&self) -> [u8; 48] { let bytes = self.public_key().to_bytes(); let mut array = [0u8; 48]; @@ -242,6 +294,16 @@ impl ExtendedBLSPrivKey { array } + /// Get the public key bytes in Dash legacy serialization. + /// + /// This is the format dashbls/DashSync use throughout the BLS HD chain. + pub fn public_key_bytes_legacy(&self) -> [u8; 48] { + let bytes = self.public_key().to_bytes_with_mode(SerializationFormat::Legacy); + let mut array = [0u8; 48]; + array.copy_from_slice(&bytes[..48.min(bytes.len())]); + array + } + /// Get the fingerprint of this key pub fn fingerprint(&self) -> Fingerprint { use dashcore_hashes::hash160; @@ -264,11 +326,27 @@ impl ExtendedBLSPrivKey { } } - /// Derive at a path + /// Derive at a path using the modern (IETF) serialization mode + /// (see [`Self::derive_priv`]). pub fn derive_path(&self, path: &DerivationPath) -> Result { + self.derive_path_with_mode(path, SerializationFormat::Modern) + } + + /// Derive at a path using the legacy Dash serialization mode + /// (see [`Self::derive_priv_legacy`]). + pub fn derive_path_legacy(&self, path: &DerivationPath) -> Result { + self.derive_path_with_mode(path, SerializationFormat::Legacy) + } + + /// Derive at a path with an explicit serialization mode. + pub fn derive_path_with_mode( + &self, + path: &DerivationPath, + format: SerializationFormat, + ) -> Result { let mut key = self.clone(); for child in path.as_ref() { - key = key.derive_priv(*child)?; + key = key.derive_priv_with_mode(*child, format)?; } Ok(key) } @@ -304,20 +382,47 @@ impl ExtendedBLSPubKey { } } - /// Derive a child public key (only for non-hardened derivation) + /// Derive a child public key using the modern (IETF) serialization mode + /// (only for non-hardened derivation). pub fn ckd_pub(&self, child: ChildNumber) -> Result { self.derive_pub(child) } - /// Derive a child public key (only for non-hardened derivation) + /// Derive a child public key using the modern (IETF) public key + /// serialization (only for non-hardened derivation). + /// + /// Equivalent to dashbls `PublicChild(i)` with the default + /// `fLegacy = false`. For Dash masternode operator keys use + /// [`Self::derive_pub_legacy`], which matches DashSync. pub fn derive_pub(&self, child: ChildNumber) -> Result { + self.derive_pub_with_mode(child, SerializationFormat::Modern) + } + + /// Derive a child public key using the legacy Dash public key + /// serialization (only for non-hardened derivation). + /// + /// Equivalent to dashbls `PublicChild(i, fLegacy = true)` — the mode + /// dashbls/DashSync use for masternode operator keys. + pub fn derive_pub_legacy(&self, child: ChildNumber) -> Result { + self.derive_pub_with_mode(child, SerializationFormat::Legacy) + } + + /// Derive a child public key with an explicit serialization mode + /// (only for non-hardened derivation). + pub fn derive_pub_with_mode( + &self, + child: ChildNumber, + format: SerializationFormat, + ) -> Result { if child.is_hardened() { return Err(Error::CannotDeriveFromHardenedPublic); } - // Build the input data for HMAC: public_key || index + // Build the input data for HMAC: public_key || index — matches + // dashbls `ExtendedPublicKey::PublicChild`, whose fLegacy flag + // corresponds to `format`. let mut input_data = Vec::new(); - input_data.extend_from_slice(&self.public_key.to_bytes()); + input_data.extend_from_slice(&self.public_key.to_bytes_with_mode(format)); let child_bytes = u32::from(child).to_be_bytes(); input_data.extend_from_slice(&child_bytes); @@ -381,7 +486,7 @@ impl ExtendedBLSPubKey { Fingerprint::from_bytes(fingerprint_bytes) } - /// Get the public key bytes + /// Get the public key bytes (modern/IETF serialization) pub fn to_bytes(&self) -> [u8; 48] { let bytes = self.public_key.to_bytes(); let mut array = [0u8; 48]; @@ -389,11 +494,38 @@ impl ExtendedBLSPubKey { array } - /// Derive at a path (only non-hardened paths allowed) + /// Get the public key bytes in Dash legacy serialization. + /// + /// This is the format dashbls/DashSync use throughout the BLS HD chain. + pub fn to_bytes_legacy(&self) -> [u8; 48] { + let bytes = self.public_key.to_bytes_with_mode(SerializationFormat::Legacy); + let mut array = [0u8; 48]; + array.copy_from_slice(&bytes[..48.min(bytes.len())]); + array + } + + /// Derive at a path using the modern (IETF) serialization mode + /// (only non-hardened paths allowed; see [`Self::derive_pub`]). pub fn derive_path(&self, path: &DerivationPath) -> Result { + self.derive_path_with_mode(path, SerializationFormat::Modern) + } + + /// Derive at a path using the legacy Dash serialization mode + /// (only non-hardened paths allowed; see [`Self::derive_pub_legacy`]). + pub fn derive_path_legacy(&self, path: &DerivationPath) -> Result { + self.derive_path_with_mode(path, SerializationFormat::Legacy) + } + + /// Derive at a path with an explicit serialization mode + /// (only non-hardened paths allowed). + pub fn derive_path_with_mode( + &self, + path: &DerivationPath, + format: SerializationFormat, + ) -> Result { let mut key = self.clone(); for child in path.as_ref() { - key = key.derive_pub(*child)?; + key = key.derive_pub_with_mode(*child, format)?; } Ok(key) } @@ -1241,6 +1373,235 @@ mod tests { assert_eq!(child_unhardened.parent_fingerprint, master.fingerprint()); } + /// Reference vectors generated with dashbls (dashpay/bls-signatures @ 0842b17, + /// the C++ library DashSync uses via FFI): `ExtendedPrivateKey::FromSeed` + + /// `PrivateChild(i, fLegacy=true)`. These pin our derivation to the exact + /// bytes Dash Core / DashSync produce (see issue #878). + mod dashbls_vectors { + use super::*; + + /// BIP39 seed for "abandon abandon ... about" (empty passphrase). + const SEED64: &str = "5eb00bbddcf069084889a8ab9155568165f5c453ccb85e70811aaed6f6da5fc19a5ac40b389cd370d086206dec8aa6c43daea6690f20ad3d8d48b2d2ce9e38e4"; + + fn master_from_seed64() -> ExtendedBLSPrivKey { + let seed = hex::decode(SEED64).unwrap(); + ExtendedBLSPrivKey::new_master(Network::Mainnet, &seed).unwrap() + } + + fn hardened(idx: u32) -> ChildNumber { + ChildNumber::from_hardened_idx(idx).unwrap() + } + + #[test] + fn master_from_seed() { + let master = master_from_seed64(); + assert_eq!( + hex::encode(master.private_key.to_be_bytes()), + "27d1e600fe5ce42e9a18fe064aa0c1b8ee6754289013a86eb1e8af985ddc55c5" + ); + assert_eq!( + hex::encode(&master.chain_code[..]), + "2a680de50ab918089c65f47e6f32363eb8fbb915a61e9a10e0f882aa1c12aef9" + ); + assert_eq!( + hex::encode(master.public_key_bytes_legacy()), + "883389cd6c289b97bfa18cc7b7c873397b4d753269d47d2fa29dda1682c1565687ccb19dd016398da7c9724f8a58bdef" + ); + assert_eq!( + hex::encode(master.public_key_bytes()), + "a83389cd6c289b97bfa18cc7b7c873397b4d753269d47d2fa29dda1682c1565687ccb19dd016398da7c9724f8a58bdef" + ); + } + + #[test] + fn mainnet_operator_account_and_keys() { + // DIP-3 operator path m/9'/5'/3'/3' — every level hardened. + let master = master_from_seed64(); + let account = master + .derive_priv(hardened(9)) + .unwrap() + .derive_priv(hardened(5)) + .unwrap() + .derive_priv(hardened(3)) + .unwrap() + .derive_priv(hardened(3)) + .unwrap(); + + assert_eq!( + hex::encode(account.private_key.to_be_bytes()), + "5f36c0e346c6e6275d6550a09857325e3f54f2a962eb09a48f61756f7b4bbfb0" + ); + assert_eq!( + hex::encode(&account.chain_code[..]), + "d9659c1bde2fd0e0f799f2f66bbbcfc7378fdea624d73d5d4749dc7222daea5e" + ); + + // Operator keys 0..2 (non-hardened children — exercises the + // legacy-serialization HMAC input). + let expected = [ + ( + "11122e1ad656d0610ce0f80d40da874d67ea656a3e66ed371c915ec3a488a43a", + "078cad04aae29eb76171937eb7101452b401b026efbc27db840f130374e6a9ec8443d917277f8921e0ba6678a7709875", + "878cad04aae29eb76171937eb7101452b401b026efbc27db840f130374e6a9ec8443d917277f8921e0ba6678a7709875", + ), + ( + "1a4e3318640cd4e50222184d0ea111abf8a0c18a0e5dc3ed45dad85009db4e31", + "0c04974d14df3b5eb23787e21642d25c47609d966bb80f504854e4675657c63f4c4c7056f3f007671b911eb390dec4f7", + "8c04974d14df3b5eb23787e21642d25c47609d966bb80f504854e4675657c63f4c4c7056f3f007671b911eb390dec4f7", + ), + ( + "107ab3ddb1f1277dd082f3f6c9187145a614c6f0c97e4e5f3c8912ba5bba6200", + "0d14cad34409c74f9dd587fc00be01ca0c527e3793f016eb2a612d78ec45c650e8942fbff3e1ab44a21a6e575ebe80a1", + "8d14cad34409c74f9dd587fc00be01ca0c527e3793f016eb2a612d78ec45c650e8942fbff3e1ab44a21a6e575ebe80a1", + ), + ]; + for (i, (sk, pk_legacy, pk_modern)) in expected.iter().enumerate() { + let child = account + .derive_priv_legacy(ChildNumber::from_normal_idx(i as u32).unwrap()) + .unwrap(); + assert_eq!(hex::encode(child.private_key.to_be_bytes()), *sk, "sk {}", i); + assert_eq!( + hex::encode(child.public_key_bytes_legacy()), + *pk_legacy, + "pk_legacy {}", + i + ); + assert_eq!(hex::encode(child.public_key_bytes()), *pk_modern, "pk_modern {}", i); + } + + // Watch-only path: same child 0 via public derivation. + let account_pub = account.to_extended_pub_key(); + let child0_pub = + account_pub.derive_pub_legacy(ChildNumber::from_normal_idx(0).unwrap()).unwrap(); + assert_eq!(hex::encode(child0_pub.to_bytes_legacy()), expected[0].1); + } + + #[test] + fn testnet_operator_account_and_key0() { + // Testnet operator path m/9'/1'/3'/3'. + let master = master_from_seed64(); + let account = master + .derive_priv(hardened(9)) + .unwrap() + .derive_priv(hardened(1)) + .unwrap() + .derive_priv(hardened(3)) + .unwrap() + .derive_priv(hardened(3)) + .unwrap(); + assert_eq!( + hex::encode(account.private_key.to_be_bytes()), + "05e18aebbe5c73f4dde3dd6a4a204da46c6efa38a38ff4fa5548b1c171154bda" + ); + let child0 = + account.derive_priv_legacy(ChildNumber::from_normal_idx(0).unwrap()).unwrap(); + assert_eq!( + hex::encode(child0.private_key.to_be_bytes()), + "3346dfd71627f9f31cad3ee66fe7b673c32cb077b2eb38c621d7e61c30e46dbd" + ); + assert_eq!( + hex::encode(child0.public_key_bytes_legacy()), + "09d8beabae708de1638487f1aff44b38e8c07d9b09f22d76329d6c8ec01e2ad4d030b660bca40ddbd222373a72c5bcef" + ); + } + + #[test] + fn chia_style_seed8_vectors() { + // dashbls test-suite seed {1, 50, 6, 244, 24, 199, 1, 25}. + let seed = [1u8, 50, 6, 244, 24, 199, 1, 25]; + let master = ExtendedBLSPrivKey::new_master(Network::Testnet, &seed).unwrap(); + assert_eq!( + hex::encode(master.private_key.to_be_bytes()), + "3e9f7b3846c1803703f94c764b51f5ace513b2f02c4d6b2c452d8ce66e5975bd" + ); + assert_eq!( + hex::encode(&master.chain_code[..]), + "d8b12555b4cc5578951e4a7c80031e22019cc0dce168b3ed88115311b8feb1e3" + ); + + // Hardened child 77' + let c77h = master.derive_priv(hardened(77)).unwrap(); + assert_eq!( + hex::encode(c77h.private_key.to_be_bytes()), + "51b31efbd83aeead1e324c5c8248f5a13bb17ba7afe29aeb5ceef7eaff49ed6f" + ); + assert_eq!( + hex::encode(&c77h.chain_code[..]), + "f2c8e4269bb3e54f8179a5c6976d92ca14c3260dd729981e9d15f53049fd698b" + ); + + // Non-hardened child 77 (legacy serialization in HMAC input) + let c77 = master.derive_priv_legacy(ChildNumber::from_normal_idx(77).unwrap()).unwrap(); + assert_eq!( + hex::encode(c77.private_key.to_be_bytes()), + "3ef4f8b4d262fb8981665532b531c7889798044f7cbe4d5fae5e30435f746044" + ); + assert_eq!( + hex::encode(&c77.chain_code[..]), + "f428f5f011f52569c0b2004aaeda0744259f4247fd5e77649d3d25e6b491cc53" + ); + } + + #[test] + fn modern_mode_vectors() { + // dashbls `PrivateChild(i)` with the default fLegacy = false — + // the mode used by `derive_priv`/`derive_pub` without suffix. + let seed = [1u8, 50, 6, 244, 24, 199, 1, 25]; + let master = ExtendedBLSPrivKey::new_master(Network::Testnet, &seed).unwrap(); + let c77 = master.derive_priv(ChildNumber::from_normal_idx(77).unwrap()).unwrap(); + assert_eq!( + hex::encode(c77.private_key.to_be_bytes()), + "0f9b101b475e449c9995032e138b432a330738b6401f675f0632385fe8d349bf" + ); + assert_eq!( + hex::encode(&c77.chain_code[..]), + "c7b09e00d6b9b1676e8714e1060e0324787734809ae557a4bc8c07e9b1304ed0" + ); + assert_eq!( + hex::encode(c77.public_key_bytes()), + "a63fa533db03b400030a5eb163433ac7c8700d2301c4242e03db58d516dea0d52768d1b0d29e9f28f7707ce96d2d6108" + ); + + // Modern-mode child 0 under the operator path from the BIP39 seed. + // Hardened levels agree across modes; the non-hardened leaf differs. + let master64 = master_from_seed64(); + let account = master64 + .derive_priv(hardened(9)) + .unwrap() + .derive_priv(hardened(5)) + .unwrap() + .derive_priv(hardened(3)) + .unwrap() + .derive_priv(hardened(3)) + .unwrap(); + let child0_modern = + account.derive_priv(ChildNumber::from_normal_idx(0).unwrap()).unwrap(); + assert_eq!( + hex::encode(child0_modern.private_key.to_be_bytes()), + "1669d6cc8ac08fa377d63dafcf83f1fa6aee09e2df58c490b1b1a0b0999417ec" + ); + assert_eq!( + hex::encode(child0_modern.public_key_bytes()), + "8f5d504fee1026394728781f004fee70480335c1f53156124b23e45386c7c1e2973efee3eab4ae60650fdaa8ae4460d0" + ); + + // Same leaf via legacy mode is a different key entirely. + let child0_legacy = + account.derive_priv_legacy(ChildNumber::from_normal_idx(0).unwrap()).unwrap(); + assert_ne!( + child0_modern.private_key.to_be_bytes(), + child0_legacy.private_key.to_be_bytes() + ); + + // Private/public derivation stays consistent in modern mode too. + let child0_pub = account + .to_extended_pub_key() + .derive_pub(ChildNumber::from_normal_idx(0).unwrap()) + .unwrap(); + assert_eq!(child0_pub.to_bytes(), child0_modern.public_key_bytes()); + } + } + #[test] fn test_zeroize_clears_key_material() { use zeroize::Zeroize; diff --git a/key-wallet/src/managed_account/address_pool.rs b/key-wallet/src/managed_account/address_pool.rs index 045098aa1..020c3f8f2 100644 --- a/key-wallet/src/managed_account/address_pool.rs +++ b/key-wallet/src/managed_account/address_pool.rs @@ -143,9 +143,11 @@ impl KeySource { #[cfg(feature = "bls")] KeySource::BLSPrivate(xprv) => { // BLS HD derivation using the proper BIP32-like derivation + // Legacy mode: BLS pools exist only for provider operator keys, + // which dashbls/DashSync derive with fLegacy = true. let mut derived = xprv.clone(); for child_num in path.as_ref() { - derived = derived.derive_priv(*child_num).map_err(|e| { + derived = derived.derive_priv_legacy(*child_num).map_err(|e| { Error::InvalidParameter(format!("BLS derivation error: {:?}", e)) })?; } @@ -161,7 +163,7 @@ impl KeySource { "Cannot derive hardened child from BLS public key".into(), )); } - derived = derived.derive_pub(*child_num).map_err(|e| { + derived = derived.derive_pub_legacy(*child_num).map_err(|e| { Error::InvalidParameter(format!("BLS public derivation error: {:?}", e)) })?; } diff --git a/key-wallet/src/tests/mod.rs b/key-wallet/src/tests/mod.rs index 8db87a128..acb7a6378 100644 --- a/key-wallet/src/tests/mod.rs +++ b/key-wallet/src/tests/mod.rs @@ -22,6 +22,8 @@ mod managed_account_collection_tests; mod performance_tests; +mod provider_key_derivation_tests; + mod special_transaction_matching_tests; mod special_transaction_tests; diff --git a/key-wallet/src/tests/provider_key_derivation_tests.rs b/key-wallet/src/tests/provider_key_derivation_tests.rs new file mode 100644 index 000000000..a9bd41d03 --- /dev/null +++ b/key-wallet/src/tests/provider_key_derivation_tests.rs @@ -0,0 +1,162 @@ +//! End-to-end tests for provider (masternode) key derivation. +//! +//! These pin the wallet-level BLS operator key and Ed25519 platform node key +//! flows to reference vectors produced by the implementations DashSync uses +//! (dashbls `ExtendedPrivateKey` with `fLegacy = true` for BLS, SLIP-0010 for +//! Ed25519), so the keys match dashwallet-ios / DashSync for the same +//! mnemonic. See . + +use crate::account::derivation::AccountDerivation; +use crate::account::AccountType; +use crate::mnemonic::{Language, Mnemonic}; +use crate::wallet::initialization::WalletAccountCreationOptions; +use crate::wallet::Wallet; +use crate::{ChildNumber, Network}; + +const TEST_MNEMONIC: &str = + "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about"; + +/// BIP39 seed for [`TEST_MNEMONIC`] with an empty passphrase. +const TEST_SEED_HEX: &str = + "5eb00bbddcf069084889a8ab9155568165f5c453ccb85e70811aaed6f6da5fc19a5ac40b389cd370d086206dec8aa6c43daea6690f20ad3d8d48b2d2ce9e38e4"; + +fn test_wallet(network: Network) -> Wallet { + let mnemonic = Mnemonic::from_phrase(TEST_MNEMONIC, Language::English).unwrap(); + Wallet::from_mnemonic(mnemonic, network, WalletAccountCreationOptions::Default).unwrap() +} + +#[cfg(feature = "bls")] +#[test] +fn bls_operator_keys_match_dashbls_reference() { + // Reference vectors generated with dashbls (dashpay/bls-signatures @ + // 0842b17): ExtendedPrivateKey::FromSeed(seed) then PrivateChild(i, + // fLegacy=true) along m/9'/5'/3'/3', then child i (non-hardened). + let wallet = test_wallet(Network::Mainnet); + let account = wallet + .accounts + .bls_account_of_type(AccountType::ProviderOperatorKeys) + .expect("operator account should be auto-created for mnemonic wallets"); + + // The stored account xpub must be the account-level key at m/9'/5'/3'/3'. + assert_eq!( + hex::encode(account.bls_public_key.to_bytes_legacy()), + "8d794d053504db3727c1f51aea2112e440fadbade687a9c0243b61523c8ab8eb64061f0a5ec5d8df4b7ec8bdfe722c19" + ); + + // Operator key 0 via watch-side (non-hardened public) derivation. + let key0_pub = + account.bls_public_key.derive_pub_legacy(ChildNumber::from_normal_idx(0).unwrap()).unwrap(); + assert_eq!( + hex::encode(key0_pub.to_bytes_legacy()), + "078cad04aae29eb76171937eb7101452b401b026efbc27db840f130374e6a9ec8443d917277f8921e0ba6678a7709875" + ); + // Same point in modern/IETF (basic-scheme) serialization, as it appears in + // v19+ ProRegTx payloads. + assert_eq!( + hex::encode(key0_pub.to_bytes()), + "878cad04aae29eb76171937eb7101452b401b026efbc27db840f130374e6a9ec8443d917277f8921e0ba6678a7709875" + ); + + // Operator secret key 0 via the seed-based signing path. + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let sk0 = account.derive_from_seed_private_key_at(&seed, 0).unwrap(); + assert_eq!( + hex::encode(sk0.to_be_bytes()), + "11122e1ad656d0610ce0f80d40da874d67ea656a3e66ed371c915ec3a488a43a" + ); + let sk1 = account.derive_from_seed_private_key_at(&seed, 1).unwrap(); + assert_eq!( + hex::encode(sk1.to_be_bytes()), + "1a4e3318640cd4e50222184d0ea111abf8a0c18a0e5dc3ed45dad85009db4e31" + ); +} + +#[cfg(feature = "bls")] +#[test] +fn bls_operator_keys_testnet_match_dashbls_reference() { + // Testnet uses coin type 1: m/9'/1'/3'/3'. + let wallet = test_wallet(Network::Testnet); + let account = wallet + .accounts + .bls_account_of_type(AccountType::ProviderOperatorKeys) + .expect("operator account should be auto-created for mnemonic wallets"); + + let key0_pub = + account.bls_public_key.derive_pub_legacy(ChildNumber::from_normal_idx(0).unwrap()).unwrap(); + assert_eq!( + hex::encode(key0_pub.to_bytes_legacy()), + "09d8beabae708de1638487f1aff44b38e8c07d9b09f22d76329d6c8ec01e2ad4d030b660bca40ddbd222373a72c5bcef" + ); + + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let sk0 = account.derive_from_seed_private_key_at(&seed, 0).unwrap(); + assert_eq!( + hex::encode(sk0.to_be_bytes()), + "3346dfd71627f9f31cad3ee66fe7b673c32cb077b2eb38c621d7e61c30e46dbd" + ); +} + +#[cfg(feature = "eddsa")] +#[test] +fn ed25519_platform_node_keys_match_slip10_reference() { + // Reference vectors computed with an independent SLIP-0010 implementation: + // master from the BIP39 seed, then hardened path m/9'/5'/3'/4' and child 0'. + let wallet = test_wallet(Network::Mainnet); + let account = wallet + .accounts + .eddsa_account_of_type(AccountType::ProviderPlatformKeys) + .expect("platform node account should be auto-created for mnemonic wallets"); + + // The stored account key must be the account-level key at m/9'/5'/3'/4'. + let expected_account_sk = + hex::decode("80035d9c2f89971a9c9fad826bba8be9328f1686ae555e912949c2c32800c379").unwrap(); + let expected_account_pk = dashcore::ed25519_dalek::SigningKey::from_bytes( + expected_account_sk.as_slice().try_into().unwrap(), + ) + .verifying_key(); + assert_eq!(account.ed25519_public_key.public_key, expected_account_pk); + + // Platform node key 0 (hardened child 0') via the seed-based signing path. + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let sk0 = account.derive_from_seed_private_key_at(&seed, 0).unwrap(); + assert_eq!( + hex::encode(sk0.to_bytes()), + "5fa238b12be77347abf9b5957bd902d16c6aaca28d25c4267ffacbd7458dceb1" + ); +} + +/// Wallets created from a bare extended private key carry no seed, so +/// DashSync-compatible BLS/Ed25519 provider keys cannot be derived for them. +/// Default account creation must skip those accounts rather than fabricate +/// keys that never correspond to anything on-chain, and explicit account +/// creation without a seed must fail. +#[cfg(all(feature = "bls", feature = "eddsa"))] +#[test] +fn seedless_wallet_skips_provider_operator_and_platform_accounts() { + use crate::bip32::ExtendedPrivKey; + use crate::Error; + + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let master = ExtendedPrivKey::new_master(Network::Testnet, &seed).unwrap(); + let mut wallet = + Wallet::from_extended_key(master, WalletAccountCreationOptions::Default).unwrap(); + + assert!(wallet.accounts.bls_account_of_type(AccountType::ProviderOperatorKeys).is_none()); + assert!(wallet.accounts.eddsa_account_of_type(AccountType::ProviderPlatformKeys).is_none()); + + // Explicit creation without a seed fails with a typed error... + assert!(matches!( + wallet.add_bls_account(AccountType::ProviderOperatorKeys, None), + Err(Error::KeylessWalletRequiresAccountKey { .. }) + )); + assert!(matches!( + wallet.add_eddsa_account(AccountType::ProviderPlatformKeys, None), + Err(Error::KeylessWalletRequiresAccountKey { .. }) + )); + + // ...but works when the seed is supplied explicitly. + wallet.add_bls_account(AccountType::ProviderOperatorKeys, Some(&seed)).unwrap(); + wallet.add_eddsa_account(AccountType::ProviderPlatformKeys, Some(&seed)).unwrap(); + assert!(wallet.accounts.bls_account_of_type(AccountType::ProviderOperatorKeys).is_some()); + assert!(wallet.accounts.eddsa_account_of_type(AccountType::ProviderPlatformKeys).is_some()); +} diff --git a/key-wallet/src/wallet/accounts.rs b/key-wallet/src/wallet/accounts.rs index 776065c54..6982252f3 100644 --- a/key-wallet/src/wallet/accounts.rs +++ b/key-wallet/src/wallet/accounts.rs @@ -73,22 +73,27 @@ impl Wallet { /// /// BLS accounts are used for Platform/masternode operations. /// + /// The seed (whether provided or taken from the wallet) is fed directly + /// into the BLS HD master key and the account's derivation path is applied + /// in the BLS scheme, matching DashSync/dashbls. + /// /// # Arguments /// * `account_type` - The type of account (must be ProviderOperatorKeys) - /// * `bls_seed` - Optional 32-byte seed for BLS key generation. If not provided, - /// the account is derived from the wallet's in-wallet private key. + /// * `bls_seed` - Optional wallet seed (typically the 64-byte BIP39 seed) + /// for BLS key generation. If not provided, the wallet's own seed is used. /// /// # Returns /// Ok(()) if the account was successfully added /// /// # Errors /// Returns [`Error::KeylessWalletRequiresAccountKey`] when `bls_seed` is `None` - /// on a keyless wallet (watch-only / external-signable): pass the seed via `Some(..)`. + /// on a wallet without a stored seed (watch-only / external-signable / + /// created from an extended private key): pass the seed via `Some(..)`. #[cfg(feature = "bls")] pub fn add_bls_account( &mut self, account_type: AccountType, - bls_seed: Option<[u8; 32]>, + bls_seed: Option<&[u8]>, ) -> Result<()> { // Validate account type if !matches!(account_type, AccountType::ProviderOperatorKeys) { @@ -100,28 +105,18 @@ impl Wallet { // Get a unique wallet ID for this wallet first let wallet_id = self.get_wallet_id(); - // Create the BLS account based on whether we have a seed or need to derive + // Use the provided seed, or fall back to the wallet's own seed. + // BLS provider keys must be derived from the raw seed in the BLS + // scheme (dashbls ExtendedPrivateKey::FromSeed) — they cannot be + // derived from a secp256k1 extended private key. let bls_account = if let Some(seed) = bls_seed { - // Use the provided seed BLSAccount::from_seed(Some(wallet_id.to_vec()), account_type, seed, self.network)? } else { - // Derive from wallet's private key - let derivation_path = account_type.derivation_path(self.network)?; - - let root_key = self.root_extended_priv_key().map_err(|_| { - Error::KeylessWalletRequiresAccountKey { - account_type, - required_key: "32-byte BLS seed", - } + let seed = self.wallet_seed_bytes().ok_or(Error::KeylessWalletRequiresAccountKey { + account_type, + required_key: "wallet seed (e.g. 64-byte BIP39 seed)", })?; - let master_key = root_key.to_extended_priv_key(self.network); - let secp = Secp256k1::new(); - let account_xpriv = - master_key.derive_priv(&secp, &derivation_path).map_err(Error::Bip32)?; - - // Create BLS seed from derived private key - let seed = account_xpriv.private_key.secret_bytes(); - BLSAccount::from_seed(Some(wallet_id.to_vec()), account_type, seed, self.network)? + BLSAccount::from_seed(Some(wallet_id.to_vec()), account_type, &seed, self.network)? }; // Check if account already exists @@ -142,22 +137,28 @@ impl Wallet { /// /// EdDSA accounts are used for Platform operations. /// + /// The seed (whether provided or taken from the wallet) is fed directly + /// into the SLIP-0010 Ed25519 master key and the account's derivation path + /// is applied in the Ed25519 scheme, matching DashSync. + /// /// # Arguments /// * `account_type` - The type of account (must be ProviderPlatformKeys) - /// * `ed25519_seed` - Optional 32-byte seed for Ed25519 key generation. If not provided, - /// the account is derived from the wallet's in-wallet private key. + /// * `ed25519_seed` - Optional wallet seed (typically the 64-byte BIP39 + /// seed) for Ed25519 key generation. If not provided, the wallet's own + /// seed is used. /// /// # Returns /// Ok(()) if the account was successfully added /// /// # Errors /// Returns [`Error::KeylessWalletRequiresAccountKey`] when `ed25519_seed` is `None` - /// on a keyless wallet (watch-only / external-signable): pass the seed via `Some(..)`. + /// on a wallet without a stored seed (watch-only / external-signable / + /// created from an extended private key): pass the seed via `Some(..)`. #[cfg(feature = "eddsa")] pub fn add_eddsa_account( &mut self, account_type: AccountType, - ed25519_seed: Option<[u8; 32]>, + ed25519_seed: Option<&[u8]>, ) -> Result<()> { // Validate account type if !matches!(account_type, AccountType::ProviderPlatformKeys) { @@ -169,28 +170,17 @@ impl Wallet { // Get a unique wallet ID for this wallet first let wallet_id = self.get_wallet_id(); - // Create the EdDSA account based on whether we have a seed or need to derive + // Use the provided seed, or fall back to the wallet's own seed. + // Platform node keys must be derived from the raw seed via SLIP-0010 — + // they cannot be derived from a secp256k1 extended private key. let eddsa_account = if let Some(seed) = ed25519_seed { - // Use the provided seed EdDSAAccount::from_seed(Some(wallet_id.to_vec()), account_type, seed, self.network)? } else { - // Derive from wallet's private key - let derivation_path = account_type.derivation_path(self.network)?; - - let root_key = self.root_extended_priv_key().map_err(|_| { - Error::KeylessWalletRequiresAccountKey { - account_type, - required_key: "32-byte Ed25519 seed", - } + let seed = self.wallet_seed_bytes().ok_or(Error::KeylessWalletRequiresAccountKey { + account_type, + required_key: "wallet seed (e.g. 64-byte BIP39 seed)", })?; - let master_key = root_key.to_extended_priv_key(self.network); - let secp = Secp256k1::new(); - let account_xpriv = - master_key.derive_priv(&secp, &derivation_path).map_err(Error::Bip32)?; - - // Create Ed25519 seed from derived private key - let seed = account_xpriv.private_key.secret_bytes(); - EdDSAAccount::from_seed(Some(wallet_id.to_vec()), account_type, seed, self.network)? + EdDSAAccount::from_seed(Some(wallet_id.to_vec()), account_type, &seed, self.network)? }; // Check if account already exists diff --git a/key-wallet/src/wallet/helper.rs b/key-wallet/src/wallet/helper.rs index c5a519ed4..a282804e8 100644 --- a/key-wallet/src/wallet/helper.rs +++ b/key-wallet/src/wallet/helper.rs @@ -101,6 +101,28 @@ impl Wallet { matches!(self.wallet_type, WalletType::Seed { .. } | WalletType::Mnemonic { .. }) } + /// Get the wallet's 64-byte seed, if it has one. + /// + /// For mnemonic wallets this is the BIP39 seed (empty passphrase — the + /// same convention wallet construction uses to derive the root key). + /// Wallets created from an extended private key, watch-only wallets and + /// external-signable wallets have no seed and return `None`. + pub fn wallet_seed_bytes(&self) -> Option<[u8; 64]> { + match &self.wallet_type { + WalletType::Mnemonic { + mnemonic, + .. + } => Some(mnemonic.to_seed("")), + WalletType::Seed { + seed, + .. + } => Some(*seed.as_bytes()), + WalletType::ExtendedPrivKey(_) + | WalletType::ExternalSignable + | WalletType::WatchOnly => None, + } + } + /// Create accounts based on the provided creation options pub(crate) fn create_accounts_from_options( &mut self, @@ -321,10 +343,18 @@ impl Wallet { // Provider keys accounts self.add_account(AccountType::ProviderVotingKeys, None)?; self.add_account(AccountType::ProviderOwnerKeys, None)?; - #[cfg(feature = "bls")] - self.add_bls_account(AccountType::ProviderOperatorKeys, None)?; - #[cfg(feature = "eddsa")] - self.add_eddsa_account(AccountType::ProviderPlatformKeys, None)?; + // Operator (BLS) and platform-node (Ed25519) keys are derived from the + // raw wallet seed in their own scheme (matching DashSync/dashbls), not + // from the secp256k1 root key — so they can only be auto-created for + // wallets that carry a seed. Seedless wallets (extended-priv-key based) + // can still add them explicitly via add_bls_account/add_eddsa_account + // with an externally supplied seed. + if self.has_seed() { + #[cfg(feature = "bls")] + self.add_bls_account(AccountType::ProviderOperatorKeys, None)?; + #[cfg(feature = "eddsa")] + self.add_eddsa_account(AccountType::ProviderPlatformKeys, None)?; + } Ok(()) } From bc6cbf962333c7bb4ef9029745a98ae1faca1ef6 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 09:15:22 +0700 Subject: [PATCH 046/149] feat(key-wallet): gate-free provider-key derivation API (operator/platform-node at index) (#881) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet): gate-free provider-key derivation entry points (#880) Add wallet-state-agnostic entry points for deriving provider keys at an index, with no is_watch_only gate in either direction: - BLSAccount::operator_public_key_at(index): legacy-scheme non-hardened public derivation off the stored account xpub; works on resident AND watch-only accounts. Returns the extended key so both the legacy and modern/IETF serializations are obtainable. Rejects non-operator accounts so a mixed-up lookup fails loudly. - BLSAccount::operator_private_key_at(seed, network, index): re-derives the full DIP-3 operator path from the raw BIP39 seed in the legacy BLS scheme, then the non-hardened child; needs no account state. - EdDSAAccount::platform_node_key_at(seed, network, index): SLIP-0010 hardened derivation of the platform node key from the raw seed. The existing convenience wrappers are gated on is_watch_only in opposite directions (public derivation requires watch-only, seed derivation requires non-watch-only), so no single upstream API served both resident and restored/external-signable wallets — forcing downstream consumers to re-compose low-level primitives (see dashpay/platform#4120 and the stale derivation bug behind #878/#879). New tests pin the entry points to the existing dashbls / SLIP-0010 golden vectors and assert they behave identically on resident and watch-only accounts. Closes #880 Co-Authored-By: Claude Fable 5 * ci: pin sanitizer jobs to nightly-2026-07-13 (rustc ICE on latest nightly) The 2026-07-14 nightly (rustc daf2e5e18) ICEs while expanding the libtest harness for dash-spv-ffi tests ("attribute is missing tokens: rustc_test_entrypoint_marker" in rustc_ast/src/attr/mod.rs), breaking the Address Sanitizer job for every PR. Pin ASAN/TSAN to the last known-good nightly until a fixed nightly ships. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- .github/workflows/sanitizer.yml | 19 +++- key-wallet/src/account/bls_account.rs | 49 ++++++++++ key-wallet/src/account/eddsa_account.rs | 20 ++++ .../tests/provider_key_derivation_tests.rs | 97 +++++++++++++++++++ 4 files changed, 180 insertions(+), 5 deletions(-) diff --git a/.github/workflows/sanitizer.yml b/.github/workflows/sanitizer.yml index f44639514..839bb8fe8 100644 --- a/.github/workflows/sanitizer.yml +++ b/.github/workflows/sanitizer.yml @@ -18,6 +18,13 @@ on: permissions: contents: read +# Pinned nightly: nightly-2026-07-14 (rustc daf2e5e18) ICEs while expanding the +# libtest harness for dash-spv-ffi ("attribute is missing tokens: +# rustc_test_entrypoint_marker" in rustc_ast/src/attr/mod.rs). Bump or restore +# the floating `nightly` once a fixed nightly ships. +env: + SANITIZER_TOOLCHAIN: nightly-2026-07-13 + concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true @@ -28,8 +35,9 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v6 - - uses: dtolnay/rust-toolchain@nightly + - uses: dtolnay/rust-toolchain@master with: + toolchain: ${{ env.SANITIZER_TOOLCHAIN }} components: rust-src - name: Run tests with ASAN @@ -41,11 +49,11 @@ jobs: SKIP_DASHD_TESTS: 1 run: | # FFI crates (C interop) - cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu \ + cargo "+$SANITIZER_TOOLCHAIN" test -Zbuild-std --target x86_64-unknown-linux-gnu \ -p key-wallet-ffi -p dash-spv-ffi --lib --tests # Core crypto crates (unsafe optimizations) - cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu \ + cargo "+$SANITIZER_TOOLCHAIN" test -Zbuild-std --target x86_64-unknown-linux-gnu \ -p dashcore -p dashcore_hashes --lib --tests tsan: @@ -53,8 +61,9 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v6 - - uses: dtolnay/rust-toolchain@nightly + - uses: dtolnay/rust-toolchain@master with: + toolchain: ${{ env.SANITIZER_TOOLCHAIN }} components: rust-src - name: Run tests with TSAN @@ -65,5 +74,5 @@ jobs: SKIP_DASHD_TESTS: 1 run: | # Async crate with concurrent code - cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu \ + cargo "+$SANITIZER_TOOLCHAIN" test -Zbuild-std --target x86_64-unknown-linux-gnu \ -p dash-spv --lib --tests diff --git a/key-wallet/src/account/bls_account.rs b/key-wallet/src/account/bls_account.rs index 5f8601eb5..226a83b32 100644 --- a/key-wallet/src/account/bls_account.rs +++ b/key-wallet/src/account/bls_account.rs @@ -136,6 +136,55 @@ impl BLSAccount { }) } + /// Derive the operator (masternode) BLS public key at `index` from the + /// stored account-level extended public key, using legacy-scheme + /// (`fLegacy = true`) non-hardened derivation, matching dashbls/DashSync. + /// + /// This is wallet-state-agnostic: it works on both resident accounts + /// (created from a mnemonic or seed) and watch-only / externally-signable + /// accounts, since only the account xpub is needed for the non-hardened + /// child step. There is no `is_watch_only` gate. + /// + /// The returned extended public key exposes both serializations of the + /// operator key: [`ExtendedBLSPubKey::to_bytes_legacy`] (legacy Dash + /// format) and [`ExtendedBLSPubKey::to_bytes`] (modern/IETF format used + /// in v19+ ProRegTx payloads). + /// + /// Returns an error if this account is not an + /// [`AccountType::ProviderOperatorKeys`] account, so a mixed-up account + /// lookup fails loudly instead of yielding valid-looking wrong keys. + pub fn operator_public_key_at(&self, index: u32) -> Result { + if self.account_type != AccountType::ProviderOperatorKeys { + return Err(Error::InvalidParameter(format!( + "operator_public_key_at requires a ProviderOperatorKeys account, got {}", + self.account_type + ))); + } + let child = ChildNumber::from_normal_idx(index)?; + Ok(self.bls_public_key.derive_pub_legacy(child)?) + } + + /// Derive the operator (masternode) BLS secret key at `index` directly + /// from a wallet seed (e.g. the 64-byte BIP39 seed). + /// + /// The full DIP-3 operator account path (`m/9'/5'/3'/3'` on mainnet, + /// `m/9'/1'/3'/3'` otherwise) is re-derived from the seed in the legacy + /// BLS scheme, then the non-hardened child `index` is applied — matching + /// dashbls/DashSync. Because derivation starts from the raw seed, no + /// account state is involved and in particular no `is_watch_only` gate + /// applies. + pub fn operator_private_key_at( + seed: &[u8], + network: Network, + index: u32, + ) -> Result> { + let master = ExtendedBLSPrivKey::new_master(network, seed)?; + let path = AccountType::ProviderOperatorKeys.derivation_path(network)?; + let account_xpriv = master.derive_path_legacy(&path)?; + let child = account_xpriv.derive_priv_legacy(ChildNumber::from_normal_idx(index)?)?; + Ok(child.private_key.clone()) + } + /// Derive a BLS key at a specific path (watch-only, non-hardened paths only) pub fn derive_bls_key_at_path(&self, path: &[u32]) -> Result { if self.is_watch_only { diff --git a/key-wallet/src/account/eddsa_account.rs b/key-wallet/src/account/eddsa_account.rs index b9c3a5cd1..bff1c5cc8 100644 --- a/key-wallet/src/account/eddsa_account.rs +++ b/key-wallet/src/account/eddsa_account.rs @@ -128,6 +128,26 @@ impl EdDSAAccount { }) } + /// Derive the Platform node Ed25519 signing key at `index` directly from + /// a wallet seed (e.g. the 64-byte BIP39 seed). + /// + /// SLIP-0010 Ed25519 only supports hardened derivation, so the key is the + /// hardened child `index'` of the platform node account path + /// (`m/9'/5'/3'/4'` on mainnet, `m/9'/1'/3'/4'` otherwise), matching + /// DashSync. Because derivation starts from the raw seed, no account + /// state is involved and in particular no `is_watch_only` gate applies. + pub fn platform_node_key_at( + seed: &[u8], + network: Network, + index: u32, + ) -> Result { + let master = ExtendedEd25519PrivKey::new_master(network, seed)?; + let path = AccountType::ProviderPlatformKeys.derivation_path(network)?; + let account_xpriv = master.derive_priv(&path)?; + let child = account_xpriv.derive_priv(&[ChildNumber::from_hardened_idx(index)?])?; + Ok(dashcore::ed25519_dalek::SigningKey::from_bytes(&child.private_key)) + } + /// Derive an Ed25519 key at a specific path /// Note: Ed25519 with SLIP-0010 only supports hardened derivation pub fn derive_ed25519_key_at_path(&self, path: &[u32]) -> Result { diff --git a/key-wallet/src/tests/provider_key_derivation_tests.rs b/key-wallet/src/tests/provider_key_derivation_tests.rs index a9bd41d03..4739095e3 100644 --- a/key-wallet/src/tests/provider_key_derivation_tests.rs +++ b/key-wallet/src/tests/provider_key_derivation_tests.rs @@ -125,6 +125,103 @@ fn ed25519_platform_node_keys_match_slip10_reference() { ); } +/// The gate-free provider-key entry points must work identically on resident +/// (non-watch-only) and watch-only accounts, and must match the dashbls +/// reference vectors. See +/// . +#[cfg(feature = "bls")] +#[test] +fn operator_key_at_is_wallet_state_agnostic() { + use crate::account::bls_account::BLSAccount; + + let wallet = test_wallet(Network::Mainnet); + let resident = wallet + .accounts + .bls_account_of_type(AccountType::ProviderOperatorKeys) + .expect("operator account should be auto-created for mnemonic wallets"); + assert!(!resident.is_watch_only); + + // A restored / externally-signable wallet stores the same account xpub but + // flags it watch-only (`BLSAccount::new` sets `is_watch_only: true`). + let watch_only = resident.to_watch_only(); + assert!(watch_only.is_watch_only); + + // The legacy-gated wrapper fails on the resident account — the new entry + // point must not. + assert!(resident.derive_bls_key_at_index(0).is_err()); + + for account in [resident, &watch_only] { + let key0 = account.operator_public_key_at(0).expect("gate-free derivation must succeed"); + assert_eq!( + hex::encode(key0.to_bytes_legacy()), + "078cad04aae29eb76171937eb7101452b401b026efbc27db840f130374e6a9ec8443d917277f8921e0ba6678a7709875" + ); + assert_eq!( + hex::encode(key0.to_bytes()), + "878cad04aae29eb76171937eb7101452b401b026efbc27db840f130374e6a9ec8443d917277f8921e0ba6678a7709875" + ); + } + + // Seed-based private derivation needs no account state at all. + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let sk0 = BLSAccount::operator_private_key_at(&seed, Network::Mainnet, 0).unwrap(); + assert_eq!( + hex::encode(sk0.to_be_bytes()), + "11122e1ad656d0610ce0f80d40da874d67ea656a3e66ed371c915ec3a488a43a" + ); + let sk1 = BLSAccount::operator_private_key_at(&seed, Network::Mainnet, 1).unwrap(); + assert_eq!( + hex::encode(sk1.to_be_bytes()), + "1a4e3318640cd4e50222184d0ea111abf8a0c18a0e5dc3ed45dad85009db4e31" + ); + + // Testnet vectors (coin type 1: m/9'/1'/3'/3'). + let sk0_testnet = BLSAccount::operator_private_key_at(&seed, Network::Testnet, 0).unwrap(); + assert_eq!( + hex::encode(sk0_testnet.to_be_bytes()), + "3346dfd71627f9f31cad3ee66fe7b673c32cb077b2eb38c621d7e61c30e46dbd" + ); +} + +/// `operator_public_key_at` refuses non-operator accounts so a mixed-up +/// account lookup fails loudly instead of yielding valid-looking wrong keys. +#[cfg(feature = "bls")] +#[test] +fn operator_public_key_at_rejects_non_operator_accounts() { + use crate::account::bls_account::BLSAccount; + + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let account = + BLSAccount::from_seed(None, AccountType::IdentityRegistration, &seed, Network::Mainnet) + .unwrap(); + assert!(account.operator_public_key_at(0).is_err()); +} + +#[cfg(feature = "eddsa")] +#[test] +fn platform_node_key_at_is_wallet_state_agnostic() { + use crate::account::eddsa_account::EdDSAAccount; + + // Seed-based hardened derivation, pinned to the SLIP-0010 reference + // vector for m/9'/5'/3'/4'/0'. No account state is involved. + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let sk0 = EdDSAAccount::platform_node_key_at(&seed, Network::Mainnet, 0).unwrap(); + assert_eq!( + hex::encode(sk0.to_bytes()), + "5fa238b12be77347abf9b5957bd902d16c6aaca28d25c4267ffacbd7458dceb1" + ); + + // Cross-check against the account stored in a resident wallet: the + // seed-based path must agree with the account-based path. + let wallet = test_wallet(Network::Mainnet); + let account = wallet + .accounts + .eddsa_account_of_type(AccountType::ProviderPlatformKeys) + .expect("platform node account should be auto-created for mnemonic wallets"); + let via_account = account.derive_from_seed_private_key_at(&seed, 0).unwrap(); + assert_eq!(sk0.to_bytes(), via_account.to_bytes()); +} + /// Wallets created from a bare extended private key carry no seed, so /// DashSync-compatible BLS/Ed25519 provider keys cannot be derived for them. /// Default account creation must skip those accounts rather than fabricate From be18425448bb531240814ac213bc81ea9f4a9ed5 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 11:21:07 +0700 Subject: [PATCH 047/149] fix(key-wallet): extend gap limit for operator keys like owner/voting keys (#882) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When a ProRegTx/ProUpRegTx marks a provider key as used, wallet_checker runs maintain_gap_limit on the account's pools — but it sourced the derivation key via extended_public_key_for_account_type, which returns None for BLS accounts, so the operator pool was silently skipped while the ECDSA owner/voting pools were extended. A wallet whose operator keys were all registered would stop detecting registrations with its next operator keys. Add Wallet::key_source_for_account_type, which yields KeySource::BLSPublic (legacy-scheme non-hardened derivation, matching how the pool addresses were generated) for the operator account and KeySource::Public for ECDSA accounts, and route wallet_checker's gap maintenance through it. Platform-node (EdDSA) pools remain excluded: SLIP-0010 supports hardened derivation only, so they cannot be extended from a public key. The regression test pins the semantics: after a ProRegTx uses operator key 0, the pool extends to highest_used + gap_limit and the freshly derived entries carry BLS public keys matching the audited operator_public_key_at entry point. Co-authored-by: Claude Fable 5 --- .../special_transaction_matching_tests.rs | 91 +++++++++++++++++++ .../transaction_checking/wallet_checker.rs | 9 +- key-wallet/src/wallet/helper.rs | 34 +++++++ 3 files changed, 129 insertions(+), 5 deletions(-) diff --git a/key-wallet/src/tests/special_transaction_matching_tests.rs b/key-wallet/src/tests/special_transaction_matching_tests.rs index 5f62ebf38..bd2d2876e 100644 --- a/key-wallet/src/tests/special_transaction_matching_tests.rs +++ b/key-wallet/src/tests/special_transaction_matching_tests.rs @@ -432,6 +432,97 @@ async fn provider_registration_with_operator_public_key_matches_provider_operato assert_matched_account_type(&result, AccountTypeToCheck::ProviderOperatorKeys); } +/// After a ProRegTx uses an operator key, gap-limit maintenance must extend +/// the operator pool with freshly derived BLS keys — exactly like the ECDSA +/// owner/voting pools. Regression test for the operator pool being skipped +/// because the wallet exposes no ECDSA xpub for BLS accounts (the checker +/// bailed before `maintain_gap_limit`). +#[cfg(feature = "bls")] +#[tokio::test] +async fn provider_registration_extends_operator_key_gap_limit() { + use crate::managed_account::address_pool::PublicKeyType; + use crate::managed_account::managed_account_type::ManagedAccountType; + use crate::AccountType; + + let (mut wallet, mut info) = make_wallet(); + let (_other_wallet, mut other_info) = make_wallet(); + + let owner_addr = other_info + .provider_owner_keys_managed_account_mut() + .expect("other owner") + .next_address(None, true) + .expect("derive owner"); + let voting_addr = other_info + .provider_voting_keys_managed_account_mut() + .expect("other voting") + .next_address(None, true) + .expect("derive voting"); + // Takes operator key 0 and marks it used, so the pool's unused window + // shrinks below the gap limit. + let operator_pk = info + .provider_operator_keys_managed_account_mut() + .expect("provider_operator_keys managed") + .next_bls_operator_key(None, true) + .expect("derive operator"); + + let operator_pool_state = |info: &ManagedWalletInfo| { + let account = + info.provider_operator_keys_managed_account().expect("operator managed account"); + match account.managed_account_type() { + ManagedAccountType::ProviderOperatorKeys { + addresses, + } => (addresses.highest_generated, addresses.highest_used, addresses.gap_limit), + _ => unreachable!("operator account holds an operator pool"), + } + }; + + let (highest_generated_before, highest_used_before, gap_limit) = operator_pool_state(&info); + assert_eq!(highest_used_before, Some(0)); + assert!( + highest_generated_before < Some(gap_limit), + "setup: the pre-generated window must not already satisfy the gap limit" + ); + + let tx = prov_reg_tx( + ProviderMasternodeType::Regular, + derive_pubkey_hash(&owner_addr), + derive_pubkey_hash(&voting_addr), + operator_pk.0.to_compressed().into(), + ScriptBuf::new(), + None, + ); + let result = + info.check_core_transaction(&tx, test_block_context(), &mut wallet, true, true).await; + + assert_matched_account_type(&result, AccountTypeToCheck::ProviderOperatorKeys); + + // The pool must now hold `gap_limit` unused keys past the used index 0. + let (highest_generated_after, highest_used_after, _) = operator_pool_state(&info); + assert_eq!(highest_used_after, Some(0)); + assert_eq!(highest_generated_after, Some(gap_limit)); + + // The freshly derived keys are surfaced to callers and carry BLS public + // keys matching the audited gate-free derivation entry point. + let bls_account = wallet + .accounts + .bls_account_of_type(AccountType::ProviderOperatorKeys) + .expect("operator account exists"); + let new_operator_keys: Vec<_> = result + .new_addresses + .iter() + .filter(|derived| derived.account_type == AccountType::ProviderOperatorKeys) + .collect(); + assert!(!new_operator_keys.is_empty(), "gap maintenance must derive new operator keys"); + for derived in new_operator_keys { + let expected = bls_account + .operator_public_key_at(derived.info.index) + .expect("gate-free public derivation") + .to_bytes() + .to_vec(); + assert_eq!(derived.info.public_key, Some(PublicKeyType::BLS(expected))); + } +} + #[cfg(feature = "eddsa")] #[tokio::test] async fn provider_registration_with_platform_node_id_matches_provider_platform_keys() { diff --git a/key-wallet/src/transaction_checking/wallet_checker.rs b/key-wallet/src/transaction_checking/wallet_checker.rs index 356ce45da..7ea653924 100644 --- a/key-wallet/src/transaction_checking/wallet_checker.rs +++ b/key-wallet/src/transaction_checking/wallet_checker.rs @@ -178,14 +178,13 @@ impl WalletTransactionChecker for ManagedWalletInfo { account.mark_address_used(&address_info.address); } - let Some(xpub) = wallet.extended_public_key_for_account_type( + let key_source = wallet.key_source_for_account_type( &account_match.account_type_match.to_account_type_to_check(), account_match.account_type_match.account_index(), - ) else { + ); + if matches!(key_source, KeySource::NoKeySource) { continue; - }; - - let key_source = KeySource::Public(xpub); + } let rev_before = result.new_addresses.len(); let owning_account_type = account.managed_account_type().to_account_type(); for pool in account.managed_account_type_mut().address_pools_mut() { diff --git a/key-wallet/src/wallet/helper.rs b/key-wallet/src/wallet/helper.rs index a282804e8..a7cfc209c 100644 --- a/key-wallet/src/wallet/helper.rs +++ b/key-wallet/src/wallet/helper.rs @@ -616,4 +616,38 @@ impl Wallet { } } } + + /// Get a [`crate::KeySource`] capable of public (watch-side) derivation + /// for a specific account type. + /// + /// This is the key-source counterpart of + /// [`Self::extended_public_key_for_account_type`]: ECDSA accounts yield + /// [`crate::KeySource::Public`], while the BLS provider operator account + /// yields [`crate::KeySource::BLSPublic`] (legacy-scheme non-hardened + /// derivation) so gap-limit maintenance can extend the operator key pool + /// just like the owner/voting pools. + /// + /// Returns [`crate::KeySource::NoKeySource`] for account types that + /// cannot derive publicly: the Ed25519 platform node account (SLIP-0010 + /// supports hardened derivation only) and Dashpay accounts (not + /// retrieved via this helper). + pub fn key_source_for_account_type( + &self, + account_type: &crate::transaction_checking::transaction_router::AccountTypeToCheck, + account_index: Option, + ) -> crate::KeySource { + match account_type { + #[cfg(feature = "bls")] + crate::transaction_checking::transaction_router::AccountTypeToCheck::ProviderOperatorKeys => self + .accounts + .provider_operator_keys + .as_ref() + .map(|a| crate::KeySource::BLSPublic(a.bls_public_key.clone())) + .unwrap_or(crate::KeySource::NoKeySource), + _ => self + .extended_public_key_for_account_type(account_type, account_index) + .map(crate::KeySource::Public) + .unwrap_or(crate::KeySource::NoKeySource), + } + } } From a9594591d8a8d47f0a1c0442ca1e5f25fe523c27 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 11:35:41 +0700 Subject: [PATCH 048/149] fix(key-wallet): platform node id is the Tenderdash node ID (SHA256[0..20]), not hash160 (#884) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(key-wallet): platform node id is the Tenderdash node ID, not hash160 (#883) key-wallet computed the masternode platform node id as HASH160(ed25519_pubkey), but the canonical value — what dashmate derives from node_key.json, what Tenderdash announces on the platform P2P network, and what ProRegTx builders (DashSync) write on-chain — is the Tenderdash/CometBFT node ID: SHA256(pubkey)[0..20]. Wallet-derived node ids therefore never matched real on-chain platform_node_id values, so ProviderPlatformKeys matching could never hit an actual evonode registration. - Add derivation_slip10::tenderdash_node_id as the single audited primitive (SHA256(pubkey)[0..20]). - Key the ProviderPlatformKeys pool entries by node id: the DerivedKey::EdDSA branch of AddressPool::generate_address_at_index and EdDSAAccount::derive_address_at now wrap the node id (not hash160) in the P2PKH-style payload the ProRegTx matcher compares against reg.platform_node_id. - SLIP-0010 fingerprints/identifier() are intentionally unchanged: BIP32 key identifiers are spec-defined as hash160 and never compared against on-chain fields. Golden vectors (cross-checked with an independent Ed25519 implementation, pyca/cryptography): platform node key 0 for the shared test mnemonic has pubkey 3130c143...8cf789 and node id 302f2615e6955cce8ed3cff81e8011bfd3a2991f. Tests pin the primitive and the pool payload end-to-end; without the fix the pool payload is the hash160 value eaa83b51... and the new tests fail. Closes #883 Co-Authored-By: Claude Fable 5 * fix(key-wallet): gate DerivedKey::EdDSA on the eddsa feature derivation_slip10 is compiled only with the eddsa feature, but the DerivedKey::EdDSA match arm referencing tenderdash_node_id was unconditional, breaking default-feature builds. Gate the variant and its arm like KeySource's EdDSA variants — the variant is only constructible from the (already gated) EdDSA key sources. Co-Authored-By: Claude Fable 5 * test(key-wallet): cover derive_address_at node-id path with golden vector Codecov flagged the EdDSAAccount::derive_address_at change as uncovered. Pin it to the same independently computed Tenderdash node id as the pool-level tests so both implementations of the convention are locked to the vector. Co-Authored-By: Claude Fable 5 * refactor(key-wallet): use infallible conversions for node-id bytes Address review: PubkeyHash::from_byte_array instead of from_slice().expect(), and copy_from_slice instead of try_into().expect() in tenderdash_node_id — no panics in library code. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- key-wallet/src/account/eddsa_account.rs | 60 +++++++++++++---- key-wallet/src/derivation_slip10.rs | 19 ++++++ .../src/managed_account/address_pool.rs | 23 +++++-- .../tests/provider_key_derivation_tests.rs | 64 +++++++++++++++++++ .../transaction_checking/account_checker.rs | 4 +- 5 files changed, 153 insertions(+), 17 deletions(-) diff --git a/key-wallet/src/account/eddsa_account.rs b/key-wallet/src/account/eddsa_account.rs index bff1c5cc8..6ad5b2490 100644 --- a/key-wallet/src/account/eddsa_account.rs +++ b/key-wallet/src/account/eddsa_account.rs @@ -329,9 +329,13 @@ impl )) } - /// Derive an Ed25519-based address at a specific chain and index. + /// Derive an Ed25519-based pseudo-address at a specific chain and index. /// - /// Creates a P2PKH-style address from the hash160 of the Ed25519 public key. + /// The payload is the Tenderdash node ID of the Ed25519 public key + /// (`SHA256(pubkey)[0..20]`, the CometBFT convention) wrapped in a + /// P2PKH-style address — the same value ProRegTx carries as + /// `platform_node_id`, so it can be matched against on-chain evonode + /// registrations. NOT hash160, which Dash only uses for ECDSA key hashes. fn derive_address_at( &self, address_pool_type: AddressPoolType, @@ -342,17 +346,11 @@ impl let ed25519_pubkey = self.derive_public_key_at(address_pool_type, index, use_hardened_with_priv_key)?; - // Get the Ed25519 public key bytes (32 bytes for Ed25519) - let pubkey_bytes = ed25519_pubkey.to_bytes(); + let node_id = crate::derivation_slip10::tenderdash_node_id(&ed25519_pubkey.to_bytes()); - // Create a P2PKH address from the hash160 of the Ed25519 public key - // This uses the same hash160 (SHA256 + RIPEMD160) as ECDSA addresses - use dashcore::hashes::{hash160, Hash}; - let pubkey_hash = hash160::Hash::hash(&pubkey_bytes); - - // Create the address from the public key hash use dashcore::address::Payload; - let payload = Payload::PubkeyHash(pubkey_hash.into()); + use dashcore::hashes::Hash; + let payload = Payload::PubkeyHash(dashcore::PubkeyHash::from_byte_array(node_id)); Ok(Address::new(self.network, payload)) } @@ -534,6 +532,46 @@ mod tests { assert!(result.is_err()); } + /// `derive_address_at` must produce Tenderdash-node-id payloads + /// (SHA256(pubkey)[0..20]) — pinned to the same golden vector as + /// `tests::provider_key_derivation_tests` (platform node key 0 at + /// m/9'/5'/3'/4'/0' for the BIP39 "abandon…about" seed, cross-checked + /// with an independent Ed25519 implementation). + #[test] + fn test_derive_address_at_uses_tenderdash_node_id() { + use dashcore::hashes::Hash; + + let seed = hex::decode( + "5eb00bbddcf069084889a8ab9155568165f5c453ccb85e70811aaed6f6da5fc1\ + 9a5ac40b389cd370d086206dec8aa6c43daea6690f20ad3d8d48b2d2ce9e38e4", + ) + .unwrap(); + let account = EdDSAAccount::from_seed( + None, + AccountType::ProviderPlatformKeys, + &seed, + Network::Mainnet, + ) + .expect("create platform keys account from seed"); + + let master = + ExtendedEd25519PrivKey::new_master(Network::Mainnet, &seed).expect("master from seed"); + let account_xpriv = master + .derive_priv( + &AccountType::ProviderPlatformKeys.derivation_path(Network::Mainnet).unwrap(), + ) + .expect("derive account xpriv"); + + let address = account + .derive_address_at(AddressPoolType::AbsentHardened, 0, Some(account_xpriv)) + .expect("derive platform node pseudo-address"); + + let dashcore::address::Payload::PubkeyHash(hash) = address.payload() else { + panic!("platform pseudo-addresses use P2PKH-style payloads"); + }; + assert_eq!(hex::encode(hash.to_byte_array()), "302f2615e6955cce8ed3cff81e8011bfd3a2991f"); + } + #[test] fn test_derive_identity_key() { let seed = [5u8; 32]; diff --git a/key-wallet/src/derivation_slip10.rs b/key-wallet/src/derivation_slip10.rs index 804154003..40926ce1b 100644 --- a/key-wallet/src/derivation_slip10.rs +++ b/key-wallet/src/derivation_slip10.rs @@ -25,6 +25,25 @@ pub use dashcore::ed25519_dalek::VerifyingKey as Ed25519PublicKey; // Use DerivationPath from bip32 pub use crate::bip32::DerivationPath; +/// Compute the Tenderdash/CometBFT node ID for an Ed25519 public key: the +/// first 20 bytes of a single SHA-256 of the 32-byte public key. +/// +/// This is the canonical value of the `platform_node_id` field in ProRegTx / +/// ProUpServTx payloads for evonodes: it is what dashmate derives from +/// `node_key.json` and what Tenderdash announces on the platform P2P network, +/// so it is the only value that can match an on-chain registration. +/// +/// Note this is **not** `hash160` (SHA-256 + RIPEMD-160). Dash uses hash160 +/// for the ECDSA owner/voting key hashes in the same payload, but the +/// platform node id follows the CometBFT node-ID convention. +pub fn tenderdash_node_id(ed25519_public_key_bytes: &[u8; 32]) -> [u8; 20] { + use dashcore_hashes::sha256; + let hash = sha256::Hash::hash(ed25519_public_key_bytes); + let mut node_id = [0u8; 20]; + node_id.copy_from_slice(&hash.as_byte_array()[..20]); + node_id +} + /// Extended Ed25519 private key for SLIP-0010 #[derive(Clone)] pub struct ExtendedEd25519PrivKey { diff --git a/key-wallet/src/managed_account/address_pool.rs b/key-wallet/src/managed_account/address_pool.rs index 020c3f8f2..0bb38e061 100644 --- a/key-wallet/src/managed_account/address_pool.rs +++ b/key-wallet/src/managed_account/address_pool.rs @@ -122,6 +122,7 @@ pub enum DerivedKey { /// BLS public key (48 bytes) BLS(Vec), /// EdDSA public key (32 bytes) + #[cfg(feature = "eddsa")] EdDSA(Vec), } @@ -490,14 +491,26 @@ impl AddressPool { (address, PublicKeyType::BLS(public_key_bytes)) } + #[cfg(feature = "eddsa")] DerivedKey::EdDSA(public_key_bytes) => { - // EdDSA addresses use Hash160 of the public key bytes - use dashcore::hashes::{hash160, Hash}; - let pubkey_hash = hash160::Hash::hash(&public_key_bytes); + // EdDSA pool entries key on the Tenderdash node ID + // (SHA256(pubkey)[0..20]) — the value ProRegTx carries as + // `platform_node_id` — wrapped in a P2PKH-style payload so it + // can live in the pool's address index. NOT hash160: a + // hash160-keyed entry can never match an on-chain evonode + // registration. + let pubkey_arr: &[u8; 32] = + public_key_bytes.as_slice().try_into().map_err(|_| { + Error::InvalidParameter(format!( + "EdDSA public key must be 32 bytes, got {}", + public_key_bytes.len() + )) + })?; + let node_id = crate::derivation_slip10::tenderdash_node_id(pubkey_arr); - // Create P2PKH address from the hash use dashcore::address::Payload; - let payload = Payload::PubkeyHash(pubkey_hash.into()); + use dashcore::hashes::Hash; + let payload = Payload::PubkeyHash(dashcore::PubkeyHash::from_byte_array(node_id)); let address = Address::new(self.network, payload); (address, PublicKeyType::EdDSA(public_key_bytes)) diff --git a/key-wallet/src/tests/provider_key_derivation_tests.rs b/key-wallet/src/tests/provider_key_derivation_tests.rs index 4739095e3..4973e3180 100644 --- a/key-wallet/src/tests/provider_key_derivation_tests.rs +++ b/key-wallet/src/tests/provider_key_derivation_tests.rs @@ -125,6 +125,70 @@ fn ed25519_platform_node_keys_match_slip10_reference() { ); } +/// The platform node id must follow the Tenderdash/CometBFT convention — +/// `SHA256(pubkey)[0..20]` — not hash160, or wallet-side matching can never +/// hit on-chain evonode registrations. See +/// . +#[cfg(feature = "eddsa")] +#[test] +fn ed25519_platform_node_id_matches_tenderdash_convention() { + use crate::account::eddsa_account::EdDSAAccount; + use crate::derivation_slip10::tenderdash_node_id; + + // Platform node key 0 (m/9'/5'/3'/4'/0') for TEST_SEED — the same key the + // SLIP-0010 vector above pins. Public key and node id cross-checked with + // an independent Ed25519 implementation (pyca/cryptography). + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let sk0 = EdDSAAccount::platform_node_key_at(&seed, Network::Mainnet, 0).unwrap(); + let pubkey = sk0.verifying_key(); + assert_eq!( + hex::encode(pubkey.to_bytes()), + "3130c14339391cf26a68d86879e180ee9a16b660f5aa91f560f67c0abe8cf789" + ); + assert_eq!( + hex::encode(tenderdash_node_id(&pubkey.to_bytes())), + "302f2615e6955cce8ed3cff81e8011bfd3a2991f" + ); +} + +/// The managed ProviderPlatformKeys pool must key its entries by the +/// Tenderdash node id, since that payload is what ProRegTx matching compares +/// against the on-chain `platform_node_id` field. +#[cfg(feature = "eddsa")] +#[test] +fn platform_pool_entries_are_keyed_by_tenderdash_node_id() { + use crate::derivation_slip10::ExtendedEd25519PrivKey; + use crate::managed_account::managed_account_trait::ManagedAccountTrait; + use crate::wallet::managed_wallet_info::ManagedWalletInfo; + use dashcore::hashes::Hash; + + let wallet = test_wallet(Network::Mainnet); + let mut info = ManagedWalletInfo::from_wallet_with_name(&wallet, "node-id".to_string(), 0); + + let seed = hex::decode(TEST_SEED_HEX).unwrap(); + let master = ExtendedEd25519PrivKey::new_master(Network::Mainnet, &seed).unwrap(); + let account_xpriv = master + .derive_priv(&AccountType::ProviderPlatformKeys.derivation_path(Network::Mainnet).unwrap()) + .unwrap(); + + let (verifying_key, entry) = info + .provider_platform_keys_managed_account_mut() + .expect("platform managed account") + .next_eddsa_platform_key(account_xpriv, true) + .expect("derive platform key 0"); + + assert_eq!( + hex::encode(verifying_key.to_bytes()), + "3130c14339391cf26a68d86879e180ee9a16b660f5aa91f560f67c0abe8cf789" + ); + // The pool entry's payload is the Tenderdash node id — the value a real + // ProRegTx carries — so registration matching can hit it. + let dashcore::address::Payload::PubkeyHash(hash) = entry.address.payload() else { + panic!("platform pool entries use P2PKH-style payloads"); + }; + assert_eq!(hex::encode(hash.to_byte_array()), "302f2615e6955cce8ed3cff81e8011bfd3a2991f"); +} + /// The gate-free provider-key entry points must work identically on resident /// (non-watch-only) and watch-only accounts, and must match the dashbls /// reference vectors. See diff --git a/key-wallet/src/transaction_checking/account_checker.rs b/key-wallet/src/transaction_checking/account_checker.rs index bbff4ba8f..e5a4f7290 100644 --- a/key-wallet/src/transaction_checking/account_checker.rs +++ b/key-wallet/src/transaction_checking/account_checker.rs @@ -1042,7 +1042,9 @@ impl ManagedCoreFundsAccount { _ => return None, }; - // Check if platform_node_id matches any of our address hashes + // Check if platform_node_id matches any of our pool entries. + // Pool payloads hold Tenderdash node ids (SHA256(pubkey)[..20]), + // matching the on-chain field's convention. for (address, &addr_index) in &addresses.address_index { if let Payload::PubkeyHash(addr_hash) = address.payload() { if addr_hash == platform_node_id { From 2045cd29efbb323df9ec6c23f542ae5b12389b0e Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 13:06:58 +0700 Subject: [PATCH 049/149] feat(dash): dedicated PlatformNodeId newtype for platform_node_id fields (#885) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(dash): dedicated PlatformNodeId newtype for platform_node_id fields The platform_node_id on ProRegTx was typed Option (and raw Option<[u8; 20]> on ProUpServTx), but the value is a Tenderdash/CometBFT node ID — SHA256(ed25519_pubkey) truncated to 20 bytes — not a hash160 public key hash. Sharing the PubkeyHash type is what let the two conventions get conflated in the first place (#883, fixed in #884). Introduce dashcore::PlatformNodeId, a 20-byte newtype with hex Display/FromStr, serde, bincode, and consensus encoding, plus from_ed25519_public_key encoding the canonical CometBFT derivation. Use it for platform_node_id on ProviderRegistrationPayload, ProviderUpdateServicePayload, and EntryMasternodeType::HighPerformance. All wire/persisted encodings stay byte-identical: consensus encoding is the same raw 20 bytes (existing test vectors still pass, and the v2 ProRegTx round-trip test now also asserts the decoded node-ID value), and a regression test locks the bincode layout to the old PubkeyHash layout so persisted dash-spv masternode-list snapshots keep decoding. key-wallet's platform-key matchers now compare raw bytes explicitly, making the hash160-vs-node-ID distinction visible at the comparison site; behavior is unchanged. The key-wallet-ffi C ABI is untouched. Follow-up to #883. Co-Authored-By: Claude Fable 5 * refactor(key-wallet): delegate tenderdash_node_id to PlatformNodeId PR #884 added derivation_slip10::tenderdash_node_id and this branch added PlatformNodeId::from_ed25519_public_key — two copies of the same SHA256(pubkey)[0..20] derivation. Keep the dash-crate newtype as the single audited implementation and delegate the key-wallet helper to it; the #884 golden-vector tests pin the output unchanged. Co-Authored-By: Claude Fable 5 * test(dash): cover PlatformNodeId hex/serde paths and zero-fill encoding Close the patch-coverage gaps codecov flagged: hex Display/FromStr round-trip (including wrong-length and non-hex failures), serde JSON round-trip as a hex string, and the zero-filled wire encoding of v2 high-performance ProRegTx / ProUpServTx payloads whose platform fields are None. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- .../provider_registration.rs | 64 +++++++- .../provider_update_service.rs | 51 +++++-- dash/src/lib.rs | 2 + dash/src/platform_node_id.rs | 142 ++++++++++++++++++ dash/src/sml/masternode_list_entry/mod.rs | 6 +- key-wallet-ffi/src/special_payload.rs | 2 +- key-wallet/src/derivation_slip10.rs | 6 +- .../special_transaction_matching_tests.rs | 6 +- .../src/tests/special_transaction_tests.rs | 6 +- .../transaction_checking/account_checker.rs | 5 +- .../transaction_router/tests/provider.rs | 9 +- 11 files changed, 263 insertions(+), 36 deletions(-) create mode 100644 dash/src/platform_node_id.rs diff --git a/dash/src/blockdata/transaction/special_transaction/provider_registration.rs b/dash/src/blockdata/transaction/special_transaction/provider_registration.rs index 95dda9157..0ba8c75e8 100644 --- a/dash/src/blockdata/transaction/special_transaction/provider_registration.rs +++ b/dash/src/blockdata/transaction/special_transaction/provider_registration.rs @@ -46,6 +46,7 @@ use crate::blockdata::transaction::special_transaction::SpecialTransactionBasePa use crate::bls_sig_utils::BLSPublicKey; use crate::consensus::{Decodable, Encodable, encode}; use crate::hash_types::{InputsHash, PubkeyHash, SpecialTransactionPayloadHash}; +use crate::platform_node_id::PlatformNodeId; use crate::prelude::*; use crate::{Address, Network, OutPoint, ScriptBuf, VarInt, io}; @@ -108,7 +109,11 @@ pub struct ProviderRegistrationPayload { pub script_payout: ScriptBuf, pub inputs_hash: InputsHash, pub signature: Vec, - pub platform_node_id: Option, + /// Tenderdash/CometBFT node ID of the masternode's Platform node: + /// `SHA256(ed25519_public_key)` truncated to 20 bytes. Not a `hash160` + /// public key hash. Only present for high-performance (Evo) masternodes + /// on payload version 2+. + pub platform_node_id: Option, pub platform_p2p_port: Option, pub platform_http_port: Option, } @@ -131,7 +136,7 @@ impl ProviderRegistrationPayload { script_payout: ScriptBuf, inputs_hash: InputsHash, signature: Vec, - platform_node_id: Option, + platform_node_id: Option, platform_p2p_port: Option, platform_http_port: Option, ) -> Self { @@ -221,10 +226,7 @@ impl SpecialTransactionBasePayloadEncodable for ProviderRegistrationPayload { len += self.inputs_hash.consensus_encode(&mut s)?; if self.version >= 2 && self.masternode_type == ProviderMasternodeType::HighPerformance { - len += self - .platform_node_id - .unwrap_or_else(PubkeyHash::all_zeros) - .consensus_encode(&mut s)?; + len += self.platform_node_id.unwrap_or_default().consensus_encode(&mut s)?; len += self.platform_p2p_port.unwrap_or_default().consensus_encode(&mut s)?; len += self.platform_http_port.unwrap_or_default().consensus_encode(&mut s)?; } @@ -267,7 +269,7 @@ impl Decodable for ProviderRegistrationPayload { let mut platform_http_port = None; if version >= 2 && provider_type == ProviderMasternodeType::HighPerformance { - platform_node_id = Some(PubkeyHash::consensus_decode(r)?); + platform_node_id = Some(PlatformNodeId::consensus_decode(r)?); platform_p2p_port = Some(u16::consensus_decode(r)?); platform_http_port = Some(u16::consensus_decode(r)?); } @@ -721,6 +723,41 @@ mod tests { assert_eq!(actual, want); } + /// A v2 high-performance payload with absent platform fields still + /// writes the zero-filled node id and ports on the wire. + #[test] + fn v2_high_performance_none_platform_fields_encode_zero_filled() { + let payload = ProviderRegistrationPayload { + version: 2, + masternode_type: ProviderMasternodeType::HighPerformance, + masternode_mode: 0, + collateral_outpoint: OutPoint { + txid: Txid::all_zeros(), + vout: 0, + }, + service_address: SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::from_bits(0), 0)), + owner_key_hash: PubkeyHash::all_zeros(), + operator_public_key: BLSPublicKey::from([0; 48]), + voting_key_hash: PubkeyHash::all_zeros(), + operator_reward: 0, + script_payout: ScriptBuf::new(), + inputs_hash: InputsHash::all_zeros(), + signature: vec![], + platform_node_id: None, + platform_p2p_port: None, + platform_http_port: None, + }; + + let mut encoded = Vec::new(); + payload.consensus_encode(&mut encoded).unwrap(); + + let decoded: ProviderRegistrationPayload = + deserialize(&encoded).expect("deserialize zero-filled payload"); + assert_eq!(decoded.platform_node_id, Some(crate::PlatformNodeId::from_byte_array([0; 20]))); + assert_eq!(decoded.platform_p2p_port, Some(0)); + assert_eq!(decoded.platform_http_port, Some(0)); + } + #[test] fn test_payload_version_2_encoding_and_decoding() { let payload_bytes = hex!( @@ -729,6 +766,19 @@ mod tests { let payload: ProviderRegistrationPayload = deserialize(&payload_bytes).expect("deserialize payload"); + assert_eq!(payload.version, 2); + assert_eq!(payload.masternode_type, ProviderMasternodeType::HighPerformance); + assert_eq!( + payload.platform_node_id, + Some( + "4cd2ca50b36e0a2bb1b6b29da140448b47eeb7a1" + .parse() + .expect("valid platform node id hex") + ) + ); + assert_eq!(payload.platform_p2p_port, Some(26656)); + assert_eq!(payload.platform_http_port, Some(443)); + let mut serialized_payload_bytes = Vec::new(); payload.consensus_encode(&mut serialized_payload_bytes).expect("serialize payload"); diff --git a/dash/src/blockdata/transaction/special_transaction/provider_update_service.rs b/dash/src/blockdata/transaction/special_transaction/provider_update_service.rs index 1d9fccb10..6068886b8 100644 --- a/dash/src/blockdata/transaction/special_transaction/provider_update_service.rs +++ b/dash/src/blockdata/transaction/special_transaction/provider_update_service.rs @@ -43,6 +43,7 @@ use crate::blockdata::transaction::special_transaction::provider_registration::P use crate::bls_sig_utils::BLSSignature; use crate::consensus::{Decodable, Encodable, encode}; use crate::hash_types::{InputsHash, SpecialTransactionPayloadHash, Txid}; +use crate::platform_node_id::PlatformNodeId; use crate::{ScriptBuf, VarInt, io}; /// ProTx version constants @@ -68,8 +69,10 @@ pub struct ProviderUpdateServicePayload { pub port: u16, pub script_payout: ScriptBuf, pub inputs_hash: InputsHash, - // Platform fields (only for BasicBLS version and Evo masternode type) - pub platform_node_id: Option<[u8; 20]>, + // Platform fields (only for BasicBLS version and Evo masternode type). + // The node ID is a Tenderdash/CometBFT node ID (SHA256 of the ed25519 + // public key truncated to 20 bytes), not a hash160 public key hash. + pub platform_node_id: Option, pub platform_p2p_port: Option, pub platform_http_port: Option, pub payload_sig: BLSSignature, @@ -88,7 +91,7 @@ impl ProviderUpdateServicePayload { port: u16, script_payout: ScriptBuf, inputs_hash: InputsHash, - platform_node_id: Option<[u8; 20]>, + platform_node_id: Option, platform_p2p_port: Option, platform_http_port: Option, payload_sig: BLSSignature, @@ -147,7 +150,7 @@ impl SpecialTransactionBasePayloadEncodable for ProviderUpdateServicePayload { if self.version >= ProTxVersion::BasicBLS as u16 && self.mn_type == Some(ProviderMasternodeType::HighPerformance as u16) { - len += s.write(&self.platform_node_id.unwrap_or([0u8; 20]))?; + len += self.platform_node_id.unwrap_or_default().consensus_encode(&mut s)?; len += self.platform_p2p_port.unwrap_or_default().consensus_encode(&mut s)?; len += self.platform_http_port.unwrap_or_default().consensus_encode(&mut s)?; } @@ -199,11 +202,7 @@ impl Decodable for ProviderUpdateServicePayload { == ProTxVersion::BasicBLS as u16 && mn_type == Some(ProviderMasternodeType::HighPerformance as u16) { - let node_id = { - let mut buf = [0u8; 20]; - r.read_exact(&mut buf)?; - buf - }; + let node_id = PlatformNodeId::consensus_decode(r)?; let p2p_port = u16::consensus_decode(r)?; let http_port = u16::consensus_decode(r)?; (Some(node_id), Some(p2p_port), Some(http_port)) @@ -244,6 +243,7 @@ mod tests { use crate::consensus::{Decodable, Encodable, deserialize}; use crate::hash_types::InputsHash; use crate::internal_macros::hex; + use crate::platform_node_id::PlatformNodeId; use crate::{Network, ScriptBuf, Transaction, Txid}; #[test] @@ -406,7 +406,7 @@ mod tests { port: 0, script_payout: ScriptBuf::from(vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 0]), inputs_hash: InputsHash::all_zeros(), - platform_node_id: Some([0; 20]), + platform_node_id: Some(PlatformNodeId::from_byte_array([0; 20])), platform_p2p_port: Some(0), platform_http_port: Some(0), payload_sig: BLSSignature::from([0; 96]), @@ -422,6 +422,37 @@ mod tests { assert_eq!(decoded, original); } + /// A v2 Evo payload with absent platform fields still writes the + /// zero-filled node id and ports on the wire. + #[test] + fn v2_evo_masternode_none_platform_fields_encode_zero_filled() { + let original = ProviderUpdateServicePayload { + version: 2, + mn_type: Some(1), // HighPerformance (Evo) + pro_tx_hash: Txid::all_zeros(), + ip_address: 0, + port: 0, + script_payout: ScriptBuf::new(), + inputs_hash: InputsHash::all_zeros(), + platform_node_id: None, + platform_p2p_port: None, + platform_http_port: None, + payload_sig: BLSSignature::from([0; 96]), + }; + + let mut encoded = Vec::new(); + original.consensus_encode(&mut encoded).unwrap(); + + // version(2) + mn_type(2) + pro_tx_hash(32) + ip(16) + port(2) + + // script(1) + inputs_hash(32) + node_id(20) + p2p(2) + http(2) + sig(96) + assert_eq!(encoded.len(), 207); + + let decoded = ProviderUpdateServicePayload::consensus_decode(&mut &encoded[..]).unwrap(); + assert_eq!(decoded.platform_node_id, Some(PlatformNodeId::from_byte_array([0; 20]))); + assert_eq!(decoded.platform_p2p_port, Some(0)); + assert_eq!(decoded.platform_http_port, Some(0)); + } + #[test] fn test_protx_update_v2_block_parsing() { use crate::blockdata::block::Block; diff --git a/dash/src/lib.rs b/dash/src/lib.rs index 1b74873db..85b527b89 100644 --- a/dash/src/lib.rs +++ b/dash/src/lib.rs @@ -107,6 +107,7 @@ pub mod ephemerealdata; pub mod error; pub mod hash_types; pub mod merkle_tree; +pub mod platform_node_id; pub mod policy; pub mod pow; pub mod sign_message; @@ -142,6 +143,7 @@ pub use crate::hash_types::{ }; pub use crate::merkle_tree::MerkleBlock; pub use crate::network::constants::{Network, ParseNetworkError}; +pub use crate::platform_node_id::PlatformNodeId; pub use crate::pow::{CompactTarget, Target, Work}; pub use crate::transaction::outpoint::OutPoint; pub use crate::transaction::txin::TxIn; diff --git a/dash/src/platform_node_id.rs b/dash/src/platform_node_id.rs new file mode 100644 index 000000000..764d400d7 --- /dev/null +++ b/dash/src/platform_node_id.rs @@ -0,0 +1,142 @@ +// Rust Dash Library +// Written by +// The Rust Dash developers +// +// To the extent possible under law, the author(s) have dedicated all +// copyright and related and neighboring rights to this software to +// the public domain worldwide. This software is distributed without +// any warranty. +// +// You should have received a copy of the CC0 Public Domain Dedication +// along with this software. +// If not, see . +// + +//! Dash Platform (Tenderdash/CometBFT) node ID. +//! +//! A platform node ID is the first 20 bytes of `SHA256(ed25519_public_key)`, +//! following the CometBFT node-ID convention. Although it is 20 bytes on the +//! wire like a [`PubkeyHash`](crate::PubkeyHash), it is *not* a `hash160` +//! public key hash, so it gets its own type to keep the two conventions from +//! being conflated. + +#[cfg(feature = "bincode")] +use bincode::{Decode, Encode}; +use hashes::{Hash, sha256}; +use internals::impl_array_newtype; + +use crate::consensus::{Decodable, Encodable, encode}; +use crate::internal_macros::impl_bytes_newtype; +use crate::io; + +/// A Dash Platform (Tenderdash/CometBFT) node ID. +/// +/// This is `SHA256(ed25519_public_key)` truncated to 20 bytes, not a `hash160` +/// public key hash. The consensus encoding is the raw 20 bytes, identical to +/// the encoding of a 20-byte key hash. +#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Default)] +#[cfg_attr(feature = "bincode", derive(Encode, Decode))] +pub struct PlatformNodeId([u8; 20]); + +impl_array_newtype!(PlatformNodeId, u8, 20); +impl_bytes_newtype!(PlatformNodeId, 20); + +impl PlatformNodeId { + /// Constructs a platform node ID from its raw 20 bytes. + pub const fn from_byte_array(bytes: [u8; 20]) -> Self { + PlatformNodeId(bytes) + } + + /// Returns the raw 20 bytes of the node ID. + pub const fn to_byte_array(self) -> [u8; 20] { + self.0 + } + + /// Returns a reference to the raw 20 bytes of the node ID. + pub const fn as_byte_array(&self) -> &[u8; 20] { + &self.0 + } + + /// Derives the node ID from an Ed25519 public key per the CometBFT + /// convention: `SHA256(public_key)[0..20]`. + pub fn from_ed25519_public_key(public_key_bytes: &[u8; 32]) -> Self { + let digest = sha256::Hash::hash(public_key_bytes); + let mut bytes = [0u8; 20]; + bytes.copy_from_slice(&digest.to_byte_array()[..20]); + PlatformNodeId(bytes) + } +} + +impl Encodable for PlatformNodeId { + fn consensus_encode(&self, w: &mut W) -> Result { + self.0.consensus_encode(w) + } +} + +impl Decodable for PlatformNodeId { + fn consensus_decode(r: &mut R) -> Result { + Ok(PlatformNodeId(<[u8; 20]>::consensus_decode(r)?)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::consensus::{deserialize, serialize}; + + #[test] + fn consensus_round_trip() { + let node_id = PlatformNodeId::from_byte_array([0xAB; 20]); + let encoded = serialize(&node_id); + assert_eq!(encoded.len(), 20); + assert_eq!(encoded, [0xAB; 20]); + let decoded: PlatformNodeId = deserialize(&encoded).expect("decode node id"); + assert_eq!(decoded, node_id); + } + + /// Persisted masternode-list snapshots were written while + /// `platform_node_id` was typed as `PubkeyHash`; the bincode layout must + /// stay identical so old snapshots keep decoding. + #[cfg(feature = "bincode")] + #[test] + fn bincode_layout_matches_pubkey_hash() { + use hashes::Hash; + + let bytes = [0xCD; 20]; + let config = bincode::config::standard(); + let node_id_bytes = bincode::encode_to_vec(PlatformNodeId::from_byte_array(bytes), config) + .expect("encode node id"); + let pubkey_hash_bytes = + bincode::encode_to_vec(crate::PubkeyHash::from_byte_array(bytes), config) + .expect("encode pubkey hash"); + assert_eq!(node_id_bytes, pubkey_hash_bytes); + } + + #[test] + fn hex_display_and_parse_round_trip() { + let hex = "4cd2ca50b36e0a2bb1b6b29da140448b47eeb7a1"; + let node_id: PlatformNodeId = hex.parse().expect("parse node id hex"); + assert_eq!(node_id.to_string(), hex); + assert_eq!(format!("{:?}", node_id), hex); + assert!("abcd".parse::().is_err(), "wrong-length hex must fail"); + assert!("zz".repeat(20).parse::().is_err(), "non-hex input must fail"); + } + + #[cfg(feature = "serde")] + #[test] + fn serde_json_round_trip_as_hex_string() { + let node_id = PlatformNodeId::from_byte_array([0x11; 20]); + let json = serde_json::to_string(&node_id).expect("serialize node id"); + assert_eq!(json, format!("\"{}\"", "11".repeat(20))); + let back: PlatformNodeId = serde_json::from_str(&json).expect("deserialize node id"); + assert_eq!(back, node_id); + } + + #[test] + fn from_ed25519_public_key_is_truncated_sha256() { + let public_key = [7u8; 32]; + let digest = sha256::Hash::hash(&public_key); + let node_id = PlatformNodeId::from_ed25519_public_key(&public_key); + assert_eq!(node_id.as_byte_array()[..], digest.to_byte_array()[..20]); + } +} diff --git a/dash/src/sml/masternode_list_entry/mod.rs b/dash/src/sml/masternode_list_entry/mod.rs index e26197fe6..6291b463c 100644 --- a/dash/src/sml/masternode_list_entry/mod.rs +++ b/dash/src/sml/masternode_list_entry/mod.rs @@ -20,7 +20,7 @@ use crate::internal_macros::impl_consensus_encoding; use crate::sml::masternode_list_entry::net_info::{ Bip155Network, ExtNetInfo, NetInfoEntry, NetInfoPurpose, }; -use crate::{ProTxHash, PubkeyHash}; +use crate::{PlatformNodeId, ProTxHash, PubkeyHash}; #[derive(Clone, Ord, PartialOrd, Eq, PartialEq, Debug)] #[cfg_attr(feature = "bincode", derive(Encode, Decode))] @@ -29,7 +29,9 @@ pub enum EntryMasternodeType { Regular, HighPerformance { platform_http_port: u16, - platform_node_id: PubkeyHash, + /// Tenderdash/CometBFT node ID (`SHA256(ed25519_pubkey)[0..20]`), + /// not a `hash160` public key hash. + platform_node_id: PlatformNodeId, }, } diff --git a/key-wallet-ffi/src/special_payload.rs b/key-wallet-ffi/src/special_payload.rs index 33cfc81aa..527de521a 100644 --- a/key-wallet-ffi/src/special_payload.rs +++ b/key-wallet-ffi/src/special_payload.rs @@ -223,7 +223,7 @@ impl From<&ProviderUpdateServicePayload> for FFIProviderUpdateServicePayload { script_payout, script_payout_len, has_platform_fields: p.platform_node_id.is_some(), - platform_node_id: p.platform_node_id.unwrap_or([0; 20]), + platform_node_id: p.platform_node_id.map(|id| id.to_byte_array()).unwrap_or([0; 20]), platform_p2p_port: p.platform_p2p_port.map_or(-1, i32::from), platform_http_port: p.platform_http_port.map_or(-1, i32::from), } diff --git a/key-wallet/src/derivation_slip10.rs b/key-wallet/src/derivation_slip10.rs index 40926ce1b..ad3883630 100644 --- a/key-wallet/src/derivation_slip10.rs +++ b/key-wallet/src/derivation_slip10.rs @@ -37,11 +37,7 @@ pub use crate::bip32::DerivationPath; /// for the ECDSA owner/voting key hashes in the same payload, but the /// platform node id follows the CometBFT node-ID convention. pub fn tenderdash_node_id(ed25519_public_key_bytes: &[u8; 32]) -> [u8; 20] { - use dashcore_hashes::sha256; - let hash = sha256::Hash::hash(ed25519_public_key_bytes); - let mut node_id = [0u8; 20]; - node_id.copy_from_slice(&hash.as_byte_array()[..20]); - node_id + dashcore::PlatformNodeId::from_ed25519_public_key(ed25519_public_key_bytes).to_byte_array() } /// Extended Ed25519 private key for SLIP-0010 diff --git a/key-wallet/src/tests/special_transaction_matching_tests.rs b/key-wallet/src/tests/special_transaction_matching_tests.rs index bd2d2876e..27a8dcda6 100644 --- a/key-wallet/src/tests/special_transaction_matching_tests.rs +++ b/key-wallet/src/tests/special_transaction_matching_tests.rs @@ -278,7 +278,7 @@ fn prov_reg_tx( voting_key_hash: dashcore::PubkeyHash, operator_public_key: dashcore::bls_sig_utils::BLSPublicKey, script_payout: ScriptBuf, - platform_node_id: Option, + platform_node_id: Option, ) -> Transaction { Transaction { version: 3, @@ -552,7 +552,9 @@ async fn provider_registration_with_platform_node_id_matches_provider_platform_k .expect("provider_platform_keys managed") .next_eddsa_platform_key(eddsa, true) .expect("derive platform"); - let platform_node_id = derive_pubkey_hash(&platform_info.address); + let platform_node_id = dashcore::PlatformNodeId::from_byte_array( + derive_pubkey_hash(&platform_info.address).to_byte_array(), + ); let tx = prov_reg_tx( ProviderMasternodeType::HighPerformance, diff --git a/key-wallet/src/tests/special_transaction_tests.rs b/key-wallet/src/tests/special_transaction_tests.rs index d88d7bc5a..06d3c84f3 100644 --- a/key-wallet/src/tests/special_transaction_tests.rs +++ b/key-wallet/src/tests/special_transaction_tests.rs @@ -11,7 +11,7 @@ use dashcore::blockdata::transaction::special_transaction::{ use dashcore::bls_sig_utils::{BLSPublicKey, BLSSignature}; use dashcore::hash_types::{InputsHash, PubkeyHash}; use dashcore::hashes::Hash; -use dashcore::{OutPoint, ScriptBuf, Transaction, TxIn, TxOut, Txid}; +use dashcore::{OutPoint, PlatformNodeId, ScriptBuf, Transaction, TxIn, TxOut, Txid}; use std::net::SocketAddr; /// Special transaction types in Dash @@ -206,7 +206,7 @@ fn create_special_transaction(tx_type: SpecialTransactionType) -> Transaction { script_payout: ScriptBuf::new(), inputs_hash: InputsHash::from_byte_array([6u8; 32]), signature: vec![7u8; 96], - platform_node_id: Some(PubkeyHash::from_byte_array([8u8; 20])), + platform_node_id: Some(PlatformNodeId::from_byte_array([8u8; 20])), platform_p2p_port: Some(26656), platform_http_port: Some(443), }; @@ -231,7 +231,7 @@ fn create_special_transaction(tx_type: SpecialTransactionType) -> Transaction { port: 19999, script_payout: ScriptBuf::new(), inputs_hash: InputsHash::from_byte_array([10u8; 32]), - platform_node_id: Some([12u8; 20]), + platform_node_id: Some(PlatformNodeId::from_byte_array([12u8; 20])), platform_p2p_port: Some(26656), platform_http_port: Some(443), payload_sig: BLSSignature::from([11u8; 96]), diff --git a/key-wallet/src/transaction_checking/account_checker.rs b/key-wallet/src/transaction_checking/account_checker.rs index e5a4f7290..e1a907d8b 100644 --- a/key-wallet/src/transaction_checking/account_checker.rs +++ b/key-wallet/src/transaction_checking/account_checker.rs @@ -14,6 +14,7 @@ use crate::managed_account::transaction_record::TransactionRecord; use crate::Address; use dashcore::address::Payload; use dashcore::blockdata::transaction::Transaction; +use dashcore::hashes::Hash as _; use dashcore::transaction::TransactionPayload; use dashcore::ScriptBuf; @@ -1047,7 +1048,7 @@ impl ManagedCoreFundsAccount { // matching the on-chain field's convention. for (address, &addr_index) in &addresses.address_index { if let Payload::PubkeyHash(addr_hash) = address.payload() { - if addr_hash == platform_node_id { + if addr_hash.as_byte_array() == platform_node_id.as_byte_array() { // Get the address info if let Some(address_info) = addresses.addresses.get(&addr_index) { return Some(AccountMatch { @@ -1517,7 +1518,7 @@ impl crate::managed_account::ManagedCoreKeysAccount { for (address, &addr_index) in &addresses.address_index { if let Payload::PubkeyHash(addr_hash) = address.payload() { - if addr_hash == platform_node_id { + if addr_hash.as_byte_array() == platform_node_id.as_byte_array() { if let Some(address_info) = addresses.addresses.get(&addr_index) { return Some(AccountMatch { account_type_match: diff --git a/key-wallet/src/transaction_checking/transaction_router/tests/provider.rs b/key-wallet/src/transaction_checking/transaction_router/tests/provider.rs index d9cb4c350..a2ca67f3b 100644 --- a/key-wallet/src/transaction_checking/transaction_router/tests/provider.rs +++ b/key-wallet/src/transaction_checking/transaction_router/tests/provider.rs @@ -688,12 +688,13 @@ async fn test_provider_registration_transaction_routing_check_platform_only() { inputs_hash: dashcore::hash_types::InputsHash::from_slice(&[6u8; 32]) .expect("Failed to create inputs hash from bytes"), signature: vec![7u8; 65], // Simplified signature - platform_node_id: Some( - *platform_node_id + platform_node_id: Some(dashcore::PlatformNodeId::from_byte_array( + platform_node_id .payload() .as_pubkey_hash() - .expect("Platform node ID address should be P2PKH"), - ), + .expect("Platform node ID address should be P2PKH") + .to_byte_array(), + )), platform_p2p_port: Some(26656), platform_http_port: Some(8080), }, From ac8828c9ed025f57d9c9ca7c5356cfd50848e833 Mon Sep 17 00:00:00 2001 From: Ivan Shumkov Date: Tue, 14 Jul 2026 17:20:41 +0700 Subject: [PATCH 050/149] feat(key-wallet): add missing DIP-15 derivation constants to dip9 (#886) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet): add the DIP-15 auto-accept feature constant to dip9 Every other DIP-9-family feature index (identities 5' with its sub-features, coinjoin 4', dashpay 15', platform payment 17') is defined in key_wallet::dip9, but the DIP-15 auto-accept feature 16' (m/9'/coin'/16'/expiry', the expiry-bounded bearer-key family behind the DashPay auto-accept QR) was never added. Downstream consumers — the platform wallet's auto_accept_derivation_path and the FFI resolver signer's raw-key export gate in dashpay/platform — each carry their own local `16` literal as a result. Define FEATURE_PURPOSE_DASHPAY_AUTO_ACCEPT = 16 next to its siblings so those call sites can reference the canonical constant. Co-Authored-By: Claude Fable 5 * feat(key-wallet): add the DIP-15 contactInfo child constants to dip9 Same gap as the auto-accept feature: the DIP-15 contactInfo hardened children (encToUserId = 65536', privateData = 65537' under the identity- auth root) had no canonical constants, so dashpay/platform carries them twice — named consts in rs-platform-wallet's crypto/contact_info.rs and raw 65536/65537 literals in rs-sdk-ffi's resolver signer (which cannot import platform-wallet). Define them beside the other DIP-15 constants. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- key-wallet/src/dip9.rs | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/key-wallet/src/dip9.rs b/key-wallet/src/dip9.rs index 276972df7..da8a26aee 100644 --- a/key-wallet/src/dip9.rs +++ b/key-wallet/src/dip9.rs @@ -136,6 +136,21 @@ pub const FEATURE_PURPOSE_IDENTITIES_SUBFEATURE_INVITATIONS: u32 = 3; pub const FEATURE_PURPOSE_ASSET_LOCK_SUBFEATURE_ADDRESS_TOPUP: u32 = 4; pub const FEATURE_PURPOSE_ASSET_LOCK_SUBFEATURE_SHIELDED_ADDRESS_TOPUP: u32 = 5; pub const FEATURE_PURPOSE_DASHPAY: u32 = 15; +/// DIP-15 auto-accept feature index: the derivation family +/// `m/9'/coin_type'/16'/expiry'` holding the shareable, expiry-bounded +/// bearer keys that a DashPay user embeds in an auto-accept QR (`dapk`) so a +/// scanned contact request auto-establishes. Consumers (the platform wallet's +/// auto-accept path builder and the FFI signer's raw-key export gate) must +/// reference this constant rather than a local literal. +pub const FEATURE_PURPOSE_DASHPAY_AUTO_ACCEPT: u32 = 16; +/// DIP-15 contactInfo `encToUserId` hardened child (`root / 65536' / index'`, +/// where `root` is the owner's identity-authentication path): derives the +/// AES key that encrypts the contact id in a `contactInfo` document. +pub const DASHPAY_CONTACT_INFO_ENC_TO_USER_ID_CHILD: u32 = 1 << 16; +/// DIP-15 contactInfo `privateData` hardened child (`root / 65537' / index'`): +/// derives the AES key that encrypts the private-data blob in a `contactInfo` +/// document. +pub const DASHPAY_CONTACT_INFO_PRIVATE_DATA_CHILD: u32 = (1 << 16) + 1; /// DIP-17: Platform Payment Addresses feature index pub const FEATURE_PURPOSE_PLATFORM_PAYMENT: u32 = 17; pub const DASH_BIP44_PATH_MAINNET: IndexConstPath<2> = IndexConstPath { From 7221d866954be113888db7fdb36923a8e1d20c04 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 17:44:08 +0700 Subject: [PATCH 051/149] fix(dash): normalize PlatformNodeId to canonical byte order at consensus boundary (#889) PlatformNodeId held bytes in two different orders depending on how it was constructed: consensus_decode kept the wire bytes as-is (Dash Core serializes the field as a uint160, whose internal blob order is reversed), while from_ed25519_public_key produced the canonical forward Tenderdash node id (SHA256(pubkey)[0..20]). Ids from the two paths never compared equal, so wallet-owned platform node ids could never be recognized from a decoded ProRegTx, and Display of a decoded id showed the reversed form that matches neither dashmate nor explorers. Normalize at the consensus boundary so the newtype always holds canonical forward bytes: consensus_decode reverses the wire bytes into canonical order and consensus_encode reverses back to wire order. from_ed25519_public_key, Display, FromHex, and the bincode layout are unchanged. Fixes #887 Co-authored-by: Claude Fable 5 --- .../provider_registration.rs | 5 +- dash/src/platform_node_id.rs | 62 ++++++++++++++++--- 2 files changed, 57 insertions(+), 10 deletions(-) diff --git a/dash/src/blockdata/transaction/special_transaction/provider_registration.rs b/dash/src/blockdata/transaction/special_transaction/provider_registration.rs index 0ba8c75e8..3c885245c 100644 --- a/dash/src/blockdata/transaction/special_transaction/provider_registration.rs +++ b/dash/src/blockdata/transaction/special_transaction/provider_registration.rs @@ -768,10 +768,13 @@ mod tests { assert_eq!(payload.version, 2); assert_eq!(payload.masternode_type, ProviderMasternodeType::HighPerformance); + // The wire carries `4cd2ca50b36e0a2bb1b6b29da140448b47eeb7a1` (Dash + // Core's internal uint160 blob order); the decoded id is the canonical + // Tenderdash form, i.e. the byte-reversal. assert_eq!( payload.platform_node_id, Some( - "4cd2ca50b36e0a2bb1b6b29da140448b47eeb7a1" + "a1b7ee478b4440a19db2b6b12b0a6eb350cad24c" .parse() .expect("valid platform node id hex") ) diff --git a/dash/src/platform_node_id.rs b/dash/src/platform_node_id.rs index 764d400d7..d01ef64a2 100644 --- a/dash/src/platform_node_id.rs +++ b/dash/src/platform_node_id.rs @@ -19,6 +19,12 @@ //! wire like a [`PubkeyHash`](crate::PubkeyHash), it is *not* a `hash160` //! public key hash, so it gets its own type to keep the two conventions from //! being conflated. +//! +//! The inner bytes are always held in canonical forward order — the order in +//! which Tenderdash and dashmate display and derive the id. Dash Core +//! serializes the field as a `uint160`, whose internal blob order is the +//! byte-reversal of the canonical form, so the consensus encoding reverses on +//! the way in and out. #[cfg(feature = "bincode")] use bincode::{Decode, Encode}; @@ -32,8 +38,9 @@ use crate::io; /// A Dash Platform (Tenderdash/CometBFT) node ID. /// /// This is `SHA256(ed25519_public_key)` truncated to 20 bytes, not a `hash160` -/// public key hash. The consensus encoding is the raw 20 bytes, identical to -/// the encoding of a 20-byte key hash. +/// public key hash. The inner bytes are always in canonical forward order +/// (as displayed by Tenderdash/dashmate); the consensus encoding is the +/// byte-reversal of that, matching Dash Core's internal `uint160` blob order. #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Default)] #[cfg_attr(feature = "bincode", derive(Encode, Decode))] pub struct PlatformNodeId([u8; 20]); @@ -42,17 +49,17 @@ impl_array_newtype!(PlatformNodeId, u8, 20); impl_bytes_newtype!(PlatformNodeId, 20); impl PlatformNodeId { - /// Constructs a platform node ID from its raw 20 bytes. + /// Constructs a platform node ID from its 20 canonical (forward-order) bytes. pub const fn from_byte_array(bytes: [u8; 20]) -> Self { PlatformNodeId(bytes) } - /// Returns the raw 20 bytes of the node ID. + /// Returns the 20 canonical (forward-order) bytes of the node ID. pub const fn to_byte_array(self) -> [u8; 20] { self.0 } - /// Returns a reference to the raw 20 bytes of the node ID. + /// Returns a reference to the 20 canonical (forward-order) bytes of the node ID. pub const fn as_byte_array(&self) -> &[u8; 20] { &self.0 } @@ -69,13 +76,19 @@ impl PlatformNodeId { impl Encodable for PlatformNodeId { fn consensus_encode(&self, w: &mut W) -> Result { - self.0.consensus_encode(w) + // The wire carries Dash Core's internal `uint160` blob order, which is + // the byte-reversal of the canonical form held in `self.0`. + let mut wire_bytes = self.0; + wire_bytes.reverse(); + wire_bytes.consensus_encode(w) } } impl Decodable for PlatformNodeId { fn consensus_decode(r: &mut R) -> Result { - Ok(PlatformNodeId(<[u8; 20]>::consensus_decode(r)?)) + let mut bytes = <[u8; 20]>::consensus_decode(r)?; + bytes.reverse(); + Ok(PlatformNodeId(bytes)) } } @@ -86,14 +99,45 @@ mod tests { #[test] fn consensus_round_trip() { - let node_id = PlatformNodeId::from_byte_array([0xAB; 20]); + let mut canonical = [0u8; 20]; + for (i, byte) in canonical.iter_mut().enumerate() { + *byte = i as u8; + } + let mut wire = canonical; + wire.reverse(); + + let node_id = PlatformNodeId::from_byte_array(canonical); let encoded = serialize(&node_id); assert_eq!(encoded.len(), 20); - assert_eq!(encoded, [0xAB; 20]); + assert_eq!(encoded, wire, "wire order is the byte-reversal of canonical order"); let decoded: PlatformNodeId = deserialize(&encoded).expect("decode node id"); assert_eq!(decoded, node_id); } + /// Mainnet evonode example from issue #887: the ProRegTx wire bytes carry + /// the `uint160` internal (reversed) order, while the canonical id — as + /// dashmate/Tenderdash derive and display it — is the byte-reversal. + #[test] + fn consensus_decode_yields_canonical_order() { + let wire_bytes = crate::internal_macros::hex!("8cb97997a418f4814a63d3564b9574a393437968"); + let node_id: PlatformNodeId = deserialize(&wire_bytes).expect("decode node id"); + assert_eq!(node_id.to_string(), "68794393a374954b56d3634a81f418a49779b98c"); + assert_eq!(serialize(&node_id), wire_bytes, "re-encoding restores wire order"); + } + + /// A node id decoded from the wire must compare equal to one derived from + /// the matching Ed25519 public key — the mismatch reported in issue #887. + #[test] + fn decoded_wire_id_matches_derived_id() { + let public_key = [7u8; 32]; + let derived = PlatformNodeId::from_ed25519_public_key(&public_key); + + let mut wire_bytes = derived.to_byte_array(); + wire_bytes.reverse(); + let decoded: PlatformNodeId = deserialize(&wire_bytes).expect("decode node id"); + assert_eq!(decoded, derived); + } + /// Persisted masternode-list snapshots were written while /// `platform_node_id` was typed as `PubkeyHash`; the bincode layout must /// stay identical so old snapshots keep decoding. From 38c689d97ddce0483d4ca455ade32efb019eb68d Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Tue, 14 Jul 2026 19:16:47 +0700 Subject: [PATCH 052/149] feat(key-wallet): stamp retained transaction records with their in-block position (#891) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes #890. Consumers aggregating a wallet's retained provider special transactions need to replay Core's same-block apply order (RebuildListFromBlock applies provider updates in block.vtx order), but the retained records carried no in-block index: BlockInfo held only height/hash/timestamp, account.transactions() iterates in txid order, and the index visible during block processing was discarded. - BlockInfo gains `position: Option` (serde default; None for contexts built without the containing block, e.g. FFI-supplied contexts and legacy persisted records), with a `position()` accessor and a `with_position` builder. `new()` keeps its 3-arg signature. - process_block_for_wallets enumerates block.txdata and stamps each record's context with its index. Chainlock promotion already copies the whole BlockInfo, so the position survives InBlock → InChainLockedBlock transitions. - Test pins that a two-tx block yields records stamped 0 and 1. Co-authored-by: Claude Fable 5 --- key-wallet-manager/src/event_tests.rs | 56 +++++++++++++++++++ key-wallet-manager/src/process_block.rs | 22 ++++---- .../transaction_context.rs | 25 +++++++++ 3 files changed, 93 insertions(+), 10 deletions(-) diff --git a/key-wallet-manager/src/event_tests.rs b/key-wallet-manager/src/event_tests.rs index 286896921..05d89c2f0 100644 --- a/key-wallet-manager/src/event_tests.rs +++ b/key-wallet-manager/src/event_tests.rs @@ -1283,3 +1283,59 @@ async fn test_block_processed_chainlocked_flag_matches_record_context() { assert!(matches!(inserted[0].context, TransactionContext::InBlock(_))); } } + +// --------------------------------------------------------------------------- +// In-block position stamping +// --------------------------------------------------------------------------- + +/// Records inserted by block processing must carry their `block.txdata` +/// index in `BlockInfo::position`, so consumers can replay Core's +/// same-block apply order (e.g. multiple provider special transactions +/// for one masternode in a single block resolve latest-wins by this +/// position in `RebuildListFromBlock`). +#[tokio::test] +async fn test_block_processing_stamps_in_block_position() { + let (mut manager, wallet_id, addr) = setup_manager_with_wallet(); + let mut rx = manager.subscribe_events(); + + let tx_first = create_tx_paying_to(&addr, 0xb1); + let tx_second = create_tx_paying_to(&addr, 0xb2); + let block = make_block(vec![tx_first.clone(), tx_second.clone()], 0xb1, 1500); + let wallets = BTreeSet::from([wallet_id]); + manager.process_block_for_wallets(&block, block.block_hash(), 300, &wallets).await; + + let events = drain_events(&mut rx); + let bp = events + .iter() + .find(|e| matches!(e, WalletEvent::BlockProcessed { .. })) + .expect("BlockProcessed expected"); + let WalletEvent::BlockProcessed { + inserted, + .. + } = bp + else { + unreachable!() + }; + assert_eq!(inserted.len(), 2, "both txs pay the wallet, got {inserted:?}"); + + let position_of = |txid| { + inserted + .iter() + .find(|r| r.txid == txid) + .expect("record for txid") + .context + .block_info() + .expect("confirmed record must carry block info") + .position() + }; + assert_eq!( + position_of(tx_first.txid()), + Some(0), + "first tx in block.txdata must be stamped position 0" + ); + assert_eq!( + position_of(tx_second.txid()), + Some(1), + "second tx in block.txdata must be stamped position 1" + ); +} diff --git a/key-wallet-manager/src/process_block.rs b/key-wallet-manager/src/process_block.rs index 1e2d511a7..11e59d74c 100644 --- a/key-wallet-manager/src/process_block.rs +++ b/key-wallet-manager/src/process_block.rs @@ -47,20 +47,22 @@ impl WalletInterface for WalletM && wallets.iter().filter_map(|wid| self.wallet_infos.get(wid)).all(|info| { info.last_applied_chain_lock().is_some_and(|cl| height <= cl.block_height) }); - let block_context = if all_chainlocked { - TransactionContext::InChainLockedBlock(info) - } else { - TransactionContext::InBlock(info) - }; - let mut per_wallet_inserted: BTreeMap> = BTreeMap::new(); let mut per_wallet_updated: BTreeMap> = BTreeMap::new(); let mut per_wallet_derived: BTreeMap> = BTreeMap::new(); - for tx in &block.txdata { - let check_result = self - .check_transaction_in_wallets(tx, block_context.clone(), wallets, true, false) - .await; + for (position, tx) in block.txdata.iter().enumerate() { + // Stamp each record with its `block.vtx` index so consumers + // can replay Core's same-block apply order (e.g. multiple + // provider updates for one masternode in a single block). + let tx_info = info.with_position(position as u32); + let block_context = if all_chainlocked { + TransactionContext::InChainLockedBlock(tx_info) + } else { + TransactionContext::InBlock(tx_info) + }; + let check_result = + self.check_transaction_in_wallets(tx, block_context, wallets, true, false).await; if !check_result.affected_wallets.is_empty() { if check_result.is_new_transaction { diff --git a/key-wallet/src/transaction_checking/transaction_context.rs b/key-wallet/src/transaction_checking/transaction_context.rs index 2377ecf19..b621532c3 100644 --- a/key-wallet/src/transaction_checking/transaction_context.rs +++ b/key-wallet/src/transaction_checking/transaction_context.rs @@ -11,6 +11,15 @@ pub struct BlockInfo { pub(crate) height: CoreBlockHeight, pub(crate) block_hash: BlockHash, pub(crate) timestamp: u32, + /// The transaction's index within `block.txdata` (`block.vtx` order), + /// when known. Same-block ordering matters for consumers that must + /// replay Core's apply-order semantics — e.g. multiple provider + /// special transactions for one masternode in a single block resolve + /// latest-wins by this position in `RebuildListFromBlock`. `None` + /// when the context was built without access to the containing block + /// (FFI-supplied contexts, legacy persisted records). + #[cfg_attr(feature = "serde", serde(default))] + pub(crate) position: Option, } impl BlockInfo { @@ -19,9 +28,19 @@ impl BlockInfo { height, block_hash, timestamp, + position: None, } } + /// Attach the transaction's in-block position (its index within + /// `block.txdata`). Chainlock promotion copies the whole `BlockInfo`, + /// so a position set at block-processing time survives context + /// transitions. + pub fn with_position(mut self, position: u32) -> Self { + self.position = Some(position); + self + } + pub fn height(&self) -> CoreBlockHeight { self.height } @@ -33,6 +52,12 @@ impl BlockInfo { pub fn timestamp(&self) -> u32 { self.timestamp } + + /// The transaction's index within its block (`block.vtx` order), when + /// it was recorded at block-processing time. + pub fn position(&self) -> Option { + self.position + } } /// Context for transaction processing From 19690d31b37ba12adc00f6bf2f8413d97e2b669d Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Wed, 15 Jul 2026 11:37:10 +0700 Subject: [PATCH 053/149] fix(dash-spv): don't preload filters below the actually stored range on filter-download start (#893) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(dash-spv): don't preload filters below the actually stored range on filter-download start FiltersManager::start_download treated stored_filters_tip — a tip watermark — as proof that filter storage is contiguous from scan_start. When only tip-region filters were ever stored (e.g. headers previously synced from a checkpoint above the wallet's birth height), the preload called load_filters over never-populated segments: a debug-assertion abort in SegmentCache::get_items (crash-loop on every launch, since the restart resumes the same scan state) and silent sentinel filter data zipped against real headers in release builds. - FilterStorage gains filter_start_height() (lowest stored height, backed by the SegmentCache start it already tracks) and clear_filters() (drop all filters; files removed on next persist). - start_download only treats the stored range as usable when it reaches down to scan_start. Otherwise the unreachable tip-region filters are discarded and the download restarts from scan_start exactly as if no filters were stored — resuming from their tip would leave a permanent hole below them that the in-order store loop can never fill. Preload and the initial batch's mark_verified are gated on the same check, and progress.stored_height is reset so batch scan gating no longer trusts the stale watermark. - SegmentCache::get_items now returns a typed InvalidArgument error for ranges that begin below the lowest stored height, replacing the debug-only panic / release sentinel reads for that case. Fixes #892 Co-Authored-By: Claude Fable 5 * fix(dash-spv): address CodeRabbit review on sparse-filter preload fix Three follow-ups to the #892 fix: - parse_segment_id: parse the entire component between the `{prefix}_` and `.{ext}` fixtures via strip_prefix/strip_suffix instead of a fixed 4-byte slice. Trailing junk (`segment_0000junk.dat`) is now rejected and ids wider than the zero-padding (`segment_100000.dat`) are accepted rather than silently truncated. Adds a dedicated parse_segment_id test covering round-trips, junk tails, 5+ digit ids, and malformed names. - SegmentCache::clear now returns StorageResult<()>: the directory scan runs to completion before any cache state is mutated, io NotFound on read_dir is treated as empty-ok, and every other error is propagated. A failed scan no longer leaves the cache reporting empty while persisted segment files survive to resurrect the sparse-storage bug after restart. The Result is threaded through clear_filters and its callers. - start_download gates stored_covers_scan on the stored start height alone (`stored_filters_start.is_some_and(|s| s <= scan_start)`). filter_tip_height collapses "empty" and "only height 0 stored" to 0, so the previous `stored_filters_tip > 0` guard misread a genesis-only store as empty and re-downloaded height 0 instead of preloading it. The discard branch is made consistent via `is_some()`. Adds a genesis-only restart test asserting the lone filter is preloaded with no clear and no re-request. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- dash-spv/src/storage/filters.rs | 81 +++++++++ dash-spv/src/storage/mod.rs | 8 + dash-spv/src/storage/segments.rs | 235 +++++++++++++++++++++++++-- dash-spv/src/sync/filters/manager.rs | 226 +++++++++++++++++++++++++- 4 files changed, 535 insertions(+), 15 deletions(-) diff --git a/dash-spv/src/storage/filters.rs b/dash-spv/src/storage/filters.rs index 37a0cd827..3578a26ea 100644 --- a/dash-spv/src/storage/filters.rs +++ b/dash-spv/src/storage/filters.rs @@ -21,6 +21,26 @@ pub trait FilterStorage: Send + Sync + 'static { async fn filter_tip_height(&self) -> StorageResult; + /// Lowest filter height present in storage, or `None` when no filters are + /// stored. + /// + /// `filter_tip_height` is a tip watermark only: storage is dense within + /// `[filter_start_height, filter_tip_height]`, but heights below the start + /// were never stored. Callers must not treat the tip as proof that storage + /// is contiguous from an arbitrary lower height. + async fn filter_start_height(&self) -> Option; + + /// Drop every stored filter, leaving the storage empty. + /// + /// Used when the stored range cannot serve the current scan (e.g. filters + /// were previously stored from a checkpoint above a wallet's rescan + /// start): keeping an unreachable island of tip-region filters would leave + /// a permanent hole below it that the contiguous store path cannot fill. + /// + /// Like `truncate_above`, the deletion is not durable until the next + /// successful `persist` call. + async fn clear_filters(&mut self) -> StorageResult<()>; + /// Drop all filters with `height > target_height`. /// /// Truncating above the current tip is a no-op, truncating below @@ -79,6 +99,14 @@ impl FilterStorage for PersistentFilterStorage { Ok(self.filters.read().await.tip_height().unwrap_or(0)) } + async fn filter_start_height(&self) -> Option { + self.filters.read().await.start_height() + } + + async fn clear_filters(&mut self) -> StorageResult<()> { + self.filters.write().await.clear() + } + async fn truncate_above(&mut self, target_height: u32) -> StorageResult<()> { self.filters.write().await.truncate_above(target_height).await } @@ -107,4 +135,57 @@ mod tests { assert_eq!(storage.filter_tip_height().await.unwrap(), 2); assert!(storage.load_filters(3..4).await.is_err()); } + + /// Filters stored only in a high region (as when headers previously + /// synced from a checkpoint): the start accessor reports the true lowest + /// stored height, reads below it fail loudly instead of returning + /// sentinels, and `clear_filters` durably empties the storage. + #[tokio::test] + async fn test_filter_start_height_and_clear_filters() { + let tmp_dir = TempDir::new().unwrap(); + let mut storage = PersistentFilterStorage::open(tmp_dir.path()).await.unwrap(); + + assert_eq!(storage.filter_start_height().await, None); + + for height in 900..=1000 { + storage.store_filter(height, &filter_bytes(height as u8)).await.unwrap(); + } + assert_eq!(storage.filter_start_height().await, Some(900)); + assert_eq!(storage.filter_tip_height().await.unwrap(), 1000); + + // Reads that begin below the stored range must error, not hand back + // sentinel data zipped against real headers. + assert!(storage.load_filters(100..200).await.is_err()); + assert!(storage.load_filters(850..950).await.is_err()); + assert_eq!(storage.load_filters(900..1001).await.unwrap().len(), 101); + + storage.persist(tmp_dir.path()).await.unwrap(); + let segment_file = + tmp_dir.path().join(PersistentFilterStorage::FOLDER_NAME).join("segment_0000.dat"); + assert!(segment_file.exists()); + + // The start height survives a reload. + let mut storage = PersistentFilterStorage::open(tmp_dir.path()).await.unwrap(); + assert_eq!(storage.filter_start_height().await, Some(900)); + + storage.clear_filters().await.unwrap(); + assert_eq!(storage.filter_start_height().await, None); + assert_eq!(storage.filter_tip_height().await.unwrap(), 0); + assert!(storage.load_filters(900..901).await.is_err()); + + storage.persist(tmp_dir.path()).await.unwrap(); + assert!(!segment_file.exists()); + + // Reopen: the clear is durable, and the storage accepts a fresh + // contiguous range from a lower start. + let mut storage = PersistentFilterStorage::open(tmp_dir.path()).await.unwrap(); + assert_eq!(storage.filter_start_height().await, None); + assert_eq!(storage.filter_tip_height().await.unwrap(), 0); + + for height in 100..=110 { + storage.store_filter(height, &filter_bytes(height as u8)).await.unwrap(); + } + assert_eq!(storage.filter_start_height().await, Some(100)); + assert_eq!(storage.load_filters(100..111).await.unwrap().len(), 11); + } } diff --git a/dash-spv/src/storage/mod.rs b/dash-spv/src/storage/mod.rs index 73135493b..248ab10d0 100644 --- a/dash-spv/src/storage/mod.rs +++ b/dash-spv/src/storage/mod.rs @@ -375,6 +375,14 @@ impl filters::FilterStorage for DiskStorageManager { self.filters.read().await.filter_tip_height().await } + async fn filter_start_height(&self) -> Option { + self.filters.read().await.filter_start_height().await + } + + async fn clear_filters(&mut self) -> StorageResult<()> { + self.filters.write().await.clear_filters().await + } + async fn truncate_above(&mut self, target_height: u32) -> StorageResult<()> { self.filters.write().await.truncate_above(target_height).await } diff --git a/dash-spv/src/storage/segments.rs b/dash-spv/src/storage/segments.rs index 49f291add..9521fcc81 100644 --- a/dash-spv/src/storage/segments.rs +++ b/dash-spv/src/storage/segments.rs @@ -107,16 +107,9 @@ impl SegmentCache { for entry in entries.flatten() { if let Some(name) = entry.file_name().to_str() { - if name.starts_with(I::SEGMENT_PREFIX) - && name.ends_with(&format!(".{}", I::DATA_FILE_EXTENSION)) - { - let segment_id_start = I::SEGMENT_PREFIX.len() + 1; - let segment_id_end = segment_id_start + 4; - - if let Ok(id) = name[segment_id_start..segment_id_end].parse::() { - max_seg_id = Some(max_seg_id.map_or(id, |max: u32| max.max(id))); - min_seg_id = Some(min_seg_id.map_or(id, |min: u32| min.min(id))); - } + if let Some(id) = Self::parse_segment_id(name) { + max_seg_id = Some(max_seg_id.map_or(id, |max: u32| max.max(id))); + min_seg_id = Some(min_seg_id.map_or(id, |min: u32| min.min(id))); } } } @@ -141,6 +134,21 @@ impl SegmentCache { Ok(cache) } + /// Parse the segment id out of a segment file name of the form + /// `{SEGMENT_PREFIX}_{id:04}.{DATA_FILE_EXTENSION}` (see + /// [`Persistable::segment_file_name`]). + /// + /// The entire remaining component between the `{prefix}_` and + /// `.{extension}` fixtures must parse as a `u32`, so trailing junk + /// (`segment_0000junk.dat`) is rejected and ids longer than the + /// zero-padding width (`segment_100000.dat`) are accepted, not truncated. + fn parse_segment_id(file_name: &str) -> Option { + let separator = format!("{}_", I::SEGMENT_PREFIX); + let suffix = format!(".{}", I::DATA_FILE_EXTENSION); + + file_name.strip_prefix(&separator)?.strip_suffix(&suffix)?.parse::().ok() + } + #[inline] fn height_to_segment_id(height: u32) -> u32 { height / Segment::::ITEMS_PER_SEGMENT @@ -226,6 +234,17 @@ impl SegmentCache { ))); } + // Reject ranges that begin below the lowest stored height: those slots + // were never populated and hold only sentinels. Without this guard the + // read would trip the `first_valid_offset` debug assertion below in + // debug builds, and silently hand back sentinel data in release builds. + if self.start_height.is_none_or(|lowest| start < lowest) { + return Err(StorageError::InvalidArgument(format!( + "get_items range {height_range:?} begins below the lowest stored height {:?}", + self.start_height + ))); + } + let mut items = Vec::with_capacity((end - start) as usize); let start_segment = Self::height_to_segment_id(start); @@ -429,6 +448,55 @@ impl SegmentCache { Ok(()) } + /// Drop every stored item, leaving the cache empty. + /// + /// All backing segment files — including segments not currently resident + /// in memory — are queued for deletion on the next `persist`. Like + /// `truncate_above`, the deletion is not durable until that `persist` + /// succeeds; a crash in between leaves the old files on disk and the + /// cache reopens with the pre-clear contents. + /// + /// The directory scan runs to completion *before* any cache state is + /// mutated: a failed scan must not leave the cache reporting empty while + /// persisted segment files survive on disk, since those would resurrect + /// after restart. A missing segments directory is treated as an empty + /// (already-cleared) cache; any other I/O error is propagated. + pub fn clear(&mut self) -> StorageResult<()> { + // Queue every on-disk segment file, discovered by directory scan since + // only up to MAX_ACTIVE_SEGMENTS of them are resident in memory. + let mut to_delete = HashSet::new(); + match fs::read_dir(&self.segments_dir) { + Ok(entries) => { + for entry in entries { + let entry = entry?; + if let Some(name) = entry.file_name().to_str() { + if let Some(id) = Self::parse_segment_id(name) { + to_delete.insert(id); + } + } + } + } + // No directory yet means nothing was ever persisted here. + Err(e) if e.kind() == std::io::ErrorKind::NotFound => {} + Err(e) => return Err(StorageError::Io(e)), + } + + // Scan succeeded — now it is safe to mutate cache state. Resident and + // evicted segments may be dirty and not persisted yet, so the scan + // above cannot see them. Queue their ids too; `persist` ignores + // missing files. + to_delete.extend(self.segments.keys().copied()); + to_delete.extend(self.evicted.keys().copied()); + + self.to_delete.extend(to_delete); + self.segments.clear(); + self.evicted.clear(); + self.tip_height = None; + self.start_height = None; + + Ok(()) + } + pub async fn persist(&mut self, segments_dir: impl Into) { let segments_dir = segments_dir.into(); @@ -1092,6 +1160,153 @@ mod tests { ); } + /// Ranges that begin below the lowest stored height read never-populated + /// sentinel slots and must fail with a typed error rather than panicking + /// on the `first_valid_offset` debug assertion (or silently returning + /// sentinels in release builds). + #[tokio::test] + async fn test_get_items_below_start_height_errors() { + let tmp_dir = TempDir::new().unwrap(); + + let items = FilterHeader::dummy_batch(0..10); + + let mut cache = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + cache.store_items_at_height(&items, 900).await.unwrap(); + assert_eq!(cache.start_height(), Some(900)); + + // Entirely below the stored range (the #892 shape: a scan start far + // below tip-region storage, within the same segment). + assert!(matches!(cache.get_items(100..200).await, Err(StorageError::InvalidArgument(_)))); + // Straddling the lowest stored height. + assert!(matches!(cache.get_items(895..905).await, Err(StorageError::InvalidArgument(_)))); + // The stored range itself is readable. + assert_eq!(cache.get_items(900..910).await.unwrap(), items); + + // Same result when the never-populated slots live in a lower segment + // than the stored data. + const ITEMS_PER_SEGMENT: u32 = Segment::::ITEMS_PER_SEGMENT; + let mut cache = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + cache.store_items_at_height(&items, ITEMS_PER_SEGMENT + 5).await.unwrap(); + assert!(matches!(cache.get_items(100..200).await, Err(StorageError::InvalidArgument(_)))); + + // An empty cache rejects every range. + let mut cache = + SegmentCache::::load_or_new(tmp_dir.path().join("empty")).await.unwrap(); + assert!(matches!(cache.get_items(0..1).await, Err(StorageError::InvalidArgument(_)))); + } + + #[tokio::test] + async fn test_segment_cache_clear() { + let tmp_dir = TempDir::new().unwrap(); + + const ITEMS_PER_SEGMENT: u32 = Segment::::ITEMS_PER_SEGMENT; + + // Two segments' worth of items, persisted to disk. + let items = FilterHeader::dummy_batch(0..ITEMS_PER_SEGMENT + 5); + let mut cache = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + cache.store_items_at_height(&items, 0).await.unwrap(); + cache.persist(tmp_dir.path()).await; + + let file_0 = tmp_dir.path().join(FilterHeader::segment_file_name(0)); + let file_1 = tmp_dir.path().join(FilterHeader::segment_file_name(1)); + assert!(file_0.exists()); + assert!(file_1.exists()); + + // Reopen (so only the min/max segments are resident) and clear. + let mut cache = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + cache.clear().unwrap(); + + assert_eq!(cache.tip_height(), None); + assert_eq!(cache.start_height(), None); + assert_eq!(cache.get_item(3).await.unwrap(), None); + assert!(cache.get_items(0..5).await.is_err()); + + // The deletion becomes durable on persist. + cache.persist(tmp_dir.path()).await; + assert!(!file_0.exists()); + assert!(!file_1.exists()); + + // Storing a fresh range from a lower origin after clear is sound. + let replacement = FilterHeader::dummy_batch(100..110); + cache.store_items_at_height(&replacement, 100).await.unwrap(); + assert_eq!(cache.start_height(), Some(100)); + assert_eq!(cache.tip_height(), Some(109)); + assert_eq!(cache.get_items(100..110).await.unwrap(), replacement); + + cache.persist(tmp_dir.path()).await; + let mut reloaded = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + assert_eq!(reloaded.start_height(), Some(100)); + assert_eq!(reloaded.tip_height(), Some(109)); + assert_eq!(reloaded.get_items(100..110).await.unwrap(), replacement); + } + + /// `clear` must also drop dirty in-memory state that was never persisted, + /// and queue previously-persisted files of those same segments. + #[tokio::test] + async fn test_segment_cache_clear_unpersisted_state() { + let tmp_dir = TempDir::new().unwrap(); + + let items = FilterHeader::dummy_batch(0..10); + let mut cache = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + cache.store_items_at_height(&items, 0).await.unwrap(); + cache.persist(tmp_dir.path()).await; + + // Dirty the resident segment again so it differs from disk, then clear + // without an intervening persist. + let more = FilterHeader::dummy_batch(100..105); + cache.store_items_at_height(&more, 10).await.unwrap(); + cache.clear().unwrap(); + + assert_eq!(cache.tip_height(), None); + assert_eq!(cache.start_height(), None); + + cache.persist(tmp_dir.path()).await; + assert!(!tmp_dir.path().join(FilterHeader::segment_file_name(0)).exists()); + + let reloaded = SegmentCache::::load_or_new(tmp_dir.path()).await.unwrap(); + assert_eq!(reloaded.tip_height(), None); + assert_eq!(reloaded.start_height(), None); + } + + #[test] + fn test_parse_segment_id() { + type Cache = SegmentCache; + + // Round-trips the writer's `{prefix}_{id:04}.{ext}` format for a range + // of ids, including ones wider than the zero-padding. + for id in [0u32, 1, 42, 9999, 10_000, 100_000, u32::MAX] { + assert_eq!(Cache::parse_segment_id(&FilterHeader::segment_file_name(id)), Some(id)); + } + + // Zero-padded low ids parse to their numeric value, not truncated. + assert_eq!(Cache::parse_segment_id("segment_0000.dat"), Some(0)); + assert_eq!(Cache::parse_segment_id("segment_0005.dat"), Some(5)); + + // Ids wider than the padding are accepted whole, never truncated to + // the first four digits. + assert_eq!(Cache::parse_segment_id("segment_100000.dat"), Some(100_000)); + + // Trailing junk between the id and the extension is rejected rather + // than silently parsed as a prefix of the component. + assert_eq!(Cache::parse_segment_id("segment_0000junk.dat"), None); + assert_eq!(Cache::parse_segment_id("segment_12ab.dat"), None); + + // Wrong prefix, wrong extension, missing separator, or a + // non-numeric/empty id component are all rejected. + assert_eq!(Cache::parse_segment_id("segment_0000.tmp"), None); + assert_eq!(Cache::parse_segment_id("other_0000.dat"), None); + assert_eq!(Cache::parse_segment_id("segment0000.dat"), None); + assert_eq!(Cache::parse_segment_id("segment_.dat"), None); + assert_eq!(Cache::parse_segment_id("segment_-1.dat"), None); + assert_eq!(Cache::parse_segment_id("segment_0000.dat.bak"), None); + + // The filter storage's item type round-trips through the same helper. + assert_eq!( + SegmentCache::>::parse_segment_id(&>::segment_file_name(7)), + Some(7) + ); + } + #[test] fn test_segment_insert_get() { let segment_id = 10; diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index 58192af13..b117b58d2 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -188,8 +188,17 @@ impl 0`) so this covers a lone above-scan filter + // stored at height 0, though that cannot arise while the start is also + // above `scan_start`. + if stored_filters_start.is_some() && !stored_covers_scan { + tracing::warn!( + "Stored filters {:?}..={} do not reach scan start {}; discarding them and re-downloading", + stored_filters_start, + stored_filters_tip, + scan_start + ); + self.filter_storage.write().await.clear_filters().await?; + self.progress.update_stored_height(0); + } + // Determine download start (where we need to download from) // Must be at least header_start_height for checkpoint-based sync - let download_start = if stored_filters_tip > 0 { + let download_start = if stored_covers_scan { (stored_filters_tip + 1).max(header_start_height) } else { scan_start @@ -268,7 +306,7 @@ impl 0 && scan_start <= stored_filters_tip { + let filters = if stored_covers_scan && scan_start <= stored_filters_tip { let end_height = stored_filters_tip.min(batch_end); tracing::info!( "Loading stored filters {} to {} into current batch", @@ -283,7 +321,7 @@ impl= batch_end { + if stored_covers_scan && stored_filters_tip >= batch_end { batch.mark_verified(); } self.active_batches.insert(scan_start, batch); @@ -2156,6 +2194,184 @@ mod tests { assert_eq!(batch.end_height(), 100); } + /// Reproduces #892: filters were only ever stored near a high tip (e.g. + /// headers previously synced from a checkpoint), and the wallet's scan + /// start is far below that region. `start_download` must not treat the + /// stored tip watermark as contiguous coverage from `scan_start`: + /// preloading `load_filters(scan_start, ..)` would read never-populated + /// segments (debug abort in `SegmentCache::get_items`, sentinel filter + /// data in release builds). The unreachable tip-region filters are + /// discarded and the download restarts from `scan_start`. + #[tokio::test] + async fn test_start_download_discards_stored_filters_above_scan_start() { + let mut manager = create_test_manager().await; + + // Block headers cover the whole range so send_pending can resolve stop hashes. + let headers = dashcore::block::Header::dummy_batch(0..1001); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); + + // Filters were only ever stored near the tip: 900..=1000. The heights + // below 900 were never populated. + { + let mut filter_storage = manager.filter_storage.write().await; + for height in 900..=1000u32 { + filter_storage.store_filter(height, &[height as u8; 8]).await.unwrap(); + } + } + // Restart shape: `new()` seeds stored_height from the tip watermark. + manager.progress.update_stored_height(1000); + manager.progress.update_filter_header_tip_height(1000); + manager.progress.update_target_height(1000); + + // Wallet committed far below the stored region, so scan_start = 100. + manager.wallet.write().await.update_wallet_synced_height(&MOCK_WALLET_ID, 99); + + let (tx, mut rx) = unbounded_channel(); + let events = manager.start_download(&RequestSender::new(tx)).await.unwrap(); + + assert!(events.is_empty()); + assert_eq!(manager.state(), SyncState::Syncing); + + // The unreachable tip-region filters were discarded entirely... + assert_eq!(manager.filter_storage.read().await.filter_tip_height().await.unwrap(), 0); + assert_eq!(manager.filter_storage.read().await.filter_start_height().await, None); + + // ...nothing was preloaded into the initial batch... + let batch = manager.active_batches.get(&100).expect("initial batch at scan_start"); + assert!(batch.filters().is_empty()); + assert!(!batch.verified()); + assert!(!batch.scanned()); + assert_eq!(batch.end_height(), 1000); + + // ...scan gating no longer sees the stale tip watermark... + assert_eq!(manager.progress.stored_height(), 0); + + // ...and both the store cursor and the download restart from + // scan_start rather than stored_filters_tip + 1. + assert_eq!(manager.next_batch_to_store, 100); + match rx.try_recv().expect("a filter request must have been sent") { + NetworkRequest::SendMessage(NetworkMessage::GetCFilters(gcf)) => { + assert_eq!(gcf.start_height, 100); + } + other => panic!("Expected GetCFilters, got {:?}", other), + } + } + + /// Counterpart to the sparse-storage case: when the stored filter range + /// actually reaches down to `scan_start`, the preload happens exactly as + /// before and nothing is discarded. + #[tokio::test] + async fn test_start_download_preloads_when_stored_filters_cover_scan_start() { + let mut manager = create_test_manager().await; + + let headers = dashcore::block::Header::dummy_batch(0..1001); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); + + // Filters stored densely from genesis through the tip. + { + let mut filter_storage = manager.filter_storage.write().await; + for height in 0..=1000u32 { + filter_storage.store_filter(height, &[height as u8; 8]).await.unwrap(); + } + } + manager.progress.update_stored_height(1000); + manager.progress.update_filter_header_tip_height(1000); + manager.progress.update_target_height(1000); + + manager.wallet.write().await.update_wallet_synced_height(&MOCK_WALLET_ID, 99); + + let (tx, mut rx) = unbounded_channel(); + manager.start_download(&RequestSender::new(tx)).await.unwrap(); + + assert_eq!(manager.state(), SyncState::Syncing); + + // Storage is untouched. + assert_eq!(manager.filter_storage.read().await.filter_tip_height().await.unwrap(), 1000); + assert_eq!(manager.filter_storage.read().await.filter_start_height().await, Some(0)); + + // The stored range 100..=1000 was preloaded and the batch is verified + // and scanned immediately. + let batch = manager.active_batches.get(&100).expect("initial batch at scan_start"); + assert_eq!(batch.filters().len(), 901); + assert!(batch.verified()); + assert!(batch.scanned()); + assert_eq!(manager.progress.stored_height(), 1000); + + // Nothing left to download: everything through the filter header tip + // is already stored. + assert_eq!(manager.next_batch_to_store, 1001); + assert!(rx.try_recv().is_err(), "no filter request expected"); + } + + /// Genesis-only storage: a single filter at height 0, scanning from 0. + /// `filter_tip_height` collapses to 0 for both an empty store and this + /// one, so gating the preload on `tip > 0` would misread it as empty and + /// needlessly re-download height 0. The stored start (Some(0)) must drive + /// the decision: the filter is preloaded and nothing is discarded or + /// re-requested. Regtest can produce exactly this shape. + #[tokio::test] + async fn test_start_download_preloads_genesis_only_stored_filter() { + let mut manager = create_test_manager().await; + + let headers = dashcore::block::Header::dummy_batch(0..1); + manager + .header_storage + .write() + .await + .store_headers( + &headers.iter().map(crate::types::HashedBlockHeader::from).collect::>(), + ) + .await + .unwrap(); + + // Exactly one filter stored, at height 0. + manager.filter_storage.write().await.store_filter(0, &[0u8; 8]).await.unwrap(); + assert_eq!(manager.filter_storage.read().await.filter_tip_height().await.unwrap(), 0); + assert_eq!(manager.filter_storage.read().await.filter_start_height().await, Some(0)); + + // Restart shape at the genesis tip. + manager.progress.update_stored_height(0); + manager.progress.update_filter_header_tip_height(0); + manager.progress.update_target_height(0); + // Wallet at genesis: scan_start = 0. + + let (tx, mut rx) = unbounded_channel(); + manager.start_download(&RequestSender::new(tx)).await.unwrap(); + + // The lone filter was NOT discarded... + assert_eq!(manager.filter_storage.read().await.filter_tip_height().await.unwrap(), 0); + assert_eq!(manager.filter_storage.read().await.filter_start_height().await, Some(0)); + + // ...it was preloaded into the initial batch, which is verified and + // scanned since the whole (single-height) range is covered... + let batch = manager.active_batches.get(&0).expect("initial batch at scan_start"); + assert_eq!(batch.filters().len(), 1); + assert!(batch.verified()); + assert!(batch.scanned()); + assert_eq!(manager.progress.stored_height(), 0); + + // ...and the download frontier sits above the stored tip, so no + // filter request goes out for the already-stored genesis height. + assert_eq!(manager.next_batch_to_store, 1); + assert!(rx.try_recv().is_err(), "no filter request expected for the genesis-only store"); + } + #[tokio::test] async fn test_handle_new_filter_headers_transitions_synced_to_syncing() { let mut manager = create_test_manager().await; From 70d4bf8e36057c58e02d56769a6e9760f701dd06 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Wed, 22 Jul 2026 23:11:43 +0800 Subject: [PATCH 054/149] feat(dash-spv): track network acceptance of broadcast transactions (#913) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(dash-spv): track network acceptance of broadcast transactions Broadcasts are now sent to a subset of connected peers (default: half, per BroadcastHoldout::Half) while being withheld from the rest. Since peers never re-announce a transaction to whoever sent it to them, an inv for our txid from a withheld peer proves the transaction relayed through the network — the broadcast is then reported Accepted. InstantSend locks, block confirmations, and already-have rejects also count as acceptance; p2p reject messages (best-effort on modern dashd) report Rejected, distinguishing txn-mempool-conflict (rejection) from txn-already-in-mempool (acceptance) under the shared Duplicate code; a configurable timeout reports Uncertain, upgradeable by a late echo. Outcomes surface as SyncEvent::TransactionBroadcastResult, an awaitable DashSpvClient::broadcast_transaction_and_wait API, and matching FFI callbacks, config setters, and a blocking FFI wait function. Rebroadcast keeps respecting the holdout while pending, re-picks holdouts from never-sent peers on disconnect, skips rejected entries, and sends deferred broadcasts as soon as a peer connects. Broadcast state survives disconnects. Verified with a new two-node dashd integration test: the transaction is sent to one node and the withheld node's inv echo confirms acceptance; a double-spend leg documents that dashd 23 sends no BIP61 rejects and correctly resolves Uncertain. Co-Authored-By: Claude Fable 5 * docs(dash-spv-ffi): regenerate FFI_API.md for broadcast acceptance functions Co-Authored-By: Claude Fable 5 * refactor(dash-spv): drop BIP61 reject handling from broadcast acceptance Modern Dash Core removed the BIP61 reject message entirely (no enablebip61, NetMsgType::REJECT, or reject codes remain in dashpay/dash), and the two-node integration test confirmed dashd 23.1.0 sends nothing for a refused transaction. The reject path was therefore dead code: remove the Reject message subscription and handler, the BroadcastResult::Rejected and BroadcastStatus::Rejected variants, and the FFI reject fields (FFIBroadcastStatus is now Accepted=0/Uncertain=1, the callback and FFIBroadcastResult lose reject_code/reject_reason, and the result struct no longer owns strings so its destroy function is gone). A transaction the network refuses now uniformly surfaces as Uncertain: no echo arrives within the acceptance timeout. The integration test's double-spend leg asserts exactly that. Co-Authored-By: Claude Fable 5 * fix(dash-spv): address review feedback on broadcast await and FFI tests - broadcast_transaction_and_wait no longer resolves on the manager's interim Uncertain event: a late echo can still upgrade the outcome to Accepted, so callers with a timeout longer than the configured acceptance timeout keep waiting until their deadline. Uncertain is now only returned on deadline expiry (or bus close). - Add FFI unit tests for the broadcast config setters: zero threshold/timeout are rejected with InvalidArgument without modifying the config; valid values map onto the ClientConfig fields; holdout half/count setters round-trip. - Drop the obsolete "or rejection signal" wording from the timeout setter doc (BIP61 reject handling was removed) and regenerate FFI_API.md. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- dash-spv-ffi/FFI_API.md | 91 +- dash-spv-ffi/src/bin/ffi_cli.rs | 22 + dash-spv-ffi/src/callbacks.rs | 90 ++ dash-spv-ffi/src/client.rs | 83 ++ dash-spv-ffi/src/config.rs | 80 ++ dash-spv-ffi/tests/dashd_sync/callbacks.rs | 1 + dash-spv-ffi/tests/test_config.rs | 74 ++ .../tests/unit/test_async_operations.rs | 1 + dash-spv/src/client/config.rs | 54 ++ dash-spv/src/client/lifecycle.rs | 1 + dash-spv/src/client/transactions.rs | 93 +- dash-spv/src/lib.rs | 1 + dash-spv/src/network/mod.rs | 9 + dash-spv/src/sync/events.rs | 19 + dash-spv/src/sync/mempool/broadcast.rs | 174 ++++ dash-spv/src/sync/mempool/manager.rs | 818 +++++++++++++++--- dash-spv/src/sync/mempool/mod.rs | 2 + dash-spv/src/sync/mempool/sync_manager.rs | 53 +- dash-spv/src/sync/mod.rs | 2 +- dash-spv/tests/dashd_sync/tests_mempool.rs | 129 +++ 20 files changed, 1676 insertions(+), 121 deletions(-) create mode 100644 dash-spv/src/sync/mempool/broadcast.rs diff --git a/dash-spv-ffi/FFI_API.md b/dash-spv-ffi/FFI_API.md index 5b708ea8b..2e81eea7f 100644 --- a/dash-spv-ffi/FFI_API.md +++ b/dash-spv-ffi/FFI_API.md @@ -4,7 +4,7 @@ This document provides a comprehensive reference for all FFI (Foreign Function I **Auto-generated**: This documentation is automatically generated from the source code. Do not edit manually. -**Total Functions**: 39 +**Total Functions**: 44 ## Table of Contents @@ -31,7 +31,7 @@ Functions: 3 ### Configuration -Functions: 15 +Functions: 19 | Function | Description | Module | |----------|-------------|--------| @@ -41,6 +41,10 @@ Functions: 15 | `dash_spv_ffi_config_get_network` | Gets the network type from the configuration # Safety - `config` must be a... | config | | `dash_spv_ffi_config_mainnet` | No description | config | | `dash_spv_ffi_config_new` | No description | config | +| `dash_spv_ffi_config_set_broadcast_acceptance_threshold` | Sets how many distinct non-recipient peers must announce a broadcast txid... | config | +| `dash_spv_ffi_config_set_broadcast_acceptance_timeout_secs` | Sets the timeout (in seconds) after which a pending broadcast with no... | config | +| `dash_spv_ffi_config_set_broadcast_holdout_count` | Withholds broadcast transactions from a fixed number of peers (clamped so... | config | +| `dash_spv_ffi_config_set_broadcast_holdout_half` | Withholds broadcast transactions from half of the connected peers (the... | config | | `dash_spv_ffi_config_set_data_dir` | Sets the data directory for storing blockchain data # Safety - `config`... | config | | `dash_spv_ffi_config_set_fetch_mempool_transactions` | Sets whether to fetch full mempool transaction data # Safety - `config`... | config | | `dash_spv_ffi_config_set_masternode_sync_enabled` | Enables or disables masternode synchronization # Safety - `config` must be... | config | @@ -63,11 +67,12 @@ Functions: 3 ### Transaction Management -Functions: 1 +Functions: 2 | Function | Description | Module | |----------|-------------|--------| | `dash_spv_ffi_client_broadcast_transaction` | Broadcasts a transaction to the Dash network via connected peers | client | +| `dash_spv_ffi_client_broadcast_transaction_and_wait` | Broadcasts a transaction and waits for its network-level outcome | client | ### Mempool Operations @@ -252,6 +257,70 @@ dash_spv_ffi_config_new(network: FFINetwork) -> *mut FFIClientConfig --- +#### `dash_spv_ffi_config_set_broadcast_acceptance_threshold` + +```c +dash_spv_ffi_config_set_broadcast_acceptance_threshold(config: *mut FFIClientConfig, threshold: u32,) -> i32 +``` + +**Description:** +Sets how many distinct non-recipient peers must announce a broadcast txid back before it is reported as accepted. Must be > 0. # Safety - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Safety:** +- `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Module:** `config` + +--- + +#### `dash_spv_ffi_config_set_broadcast_acceptance_timeout_secs` + +```c +dash_spv_ffi_config_set_broadcast_acceptance_timeout_secs(config: *mut FFIClientConfig, seconds: u32,) -> i32 +``` + +**Description:** +Sets the timeout (in seconds) after which a pending broadcast with no acceptance signal is reported as uncertain. Must be > 0. # Safety - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Safety:** +- `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Module:** `config` + +--- + +#### `dash_spv_ffi_config_set_broadcast_holdout_count` + +```c +dash_spv_ffi_config_set_broadcast_holdout_count(config: *mut FFIClientConfig, count: u32,) -> i32 +``` + +**Description:** +Withholds broadcast transactions from a fixed number of peers (clamped so that at least one peer always receives the transaction). # Safety - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Safety:** +- `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Module:** `config` + +--- + +#### `dash_spv_ffi_config_set_broadcast_holdout_half` + +```c +dash_spv_ffi_config_set_broadcast_holdout_half(config: *mut FFIClientConfig,) -> i32 +``` + +**Description:** +Withholds broadcast transactions from half of the connected peers (the default policy). The withheld peers are the source of the `inv` echo that proves a broadcast propagated through the network. # Safety - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Safety:** +- `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet - The caller must ensure the config pointer remains valid for the duration of this call + +**Module:** `config` + +--- + #### `dash_spv_ffi_config_set_data_dir` ```c @@ -458,6 +527,22 @@ Broadcasts a transaction to the Dash network via connected peers. # Safety - ` --- +#### `dash_spv_ffi_client_broadcast_transaction_and_wait` + +```c +dash_spv_ffi_client_broadcast_transaction_and_wait(client: *mut FFIDashSpvClient, tx_bytes: *const u8, length: usize, timeout_secs: u32, out_result: *mut FFIBroadcastResult,) -> i32 +``` + +**Description:** +Broadcasts a transaction and waits for its network-level outcome. Blocks until the network accepts the transaction (non-recipient peers announce it back, it is InstantSend-locked, or confirmed) or the timeout elapses (outcome `Uncertain`). `timeout_secs == 0` uses the configured acceptance timeout plus a small grace period. Requires mempool tracking to be enabled in the client config. # Safety - `client` must be a valid, non-null pointer to an initialized FFIDashSpvClient - `tx_bytes` must be a valid, non-null pointer to the transaction data - `length` must be the length of the transaction data in bytes - `out_result` must be a valid, non-null pointer to an FFIBroadcastResult + +**Safety:** +- `client` must be a valid, non-null pointer to an initialized FFIDashSpvClient - `tx_bytes` must be a valid, non-null pointer to the transaction data - `length` must be the length of the transaction data in bytes - `out_result` must be a valid, non-null pointer to an FFIBroadcastResult + +**Module:** `client` + +--- + ### Mempool Operations - Detailed #### `dash_spv_ffi_mempool_progress_destroy` diff --git a/dash-spv-ffi/src/bin/ffi_cli.rs b/dash-spv-ffi/src/bin/ffi_cli.rs index fa32ddb87..38aeb8098 100644 --- a/dash-spv-ffi/src/bin/ffi_cli.rs +++ b/dash-spv-ffi/src/bin/ffi_cli.rs @@ -133,6 +133,27 @@ extern "C" fn on_sync_complete(header_tip: u32, cycle: u32, _user_data: *mut c_v println!("[Sync] Sync complete at height: {} (cycle {})", header_tip, cycle); } +extern "C" fn on_transaction_broadcast_result( + txid: *const [u8; 32], + status: FFIBroadcastStatus, + relayed_by: u32, + _user_data: *mut c_void, +) { + let txid_hex = unsafe { &*txid }.iter().rev().fold(String::new(), |mut acc, b| { + use std::fmt::Write; + let _ = write!(acc, "{:02x}", b); + acc + }); + match status { + FFIBroadcastStatus::Accepted => { + println!("[Broadcast] {} accepted ({} peer(s) relayed it back)", txid_hex, relayed_by) + } + FFIBroadcastStatus::Uncertain => { + println!("[Broadcast] {} outcome uncertain (no network signal)", txid_hex) + } + } +} + // ============================================================================ // Network Event Callbacks // ============================================================================ @@ -511,6 +532,7 @@ fn main() { on_instantlock_received: Some(on_instantlock_received), on_manager_error: Some(on_manager_error), on_sync_complete: Some(on_sync_complete), + on_transaction_broadcast_result: Some(on_transaction_broadcast_result), user_data: ptr::null_mut(), }, network: FFINetworkEventCallbacks { diff --git a/dash-spv-ffi/src/callbacks.rs b/dash-spv-ffi/src/callbacks.rs index ff1d1c765..c9fc45ff5 100644 --- a/dash-spv-ffi/src/callbacks.rs +++ b/dash-spv-ffi/src/callbacks.rs @@ -226,6 +226,35 @@ pub type OnManagerErrorCallback = pub type OnSyncCompleteCallback = Option; +/// Network-level outcome of a transaction broadcast. +/// +/// There is no rejected state: modern Dash Core removed the BIP61 `reject` +/// message, so the p2p network provides no negative signal — a transaction +/// the network refuses surfaces as `Uncertain` (no echo within the timeout). +#[repr(C)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum FFIBroadcastStatus { + /// Non-recipient peers announced the txid back (or it was + /// InstantSend-locked/confirmed) — the network accepted it. + Accepted = 0, + /// No acceptance signal arrived within the acceptance timeout. + Uncertain = 1, +} + +/// Callback for SyncEvent::TransactionBroadcastResult +/// +/// The `txid` pointer is borrowed and only valid for the duration of the +/// callback. `relayed_by` is the number of distinct non-recipient peers that +/// announced the txid back (meaningful for `Accepted`). +pub type OnTransactionBroadcastResultCallback = Option< + extern "C" fn( + txid: *const [u8; 32], + status: FFIBroadcastStatus, + relayed_by: u32, + user_data: *mut c_void, + ), +>; + /// Sync event callbacks - one callback per SyncEvent variant. /// /// Set only the callbacks you're interested in; unset callbacks will be ignored. @@ -250,6 +279,7 @@ pub struct FFISyncEventCallbacks { pub on_instantlock_received: OnInstantLockReceivedCallback, pub on_manager_error: OnManagerErrorCallback, pub on_sync_complete: OnSyncCompleteCallback, + pub on_transaction_broadcast_result: OnTransactionBroadcastResultCallback, pub user_data: *mut c_void, } @@ -282,6 +312,7 @@ impl Default for FFISyncEventCallbacks { on_instantlock_received: None, on_manager_error: None, on_sync_complete: None, + on_transaction_broadcast_result: None, user_data: std::ptr::null_mut(), } } @@ -438,6 +469,22 @@ impl FFISyncEventCallbacks { cb(*header_tip, *cycle, self.user_data); } } + SyncEvent::TransactionBroadcastResult { + txid, + result, + } => { + if let Some(cb) = self.on_transaction_broadcast_result { + use dash_spv::BroadcastResult; + let txid_bytes = txid.as_byte_array(); + let (status, relayed_by) = match result { + BroadcastResult::Accepted { + relayed_by, + } => (FFIBroadcastStatus::Accepted, *relayed_by as u32), + BroadcastResult::Uncertain => (FFIBroadcastStatus::Uncertain, 0), + }; + cb(txid_bytes as *const [u8; 32], status, relayed_by, self.user_data); + } + } } } } @@ -1370,4 +1417,47 @@ mod tests { }); assert_eq!(FIRED.load(Ordering::SeqCst), 0); } + + /// `TransactionBroadcastResult` dispatch must marshal the outcome fields + /// (status, relayed_by) for each variant. + #[test] + fn test_transaction_broadcast_result_dispatch() { + use dash_spv::BroadcastResult; + + static STATUS: AtomicU32 = AtomicU32::new(u32::MAX); + static RELAYED: AtomicU32 = AtomicU32::new(u32::MAX); + + extern "C" fn cb( + txid: *const [u8; 32], + status: FFIBroadcastStatus, + relayed_by: u32, + _user: *mut c_void, + ) { + assert!(!txid.is_null()); + STATUS.store(status as u32, Ordering::SeqCst); + RELAYED.store(relayed_by, Ordering::SeqCst); + } + + let callbacks = FFISyncEventCallbacks { + on_transaction_broadcast_result: Some(cb), + ..FFISyncEventCallbacks::default() + }; + let txid = Txid::from_byte_array([7u8; 32]); + + callbacks.dispatch(&SyncEvent::TransactionBroadcastResult { + txid, + result: BroadcastResult::Accepted { + relayed_by: 3, + }, + }); + assert_eq!(STATUS.load(Ordering::SeqCst), FFIBroadcastStatus::Accepted as u32); + assert_eq!(RELAYED.load(Ordering::SeqCst), 3); + + callbacks.dispatch(&SyncEvent::TransactionBroadcastResult { + txid, + result: BroadcastResult::Uncertain, + }); + assert_eq!(STATUS.load(Ordering::SeqCst), FFIBroadcastStatus::Uncertain as u32); + assert_eq!(RELAYED.load(Ordering::SeqCst), 0); + } } diff --git a/dash-spv-ffi/src/client.rs b/dash-spv-ffi/src/client.rs index 6cb94c8e2..33b334ef9 100644 --- a/dash-spv-ffi/src/client.rs +++ b/dash-spv-ffi/src/client.rs @@ -348,6 +348,89 @@ pub unsafe extern "C" fn dash_spv_ffi_client_broadcast_transaction( } } +/// Network-level outcome of a broadcast, as returned by +/// `dash_spv_ffi_client_broadcast_transaction_and_wait`. +/// +/// Plain value type — nothing to free. +#[repr(C)] +pub struct FFIBroadcastResult { + /// The determined outcome. + pub status: crate::FFIBroadcastStatus, + /// Distinct non-recipient peers that announced the txid back + /// (meaningful for `Accepted`). + pub relayed_by: u32, +} + +/// Broadcasts a transaction and waits for its network-level outcome. +/// +/// Blocks until the network accepts the transaction (non-recipient peers +/// announce it back, it is InstantSend-locked, or confirmed) or the timeout +/// elapses (outcome `Uncertain`). `timeout_secs == 0` uses the configured +/// acceptance timeout plus a small grace period. +/// +/// Requires mempool tracking to be enabled in the client config. +/// +/// # Safety +/// +/// - `client` must be a valid, non-null pointer to an initialized FFIDashSpvClient +/// - `tx_bytes` must be a valid, non-null pointer to the transaction data +/// - `length` must be the length of the transaction data in bytes +/// - `out_result` must be a valid, non-null pointer to an FFIBroadcastResult +#[no_mangle] +pub unsafe extern "C" fn dash_spv_ffi_client_broadcast_transaction_and_wait( + client: *mut FFIDashSpvClient, + tx_bytes: *const u8, + length: usize, + timeout_secs: u32, + out_result: *mut FFIBroadcastResult, +) -> i32 { + null_check!(client); + null_check!(tx_bytes); + null_check!(out_result); + + let tx_bytes = std::slice::from_raw_parts(tx_bytes, length); + + let tx = match dashcore::consensus::deserialize::(tx_bytes) { + Ok(t) => t, + Err(e) => { + set_last_error(&format!("Invalid transaction: {}", e)); + return FFIErrorCode::InvalidArgument as i32; + } + }; + + let client = &(*client); + let spv_client = client.inner.clone(); + let timeout = (timeout_secs > 0).then(|| std::time::Duration::from_secs(timeout_secs as u64)); + + let result = client + .runtime + .block_on(async { spv_client.broadcast_transaction_and_wait(&tx, timeout).await }); + + match result { + Ok(outcome) => { + use dash_spv::BroadcastResult; + let ffi = match outcome { + BroadcastResult::Accepted { + relayed_by, + } => FFIBroadcastResult { + status: crate::FFIBroadcastStatus::Accepted, + relayed_by: relayed_by as u32, + }, + BroadcastResult::Uncertain => FFIBroadcastResult { + status: crate::FFIBroadcastStatus::Uncertain, + relayed_by: 0, + }, + }; + std::ptr::write(out_result, ffi); + FFIErrorCode::Success as i32 + } + Err(e) => { + set_last_error(&format!("Failed to broadcast transaction: {}", e)); + FFIErrorCode::from(e) as i32 + } + } +} + /// Destroy the client and free associated resources. /// /// # Safety diff --git a/dash-spv-ffi/src/config.rs b/dash-spv-ffi/src/config.rs index d6d7bf581..849c0a87d 100644 --- a/dash-spv-ffi/src/config.rs +++ b/dash-spv-ffi/src/config.rs @@ -325,6 +325,86 @@ pub unsafe extern "C" fn dash_spv_ffi_config_set_fetch_mempool_transactions( FFIErrorCode::Success as i32 } +/// Withholds broadcast transactions from half of the connected peers +/// (the default policy). The withheld peers are the source of the `inv` +/// echo that proves a broadcast propagated through the network. +/// +/// # Safety +/// - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet +/// - The caller must ensure the config pointer remains valid for the duration of this call +#[no_mangle] +pub unsafe extern "C" fn dash_spv_ffi_config_set_broadcast_holdout_half( + config: *mut FFIClientConfig, +) -> i32 { + null_check!(config); + + let config = unsafe { &mut *((*config).inner as *mut ClientConfig) }; + config.broadcast_holdout = dash_spv::BroadcastHoldout::Half; + FFIErrorCode::Success as i32 +} + +/// Withholds broadcast transactions from a fixed number of peers (clamped so +/// that at least one peer always receives the transaction). +/// +/// # Safety +/// - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet +/// - The caller must ensure the config pointer remains valid for the duration of this call +#[no_mangle] +pub unsafe extern "C" fn dash_spv_ffi_config_set_broadcast_holdout_count( + config: *mut FFIClientConfig, + count: u32, +) -> i32 { + null_check!(config); + + let config = unsafe { &mut *((*config).inner as *mut ClientConfig) }; + config.broadcast_holdout = dash_spv::BroadcastHoldout::Count(count as usize); + FFIErrorCode::Success as i32 +} + +/// Sets how many distinct non-recipient peers must announce a broadcast txid +/// back before it is reported as accepted. Must be > 0. +/// +/// # Safety +/// - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet +/// - The caller must ensure the config pointer remains valid for the duration of this call +#[no_mangle] +pub unsafe extern "C" fn dash_spv_ffi_config_set_broadcast_acceptance_threshold( + config: *mut FFIClientConfig, + threshold: u32, +) -> i32 { + null_check!(config); + if threshold == 0 { + set_last_error("broadcast acceptance threshold must be > 0"); + return FFIErrorCode::InvalidArgument as i32; + } + + let config = unsafe { &mut *((*config).inner as *mut ClientConfig) }; + config.broadcast_acceptance_threshold = threshold as usize; + FFIErrorCode::Success as i32 +} + +/// Sets the timeout (in seconds) after which a pending broadcast with no +/// acceptance signal is reported as uncertain. Must be > 0. +/// +/// # Safety +/// - `config` must be a valid pointer to an FFIClientConfig created by dash_spv_ffi_config_new/mainnet/testnet +/// - The caller must ensure the config pointer remains valid for the duration of this call +#[no_mangle] +pub unsafe extern "C" fn dash_spv_ffi_config_set_broadcast_acceptance_timeout_secs( + config: *mut FFIClientConfig, + seconds: u32, +) -> i32 { + null_check!(config); + if seconds == 0 { + set_last_error("broadcast acceptance timeout must be > 0"); + return FFIErrorCode::InvalidArgument as i32; + } + + let config = unsafe { &mut *((*config).inner as *mut ClientConfig) }; + config.broadcast_acceptance_timeout = std::time::Duration::from_secs(seconds as u64); + FFIErrorCode::Success as i32 +} + // Checkpoint sync configuration functions /// Sets the starting block height for synchronization diff --git a/dash-spv-ffi/tests/dashd_sync/callbacks.rs b/dash-spv-ffi/tests/dashd_sync/callbacks.rs index a36b944ef..c5335dead 100644 --- a/dash-spv-ffi/tests/dashd_sync/callbacks.rs +++ b/dash-spv-ffi/tests/dashd_sync/callbacks.rs @@ -593,6 +593,7 @@ pub(super) fn create_sync_callbacks(tracker: &Arc) -> FFISyncEv on_instantlock_received: Some(on_instantlock_received), on_manager_error: Some(on_manager_error), on_sync_complete: Some(on_sync_complete), + on_transaction_broadcast_result: None, user_data: Arc::as_ptr(tracker) as *mut c_void, } } diff --git a/dash-spv-ffi/tests/test_config.rs b/dash-spv-ffi/tests/test_config.rs index 651d7201d..e58b6e2f7 100644 --- a/dash-spv-ffi/tests/test_config.rs +++ b/dash-spv-ffi/tests/test_config.rs @@ -107,4 +107,78 @@ mod tests { dash_spv_ffi_config_destroy(config); } } + + #[test] + #[serial] + fn test_config_broadcast_holdout_setters() { + unsafe { + let config = dash_spv_ffi_config_new(FFINetwork::Testnet); + + let result = dash_spv_ffi_config_set_broadcast_holdout_count(config, 2); + assert_eq!(result, FFIErrorCode::Success as i32); + assert_eq!( + (*config).get_inner().broadcast_holdout, + dash_spv::BroadcastHoldout::Count(2) + ); + + let result = dash_spv_ffi_config_set_broadcast_holdout_half(config); + assert_eq!(result, FFIErrorCode::Success as i32); + assert_eq!((*config).get_inner().broadcast_holdout, dash_spv::BroadcastHoldout::Half); + + dash_spv_ffi_config_destroy(config); + } + } + + #[test] + #[serial] + fn test_config_broadcast_acceptance_threshold() { + unsafe { + let config = dash_spv_ffi_config_new(FFINetwork::Testnet); + + // Zero is rejected and leaves the default intact + let default_threshold = (*config).get_inner().broadcast_acceptance_threshold; + let result = dash_spv_ffi_config_set_broadcast_acceptance_threshold(config, 0); + assert_eq!(result, FFIErrorCode::InvalidArgument as i32); + assert_eq!( + (*config).get_inner().broadcast_acceptance_threshold, + default_threshold, + "rejected value must not modify the config" + ); + + // Nonzero maps to the ClientConfig field + let result = dash_spv_ffi_config_set_broadcast_acceptance_threshold(config, 3); + assert_eq!(result, FFIErrorCode::Success as i32); + assert_eq!((*config).get_inner().broadcast_acceptance_threshold, 3); + + dash_spv_ffi_config_destroy(config); + } + } + + #[test] + #[serial] + fn test_config_broadcast_acceptance_timeout() { + unsafe { + let config = dash_spv_ffi_config_new(FFINetwork::Testnet); + + // Zero is rejected and leaves the default intact + let default_timeout = (*config).get_inner().broadcast_acceptance_timeout; + let result = dash_spv_ffi_config_set_broadcast_acceptance_timeout_secs(config, 0); + assert_eq!(result, FFIErrorCode::InvalidArgument as i32); + assert_eq!( + (*config).get_inner().broadcast_acceptance_timeout, + default_timeout, + "rejected value must not modify the config" + ); + + // Nonzero maps to the ClientConfig field + let result = dash_spv_ffi_config_set_broadcast_acceptance_timeout_secs(config, 90); + assert_eq!(result, FFIErrorCode::Success as i32); + assert_eq!( + (*config).get_inner().broadcast_acceptance_timeout, + std::time::Duration::from_secs(90) + ); + + dash_spv_ffi_config_destroy(config); + } + } } diff --git a/dash-spv-ffi/tests/unit/test_async_operations.rs b/dash-spv-ffi/tests/unit/test_async_operations.rs index 0a3d9613e..ccdf9c825 100644 --- a/dash-spv-ffi/tests/unit/test_async_operations.rs +++ b/dash-spv-ffi/tests/unit/test_async_operations.rs @@ -78,6 +78,7 @@ mod tests { on_instantlock_received: None, on_manager_error: None, on_sync_complete: Some(on_sync_complete), + on_transaction_broadcast_result: None, user_data: &event_data as *const _ as *mut c_void, }; diff --git a/dash-spv/src/client/config.rs b/dash-spv/src/client/config.rs index d14aa095f..84d54f5b3 100644 --- a/dash-spv/src/client/config.rs +++ b/dash-spv/src/client/config.rs @@ -3,11 +3,13 @@ use clap::ValueEnum; use std::net::SocketAddr; use std::path::PathBuf; +use std::time::Duration; use dashcore::Network; // Serialization removed due to complex Address types use crate::client::devnet::DevnetConfig; +use crate::sync::BroadcastHoldout; use crate::types::ValidationMode; /// Strategy for handling mempool (unconfirmed) transactions. @@ -68,6 +70,21 @@ pub struct ClientConfig { /// Whether to fetch transactions from INV messages immediately. pub fetch_mempool_transactions: bool, + /// How many peers to withhold broadcast transactions from. + /// + /// Peers never re-announce a transaction to whoever sent it to them, so + /// the withheld ("holdout") peers are the only source of the `inv` echo + /// that proves a broadcast propagated through the network. + pub broadcast_holdout: BroadcastHoldout, + + /// Distinct non-recipient peers that must announce a broadcast txid back + /// before the broadcast is reported as accepted. + pub broadcast_acceptance_threshold: usize, + + /// How long a broadcast may stay pending before its outcome is reported + /// as uncertain. + pub broadcast_acceptance_timeout: Duration, + /// Start syncing from a specific block height. /// The client will use the nearest checkpoint at or before this height. pub start_from_height: Option, @@ -93,6 +110,9 @@ impl Default for ClientConfig { mempool_strategy: MempoolStrategy::FetchAll, max_mempool_transactions: 1000, fetch_mempool_transactions: true, + broadcast_holdout: BroadcastHoldout::Half, + broadcast_acceptance_threshold: 1, + broadcast_acceptance_timeout: Duration::from_secs(60), start_from_height: None, devnet: None, } @@ -180,6 +200,33 @@ impl ClientConfig { self } + /// Set the broadcast holdout policy. + pub fn with_broadcast_holdout(mut self, holdout: BroadcastHoldout) -> Self { + self.broadcast_holdout = holdout; + self + } + + /// Set how many non-recipient peer announcements mark a broadcast accepted. + pub fn with_broadcast_acceptance_threshold(mut self, threshold: usize) -> Self { + self.broadcast_acceptance_threshold = threshold; + self + } + + /// Set the timeout after which a pending broadcast is reported uncertain. + pub fn with_broadcast_acceptance_timeout(mut self, timeout: Duration) -> Self { + self.broadcast_acceptance_timeout = timeout; + self + } + + /// Bundle the broadcast acceptance settings for the mempool manager. + pub(crate) fn broadcast_config(&self) -> crate::sync::BroadcastConfig { + crate::sync::BroadcastConfig { + holdout: self.broadcast_holdout, + acceptance_threshold: self.broadcast_acceptance_threshold, + acceptance_timeout: self.broadcast_acceptance_timeout, + } + } + /// Set the starting height for synchronization. pub fn with_start_height(mut self, height: u32) -> Self { self.start_from_height = Some(height); @@ -208,6 +255,13 @@ impl ClientConfig { ); } + if self.broadcast_acceptance_threshold == 0 { + return Err("broadcast_acceptance_threshold must be > 0".to_string()); + } + if self.broadcast_acceptance_timeout.is_zero() { + return Err("broadcast_acceptance_timeout must be > 0".to_string()); + } + match (self.network == Network::Devnet, &self.devnet) { (true, Some(devnet)) => devnet.validate()?, (true, None) => { diff --git a/dash-spv/src/client/lifecycle.rs b/dash-spv/src/client/lifecycle.rs index 4ce3d0c47..46e26f71c 100644 --- a/dash-spv/src/client/lifecycle.rs +++ b/dash-spv/src/client/lifecycle.rs @@ -145,6 +145,7 @@ impl DashSpvClient DashSpvClient { - /// Broadcast a transaction to all connected peers. + /// Broadcast a transaction to the network. + /// + /// With mempool tracking enabled (the default), the transaction is sent to + /// a subset of connected peers while being withheld from the rest; an + /// `inv` announcement from a withheld peer proves the transaction relayed + /// through the network and fires + /// [`SyncEvent::TransactionBroadcastResult`]. Use + /// [`broadcast_transaction_and_wait`](Self::broadcast_transaction_and_wait) + /// to await that outcome directly. /// - /// The transaction is also injected into the local message pipeline so that - /// the mempool manager processes it immediately. + /// With mempool tracking disabled there is no acceptance tracking and the + /// transaction is simply sent to all connected peers. pub async fn broadcast_transaction(&self, tx: &dashcore::Transaction) -> Result<()> { let network_guard = self.network.lock().await; @@ -20,12 +36,79 @@ impl DashSpvClient, + ) -> Result { + let config_timeout = { + let config = self.config.read().await; + if !config.enable_mempool_tracking { + return Err(SpvError::Config( + "broadcast_transaction_and_wait requires mempool tracking".to_string(), + )); + } + config.broadcast_acceptance_timeout + }; + let wait = timeout.unwrap_or(config_timeout + AWAIT_GRACE); + let txid = tx.txid(); + + // Subscribe before dispatching so the outcome event cannot be missed. + let mut events = self.subscribe_sync_events().await; + self.broadcast_transaction(tx).await?; + + let deadline = tokio::time::Instant::now() + wait; + loop { + match tokio::time::timeout_at(deadline, events.recv()).await { + // An interim `Uncertain` (the manager's acceptance timeout + // firing) is not returned early: a late echo can still + // upgrade the outcome, so keep waiting until the deadline. + Ok(Ok(SyncEvent::TransactionBroadcastResult { + txid: event_txid, + result, + })) if event_txid == txid && !matches!(result, BroadcastResult::Uncertain) => { + return Ok(result) + } + Ok(Ok(_)) => continue, + Ok(Err(RecvError::Lagged(skipped))) => { + tracing::warn!( + "Sync event stream lagged by {} while awaiting broadcast result", + skipped + ); + continue; + } + // Bus closed or deadline elapsed: no definitive outcome observed. + Ok(Err(RecvError::Closed)) | Err(_) => return Ok(BroadcastResult::Uncertain), + } + } + } } diff --git a/dash-spv/src/lib.rs b/dash-spv/src/lib.rs index ecf8f432a..b0263f08e 100644 --- a/dash-spv/src/lib.rs +++ b/dash-spv/src/lib.rs @@ -79,6 +79,7 @@ pub use error::{ LoggingError, LoggingResult, NetworkError, SpvError, StorageError, SyncError, ValidationError, }; pub use logging::{init_console_logging, init_logging, LogFileConfig, LoggingConfig, LoggingGuard}; +pub use sync::{BroadcastHoldout, BroadcastResult}; pub use tracing::level_filters::LevelFilter; pub use types::{FilterMatch, ValidationMode}; diff --git a/dash-spv/src/network/mod.rs b/dash-spv/src/network/mod.rs index 017af47d9..20c4266c7 100644 --- a/dash-spv/src/network/mod.rs +++ b/dash-spv/src/network/mod.rs @@ -91,6 +91,15 @@ impl RequestSender { .map_err(|e| NetworkError::ProtocolError(e.to_string())) } + /// Send a transaction to a specific peer. + pub(crate) fn send_transaction( + &self, + tx: dashcore::Transaction, + peer_address: SocketAddr, + ) -> NetworkResult<()> { + self.send_message_to_peer(NetworkMessage::Tx(tx), peer_address) + } + /// Request inventory from a specific peer. pub fn request_inventory( &self, diff --git a/dash-spv/src/sync/events.rs b/dash-spv/src/sync/events.rs index 2fbf24217..ae2df5c58 100644 --- a/dash-spv/src/sync/events.rs +++ b/dash-spv/src/sync/events.rs @@ -1,3 +1,4 @@ +use crate::sync::mempool::BroadcastResult; use crate::sync::ManagerIdentifier; use dashcore::ephemerealdata::chain_lock::ChainLock; use dashcore::ephemerealdata::instant_lock::InstantLock; @@ -162,6 +163,20 @@ pub enum SyncEvent { validated: bool, }, + /// The network-level outcome of a self-broadcast transaction was + /// determined: accepted (echoed back by non-recipient peers, InstantSend + /// locked, or confirmed) or uncertain (no signal within the timeout). + /// An `Uncertain` outcome may later be followed by an `Accepted` one. + /// + /// Emitted by: `MempoolManager` + /// Consumed by: External listeners, `broadcast_transaction_and_wait` + TransactionBroadcastResult { + /// The broadcast transaction's txid + txid: Txid, + /// The determined outcome + result: BroadcastResult, + }, + /// Sync has reached the chain tip (all managers idle). /// /// Emitted on every not-synced to synced transition. Cycle 0 is the @@ -256,6 +271,10 @@ impl fmt::Display for SyncEvent { "InstantLockReceived(txid={}, validated={})", instant_lock.txid, validated ), + SyncEvent::TransactionBroadcastResult { + txid, + result, + } => write!(f, "TransactionBroadcastResult(txid={}, result={})", txid, result), SyncEvent::SyncComplete { header_tip, cycle, diff --git a/dash-spv/src/sync/mempool/broadcast.rs b/dash-spv/src/sync/mempool/broadcast.rs new file mode 100644 index 000000000..ee739c2d7 --- /dev/null +++ b/dash-spv/src/sync/mempool/broadcast.rs @@ -0,0 +1,174 @@ +//! Broadcast acceptance tracking for self-originated transactions. +//! +//! Implements the classic SPV acceptance heuristic: the transaction is sent to +//! a subset of connected peers while being withheld from the rest (the +//! "holdout" set). Peers never re-announce a transaction to the peer they +//! received it from, so an `inv` for our txid from a holdout peer proves the +//! transaction propagated through the network and entered mempools. + +use std::collections::HashSet; +use std::net::SocketAddr; +use std::time::{Duration, Instant}; + +use dashcore::Transaction; + +/// How many peers to withhold a broadcast transaction from. +/// +/// At least one peer must be withheld for acceptance detection via `inv` echo +/// to work; with a holdout of zero the outcome stays [`BroadcastResult::Uncertain`] +/// until an InstantSend lock or block confirmation arrives. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum BroadcastHoldout { + /// Withhold from half of the connected peers (rounded down). + #[default] + Half, + /// Withhold from a fixed number of peers, clamped so that the + /// transaction is always sent to at least one peer. + Count(usize), +} + +impl BroadcastHoldout { + /// Number of peers to withhold from, given `peer_count` connected peers. + pub fn count_for(&self, peer_count: usize) -> usize { + match self { + BroadcastHoldout::Half => peer_count / 2, + BroadcastHoldout::Count(k) => (*k).min(peer_count.saturating_sub(1)), + } + } +} + +/// Configuration for broadcast acceptance tracking. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct BroadcastConfig { + /// Peers to withhold the initial send from. + pub holdout: BroadcastHoldout, + /// Distinct non-recipient peers that must announce the txid back before + /// the broadcast is considered accepted. + pub acceptance_threshold: usize, + /// How long a broadcast may stay pending before its outcome is reported + /// as uncertain. + pub acceptance_timeout: Duration, +} + +impl Default for BroadcastConfig { + fn default() -> Self { + Self { + holdout: BroadcastHoldout::default(), + acceptance_threshold: 1, + acceptance_timeout: Duration::from_secs(60), + } + } +} + +/// Network-level outcome of a transaction broadcast. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum BroadcastResult { + /// The transaction propagated through the network: peers we did not send + /// it to announced it back (or it was InstantSend-locked / mined). + Accepted { + /// Number of distinct non-recipient peers that announced the txid. + /// Zero when acceptance was proven by an InstantSend lock or a block + /// confirmation before any echo arrived. + relayed_by: usize, + }, + /// No acceptance signal arrived within the configured timeout. + /// + /// This is also what an invalid transaction looks like: modern Dash Core + /// removed the BIP61 `reject` message, so there is no negative signal on + /// the p2p network — a transaction the network refuses simply never + /// echoes back. The transaction may also still be fine and confirm + /// later; a late echo, InstantSend lock, or confirmation upgrades the + /// outcome to `Accepted`. + Uncertain, +} + +impl std::fmt::Display for BroadcastResult { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + BroadcastResult::Accepted { + relayed_by, + } => write!(f, "Accepted(relayed_by={})", relayed_by), + BroadcastResult::Uncertain => write!(f, "Uncertain"), + } + } +} + +/// Internal lifecycle state of a tracked broadcast. +/// +/// Valid transitions: `Pending -> Accepted | Uncertain` and +/// `Uncertain -> Accepted`. Every transition emits exactly one +/// `SyncEvent::TransactionBroadcastResult`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum BroadcastStatus { + /// Broadcast sent, awaiting an acceptance signal. + Pending, + /// Accepted by the network (echo threshold, IS lock, or confirmation). + Accepted, + /// Timed out without a definitive signal. + Uncertain, +} + +/// Per-transaction broadcast tracking state. +/// +/// The full transaction is stored so rebroadcast works even when the wallet +/// does not consider the transaction relevant (the `transactions` map only +/// holds wallet-relevant entries). +#[derive(Debug, Clone)] +pub(crate) struct TxBroadcastState { + /// The broadcast transaction. + pub transaction: Transaction, + /// Peers the transaction was sent to directly. Announcements from these + /// peers carry no acceptance information. + pub sent_to: HashSet, + /// Peers deliberately not sent the transaction; sticky across + /// rebroadcasts so an echo remains possible. + pub holdout: HashSet, + /// Non-recipient peers that announced the txid back via `inv`. + pub announced_by: HashSet, + /// When the broadcast was first initiated (drives the acceptance timeout). + pub created_at: Instant, + /// When the transaction was last sent to the network (drives rebroadcast). + pub last_broadcast: Instant, + /// Current lifecycle status. + pub status: BroadcastStatus, +} + +impl TxBroadcastState { + pub fn new(transaction: Transaction, now: Instant) -> Self { + Self { + transaction, + sent_to: HashSet::new(), + holdout: HashSet::new(), + announced_by: HashSet::new(), + created_at: now, + last_broadcast: now, + status: BroadcastStatus::Pending, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn holdout_half_rounds_down() { + let h = BroadcastHoldout::Half; + assert_eq!(h.count_for(0), 0); + assert_eq!(h.count_for(1), 0); + assert_eq!(h.count_for(2), 1); + assert_eq!(h.count_for(3), 1); + assert_eq!(h.count_for(4), 2); + assert_eq!(h.count_for(5), 2); + } + + #[test] + fn holdout_count_clamps_to_leave_one_recipient() { + let h = BroadcastHoldout::Count(3); + assert_eq!(h.count_for(0), 0); + assert_eq!(h.count_for(1), 0); + assert_eq!(h.count_for(2), 1); + assert_eq!(h.count_for(4), 3); + assert_eq!(h.count_for(10), 3); + } +} diff --git a/dash-spv/src/sync/mempool/manager.rs b/dash-spv/src/sync/mempool/manager.rs index fc8fd23ea..dd822b0fa 100644 --- a/dash-spv/src/sync/mempool/manager.rs +++ b/dash-spv/src/sync/mempool/manager.rs @@ -17,6 +17,7 @@ use dashcore::{Amount, Transaction, Txid}; use rand::seq::IteratorRandom; use tokio::sync::RwLock; +use super::broadcast::{BroadcastConfig, BroadcastResult, BroadcastStatus, TxBroadcastState}; use super::filter::build_wallet_bloom_filter; use super::BLOOM_FALSE_POSITIVE_RATE; use crate::client::config::MempoolStrategy; @@ -55,7 +56,9 @@ pub(crate) struct MempoolManager { pub(super) progress: MempoolProgress, pub(super) wallet: Arc>, pub(super) transactions: HashMap, - pub(super) recent_sends: HashMap, + /// Self-broadcast transactions and their network-acceptance state. + pub(super) broadcasts: HashMap, + broadcast_config: BroadcastConfig, strategy: MempoolStrategy, max_transactions: usize, /// Txids we have requested via getdata but not yet received, with request time. @@ -81,12 +84,14 @@ impl MempoolManager { strategy: MempoolStrategy, max_transactions: usize, initial_monitor_revision: u64, + broadcast_config: BroadcastConfig, ) -> Self { Self { progress: MempoolProgress::default(), wallet, transactions: HashMap::new(), - recent_sends: HashMap::new(), + broadcasts: HashMap::new(), + broadcast_config, strategy, max_transactions, pending_requests: HashMap::new(), @@ -202,9 +207,14 @@ impl MempoolManager { peer: SocketAddr, requests: &RequestSender, ) -> SyncResult> { + // Acceptance detection must run before any early return: an inv for + // one of our own broadcasts from a peer we did NOT send it to proves + // the transaction relayed through the network. + let events = self.record_broadcast_echoes(inv, peer); + let mempool_full = self.transactions.len() >= self.max_transactions; if mempool_full { - return Ok(vec![]); + return Ok(events); } let total_queued: usize = @@ -237,7 +247,52 @@ impl MempoolManager { self.send_queued(requests).await?; } - Ok(vec![]) + Ok(events) + } + + /// Record `inv` announcements of our own broadcast transactions and + /// promote broadcasts to accepted once enough non-recipient peers have + /// announced them back. + fn record_broadcast_echoes(&mut self, inv: &[Inventory], peer: SocketAddr) -> Vec { + let mut events = Vec::new(); + for item in inv { + let Inventory::Transaction(txid) = item else { + continue; + }; + let Some(state) = self.broadcasts.get_mut(txid) else { + continue; + }; + // Announcements from direct recipients carry no information: a + // peer only ever announces to peers it did not get the tx from, + // so a recipient echoing back would just be a protocol quirk. + if state.sent_to.contains(&peer) || !state.announced_by.insert(peer) { + continue; + } + tracing::debug!( + "Broadcast {} announced back by non-recipient peer {} ({}/{} needed)", + txid, + peer, + state.announced_by.len(), + self.broadcast_config.acceptance_threshold + ); + if matches!(state.status, BroadcastStatus::Pending | BroadcastStatus::Uncertain) + && state.announced_by.len() >= self.broadcast_config.acceptance_threshold + { + state.status = BroadcastStatus::Accepted; + tracing::info!( + "Broadcast {} accepted by the network ({} peer(s) relayed it back)", + txid, + state.announced_by.len() + ); + events.push(SyncEvent::TransactionBroadcastResult { + txid: *txid, + result: BroadcastResult::Accepted { + relayed_by: state.announced_by.len(), + }, + }); + } + } + events } /// Drain per-peer queues and send getdata for up to `MAX_IN_FLIGHT` items. @@ -302,22 +357,28 @@ impl MempoolManager { /// Handle a received transaction. /// /// When `peer` is the local sentinel address (`0.0.0.0:0`), the transaction - /// is treated as self-originated and recorded in `recent_sends`. + /// is treated as self-originated: it is sent to a subset of connected peers + /// (withholding a holdout set) and tracked in `broadcasts` for network + /// acceptance detection. pub(super) async fn handle_tx( &mut self, tx: Transaction, peer: SocketAddr, + requests: &RequestSender, ) -> SyncResult> { let txid = tx.txid(); self.pending_requests.remove(&txid); let is_local = peer.ip().is_unspecified(); + // Send self-originated transactions to the network regardless of + // wallet relevance — the caller explicitly asked to broadcast. + if is_local { + self.start_broadcast(&tx, requests); + } + // Skip if already tracked (e.g., locally broadcast then received from a peer) if self.transactions.contains_key(&txid) { self.seen_txids.insert(txid, Instant::now()); - if is_local { - self.recent_sends.insert(txid, Instant::now()); - } return Ok(vec![]); } @@ -351,21 +412,105 @@ impl MempoolManager { result.net_amount, ); self.transactions.insert(txid, unconfirmed_tx); - if is_local { - self.recent_sends.insert(txid, Instant::now()); - } self.progress.set_tracked(self.transactions.len() as u32); Ok(vec![]) } + /// Begin tracking a self-originated transaction and send it to the + /// network, withholding it from a holdout subset of peers. + /// + /// Idempotent per txid: repeated local dispatches of the same transaction + /// do not resend (the rebroadcast timer handles resends). + pub(super) fn start_broadcast(&mut self, tx: &Transaction, requests: &RequestSender) { + let txid = tx.txid(); + if self.broadcasts.contains_key(&txid) { + return; + } + + let mut state = TxBroadcastState::new(tx.clone(), Instant::now()); + let peers: Vec = self.peers.keys().copied().collect(); + let holdout_count = self.broadcast_config.holdout.count_for(peers.len()); + state.holdout = peers + .iter() + .copied() + .choose_multiple(&mut rand::thread_rng(), holdout_count) + .into_iter() + .collect(); + + for peer in &peers { + if state.holdout.contains(peer) { + continue; + } + if requests.send_transaction(tx.clone(), *peer).is_ok() { + state.sent_to.insert(*peer); + } + } + + if peers.is_empty() { + // No peers right now; the rebroadcast timer sends once one connects. + tracing::warn!("Broadcast {} deferred: no connected peers", txid); + } else { + tracing::info!( + "Broadcast {} to {}/{} peers ({} withheld for acceptance detection)", + txid, + state.sent_to.len(), + peers.len(), + state.holdout.len() + ); + } + self.broadcasts.insert(txid, state); + } + + /// Transition pending broadcasts that outlived the acceptance timeout to + /// `Uncertain`, emitting one event per transition. A late echo, IS lock, + /// or confirmation can still upgrade them to `Accepted`. + pub(super) fn expire_broadcasts(&mut self) -> Vec { + self.expire_broadcasts_at(Instant::now()) + } + + fn expire_broadcasts_at(&mut self, now: Instant) -> Vec { + let timeout = self.broadcast_config.acceptance_timeout; + let mut events = Vec::new(); + for (txid, state) in &mut self.broadcasts { + if state.status == BroadcastStatus::Pending + && now.saturating_duration_since(state.created_at) >= timeout + { + state.status = BroadcastStatus::Uncertain; + tracing::info!( + "Broadcast {} outcome uncertain: no acceptance signal within {:?}", + txid, + timeout + ); + events.push(SyncEvent::TransactionBroadcastResult { + txid: *txid, + result: BroadcastResult::Uncertain, + }); + } + } + events + } + /// Remove transactions from the mempool that have been confirmed in a block. - pub(super) fn remove_confirmed(&mut self, txids: &[Txid]) { + /// + /// Confirmation is definitive acceptance: pending or uncertain broadcasts + /// for confirmed txids are promoted to accepted and dropped from tracking. + pub(super) fn remove_confirmed(&mut self, txids: &[Txid]) -> Vec { self.seen_txids.retain(|_, t| t.elapsed() < SEEN_TXID_EXPIRY); + let mut events = Vec::new(); let mut removed = Vec::new(); for txid in txids { + if let Some(state) = self.broadcasts.remove(txid) { + if matches!(state.status, BroadcastStatus::Pending | BroadcastStatus::Uncertain) { + events.push(SyncEvent::TransactionBroadcastResult { + txid: *txid, + result: BroadcastResult::Accepted { + relayed_by: state.announced_by.len(), + }, + }); + } + } if self.transactions.remove(txid).is_some() { - self.recent_sends.remove(txid); removed.push(*txid); } } @@ -374,17 +519,35 @@ impl MempoolManager { self.progress.set_tracked(self.transactions.len() as u32); tracing::debug!("Removed {} confirmed transactions from mempool", removed.len()); } + events } /// Mark a mempool transaction as InstantSend-locked and notify the wallet. /// /// If the transaction hasn't arrived yet, remembers the lock so it /// can be applied when the transaction is later received via `handle_tx`. - pub(super) async fn process_instant_send(&mut self, instant_lock: InstantLock) { + /// + /// An InstantSend lock is definitive acceptance: any pending or uncertain + /// broadcast for the txid is promoted to accepted and dropped from tracking. + pub(super) async fn process_instant_send( + &mut self, + instant_lock: InstantLock, + ) -> Vec { let txid = instant_lock.txid; + let mut events = Vec::new(); + if let Some(state) = self.broadcasts.remove(&txid) { + if matches!(state.status, BroadcastStatus::Pending | BroadcastStatus::Uncertain) { + tracing::info!("Broadcast {} accepted (InstantSend lock received)", txid); + events.push(SyncEvent::TransactionBroadcastResult { + txid, + result: BroadcastResult::Accepted { + relayed_by: state.announced_by.len(), + }, + }); + } + } let instant_lock_opt = if let Some(tx) = self.transactions.get_mut(&txid) { tx.is_instant_send = true; - self.recent_sends.remove(&txid); tracing::debug!("Marked mempool tx {} as InstantSend-locked", txid); Some(instant_lock) } else if self.pending_is_locks.len() < MAX_PENDING_IS_LOCKS { @@ -403,6 +566,7 @@ impl MempoolManager { let mut wallet = self.wallet.write().await; wallet.process_instant_send_lock(lock); } + events } /// Prune transactions and pending IS locks older than `timeout`. @@ -417,9 +581,9 @@ impl MempoolManager { } }); - // Prune old recent sends + // Prune old broadcast tracking states if let Some(cutoff) = Instant::now().checked_sub(timeout) { - self.recent_sends.retain(|_, &mut timestamp| timestamp > cutoff); + self.broadcasts.retain(|_, state| state.created_at > cutoff); } if !expired_txids.is_empty() { @@ -440,11 +604,17 @@ impl MempoolManager { } } - /// Rebroadcast unconfirmed self-sent transactions to all peers. + /// Rebroadcast unconfirmed self-sent transactions. /// - /// Each transaction in `recent_sends` tracks when it was last broadcast. - /// Transactions whose last broadcast was more than `REBROADCAST_INTERVAL` - /// ago are rebroadcast and their timestamp is reset. + /// Each entry in `broadcasts` tracks when it was last sent. Entries whose + /// last send was more than `REBROADCAST_INTERVAL` ago are resent: + /// - `Pending`: targeted resend that keeps respecting the holdout set so + /// an acceptance echo remains possible. If every holdout peer has + /// disconnected, replacement holdouts are picked from peers that never + /// received the transaction. + /// - `Accepted`/`Uncertain`: plain broadcast to all peers (the holdout no + /// longer matters; a late echo from a new peer can still upgrade + /// `Uncertain` to `Accepted`). pub(super) async fn rebroadcast_if_due(&mut self, requests: &RequestSender) { self.rebroadcast_if_due_at(requests, Instant::now()).await } @@ -453,20 +623,61 @@ impl MempoolManager { /// forward instead of subtracting from `Instant::now()`, which underflows on /// Windows when the QPC-based monotonic clock has a small value at boot. async fn rebroadcast_if_due_at(&mut self, requests: &RequestSender, now: Instant) { + let current_peers: Vec = self.peers.keys().copied().collect(); let mut count: usize = 0; - for (txid, last_broadcast) in &mut self.recent_sends { - if now.saturating_duration_since(*last_broadcast) < REBROADCAST_INTERVAL { + for (txid, state) in &mut self.broadcasts { + if current_peers.is_empty() { continue; } - if let Some(unconfirmed) = self.transactions.get(txid) { - let _ = requests.broadcast(NetworkMessage::Tx(unconfirmed.transaction.clone())); - *last_broadcast = now; - count += 1; + // A pending broadcast that never reached any peer (no peers were + // connected at send time) is due immediately. + let never_sent = state.status == BroadcastStatus::Pending && state.sent_to.is_empty(); + if !never_sent + && now.saturating_duration_since(state.last_broadcast) < REBROADCAST_INTERVAL + { + continue; + } + if state.status == BroadcastStatus::Pending { + // Re-pick the holdout from never-sent peers if all holdouts left. + let holdout_alive = current_peers.iter().any(|p| state.holdout.contains(p)); + if !holdout_alive { + let needed = self.broadcast_config.holdout.count_for(current_peers.len()); + let candidates: Vec = current_peers + .iter() + .filter(|p| !state.sent_to.contains(*p)) + .copied() + .collect(); + state.holdout.extend( + candidates.into_iter().choose_multiple(&mut rand::thread_rng(), needed), + ); + } + let targets: Vec = + current_peers.iter().filter(|p| !state.holdout.contains(*p)).copied().collect(); + if targets.is_empty() { + // Every connected peer is a holdout (all original + // recipients are gone); relay through everyone rather + // than letting the transaction stall. + if !current_peers.is_empty() { + let _ = requests.broadcast(NetworkMessage::Tx(state.transaction.clone())); + state.sent_to.extend(current_peers.iter().copied()); + } + } else { + for peer in targets { + if requests.send_transaction(state.transaction.clone(), peer).is_ok() { + state.sent_to.insert(peer); + } + } + } + } else { + let _ = requests.broadcast(NetworkMessage::Tx(state.transaction.clone())); } + tracing::debug!("Rebroadcast unconfirmed transaction {}", txid); + state.last_broadcast = now; + count += 1; } if count > 0 { - tracing::info!("Rebroadcast {} unconfirmed transaction(s) to all peers", count); + tracing::info!("Rebroadcast {} unconfirmed transaction(s)", count); } } @@ -544,6 +755,7 @@ impl fmt::Debug for MempoolManager { f.debug_struct("MempoolManager") .field("progress", &self.progress) .field("strategy", &self.strategy) + .field("broadcasts", &self.broadcasts.len()) .field("pending_requests", &self.pending_requests.len()) .field("peers", &self.peers.len()) .field("activated_peers", &activated) @@ -590,7 +802,13 @@ mod tests { let (tx, rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let mut manager = MempoolManager::new(wallet, MempoolStrategy::FetchAll, 1000, 0); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::FetchAll, + 1000, + 0, + BroadcastConfig::default(), + ); manager.progress.set_state(SyncState::Synced); (manager, requests, rx) @@ -602,7 +820,13 @@ mod tests { let (tx, rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let manager = MempoolManager::new(wallet, MempoolStrategy::BloomFilter, 1000, 0); + let manager = MempoolManager::new( + wallet, + MempoolStrategy::BloomFilter, + 1000, + 0, + BroadcastConfig::default(), + ); (manager, requests, rx) } @@ -671,6 +895,7 @@ mod tests { MempoolStrategy::FetchAll, 2, // Very small capacity 0, + BroadcastConfig::default(), ); let peer = test_socket_address(1); manager.peers.insert(peer, Some(VecDeque::new())); @@ -705,7 +930,13 @@ mod tests { let (tx, _rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let mut manager = MempoolManager::new(wallet, MempoolStrategy::FetchAll, 2, 0); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::FetchAll, + 2, + 0, + BroadcastConfig::default(), + ); manager.progress.set_state(SyncState::Synced); let peer = test_socket_address(1); manager.peers.insert(peer, Some(VecDeque::new())); @@ -729,7 +960,13 @@ mod tests { let (tx, _rx) = mpsc::unbounded_channel::(); let _requests = RequestSender::new(tx); - let mut manager = MempoolManager::new(wallet, MempoolStrategy::FetchAll, 1000, 0); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::FetchAll, + 1000, + 0, + BroadcastConfig::default(), + ); let fresh_txid = Txid::from_byte_array([1; 32]); let stale_txid = Txid::from_byte_array([2; 32]); @@ -747,7 +984,7 @@ mod tests { #[tokio::test] async fn test_handle_tx_irrelevant() { - let (mut manager, _requests, _rx) = create_test_manager(); + let (mut manager, requests, _rx) = create_test_manager(); let tx = Transaction { version: 1, @@ -758,7 +995,7 @@ mod tests { }; let txid = tx.txid(); - let events = manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + let events = manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); // MockWallet returns is_relevant=false by default assert!(events.is_empty()); assert_eq!(manager.progress.received(), 1); @@ -842,14 +1079,20 @@ mod tests { let (tx, _rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let manager = MempoolManager::new(wallet.clone(), MempoolStrategy::BloomFilter, 1000, 0); + let manager = MempoolManager::new( + wallet.clone(), + MempoolStrategy::BloomFilter, + 1000, + 0, + BroadcastConfig::default(), + ); (manager, requests, wallet) } #[tokio::test] async fn test_handle_tx_relevant_stores_transaction() { - let (mut manager, _requests, _wallet) = create_relevant_manager(); + let (mut manager, requests, _wallet) = create_relevant_manager(); let tx = Transaction { version: 1, @@ -860,7 +1103,7 @@ mod tests { }; let txid = tx.txid(); - let events = manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + let events = manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); assert!(events.is_empty()); // Verify transaction was stored @@ -877,7 +1120,7 @@ mod tests { output: vec![], special_transaction_payload: None, }; - let events = manager.handle_tx(tx2, test_socket_address(1)).await.unwrap(); + let events = manager.handle_tx(tx2, test_socket_address(1), &requests).await.unwrap(); assert!(events.is_empty()); assert_eq!(manager.transactions.len(), 1); @@ -888,7 +1131,7 @@ mod tests { #[tokio::test] async fn test_handle_tx_local_records_send() { - let (mut manager, _requests, _wallet) = create_relevant_manager(); + let (mut manager, requests, _wallet) = create_relevant_manager(); let tx = Transaction { version: 2, @@ -901,18 +1144,18 @@ mod tests { // Use the unspecified address to simulate a locally broadcast transaction let local_addr = SocketAddr::from(([0, 0, 0, 0], 0)); - manager.handle_tx(tx, local_addr).await.unwrap(); + manager.handle_tx(tx, local_addr, &requests).await.unwrap(); assert!(manager.transactions.contains_key(&txid)); assert!( - manager.recent_sends.contains_key(&txid), - "locally dispatched transaction should be recorded as a recent send" + manager.broadcasts.contains_key(&txid), + "locally dispatched transaction should be tracked as a broadcast" ); } #[tokio::test] async fn test_handle_tx_remote_does_not_record_send() { - let (mut manager, _requests, _wallet) = create_relevant_manager(); + let (mut manager, requests, _wallet) = create_relevant_manager(); let tx = Transaction { version: 3, @@ -923,18 +1166,18 @@ mod tests { }; let txid = tx.txid(); - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); assert!(manager.transactions.contains_key(&txid)); assert!( - !manager.recent_sends.contains_key(&txid), - "peer-received transaction should not be recorded as a recent send" + !manager.broadcasts.contains_key(&txid), + "peer-received transaction should not be tracked as a broadcast" ); } #[tokio::test] async fn test_handle_tx_clears_pending_request() { - let (mut manager, _requests, _wallet) = create_relevant_manager(); + let (mut manager, requests, _wallet) = create_relevant_manager(); let tx = Transaction { version: 1, @@ -949,7 +1192,7 @@ mod tests { manager.pending_requests.insert(txid, Instant::now()); assert!(manager.pending_requests.contains_key(&txid)); - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); // Pending request should be cleared regardless of relevance assert!(!manager.pending_requests.contains_key(&txid)); @@ -966,7 +1209,13 @@ mod tests { let (tx, rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let manager = MempoolManager::new(wallet, MempoolStrategy::BloomFilter, 1000, 0); + let manager = MempoolManager::new( + wallet, + MempoolStrategy::BloomFilter, + 1000, + 0, + BroadcastConfig::default(), + ); (manager, requests, rx) } @@ -1010,20 +1259,27 @@ mod tests { special_transaction_payload: None, }; let txid = tx.txid(); + manager.broadcasts.insert(txid, TxBroadcastState::new(tx.clone(), Instant::now())); manager.transactions.insert( txid, UnconfirmedTransaction::new(tx, Amount::from_sat(0), false, false, Vec::new(), 0), ); - manager.recent_sends.insert(txid, Instant::now()); - manager.process_instant_send(dummy_instant_lock(txid)).await; + let events = manager.process_instant_send(dummy_instant_lock(txid)).await; - // Verify IS flag and recent_sends cleanup + // Verify IS flag, broadcast promotion, and tracking cleanup assert!(manager.transactions.get(&txid).unwrap().is_instant_send); assert!( - !manager.recent_sends.contains_key(&txid), - "IS-locked transaction should be removed from recent_sends" + !manager.broadcasts.contains_key(&txid), + "IS-locked transaction should no longer be tracked as a broadcast" ); + assert!(matches!( + events.as_slice(), + [SyncEvent::TransactionBroadcastResult { + result: BroadcastResult::Accepted { .. }, + .. + }] + )); let wallet = manager.wallet.read().await; let status_changes = wallet.status_changes(); @@ -1187,7 +1443,7 @@ mod tests { #[tokio::test] async fn test_instant_send_before_transaction() { - let (mut manager, _requests, wallet) = create_relevant_manager(); + let (mut manager, requests, wallet) = create_relevant_manager(); let tx = Transaction { version: 1, @@ -1203,7 +1459,7 @@ mod tests { assert!(manager.pending_is_locks.contains_key(&txid)); // Transaction arrives - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); // Pending IS lock consumed assert!(manager.pending_is_locks.is_empty()); @@ -1225,7 +1481,7 @@ mod tests { #[tokio::test] async fn test_instant_send_before_irrelevant_transaction() { - let (mut manager, _requests, _rx) = create_test_manager(); + let (mut manager, requests, _rx) = create_test_manager(); let tx = Transaction { version: 1, @@ -1241,7 +1497,7 @@ mod tests { assert!(manager.pending_is_locks.contains_key(&txid)); // Transaction arrives but wallet says it's not relevant - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); // Pending IS lock cleaned up (no leak) assert!(manager.pending_is_locks.is_empty()); @@ -1386,7 +1642,13 @@ mod tests { let (tx, rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let mut manager = MempoolManager::new(wallet, MempoolStrategy::BloomFilter, 1000, 0); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::BloomFilter, + 1000, + 0, + BroadcastConfig::default(), + ); // Drop receiver so send_filter_load fails drop(rx); @@ -1398,7 +1660,7 @@ mod tests { #[tokio::test] async fn test_handle_tx_relevant_populates_wallet_effect_fields() { - let (mut manager, _requests, wallet) = create_relevant_manager(); + let (mut manager, requests, wallet) = create_relevant_manager(); let tx = Transaction { version: 1, @@ -1417,7 +1679,7 @@ mod tests { w.set_mempool_addresses(vec![addr.clone()]); } - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); let stored = manager.transactions.get(&txid).unwrap(); assert_eq!(stored.net_amount, 50000); @@ -1429,7 +1691,7 @@ mod tests { #[tokio::test] async fn test_handle_tx_outgoing_transaction() { - let (mut manager, _requests, wallet) = create_relevant_manager(); + let (mut manager, requests, wallet) = create_relevant_manager(); let tx = Transaction { version: 1, @@ -1445,7 +1707,7 @@ mod tests { w.set_mempool_net_amount(-30000); } - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); let stored = manager.transactions.get(&txid).unwrap(); assert_eq!(stored.net_amount, -30000); @@ -1545,23 +1807,35 @@ mod tests { }; let txid = tx.txid(); txids.push(txid); + if i < 2 { + // Mark the first two as self-broadcasts + manager.broadcasts.insert(txid, TxBroadcastState::new(tx.clone(), Instant::now())); + } manager.transactions.insert( txid, UnconfirmedTransaction::new(tx, Amount::from_sat(0), false, false, Vec::new(), 0), ); } assert_eq!(manager.transactions.len(), 3); - // Mark two as recent sends - manager.recent_sends.insert(txids[0], Instant::now()); - manager.recent_sends.insert(txids[1], Instant::now()); // Remove 2 of the 3 transactions - manager.remove_confirmed(&txids[..2]); + let events = manager.remove_confirmed(&txids[..2]); assert_eq!(manager.transactions.len(), 1); assert!(manager.transactions.contains_key(&txids[2])); - assert!(!manager.recent_sends.contains_key(&txids[0])); - assert!(!manager.recent_sends.contains_key(&txids[1])); + assert!(!manager.broadcasts.contains_key(&txids[0])); + assert!(!manager.broadcasts.contains_key(&txids[1])); + // Both pending broadcasts were confirmed => promoted to accepted + assert_eq!(events.len(), 2); + assert!(events.iter().all(|e| matches!( + e, + SyncEvent::TransactionBroadcastResult { + result: BroadcastResult::Accepted { + relayed_by: 0 + }, + .. + } + ))); assert_eq!(manager.progress.removed(), 2); assert_eq!(manager.progress.tracked(), 1); @@ -1637,35 +1911,39 @@ mod tests { ); } - #[tokio::test] - async fn test_rebroadcast_sends_old_recent_sends() { - let (mut manager, requests, mut rx) = create_test_manager(); - - let tx = Transaction { - version: 10, + fn test_transaction(version: u16) -> Transaction { + Transaction { + version, lock_time: 0, input: vec![], output: vec![], special_transaction_payload: None, - }; + } + } + + #[tokio::test] + async fn test_rebroadcast_sends_old_pending_broadcasts() { + let (mut manager, requests, mut rx) = create_test_manager(); + let peer = test_socket_address(1); + manager.peers.insert(peer, Some(VecDeque::new())); + + let tx = test_transaction(10); let txid = tx.txid(); let t0 = Instant::now(); let later = t0 + REBROADCAST_INTERVAL + Duration::from_secs(1); - manager.transactions.insert( - txid, - UnconfirmedTransaction::new(tx, Amount::from_sat(0), false, true, Vec::new(), -100_000), - ); - manager.recent_sends.insert(txid, t0); + let mut state = TxBroadcastState::new(tx, t0); + state.sent_to.insert(peer); + manager.broadcasts.insert(txid, state); manager.rebroadcast_if_due_at(&requests, later).await; - // Should have sent a BroadcastMessage for the transaction + // Pending entries are resent via targeted sends (respecting the holdout) let msg = rx.try_recv().expect("expected a rebroadcast message"); assert!( - matches!(msg, NetworkRequest::BroadcastMessage(NetworkMessage::Tx(_))), - "expected BroadcastMessage(Tx), got {:?}", + matches!(msg, NetworkRequest::SendMessageToPeer(NetworkMessage::Tx(_), p) if p == peer), + "expected SendMessageToPeer(Tx), got {:?}", msg ); @@ -1676,24 +1954,45 @@ mod tests { } #[tokio::test] - async fn test_rebroadcast_skips_recent_transactions() { + async fn test_rebroadcast_uncertain_uses_full_broadcast() { let (mut manager, requests, mut rx) = create_test_manager(); + let peer = test_socket_address(1); + manager.peers.insert(peer, Some(VecDeque::new())); - let tx = Transaction { - version: 11, - lock_time: 0, - input: vec![], - output: vec![], - special_transaction_payload: None, - }; + let tx = test_transaction(12); let txid = tx.txid(); - // Add a transaction that was just sent (within the rebroadcast interval) - manager.transactions.insert( - txid, - UnconfirmedTransaction::new(tx, Amount::from_sat(0), false, true, Vec::new(), -50_000), + let t0 = Instant::now(); + let later = t0 + REBROADCAST_INTERVAL + Duration::from_secs(1); + + let mut state = TxBroadcastState::new(tx, t0); + state.sent_to.insert(peer); + state.status = BroadcastStatus::Uncertain; + manager.broadcasts.insert(txid, state); + + manager.rebroadcast_if_due_at(&requests, later).await; + + let msg = rx.try_recv().expect("expected a rebroadcast message"); + assert!( + matches!(msg, NetworkRequest::BroadcastMessage(NetworkMessage::Tx(_))), + "expected BroadcastMessage(Tx), got {:?}", + msg ); - manager.recent_sends.insert(txid, Instant::now()); + } + + #[tokio::test] + async fn test_rebroadcast_skips_recent_transactions() { + let (mut manager, requests, mut rx) = create_test_manager(); + let peer = test_socket_address(1); + manager.peers.insert(peer, Some(VecDeque::new())); + + let tx = test_transaction(11); + let txid = tx.txid(); + + // Add a broadcast that was just sent (within the rebroadcast interval) + let mut state = TxBroadcastState::new(tx, Instant::now()); + state.sent_to.insert(peer); + manager.broadcasts.insert(txid, state); manager.rebroadcast_if_due(&requests).await; @@ -1717,4 +2016,329 @@ mod tests { assert!(manager.peers.contains_key(&peer2)); assert!(manager.peers[&peer2].is_some()); } + + // ---- Broadcast acceptance tracking ---- + + const LOCAL_SENTINEL: SocketAddr = + SocketAddr::new(std::net::IpAddr::V4(std::net::Ipv4Addr::UNSPECIFIED), 0); + + fn drain_tx_sends(rx: &mut mpsc::UnboundedReceiver) -> Vec { + let mut sends = Vec::new(); + while let Ok(msg) = rx.try_recv() { + if let NetworkRequest::SendMessageToPeer(NetworkMessage::Tx(_), peer) = msg { + sends.push(peer); + } + } + sends + } + + fn accepted_event_count(events: &[SyncEvent]) -> usize { + events + .iter() + .filter(|e| { + matches!( + e, + SyncEvent::TransactionBroadcastResult { + result: BroadcastResult::Accepted { .. }, + .. + } + ) + }) + .count() + } + + #[tokio::test] + async fn test_local_tx_sends_to_half_of_peers() { + let (mut manager, requests, mut rx) = create_test_manager(); + for i in 1..=4 { + manager.peers.insert(test_socket_address(i), None); + } + + let tx = test_transaction(20); + let txid = tx.txid(); + manager.handle_tx(tx, LOCAL_SENTINEL, &requests).await.unwrap(); + + let sends = drain_tx_sends(&mut rx); + assert_eq!(sends.len(), 2, "should send to half of 4 peers"); + + let state = manager.broadcasts.get(&txid).expect("broadcast tracked"); + assert_eq!(state.sent_to.len(), 2); + assert_eq!(state.holdout.len(), 2); + assert!(state.sent_to.is_disjoint(&state.holdout)); + assert_eq!(state.status, BroadcastStatus::Pending); + + // A second local dispatch of the same tx must not resend + let tx = test_transaction(20); + manager.handle_tx(tx, LOCAL_SENTINEL, &requests).await.unwrap(); + assert!(drain_tx_sends(&mut rx).is_empty(), "idempotent per txid"); + } + + #[tokio::test] + async fn test_local_tx_single_peer_no_holdout() { + let (mut manager, requests, mut rx) = create_test_manager(); + let peer = test_socket_address(1); + manager.peers.insert(peer, None); + + let tx = test_transaction(21); + let txid = tx.txid(); + manager.handle_tx(tx, LOCAL_SENTINEL, &requests).await.unwrap(); + + assert_eq!(drain_tx_sends(&mut rx), vec![peer]); + let state = manager.broadcasts.get(&txid).unwrap(); + assert!(state.holdout.is_empty(), "single peer leaves nobody to hold out"); + } + + #[tokio::test] + async fn test_echo_from_holdout_peer_accepts_once() { + let (mut manager, requests, mut rx) = create_test_manager(); + let recipient = test_socket_address(1); + let holdout = test_socket_address(2); + manager.peers.insert(recipient, None); + manager.peers.insert(holdout, None); + + let tx = test_transaction(22); + let txid = tx.txid(); + let mut state = TxBroadcastState::new(tx, Instant::now()); + state.sent_to.insert(recipient); + state.holdout.insert(holdout); + manager.broadcasts.insert(txid, state); + drain_tx_sends(&mut rx); + + let inv = vec![Inventory::Transaction(txid)]; + + // Echo from the recipient peer carries no information + let events = manager.handle_inv(&inv, recipient, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 0); + assert_eq!(manager.broadcasts[&txid].status, BroadcastStatus::Pending); + + // Echo from the holdout peer proves propagation + let events = manager.handle_inv(&inv, holdout, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 1); + assert_eq!(manager.broadcasts[&txid].status, BroadcastStatus::Accepted); + + // A repeat announcement must not emit a second event + let events = manager.handle_inv(&inv, holdout, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 0); + } + + #[tokio::test] + async fn test_echo_detected_when_mempool_full() { + let wallet = Arc::new(RwLock::new(MockWallet::new())); + let (tx_chan, _rx) = mpsc::unbounded_channel::(); + let requests = RequestSender::new(tx_chan); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::FetchAll, + 1, // capacity of one + 0, + BroadcastConfig::default(), + ); + + // Fill the mempool to capacity with an unrelated transaction + let filler = test_transaction(23); + manager.transactions.insert( + filler.txid(), + UnconfirmedTransaction::new(filler, Amount::from_sat(0), false, false, Vec::new(), 0), + ); + + let holdout = test_socket_address(2); + let tx = test_transaction(24); + let txid = tx.txid(); + let mut state = TxBroadcastState::new(tx, Instant::now()); + state.sent_to.insert(test_socket_address(1)); + state.holdout.insert(holdout); + manager.broadcasts.insert(txid, state); + + // The mempool-full early return must not swallow the acceptance echo + let inv = vec![Inventory::Transaction(txid)]; + let events = manager.handle_inv(&inv, holdout, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 1); + } + + #[tokio::test] + async fn test_timeout_uncertain_then_late_echo_upgrades() { + let (mut manager, requests, _rx) = create_test_manager(); + let holdout = test_socket_address(2); + + let tx = test_transaction(28); + let txid = tx.txid(); + let t0 = Instant::now(); + let mut state = TxBroadcastState::new(tx, t0); + state.sent_to.insert(test_socket_address(1)); + state.holdout.insert(holdout); + manager.broadcasts.insert(txid, state); + + // Before the timeout nothing expires + let events = manager.expire_broadcasts_at(t0 + Duration::from_secs(1)); + assert!(events.is_empty()); + + // At the timeout the broadcast becomes uncertain, exactly once + let deadline = t0 + BroadcastConfig::default().acceptance_timeout; + let events = manager.expire_broadcasts_at(deadline); + assert!(matches!( + events.as_slice(), + [SyncEvent::TransactionBroadcastResult { + result: BroadcastResult::Uncertain, + .. + }] + )); + assert!(manager.expire_broadcasts_at(deadline + Duration::from_secs(9)).is_empty()); + assert_eq!(manager.broadcasts[&txid].status, BroadcastStatus::Uncertain); + + // A late echo still upgrades the outcome to accepted + let inv = vec![Inventory::Transaction(txid)]; + let events = manager.handle_inv(&inv, holdout, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 1); + assert_eq!(manager.broadcasts[&txid].status, BroadcastStatus::Accepted); + } + + #[tokio::test] + async fn test_zero_peers_at_broadcast_sends_on_next_tick() { + let (mut manager, requests, mut rx) = create_test_manager(); + + // No peers connected at broadcast time + let tx = test_transaction(29); + let txid = tx.txid(); + manager.handle_tx(tx, LOCAL_SENTINEL, &requests).await.unwrap(); + assert!(drain_tx_sends(&mut rx).is_empty()); + assert!(manager.broadcasts[&txid].sent_to.is_empty()); + + // A peer connects; the never-sent broadcast is due immediately + let peer = test_socket_address(1); + manager.peers.insert(peer, None); + manager.rebroadcast_if_due(&requests).await; + assert_eq!(drain_tx_sends(&mut rx), vec![peer]); + assert!(manager.broadcasts[&txid].sent_to.contains(&peer)); + } + + #[tokio::test] + async fn test_holdout_sticky_across_rebroadcasts() { + let (mut manager, requests, mut rx) = create_test_manager(); + let recipient = test_socket_address(1); + let holdout = test_socket_address(2); + manager.peers.insert(recipient, None); + manager.peers.insert(holdout, None); + + let tx = test_transaction(30); + let txid = tx.txid(); + let t0 = Instant::now(); + let mut state = TxBroadcastState::new(tx, t0); + state.sent_to.insert(recipient); + state.holdout.insert(holdout); + manager.broadcasts.insert(txid, state); + + let later = t0 + REBROADCAST_INTERVAL + Duration::from_secs(1); + manager.rebroadcast_if_due_at(&requests, later).await; + + // Only the original recipient is resent to; the holdout stays withheld + assert_eq!(drain_tx_sends(&mut rx), vec![recipient]); + assert_eq!(manager.broadcasts[&txid].holdout, [holdout].into_iter().collect()); + } + + #[tokio::test] + async fn test_holdout_repicked_when_all_holdouts_disconnect() { + let (mut manager, requests, mut rx) = create_test_manager(); + let recipient = test_socket_address(1); + let new_peer = test_socket_address(3); + manager.peers.insert(recipient, None); + manager.peers.insert(new_peer, None); + + let tx = test_transaction(31); + let txid = tx.txid(); + let t0 = Instant::now(); + let mut state = TxBroadcastState::new(tx, t0); + state.sent_to.insert(recipient); + // Original holdout peer already disconnected (not in manager.peers) + state.holdout.insert(test_socket_address(2)); + manager.broadcasts.insert(txid, state); + + let later = t0 + REBROADCAST_INTERVAL + Duration::from_secs(1); + manager.rebroadcast_if_due_at(&requests, later).await; + + // The never-sent connected peer becomes the replacement holdout, + // so the resend goes only to the original recipient. + assert_eq!(drain_tx_sends(&mut rx), vec![recipient]); + assert!(manager.broadcasts[&txid].holdout.contains(&new_peer)); + assert!(!manager.broadcasts[&txid].sent_to.contains(&new_peer)); + } + + #[tokio::test] + async fn test_acceptance_threshold_two_requires_two_peers() { + let wallet = Arc::new(RwLock::new(MockWallet::new())); + let (tx_chan, _rx) = mpsc::unbounded_channel::(); + let requests = RequestSender::new(tx_chan); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::FetchAll, + 1000, + 0, + BroadcastConfig { + acceptance_threshold: 2, + ..BroadcastConfig::default() + }, + ); + + let holdout1 = test_socket_address(2); + let holdout2 = test_socket_address(3); + let tx = test_transaction(32); + let txid = tx.txid(); + let mut state = TxBroadcastState::new(tx, Instant::now()); + state.sent_to.insert(test_socket_address(1)); + state.holdout.extend([holdout1, holdout2]); + manager.broadcasts.insert(txid, state); + + let inv = vec![Inventory::Transaction(txid)]; + let events = manager.handle_inv(&inv, holdout1, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 0, "one echo below threshold"); + + let events = manager.handle_inv(&inv, holdout2, &requests).await.unwrap(); + assert_eq!(accepted_event_count(&events), 1, "second distinct peer meets threshold"); + assert!(matches!( + events.as_slice(), + [SyncEvent::TransactionBroadcastResult { + result: BroadcastResult::Accepted { + relayed_by: 2 + }, + .. + }] + )); + } + + #[tokio::test] + async fn test_clear_pending_preserves_broadcasts() { + let (mut manager, _requests, _rx) = create_test_manager(); + + let tx = test_transaction(33); + let txid = tx.txid(); + manager.broadcasts.insert(txid, TxBroadcastState::new(tx, Instant::now())); + + manager.clear_pending(); + + assert!( + manager.broadcasts.contains_key(&txid), + "broadcast tracking must survive disconnects" + ); + } + + #[test] + fn test_prune_expired_removes_old_broadcasts() { + let (mut manager, _requests, _rx) = create_test_manager(); + let timeout = Duration::from_secs(2); + + let fresh = test_transaction(34); + let fresh_txid = fresh.txid(); + manager.broadcasts.insert(fresh_txid, TxBroadcastState::new(fresh, Instant::now())); + + let stale = test_transaction(35); + let stale_txid = stale.txid(); + let mut stale_state = + TxBroadcastState::new(stale, Instant::now() - timeout - Duration::from_secs(1)); + stale_state.status = BroadcastStatus::Uncertain; + manager.broadcasts.insert(stale_txid, stale_state); + + manager.prune_expired(timeout); + + assert!(manager.broadcasts.contains_key(&fresh_txid)); + assert!(!manager.broadcasts.contains_key(&stale_txid)); + } } diff --git a/dash-spv/src/sync/mempool/mod.rs b/dash-spv/src/sync/mempool/mod.rs index 94f1a84e0..f87322f2d 100644 --- a/dash-spv/src/sync/mempool/mod.rs +++ b/dash-spv/src/sync/mempool/mod.rs @@ -1,8 +1,10 @@ +mod broadcast; mod filter; mod manager; mod progress; mod sync_manager; +pub use broadcast::{BroadcastConfig, BroadcastHoldout, BroadcastResult}; pub(crate) use manager::MempoolManager; pub use progress::MempoolProgress; diff --git a/dash-spv/src/sync/mempool/sync_manager.rs b/dash-spv/src/sync/mempool/sync_manager.rs index f818ed42c..1e0e15569 100644 --- a/dash-spv/src/sync/mempool/sync_manager.rs +++ b/dash-spv/src/sync/mempool/sync_manager.rs @@ -50,7 +50,9 @@ impl SyncManager for MempoolManager { ) -> SyncResult> { match msg.inner() { NetworkMessage::Inv(inv) => self.handle_inv(inv, msg.peer_address(), requests).await, - NetworkMessage::Tx(tx) => self.handle_tx(tx.clone(), msg.peer_address()).await, + NetworkMessage::Tx(tx) => { + self.handle_tx(tx.clone(), msg.peer_address(), requests).await + } _ => Ok(vec![]), } } @@ -88,24 +90,26 @@ impl SyncManager for MempoolManager { // Remove confirmed transactions from mempool. // Bloom filter rebuild is handled by the tick's revision check. if !confirmed_txids.is_empty() { - self.remove_confirmed(confirmed_txids); + return Ok(self.remove_confirmed(confirmed_txids)); } Ok(vec![]) } SyncEvent::InstantLockReceived { instant_lock, .. - } => { - self.process_instant_send(instant_lock.clone()).await; - Ok(vec![]) - } + } => Ok(self.process_instant_send(instant_lock.clone()).await), _ => Ok(vec![]), } } async fn tick(&mut self, requests: &RequestSender) -> SyncResult> { + // Broadcast bookkeeping runs regardless of sync state: broadcasts can + // be initiated (and time out) before the mempool phase is synced. + let events = self.expire_broadcasts(); + self.rebroadcast_if_due(requests).await; + if self.state() != SyncState::Synced { - return Ok(vec![]); + return Ok(events); } // Prune expired transactions periodically @@ -117,9 +121,6 @@ impl SyncManager for MempoolManager { // Send queued getdata requests now that slots may have freed up self.send_queued(requests).await?; - // Rebroadcast unconfirmed self-sent transactions on a randomized interval - self.rebroadcast_if_due(requests).await; - // Rebuild bloom filter if the wallet's monitored set has changed. // // We poll the revision counter rather than using push-based wallet events @@ -137,7 +138,7 @@ impl SyncManager for MempoolManager { self.last_monitor_revision = current_revision; } - Ok(vec![]) + Ok(events) } async fn handle_network_event( @@ -191,6 +192,7 @@ mod tests { use super::*; use crate::client::config::MempoolStrategy; use crate::network::NetworkRequest; + use crate::sync::BroadcastConfig; use crate::test_utils::test_socket_address; use dashcore::hashes::Hash; use key_wallet_manager::test_utils::MockWallet; @@ -204,7 +206,13 @@ mod tests { let (tx, rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let manager = MempoolManager::new(wallet, MempoolStrategy::FetchAll, 1000, 0); + let manager = MempoolManager::new( + wallet, + MempoolStrategy::FetchAll, + 1000, + 0, + BroadcastConfig::default(), + ); (manager, requests, rx) } @@ -556,7 +564,13 @@ mod tests { let (tx, mut rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let mut manager = MempoolManager::new(wallet, MempoolStrategy::BloomFilter, 1000, 0); + let mut manager = MempoolManager::new( + wallet, + MempoolStrategy::BloomFilter, + 1000, + 0, + BroadcastConfig::default(), + ); let peer = test_socket_address(1); manager.handle_peer_connected(peer); @@ -631,6 +645,7 @@ mod tests { MempoolStrategy::BloomFilter, 1000, initial_revision, + BroadcastConfig::default(), ); let peer = test_socket_address(1); @@ -686,7 +701,13 @@ mod tests { let (tx, mut rx) = mpsc::unbounded_channel::(); let requests = RequestSender::new(tx); - let mut manager = MempoolManager::new(wallet.clone(), MempoolStrategy::FetchAll, 1000, 0); + let mut manager = MempoolManager::new( + wallet.clone(), + MempoolStrategy::FetchAll, + 1000, + 0, + BroadcastConfig::default(), + ); let peer = test_socket_address(1); manager.handle_peer_connected(peer); @@ -740,6 +761,7 @@ mod tests { MempoolStrategy::BloomFilter, 1000, initial_revision, + BroadcastConfig::default(), ); let peer = test_socket_address(1); @@ -789,6 +811,7 @@ mod tests { MempoolStrategy::BloomFilter, 1000, initial_revision, + BroadcastConfig::default(), ); let peer = test_socket_address(1); @@ -808,7 +831,7 @@ mod tests { output: vec![], special_transaction_payload: None, }; - manager.handle_tx(tx, test_socket_address(1)).await.unwrap(); + manager.handle_tx(tx, test_socket_address(1), &requests).await.unwrap(); let has_filter_load = std::iter::from_fn(|| rx.try_recv().ok()).any(|msg| { matches!(msg, NetworkRequest::SendMessageToPeer(NetworkMessage::FilterLoad(_), _)) diff --git a/dash-spv/src/sync/mod.rs b/dash-spv/src/sync/mod.rs index 8a6925ef7..f059ebc09 100644 --- a/dash-spv/src/sync/mod.rs +++ b/dash-spv/src/sync/mod.rs @@ -23,7 +23,7 @@ pub use filters::{FiltersManager, FiltersProgress}; pub use instantsend::{InstantSendManager, InstantSendProgress}; pub use masternodes::{MasternodesManager, MasternodesProgress}; pub(crate) use mempool::MempoolManager; -pub use mempool::MempoolProgress; +pub use mempool::{BroadcastConfig, BroadcastHoldout, BroadcastResult, MempoolProgress}; pub use events::SyncEvent; pub use identifier::ManagerIdentifier; diff --git a/dash-spv/tests/dashd_sync/tests_mempool.rs b/dash-spv/tests/dashd_sync/tests_mempool.rs index 29c407dd5..becb4598f 100644 --- a/dash-spv/tests/dashd_sync/tests_mempool.rs +++ b/dash-spv/tests/dashd_sync/tests_mempool.rs @@ -570,3 +570,132 @@ async fn test_broadcast_transaction_local_detection() { bf.stop().await; tracing::info!("test_broadcast_transaction_local_detection passed"); } + +/// Verify network acceptance detection with two dashd nodes. +/// +/// The SPV client connects to both nodes. A broadcast is sent to only a +/// subset of peers (half, per `BroadcastHoldout::Half`), so with two peers +/// exactly one node receives it directly. Acceptance is proven when the +/// withheld node — which can only learn the transaction through node-to-node +/// relay — announces the txid back to the SPV client via `inv`. +#[tokio::test] +async fn test_broadcast_transaction_network_acceptance() { + let Some(ctx) = TestContext::new(TestChain::Minimal).await else { + return; + }; + if !ctx.dashd.supports_mining { + eprintln!("Skipping test (dashd RPC miner not available)"); + return; + } + + // Second dashd node on the same chain, connected to the first so + // transactions relay between them. + let Some(dashd2) = DashdTestContext::new(TestChain::Minimal).await else { + return; + }; + dashd2.node.connect_to_node(ctx.dashd.addr).await; + + // SPV client connected to BOTH nodes. + let storage = tempfile::TempDir::new().expect("Failed to create client temp dir"); + let mut config = dash_spv::ClientConfig::regtest() + .with_storage_path(storage.path().to_path_buf()) + .without_masternodes(); + config.add_peer(ctx.dashd.addr); + config.add_peer(dashd2.addr); + let (wallet, _) = create_test_wallet(&ctx.dashd.wallet.mnemonic, dash_spv::Network::Regtest); + let mut handle = create_and_start_client(&config, wallet).await; + + super::helpers::wait_for_sync(&mut handle.progress_receiver, ctx.dashd.initial_height).await; + assert!( + super::helpers::wait_for_mempool_synced(&mut handle.progress_receiver).await, + "expected mempool to reach Synced state" + ); + + // Wait until the network layer reports both peers connected — with a + // single peer no holdout is possible and the echo can never arrive. + let both_connected = wait_for_network_event( + &mut handle.network_event_receiver, + |event| { + matches!(event, NetworkEvent::PeersUpdated { connected_count, .. } if *connected_count >= 2) + }, + MEMPOOL_TIMEOUT, + ) + .await; + assert!(both_connected, "expected the SPV client to connect to both dashd nodes"); + + // Fund the SPV wallet and confirm the funding. + let receive_address = ctx.receive_address().await; + let funding_txid = + ctx.dashd.node.send_to_address(&receive_address, Amount::from_sat(200_000_000)); + super::helpers::wait_for_mempool_tx(&mut handle.wallet_event_receiver, MEMPOOL_TIMEOUT) + .await + .expect("Expected mempool event for funding tx"); + + let miner_address = ctx.dashd.node.get_new_address_from_wallet("default"); + ctx.dashd.node.generate_blocks(1, &miner_address); + super::helpers::wait_for_sync(&mut handle.progress_receiver, ctx.dashd.initial_height + 1) + .await; + + // Create a signed transaction without broadcasting it via dashd. + let wallet_name = &ctx.dashd.wallet.wallet_name; + let utxos = ctx.dashd.node.list_unspent_from_wallet(wallet_name); + let utxo = + utxos.iter().find(|u| u.txid == funding_txid).expect("Funding tx UTXO not found in wallet"); + let external_address = ctx.dashd.node.get_new_address_from_wallet("default"); + let signed_tx = ctx.dashd.node.create_signed_transaction( + wallet_name, + utxo.txid, + utxo.vout, + utxo.amount, + &external_address, + Amount::from_sat(10_000), + ); + let txid = signed_tx.txid(); + + // Broadcast through the SPV client and await the network outcome: the tx + // goes to one node, relays to the other, and the other's inv proves + // acceptance. + let result = handle + .client + .broadcast_transaction_and_wait(&signed_tx, Some(Duration::from_secs(45))) + .await + .expect("broadcast_transaction_and_wait failed"); + tracing::info!("Broadcast {} outcome: {}", txid, result); + match result { + dash_spv::BroadcastResult::Accepted { + relayed_by, + } => { + assert!(relayed_by >= 1, "acceptance requires at least one echoing peer") + } + other => panic!("expected network acceptance for {}, got {}", txid, other), + } + + // Negative leg: a double-spend of the same UTXO must NOT be reported as + // accepted. The receiving node refuses it (mempool conflict), so it never + // relays to the holdout peer and no echo arrives. Modern dashd sends no + // BIP61 reject, so the outcome resolves Uncertain via the timeout. + let double_spend = ctx.dashd.node.create_signed_transaction( + wallet_name, + utxo.txid, + utxo.vout, + utxo.amount, + &external_address, + Amount::from_sat(20_000), + ); + let ds_txid = double_spend.txid(); + assert_ne!(ds_txid, txid, "double spend must have a distinct txid"); + let ds_result = handle + .client + .broadcast_transaction_and_wait(&double_spend, Some(Duration::from_secs(10))) + .await + .expect("broadcast_transaction_and_wait failed for double spend"); + tracing::info!("Double-spend {} outcome: {}", ds_txid, ds_result); + assert!( + matches!(ds_result, dash_spv::BroadcastResult::Uncertain), + "double spend must resolve uncertain (no echo, no BIP61 reject), got {}", + ds_result + ); + + handle.stop().await; + tracing::info!("test_broadcast_transaction_network_acceptance passed"); +} From 4c1b468b7ab25d797f19c3bee21f12f601760c17 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Mon, 27 Jul 2026 05:11:32 -0700 Subject: [PATCH 055/149] fix(key-wallet): fix issue 911 (#914) --- .../managed_wallet_info/coin_selection.rs | 252 +++++++++++++++--- .../transaction_builder.rs | 42 ++- 2 files changed, 249 insertions(+), 45 deletions(-) diff --git a/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs b/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs index 5bbd7d53a..7cd72cc2e 100644 --- a/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs +++ b/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs @@ -7,6 +7,10 @@ use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::Utxo; use core::cmp::Reverse; +pub(crate) const TX_OUTPUT_SIZE: usize = 34; +pub(crate) const TX_INPUT_SIZE: usize = 148; +pub(crate) const CHANGE_OUTPUT_SIZE: usize = TX_OUTPUT_SIZE; + /// UTXO selection strategy #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum SelectionStrategy { @@ -110,9 +114,9 @@ impl CoinSelector { where I: IntoIterator, { - // Default base size assumes 2 outputs (target + change) - let default_base_size = 10 + (34 * 2); - let input_size = 148; + // Default base size assumes 2 outputs: the target output and one change output. + let default_base_size = 10 + TX_OUTPUT_SIZE + CHANGE_OUTPUT_SIZE; + let input_size = TX_INPUT_SIZE; self.select_coins_with_size( utxos, target_amount, @@ -120,10 +124,15 @@ impl CoinSelector { current_height, default_base_size, input_size, + CHANGE_OUTPUT_SIZE, ) } - /// Select UTXOs for a target amount with custom transaction size parameters + /// Select UTXOs for a target amount with custom transaction size parameters. + /// + /// `base_size` includes the change output; `change_output_size` is its byte count (`0` when + /// there is no change address), letting the accumulator drop it to size a no-change fee. + #[allow(clippy::too_many_arguments)] pub fn select_coins_with_size<'a, I>( &self, utxos: I, @@ -132,6 +141,7 @@ impl CoinSelector { current_height: u32, base_size: usize, input_size: usize, + change_output_size: usize, ) -> Result where I: IntoIterator, @@ -170,6 +180,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ) } SelectionStrategy::LargestFirst => { @@ -180,6 +191,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ) } SelectionStrategy::SmallestFirstTill(threshold) => { @@ -196,6 +208,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ) } else { // Split at threshold @@ -213,6 +226,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ) } } @@ -225,6 +239,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ) } SelectionStrategy::OptimalConsolidation => self @@ -234,6 +249,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ), _ => unreachable!(), } @@ -250,6 +266,7 @@ impl CoinSelector { fee_rate, base_size, input_size, + change_output_size, ) } SelectionStrategy::All => { @@ -287,7 +304,9 @@ impl CoinSelector { } } - /// Simple accumulation strategy with custom transaction size parameters + /// Accumulate UTXOs. `change_output_size` = the change output's bytes within `base_size` + /// (`0` if none is budgeted), shaved off to size the no-change fee. + #[allow(clippy::too_many_arguments)] fn accumulate_coins_with_size<'a, I>( &self, utxos: I, @@ -295,52 +314,79 @@ impl CoinSelector { fee_rate: FeeRate, base_size: usize, input_size: usize, + change_output_size: usize, ) -> Result where I: IntoIterator, { let mut selected = Vec::new(); let mut total_value = 0u64; + // Reported on shortfall so `available < required`, not the fee-excluded `target_amount` + // that looked like `available > required` (#911). + let mut required_no_change = target_amount; for utxo in utxos { total_value += utxo.value(); selected.push(utxo.clone()); - // Calculate size with current inputs + // `base_size` budgets a change output; a no-change send omits it and pays less. let estimated_size = base_size + (input_size * selected.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); - let required_amount = target_amount + estimated_fee; - - if total_value >= required_amount { - let change_amount = total_value - required_amount; + let size_no_change = estimated_size.saturating_sub(change_output_size); + let fee_no_change = fee_rate.calculate_fee(size_no_change); + required_no_change = target_amount + fee_no_change; - // Check if change is dust - let (final_change, exact_match) = if change_amount < self.dust_threshold { - // Add dust to fee - (0, change_amount == 0) - } else { - (change_amount, false) - }; + if total_value < required_no_change { + continue; + } + // Keep change only if one is budgeted and it clears dust (`>`, like the builder). + let required_with_change = target_amount + estimated_fee; + if change_output_size > 0 + && total_value >= required_with_change + && total_value - required_with_change > self.dust_threshold + { return Ok(SelectionResult { selected, total_value, target_amount, - change_amount: final_change, + change_amount: total_value - required_with_change, estimated_size, - estimated_fee: if final_change == 0 { - total_value - target_amount - } else { - estimated_fee - }, - exact_match, + estimated_fee, + exact_match: false, }); } + + // No change output budgeted but a real remainder: surface it as change so the builder + // adds one or errors, instead of burning a large surplus into the fee. + let remainder = total_value - required_no_change; + if change_output_size == 0 && remainder > self.dust_threshold { + return Ok(SelectionResult { + selected, + total_value, + target_amount, + change_amount: remainder, + estimated_size: size_no_change, + estimated_fee: fee_no_change, + exact_match: false, + }); + } + + // Otherwise the remainder is dust: drop the change output, fold it into the fee (#911). + return Ok(SelectionResult { + selected, + total_value, + target_amount, + change_amount: 0, + estimated_size: size_no_change, + estimated_fee: total_value - target_amount, + exact_match: total_value == required_no_change, + }); } Err(SelectionError::InsufficientFunds { available: total_value, - required: target_amount, + required: required_no_change, }) } @@ -357,6 +403,7 @@ impl CoinSelector { /// - Pros: Faster to find solutions due to aggressive pruning /// - Cons: May leave small UTXOs unconsolidated, leading to wallet fragmentation /// - Cons: Less likely to find exact matches with larger denominations + #[allow(clippy::too_many_arguments)] fn branch_and_bound_with_size<'a, I>( &self, utxos: I, @@ -364,6 +411,7 @@ impl CoinSelector { fee_rate: FeeRate, base_size: usize, input_size: usize, + change_output_size: usize, ) -> Result where I: IntoIterator, @@ -400,15 +448,14 @@ impl CoinSelector { }); } - // Fall back to accumulation if no exact match found - // For fallback, assume change output is needed - let base_size_with_change = base_size + 34; + // No exact match: accumulate. `base_size` already budgets the change output. self.accumulate_coins_with_size( sorted_refs, target_amount, fee_rate, - base_size_with_change, + base_size, input_size, + change_output_size, ) } @@ -435,6 +482,7 @@ impl CoinSelector { /// - During low-fee periods when consolidation is cheaper /// - For wallets that receive many small payments /// - When exact change is preferred to minimize privacy leaks + #[allow(clippy::too_many_arguments)] fn optimal_consolidation_with_size<'a>( &self, utxos: &[&'a Utxo], @@ -442,6 +490,7 @@ impl CoinSelector { fee_rate: FeeRate, base_size: usize, input_size: usize, + change_output_size: usize, ) -> Result { // First, try to find an exact match using smaller UTXOs // Sort by value ascending to prioritize using smaller UTXOs @@ -475,9 +524,8 @@ impl CoinSelector { } } - // If no exact match, try to minimize change while consolidating small UTXOs - // Use a combination of smallest UTXOs that slightly exceeds the target - let base_size_with_change = base_size + 34; // Add change output to base size + // Minimize change while consolidating small UTXOs. `base_size` already budgets the + // change output. let mut best_selection: Option> = None; let mut best_change = u64::MAX; @@ -490,13 +538,13 @@ impl CoinSelector { current_total += utxo.value(); } - let estimated_size = base_size_with_change + (input_size * current.len()); + let estimated_size = base_size + (input_size * current.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); let required = target_amount + estimated_fee; if current_total >= required { let change = current_total - required; - if change < best_change && change >= self.dust_threshold { + if change < best_change && change > self.dust_threshold { best_selection = Some(current); best_change = change; } @@ -504,7 +552,7 @@ impl CoinSelector { } if let Some(selected) = best_selection { - let estimated_size = base_size_with_change + (input_size * selected.len()); + let estimated_size = base_size + (input_size * selected.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); let total_value: u64 = selected.iter().map(|u| u.value()).sum(); @@ -519,15 +567,14 @@ impl CoinSelector { }); } - // Fall back to accumulate if we couldn't find a good solution - // For fallback, assume change output is needed - let base_size_with_change = base_size + 34; + // Fall back to accumulate; `base_size` already budgets the change output. self.accumulate_coins_with_size( sorted_asc, target_amount, fee_rate, - base_size_with_change, + base_size, input_size, + change_output_size, ) } @@ -694,6 +741,7 @@ impl std::error::Error for SelectionError {} #[cfg(test)] mod tests { use super::*; + use test_case::test_case; #[test] fn test_select_all_drains_everything() { @@ -767,6 +815,132 @@ mod tests { assert!(matches!(result, Err(SelectionError::InsufficientFunds { .. }))); } + /// #911: a no-change send is wrongly rejected with a misleading `InsufficientFunds` + /// (available > required) because the fee is sized for a change output that won't exist. + /// UTXO 150_200, send 150_000 (fee 200, no change) — must succeed for every strategy. + #[test_case(SelectionStrategy::SmallestFirst ; "smallest_first")] + #[test_case(SelectionStrategy::LargestFirst ; "largest_first")] + #[test_case(SelectionStrategy::SmallestFirstTill(2) ; "smallest_first_till")] + #[test_case(SelectionStrategy::BranchAndBound ; "branch_and_bound")] + #[test_case(SelectionStrategy::OptimalConsolidation ; "optimal_consolidation")] + #[test_case(SelectionStrategy::Random ; "random")] + fn test_zero_or_dust_change_send_not_rejected_issue_911(strategy: SelectionStrategy) { + let utxos = vec![Utxo::dummy(0, 150_200, 100, false, true)]; + let target = 150_000; + + let result = + CoinSelector::new(strategy).select_coins(&utxos, target, FeeRate::normal(), 200); + + // The bug's tell-tale: InsufficientFunds with available > required. + if let Err(SelectionError::InsufficientFunds { + available, + required, + }) = &result + { + assert!( + available < required, + "{strategy:?}: misleading InsufficientFunds — available {available} > required \ + {required}; a no-change tx (fee {}) fits, so this send must not be rejected", + available - target + ); + } + + let selection = + result.unwrap_or_else(|e| panic!("{strategy:?}: valid no-change send rejected: {e:?}")); + assert_eq!( + selection.change_amount, 0, + "{strategy:?}: send-everything-minus-fee has no change" + ); + assert_eq!(selection.total_value, 150_200, "{strategy:?}"); + assert_eq!(selection.estimated_fee, 200, "{strategy:?}: whole remainder becomes the fee"); + } + + /// #911 with its exact figures, through the fallback path: `BranchAndBound` (the builder + /// default) and `OptimalConsolidation` used to hand the accumulator a `base_size` inflated by + /// a phantom change output. UTXO 10_000_000, send 9_999_780 (fee 220, no change) — must hold + /// for every strategy, not just those that skip the fallback. + #[test_case(SelectionStrategy::SmallestFirst ; "smallest_first")] + #[test_case(SelectionStrategy::LargestFirst ; "largest_first")] + #[test_case(SelectionStrategy::SmallestFirstTill(2) ; "smallest_first_till")] + #[test_case(SelectionStrategy::BranchAndBound ; "branch_and_bound_default")] + #[test_case(SelectionStrategy::OptimalConsolidation ; "optimal_consolidation")] + #[test_case(SelectionStrategy::Random ; "random")] + fn test_issue_911_exact_figures_no_phantom_change(strategy: SelectionStrategy) { + let utxos = vec![Utxo::dummy(0, 10_000_000, 100, false, true)]; + let target = 9_999_780; + + let result = + CoinSelector::new(strategy).select_coins(&utxos, target, FeeRate::normal(), 200); + + let selection = result.unwrap_or_else(|e| { + panic!("{strategy:?}: #911 send rejected (phantom change output in fallback?): {e:?}") + }); + assert_eq!(selection.change_amount, 0, "{strategy:?}: no change on a send-everything"); + assert_eq!( + selection.estimated_fee, 220, + "{strategy:?}: the whole 220-duff remainder is the fee (no change output budgeted)" + ); + } + + /// #911 gap 2: with no change address, `base_size` has no change output and + /// `change_output_size` is 0 — the accumulator must NOT shave a phantom 34 bytes off the fee. + /// Sending everything but a 158-duff remainder underpays the real 192-byte fee and must be + /// rejected, not silently accepted below the min-relay fee. + #[test_case(SelectionStrategy::SmallestFirst ; "smallest_first")] + #[test_case(SelectionStrategy::LargestFirst ; "largest_first")] + #[test_case(SelectionStrategy::SmallestFirstTill(2) ; "smallest_first_till")] + #[test_case(SelectionStrategy::BranchAndBound ; "branch_and_bound")] + #[test_case(SelectionStrategy::OptimalConsolidation ; "optimal_consolidation")] + #[test_case(SelectionStrategy::Random ; "random")] + fn test_no_change_address_does_not_underpay_issue_911_gap2(strategy: SelectionStrategy) { + let utxos = vec![Utxo::dummy(0, 10_000_000, 100, false, true)]; + let target = 10_000_000 - 158; // only affordable if the fee were under-sized to 158 + let base_size = 10 + 34; // overhead + target output, NO change output + let result = CoinSelector::new(strategy).select_coins_with_size( + &utxos, + target, + FeeRate::normal(), + 200, + base_size, + 148, + 0, // no change address => no change output to drop + ); + assert!( + matches!(result, Err(SelectionError::InsufficientFunds { .. })), + "{strategy:?}: accepted an underpaid (158 < 192) no-change tx: {result:?}" + ); + } + + /// Review finding: with no change address (`change_output_size == 0`) and a remainder well + /// above dust, the leftover must be reported as change (so the builder can add a change output + /// or error), NOT silently folded into the fee. Sending 100_000 out of a 10_000_000 UTXO leaves + /// ~9.9M that must not become fee. + #[test_case(SelectionStrategy::SmallestFirst ; "smallest_first")] + #[test_case(SelectionStrategy::LargestFirst ; "largest_first")] + #[test_case(SelectionStrategy::SmallestFirstTill(2) ; "smallest_first_till")] + #[test_case(SelectionStrategy::BranchAndBound ; "branch_and_bound")] + #[test_case(SelectionStrategy::OptimalConsolidation ; "optimal_consolidation")] + #[test_case(SelectionStrategy::Random ; "random")] + fn test_no_change_address_surplus_not_burned_as_fee(strategy: SelectionStrategy) { + let utxos = vec![Utxo::dummy(0, 10_000_000, 100, false, true)]; + let target = 100_000; + let base_size = 10 + 34; // one (target) output, no change output budgeted + let selection = CoinSelector::new(strategy) + .select_coins_with_size(&utxos, target, FeeRate::normal(), 200, base_size, 148, 0) + .unwrap_or_else(|e| panic!("{strategy:?}: {e:?}")); + + // No-change fee is 192 (44 + 148); the ~9.9M remainder is change, not fee. + assert_eq!( + selection.estimated_fee, 192, + "{strategy:?}: only the no-change fee is charged, not the whole surplus" + ); + assert_eq!( + selection.change_amount, + 10_000_000 - target - 192, + "{strategy:?}: the surplus is surfaced as change, not burned" + ); + } + #[test] fn test_optimal_consolidation_strategy() { // Test that OptimalConsolidation strategy works correctly diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index b2afbefe7..4d9743aa3 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -5,7 +5,9 @@ use crate::managed_account::reservation::ReservationSet; use crate::managed_account::ManagedCoreFundsAccount; -use crate::wallet::managed_wallet_info::coin_selection::{CoinSelector, SelectionStrategy}; +use crate::wallet::managed_wallet_info::coin_selection::{ + CoinSelector, SelectionStrategy, CHANGE_OUTPUT_SIZE, TX_INPUT_SIZE, TX_OUTPUT_SIZE, +}; use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::{Account, DerivationPath, Signer, Utxo, Wallet}; use core::fmt; @@ -186,12 +188,12 @@ impl TransactionBuilder { }, ); - // Add outputs size (TX_OUTPUT_SIZE = 34 bytes per P2PKH output) - size += outputs_count * 34; + // Add outputs size (P2PKH output) + size += outputs_count * TX_OUTPUT_SIZE; // Add change output if we have a change address if self.change_addr.is_some() { - size += 34; // TX_OUTPUT_SIZE + size += CHANGE_OUTPUT_SIZE; } // Add special payload size if present @@ -263,7 +265,8 @@ impl TransactionBuilder { } TransactionPayload::AssetLockPayloadType(p) => { // version (1) + creditOutputsCount + creditOutputs - 1 + varint_size(p.credit_outputs.len()) + p.credit_outputs.len() * 34 + 1 + varint_size(p.credit_outputs.len()) + + p.credit_outputs.len() * TX_OUTPUT_SIZE } TransactionPayload::AssetUnlockPayloadType(_p) => { // version (1) + index (8) + fee (4) + requestHeight (4) + quorumHash (32) + quorumSig (96) @@ -308,6 +311,13 @@ impl TransactionBuilder { self.inputs.retain(|utxo| utxo.is_confirmed || utxo.is_instantlocked); } + // Matches `calculate_base_size`: a change output is only budgeted with a change address. + let change_output_size = if self.change_addr.is_some() { + CHANGE_OUTPUT_SIZE + } else { + 0 + }; + let selection = CoinSelector::new(self.selection_strategy) .select_coins_with_size( self.inputs.iter(), @@ -315,7 +325,8 @@ impl TransactionBuilder { self.fee_rate, self.current_height, self.calculate_base_size(), - 148, // Size per P2PKH input + TX_INPUT_SIZE, + change_output_size, ) .map_err(BuilderError::CoinSelection)?; @@ -697,6 +708,25 @@ mod tests { )); } + #[test] + fn test_no_change_address_large_surplus_errors_not_burned() { + // Review finding: a send that leaves a large remainder but has no change address must + // error (NoChangeAddress), not silently pay the entire surplus to miners. + let utxo = Utxo::dummy(0, 10_000_000, 100, false, true); + let destination = Address::dummy(Network::Testnet, 0); + + let result = TransactionBuilder::new() + .set_current_height(200) + .add_inputs([utxo]) // no set_change_address + .add_output(&destination, 100_000) + .build_unsigned(); + + assert!( + matches!(result, Err(BuilderError::NoChangeAddress)), + "surplus with no change address must error, got {result:?}" + ); + } + #[test] fn test_asset_lock_transaction() { // Test based on DSTransactionTests.m testAssetLockTx1 From ce4f26ca32a85bcfb3e3e57cc622a2027066a91b Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Mon, 27 Jul 2026 05:29:11 -0700 Subject: [PATCH 056/149] feat(dash-spv-bench): multi-scenario sync benchmark harness (#908) A docker-driven benchmark that syncs the SPV client against a frozen chain snapshot (local peers via tc netem) or the real testnet/mainnet, reporting sync time and a live dashboard. Base branch for the network-rewrite PR stacked on top; it drives the rewritten client and does not build on dev alone. Claude-Session: https://claude.ai/code/session_01CUb3bkX9C1gBFA65GFsN53 Co-authored-by: Claude Opus 4.8 (1M context) --- .github/ci-groups.yml | 1 + Cargo.toml | 2 +- dash-spv-bench/.gitignore | 24 ++ dash-spv-bench/Cargo.toml | 25 ++ dash-spv-bench/Dockerfile | 24 ++ dash-spv-bench/bench-scenarios.sh | 88 +++++ dash-spv-bench/run.sh | 339 ++++++++++++++++++ dash-spv-bench/scenario.example.yml | 38 ++ dash-spv-bench/scenarios/congested-2core.yml | 5 + dash-spv-bench/scenarios/ideal-1-peer.yml | 5 + dash-spv-bench/scenarios/ideal-3-peers.yml | 5 + dash-spv-bench/scenarios/ideal-5-peers.yml | 5 + .../scenarios/no-lag-ideal-1-peer.yml | 5 + .../scenarios/no-lag-ideal-3-peers.yml | 5 + .../scenarios/no-lag-ideal-5-peers.yml | 5 + dash-spv-bench/scenarios/one-bad-peer.yml | 6 + dash-spv-bench/scenarios/real-mainnet.yml | 3 + dash-spv-bench/scenarios/real-testnet.yml | 3 + dash-spv-bench/scenarios/slow-many-peers.yml | 11 + dash-spv-bench/snapshot-chain.sh | 29 ++ dash-spv-bench/src/dashboard.rs | 217 +++++++++++ dash-spv-bench/src/main.rs | 239 ++++++++++++ dash-spv-bench/src/metrics.rs | 154 ++++++++ 23 files changed, 1237 insertions(+), 1 deletion(-) create mode 100644 dash-spv-bench/.gitignore create mode 100644 dash-spv-bench/Cargo.toml create mode 100644 dash-spv-bench/Dockerfile create mode 100755 dash-spv-bench/bench-scenarios.sh create mode 100755 dash-spv-bench/run.sh create mode 100644 dash-spv-bench/scenario.example.yml create mode 100644 dash-spv-bench/scenarios/congested-2core.yml create mode 100644 dash-spv-bench/scenarios/ideal-1-peer.yml create mode 100644 dash-spv-bench/scenarios/ideal-3-peers.yml create mode 100644 dash-spv-bench/scenarios/ideal-5-peers.yml create mode 100644 dash-spv-bench/scenarios/no-lag-ideal-1-peer.yml create mode 100644 dash-spv-bench/scenarios/no-lag-ideal-3-peers.yml create mode 100644 dash-spv-bench/scenarios/no-lag-ideal-5-peers.yml create mode 100644 dash-spv-bench/scenarios/one-bad-peer.yml create mode 100644 dash-spv-bench/scenarios/real-mainnet.yml create mode 100644 dash-spv-bench/scenarios/real-testnet.yml create mode 100644 dash-spv-bench/scenarios/slow-many-peers.yml create mode 100755 dash-spv-bench/snapshot-chain.sh create mode 100644 dash-spv-bench/src/dashboard.rs create mode 100644 dash-spv-bench/src/main.rs create mode 100644 dash-spv-bench/src/metrics.rs diff --git a/.github/ci-groups.yml b/.github/ci-groups.yml index cfd026315..aae748262 100644 --- a/.github/ci-groups.yml +++ b/.github/ci-groups.yml @@ -29,4 +29,5 @@ excluded: - integration_test # Requires live Dash node - dash-fuzz # Honggfuzz binary targets, tested by fuzz.yml - masternode-seeds-fetcher # Tooling binary; needs live dashd RPC, exercised by update-masternode-seeds.yml + - dash-spv-bench # Benchmarking binary - git-state # Compile-time proc-macro with no tests, exercised via dash-spv diff --git a/Cargo.toml b/Cargo.toml index f885763f1..63049fbea 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,5 +1,5 @@ [workspace] -members = ["dash", "dash-network", "hashes", "internals", "fuzz", "rpc-client", "rpc-json", "rpc-integration-test", "key-wallet", "key-wallet-manager", "key-wallet-ffi", "dash-spv", "dash-spv-ffi", "git-state", "dash-network-seeds", "masternode-seeds-fetcher"] +members = ["dash", "dash-network", "hashes", "internals", "fuzz", "rpc-client", "rpc-json", "rpc-integration-test", "key-wallet", "key-wallet-manager", "key-wallet-ffi", "dash-spv", "dash-spv-ffi", "git-state", "dash-network-seeds", "masternode-seeds-fetcher", "dash-spv-bench"] resolver = "2" [workspace.package] diff --git a/dash-spv-bench/.gitignore b/dash-spv-bench/.gitignore new file mode 100644 index 000000000..fb948c9f4 --- /dev/null +++ b/dash-spv-bench/.gitignore @@ -0,0 +1,24 @@ +bench-results/ +bench-storage/ +profiles/ + +*.lock + +# Persistent testnet chain snapshot grown by snapshot-chain.sh (multi-GB; never committed). +chain-data/ + +# Transient per-peer CoW clones created by run.sh + the file that remembers their dir. +.bench-clones.* +.clonedir + +# Scenario matrix outputs (bench-scenarios.sh): generated reports/logs/compose, and the +# bootstrapped yq binary. Reports are meant to be copied out, not committed. +.scenario.*.yml +results/ +.bin/ + +# FlameGraph tooling clone (run.sh --flame), fetched on demand. +.flamegraph/ + +# Wallet mnemonics (one BIP39 phrase per line) — secrets, never committed. +wallets.txt diff --git a/dash-spv-bench/Cargo.toml b/dash-spv-bench/Cargo.toml new file mode 100644 index 000000000..0cf11a8b9 --- /dev/null +++ b/dash-spv-bench/Cargo.toml @@ -0,0 +1,25 @@ +[package] +name = "dash-spv-bench" +version = { workspace = true } +edition = "2021" +license = "MIT" +publish = false + +[dependencies] +dash-spv = { path = "../dash-spv" } +dashcore = { path = "../dash", features = ["serde", "core-block-hash-use-x11", "message_verification", "bls", "quorum_validation"] } +key-wallet = { path = "../key-wallet" } +key-wallet-manager = { path = "../key-wallet-manager", features = ["parallel-filters"] } + +tokio = { version = "1.0", features = ["full"] } + +anyhow = "1.0" +tempfile = "3.0" + +tracing = "0.1" +tracing-subscriber = { version = "0.3.20", features = ["env-filter"] } +indicatif = "0.18" + +[[bin]] +name = "dash-spv-bench" +path = "src/main.rs" diff --git a/dash-spv-bench/Dockerfile b/dash-spv-bench/Dockerfile new file mode 100644 index 000000000..65b6fde46 --- /dev/null +++ b/dash-spv-bench/Dockerfile @@ -0,0 +1,24 @@ +FROM debian:bookworm-slim + +ARG DASHVERSION=23.1.7 + +RUN apt-get update \ + && apt-get install -y --no-install-recommends ca-certificates curl iproute2 \ + && rm -rf /var/lib/apt/lists/* + +# Download the Linux dashd matching the host CPU arch (amd64 -> x86_64, arm64 -> aarch64). +# The version is pinned so benchmark numbers stay comparable; run.sh and snapshot-chain.sh tag the +# image with it and pass it back as --build-arg, so bumping it there rebuilds instead of reusing. +RUN set -eux; \ + arch="$(dpkg --print-architecture)"; \ + case "$arch" in \ + amd64) dlarch=x86_64 ;; \ + arm64) dlarch=aarch64 ;; \ + *) echo "unsupported arch $arch" >&2; exit 1 ;; \ + esac; \ + url="https://github.com/dashpay/dash/releases/download/v${DASHVERSION}/dashcore-${DASHVERSION}-${dlarch}-linux-gnu.tar.gz"; \ + curl -fsSL "$url" -o /tmp/dashcore.tgz; \ + tar -xzf /tmp/dashcore.tgz -C /tmp; \ + cp "/tmp/dashcore-${DASHVERSION}/bin/dashd" /usr/local/bin/dashd; \ + rm -rf /tmp/dashcore*; \ + dashd --version | head -1 diff --git a/dash-spv-bench/bench-scenarios.sh b/dash-spv-bench/bench-scenarios.sh new file mode 100755 index 000000000..7e1ffd744 --- /dev/null +++ b/dash-spv-bench/bench-scenarios.sh @@ -0,0 +1,88 @@ +#!/bin/bash +# +# dash-spv-bench scenario matrix (LOCAL mode only). +# +# ./bench-scenarios.sh [-d scenarios] +# +# Thin loop over scenario files: for each scenarios/*.yml it calls `run.sh scenario ` +# once per its `repeats:` count, captures the timings run.sh prints, and writes an organized +# Markdown report (median over repeats). All the real work — compose generation, bring-up, +# CPU pinning, teardown — lives in run.sh; this only orchestrates and aggregates. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +cd "${SCRIPT_DIR}" + +SCN_DIR="${SCRIPT_DIR}/scenarios" +RUN_CMD="${RUN_CMD:-${SCRIPT_DIR}/run.sh}" # overridable for testing the loop without docker +while [ $# -gt 0 ]; do + case "$1" in + -d | --dir) SCN_DIR="$2"; shift 2 ;; + -h | --help) sed -n '3,10p' "$0"; exit 0 ;; + *) echo "unknown arg: $1" >&2; exit 1 ;; + esac +done + +shopt -s nullglob +FILES=("${SCN_DIR}"/*.yml) +[ "${#FILES[@]}" -gt 0 ] || { echo "Error: no scenarios (*.yml) in ${SCN_DIR}" >&2; exit 1; } + +RUN_TS="$(date +%Y%m%d-%H%M%S)" +OUT_DIR="${SCRIPT_DIR}/results/${RUN_TS}" +mkdir -p "${OUT_DIR}" +REPORT="${OUT_DIR}/report.md" +TSV="${OUT_DIR}/results.tsv" +: >"${TSV}" + +metric() { awk -F':[[:space:]]*' -v k="$2" '$1==k {gsub(/[[:space:]]+$/,"",$2); print $2; exit}' "$1"; } +median() { sort -n | awk '{a[NR]=$1} END{if(NR==0){print"-";exit} print (NR%2)?a[(NR+1)/2]:int((a[NR/2]+a[NR/2+1])/2)}'; } + +for f in "${FILES[@]}"; do + name="$(basename "${f}" .yml)" + repeats="$(awk '/^repeats:/{print $2; exit}' "${f}")"; repeats="${repeats:-1}" + echo "===== scenario '${name}' × ${repeats} =====" + for r in $(seq 1 "${repeats}"); do + echo "-- ${name} repeat ${r}/${repeats}" + log="${OUT_DIR}/${name}.r${r}.log" + "${RUN_CMD}" "${f}" >"${log}" 2>&1 || echo " (run.sh exited non-zero; see ${log})" + + # Scenario metadata comes straight from run.sh's own echo lines, so we never re-parse the YAML. + meta="$(grep -m1 "==> scenario '" "${log}" || true)" + cpus="$(sed -n "s/.*cpus=\([^,]*\),.*/\1/p" <<<"${meta}")" + blocks="$(sed -n "s/.*blocks=\([0-9]*\).*/\1/p" <<<"${meta}")" + peers="$(sed -n "s/.*\[\(.*\)\], cpus=.*/\1/p" <<<"${meta}")" + desc="$(grep -m1 "==> description:" "${log}" | sed 's/.*==> description: //' || true)" + + printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + "${name}" "${cpus}" "${blocks}" "${peers}" "${r}" \ + "$(metric "${log}" completed)" "$(metric "${log}" total_ms)" \ + "$(metric "${log}" block_headers_ms)" "$(metric "${log}" filter_headers_ms)" \ + "$(metric "${log}" filters_ms)" "$(metric "${log}" filter_hdr_lag_ms)" "${desc}" \ + >>"${TSV}" + done +done + +# --- report --------------------------------------------------------------------------------- +{ + echo "# dash-spv-bench scenario report" + echo + echo "- **Date:** $(date -u '+%Y-%m-%d %H:%M:%SZ')" + echo "- **Host:** $(nproc) cores — $(grep -m1 'model name' /proc/cpuinfo 2>/dev/null | cut -d: -f2 | sed 's/^ //' || uname -m)" + echo "- **git:** $(git -C "${REPO_ROOT}" rev-parse --abbrev-ref HEAD 2>/dev/null)@$(git -C "${REPO_ROOT}" rev-parse --short HEAD 2>/dev/null)" + echo "- **Scenarios:** \`$(basename "${SCN_DIR}")/\` (${#FILES[@]} files)" + echo + echo "Timings are the **median** over each scenario's repeats (ms)." + echo + echo "| scenario | description | cpus | blocks | peers | completed | total | block_hdrs | filter_hdrs | filters | fh_lag |" + echo "|---|---|---|---|---|---|---|---|---|---|---|" + for name in $(cut -f1 "${TSV}" | awk '!seen[$0]++'); do + row() { awk -F'\t' -v n="${name}" '$1==n{print $'"$1"'; exit}' "${TSV}"; } + med() { awk -F'\t' -v n="${name}" '$1==n && $'"$1"'!="" && $'"$1"'!="-"{print $'"$1"'}' "${TSV}" | median; } + echo "| ${name} | $(row 12) | $(row 2) | $(row 3) | $(row 4) | $(row 6) | $(med 7) | $(med 8) | $(med 9) | $(med 10) | $(med 11) |" + done + echo + echo "Raw per-repeat data: \`results.tsv\`; per-run logs alongside this report." +} >"${REPORT}" + +echo "==> wrote ${REPORT}" diff --git a/dash-spv-bench/run.sh b/dash-spv-bench/run.sh new file mode 100755 index 000000000..cd56ec8a5 --- /dev/null +++ b/dash-spv-bench/run.sh @@ -0,0 +1,339 @@ +#!/bin/bash +# +# dash-spv benchmark driver — runs ONE scenario end to end (build, bring up peers, sync, report). +# +# Usage: +# ./run.sh Run the scenario +# ./run.sh --flame Same, under the sampler (perf on Linux, sample on macOS) +# -> profiles/flamegraph.svg +# --wallets Wallets for this run: a file with one BIP39 mnemonic per +# line. No file => the run has no wallet. +# +# RUST_LOG can be set to tweak logging, e.g. RUST_LOG=info ./run.sh scenarios/ideal-1-peer.yml +# It is the tracing filter for BOTH log sinks and overrides the defaults, which are +# terminal = "warn,dash_spv_bench=info" (kept light so the live bars stay readable) +# file = "warn,dash_spv=debug,dash_spv_bench=debug" (debug, for offline analysis) +# +# Outputs land in bench-storage/ (gitignored, wiped at the start of each run): +# run.log full trace of the sync (debug by default) for offline analysis +# summary.txt metrics + per-wallet tx/balance fingerprint (also printed to stdout) +# plus the SPV storage the run produced (block_headers/, filters/, ...) +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +COMPOSE_FILE="" # local mode generates one here; deleted on exit +DASH_VERSION="23.1.7" +IMAGE="dash-spv-bench/dashd:${DASH_VERSION}" +PROJECT="spv-bench" +STATE="${SCRIPT_DIR}/.clonedir" +FLAME_SVG="${SCRIPT_DIR}/profiles/flamegraph.svg" +FLAMEGRAPH_DIR="${SCRIPT_DIR}/.flamegraph" # FlameGraph tooling clone (gitignored, inside the package) +CHAIN_DIR="${SCRIPT_DIR}/chain-data" + +INVOCATION_DIR="${PWD}" +abspath() { case "$1" in /*) printf '%s\n' "$1" ;; *) printf '%s\n' "${INVOCATION_DIR%/}/$1" ;; esac; } + +cd "${SCRIPT_DIR}" + +FLAME=0 +SCN_FILE="" +WALLETS_ARG="" +while [ $# -gt 0 ]; do + case "$1" in + --flame) FLAME=1; shift ;; + --wallets) WALLETS_ARG="${2:?--wallets needs a file path}"; shift 2 ;; + -h | --help) sed -n '3,20p' "$0"; exit 0 ;; + -*) echo "unknown flag: $1" >&2; exit 1 ;; + *) SCN_FILE="$1"; shift ;; + esac +done +[ -n "${SCN_FILE}" ] || { echo "usage: $0 [--flame] [--wallets ]" >&2; exit 1; } +SCN_FILE="$(abspath "${SCN_FILE}")" +[ -f "${SCN_FILE}" ] || { echo "Error: scenario file not found: ${SCN_FILE}" >&2; exit 1; } + +YQ_VERSION="v4.44.6" +ensure_yq() { + if command -v yq >/dev/null 2>&1; then YQ=yq; return 0; fi + local bin="${SCRIPT_DIR}/.bin/yq" + if [ ! -x "${bin}" ]; then + mkdir -p "${SCRIPT_DIR}/.bin" + local os arch + os="$(uname -s | tr '[:upper:]' '[:lower:]')"; arch="$(uname -m)" + case "${arch}" in x86_64 | amd64) arch=amd64 ;; aarch64 | arm64) arch=arm64 ;; esac + echo "==> fetching yq ${YQ_VERSION} (${os}/${arch}) into .bin/yq" + curl -fsSL "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_${os}_${arch}" \ + -o "${bin}" || { echo "Error: could not download yq; install it manually." >&2; exit 1; } + chmod +x "${bin}" + fi + YQ="${bin}" +} +ensure_yq +scn() { "${YQ}" "$1" "${SCN_FILE}"; } # evaluate a yq expression against the scenario file + +_peer_group() { + "${YQ}" ".peers[$1] | [.count, .latency_ms // 0, .jitter_ms // 0, .loss_pct // 0, .rate_kbit // 0, .corrupt_pct // 0, .reorder_pct // 0] | @tsv" "${SCN_FILE}" +} + +build_netem() { + local lat="$1" jit="$2" loss="$3" rate="$4" corrupt="$5" reorder="$6" a="" + [ "${lat}" != 0 ] && { a="delay ${lat}ms"; [ "${jit}" != 0 ] && a="${a} ${jit}ms"; } + [ "${loss}" != 0 ] && a="${a} loss ${loss}%" + [ "${rate}" != 0 ] && a="${a} rate ${rate}kbit" + [ "${corrupt}" != 0 ] && a="${a} corrupt ${corrupt}%" + [ "${reorder}" != 0 ] && a="${a} reorder ${reorder}%" + echo "${a# }" +} + +peers_summary() { + local ng g count lat jit loss rate corrupt reorder tag out="" + ng="$(scn '.peers | length')" + for g in $(seq 0 $((ng - 1))); do + read -r count lat jit loss rate corrupt reorder <<<"$(_peer_group "${g}")" + tag="${lat}ms" + [ "${jit}" != 0 ] && tag="${tag}±${jit}" + [ "${loss}" != 0 ] && tag="${tag}/${loss}%loss" + [ "${rate}" != 0 ] && tag="${tag}/${rate}kbit" + out="${out}, ${count}×${tag}" + done + echo "${out#, }" +} + +emit_compose() { + local out="$1" peer_cpus="$2" + cat >"${out}" <
>"${out}" <<'ANCHOR' + build: + context: . + dockerfile: Dockerfile + cap_add: [NET_ADMIN] + entrypoint: ["/bin/sh", "-ec"] + command: + - | + if [ -n "$${NETEM_ARGS:-}" ]; then + tc qdisc add dev eth0 root netem $${NETEM_ARGS} \ + && echo "netem: $${NETEM_ARGS}" || echo "WARNING: netem failed (NET_ADMIN/sch_netem?)" + fi + exec dashd -testnet -datadir=/data -port=19400 -rpcport=19500 -server=1 -daemon=0 \ + -connect=0 -bind=0.0.0.0 -listen=1 -rpcbind=0.0.0.0 -rpcallowip=0.0.0.0/0 \ + -whitelist=0.0.0.0/0 -disablewallet=1 -peerbloomfilters=1 \ + -dbcache=64 -fallbackfee=0.00001 -txindex=0 -addressindex=0 $${FILTER_FLAGS} + +services: +ANCHOR + local ng g count lat jit loss rate corrupt reorder netem n peer=0 + ng="$(scn '.peers | length')" + for g in $(seq 0 $((ng - 1))); do + read -r count lat jit loss rate corrupt reorder <<<"$(_peer_group "${g}")" + netem="$(build_netem "${lat}" "${jit}" "${loss}" "${rate}" "${corrupt}" "${reorder}")" + for n in $(seq 1 "${count}"); do + peer=$((peer + 1)) + cat >>"${out}" <&2; exit 1 ;; esac +export BENCH_MODE="${MODE}" +export CLONE_DIR="${CLONE_DIR:-/nonexistent}" + +BENCH_CPUS="$(scn '.cpus // ""')" +BENCH_MAX_PEERS="$(scn '.max_peers // ""')" # only if set; else the ClientConfig default +export BENCH_MAX_PEERS + +BENCH_WALLET_FILE="${SCRIPT_DIR}/wallets.txt" +[ -n "${WALLETS_ARG}" ] && BENCH_WALLET_FILE="$(abspath "${WALLETS_ARG}")" +export BENCH_WALLET_FILE +export BENCH_STORAGE_DIR="${SCRIPT_DIR}/bench-storage" + +DESC="$(scn '.description // ""')" +[ -n "${DESC}" ] && echo "==> description: ${DESC}" + +BLOCKS="" +if [ "${MODE}" = local ]; then + BLOCKS="$(scn '.blocks // 1000000')" + export BENCH_HEIGHT="${BLOCKS}" + unset BENCH_START_HEIGHT # local always syncs from genesis +else + BENCH_PEERS="$(scn '.peers // [] | join(",")')" + export BENCH_PEERS + sh="$(scn '.start_height // ""')"; [ -n "${sh}" ] && export BENCH_START_HEIGHT="${sh}" +fi + +expand_cpus() { + local part lo hi + local IFS=, + for part in $1; do + case "${part}" in + *-*) lo="${part%-*}"; hi="${part#*-}"; seq "${lo}" "${hi}" ;; + *) echo "${part}" ;; + esac + done +} + +CPU_PREFIX=() +if [ -n "${BENCH_CPUS}" ]; then + ncpu="$(getconf _NPROCESSORS_ONLN 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || nproc 2>/dev/null || echo 0)" + bench_cores="$(expand_cpus "${BENCH_CPUS}" | sort -nu)" + if [ "${ncpu}" -gt 0 ]; then + peer_cores="" + for i in $(seq 0 $((ncpu - 1))); do + grep -qxF "${i}" <<<"${bench_cores}" || peer_cores="${peer_cores},${i}" + done + BENCH_PEER_CPUS="${peer_cores#,}" + fi + + if command -v taskset >/dev/null 2>&1; then + CPU_PREFIX=(taskset -c "${BENCH_CPUS}") + echo "==> pinning the measured run to CPUs ${BENCH_CPUS}${BENCH_PEER_CPUS:+; docker peers to ${BENCH_PEER_CPUS}}" + else + echo "==> note: 'taskset' not found (e.g. macOS); running the measured binary UNPINNED${BENCH_PEER_CPUS:+ (docker peers still pinned to ${BENCH_PEER_CPUS})}" >&2 + fi +fi + +if [ "${MODE}" = local ]; then + COMPOSE_FILE="$(mktemp "${SCRIPT_DIR}/.scenario.XXXXXX")" + mv "${COMPOSE_FILE}" "${COMPOSE_FILE}.yml" + COMPOSE_FILE="${COMPOSE_FILE}.yml" + npeers="$(emit_compose "${COMPOSE_FILE}" "${BENCH_PEER_CPUS:-}")" + echo "==> scenario '$(basename "${SCN_FILE}" .yml)': ${npeers} peers [$(peers_summary)], cpus=${BENCH_CPUS:-}, blocks=${BLOCKS}" +fi + +compose() { docker compose -p "${PROJECT}" -f "${COMPOSE_FILE}" "$@"; } + +wait_loaded() { + local c logs + for _ in $(seq 1 400); do + local all=1 + for c in "$@"; do + logs="$(docker logs "${c}" 2>&1)" || { all=0; break; } + case "${logs}" in *"init message: Done loading"*) ;; *) all=0; break ;; esac + done + [ "${all}" -eq 1 ] && return 0 + echo -n "."; sleep 3 + done + return 1 +} + +teardown() { + [ -n "${COMPOSE_FILE}" ] && compose down --remove-orphans >/dev/null 2>&1 || true + [ -f "${STATE}" ] && { rm -rf "$(cat "${STATE}")" 2>/dev/null || true; rm -f "${STATE}"; } + rm -rf "${SCRIPT_DIR}"/.bench-clones.* 2>/dev/null || true +} + +arm_teardown() { + # On exit: tear down, THEN delete the generated compose (down needs it to still exist). + trap 'teardown; [ -n "${COMPOSE_FILE}" ] && rm -f "${COMPOSE_FILE}"' EXIT + trap 'exit 130' INT + trap 'exit 143' TERM + trap 'exit 129' HUP +} + +bring_up() { + echo "==> ensuring a clean network" + teardown + docker image inspect "${IMAGE}" >/dev/null 2>&1 || { echo "==> building peer image"; compose build; } + + local services; services="$(compose config --services)" + local n; n="$(echo ${services} | wc -w | tr -d ' ')" + # BENCH_PEERS is derived from the generated peers (host ports 19401..). + local peers_csv="" + for svc in ${services}; do peers_csv="${peers_csv},127.0.0.1:$((19400 + ${svc#dashd}))"; done + export BENCH_PEERS="${peers_csv#,}" + + echo "==> CoW-cloning ${CHAIN_DIR} for ${n} peers (instant)" + local clone_dir; clone_dir="$(mktemp -d "${SCRIPT_DIR}/.bench-clones.XXXXXX")" + echo "${clone_dir}" > "${STATE}" + for svc in ${services}; do + local dst="${clone_dir}/peer${svc#dashd}" + cp -c -R "${CHAIN_DIR}" "${dst}" 2>/dev/null \ + || cp --reflink=auto -R "${CHAIN_DIR}" "${dst}" 2>/dev/null \ + || cp -R "${CHAIN_DIR}" "${dst}" + done + + local batch=4 + echo "==> starting ${n} peers in batches of ${batch}" + local started="" count=0 + for svc in ${services}; do + CLONE_DIR="${clone_dir}" compose up -d "${svc}" >/dev/null 2>&1 + started="${started} spv-bench-${svc}"; count=$((count + 1)) + if [ $((count % batch)) -eq 0 ]; then + echo -n " loaded ${count}/${n} " + wait_loaded ${started} || { echo " timeout loading batch" >&2; exit 1; } + echo " ok" + fi + done + if [ $((count % batch)) -ne 0 ]; then # wait on the trailing partial batch too + echo -n " loaded ${count}/${n} " + wait_loaded ${started} || { echo " timeout loading batch" >&2; exit 1; } + echo " ok" + fi + + echo "==> ${n} peers started" +} + +build_bin() { + echo "==> building bench binary (release + line-table symbols)" + ( cd "${REPO_ROOT}" && CARGO_PROFILE_RELEASE_DEBUG=line-tables-only \ + cargo build --release -p dash-spv-bench ) + BIN="${REPO_ROOT}/target/release/dash-spv-bench" +} + +FLAME_TOOL="" +if [ "${FLAME}" -eq 1 ]; then # fail fast on missing profiler deps, before building + if command -v perf >/dev/null; then FLAME_TOOL=perf # Linux + elif command -v /usr/bin/sample >/dev/null; then FLAME_TOOL=sample # macOS + else echo "flame mode needs 'perf' (Linux) or '/usr/bin/sample' (macOS)" >&2; exit 1; fi + [ -f "${FLAMEGRAPH_DIR}/flamegraph.pl" ] || \ + git clone --depth 1 https://github.com/brendangregg/FlameGraph "${FLAMEGRAPH_DIR}" +fi + +build_bin + +if [ "${MODE}" = local ]; then + arm_teardown + bash "${SCRIPT_DIR}/snapshot-chain.sh" # builds ./chain-data to BENCH_HEIGHT via docker + bring_up +else + echo "==> testnet mode, peers: ${BENCH_PEERS:-}" +fi + +if [ "${FLAME}" -eq 0 ]; then + echo "==> running sync" + "${CPU_PREFIX[@]+"${CPU_PREFIX[@]}"}" "${BIN}" +elif [ "${FLAME_TOOL}" = perf ]; then + echo "==> running sync under perf" + mkdir -p "$(dirname "${FLAME_SVG}")" + perf record -F 499 -g -o /tmp/bench-perf.data -- "${CPU_PREFIX[@]+"${CPU_PREFIX[@]}"}" "${BIN}" + perf script -i /tmp/bench-perf.data \ + | "${FLAMEGRAPH_DIR}/stackcollapse-perf.pl" \ + | "${FLAMEGRAPH_DIR}/flamegraph.pl" --title "dash-spv sync" --colors hot > "${FLAME_SVG}" + echo "==> wrote ${FLAME_SVG}" +else + echo "==> running sync under the sampler" + "${CPU_PREFIX[@]+"${CPU_PREFIX[@]}"}" "${BIN}" & bpid=$! + sleep 3 + /usr/bin/sample "${bpid}" 2000 1 -file /tmp/bench.sample.txt -mayDie >/dev/null 2>&1 || true + wait "${bpid}" 2>/dev/null || true + mkdir -p "$(dirname "${FLAME_SVG}")" + "${FLAMEGRAPH_DIR}/stackcollapse-sample.awk" /tmp/bench.sample.txt \ + | sed -E 's/^Thread_[^;]*;//' \ + | "${FLAMEGRAPH_DIR}/flamegraph.pl" --title "dash-spv sync" --colors hot > "${FLAME_SVG}" + echo "==> wrote ${FLAME_SVG}" +fi diff --git a/dash-spv-bench/scenario.example.yml b/dash-spv-bench/scenario.example.yml new file mode 100644 index 000000000..34efaba12 --- /dev/null +++ b/dash-spv-bench/scenario.example.yml @@ -0,0 +1,38 @@ +description: "One-line summary of this scenario" # optional; shown in the report table. + +mode: local # "local" (docker peers you control), "testnet" or "mainnet" (real network). Default: local. + +cpus: "0-3" # optional. taskset CPU list pinning the MEASURED client (here cores 0,1,2,3). + # In local mode the docker peers get every OTHER core. Omit => no pinning. + +max_peers: 3 # optional. Max simultaneous peers. Omit => the ClientConfig default. + + +# --------------------------------------------------------------------------------------------- +# LOCAL mode (mode: local) — docker dashd peers serving the frozen ./chain-data snapshot +# --------------------------------------------------------------------------------------------- +blocks: 1000000 # optional (default 1000000). The target HEIGHT: snapshot-chain.sh builds + # ./chain-data up to here (via docker) and the client syncs the full + # range genesis..blocks. Lower it for a shorter run. + +# Peer groups: one entry per group; each becomes `count` dashd containers sharing a connection +# quality (via `tc netem`). List several groups to mix good and bad peers in one network. +peers: + - count: 3 # required: how many peers in this group + latency_ms: 50 # optional: added one-way delay (netem delay) + jitter_ms: 0 # optional: delay jitter (netem delay ) + loss_pct: 0 # optional: packet loss % (netem loss) + rate_kbit: 0 # optional: bandwidth cap in kbit (netem rate) + corrupt_pct: 0 # optional: bit corruption % (netem corrupt) + reorder_pct: 0 # optional: packet reordering % (netem reorder) + # A second, degraded group ("bad peers"): + # - { count: 1, latency_ms: 800, loss_pct: 8, rate_kbit: 500 } + +# --------------------------------------------------------------------------------------------- +# TESTNET / MAINNET mode — connect to the real Dash network instead of docker peers. +# Set `mode: testnet` or `mode: mainnet` above, then use these instead of blocks/peers: +# --------------------------------------------------------------------------------------------- +# peers: # explicit nodes as "host:port" strings; omit for DNS discovery. +# - "1.2.3.4:9999" # (testnet uses :19999, mainnet :9999) +# - "5.6.7.8:9999" +# start_height: 2000000 # optional checkpoint: skip genesis..start and sync a bounded range. diff --git a/dash-spv-bench/scenarios/congested-2core.yml b/dash-spv-bench/scenarios/congested-2core.yml new file mode 100644 index 000000000..bbb6455d0 --- /dev/null +++ b/dash-spv-bench/scenarios/congested-2core.yml @@ -0,0 +1,5 @@ +description: "CPU-starved client (2 cores), 5 peers on a jittery 300ms link." +cpus: "0-1" +repeats: 3 +peers: + - { count: 5, latency_ms: 300, jitter_ms: 100 } diff --git a/dash-spv-bench/scenarios/ideal-1-peer.yml b/dash-spv-bench/scenarios/ideal-1-peer.yml new file mode 100644 index 000000000..10b9c68ce --- /dev/null +++ b/dash-spv-bench/scenarios/ideal-1-peer.yml @@ -0,0 +1,5 @@ +description: "Ideal link, single peer: 4 CPUs, 1 peer @ 50ms, full 1M sync." +cpus: "0-3" +repeats: 3 +peers: + - { count: 1, latency_ms: 50 } diff --git a/dash-spv-bench/scenarios/ideal-3-peers.yml b/dash-spv-bench/scenarios/ideal-3-peers.yml new file mode 100644 index 000000000..2ba28c255 --- /dev/null +++ b/dash-spv-bench/scenarios/ideal-3-peers.yml @@ -0,0 +1,5 @@ +description: "Ideal link: 4 CPUs, 3 peers @ 50ms, full 1M sync." +cpus: "0-3" +repeats: 3 +peers: + - { count: 3, latency_ms: 50 } diff --git a/dash-spv-bench/scenarios/ideal-5-peers.yml b/dash-spv-bench/scenarios/ideal-5-peers.yml new file mode 100644 index 000000000..95e299642 --- /dev/null +++ b/dash-spv-bench/scenarios/ideal-5-peers.yml @@ -0,0 +1,5 @@ +description: "Ideal link, more peers: 4 CPUs, 5 peers @ 50ms, full 1M sync." +cpus: "0-3" +repeats: 3 +peers: + - { count: 5, latency_ms: 50 } diff --git a/dash-spv-bench/scenarios/no-lag-ideal-1-peer.yml b/dash-spv-bench/scenarios/no-lag-ideal-1-peer.yml new file mode 100644 index 000000000..237e2382b --- /dev/null +++ b/dash-spv-bench/scenarios/no-lag-ideal-1-peer.yml @@ -0,0 +1,5 @@ +description: "No-lag ideal, single peer: 12 CPUs, 1 peer @ 0ms, full 1M sync." +cpus: "0-11" +repeats: 3 +peers: + - { count: 1, latency_ms: 0 } diff --git a/dash-spv-bench/scenarios/no-lag-ideal-3-peers.yml b/dash-spv-bench/scenarios/no-lag-ideal-3-peers.yml new file mode 100644 index 000000000..8f4efdced --- /dev/null +++ b/dash-spv-bench/scenarios/no-lag-ideal-3-peers.yml @@ -0,0 +1,5 @@ +description: "No-lag ideal: 12 CPUs, 3 peers @ 0ms, full 1M sync." +cpus: "0-11" +repeats: 3 +peers: + - { count: 3, latency_ms: 0 } diff --git a/dash-spv-bench/scenarios/no-lag-ideal-5-peers.yml b/dash-spv-bench/scenarios/no-lag-ideal-5-peers.yml new file mode 100644 index 000000000..a0d7dda01 --- /dev/null +++ b/dash-spv-bench/scenarios/no-lag-ideal-5-peers.yml @@ -0,0 +1,5 @@ +description: "No-lag ideal, more peers: 12 CPUs, 5 peers @ 0ms, full 1M sync." +cpus: "0-11" +repeats: 3 +peers: + - { count: 5, latency_ms: 0 } diff --git a/dash-spv-bench/scenarios/one-bad-peer.yml b/dash-spv-bench/scenarios/one-bad-peer.yml new file mode 100644 index 000000000..3eb8187d6 --- /dev/null +++ b/dash-spv-bench/scenarios/one-bad-peer.yml @@ -0,0 +1,6 @@ +description: "Two good peers plus one slow/lossy/capped peer (a bad peer in the set)." +cpus: "0-3" +repeats: 3 +peers: + - { count: 2, latency_ms: 50 } + - { count: 1, latency_ms: 800, loss_pct: 8, rate_kbit: 500 } diff --git a/dash-spv-bench/scenarios/real-mainnet.yml b/dash-spv-bench/scenarios/real-mainnet.yml new file mode 100644 index 000000000..b1371ff5d --- /dev/null +++ b/dash-spv-bench/scenarios/real-mainnet.yml @@ -0,0 +1,3 @@ +description: "Mainnet sync from genesis via DNS peer discovery." +max_peers: 3 +mode: mainnet diff --git a/dash-spv-bench/scenarios/real-testnet.yml b/dash-spv-bench/scenarios/real-testnet.yml new file mode 100644 index 000000000..fa0348c5b --- /dev/null +++ b/dash-spv-bench/scenarios/real-testnet.yml @@ -0,0 +1,3 @@ +description: "Testnet sync from genesis via DNS peer discovery." +max_peers: 3 +mode: testnet diff --git a/dash-spv-bench/scenarios/slow-many-peers.yml b/dash-spv-bench/scenarios/slow-many-peers.yml new file mode 100644 index 000000000..c0808de3a --- /dev/null +++ b/dash-spv-bench/scenarios/slow-many-peers.yml @@ -0,0 +1,11 @@ +description: "Many slow peers: 8 peers @ 300ms/30ms jitter, 8 Mbit each, full 1M sync. Stresses per-peer cap growth — saturating a high-latency rate-capped link needs several in-flight requests per peer, so caps must climb well above the floor." +mode: local +cpus: "0-5" +max_peers: 8 +repeats: 2 +blocks: 1000000 +peers: + - count: 8 + latency_ms: 300 + jitter_ms: 30 + rate_kbit: 8000 diff --git a/dash-spv-bench/snapshot-chain.sh b/dash-spv-bench/snapshot-chain.sh new file mode 100755 index 000000000..c6a873554 --- /dev/null +++ b/dash-spv-bench/snapshot-chain.sh @@ -0,0 +1,29 @@ +#!/bin/bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +CHAIN_DIR="${SCRIPT_DIR}/chain-data" +SUBDIR="testnet3" +DASH_VERSION="23.1.7" # keep in sync with run.sh +IMAGE="dash-spv-bench/dashd:${DASH_VERSION}" + +cd "${SCRIPT_DIR}" + +: "${BENCH_HEIGHT:?BENCH_HEIGHT must be set — run.sh exports it from the scenario blocks}" +command -v docker >/dev/null 2>&1 || { echo "Error: docker is required to build the snapshot." >&2; exit 1; } + +docker image inspect "${IMAGE}" >/dev/null 2>&1 \ + || { echo "==> building peer image" + docker build --build-arg "DASHVERSION=${DASH_VERSION}" \ + -t "${IMAGE}" -f "${SCRIPT_DIR}/Dockerfile" "${SCRIPT_DIR}"; } + +mkdir -p "${CHAIN_DIR}" + +echo "==> extending snapshot to height ${BENCH_HEIGHT} via docker (downloads only the missing blocks)..." + +docker run --rm --user "$(id -u):$(id -g)" -v "${CHAIN_DIR}:/data" "${IMAGE}" \ + dashd -testnet -datadir=/data -daemon=0 -server=1 \ + -blockfilterindex=1 -peerblockfilters=1 -stopatheight="${BENCH_HEIGHT}" \ + -txindex=0 -prune=0 -disablewallet=1 + +echo "==> done. Snapshot is at ${CHAIN_DIR}/${SUBDIR} (height ${BENCH_HEIGHT})." diff --git a/dash-spv-bench/src/dashboard.rs b/dash-spv-bench/src/dashboard.rs new file mode 100644 index 000000000..d8f0aa7ce --- /dev/null +++ b/dash-spv-bench/src/dashboard.rs @@ -0,0 +1,217 @@ +use std::io::{self, IsTerminal, Write}; + +use dash_spv::sync::{ProgressPercentage, SyncProgress}; +use indicatif::{MultiProgress, ProgressBar, ProgressDrawTarget, ProgressStyle}; +use tracing_subscriber::fmt::MakeWriter; + +const REDRAW_HZ: u8 = 10; +const BAR_WIDTH: usize = 32; +const NOT_ENABLED: &str = "not enabled"; + +pub struct Dashboard { + multi: MultiProgress, + block_headers: ProgressBar, + filter_headers: ProgressBar, + filters: ProgressBar, + masternodes: ProgressBar, + blocks: ProgressBar, + chainlocks: ProgressBar, + instantsend: ProgressBar, + mempool: ProgressBar, +} + +impl Dashboard { + pub fn new() -> Self { + let target = if io::stderr().is_terminal() { + ProgressDrawTarget::stderr_with_hz(REDRAW_HZ) + } else { + ProgressDrawTarget::hidden() + }; + let multi = MultiProgress::with_draw_target(target); + + let bar_style = ProgressStyle::with_template(&format!( + " {{prefix:<15}} [{{bar:{BAR_WIDTH}}}] {{percent:>3}}% {{msg}}" + )) + .expect("static template") + .progress_chars("=> "); + + let info_style = ProgressStyle::with_template(&format!( + " {{prefix:<15}} {:BAR_WIDTH$} {{msg}}", + "" + )) + .expect("static template"); + + let mk = |label: &'static str, style: &ProgressStyle| { + let pb = multi.add(ProgressBar::new(1)); + pb.set_style(style.clone()); + pb.set_prefix(label); + pb.set_message(NOT_ENABLED); + pb + }; + + Self { + block_headers: mk("block headers", &bar_style), + filter_headers: mk("filter headers", &bar_style), + filters: mk("filters", &bar_style), + masternodes: mk("masternodes", &bar_style), + blocks: mk("matched blocks", &bar_style), + chainlocks: mk("chainlocks", &info_style), + instantsend: mk("instantsend", &info_style), + mempool: mk("mempool", &info_style), + multi, + } + } + + pub fn log_writer(&self) -> BarWriter { + BarWriter(self.multi.clone()) + } + + pub fn render(&self, progress: &SyncProgress) { + fn set(pb: &ProgressBar, p: &impl ProgressPercentage) { + let (current, target) = (p.current_height(), p.target_height()); + pb.set_length(target.max(1) as u64); + pb.set_position(current.min(target) as u64); + pb.set_message(format!("{} / {}", current, target)); + } + + if let Ok(p) = progress.headers() { + set(&self.block_headers, p); + } + if let Ok(p) = progress.filter_headers() { + let (current, target) = (p.current_height(), p.target_height()); + self.filter_headers.set_length(target.max(1) as u64); + self.filter_headers.set_position(current.min(target) as u64); + self.filter_headers.set_message(format!( + "{} / {} (verified {})", + current, + target, + p.current_height() + )); + } + if let Ok(p) = progress.filters() { + set(&self.filters, p); + } + if let Ok(p) = progress.masternodes() { + let (current, target) = (p.current_height(), p.target_height()); + self.masternodes.set_length(target.max(1) as u64); + self.masternodes.set_position(current.min(target) as u64); + self.masternodes.set_message(format!( + "{} / {} ({} diffs, {} cycles)", + current, + target, + p.diffs_processed(), + p.validated_cycles() + )); + } + if let Ok(p) = progress.blocks() { + let tip = progress.headers().map(|h| h.target_height()).unwrap_or(0); + let done = p.last_processed(); + self.blocks.set_length(tip.max(1) as u64); + self.blocks.set_position(done.min(tip) as u64); + self.blocks.set_message(format!( + "{} / {} ({} blocks, {} txs)", + done, + tip, + p.processed(), + p.transactions() + )); + } + if let Ok(p) = progress.chainlocks() { + self.chainlocks.set_message(format!( + "{:?} best {} ({} valid, {} invalid)", + p.state(), + p.best_validated_height(), + p.valid(), + p.invalid() + )); + } + if let Ok(p) = progress.instantsend() { + self.instantsend.set_message(format!( + "{:?} {} valid, {} invalid, {} pending", + p.state(), + p.valid(), + p.invalid(), + p.pending() + )); + } + if let Ok(p) = progress.mempool() { + self.mempool.set_message(format!( + "{:?} {} tracked ({} relevant of {} seen)", + p.state(), + p.tracked(), + p.relevant(), + p.received() + )); + } + } + + pub fn finish(&self) { + for pb in [ + &self.block_headers, + &self.filter_headers, + &self.filters, + &self.masternodes, + &self.blocks, + &self.chainlocks, + &self.instantsend, + &self.mempool, + ] { + if pb.message() == NOT_ENABLED { + pb.abandon(); + } else { + pb.finish(); + } + } + } +} + +impl Default for Dashboard { + fn default() -> Self { + Self::new() + } +} + +#[derive(Clone)] +pub struct BarWriter(MultiProgress); + +impl<'a> MakeWriter<'a> for BarWriter { + type Writer = BarWriterGuard; + + fn make_writer(&'a self) -> Self::Writer { + BarWriterGuard { + multi: self.0.clone(), + buf: Vec::new(), + } + } +} + +pub struct BarWriterGuard { + multi: MultiProgress, + buf: Vec, +} + +impl Write for BarWriterGuard { + fn write(&mut self, buf: &[u8]) -> io::Result { + self.buf.extend_from_slice(buf); + Ok(buf.len()) + } + + fn flush(&mut self) -> io::Result<()> { + if self.buf.is_empty() { + return Ok(()); + } + let line = std::mem::take(&mut self.buf); + self.multi.suspend(|| { + let mut err = io::stderr().lock(); + let _ = err.write_all(&line); + let _ = err.flush(); + }); + Ok(()) + } +} + +impl Drop for BarWriterGuard { + fn drop(&mut self) { + let _ = self.flush(); + } +} diff --git a/dash-spv-bench/src/main.rs b/dash-spv-bench/src/main.rs new file mode 100644 index 000000000..f3be43593 --- /dev/null +++ b/dash-spv-bench/src/main.rs @@ -0,0 +1,239 @@ +mod dashboard; +mod metrics; + +use std::net::SocketAddr; +use std::path::PathBuf; +use std::sync::Arc; +use std::time::Duration; + +use anyhow::{anyhow, Context, Result}; +use dash_spv::storage::DiskStorageManager; +use dash_spv::{ClientConfig, DashSpvClient, EventHandler}; +use dashcore::Network; +use key_wallet::wallet::initialization::WalletAccountCreationOptions; +use key_wallet::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; +use key_wallet::wallet::ManagedWalletInfo; +use key_wallet_manager::WalletManager; +use std::collections::BTreeSet; +use tokio::sync::RwLock; +use tracing_subscriber::prelude::*; + +use crate::dashboard::Dashboard; +use crate::metrics::BenchEventHandler; + +fn env_or(key: &str, default: &str) -> String { + std::env::var(key).unwrap_or_else(|_| default.to_string()) +} + +fn load_mnemonics() -> Vec { + std::env::var("BENCH_WALLET_FILE") + .ok() + .filter(|p| !p.trim().is_empty()) + .and_then(|path| std::fs::read_to_string(&path).ok()) + .map(|contents| { + contents + .lines() + .map(str::trim) + .filter(|line| !line.is_empty() && !line.starts_with('#')) + .map(String::from) + .collect() + }) + .unwrap_or_default() +} + +#[tokio::main] +async fn main() -> Result<()> { + let dashboard = Arc::new(Dashboard::new()); + + let (output_dir, _tempdir_guard): (PathBuf, Option) = + match std::env::var("BENCH_STORAGE_DIR") { + Ok(dir) if !dir.trim().is_empty() => { + let path = PathBuf::from(dir.trim()); + let _ = std::fs::remove_dir_all(&path); + std::fs::create_dir_all(&path).context("create bench storage dir")?; + (path, None) + } + _ => { + let td = tempfile::tempdir().context("temp storage dir")?; + (td.path().to_path_buf(), Some(td)) + } + }; + + let log_file = std::fs::File::create(output_dir.join("run.log")).context("create run.log")?; + let terminal_filter = tracing_subscriber::EnvFilter::try_from_default_env() + .unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("warn,dash_spv_bench=info")); + let file_filter = tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| { + tracing_subscriber::EnvFilter::new("warn,dash_spv=debug,dash_spv_bench=debug") + }); + tracing_subscriber::registry() + .with( + tracing_subscriber::fmt::layer() + .with_writer(dashboard.log_writer()) + .with_filter(terminal_filter), + ) + .with( + tracing_subscriber::fmt::layer() + .with_ansi(false) + .with_writer(std::sync::Mutex::new(log_file)) + .with_filter(file_filter), + ) + .init(); + + let mode = env_or("BENCH_MODE", "local").trim().to_ascii_lowercase(); + let (network, remote) = match mode.as_str() { + "local" => (Network::Testnet, false), + "testnet" => (Network::Testnet, true), + "mainnet" => (Network::Mainnet, true), + other => { + return Err(anyhow!( + "BENCH_MODE must be 'local', 'testnet' or 'mainnet', got '{other}'" + )) + } + }; + + let peers: Vec = std::env::var("BENCH_PEERS") + .unwrap_or_default() + .split(',') + .map(str::trim) + .filter(|s| !s.is_empty()) + .map(|p| p.parse::().with_context(|| format!("bad peer {p}"))) + .collect::>()?; + + if !remote && peers.is_empty() { + return Err(anyhow!("BENCH_MODE=local requires BENCH_PEERS (the docker peer addresses)")); + } + + let dns_mode = remote && peers.is_empty(); + + let start_height: Option = + std::env::var("BENCH_START_HEIGHT").ok().and_then(|v| v.trim().parse().ok()); + + let mnemonics = load_mnemonics(); + let timeout = Duration::from_secs(1800); + + tracing::info!( + "dash-spv-bench: mode={} ({}) on {:?}, wallets={}", + mode, + if dns_mode { + "DNS discovery".to_string() + } else { + format!("{} peers", peers.len()) + }, + network, + mnemonics.len(), + ); + + let mut config = ClientConfig::new(network).with_storage_path(output_dir.clone()); + config.start_from_height = start_height; + config.restrict_to_configured_peers = !dns_mode; + if let Ok(v) = std::env::var("BENCH_MAX_PEERS") { + if let Ok(n) = v.trim().parse() { + config.max_peers = n; + } + } + for addr in &peers { + config.add_peer(*addr); + } + + let storage_manager = + DiskStorageManager::new(&config).await.map_err(|e| anyhow!("storage manager: {e}"))?; + + // Wallets: one BIP44 account 0 per mnemonic, so filter matches drive block download. + let mut wallet_manager = WalletManager::::new(network); + let birth_height = start_height.unwrap_or(0); + let mut wallet_ids = Vec::with_capacity(mnemonics.len()); + for mnemonic in &mnemonics { + let id = wallet_manager + .create_wallet_from_mnemonic(mnemonic, birth_height, account_creation_options()) + .map_err(|e| anyhow!("wallet from mnemonic: {e}"))?; + wallet_ids.push(id); + } + let wallet = Arc::new(RwLock::new(wallet_manager)); + let wallet_probe = wallet.clone(); + + let handler = Arc::new(BenchEventHandler::new(dashboard.clone())); + let network = dash_spv::network::PeerNetworkManager::new(&config) + .await + .map_err(|e| anyhow!("network new: {e}"))?; + + let client = DashSpvClient::new( + config, + network, + storage_manager, + wallet, + vec![handler.clone() as Arc], + ) + .await + .map_err(|e| anyhow!("client new: {e}"))?; + + let run_client = client.clone(); + let run_handle = tokio::spawn(async move { + if let Err(e) = run_client.run().await { + tracing::error!("client run() exited with error: {e}"); + } + }); + + tokio::select! { + _ = handler.wait_done() => {} + _ = tokio::time::sleep(timeout) => {} + _ = tokio::signal::ctrl_c() => eprintln!("ctrl-c received, shutting down..."), + } + let m = handler.snapshot(); + + let _ = client.shutdown().await; + run_handle.abort(); + let _ = run_handle.await; + + use std::fmt::Write as _; + let mut report = format!("{m}\n"); + + if !wallet_ids.is_empty() { + use key_wallet_manager::WalletInterface; + let w = wallet_probe.read().await; + let _ = writeln!(report, "wallet_synced_height: {}", w.synced_height()); + let _ = writeln!(report, "wallet_addresses: {}", w.monitored_addresses().len()); + + for (i, id) in wallet_ids.iter().enumerate() { + let txs = w + .get_wallet_info(id) + .map(|info| { + info.accounts() + .all_accounts() + .iter() + .flat_map(|a| a.transactions().keys()) + .collect::>() + .len() + }) + .unwrap_or(0); + + let (confirmed, total) = + w.get_wallet_balance(id).map(|b| (b.confirmed(), b.total())).unwrap_or((0, 0)); + + let _ = writeln!( + report, + "wallet[{i}] txs={txs} confirmed_sat={confirmed} total_sat={total}" + ); + } + + let _ = writeln!(report, "wallet_describe:\n{}", w.describe().await); + } + + print!("\n{report}"); + if let Err(e) = std::fs::write(output_dir.join("summary.txt"), &report) { + tracing::warn!("could not write summary.txt: {e}"); + } + tracing::info!("run outputs written to {}", output_dir.display()); + + Ok(()) +} + +fn account_creation_options() -> WalletAccountCreationOptions { + WalletAccountCreationOptions::SpecificAccounts( + BTreeSet::from([0]), + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + BTreeSet::new(), + None, + ) +} diff --git a/dash-spv-bench/src/metrics.rs b/dash-spv-bench/src/metrics.rs new file mode 100644 index 000000000..930eb1332 --- /dev/null +++ b/dash-spv-bench/src/metrics.rs @@ -0,0 +1,154 @@ +use std::collections::BTreeMap; +use std::fmt; +use std::sync::{Arc, Mutex}; +use std::time::Instant; + +use dash_spv::sync::{SyncEvent, SyncProgress}; +use dash_spv::EventHandler; +use tokio::sync::Notify; + +use crate::dashboard::Dashboard; + +pub const MILESTONE_BLOCK_HEADERS: &str = "block_headers_complete"; +pub const MILESTONE_FILTER_HEADERS: &str = "filter_headers_complete"; +pub const MILESTONE_FILTERS: &str = "filters_complete"; +pub const MILESTONE_SYNC: &str = "sync_complete"; + +#[derive(Debug)] +pub struct RunMetrics { + total_ms: u64, + completed: bool, + block_headers_ms: Option, + filter_headers_ms: Option, + filters_ms: Option, + error: Option, +} + +impl fmt::Display for RunMetrics { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let ms = |o: Option| o.map(|v| v.to_string()).unwrap_or_else(|| "-".to_string()); + + writeln!(f, "=== dash-spv sync ===")?; + writeln!( + f, + "completed: {}{}", + self.completed, + if self.completed { + "" + } else { + " (TIMED OUT)" + } + )?; + writeln!(f, "total_ms: {}", self.total_ms)?; + writeln!(f, "block_headers_ms: {}", ms(self.block_headers_ms))?; + writeln!(f, "filter_headers_ms: {}", ms(self.filter_headers_ms))?; + write!(f, "filters_ms: {}", ms(self.filters_ms))?; + if let Some(e) = &self.error { + write!(f, "\nerror: {e}")?; + } + + Ok(()) + } +} + +pub struct BenchEventHandler { + start: Instant, + inner: Mutex, + done: Notify, + dashboard: Arc, +} + +struct Inner { + milestones: BTreeMap<&'static str, u64>, + error: Option, + completed: bool, +} + +impl BenchEventHandler { + pub fn new(dashboard: Arc) -> Self { + Self { + start: Instant::now(), + inner: Mutex::new(Inner { + milestones: BTreeMap::new(), + error: None, + completed: false, + }), + done: Notify::new(), + dashboard, + } + } + + fn record(&self, label: &'static str) { + let elapsed = self.start.elapsed().as_millis() as u64; + let mut inner = self.inner.lock().unwrap(); + inner.milestones.entry(label).or_insert(elapsed); + } + + pub async fn wait_done(&self) { + let notified = self.done.notified(); + tokio::pin!(notified); + notified.as_mut().enable(); + + if self.inner.lock().unwrap().completed { + return; + } + + notified.await; + } + + pub fn snapshot(&self) -> RunMetrics { + let inner = self.inner.lock().unwrap(); + let bh = inner.milestones.get(MILESTONE_BLOCK_HEADERS).copied(); + let fh = inner.milestones.get(MILESTONE_FILTER_HEADERS).copied(); + let fl = inner.milestones.get(MILESTONE_FILTERS).copied(); + let sc = inner.milestones.get(MILESTONE_SYNC).copied(); + + RunMetrics { + total_ms: sc.unwrap_or_else(|| self.start.elapsed().as_millis() as u64), + completed: inner.completed, + block_headers_ms: bh, + filter_headers_ms: fh, + filters_ms: fl, + error: inner.error.clone(), + } + } +} + +impl EventHandler for BenchEventHandler { + fn on_sync_event(&self, event: &SyncEvent) { + match event { + SyncEvent::BlockHeaderSyncComplete { + .. + } => self.record(MILESTONE_BLOCK_HEADERS), + SyncEvent::FilterHeadersSyncComplete { + .. + } => self.record(MILESTONE_FILTER_HEADERS), + SyncEvent::FiltersSyncComplete { + .. + } => self.record(MILESTONE_FILTERS), + SyncEvent::SyncComplete { + .. + } => { + self.record(MILESTONE_SYNC); + self.dashboard.finish(); + let mut inner = self.inner.lock().unwrap(); + inner.completed = true; + drop(inner); + self.done.notify_waiters(); + } + _ => {} + } + } + + /// Every progress change repaints the bars (throttled inside). + fn on_progress(&self, progress: &SyncProgress) { + self.dashboard.render(progress); + } + + fn on_error(&self, error: &str) { + let mut inner = self.inner.lock().unwrap(); + if inner.error.is_none() { + inner.error = Some(error.to_string()); + } + } +} From 05702cdc908b4bfe8221623d3066fd64a55db028 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Mon, 27 Jul 2026 05:30:39 -0700 Subject: [PATCH 057/149] fix(dash-spv): reduced debug log verbosity (#917) --- dash-spv/src/network/manager.rs | 10 +++++----- dash-spv/src/network/peer.rs | 4 ++-- dash-spv/src/storage/segments.rs | 2 +- dash-spv/src/sync/filters/manager.rs | 2 +- dash-spv/src/sync/filters/pipeline.rs | 4 ++-- dash-spv/src/sync/masternodes/pipeline.rs | 2 +- dash-spv/src/sync/masternodes/sync_manager.rs | 2 +- dash-spv/src/validation/filter.rs | 2 +- 8 files changed, 14 insertions(+), 14 deletions(-) diff --git a/dash-spv/src/network/manager.rs b/dash-spv/src/network/manager.rs index 680eedf4d..dbfccb443 100644 --- a/dash-spv/src/network/manager.rs +++ b/dash-spv/src/network/manager.rs @@ -483,7 +483,7 @@ impl PeerNetworkManager { match msg_result { Ok(Some(msg)) => { // Log all received messages at debug level to help troubleshoot - tracing::debug!("Received {:?} from {}", msg.cmd(), addr); + tracing::trace!("Received {:?} from {}", msg.cmd(), addr); // Handle some messages directly match &msg.inner() { @@ -797,7 +797,7 @@ impl PeerNetworkManager { request = request_rx.recv() => { match request { Some(NetworkRequest::SendMessage(msg)) => { - tracing::debug!("Request processor: sending {}", msg.cmd()); + tracing::trace!("Request processor: sending {}", msg.cmd()); // Spawn each send concurrently to allow parallel requests across peers. let this = this.clone(); tokio::spawn(async move { @@ -822,7 +822,7 @@ impl PeerNetworkManager { }); } Some(NetworkRequest::SendMessageToPeer(msg, peer_address)) => { - tracing::debug!("Request processor: sending {} to peer {}", msg.cmd(), peer_address); + tracing::trace!("Request processor: sending {} to peer {}", msg.cmd(), peer_address); let this = this.clone(); tokio::spawn(async move { let fallback_msg = msg.clone(); @@ -1064,7 +1064,7 @@ impl PeerNetworkManager { while !this.shutdown_token.is_cancelled() { tokio::select! { _ = maintenance_interval.tick() => { - tracing::debug!("Maintenance interval elapsed"); + tracing::trace!("Maintenance interval elapsed"); this.maintenance_tick().await; } _ = dns_interval.tick(), if !this.exclusive_mode => { @@ -1186,7 +1186,7 @@ impl PeerNetworkManager { let (addr, peer) = self.next_peer(&selected_peers); - tracing::debug!("Distributing {} request to peer {}", message.cmd(), addr); + tracing::trace!("Distributing {} request to peer {}", message.cmd(), addr); self.send_message_to_peer(&addr, &peer, message).await } diff --git a/dash-spv/src/network/peer.rs b/dash-spv/src/network/peer.rs index 2966ac52d..48e2f631c 100644 --- a/dash-spv/src/network/peer.rs +++ b/dash-spv/src/network/peer.rs @@ -351,7 +351,7 @@ impl Peer { tracing::warn!("Failed to flush socket {}: {}", self.address, e); } self.bytes_sent += serialized.len() as u64; - tracing::debug!("Sent message to {}: {:?}", self.address, raw_message.payload); + tracing::trace!("Sent message to {}: {:?}", self.address, raw_message.payload); Ok(()) } Err(e) => { @@ -681,7 +681,7 @@ impl Peer { let pong_message = NetworkMessage::Pong(nonce); self.send_message(pong_message).await?; - tracing::debug!("Responded to ping from {} with pong nonce {}", self.address, nonce); + tracing::trace!("Responded to ping from {} with pong nonce {}", self.address, nonce); Ok(()) } diff --git a/dash-spv/src/storage/segments.rs b/dash-spv/src/storage/segments.rs index 9521fcc81..f76566aa0 100644 --- a/dash-spv/src/storage/segments.rs +++ b/dash-spv/src/storage/segments.rs @@ -359,7 +359,7 @@ impl SegmentCache { let mut height = start_height; - tracing::debug!( + tracing::trace!( "SegmentsCache: storing {} items starting at height {}", items.len(), height, diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index b117b58d2..376a80c8c 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -843,7 +843,7 @@ impl 0 && received % 100 == 0 { - tracing::debug!( + tracing::trace!( "Filter batch {} progress: {}/{} filters received", batch_start, received, diff --git a/dash-spv/src/sync/masternodes/pipeline.rs b/dash-spv/src/sync/masternodes/pipeline.rs index 8c7a0958b..9a67149ec 100644 --- a/dash-spv/src/sync/masternodes/pipeline.rs +++ b/dash-spv/src/sync/masternodes/pipeline.rs @@ -89,7 +89,7 @@ impl MnListDiffPipeline { requests.request_mnlist_diff(base_hash, target_hash)?; self.coordinator.mark_sent(&[target_hash]); - tracing::debug!( + tracing::trace!( "Sent GetMnListDiff: base={}, target={} ({} active, {} pending)", base_hash, target_hash, diff --git a/dash-spv/src/sync/masternodes/sync_manager.rs b/dash-spv/src/sync/masternodes/sync_manager.rs index e949c0d2d..69c10209a 100644 --- a/dash-spv/src/sync/masternodes/sync_manager.rs +++ b/dash-spv/src/sync/masternodes/sync_manager.rs @@ -162,7 +162,7 @@ pub(super) async fn feed_qrinfo_heights_to_engine( if let Ok(Some(height)) = storage.get_header_height_by_hash(&block_hash).await { engine.feed_block_height(height, block_hash); fed_count += 1; - tracing::debug!("Fed height {} for block {}", height, block_hash); + tracing::trace!("Fed height {} for block {}", height, block_hash); } } diff --git a/dash-spv/src/validation/filter.rs b/dash-spv/src/validation/filter.rs index d9a0f8f3f..86f82706d 100644 --- a/dash-spv/src/validation/filter.rs +++ b/dash-spv/src/validation/filter.rs @@ -122,7 +122,7 @@ impl Validator> for FilterValidator { ))); } - tracing::debug!("Verified {} filters successfully", input.filters.len()); + tracing::trace!("Verified {} filters successfully", input.filters.len()); Ok(()) } } From 6e5d2f0ec5553497e076cd70409f83eac18e2158 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Mon, 27 Jul 2026 22:31:18 +1000 Subject: [PATCH 058/149] feat(key-wallet): reserve receive addresses on hand-out (#818) * feat(key-wallet): reserve receive addresses on hand-out Adds a reservation lifecycle to addresses so a receive address handed out to a caller is not re-issued before it is funded or explicitly released. This closes the hand-out race where two sequential requests returned the same address. - Model address lifecycle as an `AddressState` enum (`Available`, `Reserved { at }`, `Used { at }`) on `AddressInfo`, replacing the separate `used`/`used_at` fields. The states are mutually exclusive by construction, so the invariant "a used address is never reserved" holds structurally instead of being maintained by hand. - Add `next_unused_and_reserve`, `release_reservation`, and `sweep_expired_reservations` (TTL backstop) on `AddressPool`; reserved addresses are excluded from hand-out, count against the gap limit, and are never pruned or aged out when clockless. - Wire `next_receive_address_and_reserve`, `release_receive_reservation`, and `sweep_expired_receive_reservations` through `ManagedCoreFundsAccount`, bumping the monitor revision on change. - Add `reserved_count` to `PoolStats`. - Cover reserve/release/sweep, serde and bincode round-trips, gap-limit and prune interaction, and end-to-end promotion on funding. * fix(key-wallet): return `InvalidState` instead of panicking on pool invariant miss `next_unused_and_reserve` and `maintain_gap_limit` guarded the "entry must exist after `generate_address_at_index(add_to_state=true)`" invariant with `expect()` and `panic!()`. This crate's error-handling philosophy is to never panic in library code, so both sites now propagate a new `Error::InvalidState` variant through their existing `Result` return type. The FFI maps it to `FFIErrorCode::InvalidState`. Addresses CodeRabbit review comment on PR #818 https://github.com/dashpay/rust-dashcore/pull/818#discussion_r3456738207 * feat(dash-spv): periodically sweep expired receive-address reservations Reservations handed out by `next_receive_address_and_reserve` pin gap-limit headroom until funded or released. The TTL sweep is the backstop for ones never paid and never released, but `key-wallet` is clockless by design, so nothing drove it automatically. Adds a clock-owning driver at the SPV layer that runs the sweep on a fixed interval, on by default with opt-out via `ClientConfig::without_reservation_sweep` and a configurable `reservation_ttl_secs` (default 1 hour). `key-wallet` stays clockless: the SPV task supplies `now`. Wiring: `WalletInfoInterface::sweep_expired_reservations` fans out over every funding account, `WalletInterface::sweep_expired_reservations` sums across managed wallets (default `0` for implementations without reservations), and `spawn_reservation_sweep` ticks the call under the existing monitor-shutdown token alongside the chainlock dispatcher. * fix(dash-spv): reject `reservation_sweep_ttl_secs` of `Some(0)` in `validate` A TTL of `0` makes `sweep_expired_reservations` reclaim nothing, so `with_reservation_ttl_secs(0)` would spawn a live, lock-taking sweep task that does nothing every tick while the API still describes sweeping as enabled. Reject it in `validate` alongside the other zero-value checks, pointing callers at `without_reservation_sweep` to disable the sweep instead. Addresses CodeRabbit review comment on PR #818 https://github.com/dashpay/rust-dashcore/pull/818#discussion_r3489945978 * fix(dash-spv): spawn reservation sweep only after `start` succeeds The sweep mutates wallet state and a tokio interval fires its first tick immediately, so spawning it before `self.start().await` let it reclaim reservations and bump monitor revisions even when startup ultimately failed and `run` returned an error. Move the spawn below the successful-startup path, which also lets the now-unreachable error-path cleanup go away. Addresses CodeRabbit review comment on PR #818 https://github.com/dashpay/rust-dashcore/pull/818#discussion_r3489945986 * refactor(key-wallet): drop unused address first-used and generated timestamps `AddressState::Used` carried a first-used `at` timestamp and `AddressInfo` carried a `generated_at`, both hardcoded to `0` and never read by any code: they were only copied into the FFI `FFIAddressInfo` struct, where the values were always `0`. Nothing computed anything from them, and the first-use time is already derivable from the transaction records that touch the address. Remove both. `AddressState::Used` becomes a unit variant, `AddressInfo::generated_at` and the `used_at()` accessor are gone, and the two dead fields drop out of `FFIAddressInfo`. The reservation timestamp (`AddressState::Reserved { at }`) stays, since `sweep_expired_reservations` relies on it to reclaim stale reservations. This changes the `#[repr(C)] FFIAddressInfo` layout, so C/Swift consumers must drop the `generated_at` and `used_at` fields. --------- Co-authored-by: lklimek <842586+lklimek@users.noreply.github.com> --- dash-spv/src/client/config.rs | 26 + dash-spv/src/client/config_test.rs | 12 + dash-spv/src/client/event_handler.rs | 43 ++ dash-spv/src/client/sync_coordinator.rs | 12 + key-wallet-ffi/src/address_pool.rs | 15 +- key-wallet-ffi/src/error.rs | 3 +- key-wallet-manager/src/events.rs | 6 +- key-wallet-manager/src/process_block.rs | 27 + key-wallet-manager/src/wallet_interface.rs | 11 + key-wallet/src/error.rs | 3 + .../src/managed_account/address_pool.rs | 536 ++++++++++++++++-- .../managed_core_funds_account.rs | 87 +++ key-wallet/src/tests/address_pool_tests.rs | 30 +- .../src/tests/address_reservation_tests.rs | 168 ++++++ key-wallet/src/tests/mod.rs | 2 + .../wallet_info_interface.rs | 14 + 16 files changed, 923 insertions(+), 72 deletions(-) create mode 100644 key-wallet/src/tests/address_reservation_tests.rs diff --git a/dash-spv/src/client/config.rs b/dash-spv/src/client/config.rs index 84d54f5b3..00cd577de 100644 --- a/dash-spv/src/client/config.rs +++ b/dash-spv/src/client/config.rs @@ -89,6 +89,14 @@ pub struct ClientConfig { /// The client will use the nearest checkpoint at or before this height. pub start_from_height: Option, + /// Time-to-live (seconds) for unfunded receive-address reservations, + /// periodically reclaimed by the sweep so a hand-out that is never paid and + /// never released does not pin gap-limit headroom forever. `Some(secs)` + /// enables the sweep with that TTL; `None` disables it. Defaults to one hour. + /// Set with [`Self::with_reservation_ttl_secs`] or clear with + /// [`Self::without_reservation_sweep`]. + pub reservation_sweep_ttl_secs: Option, + /// Devnet-only configuration. Must be `Some` iff `network == Network::Devnet`. pub devnet: Option, } @@ -114,6 +122,7 @@ impl Default for ClientConfig { broadcast_acceptance_threshold: 1, broadcast_acceptance_timeout: Duration::from_secs(60), start_from_height: None, + reservation_sweep_ttl_secs: Some(3600), devnet: None, } } @@ -233,6 +242,19 @@ impl ClientConfig { self } + /// Disable the periodic receive-address reservation sweep. + pub fn without_reservation_sweep(mut self) -> Self { + self.reservation_sweep_ttl_secs = None; + self + } + + /// Enable the periodic receive-address reservation sweep with the given TTL + /// (seconds): a reservation older than `secs` is reclaimed. + pub fn with_reservation_ttl_secs(mut self, secs: u64) -> Self { + self.reservation_sweep_ttl_secs = Some(secs); + self + } + /// Attach a [`DevnetConfig`]. The network must be `Network::Devnet`. /// [`validate`](Self::validate) enforces the biconditional. pub fn with_devnet(mut self, devnet: DevnetConfig) -> Self { @@ -255,6 +277,10 @@ impl ClientConfig { ); } + if self.reservation_sweep_ttl_secs == Some(0) { + return Err("reservation_sweep_ttl_secs must be > 0 when set; use without_reservation_sweep() to disable the sweep".to_string()); + } + if self.broadcast_acceptance_threshold == 0 { return Err("broadcast_acceptance_threshold must be > 0".to_string()); } diff --git a/dash-spv/src/client/config_test.rs b/dash-spv/src/client/config_test.rs index c47dbba61..e954bb8e1 100644 --- a/dash-spv/src/client/config_test.rs +++ b/dash-spv/src/client/config_test.rs @@ -197,6 +197,18 @@ mod tests { assert!(result.unwrap_err().contains("max_mempool_transactions must be > 0")); } + #[test] + fn test_validation_rejects_zero_reservation_ttl() { + let config = ClientConfig { + reservation_sweep_ttl_secs: Some(0), + ..Default::default() + }; + + let result = config.validate(); + assert!(result.is_err()); + assert!(result.unwrap_err().contains("reservation_sweep_ttl_secs must be > 0")); + } + #[test] fn test_apply_global_overrides_no_devnet_is_noop() { let tmp = TempDir::new().unwrap(); diff --git a/dash-spv/src/client/event_handler.rs b/dash-spv/src/client/event_handler.rs index a6ef1aace..1e4b57860 100644 --- a/dash-spv/src/client/event_handler.rs +++ b/dash-spv/src/client/event_handler.rs @@ -6,6 +6,7 @@ use std::fmt::Display; use std::sync::Arc; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; use tokio::sync::{broadcast, mpsc, watch, RwLock}; use tokio::task::JoinHandle; @@ -217,6 +218,48 @@ where }) } +/// How often the reservation-sweep task wakes to reclaim expired reservations. +const RESERVATION_SWEEP_INTERVAL: Duration = Duration::from_secs(60); + +/// Periodically reclaim receive-address reservations that were handed out but +/// never funded and never explicitly released, which would otherwise pin +/// gap-limit headroom forever. +/// +/// The wallet keeps no clock, so this task supplies `now` from the system clock +/// and the caller-supplied `ttl` (seconds). A system-clock failure yields a +/// `now` of `0`, which the wallet treats as "no clock" and reclaims nothing. +/// Runs until `shutdown` is cancelled. +pub(crate) fn spawn_reservation_sweep( + wallet: Arc>, + ttl_secs: u64, + shutdown: CancellationToken, +) -> JoinHandle<()> +where + W: WalletInterface + 'static, +{ + const NAME: &str = "Reservation sweep"; + tokio::spawn(async move { + tracing::debug!("{} task started", NAME); + let mut interval = tokio::time::interval(RESERVATION_SWEEP_INTERVAL); + loop { + tokio::select! { + _ = interval.tick() => { + let now = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0); + let reclaimed = wallet.write().await.sweep_expired_reservations(now, ttl_secs); + if reclaimed > 0 { + tracing::debug!("{}: reclaimed {} expired reservation(s)", NAME, reclaimed); + } + } + _ = shutdown.cancelled() => break, + } + } + tracing::debug!("{} task exiting", NAME); + }) +} + #[cfg(test)] mod tests { use std::net::SocketAddr; diff --git a/dash-spv/src/client/sync_coordinator.rs b/dash-spv/src/client/sync_coordinator.rs index e428bcb80..2f0efaa93 100644 --- a/dash-spv/src/client/sync_coordinator.rs +++ b/dash-spv/src/client/sync_coordinator.rs @@ -7,6 +7,7 @@ use tokio_util::sync::CancellationToken; use super::event_handler::{ spawn_broadcast_monitor, spawn_chainlock_wallet_dispatch, spawn_progress_monitor, + spawn_reservation_sweep, }; use super::DashSpvClient; use crate::error::Result; @@ -103,6 +104,14 @@ impl DashSpvClient DashSpvClient FFIAddressInfo { public_key_len, index: info.index, path: path_str, - used: info.used, - generated_at: info.generated_at, - used_at: info.used_at.unwrap_or(0), + used: info.is_used(), tx_count: info.tx_count, total_received: info.total_received, total_sent: info.total_sent, @@ -879,6 +873,7 @@ pub unsafe extern "C" fn address_info_array_free(infos: *mut *mut FFIAddressInfo mod tests { use super::*; use dash_network::ffi::FFINetwork; + use key_wallet::managed_account::address_pool::AddressState; #[test] fn test_address_pool_type_values() { @@ -914,9 +909,7 @@ mod tests { public_key: Some(PublicKeyType::ECDSA(vec![0x02, 0x03, 0x04])), index: 0, path: test_path, - used: false, - generated_at: 1234567890, - used_at: None, + state: AddressState::Available, tx_count: 0, total_received: 0, total_sent: 0, @@ -931,8 +924,6 @@ mod tests { // Verify basic fields assert_eq!(ffi_info.index, 0); assert!(!ffi_info.used); - assert_eq!(ffi_info.generated_at, 1234567890); - assert_eq!(ffi_info.used_at, 0); assert_eq!(ffi_info.public_key_len, 3); assert!(ffi_info.script_pubkey_len > 0); diff --git a/key-wallet-ffi/src/error.rs b/key-wallet-ffi/src/error.rs index bc2fddf58..db38bf4d1 100644 --- a/key-wallet-ffi/src/error.rs +++ b/key-wallet-ffi/src/error.rs @@ -250,7 +250,8 @@ impl From for FFIError { Error::InvalidNetwork => FFIErrorCode::InvalidNetwork, Error::InvalidAddress(_) => FFIErrorCode::InvalidAddress, Error::Serialization(_) => FFIErrorCode::SerializationError, - Error::WatchOnly + Error::InvalidState(_) + | Error::WatchOnly | Error::CoinJoinNotEnabled | Error::NoKeySource | Error::KeylessWalletRequiresAccountKey { diff --git a/key-wallet-manager/src/events.rs b/key-wallet-manager/src/events.rs index d5d597cba..77a5af1e7 100644 --- a/key-wallet-manager/src/events.rs +++ b/key-wallet-manager/src/events.rs @@ -434,7 +434,7 @@ mod project_derived_addresses_tests { use super::*; use key_wallet::account::StandardAccountType; use key_wallet::bip32::{ChildNumber, DerivationPath}; - use key_wallet::managed_account::address_pool::AddressInfo; + use key_wallet::managed_account::address_pool::{AddressInfo, AddressState}; use std::collections::BTreeMap; /// Compressed encoding of the secp256k1 generator point (G). @@ -488,9 +488,7 @@ mod project_derived_addresses_tests { public_key: Some(PublicKeyType::ECDSA(pubkey_bytes)), index, path, - used: false, - generated_at: 0, - used_at: None, + state: AddressState::Available, tx_count: 0, total_received: 0, total_sent: 0, diff --git a/key-wallet-manager/src/process_block.rs b/key-wallet-manager/src/process_block.rs index 11e59d74c..b5aabca86 100644 --- a/key-wallet-manager/src/process_block.rs +++ b/key-wallet-manager/src/process_block.rs @@ -305,6 +305,10 @@ impl WalletInterface for WalletM self.event_sender.subscribe() } + fn sweep_expired_reservations(&mut self, now: u64, ttl: u64) -> usize { + self.wallet_infos.values_mut().map(|info| info.sweep_expired_reservations(now, ttl)).sum() + } + fn apply_chain_lock(&mut self, chain_lock: ChainLock) { for (wallet_id, info) in self.wallet_infos.iter_mut() { let outcome = info.apply_chain_lock(chain_lock.clone()); @@ -539,6 +543,29 @@ mod tests { assert_eq!(manager.last_processed_height(), 0); } + #[tokio::test] + async fn test_sweep_expired_reservations_fans_out_over_wallets() { + let (mut manager, wallet_id, _addr) = setup_manager_with_wallet(); + + // Reserve a receive address on the wallet's funding account. + manager + .get_wallet_info_mut(&wallet_id) + .expect("wallet info") + .first_bip44_managed_account_mut() + .expect("managed account") + .next_receive_address_and_reserve(None, 1_000) + .expect("reserve"); + + // The manager fans out across every managed wallet. Before the ttl + // elapses nothing is reclaimed. + assert_eq!(manager.sweep_expired_reservations(1_000, 500), 0); + + // Once the reservation ages past the ttl the manager reclaims it, and a + // follow-up sweep finds nothing left. + assert_eq!(manager.sweep_expired_reservations(2_000, 500), 1); + assert_eq!(manager.sweep_expired_reservations(3_000, 500), 0); + } + #[tokio::test] async fn test_process_mempool_transaction_emits_event() { let (mut manager, _wallet_id, addr) = setup_manager_with_wallet(); diff --git a/key-wallet-manager/src/wallet_interface.rs b/key-wallet-manager/src/wallet_interface.rs index 60ade9b29..754ece0cf 100644 --- a/key-wallet-manager/src/wallet_interface.rs +++ b/key-wallet-manager/src/wallet_interface.rs @@ -156,6 +156,17 @@ pub trait WalletInterface: Send + Sync + 'static { 0 } + /// Reclaim expired receive-address reservations across every managed + /// wallet, returning the total number reclaimed. + /// + /// `now` and `ttl` are caller-supplied (seconds); the wallet keeps no + /// clock. A reservation is reclaimed when `now - reserved_at >= ttl`. The + /// default returns `0` for wallet implementations that do not track + /// reservations. + fn sweep_expired_reservations(&mut self, _now: u64, _ttl: u64) -> usize { + 0 + } + /// Subscribe to wallet events (e.g. transactions received, balance changes). fn subscribe_events(&self) -> broadcast::Receiver; diff --git a/key-wallet/src/error.rs b/key-wallet/src/error.rs index dd11fb10d..797a3ff2a 100644 --- a/key-wallet/src/error.rs +++ b/key-wallet/src/error.rs @@ -37,6 +37,8 @@ pub enum Error { Serialization(String), /// Invalid parameter InvalidParameter(String), + /// Internal invariant violated (inconsistent wallet state) + InvalidState(String), /// Watch-only wallet (no private keys available) WatchOnly, /// No key source available for address derivation @@ -71,6 +73,7 @@ impl fmt::Display for Error { Error::CoinJoinNotEnabled => write!(f, "CoinJoin not enabled for this account"), Error::Serialization(s) => write!(f, "Serialization error: {}", s), Error::InvalidParameter(s) => write!(f, "Invalid parameter: {}", s), + Error::InvalidState(s) => write!(f, "Invalid state: {}", s), Error::WatchOnly => write!(f, "Watch-only wallet: private keys not available"), Error::NoKeySource => write!(f, "No key source available for address derivation"), Error::KeylessWalletRequiresAccountKey { diff --git a/key-wallet/src/managed_account/address_pool.rs b/key-wallet/src/managed_account/address_pool.rs index 0bb38e061..a421421fb 100644 --- a/key-wallet/src/managed_account/address_pool.rs +++ b/key-wallet/src/managed_account/address_pool.rs @@ -210,6 +210,28 @@ impl KeySource { } } +/// Lifecycle state of an address in the pool. +/// +/// The three states are mutually exclusive by construction, so the invariant +/// "a used address is never reserved" holds structurally rather than being +/// maintained by hand. The reservation timestamp is caller-supplied (seconds), +/// and `0` means the caller had no clock available. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))] +#[cfg_attr(feature = "bincode", derive(Encode, Decode))] +pub enum AddressState { + /// Free to hand out: neither reserved nor used. + #[default] + Available, + /// Handed out but not yet funded. `at` is when it was reserved. + Reserved { + /// Reservation timestamp. + at: u64, + }, + /// Funds have been seen at this address. + Used, +} + /// Information about a single address in the pool #[derive(Debug, Clone)] #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))] @@ -225,12 +247,8 @@ pub struct AddressInfo { pub index: u32, /// Full derivation path pub path: DerivationPath, - /// Whether this address has been used - pub used: bool, - /// When the address was first generated (timestamp) - pub generated_at: u64, - /// When the address was first used (timestamp) - pub used_at: Option, + /// Lifecycle state: available, reserved, or used + pub state: AddressState, /// Transaction count for this address pub tx_count: u32, /// Total received amount @@ -260,9 +278,7 @@ impl AddressInfo { public_key: Some(public_key), index, path, - used: false, - generated_at: 0, // Should use actual timestamp - used_at: None, + state: AddressState::Available, tx_count: 0, total_received: 0, total_sent: 0, @@ -294,9 +310,7 @@ impl AddressInfo { public_key: None, // Public key not available from script alone index, path, - used: false, - generated_at: 0, // Should use actual timestamp - used_at: None, + state: AddressState::Available, tx_count: 0, total_received: 0, total_sent: 0, @@ -306,11 +320,38 @@ impl AddressInfo { }) } - /// Mark this address as used + /// Mark this address as used. + /// + /// Transitions to [`AddressState::Used`], which structurally drops any + /// reservation so the invariant "a used address is never reserved" holds + /// even when a reserved address is promoted directly to used. fn mark_used(&mut self) { - if !self.used { - self.used = true; - self.used_at = Some(0); // Should use actual timestamp + self.state = AddressState::Used; + } + + /// Whether this address is available to be handed out: neither used nor + /// reserved. + pub fn is_available(&self) -> bool { + matches!(self.state, AddressState::Available) + } + + /// Whether this address is currently reserved (handed out but not yet used). + pub fn is_reserved(&self) -> bool { + matches!(self.state, AddressState::Reserved { .. }) + } + + /// Whether this address has been used. + pub fn is_used(&self) -> bool { + matches!(self.state, AddressState::Used) + } + + /// Timestamp the address was reserved, or `None` if not reserved. + pub fn reserved_at(&self) -> Option { + match self.state { + AddressState::Reserved { + at, + } => Some(at), + _ => None, } } @@ -538,7 +579,7 @@ impl AddressPool { // First, try to find an already generated unused address for i in 0..=self.highest_generated.unwrap_or(0) { if let Some(info) = self.addresses.get(&i) { - if !info.used { + if info.is_available() { return Ok(info.address.clone()); } } @@ -563,7 +604,7 @@ impl AddressPool { // First, try to find an already generated unused address for i in 0..=self.highest_generated.unwrap_or(0) { if let Some(info) = self.addresses.get(&i) { - if !info.used { + if info.is_available() { return Ok(info.clone()); } } @@ -584,6 +625,112 @@ impl AddressPool { }) } + /// Reserve and return the next available address. + /// + /// Scans for the first available index (neither used nor reserved), marks it + /// reserved with `now`, and returns its address. Because the scan and the + /// reserve happen under a single exclusive `&mut self` borrow, two + /// sequential hand-outs cannot both observe the same index as available: the + /// first reserves it before the second runs. When no generated address is + /// available a fresh one is derived and always materialized into the pool, + /// since the reservation is stored on the address entry. + /// + /// `now` is a caller-supplied timestamp (seconds) that stamps the + /// reservation and is what [`AddressPool::sweep_expired_reservations`] later + /// compares against. The pool keeps no clock of its own. + /// + /// This path derives a fresh address whenever every existing address is + /// used or reserved, with no upper bound. Outstanding reservations are + /// reclaimed only by [`AddressPool::release_reservation`] and + /// [`AddressPool::sweep_expired_reservations`], so callers are assumed + /// trusted. Gate or cap this before exposing it to untrusted input. + pub fn next_unused_and_reserve(&mut self, key_source: &KeySource, now: u64) -> Result
{ + for i in 0..=self.highest_generated.unwrap_or(0) { + if let Some(info) = self.addresses.get_mut(&i) { + if info.is_available() { + info.state = AddressState::Reserved { + at: now, + }; + return Ok(info.address.clone()); + } + } + } + + if matches!(key_source, KeySource::NoKeySource) { + return Err(Error::NoKeySource); + } + let next_index = self.highest_generated.map(|h| h + 1).unwrap_or(0); + let address = self.generate_address_at_index(next_index, key_source, true)?; + // `generate_address_at_index` with `add_to_state = true` always inserts + // at `next_index`. If `highest_generated` ever drifts out of sync with + // `addresses` a silent lookup miss would return an unreserved address, + // reintroducing the hand-out race this method exists to close, so report + // the broken invariant as an error rather than fail open. + let info = self.addresses.get_mut(&next_index).ok_or_else(|| { + Error::InvalidState(format!( + "next_unused_and_reserve: generate_address_at_index({}) succeeded but \ + the entry was not stored; pool invariant broken", + next_index + )) + })?; + debug_assert!(info.is_available()); + info.state = AddressState::Reserved { + at: now, + }; + Ok(address) + } + + /// Release a reservation, returning the address to the available pool. + /// + /// Idempotent: returns `false` when the index is not currently reserved + /// (never reserved, already released, already used, or unknown index), and + /// `true` when a reservation was actually cleared. + pub fn release_reservation(&mut self, index: u32) -> bool { + if let Some(info) = self.addresses.get_mut(&index) { + if info.is_reserved() { + info.state = AddressState::Available; + return true; + } + } + false + } + + /// Reclaim reservations older than `ttl`, returning their addresses to the + /// available pool. + /// + /// A reservation is expired when `now - reserved_at >= ttl`. Returns how + /// many reservations were reclaimed. `now` and `ttl` are caller-supplied + /// (seconds). The pool keeps no clock. This is the backstop against a + /// hand-out that is reserved but never paid and never explicitly released, + /// which would otherwise pin gap-limit headroom across restarts forever. + /// + /// A `now` of 0 means the caller has no clock available, so the sweep + /// declines to judge staleness and reclaims nothing. A `ttl` of 0 means no + /// expiry window, so nothing is reclaimed either. A reservation stamped with + /// `at == 0` (its caller had no clock) is likewise never aged out. + pub fn sweep_expired_reservations(&mut self, now: u64, ttl: u64) -> usize { + if now == 0 || ttl == 0 { + return 0; + } + let mut reclaimed = 0; + for info in self.addresses.values_mut() { + if let AddressState::Reserved { + at, + } = info.state + { + // A clockless stamp cannot be judged stale against a real clock. + if at == 0 { + continue; + } + if now.saturating_sub(at) >= ttl { + info.state = AddressState::Available; + reclaimed += 1; + } + } + } + reclaimed + } + /// Get multiple unused addresses at once /// /// Returns the requested number of unused addresses, generating new ones if needed. @@ -604,7 +751,7 @@ impl AddressPool { break; } if let Some(info) = self.addresses.get(&i) { - if !info.used { + if info.is_available() { addresses.push(info.address.clone()); collected += 1; } @@ -660,7 +807,7 @@ impl AddressPool { break; } if let Some(info) = self.addresses.get(&i) { - if !info.used { + if info.is_available() { result.push((info.address.clone(), info.clone())); collected += 1; } @@ -711,7 +858,7 @@ impl AddressPool { && self.highest_generated.map(|h| current_index <= h).unwrap_or(false) { if let Some(info) = self.addresses.get(¤t_index) { - if !info.used { + if info.is_available() { unused.push(info.address.clone()); } } @@ -732,7 +879,7 @@ impl AddressPool { pub fn mark_used(&mut self, address: &Address) -> bool { if let Some(&index) = self.address_index.get(address) { if let Some(info) = self.addresses.get_mut(&index) { - if !info.used { + if !info.is_used() { info.mark_used(); self.used_indices.insert(index); @@ -752,7 +899,7 @@ impl AddressPool { /// Mark an address at a specific index as used pub fn mark_index_used(&mut self, index: u32) -> bool { if let Some(info) = self.addresses.get_mut(&index) { - if !info.used { + if !info.is_used() { info.mark_used(); self.used_indices.insert(index); @@ -776,7 +923,7 @@ impl AddressPool { let mut found = Vec::new(); for (_, info) in self.addresses.iter_mut() { - if !info.used && check_fn(&info.address) { + if !info.is_used() && check_fn(&info.address) { info.mark_used(); self.used_indices.insert(info.index); found.push(info.address.clone()); @@ -804,12 +951,20 @@ impl AddressPool { /// Get only used addresses pub fn used_addresses(&self) -> Vec
{ - self.addresses.values().filter(|info| info.used).map(|info| info.address.clone()).collect() + self.addresses + .values() + .filter(|info| info.is_used()) + .map(|info| info.address.clone()) + .collect() } - /// Get only unused addresses + /// Get only available addresses (neither used nor reserved) pub fn unused_addresses(&self) -> Vec
{ - self.addresses.values().filter(|info| !info.used).map(|info| info.address.clone()).collect() + self.addresses + .values() + .filter(|info| info.is_available()) + .map(|info| info.address.clone()) + .collect() } /// Get address at specific index @@ -892,9 +1047,10 @@ impl AddressPool { /// Check if we need to generate more addresses pub fn needs_more_addresses(&self) -> bool { - let unused_count = self.addresses.values().filter(|info| !info.used).count() as u32; + let available_count = + self.addresses.values().filter(|info| info.is_available()).count() as u32; - unused_count < self.gap_limit + available_count < self.gap_limit } /// Raise the gap limit and generate the addresses the wider limit now requires. @@ -928,17 +1084,17 @@ impl AddressPool { let next_index = self.highest_generated.map(|h| h + 1).unwrap_or(0); self.generate_address_at_index(next_index, key_source, true)?; // `generate_address_at_index` with `add_to_state = true` always - // inserts at `next_index`. Asserting the invariant explicitly - // here turns a regression that breaks it (e.g. a refactor that - // hits the early-return branch on a re-derivation) into a loud - // panic instead of an infinite loop on the outer `while`. - let info = self.addresses.get(&next_index).cloned().unwrap_or_else(|| { - panic!( + // inserts at `next_index`. Surfacing a regression that breaks it + // (e.g. a refactor that hits the early-return branch on a + // re-derivation) as an error keeps the outer `while` from spinning + // into an infinite loop. + let info = self.addresses.get(&next_index).cloned().ok_or_else(|| { + Error::InvalidState(format!( "maintain_gap_limit: generate_address_at_index({}) succeeded but \ the entry was not stored; pool invariant broken", next_index - ) - }); + )) + })?; new_addresses.push(info); } @@ -968,11 +1124,14 @@ impl AddressPool { /// Get pool statistics pub fn stats(&self) -> PoolStats { let used_count = self.used_indices.len() as u32; - let unused_count = self.addresses.len() as u32 - used_count; + let reserved_count = + self.addresses.values().filter(|info| info.is_reserved()).count() as u32; + let unused_count = self.addresses.len() as u32 - used_count - reserved_count; PoolStats { total_generated: self.addresses.len() as u32, used_count, + reserved_count, unused_count, highest_used: self.highest_used, highest_generated: self.highest_generated, @@ -984,8 +1143,7 @@ impl AddressPool { /// Reset the pool (for rescan) pub fn reset_usage(&mut self) { for info in self.addresses.values_mut() { - info.used = false; - info.used_at = None; + info.state = AddressState::Available; info.tx_count = 0; info.total_received = 0; info.total_sent = 0; @@ -1005,9 +1163,11 @@ impl AddressPool { let mut pruned = 0; let indices_to_remove: Vec = self .addresses - .keys() - .filter(|&&idx| idx > keep_until && !self.used_indices.contains(&idx)) - .copied() + .iter() + .filter(|(&idx, info)| { + idx > keep_until && !self.used_indices.contains(&idx) && !info.is_reserved() + }) + .map(|(&idx, _)| idx) .collect(); for idx in indices_to_remove { @@ -1033,7 +1193,9 @@ pub struct PoolStats { pub total_generated: u32, /// Number of used addresses pub used_count: u32, - /// Number of unused addresses + /// Number of reserved addresses (handed out but not yet used) + pub reserved_count: u32, + /// Number of unused addresses (neither used nor reserved) pub unused_count: u32, /// Highest used index pub highest_used: Option, @@ -1049,7 +1211,7 @@ impl fmt::Display for PoolStats { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!( f, - "{} pool: {} addresses ({} used, {} unused), gap limit: {}", + "{} pool: {} addresses ({} used, {} reserved, {} unused), gap limit: {}", if self.is_internal { "Internal" } else { @@ -1057,6 +1219,7 @@ impl fmt::Display for PoolStats { }, self.total_generated, self.used_count, + self.reserved_count, self.unused_count, self.gap_limit ) @@ -1276,6 +1439,266 @@ mod tests { assert_eq!(pool.addresses.len(), crate::gap_limit::MAX_GAP_LIMIT as usize); } + #[test] + fn test_reserve_returns_distinct_addresses() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + // Without reserving, two hand-outs return the same address (the race). + let peek1 = pool.next_unused(&key_source, true).unwrap(); + let peek2 = pool.next_unused(&key_source, true).unwrap(); + assert_eq!(peek1, peek2); + + // Reserving on hand-out makes two sequential calls return distinct + // addresses: the first is no longer available to the second. + let addr1 = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + let addr2 = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + assert_ne!(addr1, addr2); + + // A plain peek must skip the reserved addresses too. + let peek3 = pool.next_unused(&key_source, true).unwrap(); + assert_ne!(peek3, addr1); + assert_ne!(peek3, addr2); + } + + #[test] + fn test_release_returns_address_to_pool() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let addr = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + let index = pool.address_index(&addr).unwrap(); + + // A reserved address is excluded from the available set. + assert!(!pool.unused_addresses().contains(&addr)); + + // After releasing, the same address becomes available again. + assert!(pool.release_reservation(index)); + assert!(pool.unused_addresses().contains(&addr)); + let reissued = pool.next_unused_and_reserve(&key_source, 200).unwrap(); + assert_eq!(addr, reissued); + } + + #[test] + fn test_release_is_idempotent() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let addr = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + let index = pool.address_index(&addr).unwrap(); + + // First release clears the reservation, a second is a no-op. + assert!(pool.release_reservation(index)); + assert!(!pool.release_reservation(index)); + // Releasing a never-reserved index is a no-op too. + let other = pool.next_unused(&key_source, true).unwrap(); + let other_index = pool.address_index(&other).unwrap(); + assert!(!pool.release_reservation(other_index)); + // Unknown index is a no-op. + assert!(!pool.release_reservation(9_999)); + } + + #[test] + fn test_reserved_promoted_to_used_clears_reservation() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let addr = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + let index = pool.address_index(&addr).unwrap(); + assert!(pool.addresses.get(&index).unwrap().is_reserved()); + + // Funds arriving mark the reserved address used, so the reservation is + // structurally dropped (a used address is never reserved). + assert!(pool.mark_used(&addr)); + let info = pool.addresses.get(&index).unwrap(); + assert!(info.is_used()); + assert!(!info.is_reserved()); + assert!(info.reserved_at().is_none()); + // Releasing a now-used index is a no-op. + assert!(!pool.release_reservation(index)); + } + + #[cfg(feature = "serde")] + #[test] + fn test_reservation_survives_serde_round_trip() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let reserved = pool.next_unused_and_reserve(&key_source, 12_345).unwrap(); + let index = pool.address_index(&reserved).unwrap(); + + // Unlike an ephemeral reservation, the reserved state must survive a + // round trip so a handed-out address is not re-issued after a restart. + let json = serde_json::to_string(&pool).unwrap(); + let mut restored: AddressPool = serde_json::from_str(&json).unwrap(); + + assert_eq!(restored.addresses.get(&index).unwrap().reserved_at(), Some(12_345)); + assert!(restored.addresses.get(&index).unwrap().is_reserved()); + + // A fresh hand-out on the restored pool skips the still-reserved address. + let next = restored.next_unused_and_reserve(&key_source, 12_346).unwrap(); + assert_ne!(next, reserved); + } + + #[cfg(feature = "bincode")] + #[test] + fn test_reservation_survives_bincode_round_trip() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let reserved = pool.next_unused_and_reserve(&key_source, 12_345).unwrap(); + let index = pool.address_index(&reserved).unwrap(); + + // The reserved state must survive a bincode round trip so a handed-out + // address is not re-issued after a restart. + let bytes = bincode::encode_to_vec(&pool, bincode::config::standard()).unwrap(); + let (mut restored, _): (AddressPool, usize) = + bincode::decode_from_slice(&bytes, bincode::config::standard()).unwrap(); + + assert_eq!(restored.addresses.get(&index).unwrap().reserved_at(), Some(12_345)); + assert!(restored.addresses.get(&index).unwrap().is_reserved()); + + // A fresh hand-out on the restored pool skips the still-reserved address. + let next = restored.next_unused_and_reserve(&key_source, 12_346).unwrap(); + assert_ne!(next, reserved); + } + + #[test] + fn test_sweep_expired_reservations() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let old = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + let recent = pool.next_unused_and_reserve(&key_source, 200).unwrap(); + let clockless = pool.next_unused_and_reserve(&key_source, 0).unwrap(); + let old_index = pool.address_index(&old).unwrap(); + let recent_index = pool.address_index(&recent).unwrap(); + let clockless_index = pool.address_index(&clockless).unwrap(); + + // now == 0 means no clock: nothing is reclaimed. + assert_eq!(pool.sweep_expired_reservations(0, 100), 0); + assert!(pool.addresses.get(&old_index).unwrap().is_reserved()); + + // ttl == 0 means no expiry window: live reservations are left intact. + assert_eq!(pool.sweep_expired_reservations(250, 0), 0); + assert!(pool.addresses.get(&old_index).unwrap().is_reserved()); + assert!(pool.addresses.get(&recent_index).unwrap().is_reserved()); + + // At now=250 with ttl=100: old (age 150) expires, recent (age 50) stays. + // A clockless reservation (reserved_at == 0) is never aged out. + assert_eq!(pool.sweep_expired_reservations(250, 100), 1); + assert!(!pool.addresses.get(&old_index).unwrap().is_reserved()); + assert!(pool.addresses.get(&recent_index).unwrap().is_reserved()); + assert!(pool.addresses.get(&clockless_index).unwrap().is_reserved()); + + // The reclaimed address is available for hand-out again. + let reissued = pool.next_unused_and_reserve(&key_source, 300).unwrap(); + assert_eq!(old, reissued); + } + + #[test] + fn test_reset_usage_clears_reservations() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + 5, + Network::Testnet, + ); + let key_source = test_key_source(); + + let reserved = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + let index = pool.address_index(&reserved).unwrap(); + assert!(pool.addresses.get(&index).unwrap().is_reserved()); + + // A rescan reset must drop reservations too, otherwise the address stays + // pinned forever and is never handed out again. + pool.reset_usage(); + assert!(!pool.addresses.get(&index).unwrap().is_reserved()); + + let reissued = pool.next_unused_and_reserve(&key_source, 200).unwrap(); + assert_eq!(reserved, reissued); + } + + #[test] + fn test_prune_never_drops_reserved() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let gap_limit = 5; + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + gap_limit, + Network::Testnet, + ); + let key_source = test_key_source(); + pool.generate_addresses(10, &key_source, true).unwrap(); + + // Mark index 0 used so keep_until = highest_used + gap_limit = 5. + pool.mark_index_used(0); + + // Reserve indices 1..=7 (first available each time, index 0 is used). + let mut reserved = Vec::new(); + for t in 0..7 { + reserved.push(pool.next_unused_and_reserve(&key_source, 100 + t).unwrap()); + } + + // Indices 6 and 7 are reserved and beyond keep_until. Indices 8 and 9 + // are plain unused beyond keep_until and are the only prunable entries. + let pruned = pool.prune_unused(); + assert_eq!(pruned, 2); + assert!(!pool.addresses.contains_key(&8)); + assert!(!pool.addresses.contains_key(&9)); + + // Every reserved address survived the prune. + for addr in &reserved { + assert!(pool.address_index(addr).is_some()); + } + assert!(pool.addresses.get(&6).unwrap().is_reserved()); + assert!(pool.addresses.get(&7).unwrap().is_reserved()); + } + #[test] fn test_gap_limit_maintenance() { let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); @@ -1327,6 +1750,29 @@ mod tests { assert_eq!(pool.addresses.len(), gap_limit as usize + 3); } + #[test] + fn test_reserved_addresses_count_against_gap_limit() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let key_source = test_key_source(); + let gap_limit = 5; + + // A pool with exactly gap_limit available addresses needs no more. + let mut pool = AddressPool::new( + base_path, + AddressPoolType::External, + gap_limit, + Network::Testnet, + &key_source, + ) + .unwrap(); + assert!(!pool.needs_more_addresses()); + + // Reserving one drops available headroom below the gap limit, so the + // pool now reports it needs more even though nothing was used. + pool.next_unused_and_reserve(&key_source, 100).unwrap(); + assert!(pool.needs_more_addresses()); + } + #[test] fn test_address_pool_builder() { let pool = AddressPoolBuilder::new() diff --git a/key-wallet/src/managed_account/managed_core_funds_account.rs b/key-wallet/src/managed_account/managed_core_funds_account.rs index da4c2c13a..1f7001f67 100644 --- a/key-wallet/src/managed_account/managed_core_funds_account.rs +++ b/key-wallet/src/managed_account/managed_core_funds_account.rs @@ -617,6 +617,93 @@ impl ManagedCoreFundsAccount { } } + /// Generate and reserve the next receive address. + /// + /// Like [`Self::next_receive_address`] but atomically reserves the returned + /// address so a concurrent hand-out cannot return the same one. The + /// reservation persists until funds arrive (promoting it to used), + /// [`Self::release_receive_reservation`] is called, or it is reclaimed by a + /// TTL sweep. `now` is a caller-supplied timestamp (seconds) stamping the + /// reservation. Only valid for Standard accounts. + /// + /// Derivation of fresh addresses on this path is unbounded, so callers are + /// assumed trusted. See [`address_pool::AddressPool::next_unused_and_reserve`]. + pub fn next_receive_address_and_reserve( + &mut self, + account_xpub: Option<&ExtendedPubKey>, + now: u64, + ) -> Result { + if let ManagedAccountType::Standard { + external_addresses, + .. + } = self.keys.managed_account_type_mut() + { + let key_source = match account_xpub { + Some(xpub) => address_pool::KeySource::Public(*xpub), + None => address_pool::KeySource::NoKeySource, + }; + + let addr = external_addresses.next_unused_and_reserve(&key_source, now).map_err( + |e| match e { + crate::error::Error::NoKeySource => { + "No unused addresses available and no key source provided" + } + _ => "Failed to generate receive address", + }, + )?; + self.keys.bump_monitor_revision(); + Ok(addr) + } else { + Err("Cannot generate receive address for non-standard account type") + } + } + + /// Release a previously reserved receive address back to the available pool. + /// + /// Idempotent: returns `false` if the address is unknown to the external + /// pool or is not currently reserved. Bumps the monitor revision only when + /// a reservation is actually cleared. + pub fn release_receive_reservation(&mut self, address: &Address) -> bool { + if let ManagedAccountType::Standard { + external_addresses, + .. + } = self.keys.managed_account_type_mut() + { + if let Some(index) = external_addresses.address_index(address) { + if external_addresses.release_reservation(index) { + self.keys.bump_monitor_revision(); + return true; + } + } + } + false + } + + /// Reclaim receive reservations older than `ttl`, returning their addresses + /// to the available pool. + /// + /// Backstop for reservations handed out but never funded and never + /// explicitly released, which would otherwise pin gap-limit headroom + /// forever. `now` and `ttl` are caller-supplied (seconds); the wallet keeps + /// no clock. Returns the number of reservations reclaimed and bumps the + /// monitor revision when that is non-zero. See + /// [`address_pool::AddressPool::sweep_expired_reservations`]. + pub fn sweep_expired_receive_reservations(&mut self, now: u64, ttl: u64) -> usize { + if let ManagedAccountType::Standard { + external_addresses, + .. + } = self.keys.managed_account_type_mut() + { + let reclaimed = external_addresses.sweep_expired_reservations(now, ttl); + if reclaimed > 0 { + self.keys.bump_monitor_revision(); + } + reclaimed + } else { + 0 + } + } + /// Generate multiple receive addresses at once using the optionally provided extended public key. /// Only valid for Standard accounts. pub fn next_receive_addresses( diff --git a/key-wallet/src/tests/address_pool_tests.rs b/key-wallet/src/tests/address_pool_tests.rs index 25ec66b1a..7da10b5cb 100644 --- a/key-wallet/src/tests/address_pool_tests.rs +++ b/key-wallet/src/tests/address_pool_tests.rs @@ -54,15 +54,20 @@ fn test_next_unused_multiple() { pool.mark_used(&addresses[0]); pool.mark_used(&addresses[2]); - // Request more addresses - should get 3 unused + 2 new + // Reserve the first remaining unused address: it must be skipped too. + let reserved = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + assert_eq!(reserved, addresses[1]); + + // Request more addresses: should skip used and reserved, returning the + // remaining generated ones plus freshly derived ones to satisfy the count. let more_addresses = pool.next_unused_multiple(5, &key_source, true); assert_eq!(more_addresses.len(), 5); - assert_eq!(more_addresses[0], addresses[1]); // First unused - assert_eq!(more_addresses[1], addresses[3]); // Second unused - assert_eq!(more_addresses[2], addresses[4]); // Third unused - // more_addresses[3] and [4] should be newly generated + assert!(!more_addresses.contains(&reserved)); + assert_eq!(more_addresses[0], addresses[3]); // First available + assert_eq!(more_addresses[1], addresses[4]); // Second available + // remaining entries are newly generated - assert_eq!(pool.highest_generated, Some(6)); // Generated 2 more + assert_eq!(pool.highest_generated, Some(7)); // Generated 3 more } #[test] @@ -84,18 +89,23 @@ fn test_next_unused_multiple_with_info() { for (i, (addr, info)) in address_infos.iter().enumerate() { assert_eq!(addr, &info.address); assert_eq!(info.index, i as u32); - assert!(!info.used); + assert!(!info.is_used()); assert!(info.public_key.is_some()); } // Mark first one as used pool.mark_used(&address_infos[0].0); - // Get more with info - should skip the used one + // Reserve the second one: it must be skipped alongside the used one. + let reserved = pool.next_unused_and_reserve(&key_source, 100).unwrap(); + assert_eq!(reserved, address_infos[1].0); + + // Get more with info: should skip both the used and the reserved address, + // still returning the requested count by deriving fresh addresses. let more_infos = pool.next_unused_multiple_with_info(3, &key_source, true); assert_eq!(more_infos.len(), 3); - assert_eq!(more_infos[0].0, address_infos[1].0); // Should skip the first (used) one - assert_eq!(more_infos[1].0, address_infos[2].0); + assert!(!more_infos.iter().any(|(addr, _)| addr == &reserved)); + assert_eq!(more_infos[0].0, address_infos[2].0); // First available after used + reserved } #[test] diff --git a/key-wallet/src/tests/address_reservation_tests.rs b/key-wallet/src/tests/address_reservation_tests.rs new file mode 100644 index 000000000..58c748dbb --- /dev/null +++ b/key-wallet/src/tests/address_reservation_tests.rs @@ -0,0 +1,168 @@ +//! End-to-end coverage for receive-address reservation through the managed +//! wallet: two reserved hand-outs are distinct, a reservation survives until +//! funds arrive, and the normal transaction-checking path promotes the +//! reserved address to used (clearing the reservation). + +use crate::account::{Account, AccountType, ManagedCoreFundsAccount}; +use crate::managed_account::address_pool::KeySource; +use crate::managed_account::managed_account_trait::ManagedAccountTrait; +use crate::managed_account::managed_account_type::ManagedAccountType; +use crate::mnemonic::{Language, Mnemonic}; +use crate::test_utils::TestWalletContext; +use crate::transaction_checking::TransactionContext; +use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; +use crate::{ExtendedPrivKey, Network}; +use dashcore::{Address, Transaction}; +use secp256k1::Secp256k1; +use std::str::FromStr; + +#[tokio::test] +async fn test_reserved_receive_address_promotes_on_funding() { + let mut ctx = TestWalletContext::new_random(); + let xpub = ctx.xpub; + + let revision_before = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("managed account") + .monitor_revision(); + + let a = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("managed account") + .next_receive_address_and_reserve(Some(&xpub), 1_000) + .expect("reserve a"); + let b = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("managed account") + .next_receive_address_and_reserve(Some(&xpub), 1_000) + .expect("reserve b"); + + // Each reservation must bump the monitor revision so observers re-read the + // address set. + let revision_after_reserve = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("managed account") + .monitor_revision(); + assert!(revision_after_reserve > revision_before); + + // Two sequential receive hand-outs return distinct addresses: reserving the + // first removes it from the available pool seen by the second. + assert_ne!(a, b); + + // Funds arriving at `a` promote it Reserved -> Used through the normal + // checker, which clears the reservation at the single mark_used chokepoint. + let tx = Transaction::dummy(&a, 0..1, &[150_000]); + let result = ctx.check_transaction(&tx, TransactionContext::Mempool).await; + assert!(result.is_relevant); + + let acc = ctx.managed_wallet.first_bip44_managed_account_mut().expect("managed account"); + // `a` is used now, so its reservation is already gone: releasing is a no-op. + assert!(!acc.release_receive_reservation(&a)); + // `b` is still reserved, so releasing it returns it to the available pool + // and bumps the monitor revision again. + let revision_before_release = acc.monitor_revision(); + assert!(acc.release_receive_reservation(&b)); + assert!(acc.monitor_revision() > revision_before_release); + + // Releasing an address the pool has never seen is a no-op. + let stray = Address::from_str("yTb47qEBpNmgXvYYsHEN4nh8yJwa5iC4Cs") + .unwrap() + .require_network(Network::Testnet) + .unwrap(); + assert!(!acc.release_receive_reservation(&stray)); +} + +#[test] +fn test_reserve_receive_without_key_source_errors() { + // A standard account whose external pool has no pre-generated addresses + // and no key source cannot hand out a reservation. + let mut account = ManagedCoreFundsAccount::dummy_bip44(); + let err = account + .next_receive_address_and_reserve(None, 1_000) + .expect_err("no key source and an empty pool must fail"); + assert_eq!(err, "No unused addresses available and no key source provided"); +} + +#[test] +fn test_reserve_receive_on_non_standard_account_errors() { + let network = Network::Testnet; + let mnemonic = Mnemonic::from_phrase( + "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about", + Language::English, + ) + .unwrap(); + let seed = mnemonic.to_seed(""); + let master = ExtendedPrivKey::new_master(network, &seed).unwrap(); + let secp = Secp256k1::new(); + + let account_type = AccountType::CoinJoin { + index: 0, + }; + let derivation_path = account_type.derivation_path(network).unwrap(); + let account_key = master.derive_priv(&secp, &derivation_path).unwrap(); + let account = Account::from_xpriv(Some([0u8; 32]), account_type, account_key, network).unwrap(); + + let key_source = KeySource::Public(account.account_xpub); + let managed_type = + ManagedAccountType::from_account_type(account.account_type, network, &key_source).unwrap(); + let mut managed = ManagedCoreFundsAccount::new(managed_type, network); + + let err = managed + .next_receive_address_and_reserve(Some(&account.account_xpub), 1_000) + .expect_err("non-standard account must not hand out receive reservations"); + assert_eq!(err, "Cannot generate receive address for non-standard account type"); + + // The sweep wrapper is likewise inert on a non-standard account. + assert_eq!(managed.sweep_expired_receive_reservations(10_000, 1), 0); +} + +#[test] +fn test_sweep_expired_receive_reservations_reclaims_through_account() { + let mut ctx = TestWalletContext::new_random(); + let xpub = ctx.xpub; + + let acc = ctx.managed_wallet.first_bip44_managed_account_mut().expect("managed account"); + let reserved = acc.next_receive_address_and_reserve(Some(&xpub), 1_000).expect("reserve"); + let revision_after_reserve = acc.monitor_revision(); + + // A sweep before the ttl elapses leaves the reservation intact and does not + // bump the monitor revision. + assert_eq!(acc.sweep_expired_receive_reservations(1_000, 500), 0); + assert_eq!(acc.monitor_revision(), revision_after_reserve); + + // Once the reservation is older than the ttl it is reclaimed, the revision + // bumps, and releasing the now-available address is a no-op. + assert_eq!(acc.sweep_expired_receive_reservations(2_000, 500), 1); + assert!(acc.monitor_revision() > revision_after_reserve); + assert!(!acc.release_receive_reservation(&reserved)); +} + +#[test] +fn test_sweep_expired_reservations_reclaims_across_wallet() { + let mut ctx = TestWalletContext::new_random(); + let xpub = ctx.xpub; + + let reserved = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("managed account") + .next_receive_address_and_reserve(Some(&xpub), 1_000) + .expect("reserve"); + + // The wallet-level wrapper fans out over every funding account. Before the + // ttl elapses nothing is reclaimed. + assert_eq!(ctx.managed_wallet.sweep_expired_reservations(1_000, 500), 0); + + // Once the reservation ages past the ttl the wrapper reclaims it and the + // address is available again, so releasing it is a no-op. + assert_eq!(ctx.managed_wallet.sweep_expired_reservations(2_000, 500), 1); + assert!(!ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("managed account") + .release_receive_reservation(&reserved)); +} diff --git a/key-wallet/src/tests/mod.rs b/key-wallet/src/tests/mod.rs index acb7a6378..0e356a976 100644 --- a/key-wallet/src/tests/mod.rs +++ b/key-wallet/src/tests/mod.rs @@ -8,6 +8,8 @@ mod balance_tests; mod address_pool_tests; +mod address_reservation_tests; + mod advanced_transaction_tests; mod backup_restore_tests; diff --git a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs index ca7aca184..559549162 100644 --- a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs +++ b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs @@ -152,6 +152,20 @@ pub trait WalletInfoInterface: Sized + WalletTransactionChecker + ManagedAccount self.funds_account(preference, index).map(|account| account.balance) } + /// Reclaim expired receive-address reservations across every funding + /// account, returning the total number reclaimed. + /// + /// `now` and `ttl` are caller-supplied (seconds); the wallet keeps no + /// clock. Delegates to `ManagedCoreFundsAccount::sweep_expired_receive_reservations` + /// for each funding account. + fn sweep_expired_reservations(&mut self, now: u64, ttl: u64) -> usize { + self.accounts_mut() + .all_funding_accounts_mut() + .into_iter() + .map(|account| account.sweep_expired_receive_reservations(now, ttl)) + .sum() + } + /// Get transaction history fn transaction_history(&self) -> Vec<&TransactionRecord>; From 9c87ccb33ff79803406411d9d0448da7379f325a Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Mon, 27 Jul 2026 20:21:35 +0700 Subject: [PATCH 059/149] feat(key-wallet): CoinJoin funding account + drain mode for asset-lock builds (#915) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet): CoinJoin funding account + drain mode for asset-lock builds Asset-lock builders (build_asset_lock / build_asset_lock_with_signer) previously hardcoded the funding source to standard BIP44 accounts. This adds: - AssetLockFundingAccount { Bip44, CoinJoin }: which account family supplies (and signs) the funding UTXOs. CoinJoin lets mixed coins fund an asset lock directly, without first sweeping them through a transparent BIP44 address. - drain mode: consume every final UTXO of the funding account and rewrite the single credit output's value to sum(inputs) - fee. The TransactionBuilder's SelectionStrategy::All path now also rewrites the asset-lock payload credit output so the payload always mirrors the on-chain OP_RETURN burn (a mismatch is consensus-invalid), and rejects a zero-value drain. - CoinJoin funding is drain-only: partial spends would need CoinJoin change re-denomination, which the builder does not do; a non-drain CoinJoin build is rejected with InvalidData. The key-wallet-ffi build_asset_lock entry point keeps its behavior (BIP44, non-drain). Co-Authored-By: Claude Fable 5 * refactor(key-wallet): address review — hoist zero-drain guard, dedup drain validation - The `drained == 0` rejection now guards every `SelectionStrategy::All` build, not just the asset-lock payload branch, so a plain drain can no longer produce a zero-value output. - Extract the duplicated drain/CoinJoin funding validation from `build_asset_lock` / `build_asset_lock_with_signer` into a shared `validate_drain_funding` helper. - cargo fmt over the new asset-lock builder tests (CI pre-commit failure). Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- key-wallet-ffi/src/transaction.rs | 5 +- .../managed_wallet_info/asset_lock_builder.rs | 419 ++++++++++++++++-- .../transaction_builder.rs | 24 +- 3 files changed, 409 insertions(+), 39 deletions(-) diff --git a/key-wallet-ffi/src/transaction.rs b/key-wallet-ffi/src/transaction.rs index 7878506bd..700d2c192 100644 --- a/key-wallet-ffi/src/transaction.rs +++ b/key-wallet-ffi/src/transaction.rs @@ -840,9 +840,12 @@ pub unsafe extern "C" fn wallet_build_and_sign_asset_lock_transaction( let result = unwrap_or_return!(managed_wallet.build_asset_lock( wallet_ref.inner(), - account_index, + key_wallet::wallet::managed_wallet_info::asset_lock_builder::AssetLockFundingAccount::Bip44 { + account_index, + }, fundings, fee_per_kb, + false, ).await, error); // Write outputs diff --git a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs index 4542ace06..8c00a9a9e 100644 --- a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs @@ -12,6 +12,7 @@ use std::fmt; use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::managed_account::ManagedCoreKeysAccount; use crate::signer::Signer; +use crate::wallet::managed_wallet_info::coin_selection::SelectionStrategy; use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::wallet::managed_wallet_info::transaction_builder::{BuilderError, TransactionBuilder}; use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; @@ -37,6 +38,43 @@ pub enum AssetLockFundingType { AssetLockShieldedAddressTopUp, } +/// Which wallet account supplies the funding UTXOs (and signs the inputs) +/// of an asset lock transaction. +/// +/// `Bip44` is the standard spendable balance — the historical behavior of +/// the builders below. `CoinJoin` lets mixed coins fund an asset lock +/// directly, without first sweeping them through a transparent BIP44 +/// address (which would link the mixed UTXOs to a reusable transparent +/// address for an extra hop). +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum AssetLockFundingAccount { + /// Standard BIP44 account (`m/44'/coinType'/index'`). + Bip44 { + /// BIP44 account index. + account_index: u32, + }, + /// CoinJoin account (`m/9'/coinType'/4'/index'`). + CoinJoin { + /// CoinJoin account index. + account_index: u32, + }, +} + +impl AssetLockFundingAccount { + /// The account index within its family. + pub fn account_index(&self) -> u32 { + match self { + Self::Bip44 { + account_index, + } + | Self::CoinJoin { + account_index, + } => *account_index, + } + } +} + /// Per-credit-output funding specification. pub struct CreditOutputFunding { /// The credit output (script + amount). @@ -93,7 +131,7 @@ pub enum AssetLockError { SigningFailed(String), /// The wallet does not have a private key (watch-only). WatchOnlyWallet, - /// The specified BIP44 account was not found. + /// The specified funding account (BIP44 or CoinJoin, by index) was not found. AccountNotFound(u32), /// No change address available. NoChangeAddress, @@ -114,7 +152,7 @@ impl fmt::Display for AssetLockError { Self::Signer(msg) => write!(f, "Signer error: {msg}"), Self::SigningFailed(msg) => write!(f, "Signing failed: {msg}"), Self::WatchOnlyWallet => write!(f, "Cannot sign with watch-only wallet"), - Self::AccountNotFound(idx) => write!(f, "BIP44 account {} not found", idx), + Self::AccountNotFound(idx) => write!(f, "funding account {} not found", idx), Self::NoChangeAddress => write!(f, "No change address available"), Self::Builder(e) => write!(f, "Transaction builder error: {e}"), } @@ -168,6 +206,28 @@ fn resolve_funding_account( } } +/// Shared guard for both asset-lock builders: a drain rewrites exactly one +/// credit output, and CoinJoin accounts have no change-address pool semantics +/// for asset locks (change would need re-denomination), so they only support +/// the whole-balance drain. +fn validate_drain_funding( + funding_account: AssetLockFundingAccount, + credit_output_count: usize, + drain: bool, +) -> Result<(), AssetLockError> { + if drain && credit_output_count != 1 { + return Err(AssetLockError::Builder(BuilderError::InvalidData( + "drain asset lock requires exactly one credit output".into(), + ))); + } + if matches!(funding_account, AssetLockFundingAccount::CoinJoin { .. }) && !drain { + return Err(AssetLockError::Builder(BuilderError::InvalidData( + "CoinJoin-funded asset locks support drain mode only".into(), + ))); + } + Ok(()) +} + impl ManagedWalletInfo { /// Build and sign an asset lock transaction. /// @@ -176,12 +236,20 @@ impl ManagedWalletInfo { /// /// The transaction is built first, and keys are only derived after a successful /// build — so no addresses are consumed if the build fails. + /// + /// `funding_account` picks which account family supplies (and signs) the + /// funding UTXOs — see [`AssetLockFundingAccount`]. `drain` locks the + /// account's whole spendable balance: every final UTXO is consumed and + /// the single credit output's value is rewritten to `Σ inputs − fee` + /// (the caller's credit-output value is ignored; exactly one credit + /// output is required). pub async fn build_asset_lock( &mut self, wallet: &Wallet, - account_index: u32, + funding_account: AssetLockFundingAccount, credit_output_fundings: Vec, fee_per_kb: u64, + drain: bool, ) -> Result { // Surface watch-only / no-private-key wallets here so we don't reserve // a change index before the build can possibly succeed. @@ -191,27 +259,54 @@ impl ManagedWalletInfo { let network = self.network; let height = self.last_processed_height(); - let acc = &wallet - .get_bip44_account(account_index) - .ok_or(AssetLockError::AccountNotFound(account_index))?; + let account_index = funding_account.account_index(); + let acc = match funding_account { + AssetLockFundingAccount::Bip44 { + .. + } => wallet + .get_bip44_account(account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))?, + AssetLockFundingAccount::CoinJoin { + .. + } => wallet + .get_coinjoin_account(account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))?, + }; + + let funds_acc = match funding_account { + AssetLockFundingAccount::Bip44 { + .. + } => self + .accounts + .standard_bip44_accounts + .get_mut(&account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))?, + AssetLockFundingAccount::CoinJoin { + .. + } => self + .accounts + .coinjoin_accounts + .get_mut(&account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))?, + }; - let funds_acc = self - .accounts - .standard_bip44_accounts - .get_mut(&account_index) - .ok_or(AssetLockError::AccountNotFound(account_index))?; + validate_drain_funding(funding_account, credit_output_fundings.len(), drain)?; let credit_outputs: Vec = credit_output_fundings.iter().map(|f| f.output.clone()).collect(); // Build first, derive credit keys after — a build failure must not // consume any funding-key indices. - let (transaction, fee) = TransactionBuilder::new() + let mut builder = TransactionBuilder::new() .set_fee_rate(FeeRate::new(fee_per_kb)) .set_current_height(height) .set_special_payload(TransactionPayload::AssetLockPayloadType(AssetLockPayload::new( credit_outputs, - ))) + ))); + if drain { + builder = builder.set_selection_strategy(SelectionStrategy::All); + } + let (transaction, fee) = builder .set_funding(funds_acc, acc) .require_final_inputs() .build_signed(wallet, |addr| funds_acc.address_derivation_path(&addr)) @@ -253,36 +348,67 @@ impl ManagedWalletInfo { /// one per credit output in payload order. The caller uses the paths to /// request signatures from the same signer when later consuming the /// credits on Platform. + /// + /// `funding_account` / `drain` — see [`Self::build_asset_lock`]. pub async fn build_asset_lock_with_signer( &mut self, wallet: &Wallet, - account_index: u32, + funding_account: AssetLockFundingAccount, credit_output_fundings: Vec, fee_per_kb: u64, + drain: bool, signer: &S, ) -> Result { let height = self.last_processed_height(); - let acc = wallet - .get_bip44_account(account_index) - .ok_or(AssetLockError::AccountNotFound(account_index))? - .clone(); + let account_index = funding_account.account_index(); + let acc = match funding_account { + AssetLockFundingAccount::Bip44 { + .. + } => wallet + .get_bip44_account(account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))? + .clone(), + AssetLockFundingAccount::CoinJoin { + .. + } => wallet + .get_coinjoin_account(account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))? + .clone(), + }; - let funds_acc = self - .accounts - .standard_bip44_accounts - .get_mut(&account_index) - .ok_or(AssetLockError::AccountNotFound(account_index))?; + let funds_acc = match funding_account { + AssetLockFundingAccount::Bip44 { + .. + } => self + .accounts + .standard_bip44_accounts + .get_mut(&account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))?, + AssetLockFundingAccount::CoinJoin { + .. + } => self + .accounts + .coinjoin_accounts + .get_mut(&account_index) + .ok_or(AssetLockError::AccountNotFound(account_index))?, + }; + + validate_drain_funding(funding_account, credit_output_fundings.len(), drain)?; let credit_outputs: Vec = credit_output_fundings.iter().map(|f| f.output.clone()).collect(); - let (transaction, fee) = TransactionBuilder::new() + let mut builder = TransactionBuilder::new() .set_fee_rate(FeeRate::new(fee_per_kb)) .set_current_height(height) .set_special_payload(TransactionPayload::AssetLockPayloadType(AssetLockPayload::new( credit_outputs, - ))) + ))); + if drain { + builder = builder.set_selection_strategy(SelectionStrategy::All); + } + let (transaction, fee) = builder .set_funding(funds_acc, &acc) .require_final_inputs() .build_signed(signer, |addr| funds_acc.address_derivation_path(&addr)) @@ -411,6 +537,149 @@ mod tests { outpoint } + /// CoinJoin-account sibling of [`insert_funded_utxo`]: fund CoinJoin + /// account 0 with a UTXO at a fresh receive address. + fn insert_funded_coinjoin_utxo( + info: &mut ManagedWalletInfo, + wallet: &Wallet, + txid_byte: u8, + value: u64, + is_confirmed: bool, + ) -> OutPoint { + let account_xpub = wallet.get_coinjoin_account(0).unwrap().account_xpub; + let account = info.accounts.coinjoin_accounts.get_mut(&0).unwrap(); + // CoinJoin accounts have no `next_receive_address` wrapper (that is + // Standard-only); draw from the external (mixed-coin) pool directly. + let funding_address = match account.managed_account_type_mut() { + crate::ManagedAccountType::CoinJoin { + external_addresses, + .. + } => external_addresses + .next_unused(&crate::KeySource::Public(account_xpub), true) + .unwrap(), + _ => unreachable!("coinjoin account must carry the CoinJoin managed type"), + }; + let outpoint = OutPoint { + txid: Txid::from_byte_array([txid_byte; 32]), + vout: 0, + }; + let utxo = Utxo { + outpoint, + txout: TxOut { + value, + script_pubkey: funding_address.script_pubkey(), + }, + address: funding_address, + height: 1000, + is_coinbase: false, + is_confirmed, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }; + account.utxos.insert(outpoint, utxo); + outpoint + } + + /// Drain from the CoinJoin account: every CoinJoin UTXO is consumed, the + /// single credit output's value is rewritten to `Σ inputs − fee`, the + /// on-chain burn mirrors it exactly, and there is no change output. + #[tokio::test] + async fn test_drain_coinjoin_asset_lock() { + let (wallet, mut info) = test_wallet_and_info(); + insert_funded_coinjoin_utxo(&mut info, &wallet, 0x41, 400_000, true); + insert_funded_coinjoin_utxo(&mut info, &wallet, 0x42, 300_000, true); + insert_funded_coinjoin_utxo(&mut info, &wallet, 0x43, 300_000, true); + info.update_last_processed_height(1100); + + // The caller's credit amount (0) is ignored — drain rewrites it. + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::CoinJoin { + account_index: 0, + }, + test_credit_outputs(&[0]), + 1000, + true, + ) + .await + .expect("drain from a funded CoinJoin account should succeed"); + + assert_eq!(result.transaction.input.len(), 3, "drain must consume every CoinJoin UTXO"); + assert_eq!(result.transaction.output.len(), 1, "drain emits no change output"); + let burn = &result.transaction.output[0]; + assert!(burn.script_pubkey.is_op_return(), "single output must be the OP_RETURN burn"); + assert_eq!(burn.value, 1_000_000 - result.fee, "burn must carry Σ inputs − fee"); + + let payload = match &result.transaction.special_transaction_payload { + Some(TransactionPayload::AssetLockPayloadType(p)) => p, + other => panic!("expected asset-lock payload, got {:?}", other), + }; + assert_eq!(payload.credit_outputs.len(), 1); + assert_eq!( + payload.credit_outputs[0].value, burn.value, + "payload credit output must mirror the burn value" + ); + + for (i, txin) in result.transaction.input.iter().enumerate() { + assert!(!txin.script_sig.is_empty(), "input {i} not signed"); + } + } + + /// A drain build requires exactly one credit output — anything else is + /// rejected before any wallet state is touched. + #[tokio::test] + async fn test_drain_requires_single_credit_output() { + let (wallet, mut info) = test_wallet_and_info(); + insert_funded_coinjoin_utxo(&mut info, &wallet, 0x44, 500_000, true); + info.update_last_processed_height(1100); + + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::CoinJoin { + account_index: 0, + }, + test_credit_outputs(&[0, 0]), + 1000, + true, + ) + .await; + assert!( + matches!(result, Err(AssetLockError::Builder(BuilderError::InvalidData(_)))), + "two credit outputs must be rejected in drain mode, got: {:?}", + result.err() + ); + } + + /// CoinJoin funding is drain-only: a partial (non-drain) CoinJoin-funded + /// build is rejected before any wallet state is touched (change would + /// need CoinJoin re-denomination, which the builder does not do). + #[tokio::test] + async fn test_coinjoin_funding_requires_drain() { + let (wallet, mut info) = test_wallet_and_info(); + insert_funded_coinjoin_utxo(&mut info, &wallet, 0x45, 1_000_000, true); + info.update_last_processed_height(1100); + + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::CoinJoin { + account_index: 0, + }, + test_credit_outputs(&[200_000]), + 1000, + false, + ) + .await; + assert!( + matches!(result, Err(AssetLockError::Builder(BuilderError::InvalidData(_)))), + "non-drain CoinJoin funding must be rejected, got: {:?}", + result.err() + ); + } + // -- Error type tests -- #[test] @@ -419,7 +688,7 @@ mod tests { AssetLockError::WatchOnlyWallet.to_string(), "Cannot sign with watch-only wallet" ); - assert_eq!(AssetLockError::AccountNotFound(5).to_string(), "BIP44 account 5 not found"); + assert_eq!(AssetLockError::AccountNotFound(5).to_string(), "funding account 5 not found"); assert_eq!(AssetLockError::NoChangeAddress.to_string(), "No change address available"); } @@ -435,15 +704,34 @@ mod tests { #[tokio::test] async fn test_empty_credit_outputs_rejected() { let (wallet, mut info) = test_wallet_and_info(); - let result = info.build_asset_lock(&wallet, 0, vec![], 1000).await; + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, + vec![], + 1000, + false, + ) + .await; assert!(matches!(result, Err(AssetLockError::Builder(BuilderError::NoOutputs)))); } #[tokio::test] async fn test_invalid_account_index() { let (wallet, mut info) = test_wallet_and_info(); - let result = - info.build_asset_lock(&wallet, 99, test_credit_outputs(&[100_000]), 1000).await; + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 99, + }, + test_credit_outputs(&[100_000]), + 1000, + false, + ) + .await; assert!(matches!(result, Err(AssetLockError::AccountNotFound(99)))); } @@ -451,7 +739,17 @@ mod tests { async fn test_insufficient_funds() { // Wallet has no UTXOs, so coin selection should fail let (wallet, mut info) = test_wallet_and_info(); - let result = info.build_asset_lock(&wallet, 0, test_credit_outputs(&[500_000]), 1000).await; + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, + test_credit_outputs(&[500_000]), + 1000, + false, + ) + .await; assert!( matches!(result, Err(AssetLockError::Builder(_))), "Expected Builder error for insufficient funds, got: {:?}", @@ -469,7 +767,17 @@ mod tests { insert_funded_utxo(&mut info, &wallet, 0x11, 1_000_000, false); info.update_last_processed_height(1100); - let result = info.build_asset_lock(&wallet, 0, test_credit_outputs(&[200_000]), 1000).await; + let result = info + .build_asset_lock( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, + test_credit_outputs(&[200_000]), + 1000, + false, + ) + .await; assert!( matches!(result, Err(AssetLockError::Builder(_))), "asset lock must not be built on unconfirmed funds, got: {:?}", @@ -487,7 +795,15 @@ mod tests { info.update_last_processed_height(1100); let result = info - .build_asset_lock(&wallet, 0, test_credit_outputs(&[200_000]), 1000) + .build_asset_lock( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, + test_credit_outputs(&[200_000]), + 1000, + false, + ) .await .expect("confirmed funds should cover the asset lock"); assert!(!result.transaction.input.is_empty()); @@ -605,7 +921,18 @@ mod tests { root, network: Network::Testnet, }; - let result = info.build_asset_lock_with_signer(&wallet, 0, vec![], 1000, &signer).await; + let result = info + .build_asset_lock_with_signer( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, + vec![], + 1000, + false, + &signer, + ) + .await; assert!(matches!(result, Err(AssetLockError::Builder(BuilderError::NoOutputs)))); } @@ -626,9 +953,12 @@ mod tests { let result = info .build_asset_lock_with_signer( &wallet, - 99, + AssetLockFundingAccount::Bip44 { + account_index: 99, + }, test_credit_outputs(&[100_000]), 1000, + false, &signer, ) .await; @@ -663,9 +993,12 @@ mod tests { let result = info .build_asset_lock_with_signer( &wallet, - 0, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, test_credit_outputs(&[100_000]), 1000, + false, &NoDigestSigner, ) .await; @@ -698,7 +1031,16 @@ mod tests { let credit_amounts = [200_000u64, 300_000u64]; let fundings = test_credit_outputs(&credit_amounts); let result = info - .build_asset_lock_with_signer(&wallet, 0, fundings, 1000, &signer) + .build_asset_lock_with_signer( + &wallet, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, + fundings, + 1000, + false, + &signer, + ) .await .expect("build_asset_lock_with_signer should succeed with funded wallet"); @@ -753,9 +1095,12 @@ mod tests { let result = info .build_asset_lock_with_signer( &wallet, - 0, + AssetLockFundingAccount::Bip44 { + account_index: 0, + }, test_credit_outputs(&[500_000]), 1000, + false, &signer, ) .await; diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 4d9743aa3..c1f9677a2 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -361,12 +361,34 @@ impl TransactionBuilder { if self.selection_strategy == SelectionStrategy::All { // Drain: the single output takes the whole balance minus fee (the caller's amount is // ignored); no change. + let drained = total_input.saturating_sub(selection.estimated_fee); + if drained == 0 { + return Err(BuilderError::InsufficientFunds { + available: total_input, + required: selection.estimated_fee, + }); + } let [out] = tx_outputs.as_mut_slice() else { return Err(BuilderError::InvalidData( "SelectionStrategy::All requires exactly one output (the destination)".into(), )); }; - out.value = total_input.saturating_sub(selection.estimated_fee); + out.value = drained; + // An asset-lock drain must also rewrite the payload's credit + // output: the on-chain OP_RETURN burn (`out` above) mirrors the + // payload credit sum, and a mismatch is consensus-invalid. + if let Some(TransactionPayload::AssetLockPayloadType(payload)) = + &mut self.special_payload + { + let [credit] = payload.credit_outputs.as_mut_slice() else { + return Err(BuilderError::InvalidData( + "SelectionStrategy::All with an asset-lock payload requires exactly one \ + credit output" + .into(), + )); + }; + credit.value = drained; + } } else if change_amount > 546 { // Add change output if above dust threshold let Some(change_addr) = self.change_addr else { From 9cbe4e75989bcab7bc32b9989e09c9580f657c0f Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Thu, 30 Jul 2026 09:38:33 -0700 Subject: [PATCH 060/149] fix(key-wallet): rewrite branch-and-bound coin selection (#918) (#919) --- .../managed_wallet_info/coin_selection.rs | 482 ++++++++++++++---- .../transaction_builder.rs | 3 +- 2 files changed, 376 insertions(+), 109 deletions(-) diff --git a/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs b/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs index 7cd72cc2e..e30477116 100644 --- a/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs +++ b/key-wallet/src/wallet/managed_wallet_info/coin_selection.rs @@ -11,6 +11,76 @@ pub(crate) const TX_OUTPUT_SIZE: usize = 34; pub(crate) const TX_INPUT_SIZE: usize = 148; pub(crate) const CHANGE_OUTPUT_SIZE: usize = TX_OUTPUT_SIZE; +/// Nodes the search may visit before giving up and falling back to accumulation. +/// Caps the worst case so no input set can stall a build. Same figure as Bitcoin Core. +const BNB_NODE_BUDGET: u32 = 100_000; + +#[derive(Clone, Copy)] +struct BnbCandidate { + /// Position in the caller's slice, so the winner can be mapped back. + index: usize, + /// Value net of the fee to spend it. + effective_value: u64, +} + +/// Depth-first search over subsets for one that funds the target with no change +/// output, burning as little surplus as possible. +/// +/// Two bounds, because one alone is not enough: abandon a branch that has overshot, +/// *and* one that cannot reach the target even taking every UTXO left (#918). +struct BnbSearch<'a> { + /// Descending effective value, so the bounds bite early. + candidates: &'a [BnbCandidate], + /// `suffix[i]`: effective value still available from `i` onwards. + suffix: &'a [u64], + target: u64, + /// How far above `target` is still acceptable. + window: u64, + budget: u32, + /// Lowest-surplus selection so far, as candidate indices. + best: Option<(Vec, u64)>, +} + +impl BnbSearch<'_> { + fn run(&mut self, index: usize, chosen: &mut Vec, sum: u64) { + if self.budget == 0 { + return; + } + self.budget -= 1; + + if sum >= self.target { + // Adding more to a funded selection only burns more; this branch ends here. + let surplus = sum - self.target; + if surplus <= self.window && self.best.as_ref().is_none_or(|(_, b)| surplus < *b) { + self.best = Some((chosen.clone(), surplus)); + } + return; + } + + // Even taking everything left falls short. + if sum.saturating_add(self.suffix[index]) < self.target { + return; + } + + for i in index..self.candidates.len() { + let next = sum.saturating_add(self.candidates[i].effective_value); + // Overshoots on its own; later candidates are smaller, so skip, don't break. + if next > self.target.saturating_add(self.window) { + continue; + } + + chosen.push(i); + self.run(i + 1, chosen, next); + chosen.pop(); + + // Nothing wasted cannot be improved on. + if self.best.as_ref().is_some_and(|(_, surplus)| *surplus == 0) { + return; + } + } + } +} + /// UTXO selection strategy #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum SelectionStrategy { @@ -116,14 +186,12 @@ impl CoinSelector { { // Default base size assumes 2 outputs: the target output and one change output. let default_base_size = 10 + TX_OUTPUT_SIZE + CHANGE_OUTPUT_SIZE; - let input_size = TX_INPUT_SIZE; self.select_coins_with_size( utxos, target_amount, fee_rate, current_height, default_base_size, - input_size, CHANGE_OUTPUT_SIZE, ) } @@ -132,7 +200,7 @@ impl CoinSelector { /// /// `base_size` includes the change output; `change_output_size` is its byte count (`0` when /// there is no change address), letting the accumulator drop it to size a no-change fee. - #[allow(clippy::too_many_arguments)] + /// Input size is always `TX_INPUT_SIZE`. pub fn select_coins_with_size<'a, I>( &self, utxos: I, @@ -140,7 +208,6 @@ impl CoinSelector { fee_rate: FeeRate, current_height: u32, base_size: usize, - input_size: usize, change_output_size: usize, ) -> Result where @@ -179,7 +246,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } @@ -190,7 +256,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } @@ -207,7 +272,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } else { @@ -225,7 +289,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } @@ -238,7 +301,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } @@ -248,7 +310,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ), _ => unreachable!(), @@ -265,7 +326,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } @@ -278,7 +338,7 @@ impl CoinSelector { } let total_value: u64 = selected.iter().map(|u| u.value()).sum(); - let estimated_size = base_size + selected.len() * input_size; + let estimated_size = base_size + selected.len() * TX_INPUT_SIZE; let estimated_fee = fee_rate.calculate_fee(estimated_size); // The caller's `target_amount` is ignored: a drain spends everything, there is no @@ -306,14 +366,12 @@ impl CoinSelector { /// Accumulate UTXOs. `change_output_size` = the change output's bytes within `base_size` /// (`0` if none is budgeted), shaved off to size the no-change fee. - #[allow(clippy::too_many_arguments)] fn accumulate_coins_with_size<'a, I>( &self, utxos: I, target_amount: u64, fee_rate: FeeRate, base_size: usize, - input_size: usize, change_output_size: usize, ) -> Result where @@ -330,7 +388,7 @@ impl CoinSelector { selected.push(utxo.clone()); // `base_size` budgets a change output; a no-change send omits it and pays less. - let estimated_size = base_size + (input_size * selected.len()); + let estimated_size = base_size + (TX_INPUT_SIZE * selected.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); let size_no_change = estimated_size.saturating_sub(change_output_size); let fee_no_change = fee_rate.calculate_fee(size_no_change); @@ -403,62 +461,126 @@ impl CoinSelector { /// - Pros: Faster to find solutions due to aggressive pruning /// - Cons: May leave small UTXOs unconsolidated, leading to wallet fragmentation /// - Cons: Less likely to find exact matches with larger denominations - #[allow(clippy::too_many_arguments)] fn branch_and_bound_with_size<'a, I>( &self, utxos: I, target_amount: u64, fee_rate: FeeRate, base_size: usize, - input_size: usize, change_output_size: usize, ) -> Result where I: IntoIterator, { - // Collect the UTXOs - they should already be in the right order if needed let sorted_refs: Vec<&'a Utxo> = utxos.into_iter().collect(); - // Try to find an exact match first + // `base_size` budgets a change output, which a change-free send does not pay for. + let base_size_no_change = base_size.saturating_sub(change_output_size); - // Use a simple recursive approach with memoization - let result = self.find_exact_match( - &sorted_refs, - target_amount, - fee_rate, - base_size, - input_size, - 0, - Vec::new(), - 0, - ); - - if let Some((selected, total)) = result { - let estimated_size = base_size + (input_size * selected.len()); - let estimated_fee = fee_rate.calculate_fee(estimated_size); + if let Some(selected) = + self.branch_and_bound(&sorted_refs, target_amount, fee_rate, base_size_no_change) + { + let total_value: u64 = selected.iter().map(|u| u.value()).sum(); + let estimated_size = base_size_no_change + TX_INPUT_SIZE * selected.len(); return Ok(SelectionResult { selected, - total_value: total, + total_value, target_amount, + // The surplus is smaller than a change output would cost (#911). change_amount: 0, estimated_size, - estimated_fee, + estimated_fee: total_value - target_amount, exact_match: true, }); } - // No exact match: accumulate. `base_size` already budgets the change output. + // Nothing change-free found: accumulate. self.accumulate_coins_with_size( sorted_refs, target_amount, fee_rate, base_size, - input_size, change_output_size, ) } + /// Choose UTXOs that fund `target_amount` with **no change output**. + /// + /// `base_size_no_change` is the transaction with no inputs and no change output — + /// the shape being aimed for. Input and change sizes come from [`TX_INPUT_SIZE`] + /// and [`CHANGE_OUTPUT_SIZE`]. + /// + /// `None` when nothing lands close enough, or the node budget runs out. + fn branch_and_bound( + &self, + utxos: &[&Utxo], + target_amount: u64, + fee_rate: FeeRate, + base_size_no_change: usize, + ) -> Option> { + // Effective values — value net of the fee to spend it — make the acceptance + // test independent of how many inputs are chosen. Sizing the fee from a + // guessed input count is what produced the old off-by-one. + let input_fee = fee_rate.calculate_fee(TX_INPUT_SIZE); + let mut candidates: Vec = utxos + .iter() + .enumerate() + .filter_map(|(index, utxo)| { + // Worth no more than its own input costs: never helps. + utxo.value().checked_sub(input_fee).filter(|ev| *ev > 0).map(|effective_value| { + BnbCandidate { + index, + effective_value, + } + }) + }) + .collect(); + candidates.sort_by_key(|c| Reverse(c.effective_value)); + + // What the effective values must cover. + let target = target_amount.saturating_add(fee_rate.calculate_fee(base_size_no_change)); + let window = self.cost_of_change(fee_rate); + + // Lets the search abandon a branch that can no longer reach the target. + let mut suffix = vec![0u64; candidates.len() + 1]; + for i in (0..candidates.len()).rev() { + suffix[i] = suffix[i + 1].saturating_add(candidates[i].effective_value); + } + + let mut search = BnbSearch { + candidates: &candidates, + suffix: &suffix, + target, + window, + budget: BNB_NODE_BUDGET, + best: None, + }; + search.run(0, &mut Vec::new(), 0); + + let (chosen, _) = search.best?; + let selected: Vec = + chosen.iter().map(|&c| utxos[candidates[c].index].clone()).collect(); + + // Effective values round each input's fee up on its own; the real transaction + // rounds once over its whole size. Re-check against the real fee. + let total_value: u64 = selected.iter().map(|u| u.value()).sum(); + let size = base_size_no_change + TX_INPUT_SIZE * selected.len(); + let required = target_amount.saturating_add(fee_rate.calculate_fee(size)); + (total_value >= required && total_value - required <= window).then_some(selected) + } + + /// What a change output costs: creating it now plus spending it later. A surplus + /// smaller than this is cheaper given to the miner than returned, which is what + /// makes a near-exact match as good as an exact one. + /// + /// Floored at the dust threshold: below that it could not be change anyway (#911). + fn cost_of_change(&self, fee_rate: FeeRate) -> u64 { + let create = fee_rate.calculate_fee(CHANGE_OUTPUT_SIZE); + let spend_later = fee_rate.calculate_fee(TX_INPUT_SIZE); + create.saturating_add(spend_later).max(self.dust_threshold) + } + /// Optimal consolidation strategy with custom sizes /// Tries to find combinations that either: /// 1. Match exactly (no change needed) @@ -482,14 +604,12 @@ impl CoinSelector { /// - During low-fee periods when consolidation is cheaper /// - For wallets that receive many small payments /// - When exact change is preferred to minimize privacy leaks - #[allow(clippy::too_many_arguments)] fn optimal_consolidation_with_size<'a>( &self, utxos: &[&'a Utxo], target_amount: u64, fee_rate: FeeRate, base_size: usize, - input_size: usize, change_output_size: usize, ) -> Result { // First, try to find an exact match using smaller UTXOs @@ -506,10 +626,9 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, max_inputs, ) { - let estimated_size = base_size + (input_size * combination.len()); + let estimated_size = base_size + (TX_INPUT_SIZE * combination.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); return Ok(SelectionResult { @@ -538,7 +657,7 @@ impl CoinSelector { current_total += utxo.value(); } - let estimated_size = base_size + (input_size * current.len()); + let estimated_size = base_size + (TX_INPUT_SIZE * current.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); let required = target_amount + estimated_fee; @@ -552,7 +671,7 @@ impl CoinSelector { } if let Some(selected) = best_selection { - let estimated_size = base_size + (input_size * selected.len()); + let estimated_size = base_size + (TX_INPUT_SIZE * selected.len()); let estimated_fee = fee_rate.calculate_fee(estimated_size); let total_value: u64 = selected.iter().map(|u| u.value()).sum(); @@ -573,7 +692,6 @@ impl CoinSelector { target_amount, fee_rate, base_size, - input_size, change_output_size, ) } @@ -585,14 +703,13 @@ impl CoinSelector { target: u64, fee_rate: FeeRate, base_size: usize, - input_size: usize, max_inputs: usize, ) -> Option> { // Simple subset sum solver for exact matches // This is a simplified version - could be optimized with dynamic programming for num_inputs in 1..=max_inputs.min(utxos.len()) { - let estimated_size = base_size + (input_size * num_inputs); + let estimated_size = base_size + (TX_INPUT_SIZE * num_inputs); let estimated_fee = fee_rate.calculate_fee(estimated_size); let required = target + estimated_fee; @@ -647,64 +764,6 @@ impl CoinSelector { None } - - /// Recursive helper for finding exact match - #[allow(clippy::too_many_arguments)] - fn find_exact_match( - &self, - utxos: &[&Utxo], - target: u64, - fee_rate: FeeRate, - base_size: usize, - input_size: usize, - index: usize, - mut current: Vec, - current_total: u64, - ) -> Option<(Vec, u64)> { - // Calculate required amount including fee - let estimated_size = base_size + (input_size * (current.len() + 1)); - let estimated_fee = fee_rate.calculate_fee(estimated_size); - let required = target + estimated_fee; - - // Check if we've found an exact match - if current_total == required { - return Some((current, current_total)); - } - - // Prune if we've exceeded the target - if current_total > required + self.dust_threshold { - return None; - } - - // Try remaining UTXOs - for i in index..utxos.len() { - let new_total = current_total + utxos[i].value(); - - // Skip if this would exceed our target by too much - if new_total > required + self.dust_threshold * 10 { - continue; - } - - current.push(utxos[i].clone()); - - if let Some(result) = self.find_exact_match( - utxos, - target, - fee_rate, - base_size, - input_size, - i + 1, - current.clone(), - new_total, - ) { - return Some(result); - } - - current.pop(); - } - - None - } } /// Errors that can occur during coin selection @@ -902,7 +961,6 @@ mod tests { FeeRate::normal(), 200, base_size, - 148, 0, // no change address => no change output to drop ); assert!( @@ -926,7 +984,7 @@ mod tests { let target = 100_000; let base_size = 10 + 34; // one (target) output, no change output budgeted let selection = CoinSelector::new(strategy) - .select_coins_with_size(&utxos, target, FeeRate::normal(), 200, base_size, 148, 0) + .select_coins_with_size(&utxos, target, FeeRate::normal(), 200, base_size, 0) .unwrap_or_else(|e| panic!("{strategy:?}: {e:?}")); // No-change fee is 192 (44 + 148); the ~9.9M remainder is change, not fee. @@ -968,4 +1026,214 @@ mod tests { let has_small_utxos = selected_values.iter().any(|&v| v <= 500); assert!(has_small_utxos, "Should include at least one small UTXO for consolidation"); } + + /// Size of a no-change transaction with `n` inputs. + fn no_change_size(n: usize) -> usize { + (10 + TX_OUTPUT_SIZE + CHANGE_OUTPUT_SIZE) - CHANGE_OUTPUT_SIZE + TX_INPUT_SIZE * n + } + + /// The point of the strategy: fund the payment with no change output at all. + #[test] + fn test_branch_and_bound_finds_an_exact_match() { + let selector = CoinSelector::new(SelectionStrategy::BranchAndBound); + let fee_rate = FeeRate::new(1000); + let utxos = vec![ + Utxo::dummy(0, 40_000, 100, false, true), + Utxo::dummy(0, 30_000, 100, false, true), + Utxo::dummy(0, 10_000, 100, false, true), + ]; + + // Sized from the inputs actually chosen. Before the rewrite this needed the + // fee for *three* inputs, because the search sized it as `current.len() + 1`. + let fee = fee_rate.calculate_fee(no_change_size(2)); + let target = 50_000 - fee; + + let result = selector.select_coins(&utxos, target, fee_rate, 200).unwrap(); + + assert!(result.exact_match); + assert_eq!(result.change_amount, 0, "an exact match needs no change output"); + assert_eq!(result.total_value, 50_000); + assert_eq!(result.selected.len(), 2); + assert_eq!(result.estimated_fee, fee, "nothing is burned beyond the real fee"); + } + + /// A surplus smaller than a change output would cost is taken anyway, folded into + /// the fee. Demanding the exact satoshi would reject it for a costlier transaction. + #[test] + fn test_branch_and_bound_prefers_a_near_exact_match_over_change() { + let selector = CoinSelector::new(SelectionStrategy::BranchAndBound); + let fee_rate = FeeRate::new(1000); + let utxos = vec![ + Utxo::dummy(0, 40_000, 100, false, true), + Utxo::dummy(0, 30_000, 100, false, true), + Utxo::dummy(0, 10_000, 100, false, true), + ]; + + let fee = fee_rate.calculate_fee(no_change_size(2)); + let surplus = 300; // under the dust threshold, so it could never be change + let target = 50_000 - fee - surplus; + + let result = selector.select_coins(&utxos, target, fee_rate, 200).unwrap(); + + assert_eq!(result.total_value, 50_000); + assert_eq!(result.change_amount, 0, "the surplus is not worth a change output"); + assert_eq!(result.estimated_fee, fee + surplus, "the surplus goes to the fee, not lost"); + } + + /// The mirror image: a surplus worth keeping must come back, not be burned. + #[test] + fn test_branch_and_bound_returns_change_when_the_surplus_is_worth_keeping() { + let selector = CoinSelector::new(SelectionStrategy::BranchAndBound); + let utxos = vec![Utxo::dummy(0, 60_000, 100, false, true)]; + + let result = selector.select_coins(&utxos, 50_000, FeeRate::new(1000), 200).unwrap(); + + assert!(result.change_amount > 0, "a large surplus must come back as change"); + assert!(!result.exact_match); + } + + /// A UTXO worth no more than the fee to spend it contributes nothing. + #[test] + fn test_branch_and_bound_skips_uneconomic_utxos() { + let selector = CoinSelector::new(SelectionStrategy::BranchAndBound); + let fee_rate = FeeRate::new(1000); + // An input costs 148 sat at this rate, so 120 is uneconomic. + let utxos = vec![ + Utxo::dummy(0, 40_000, 100, false, true), + Utxo::dummy(0, 10_000, 100, false, true), + Utxo::dummy(0, 120, 100, false, true), + ]; + let target = 50_000 - fee_rate.calculate_fee(no_change_size(2)); + + let result = selector.select_coins(&utxos, target, fee_rate, 200).unwrap(); + + assert!( + !result.selected.iter().any(|u| u.value() == 120), + "an input that costs more than it brings must not be selected" + ); + } + + /// Issue #918: a near-total target defeated the only bound the search had + /// (overshoot) — ~3s at 22 UTXOs, non-terminating past 32. + #[test] + fn test_branch_and_bound_near_total_balance_is_bounded() { + let selector = CoinSelector::new(SelectionStrategy::BranchAndBound); + + for n in [24usize, 32, 48, 64] { + let utxos: Vec = (0..n) + .map(|i| Utxo::dummy(0, 100_000 + (i as u64) * 7_919, 100, false, true)) + .collect(); + let total: u64 = utxos.iter().map(|u| u.value()).sum(); + + let started = std::time::Instant::now(); + let result = selector + .select_coins(&utxos, total - 50_000, FeeRate::new(1000), 200) + .expect("a near-total target is still fundable"); + let elapsed = started.elapsed(); + + assert!(result.total_value >= total - 50_000); + // Generous on purpose: this only has to catch a return to unbounded + // search, not police runner speed. The node-count test below is what + // actually pins the bound. + assert!( + elapsed < std::time::Duration::from_secs(30), + "selection over {} UTXOs took {:?}; the search is unbounded again", + n, + elapsed + ); + } + } + + /// A low target leaves a huge feasible space; the node budget bounds it. + #[test] + fn test_branch_and_bound_mid_range_target_is_bounded() { + let selector = CoinSelector::new(SelectionStrategy::BranchAndBound); + let utxos: Vec = (0..64) + .map(|i| Utxo::dummy(0, 100_000 + (i as u64) * 7_919, 100, false, true)) + .collect(); + let total: u64 = utxos.iter().map(|u| u.value()).sum(); + + let started = std::time::Instant::now(); + let result = selector.select_coins(&utxos, total / 3, FeeRate::new(1000), 200).unwrap(); + let elapsed = started.elapsed(); + + assert!(result.total_value >= total / 3); + assert!( + elapsed < std::time::Duration::from_secs(30), + "selection took {:?}; the node budget is not bounding the search", + elapsed + ); + } + + /// What actually fixes #918 is the feasibility bound, not the node budget: with + /// a near-total target the search settles in a handful of nodes instead of + /// walking 2^N. Asserted on nodes visited, so it does not depend on machine speed. + #[test] + fn test_feasibility_bound_settles_a_near_total_target_in_few_nodes() { + let fee_rate = FeeRate::new(1000); + let values: Vec = (0..32u64).map(|i| 100_000 + i * 7_919).collect(); + let total: u64 = values.iter().sum(); + + let (candidates, suffix) = bnb_setup(&values, fee_rate); + let visited = run_bnb(&candidates, &suffix, total - 50_000); + assert!( + visited < 1_000, + "a near-total target should be bounded in a few nodes, visited {}", + visited + ); + } + + /// Candidates and suffix sums as `branch_and_bound` builds them, so a test can + /// drive the search directly and count what it costs. + fn bnb_setup(values: &[u64], fee_rate: FeeRate) -> (Vec, Vec) { + let input_fee = fee_rate.calculate_fee(TX_INPUT_SIZE); + let mut candidates: Vec = values + .iter() + .enumerate() + .map(|(index, v)| BnbCandidate { + index, + effective_value: v - input_fee, + }) + .collect(); + candidates.sort_by_key(|c| Reverse(c.effective_value)); + + let mut suffix = vec![0u64; candidates.len() + 1]; + for i in (0..candidates.len()).rev() { + suffix[i] = suffix[i + 1] + candidates[i].effective_value; + } + (candidates, suffix) + } + + /// Run the search over a full budget and report the nodes it visited. + fn run_bnb(candidates: &[BnbCandidate], suffix: &[u64], target: u64) -> u32 { + let mut search = BnbSearch { + candidates, + suffix, + target, + window: 546, + budget: BNB_NODE_BUDGET, + best: None, + }; + search.run(0, &mut Vec::new(), 0); + BNB_NODE_BUDGET - search.budget + } + + /// The mirror of the near-total case: a low target leaves a huge feasible space + /// with no exact match in it, so the bounds cannot settle it and the *budget* is + /// what has to stop the search. Asserted on nodes visited rather than the clock. + #[test] + fn test_node_budget_stops_a_mid_range_search() { + let fee_rate = FeeRate::new(1000); + let values: Vec = (0..64u64).map(|i| 100_000 + i * 7_919).collect(); + let total: u64 = values.iter().sum(); + + let (candidates, suffix) = bnb_setup(&values, fee_rate); + let visited = run_bnb(&candidates, &suffix, total / 3); + + assert_eq!( + visited, BNB_NODE_BUDGET, + "the budget should be the binding constraint here; if the bounds now settle \ + this case in {visited} nodes the budget may be sized wrong" + ); + } } diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index c1f9677a2..747f23e8a 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -6,7 +6,7 @@ use crate::managed_account::reservation::ReservationSet; use crate::managed_account::ManagedCoreFundsAccount; use crate::wallet::managed_wallet_info::coin_selection::{ - CoinSelector, SelectionStrategy, CHANGE_OUTPUT_SIZE, TX_INPUT_SIZE, TX_OUTPUT_SIZE, + CoinSelector, SelectionStrategy, CHANGE_OUTPUT_SIZE, TX_OUTPUT_SIZE, }; use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::{Account, DerivationPath, Signer, Utxo, Wallet}; @@ -325,7 +325,6 @@ impl TransactionBuilder { self.fee_rate, self.current_height, self.calculate_base_size(), - TX_INPUT_SIZE, change_output_size, ) .map_err(BuilderError::CoinSelection)?; From 8f78baa6b7979b9bea56501ad75b5a7b7150a711 Mon Sep 17 00:00:00 2001 From: bfoss765 <38437574+bfoss765@users.noreply.github.com> Date: Wed, 5 Aug 2026 01:09:02 -0400 Subject: [PATCH 061/149] feat(key-wallet): owner-tagged reservations to close the broadcast-release TOCTOU (platform#4185) (#916) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet): owner-tagged reservations + release_if_owner to close the broadcast-release TOCTOU (platform#4185) ReservationSet previously mapped OutPoint -> reserved-at-height with no per-reservation owner, and released unconditionally by outpoint. The platform broadcast path (dashpay/platform#4185) reserves an asset-lock/deferred send's inputs, awaits the broadcast, and on a Rejected result releases them by outpoint. If key-wallet's TTL sweep reclaims that reservation mid-await and a different concurrent build re-reserves the same outpoint (same wallet generation, so any Arc::ptr_eq generation guard still passes), the rejected build's cleanup frees the OTHER build's reservation -> a double-spend window. The platform layer cannot detect this because the sweep is invisible inside key-wallet; the "release only if still mine" check must be atomic under the ReservationSet mutex. Stamp every reservation with a ReservationToken (a Copy newtype over a monotonic per-set counter, unique even across same-height reserves). reserve() returns the token it stamped; release_if_owner(outpoints, token) removes only outpoints still owned by that token, atomically under the mutex, and is a no-op for any swept-and-re-reserved outpoint. The unconditional release() is kept for the processed-spend path, where the coin is genuinely spent and must leave the set regardless of owner. Wiring: - assemble_unsigned returns the stamped token; build_signed's sign-failure path now releases owner-guarded (it too awaits, so it shares the hazard). - Additive token-returning builds: build_unsigned_reserved / build_signed_reserved. build_unsigned / build_signed keep their signatures. - Public seam for platform: ManagedCoreFundsAccount::release_reservation_if_owner and AssetLockResult::reservation_token; ReservationToken re-exported. Co-Authored-By: Claude Fable 5 * style: cargo fmt Co-Authored-By: Claude Fable 5 * fix(key-wallet): release owner-tagged reservation on asset-lock downstream failures Addresses the CodeRabbit Major (PR #916, asset_lock_builder.rs :219-252 / :293-357). Both asset-lock builders took a reservation via build_signed_reserved and captured its reservation_token, but a failure in the code that runs AFTER the successful signed build — credit-key derivation in build_asset_lock, and the Phase 1-3 signer/bookkeeping loop in build_asset_lock_with_signer — returned Err without releasing the reservation. The already-signed transaction's inputs then stranded until the 24-block TTL sweep, since the caller never received the token to release them itself. Clone the funding account's ReservationSet (a shared Arc view) before each loop re-borrows self.accounts, and on any failure release only THIS build's reservation with the owner-guarded release_if_owner(&reserved, token) — never the unconditional release, which could free a concurrent build's inputs that the TTL sweep + re-reserve handed over during this build (the very TOCTOU of dashpay/platform#4185 this PR closes). The success path is unchanged: a successful build consumes the reservation as designed, and the signer-failure release inside build_signed_reserved returns via `?` before this new code runs, so there is no double-release. Co-Authored-By: Claude Fable 5 * feat(sml): masternodes_by_voting_key lookup on MasternodeList Adds a by-voting-key lookup (returns matching pro_reg_tx_hashes) so SDK consumers can resolve which masternodes an imported voting key controls without the legacy dashj masternode list — needed for contested-username voting after the dashj engine is held post-cutover. Data was already present on every entry (key_id_voting, pro_reg_tx_hash); this is the missing index/helper plus a unit test covering multi-match, single-match and no-match. Co-Authored-By: Claude Opus 4.8 * docs(sml): guarantee ascending ProTxHash order for masternodes_by_voting_key Addresses the one unresolved CodeRabbit review thread on PR #916 (masternode_helpers.rs:116-121): the function returned matches in `BTreeMap` order but the public docs never promised an order, and the test hid that by sorting the result before asserting on it. Document the ordering as a guaranteed part of the API rather than an implementation accident, and say why it holds: `masternodes` is a `BTreeMap` (std guarantees ascending-key iteration) and `ProTxHash` derives `Ord` over its 32 internal bytes. Also note that because `ProTxHash` is `#[hash_newtype(forward)]`, the internal byte order is the same order its hex `Display` reads in — so the sequence is sorted the way it prints, which is the non-obvious part for a codebase that reverses byte order on several other hash types. Test changes: - `masternodes_by_voting_key_filters_and_collects` no longer sorts the result. It asserts the whole `Vec` at once, so it now pins ordering as well as contents. - New `masternodes_by_voting_key_returns_ascending_pro_tx_hash_order` seeds the list in DESCENDING proTxHash order, so a result that merely echoed insertion order would come back reversed and fail. It also asserts strict ascent through the public `Ord` impl, which is the comparison callers would actually use. The ordering claim is verified by the new test, not assumed. Also fixes a `cargo fmt` violation in this file's `list_from` test helper that was already present on the branch (`cargo fmt --all -- --check` was failing before this commit and passes after). cargo fmt --all -- --check -> clean cargo test -p dashcore --lib -> 577 passed, 0 failed Co-Authored-By: Claude Opus 4.8 * chore(sml): drop unrelated masternode_helpers changes from this PR The `masternodes_by_voting_key` lookup and its tests are independent of the owner-tagged reservation work and only made this PR harder to review. Revert the file to its `dev` state; the lookup can land in its own PR. Addresses review comment on masternode_helpers.rs (scope). Co-Authored-By: Claude Opus 4.8 * refactor(key-wallet): address reservation review feedback - Remove the `build_unsigned` convenience wrapper. It reserved the selected inputs but discarded the ReservationToken, leaving callers no way to do an owner-guarded release and inviting the exact misuse the token exists to prevent. It had no production callers; the only users were tests, now moved to `build_unsigned_reserved`. - `ReservationSet::release_if_owner`: replace the two-lookup (get-then-remove) per outpoint with a single Entry-API lookup ("remove only if still mine"), preserving exact semantics. - Consolidate the platform#4185 ownership rationale. It was restated in full in several places that would drift apart; keep the single canonical explanation on `ReservationSet::release_if_owner` and have the module doc, the asset-lock builder, the AssetLockResult token field, and `release_reservation_if_owner` point to it. Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Claude Fable 5 --- key-wallet/src/lib.rs | 1 + .../managed_core_funds_account.rs | 21 +- key-wallet/src/managed_account/mod.rs | 1 + key-wallet/src/managed_account/reservation.rs | 233 ++++++++++++++++-- .../managed_wallet_info/asset_lock_builder.rs | 175 ++++++++----- .../transaction_builder.rs | 118 ++++++--- .../transaction_building.rs | 18 +- 7 files changed, 453 insertions(+), 114 deletions(-) diff --git a/key-wallet/src/lib.rs b/key-wallet/src/lib.rs index 89b64b949..b777b8b9d 100644 --- a/key-wallet/src/lib.rs +++ b/key-wallet/src/lib.rs @@ -60,6 +60,7 @@ pub use managed_account::address_pool::{AddressInfo, AddressPool, KeySource, Poo pub use managed_account::managed_account_type::ManagedAccountType; pub use managed_account::managed_platform_account::ManagedPlatformAccount; pub use managed_account::platform_address::PlatformP2PKHAddress; +pub use managed_account::ReservationToken; pub use mnemonic::{Language, Mnemonic}; pub use seed::Seed; pub use signer::{ExtendedPubKeySigner, Signer, SignerMethod, TransactionCategory}; diff --git a/key-wallet/src/managed_account/managed_core_funds_account.rs b/key-wallet/src/managed_account/managed_core_funds_account.rs index 1f7001f67..2418152dc 100644 --- a/key-wallet/src/managed_account/managed_core_funds_account.rs +++ b/key-wallet/src/managed_account/managed_core_funds_account.rs @@ -19,7 +19,7 @@ use crate::managed_account::address_pool; use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::managed_account::managed_account_type::ManagedAccountType; use crate::managed_account::managed_core_keys_account::ManagedCoreKeysAccount; -use crate::managed_account::reservation::ReservationSet; +use crate::managed_account::reservation::{ReservationSet, ReservationToken}; use crate::managed_account::transaction_record::{ InputDetail, OutputDetail, OutputRole, TransactionDirection, }; @@ -130,6 +130,25 @@ impl ManagedCoreFundsAccount { self.reservations.release(tx.input.iter().map(|input| &input.previous_output)); } + /// Owner-guarded release of `tx`'s input reservations: releases an input + /// only if it is *still owned by* `token`, the [`ReservationToken`] returned + /// when this build reserved its inputs (from + /// [`build_unsigned_reserved`]/[`build_signed_reserved`]). + /// + /// This is the release a caller must use when it abandons a transaction + /// *after having `.await`ed something* between reserving and releasing — + /// above all the platform broadcast path — never the unconditional + /// [`Self::release_reservation`]. See `ReservationSet::release_if_owner` for + /// the release/re-reserve race this closes and why the owner check must live + /// inside key-wallet (`dashpay/platform#4185`). + /// + /// [`build_unsigned_reserved`]: crate::wallet::managed_wallet_info::transaction_builder::TransactionBuilder::build_unsigned_reserved + /// [`build_signed_reserved`]: crate::wallet::managed_wallet_info::transaction_builder::TransactionBuilder::build_signed_reserved + pub fn release_reservation_if_owner(&self, tx: &Transaction, token: ReservationToken) { + let outpoints: Vec = tx.input.iter().map(|input| input.previous_output).collect(); + self.reservations.release_if_owner(&outpoints, token); + } + /// Get a reference to the inner keys-account state. pub fn keys(&self) -> &ManagedCoreKeysAccount { &self.keys diff --git a/key-wallet/src/managed_account/mod.rs b/key-wallet/src/managed_account/mod.rs index 5d1766c5f..7a5b5daff 100644 --- a/key-wallet/src/managed_account/mod.rs +++ b/key-wallet/src/managed_account/mod.rs @@ -27,3 +27,4 @@ pub mod transaction_record; pub use managed_account_ref::{ManagedAccountRef, ManagedAccountRefMut, OwnedManagedCoreAccount}; pub use managed_core_funds_account::ManagedCoreFundsAccount; pub use managed_core_keys_account::ManagedCoreKeysAccount; +pub use reservation::ReservationToken; diff --git a/key-wallet/src/managed_account/reservation.rs b/key-wallet/src/managed_account/reservation.rs index db6fd9adf..1715c5fd4 100644 --- a/key-wallet/src/managed_account/reservation.rs +++ b/key-wallet/src/managed_account/reservation.rs @@ -12,6 +12,16 @@ //! held survives the lock being released and is observed by the next build. It //! is never persisted: after a restart it is empty, where chain and mempool //! sync are the source of truth for which coins are already spent. +//! +//! # Why reservations carry an owner token +//! +//! Releasing a reservation by outpoint alone is unsafe once *releasing* and +//! *re-reserving* can interleave. Every reservation is therefore stamped with a +//! [`ReservationToken`] identifying the build that made it, and the +//! abandon/rejected-broadcast release path ([`ReservationSet::release_if_owner`]) +//! removes an outpoint only if it is *still owned by the releasing build*. See +//! [`ReservationSet::release_if_owner`] for the concrete hazard this closes and +//! why the check must be atomic under this set's mutex (`dashpay/platform#4185`). use std::collections::{HashMap, HashSet}; use std::sync::{Arc, Mutex, MutexGuard}; @@ -31,24 +41,68 @@ use dashcore::blockdata::transaction::OutPoint; /// the very double-spend this guards against. const RESERVATION_TTL_BLOCKS: u32 = 24; +/// Opaque, per-`reserve`-call identity stamped onto every outpoint that call +/// reserves. +/// +/// A token is the *proof of ownership* a build presents to the reservation +/// set's `release_if_owner` so a release only removes the build's own +/// reservation. Each `reserve` call mints a fresh token from a monotonic +/// counter, so two reservations never share one — not even two reservations +/// taken at the same block height, which is why the height (which collides +/// freely) cannot serve as the identity. +/// +/// The inner counter is private and there is no public constructor: a token can +/// only originate from a real `reserve` call. That is deliberate — it prevents a +/// caller from forging a token that happens to match another build's ownership +/// and releasing inputs out from under it. +/// +/// Copy semantics let a build hold its token cheaply across an `.await` (e.g. +/// the platform broadcast path in `dashpay/platform#4185`) and present it again +/// when releasing. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct ReservationToken(u64); + +/// A single reserved outpoint: when it was reserved (for the TTL backstop) and +/// which build owns it (for owner-guarded release). +#[derive(Debug, Clone, Copy)] +struct Reservation { + /// Block height at which the reservation was taken; drives the TTL sweep. + reserved_at_height: u32, + /// The build that reserved this outpoint. Only this owner may release it via + /// [`ReservationSet::release_if_owner`]. + owner: ReservationToken, +} + +/// Mutex-guarded interior: the reservations keyed by outpoint plus the counter +/// that mints the next [`ReservationToken`]. +#[derive(Debug, Default)] +struct Reserved { + entries: HashMap, + /// Monotonically increasing source of unique tokens. Never persisted and + /// only ever incremented, so within a process every issued token is unique. + /// (Wraparound would need ~2^64 reserves in one process lifetime, which is + /// unreachable in practice.) + next_token: u64, +} + /// Ephemeral, in-memory set of reserved outpoints. Cloning shares the /// underlying state, which is what lets a build's reservation outlive the /// wallet lock and be seen by the next build. #[derive(Debug, Clone, Default)] pub(crate) struct ReservationSet { - inner: Arc>>, + inner: Arc>, } impl ReservationSet { /// Recovers from a poisoned mutex rather than panicking: the guarded data is - /// a plain `HashMap` with no invariant a partial write could break, and - /// panicking here would strand all later coin selection in a long-running - /// node. - fn lock(&self) -> MutexGuard<'_, HashMap> { + /// a plain map plus a counter with no invariant a partial write could break, + /// and panicking here would strand all later coin selection in a + /// long-running node. + fn lock(&self) -> MutexGuard<'_, Reserved> { self.inner.lock().unwrap_or_else(|poisoned| poisoned.into_inner()) } - fn sweep(reserved: &mut HashMap, current_height: u32) { + fn sweep(reserved: &mut Reserved, current_height: u32) { // Height 0 means the wallet has no processed height yet, so the elapsed // span is unknown and no entry can be reliably judged stale. An entry // stamped at height 0 therefore relies on a later non-zero height being @@ -56,34 +110,101 @@ impl ReservationSet { if current_height == 0 { return; } - reserved.retain(|_, reserved_at| { - current_height.saturating_sub(*reserved_at) < RESERVATION_TTL_BLOCKS + reserved.entries.retain(|_, reservation| { + current_height.saturating_sub(reservation.reserved_at_height) < RESERVATION_TTL_BLOCKS }); } /// Reserve `outpoints` as of `current_height`, dropping expired entries - /// first. Re-reserving an outpoint refreshes its height. - pub(crate) fn reserve(&self, outpoints: &[OutPoint], current_height: u32) { + /// first, and return the [`ReservationToken`] stamped onto all of them. + /// + /// Every outpoint in a single call shares the one returned token: the caller + /// keeps it and later presents it to [`Self::release_if_owner`] to release + /// only what this call reserved. Re-reserving an outpoint (a later `reserve` + /// naming it again) refreshes its height *and* transfers ownership to the new + /// token — the previous owner's [`Self::release_if_owner`] then becomes a + /// no-op for it, which is precisely the behavior that closes the + /// release/re-reserve race described in the module docs (`platform#4185`). + pub(crate) fn reserve(&self, outpoints: &[OutPoint], current_height: u32) -> ReservationToken { let mut reserved = self.lock(); Self::sweep(&mut reserved, current_height); + + let owner = ReservationToken(reserved.next_token); + // Increment even on an empty `outpoints` slice so a token is never + // reissued; wrapping is documented-unreachable but avoids a debug panic. + reserved.next_token = reserved.next_token.wrapping_add(1); + for outpoint in outpoints { - reserved.insert(*outpoint, current_height); + reserved.entries.insert( + *outpoint, + Reservation { + reserved_at_height: current_height, + owner, + }, + ); } + owner } /// Return the currently reserved outpoints, dropping expired entries first. pub(crate) fn reserved(&self, current_height: u32) -> HashSet { let mut reserved = self.lock(); Self::sweep(&mut reserved, current_height); - reserved.keys().copied().collect() + reserved.entries.keys().copied().collect() } - /// Release the given outpoints. Idempotent: releasing an outpoint that is - /// not reserved does nothing. + /// Unconditionally release the given outpoints, regardless of owner. + /// Idempotent: releasing an outpoint that is not reserved does nothing. + /// + /// This is correct only where the coins are *known spent* and so must leave + /// the set no matter which build reserved them — e.g. when a processed spend + /// hands its inputs from this ephemeral set to the durable spent set. A + /// caller that is merely abandoning an in-flight build (rejected broadcast, + /// cancelled send) must use [`Self::release_if_owner`] instead, so it cannot + /// free a reservation another build has since taken over. pub(crate) fn release<'a>(&self, outpoints: impl IntoIterator) { let mut reserved = self.lock(); for outpoint in outpoints { - reserved.remove(outpoint); + reserved.entries.remove(outpoint); + } + } + + /// Release only the `outpoints` still owned by `token`, atomically under this + /// set's mutex. An outpoint reserved by a different owner — because the TTL + /// sweep reclaimed this build's reservation and another build re-reserved it + /// meanwhile — is left untouched. + /// + /// This is the owner-guarded counterpart to [`Self::release`] and the whole + /// reason reservations carry a [`ReservationToken`]; it is the canonical + /// explanation the rest of the reservation machinery points back to. It is + /// the release a build must use when it abandons an in-flight transaction + /// after having `.await`ed something (a broadcast, an external signer): + /// during that await the TTL sweep can reclaim this build's reservation and a + /// *different* concurrent build can re-reserve the very same outpoint under a + /// new token (same wallet generation, so any `Arc::ptr_eq` generation guard + /// still matches). An unconditional release-by-outpoint would then free that + /// other build's inputs, letting coin selection hand them to a second + /// transaction — a double-spend window (`dashpay/platform#4185`). The + /// platform layer cannot detect this because the sweep happens inside + /// key-wallet invisibly; because the ownership check and the removal happen + /// together while the mutex is held, no sweep or re-reserve can interleave + /// between them. + /// + /// Idempotent and a no-op for any outpoint that is unreserved or owned by a + /// different token — including one this build's own reservation already lost + /// to a sweep, so a late release after reclamation is harmless. + pub(crate) fn release_if_owner(&self, outpoints: &[OutPoint], token: ReservationToken) { + use std::collections::hash_map::Entry; + let mut reserved = self.lock(); + for outpoint in outpoints { + // Single hash lookup per outpoint: the `Entry` locates the slot once, + // and `remove` reuses it — "remove only if still mine", no second + // `get` before the `remove`. + if let Entry::Occupied(entry) = reserved.entries.entry(*outpoint) { + if entry.get().owner == token { + entry.remove(); + } + } } } } @@ -175,4 +296,86 @@ mod tests { set.reserve(&[a], 1); assert!(clone.reserved(1).contains(&a)); } + + #[test] + fn each_reserve_call_mints_a_distinct_token() { + let set = ReservationSet::default(); + // Two reserves at the SAME height must still get different tokens — the + // whole point of not keying ownership on height, which collides. + let token_a = set.reserve(&[outpoint(0x10, 0)], 100); + let token_b = set.reserve(&[outpoint(0x11, 0)], 100); + assert_ne!(token_a, token_b); + } + + #[test] + fn release_if_owner_releases_only_the_owning_builds_outpoints() { + let set = ReservationSet::default(); + let mine = outpoint(0x20, 0); + let theirs = outpoint(0x21, 0); + + let my_token = set.reserve(&[mine], 100); + let _their_token = set.reserve(&[theirs], 100); + + // Releasing with my token frees only my outpoint; theirs is untouched. + set.release_if_owner(&[mine, theirs], my_token); + assert!(!set.reserved(100).contains(&mine)); + assert!(set.reserved(100).contains(&theirs)); + } + + #[test] + fn release_if_owner_is_a_noop_for_unreserved_or_wrong_token() { + let set = ReservationSet::default(); + let a = outpoint(0x22, 0); + let stale_token = set.reserve(&[a], 100); + + // Simulate the outpoint being released and re-reserved by someone else. + set.release([&a]); + let _new_token = set.reserve(&[a], 100); + + // The stale token no longer owns `a`, so its release must not remove it. + set.release_if_owner(&[a], stale_token); + assert!(set.reserved(100).contains(&a)); + + // A token for an outpoint that was never reserved is simply ignored. + let never = outpoint(0x23, 0); + set.release_if_owner(&[never], stale_token); + assert!(!set.reserved(100).contains(&never)); + } + + /// The core TOCTOU regression: reserve X under token A, then simulate the + /// TTL sweep reclaiming X and a *different* concurrent build re-reserving X + /// under token B. Token A's late `release_if_owner` (the rejected-broadcast + /// cleanup) must NOT remove X, because X now belongs to build B — releasing + /// it would hand B's input to coin selection and open a double-spend window. + /// See `dashpay/platform#4185`. + #[test] + fn release_if_owner_does_not_free_a_reservation_taken_over_after_a_sweep() { + let set = ReservationSet::default(); + let x = outpoint(0x30, 0); + + // Build A reserves X. + let token_a = set.reserve(&[x], 100); + assert!(set.reserved(100).contains(&x)); + + // TTL sweep reclaims A's reservation mid-await (modeled by advancing the + // height past the TTL so the next reserve's sweep drops A's entry)... + let swept_height = 100 + RESERVATION_TTL_BLOCKS; + // ...and build B re-reserves the very same outpoint under a new token. + let token_b = set.reserve(&[x], swept_height); + assert_ne!(token_a, token_b); + assert!(set.reserved(swept_height).contains(&x)); + + // Build A's rejected-broadcast cleanup releases by (outpoint, token A). + set.release_if_owner(&[x], token_a); + + // X must still be reserved — it is now owned by build B. + assert!( + set.reserved(swept_height).contains(&x), + "release_if_owner with the stale owner token must not free B's reservation" + ); + + // And B can still release its own reservation normally. + set.release_if_owner(&[x], token_b); + assert!(!set.reserved(swept_height).contains(&x)); + } } diff --git a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs index 8c00a9a9e..26595961d 100644 --- a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs @@ -5,12 +5,12 @@ use dashcore::blockdata::transaction::special_transaction::asset_lock::AssetLockPayload; use dashcore::blockdata::transaction::special_transaction::TransactionPayload; -use dashcore::{Transaction, TxOut}; +use dashcore::{OutPoint, Transaction, TxOut}; use secp256k1::PublicKey; use std::fmt; use crate::managed_account::managed_account_trait::ManagedAccountTrait; -use crate::managed_account::ManagedCoreKeysAccount; +use crate::managed_account::{ManagedCoreKeysAccount, ReservationToken}; use crate::signer::Signer; use crate::wallet::managed_wallet_info::coin_selection::SelectionStrategy; use crate::wallet::managed_wallet_info::fee::FeeRate; @@ -110,6 +110,18 @@ pub struct AssetLockResult { /// Per-credit-output key material. See [`AssetLockCreditKeys`] for /// ordering and variant semantics. pub keys: AssetLockCreditKeys, + /// Owner token for the reservation this build took on the funding inputs, + /// or `None` if the funding account carried no reservation set. + /// + /// The caller broadcasts `transaction` and, on a rejected broadcast, must + /// release the reserved inputs with + /// [`ManagedCoreFundsAccount::release_reservation_if_owner`] passing this + /// token — never the unconditional `release_reservation`. See + /// `ReservationSet::release_if_owner` for why owner-guarded release is + /// required here (`dashpay/platform#4185`). + /// + /// [`ManagedCoreFundsAccount::release_reservation_if_owner`]: crate::managed_account::ManagedCoreFundsAccount::release_reservation_if_owner + pub reservation_token: Option, } /// Errors specific to asset lock transaction building. @@ -306,30 +318,55 @@ impl ManagedWalletInfo { if drain { builder = builder.set_selection_strategy(SelectionStrategy::All); } - let (transaction, fee) = builder + let (transaction, fee, reservation_token) = builder .set_funding(funds_acc, acc) .require_final_inputs() - .build_signed(wallet, |addr| funds_acc.address_derivation_path(&addr)) + .build_signed_reserved(wallet, |addr| funds_acc.address_derivation_path(&addr)) .await?; - // Derive one private key per credit output. - let mut keys = Vec::with_capacity(credit_output_fundings.len()); - for funding in &credit_output_fundings { - let funding_key_account = resolve_funding_account( - &mut self.accounts, - funding.funding_type, - funding.identity_index, - )?; - let key = funding_key_account - .next_private_key(&root_xpriv, network) - .map_err(|e| AssetLockError::KeyDerivation(e.to_string()))?; - keys.push(key); - } + // The build above reserved the funding inputs. Clone the reservation + // handle (a shared `Arc` view of the same set) now, before the loop + // below re-borrows `self.accounts` — a mid-loop failure can no longer + // reach `funds_acc` to release, and the caller never received the token + // to release it either, so a leaked reservation would strand the + // already-signed inputs until the 24-block TTL sweep. Owner-guarded + // release only (see `ReservationSet::release_if_owner`, + // `dashpay/platform#4185`). + let reservations = funds_acc.reservations().clone(); + let reserved: Vec = + transaction.input.iter().map(|input| input.previous_output).collect(); + + // Derive one private key per credit output. On any failure, release + // this build's own reservation before returning. + let keys = match (|| -> Result, AssetLockError> { + let mut keys = Vec::with_capacity(credit_output_fundings.len()); + for funding in &credit_output_fundings { + let funding_key_account = resolve_funding_account( + &mut self.accounts, + funding.funding_type, + funding.identity_index, + )?; + let key = funding_key_account + .next_private_key(&root_xpriv, network) + .map_err(|e| AssetLockError::KeyDerivation(e.to_string()))?; + keys.push(key); + } + Ok(keys) + })() { + Ok(keys) => keys, + Err(e) => { + if let Some(token) = reservation_token { + reservations.release_if_owner(&reserved, token); + } + return Err(e); + } + }; Ok(AssetLockResult { transaction, fee, keys: AssetLockCreditKeys::Private(keys), + reservation_token, }) } @@ -408,12 +445,23 @@ impl ManagedWalletInfo { if drain { builder = builder.set_selection_strategy(SelectionStrategy::All); } - let (transaction, fee) = builder + let (transaction, fee, reservation_token) = builder .set_funding(funds_acc, &acc) .require_final_inputs() - .build_signed(signer, |addr| funds_acc.address_derivation_path(&addr)) + .build_signed_reserved(signer, |addr| funds_acc.address_derivation_path(&addr)) .await?; + // The build above reserved the funding inputs. Clone the reservation + // handle (a shared `Arc` view of the same set) before the bookkeeping + // loop below re-borrows `self.accounts`, so a failure during Phase 1–3 + // — which runs after the transaction is already signed — can still + // release THIS build's reservation instead of stranding the signed + // inputs until the 24-block TTL sweep. Owner-guarded release only (see + // `ReservationSet::release_if_owner`, `dashpay/platform#4185`). + let reservations = funds_acc.reservations().clone(); + let reserved: Vec = + transaction.input.iter().map(|input| input.previous_output).collect(); + // Credit-output bookkeeping: for each funding, peek the next unused // path on its account, ask the signer for the matching pubkey, and // only mark the index used once the signer has succeeded. @@ -421,50 +469,65 @@ impl ManagedWalletInfo { // This protects against a signer failure mid-loop leaving earlier // fundings' pool indices irreversibly consumed: if `public_key` // errors, the current funding's index is still free, and no - // subsequent fundings have touched their pools yet. - let mut credit_output_keys = Vec::with_capacity(credit_output_fundings.len()); - for funding in &credit_output_fundings { - // Phase 1 (sync): peek without marking used. Borrow is scoped - // to the block so we can re-resolve the account after the - // signer await. - let (path, index) = { - let funding_key_account = resolve_funding_account( - &mut self.accounts, - funding.funding_type, - funding.identity_index, - )?; - funding_key_account - .peek_next_path() - .map_err(|e| AssetLockError::KeyDerivation(e.to_string()))? - }; - - // Phase 2 (async): signer round-trip. If this errors, we return - // without ever calling mark_first_pool_index_used — index stays - // free for a retry. - let pubkey = signer - .public_key(&path) - .await - .map_err(|e| AssetLockError::Signer(e.to_string()))?; - - // Phase 3 (sync): signer succeeded, commit the index. - { - let funding_key_account = resolve_funding_account( - &mut self.accounts, - funding.funding_type, - funding.identity_index, - )?; - funding_key_account - .mark_first_pool_index_used(index) - .map_err(|e| AssetLockError::KeyDerivation(e.to_string()))?; + // subsequent fundings have touched their pools yet. On any failure we + // also release this build's own reservation before returning. + let credit_output_keys = match async { + let mut credit_output_keys = Vec::with_capacity(credit_output_fundings.len()); + for funding in &credit_output_fundings { + // Phase 1 (sync): peek without marking used. Borrow is scoped + // to the block so we can re-resolve the account after the + // signer await. + let (path, index) = { + let funding_key_account = resolve_funding_account( + &mut self.accounts, + funding.funding_type, + funding.identity_index, + )?; + funding_key_account + .peek_next_path() + .map_err(|e| AssetLockError::KeyDerivation(e.to_string()))? + }; + + // Phase 2 (async): signer round-trip. If this errors, we return + // without ever calling mark_first_pool_index_used — index stays + // free for a retry. + let pubkey = signer + .public_key(&path) + .await + .map_err(|e| AssetLockError::Signer(e.to_string()))?; + + // Phase 3 (sync): signer succeeded, commit the index. + { + let funding_key_account = resolve_funding_account( + &mut self.accounts, + funding.funding_type, + funding.identity_index, + )?; + funding_key_account + .mark_first_pool_index_used(index) + .map_err(|e| AssetLockError::KeyDerivation(e.to_string()))?; + } + + credit_output_keys.push((pubkey, path)); } - - credit_output_keys.push((pubkey, path)); + Ok::<_, AssetLockError>(credit_output_keys) } + .await + { + Ok(keys) => keys, + Err(e) => { + if let Some(token) = reservation_token { + reservations.release_if_owner(&reserved, token); + } + return Err(e); + } + }; Ok(AssetLockResult { transaction, fee, keys: AssetLockCreditKeys::Public(credit_output_keys), + reservation_token, }) } } diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 747f23e8a..076581ed6 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -3,7 +3,7 @@ //! This module provides high-level transaction building functionality //! using types from the dashcore crate. -use crate::managed_account::reservation::ReservationSet; +use crate::managed_account::reservation::{ReservationSet, ReservationToken}; use crate::managed_account::ManagedCoreFundsAccount; use crate::wallet::managed_wallet_info::coin_selection::{ CoinSelector, SelectionStrategy, CHANGE_OUTPUT_SIZE, TX_OUTPUT_SIZE, @@ -282,7 +282,18 @@ impl TransactionBuilder { size } - fn assemble_unsigned(mut self) -> Result<(Transaction, Vec), BuilderError> { + /// Select inputs, build the unsigned transaction, and reserve the chosen + /// inputs. The optional [`ReservationToken`] is `Some` exactly when a + /// reservation set was attached (via [`set_funding`]) and identifies the + /// reservation this build just took, so a caller that later abandons the + /// build can release *only* its own inputs via + /// [`ReservationSet::release_if_owner`]. It is `None` for builds with no + /// reservation set (nothing was reserved, nothing to release). + /// + /// [`set_funding`]: Self::set_funding + fn assemble_unsigned( + mut self, + ) -> Result<(Transaction, Vec, Option), BuilderError> { if let Some(TransactionPayload::AssetLockPayloadType(p)) = &self.special_payload { if p.credit_outputs.is_empty() { return Err(BuilderError::NoOutputs); @@ -424,14 +435,16 @@ impl TransactionBuilder { // Reserve the chosen inputs so a concurrent build skips them until the // broadcast transaction is processed back into the wallet (which - // releases the reservation) or the TTL backstop reclaims it. - if let Some(reservations) = &self.reservations { + // releases the reservation) or the TTL backstop reclaims it. Keep the + // stamped owner token so the caller can release only this reservation + // if the build is later abandoned (see `release_if_owner`). + let reservation_token = self.reservations.as_ref().map(|reservations| { let outpoints: Vec = selected_inputs.iter().map(|utxo| utxo.outpoint).collect(); - reservations.reserve(&outpoints, self.current_height); - } + reservations.reserve(&outpoints, self.current_height) + }); - return Ok((transaction, selected_inputs)); + return Ok((transaction, selected_inputs, reservation_token)); // BIP-69: Sort outputs by amount first, then by scriptPubKey // lexicographically. @@ -454,17 +467,31 @@ impl TransactionBuilder { } } - /// Build the unsigned transaction. The returned fee is the fee the - /// transaction actually pays: Σ(selected input values) − Σ(output values). - /// This can exceed the size-based fee target when a dust change remainder - /// (≤ 546 duffs) is dropped and left to miners. - pub fn build_unsigned(self) -> Result<(Transaction, u64), BuilderError> { - let (tx, inputs) = self.assemble_unsigned()?; + /// Build the unsigned transaction, returning it alongside the fee it pays + /// and the [`ReservationToken`] stamped onto the inputs this build reserved + /// (`None` when no reservation set is attached). + /// + /// The returned fee is the fee the transaction actually pays: + /// Σ(selected input values) − Σ(output values). This can exceed the + /// size-based fee target when a dust change remainder (≤ 546 duffs) is + /// dropped and left to miners. + /// + /// Hold the returned token when the transaction may be abandoned after an + /// `.await` that releases the wallet lock — most importantly the platform + /// broadcast path, which reserves inputs, awaits the broadcast, and on + /// rejection must release them — and release with + /// [`ManagedCoreFundsAccount::release_reservation_if_owner`]. See + /// `ReservationSet::release_if_owner` for why owner-guarded release is + /// required (`dashpay/platform#4185`). + pub fn build_unsigned_reserved( + self, + ) -> Result<(Transaction, u64, Option), BuilderError> { + let (tx, inputs, reservation) = self.assemble_unsigned()?; let total_input: u64 = inputs.iter().map(|utxo| utxo.value()).sum(); let total_output: u64 = tx.output.iter().map(|out| out.value).sum(); - Ok((tx, total_input.saturating_sub(total_output))) + Ok((tx, total_input.saturating_sub(total_output), reservation)) } /// Build and sign the transaction. The `path_resolver` maps each input @@ -478,24 +505,49 @@ impl TransactionBuilder { signer: &S, path_resolver: P, ) -> Result<(Transaction, u64), BuilderError> + where + S: TransactionSigner + ?Sized + Sync, + P: Fn(Address) -> Option + Send, + { + let (tx, fee, _reservation) = self.build_signed_reserved(signer, path_resolver).await?; + Ok((tx, fee)) + } + + /// Like [`Self::build_signed`], but also returns the [`ReservationToken`] + /// stamped onto the inputs this build reserved (`None` when no reservation + /// set is attached), for callers that may later abandon the transaction + /// after awaiting a broadcast. See [`Self::build_unsigned_reserved`] for why + /// the token is needed and how to release with it. + pub async fn build_signed_reserved( + self, + signer: &S, + path_resolver: P, + ) -> Result<(Transaction, u64, Option), BuilderError> where S: TransactionSigner + ?Sized + Sync, P: Fn(Address) -> Option + Send, { let reservations = self.reservations.clone(); - let (tx, inputs) = self.assemble_unsigned()?; + let (tx, inputs, reservation) = self.assemble_unsigned()?; let total_input: u64 = inputs.iter().map(|utxo| utxo.value()).sum(); // Signing never reaches the network for a local key, but an external // signer can fail. A failed sign means the reserved inputs are still // spendable, so release them now instead of stranding the funds until // the TTL backstop reclaims them. + // + // Release owner-guarded: `sign_tx` is an `.await`, and while it runs the + // TTL sweep could reclaim this build's reservation and a concurrent + // build could re-reserve the same outpoint under a new token. An + // unconditional release-by-outpoint would then free that other build's + // inputs (the double-spend window of `dashpay/platform#4185`), so we + // release only outpoints still owned by the token this build stamped. let reserved: Vec = inputs.iter().map(|utxo| utxo.outpoint).collect(); let tx = match signer.sign_tx(tx, inputs, path_resolver).await { Ok(tx) => tx, Err(err) => { - if let Some(reservations) = &reservations { - reservations.release(reserved.iter()); + if let (Some(reservations), Some(token)) = (&reservations, reservation) { + reservations.release_if_owner(&reserved, token); } return Err(err); } @@ -503,7 +555,7 @@ impl TransactionBuilder { let total_output: u64 = tx.output.iter().map(|out| out.value).sum(); - Ok((tx, total_input.saturating_sub(total_output))) + Ok((tx, total_input.saturating_sub(total_output), reservation)) } } @@ -702,8 +754,8 @@ mod tests { .add_inputs([utxo]) .add_output(&destination, 50000) .set_change_address(change) - .build_unsigned() - .map(|(tx, _)| tx); + .build_unsigned_reserved() + .map(|(tx, _, _)| tx); assert!(tx.is_ok()); let transaction = tx.unwrap(); @@ -720,7 +772,7 @@ mod tests { .set_current_height(200) .add_inputs([utxo]) .add_output(&destination, 50000) - .build_unsigned(); + .build_unsigned_reserved(); // Insufficient funds now surface via the coin selector wrapper too. assert!(matches!( @@ -740,7 +792,7 @@ mod tests { .set_current_height(200) .add_inputs([utxo]) // no set_change_address .add_output(&destination, 100_000) - .build_unsigned(); + .build_unsigned_reserved(); assert!( matches!(result, Err(BuilderError::NoChangeAddress)), @@ -844,7 +896,7 @@ mod tests { .set_change_address(change_address.clone()) .add_inputs(utxos) .add_output(&recipient_address, 500000) - .build_unsigned() + .build_unsigned_reserved() .unwrap() .0; @@ -873,7 +925,7 @@ mod tests { .set_change_address(change_address.clone()) .add_inputs(utxos) .add_output(&recipient_address, 150000) - .build_unsigned() + .build_unsigned_reserved() .unwrap() .0; @@ -893,14 +945,14 @@ mod tests { let recipient_address = Address::dummy(Network::Testnet, 0); let change_address = Address::dummy(Network::Testnet, 0); - let (tx, fee) = TransactionBuilder::new() + let (tx, fee, _) = TransactionBuilder::new() .set_current_height(200) .set_selection_strategy(SelectionStrategy::SmallestFirst) .set_fee_rate(FeeRate::normal()) .set_change_address(change_address) .add_inputs(utxos) .add_output(&recipient_address, 150000) - .build_unsigned() + .build_unsigned_reserved() .unwrap(); assert_eq!(tx.output.len(), 1, "dust change must be dropped"); @@ -926,13 +978,13 @@ mod tests { }], }; - let (tx, fee) = TransactionBuilder::new() + let (tx, fee, _) = TransactionBuilder::new() .set_current_height(200) .set_fee_rate(FeeRate::normal()) .set_change_address(change_address) .set_special_payload(TransactionPayload::AssetLockPayloadType(asset_lock_payload)) .add_inputs(utxos) - .build_unsigned() + .build_unsigned_reserved() .unwrap(); assert_eq!(tx.output.len(), 2, "OP_RETURN burn output + change"); @@ -1035,7 +1087,7 @@ mod tests { .add_output(&address1, 300000) // Higher amount .add_output(&address2, 100000) // Lower amount .add_output(&address1, 200000) // Middle amount - .build_unsigned() + .build_unsigned_reserved() .unwrap() .0; @@ -1107,7 +1159,7 @@ mod tests { .add_inputs([utxo2.clone()]) .add_inputs([utxo3.clone()]) .add_output(&destination, 500000) - .build_unsigned() + .build_unsigned_reserved() .unwrap() .0; @@ -1173,7 +1225,7 @@ mod tests { .set_special_payload(TransactionPayload::AssetLockPayloadType(asset_lock_payload)) .add_output(&recipient_address, 50000) .add_inputs(utxos) - .build_unsigned() + .build_unsigned_reserved() .unwrap() .0; @@ -1245,13 +1297,13 @@ mod tests { funds.utxos.insert(utxo.outpoint, utxo.clone()); let destination = Address::dummy(Network::Testnet, 0); - let (tx, _) = TransactionBuilder::new() + let (tx, _, _) = TransactionBuilder::new() .set_current_height(200) .set_fee_rate(FeeRate::normal()) .set_funding(&mut funds, &account) .set_change_address(Address::dummy(Network::Testnet, 1)) .add_output(&destination, 500_000) - .build_unsigned() + .build_unsigned_reserved() .expect("build unsigned"); // Every input the build selected is reserved, so a later build observes diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index 029ab4782..425d36ac7 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -158,7 +158,7 @@ mod tests { .set_change_address(change_address.clone()) .add_output(&recipient_address, 150000) .add_inputs(utxos) - .build_unsigned() + .build_unsigned_reserved() .unwrap() .0; @@ -190,13 +190,13 @@ mod tests { .require_network(Network::Testnet) .unwrap(); - let (tx, _fee) = TransactionBuilder::new() + let (tx, _fee, _) = TransactionBuilder::new() .set_fee_rate(FeeRate::normal()) .set_current_height(200) .set_change_address(change_address) .add_output(&recipient_address, 150000) .add_inputs(utxos) - .build_unsigned() + .build_unsigned_reserved() .expect("ordinary spend of an unconfirmed UTXO must succeed"); assert!(!tx.input.is_empty()); } @@ -217,13 +217,13 @@ mod tests { let total = 600_000u64; let fee = FeeRate::normal().calculate_fee(8 + 1 + 1 + 34 + 3 * 148); let deliverable = total - fee; - let (tx, _fee) = TransactionBuilder::new() + let (tx, _fee, _) = TransactionBuilder::new() .set_fee_rate(FeeRate::normal()) .set_current_height(200) .set_selection_strategy(SelectionStrategy::All) .add_inputs(utxos) .add_output(&dest, deliverable) - .build_unsigned() + .build_unsigned_reserved() .unwrap(); assert_eq!(tx.input.len(), 3, "sweep spends every input"); @@ -312,7 +312,7 @@ mod tests { .add_output(&recipient_address, 150000) .add_inputs(utxos); - let tx = builder.build_unsigned().unwrap().0; + let tx = builder.build_unsigned_reserved().unwrap().0; let serialized = dashcore::consensus::encode::serialize(&tx); // Size should be close to our estimation @@ -345,7 +345,7 @@ mod tests { .add_output(&recipient_address, 500000) .add_inputs(utxos); - let tx = builder.build_unsigned().unwrap().0; + let tx = builder.build_unsigned_reserved().unwrap().0; // Total input: 1000000 // Output to recipient: 500000 @@ -376,7 +376,7 @@ mod tests { .set_change_address(change_address.clone()) .add_output(&recipient_address, 1000000) // More than available .add_inputs(utxos) - .build_unsigned(); + .build_unsigned_reserved(); assert!(result.is_err()); } @@ -402,7 +402,7 @@ mod tests { .add_output(&recipient_address, 150000) .add_inputs(utxos); - let tx = builder.build_unsigned().unwrap().0; + let tx = builder.build_unsigned_reserved().unwrap().0; // Should only have 1 output (no change) assert_eq!(tx.output.len(), 1); From 712bab113973c9d319521a3457b64c4657ef6863 Mon Sep 17 00:00:00 2001 From: bfoss765 <38437574+bfoss765@users.noreply.github.com> Date: Wed, 5 Aug 2026 20:25:09 -0400 Subject: [PATCH 062/149] fix(dash-spv): recover InstantSend locks received before quorum sync completes (#895) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(dash-spv): recover InstantSend locks received before quorum sync completes A wallet integration observed a shielded fund-from-asset-lock stall for ~3.5 minutes on testnet. The wallet broadcast an asset-lock transaction and the network produced an InstantSend lock for it within ~2 seconds, but a freshly (re)started dash-spv client never used the islock — it fell back to waiting for the transaction to be mined before the asset-lock proof flow could proceed. Root cause: an islock that arrives before quorum sync has loaded the quorum needed to verify its BLS signature is correctly queued in `InstantSendManager::pending_instantlocks`, but re-validation of that queue was edge-triggered exclusively by the `MasternodeStateUpdated` sync event (`instantsend/sync_manager.rs`). After the initial masternode sync completes that event only re-fires on a new MnListDiff/QRInfo — i.e. on a new block — so a lock queued just after a fresh start is not re-checked until the next block, which for a just-broadcast transaction is effectively the block that mines it. `tick` only pruned the cache and never retried validation. Fix: make re-validation level-triggered. `tick` now re-runs `validate_pending` as soon as the masternode engine advances past the height at which the queue was last checked (tracked by `last_validated_engine_height`), so a queued islock is verified the moment the required quorum data lands instead of waiting for the next event. Pending expiry is switched from an attempt counter to a time-based TTL, since tick-driven retries would otherwise exhaust the counter long before the hour-long window elapses during a fresh sync. The existing `MasternodeStateUpdated` path is retained and also updates the marker. Tests (deterministic, no dashd required): - test_tick_revalidates_pending_when_engine_advances - test_tick_skips_revalidation_without_engine_advance - test_pending_expires_after_ttl_on_revalidation - test_on_disconnect_resets_validation_marker Co-Authored-By: Claude Fable 5 * address review: clear unvalidated cache on disconnect, TTL expiry independent of engine advance, synced-state transition after tick validation - on_disconnect now drops unvalidated instantlock cache entries (via clear_unvalidated_on_disconnect) alongside the pending queue, so a lock re-announced after reconnect is no longer deduplicated against a stale cache entry and is re-queued/re-validated fresh. - tick runs a cheap, advancement-independent expire_pending pass on every tick, so pending locks past PENDING_TTL are dropped even when the engine height is static; full BLS re-validation stays gated on engine advance. - tick applies the synced-state transition after resolving pending work (via the shared transition_synced_if_idle helper, also used by handle_sync_event), so a final lock resolved by tick no longer leaves the manager stuck in Syncing/WaitForEvents. - tests: re-announce after disconnect is not a duplicate; pending expires by TTL under a static engine height; tick resolving the last pending lock transitions out of Syncing. Existing tick tests updated for the new expiry/validation split. Co-Authored-By: Claude Opus 4.8 * fix(dash-spv): use monotonic Instant for pending islock TTL Wall-clock SystemTime can jump backward/forward, corrupting the elapsed-time check that expires unverifiable pending InstantLocks. Switch PendingInstantLock::first_seen from SystemTime to std::time::Instant, which is monotonic and correct for measuring an elapsed TTL. Instant::elapsed returns a Duration directly, so the expiry check simplifies to a plain comparison (matching the existing UnconfirmedTx pattern in types.rs). Co-Authored-By: Claude Fable 5 * fix(spv): make instantsend pending-TTL expiry portable on Windows Commit 985339e switched `PendingInstantLock::first_seen` to a monotonic `Instant`, and the tests simulated an expired lock by back-dating it as `Instant::now() - PENDING_TTL`. Subtracting from an `Instant` panics when the result would precede the `Instant` epoch, and on Windows that epoch is the boot time, so on a freshly booted CI runner `Instant::now() - 3600s` underflows. That turned the `Windows / spv` job red in four tests. An `Instant` cannot portably represent a time before its epoch, so expiry can no longer be simulated by back-dating `first_seen`. Instead, inject the current time: - Add `PendingInstantLock::is_expired(now)`, using panic-free `saturating_duration_since` instead of `elapsed()`/subtraction. - Thread an injected `now` through `expire_pending`, `validate_pending`, and a new inherent `tick_at`. Production call sites (`tick`, `handle_sync_event`) pass `Instant::now()`, so behavior is unchanged. - Rewrite the four fixtures: build a fresh `first_seen = Instant::now()` and drive expiry with a future `now = first_seen + PENDING_TTL + 1s` (`Instant + Duration` never underflows). No `Instant` subtraction or `duration_since().expect()` remains in the instantsend module; only `saturating_duration_since` and additive construction, eliminating the Windows underflow class. Co-Authored-By: Claude Fable 5 * refactor(spv): single source for the pending-islock expiry check + log The `is_expired` test and its "Dropping InstantLock for txid ... after awaiting quorum data" warning were duplicated between `validate_pending` (the re-validation loop) and `expire_pending` (the advancement-independent sweep), so a future change to PENDING_TTL or to the message would have to be made in both places. Fold both into `PendingInstantLock::is_expired_logged`, next to the `is_expired` predicate and the `first_seen` field it reads. Callers keep their own bookkeeping — `validate_pending` still counts the drop as invalid and skips re-verification, `expire_pending` still filters the queue and updates progress — so behaviour is unchanged. Addresses the remaining CodeRabbit nitpick on #895. --------- Co-authored-by: Claude Fable 5 Co-authored-by: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> --- dash-spv/src/sync/instantsend/manager.rs | 409 +++++++++++++++++- dash-spv/src/sync/instantsend/sync_manager.rs | 83 +++- 2 files changed, 457 insertions(+), 35 deletions(-) diff --git a/dash-spv/src/sync/instantsend/manager.rs b/dash-spv/src/sync/instantsend/manager.rs index eb0ed5509..312e61255 100644 --- a/dash-spv/src/sync/instantsend/manager.rs +++ b/dash-spv/src/sync/instantsend/manager.rs @@ -5,7 +5,7 @@ use std::collections::HashMap; use std::sync::Arc; -use std::time::{Duration, SystemTime}; +use std::time::{Duration, Instant, SystemTime}; use dashcore::ephemerealdata::instant_lock::InstantLock; use dashcore::hashes::Hash; @@ -14,7 +14,7 @@ use dashcore::Txid; use tokio::sync::RwLock; use crate::error::SyncResult; -use crate::sync::{InstantSendProgress, SyncEvent}; +use crate::sync::{InstantSendProgress, SyncEvent, SyncState}; /// Maximum number of pending InstantLocks awaiting validation. const MAX_PENDING_INSTANTLOCKS: usize = 500; @@ -25,8 +25,12 @@ const MAX_CACHE_SIZE: usize = 5000; /// TTL for cached InstantLocks (1 hour). const CACHE_TTL: Duration = Duration::from_secs(3600); -/// Maximum retry attempts before dropping a pending InstantLock (~1 hour at 2.5min blocks). -const MAX_RETRIES: u32 = 24; +/// How long a pending InstantLock is retained while awaiting the quorum data +/// needed to verify it. Locks that stay unverifiable for longer than this are +/// dropped. Expiry is time-based (not attempt-based) because re-validation can +/// now be driven from `tick` as the engine advances, so an attempt counter +/// would overrun long before an hour elapsed during a fresh sync. +const PENDING_TTL: Duration = Duration::from_secs(3600); /// Entry in the InstantLock cache. #[derive(Debug, Clone)] @@ -39,13 +43,53 @@ pub struct InstantLockEntry { pub validated: bool, } -/// Pending InstantLock awaiting validation with retry tracking. +/// Pending InstantLock awaiting the quorum data required to verify it. #[derive(Debug, Clone)] pub(super) struct PendingInstantLock { /// The InstantLock data. instant_lock: InstantLock, - /// Number of validation retry attempts. - retry_count: u32, + /// When the lock was first received, used to expire locks that never + /// become verifiable. Uses a monotonic `Instant` so the elapsed-time TTL + /// check is unaffected by wall-clock adjustments. + first_seen: Instant, +} + +impl PendingInstantLock { + /// Whether this pending lock has outlived [`PENDING_TTL`] as of `now`. + /// + /// Uses [`Instant::saturating_duration_since`], which clamps to zero + /// instead of panicking when `now` precedes `first_seen`. The current time + /// is threaded in by the caller (production passes `Instant::now()`; tests + /// pass a fabricated future instant) so expiry can be exercised without + /// back-dating `first_seen`. Back-dating an `Instant` by subtraction + /// (`Instant::now() - PENDING_TTL`) underflows and panics on platforms + /// whose `Instant` epoch sits near zero — notably Windows, where the clock + /// starts at boot — because an `Instant` cannot represent a time before its + /// epoch. + fn is_expired(&self, now: Instant) -> bool { + now.saturating_duration_since(self.first_seen) > PENDING_TTL + } + + /// [`is_expired`](Self::is_expired), logging the drop when it returns true. + /// + /// Both expiry paths — the advancement-independent sweep in + /// [`expire_pending`](InstantSendManager::expire_pending) and the + /// re-validation loop in + /// [`validate_pending`](InstantSendManager::validate_pending) — report a + /// dropped lock identically, so the check and its warning live here and + /// cannot drift apart when the TTL or the message changes. Callers remain + /// responsible for their own bookkeeping (progress counters, removal). + fn is_expired_logged(&self, now: Instant) -> bool { + let expired = self.is_expired(now); + if expired { + tracing::warn!( + "Dropping InstantLock for txid {} after awaiting quorum data for over {}s", + self.instant_lock.txid, + PENDING_TTL.as_secs() + ); + } + expired + } } /// InstantSend manager. @@ -62,8 +106,17 @@ pub struct InstantSendManager { engine: Arc>, /// InstantLocks indexed by txid. instantlocks: HashMap, - /// Pending InstantLocks awaiting validation with retry tracking. + /// Pending InstantLocks awaiting the quorum data required to verify them. pub(super) pending_instantlocks: Vec, + /// Highest masternode-list height at which every pending InstantLock has + /// already been (re)validated. + /// + /// `tick` uses this to re-run validation only once the engine has advanced + /// past this height, so a lock received before quorum sync completed is + /// re-checked as soon as new quorum data lands — instead of waiting for the + /// next `MasternodeStateUpdated` event, which for a freshly broadcast + /// transaction may not arrive until the block that mines it. + pub(super) last_validated_engine_height: Option, } impl InstantSendManager { @@ -74,6 +127,7 @@ impl InstantSendManager { engine, instantlocks: HashMap::new(), pending_instantlocks: Vec::new(), + last_validated_engine_height: None, } } @@ -107,7 +161,7 @@ impl InstantSendManager { } else { self.queue_pending(PendingInstantLock { instant_lock: instantlock.clone(), - retry_count: 0, + first_seen: Instant::now(), }); self.progress.update_pending(self.pending_instantlocks.len()); } @@ -201,22 +255,35 @@ impl InstantSendManager { } } - /// Validate pending InstantLocks after masternode engine becomes available. - pub(super) async fn validate_pending(&mut self) -> SyncResult> { + /// Latest masternode-list height known to the shared engine, if any. + pub(super) async fn engine_height(&self) -> Option { + self.engine.read().await.masternode_lists.keys().next_back().copied() + } + + /// Re-validate every pending InstantLock against the current engine state. + /// + /// Called both when a `MasternodeStateUpdated` event arrives and from `tick` + /// once the engine has advanced. Locks that verify are emitted as validated + /// `InstantLockReceived` events; locks that still can't be verified are + /// re-queued until they either become verifiable or exceed [`PENDING_TTL`]. + /// + /// `now` is the current time used for TTL expiry; production callers pass + /// `Instant::now()`. It is threaded in (rather than read here) so tests can + /// drive expiry with a future instant without back-dating `first_seen` — + /// see [`PendingInstantLock::is_expired`]. + pub(super) async fn validate_pending(&mut self, now: Instant) -> SyncResult> { + // Snapshot the height we are validating against before doing the work so + // `tick` knows not to re-run until the engine advances past it. + let engine_height = self.engine_height().await; + let pending = std::mem::take(&mut self.pending_instantlocks); let mut events = Vec::new(); - for mut pending_lock in pending { - pending_lock.retry_count += 1; + for pending_lock in pending { let txid = pending_lock.instant_lock.txid; - // Check if max retries exceeded - if pending_lock.retry_count > MAX_RETRIES { - tracing::warn!( - "Dropping InstantLock for txid {} after {} retries", - txid, - pending_lock.retry_count - ); + // Drop locks that have been awaiting quorum data for too long. + if pending_lock.is_expired_logged(now) { self.progress.add_invalid(1); continue; } @@ -239,6 +306,7 @@ impl InstantSendManager { } } + self.last_validated_engine_height = engine_height; self.progress.update_pending(self.pending_instantlocks.len()); Ok(events) } @@ -251,6 +319,60 @@ impl InstantSendManager { }); } + /// Drop cached InstantLock entries that were never BLS-validated, and clear + /// the queued locks that fed them. + /// + /// Called on disconnect. The `instantlocks` cache is what + /// [`process_instantlock`](Self::process_instantlock) dedupes against, so if + /// an unvalidated entry survived a disconnect the same lock re-announced + /// after reconnect would be treated as a duplicate and never re-queued or + /// re-validated. Validated entries are kept — their locks are already final. + pub(super) fn clear_unvalidated_on_disconnect(&mut self) { + self.pending_instantlocks.clear(); + self.instantlocks.retain(|_, entry| entry.validated); + self.progress.update_pending(0); + } + + /// Drop pending locks that have outlived [`PENDING_TTL`] without running BLS + /// verification. + /// + /// This is the cheap, advancement-independent half of expiry: `tick` calls + /// it on every tick so a lock that never becomes verifiable is dropped after + /// an hour even when the masternode engine height is static. Full + /// re-validation (which does run BLS) stays in + /// [`validate_pending`](Self::validate_pending) and only runs when the engine + /// advances. Returns the number of locks expired. + /// + /// `now` is the current time used for TTL expiry; production callers pass + /// `Instant::now()`. It is threaded in (rather than read here) so tests can + /// drive expiry with a future instant without back-dating `first_seen` — + /// see [`PendingInstantLock::is_expired`]. + pub(super) fn expire_pending(&mut self, now: Instant) -> usize { + let before = self.pending_instantlocks.len(); + self.pending_instantlocks.retain(|pending| !pending.is_expired_logged(now)); + let expired = before - self.pending_instantlocks.len(); + if expired > 0 { + self.progress.add_invalid(expired as u32); + self.progress.update_pending(self.pending_instantlocks.len()); + } + expired + } + + /// Transition to [`SyncState::Synced`] once no pending validations remain. + /// + /// Shared by the event-driven (`handle_sync_event`) and tick-driven + /// revalidation paths so both resolve the manager's state the same way when + /// the last pending lock is validated or expired. No-op unless the manager + /// is currently `Syncing` or `WaitForEvents`. + pub(super) fn transition_synced_if_idle(&mut self) { + if self.pending_instantlocks.is_empty() + && matches!(self.progress.state(), SyncState::Syncing | SyncState::WaitForEvents) + { + self.progress.set_state(SyncState::Synced); + tracing::info!("InstantSend manager synced (no pending validations)"); + } + } + /// Get an InstantLock by transaction ID. pub fn get_instantlock(&self, txid: &Txid) -> Option<&InstantLockEntry> { self.instantlocks.get(txid) @@ -284,12 +406,55 @@ impl std::fmt::Debug for InstantSendManager { #[cfg(test)] mod tests { use super::*; - use crate::network::MessageType; + use crate::network::{MessageType, RequestSender}; use crate::sync::{ManagerIdentifier, SyncManager, SyncManagerProgress, SyncState}; use dashcore::bls_sig_utils::BLSSignature; use dashcore::hash_types::CycleHash; use dashcore::hashes::Hash; - use dashcore::OutPoint; + use dashcore::sml::masternode_list::MasternodeList; + use dashcore::{BlockHash, OutPoint}; + use tokio::sync::mpsc::unbounded_channel; + + /// Insert an empty masternode list at `height` so the shared engine reports a + /// new tip height. Empty lists carry no rotated quorums, so InstantLock + /// verification still fails — which is exactly what we want when exercising + /// the "engine advanced but the needed quorum is still absent" path. + async fn advance_engine_height(manager: &InstantSendManager, height: u32) { + let mut engine = manager.engine.write().await; + engine.masternode_lists.insert( + height, + MasternodeList::empty(BlockHash::from_byte_array([height as u8; 32]), height), + ); + } + + fn no_op_requests() -> RequestSender { + let (tx, _rx) = unbounded_channel(); + RequestSender::new(tx) + } + + /// A pending lock whose `first_seen` is the current instant. Expiry is + /// driven by the `now` passed to `tick_at`/`expire_pending`, not by + /// back-dating this field — see [`past_ttl`]. + fn fresh_pending(txid: Txid) -> PendingInstantLock { + PendingInstantLock { + instant_lock: create_test_instantlock(txid), + first_seen: Instant::now(), + } + } + + /// An injected "now" far enough past a [`fresh_pending`] lock's `first_seen` + /// that the lock counts as expired (`first_seen + PENDING_TTL + 1s`). + /// + /// Built additively from `Instant::now()`, so it never underflows. Expiry + /// used to be simulated by back-dating `first_seen` to + /// `Instant::now() - PENDING_TTL`, but subtracting from an `Instant` panics + /// on platforms whose `Instant` epoch is near zero — notably Windows, whose + /// monotonic clock starts at boot — because an `Instant` cannot represent a + /// time before its epoch. Driving expiry with a future `now` instead keeps + /// the test portable. + fn past_ttl() -> Instant { + Instant::now() + PENDING_TTL + Duration::from_secs(1) + } fn create_test_instantlock(txid: Txid) -> InstantLock { InstantLock { @@ -456,4 +621,204 @@ mod tests { // Should be capped at MAX_CACHE_SIZE assert!(manager.cached_count() <= MAX_CACHE_SIZE); } + + /// A queued InstantLock must be re-validated by `tick` as soon as the + /// masternode engine advances — not only when a `MasternodeStateUpdated` + /// event happens to arrive. This is the regression behind the on-device + /// incident: an islock received right after a fresh SPV start stayed pending + /// until the transaction was mined because nothing re-checked it. + #[tokio::test] + async fn test_tick_revalidates_pending_when_engine_advances() { + let mut manager = create_test_manager(); + let requests = no_op_requests(); + + // A lock arrives before quorum sync: the empty engine can't verify it, + // so it is queued. + let txid = Txid::from_byte_array([3u8; 32]); + let _ = manager.process_instantlock(&create_test_instantlock(txid)).await.unwrap(); + assert_eq!(manager.pending_count(), 1); + assert_eq!(manager.last_validated_engine_height, None); + + // Engine has not advanced yet (still empty): tick must not re-validate, + // and the pending-validation marker stays untouched. + let events = manager.tick(&requests).await.unwrap(); + assert!(events.is_empty()); + assert_eq!(manager.pending_count(), 1); + assert_eq!(manager.last_validated_engine_height, None); + + // The engine gains a new masternode-list height: tick now re-validates. + // The needed rotated quorum is still absent, so the lock is re-queued, + // but the marker records the height we validated against. + advance_engine_height(&manager, 200).await; + let events = manager.tick(&requests).await.unwrap(); + assert!(events.is_empty()); + assert_eq!(manager.pending_count(), 1); + assert_eq!(manager.last_validated_engine_height, Some(200)); + + // No further advance: tick must not spend work re-validating again. + let events = manager.tick(&requests).await.unwrap(); + assert!(events.is_empty()); + assert_eq!(manager.last_validated_engine_height, Some(200)); + + // Another advance re-arms re-validation. + advance_engine_height(&manager, 201).await; + let _ = manager.tick(&requests).await.unwrap(); + assert_eq!(manager.last_validated_engine_height, Some(201)); + } + + /// Without an engine advance, `tick` must not run (potentially expensive) BLS + /// re-validation. A *fresh* pending lock (well within its TTL) must survive + /// the tick untouched: the advancement-independent expiry pass leaves it in + /// place and validation never runs. + #[tokio::test] + async fn test_tick_skips_revalidation_without_engine_advance() { + let mut manager = create_test_manager(); + let requests = no_op_requests(); + + // Engine at height 100, but the marker is deliberately set ABOVE it so + // the advancement gate (current > last) stays closed. If `validate_pending` + // ran it would overwrite the marker down to 100 — so an unchanged marker + // proves it did not run. + advance_engine_height(&manager, 100).await; + manager.last_validated_engine_height = Some(150); + + manager.pending_instantlocks.push(fresh_pending(Txid::from_byte_array([7u8; 32]))); + + let events = manager.tick(&requests).await.unwrap(); + assert!(events.is_empty()); + assert_eq!(manager.pending_count(), 1); + assert_eq!(manager.last_validated_engine_height, Some(150)); + if let SyncManagerProgress::InstantSend(progress) = manager.progress() { + assert_eq!(progress.invalid(), 0); + } else { + panic!("Expected SyncManagerProgress::InstantSend"); + } + } + + /// TTL expiry must be independent of engine advancement (finding: expire + /// pending locks even when the engine height is static). A lock past its TTL + /// is dropped by `tick` without the engine ever advancing and without running + /// BLS re-validation. + #[tokio::test] + async fn test_tick_expires_pending_without_engine_advance() { + let mut manager = create_test_manager(); + let requests = no_op_requests(); + + // Close the advancement gate: engine at 100, already validated at 100. + // `validate_pending` therefore cannot run this tick. + advance_engine_height(&manager, 100).await; + manager.last_validated_engine_height = Some(100); + + manager.pending_instantlocks.push(fresh_pending(Txid::from_byte_array([8u8; 32]))); + + // Drive the tick with a `now` past the lock's TTL so the cheap expiry + // pass drops it without back-dating `first_seen`. + let events = manager.tick_at(past_ttl(), &requests).await.unwrap(); + + assert!(events.is_empty()); + // Expired purely by the advancement-independent pass. + assert_eq!(manager.pending_count(), 0); + // Marker untouched: only cheap expiry ran, not validation. + assert_eq!(manager.last_validated_engine_height, Some(100)); + if let SyncManagerProgress::InstantSend(progress) = manager.progress() { + assert_eq!(progress.invalid(), 1); + } else { + panic!("Expected SyncManagerProgress::InstantSend"); + } + } + + /// When `tick` resolves the final pending lock — here by expiring it under a + /// static engine height — the manager must leave `Syncing`, exactly as it + /// would had the resolution come from a `MasternodeStateUpdated` event via + /// `handle_sync_event`. Otherwise it stays stuck in `Syncing` indefinitely. + #[tokio::test] + async fn test_tick_transitions_synced_when_last_pending_resolved() { + let mut manager = create_test_manager(); + let requests = no_op_requests(); + + manager.set_state(SyncState::Syncing); + manager.pending_instantlocks.push(fresh_pending(Txid::from_byte_array([4u8; 32]))); + + // A `now` past the lock's TTL expires it during the tick, draining the + // last pending lock and forcing the synced-state transition. + let events = manager.tick_at(past_ttl(), &requests).await.unwrap(); + + assert!(events.is_empty()); + assert_eq!(manager.pending_count(), 0); + assert_eq!(manager.state(), SyncState::Synced); + } + + /// A lock re-announced with the same txid after a disconnect must be + /// processed fresh, not silently dropped as a duplicate. `on_disconnect` has + /// to evict the unvalidated cache entry that `process_instantlock` dedupes + /// against, otherwise the re-announced lock is never re-queued/re-validated. + #[tokio::test] + async fn test_reannounce_after_disconnect_is_not_duplicate() { + let mut manager = create_test_manager(); + + let txid = Txid::from_byte_array([5u8; 32]); + + // First announcement: the empty engine can't verify it, so it is queued + // and cached as an unvalidated entry. + let events = manager.process_instantlock(&create_test_instantlock(txid)).await.unwrap(); + assert_eq!(events.len(), 1); + assert_eq!(manager.pending_count(), 1); + assert!(manager.get_instantlock(&txid).is_some()); + + // Disconnect must clear the queue AND the unvalidated cache entry. + manager.on_disconnect(); + assert_eq!(manager.pending_count(), 0); + assert!(manager.get_instantlock(&txid).is_none()); + + // Re-announcement of the same txid is processed fresh, not deduped away. + let events = manager.process_instantlock(&create_test_instantlock(txid)).await.unwrap(); + assert_eq!(events.len(), 1); + assert_eq!(manager.pending_count(), 1); + } + + /// A pending lock past its [`PENDING_TTL`] is dropped and counted invalid, + /// even when the engine has also advanced. Expiry is handled by the cheap + /// advancement-independent pass, which drains the queue before the + /// validation gate — so `validate_pending` never runs and the marker is left + /// untouched (it is meaningless with no pending locks anyway). + #[tokio::test] + async fn test_pending_expires_after_ttl_on_revalidation() { + let mut manager = create_test_manager(); + let requests = no_op_requests(); + + manager.pending_instantlocks.push(fresh_pending(Txid::from_byte_array([9u8; 32]))); + + // The lock is past its TTL (driven by the injected `now`) and still + // unverifiable, so it is dropped and counted invalid regardless of the + // engine advancing to height 300. + advance_engine_height(&manager, 300).await; + let events = manager.tick_at(past_ttl(), &requests).await.unwrap(); + + assert!(events.is_empty()); + assert_eq!(manager.pending_count(), 0); + // Expiry drained the queue before the validation gate, so the marker was + // never set by a validation pass. + assert_eq!(manager.last_validated_engine_height, None); + if let SyncManagerProgress::InstantSend(progress) = manager.progress() { + assert_eq!(progress.invalid(), 1); + } else { + panic!("Expected SyncManagerProgress::InstantSend"); + } + } + + /// `on_disconnect` clears queued locks and resets the re-validation marker so + /// a lock re-received after reconnect is validated fresh. + #[tokio::test] + async fn test_on_disconnect_resets_validation_marker() { + let mut manager = create_test_manager(); + + advance_engine_height(&manager, 100).await; + manager.last_validated_engine_height = Some(100); + manager.pending_instantlocks.push(fresh_pending(Txid::from_byte_array([1u8; 32]))); + + manager.on_disconnect(); + + assert_eq!(manager.pending_count(), 0); + assert_eq!(manager.last_validated_engine_height, None); + } } diff --git a/dash-spv/src/sync/instantsend/sync_manager.rs b/dash-spv/src/sync/instantsend/sync_manager.rs index e9ee4e96e..c8aa2b7cf 100644 --- a/dash-spv/src/sync/instantsend/sync_manager.rs +++ b/dash-spv/src/sync/instantsend/sync_manager.rs @@ -6,6 +6,7 @@ use crate::sync::{ use async_trait::async_trait; use dashcore::network::message::NetworkMessage; use dashcore::network::message_blockdata::Inventory; +use std::time::Instant; #[async_trait] impl SyncManager for InstantSendManager { @@ -26,7 +27,14 @@ impl SyncManager for InstantSendManager { } fn on_disconnect(&mut self) { - self.pending_instantlocks.clear(); + // Clear the queue *and* drop unvalidated cache entries. Leaving the + // latter behind would make `process_instantlock` treat a lock + // re-announced after reconnect as a duplicate, so it would never be + // re-queued or re-validated. + self.clear_unvalidated_on_disconnect(); + // Nothing is queued anymore, so the height at which we last validated is + // meaningless; reset it so re-validation runs again after reconnect. + self.last_validated_engine_height = None; } async fn handle_message( @@ -81,18 +89,13 @@ impl SyncManager for InstantSendManager { "Masternode state updated, validating {} pending InstantLocks", pending ); - self.validate_pending().await? + self.validate_pending(Instant::now()).await? } else { vec![] }; - // Transition to Synced when no pending validations after masternode sync - if self.pending_count() == 0 - && matches!(self.state(), SyncState::Syncing | SyncState::WaitForEvents) - { - self.set_state(SyncState::Synced); - tracing::info!("InstantSend manager synced (no pending validations)"); - } + // Transition to Synced when no pending validations after masternode sync. + self.transition_synced_if_idle(); return Ok(events); } @@ -100,13 +103,67 @@ impl SyncManager for InstantSendManager { Ok(vec![]) } - async fn tick(&mut self, _requests: &RequestSender) -> SyncResult> { - // Prune old entries periodically - self.prune_old_entries(); - Ok(vec![]) + async fn tick(&mut self, requests: &RequestSender) -> SyncResult> { + self.tick_at(Instant::now(), requests).await } fn progress(&self) -> SyncManagerProgress { SyncManagerProgress::InstantSend(self.progress.clone()) } } + +impl InstantSendManager { + /// `tick` with the current time injected, so tests can drive TTL expiry with + /// a future instant instead of back-dating `first_seen` (an `Instant` cannot + /// portably be back-dated by subtraction — `PendingInstantLock::is_expired` + /// explains why). The trait `tick` passes `Instant::now()`. + pub(super) async fn tick_at( + &mut self, + now: Instant, + _requests: &RequestSender, + ) -> SyncResult> { + // Prune old entries periodically + self.prune_old_entries(); + + // Whether there was work to resolve at the start of this tick. Guards the + // synced-state transition below so a freshly created manager sitting idle + // in `WaitForEvents` isn't marked `Synced` without ever having done work. + let had_pending = !self.pending_instantlocks.is_empty(); + + // Expire pending locks that have outlived their TTL. This runs on every + // tick, independent of engine advancement — otherwise, with a static + // engine height, `validate_pending` (the only other place expiry runs) + // would never be reached and a lock could stay pending forever. + self.expire_pending(now); + + // Re-validate queued InstantLocks once the masternode engine has advanced + // past the height at which they were last checked. A lock received before + // quorum sync completed (e.g. right after a fresh SPV start, for a + // transaction the wallet just broadcast) is then verified as soon as the + // needed quorum data lands, rather than waiting for the next + // `MasternodeStateUpdated` event — which, for a just-broadcast + // transaction, may not arrive until the block that mines it. + let mut events = Vec::new(); + if !self.pending_instantlocks.is_empty() { + let engine_height = self.engine_height().await; + let advanced = match (engine_height, self.last_validated_engine_height) { + (Some(current), Some(last)) => current > last, + (Some(_), None) => true, + (None, _) => false, + }; + if advanced { + events = self.validate_pending(now).await?; + } + } + + // If this tick started with pending work that has now fully drained (via + // expiry or validation), mirror `handle_sync_event` and resolve the + // manager's state — otherwise a final lock resolved by tick rather than a + // `MasternodeStateUpdated` event would leave it stuck in `Syncing`. + if had_pending { + self.transition_synced_if_idle(); + } + + Ok(events) + } +} From a041f445657fee4160debc1eeebaee70b5a8726f Mon Sep 17 00:00:00 2001 From: bfoss765 <38437574+bfoss765@users.noreply.github.com> Date: Thu, 6 Aug 2026 05:26:51 -0400 Subject: [PATCH 063/149] =?UTF-8?q?fix(key-wallet):=20out-of-order=20UTXO?= =?UTF-8?q?=20spend=20recorded=20nowhere=20=E2=80=94=20phantom=20unspent?= =?UTF-8?q?=20balance=20(#649)=20(#909)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(key-wallet-manager): deterministic repro of out-of-order spend-before-funding (#649) A spend processed BEFORE the transaction that funded the UTXO it spends (out-of-order block delivery during a cold rescan) leaves that UTXO permanently in the wallet's tracked set, producing phantom spendable balance. Device evidence (testnet): output 2febe5d7e8ad1dd0fb633004a82a24783d9b2e9095883541576e5f1344eb9975:0 (1,000,000 duffs) is counted unspent by the SDK while dashj has it spent, and the phantom +0.01 survives a full wallet rebuild from seed (fresh re-derivation + rescan reproduces it deterministically), proving the miss lives in the scan/processing path, not just live mempool ingestion. This test models that scenario at the WalletManager level: fund 1,000,000 duffs to a wallet address, then deliver the spending block (height 200) BEFORE the funding block (height 100). It asserts the funding outpoint is NOT still tracked afterward. FAILS on the current pin (which already contains #837/#864/#891/#893) — those do not address this defect. Refs: dashpay/rust-dashcore#649 Co-Authored-By: Claude Fable 5 * fix(key-wallet): track wallet-level observed_spent_outpoints to fix #649 Root cause: the "already spent" guard in managed_core_funds_account.rs::update_utxos keys off the ACCOUNT-LOCAL `spent_outpoints` set, which is only populated when the account itself processes the spending transaction. When a spend is delivered before its funding tx (out-of-order rescan), the wallet does not yet own the input, so the spend is classified as irrelevant, update_utxos never runs for it, and nothing records the spend. When the funding tx is processed later, the output is (re-)inserted as a fresh, spendable UTXO -> phantom balance that survives a full from-seed rescan. Fix (adapted from dashpay/rust-dashcore#851): record every spend observed in a block into a new wallet-level `observed_spent_outpoints` map (ManagedWalletInfo), independent of the spending tx's classification or account attribution. update_utxos and record_transaction consult this map: - update_utxos skips any output already observed spent (spend-first ordering: funding arrives after the spend). - remove_spent_from_accounts drops a coin the matched-account path missed (funding-first ordering: spend routed to another account). - TransactionRecord::compensate_for_observed_spends keeps net_amount / output_details consistent with the observed spend (declarative, so it is idempotent across rescan replays). The set is bounded-permanent: entries are evicted by prune_finalized_observed_spends once the spend height is provably final (<= min(chainlock height, synced_height)); add-account rewinds the sync checkpoint so a late account gets filter coverage before pruning can run. A dash-spv commit-time contiguity guard keeps a mid-flight account-add rescan from being silently clobbered forward. Also fixes AddressPool::prune_unused to clear script_pubkey_index alongside address_index. Adds manager-level regression tests (multi-wallet, large-block stress) that exercise the fix through the public WalletManager API. The repro test from the previous commit now passes; key-wallet (549), key-wallet-manager (all) and dash-spv lib (482) suites are green. The pre-existing masternode-network integration failures (test_utils/masternode_network.rs:106) are unrelated and fail identically on the clean pin. Refs: dashpay/rust-dashcore#649 Adapted-from: dashpay/rust-dashcore#851 Co-Authored-By: Claude Fable 5 * test(key-wallet): cover observed_spent guard branches for #649 patch coverage Codecov flagged the #649 fix's previously-uncovered branches: the wallet-level `observed_spent_outpoints` serde adapter, finality-boundary pruning, the funding-first removal guard, the account-add sync rewind, and the AddressPool `script_pubkey_index` prune fix. The manager-level integration tests only drive the spend-first ordering end-to-end, leaving these reachable only from the crate-internal `pub(crate)` surface. Add `key-wallet/src/tests/observed_spent_outpoints_tests.rs` (the sibling file already referenced by observed_spent_large_block_stress_test.rs) with five white-box tests, plus one AddressPool prune test: - observed_spent_outpoints_survive_serde_round_trip: exercises the (OutPoint, height) sequence serde adapter (serialize + deserialize visitor) and the empty-map / `#[serde(default)]` path, isolated on an account-less wallet so the populated-account `script_pubkey_index` JSON-key blocker does not apply. - prune_finalized_observed_spends_respects_finality_boundary: no-op without a chainlock; otherwise evicts exactly entries at/below min(chainlock height, synced_height), keeping the rescan case (chainlock above sync checkpoint) from over-pruning. - funding_first_guard_removes_held_coin_and_compensates_record: the un-gated remove_spent_from_accounts / finalize_guard_removed_utxo path — coin dropped, reservation released, funding record compensated to net 0; idempotent; coinbase skipped. - wallet_level_set_outlives_account_local_reload: the account-local spent_outpoints derived set (rebuilt from recorded txs via simulate_reload_rebuild_spent_outpoints) forgets an unrecorded spend, but the persisted wallet-level set still prevents resurrection on funding re-delivery. - adding_account_from_xpub_rewinds_sync_checkpoint: standalone-account add collapses synced_height to birth_height - 1; a still-behind checkpoint is left untouched. - prune_unused_clears_script_pubkey_index (address_pool_tests.rs): regression guard for the missing script_pubkey_index.remove in AddressPool::prune_unused. key-wallet lib (554) and key-wallet-manager (all) suites green. Refs: dashpay/rust-dashcore#649 Co-Authored-By: Claude Fable 5 * review fixes: gate net_amount recompute, surface observed-spend state modification, tighten deser cap - compensate_for_observed_spends: only replace the match-derived net_amount when the compensation actually dropped an output detail, keeping the no-observed-spend path byte-identical to pre-#649 behavior; pinned by a new unit test. - record_observed_spends: report whether the persisted observed-spent map actually changed, and surface that as state_modified in check_core_transaction — a consumer persisting only on reported modifications must not lose a recorded spend across a restart; pinned by a new regression test (new spend reports, unchanged redelivery and mempool spends do not). - Replace a comment reference to a nonexistent test with the inline rationale for why input_details and account_match.sent populate together. - Tighten MAX_OBSERVED_SPENT_OUTPOINTS 10M -> 1M (load-time allocation cap from a few hundred MB to a few tens of MB), still far above any legitimate size. Co-Authored-By: Claude Fable 5 * review fixes round 2: generation guard for mid-flight account adds, surface guard-rewritten records, restore public account API Addresses three external review findings on top of PR #909: 1. [P1] The commit-time contiguity guard could still certify unscanned coverage for a newly added account: a rewind landing INSIDE a scanned batch's range passed the height check, and an account add that moved no heights (checkpoint already at the birth floor) was undetectable by any height comparison. ManagedWalletInfo now carries an in-memory account_generation counter bumped on every account add (even height-invisible ones); filter scan snapshots it per wallet and commit refuses to advance a wallet whose generation changed since scan. Pinned by three new dash-spv tests including the mid-batch rewind repro (9000 -> scan [5000..9999] -> rewind 7499 -> commit keeps 7499) and the unmoved-checkpoint case. 2. [P2] The funding-first guard rewrote funding records without ever surfacing them: remove_spent_from_accounts now returns post-compensation clones of every rewritten record, the checker adds them to updated_records on both the relevant and irrelevant paths, and the manager propagates updated_records independent of is_relevant, so consumers persisting per-record updates see the rewrite. 3. [P2] ManagedAccountRefMut::record_transaction/confirm_transaction had silently gone pub -> pub(crate) with changed signatures. The public methods are restored with their original signatures (recording with no observed-spend context, the pre-#649 behavior); the checker uses new pub(crate) *_with_observed_spends variants. Co-Authored-By: Claude Fable 5 * test(key-wallet): pin born-fully-spent recovered transactions stay in history (#649/#846) HashEngineering reported (against #851/#866) that a funding transaction recovered after its spend was already observed -- every wallet-relevant output already in observed_spent_outpoints before the funding is applied -- was dropped from history entirely: the spend-first path made it come out not-relevant, so no TransactionRecord and no detection event were produced, while balance and UTXO set stayed exact. That record-loss "belongs in #851"; #851 is superseded by #909. Investigation of #909 shows its core commit (ebcd40af) already implements the suggested remedy, so no behavior change is needed: - relevance in check_transaction_for_match is address-membership based and is never gated on spent-status, so a fully-spent funding tx is still classified relevant; - ManagedCoreFundsAccount::record_transaction unconditionally inserts the record after TransactionRecord::compensate_for_observed_spends zeroes the already-spent outputs (net 0, no UTXO); - the "never insert already-spent value" guard lives in update_utxos (UTXO insertion only), not in recording. The QuantumExplorer "surface updated records independent of relevance" review fix covers the separate funding-first UPDATE case (remove_spent_from_accounts rewriting an existing funding record); the born-fully-spent NEW-record insertion is covered independently by the record_transaction compensate path. The existing observed-spent tests assert only UTXO/balance, leaving the history-record guarantee uncovered. This adds that coverage: two tests pin that a born-fully-spent recovered tx -- a plain funding tx, and a CoinJoin- style intermediate hop that spends a live coin -- is surfaced as a new record and recorded in the account's transaction history, while balance and UTXOs stay at zero. Verified across InBlock and InChainLockedBlock (chainlocked recovery) contexts and the WalletManager block path during investigation. Co-Authored-By: Claude Fable 5 * docs(key-wallet): fix private intra-doc links so the Documentation CI job passes Co-Authored-By: Claude Fable 5 * test(key-wallet-manager): parameterize network in the spend_tx test helper CodeRabbit: the shared `spend_tx` helper hardcoded `Address::dummy(Network::Testnet, ..)`, forcing every observed-spend test onto Testnet (coding guideline: never hardcode network parameters in a shared helper). Add a `network: Network` parameter and thread it into `Address::dummy`; each caller passes the network its manager uses. All key-wallet-manager tests pass. Co-Authored-By: Claude Opus 4.8 * refactor(key-wallet): reduce #649 fix to the two-piece observed-spend mechanism Applies ZocoLini's requested simplification (PR #909 review): the #649 out-of-order-spend fix is the two-piece mechanism only — (a) record every input seen in a block-context tx into `observed_spent_outpoints`, independent of classification (`wallet_checker.rs`), and (b) in `update_utxos`, skip inserting an output whose outpoint is already in that map (`managed_core_funds_account.rs`). Removes the separate unattributable-spend compensation machinery that was bundled in, which also made transaction history order-dependent (a receive delivered before its spend was rewritten to a 0-value entry, erasing it from history — a spend delivered first leaves it intact): - `TransactionRecord::compensate_for_observed_spends` and its unit tests - `ManagedWalletInfo::remove_spent_from_accounts` (both call sites in `check_core_transaction`) and its `finalize_guard_removed_utxo` helper - the now-dead account-local helpers `mark_outpoint_spent`, `release_reservation_for`, and the test-only `simulate_reload_rebuild_spent_outpoints` - the `updated_records`-independent-of-relevance change in `WalletManager` (its only source was the removed funding-first guard) The `record_transaction_with_observed_spends` / `confirm_transaction_with_observed_spends` pair is kept: it is the plumbing that delivers the wallet-level observed map to `update_utxos`, i.e. piece (b) itself. A born-fully-spent funding tx is still recorded in history; its already-spent output is simply never (re-)tracked as a UTXO (balance/UTXO correctness comes from piece (b), not from rewriting the record). Tests updated to pin that the receive is preserved in history rather than erased. Test-helper cleanup (`key-wallet-manager/tests/common/mod.rs`): drop the superfluous `Network` parameter from `spend_tx` (every caller passed Testnet and the payee's network is irrelevant to observed-spend logic) and build the external payee script directly, so the helper needs no network at all. All three observed-spend integration tests (incl. the deterministic repro) and `cargo test -p key-wallet --lib` pass; workspace clippy (`-D warnings`, debug + release) and rustfmt are clean. Co-Authored-By: Claude Opus 4.8 * test(key-wallet-manager): name the out-of-order repro after the invariant it pins The test asserted `!still_tracked` — the funding UTXO must NOT remain in the tracked set once its spend was observed first — but was named `..._leaves_utxo_permanently_tracked`, i.e. after the #649 bug rather than after the pinned behaviour. A failure therefore read as the expected outcome. Rename to `..._does_not_leave_utxo_tracked`; assertions and scenario are unchanged. --------- Co-authored-by: Claude Fable 5 Co-authored-by: Quantum Explorer --- dash-spv/src/sync/filters/batch.rs | 28 +- dash-spv/src/sync/filters/manager.rs | 329 ++++++++++++- key-wallet-manager/src/process_block.rs | 4 + .../src/test_utils/mock_wallet.rs | 7 + key-wallet-manager/src/wallet_interface.rs | 11 + key-wallet-manager/tests/common/mod.rs | 87 ++++ .../observed_spent_large_block_stress_test.rs | 136 ++++++ .../tests/observed_spent_multi_wallet_test.rs | 67 +++ .../tests/out_of_order_spend_repro_test.rs | 81 ++++ .../src/managed_account/address_pool.rs | 1 + .../managed_account/managed_account_ref.rs | 58 ++- .../managed_core_funds_account.rs | 72 ++- key-wallet/src/tests/address_pool_tests.rs | 42 ++ key-wallet/src/tests/mod.rs | 2 + .../tests/observed_spent_outpoints_tests.rs | 444 ++++++++++++++++++ .../transaction_checking/wallet_checker.rs | 63 ++- .../managed_wallet_info/managed_accounts.rs | 18 + .../src/wallet/managed_wallet_info/mod.rs | 279 ++++++++++- .../wallet_info_interface.rs | 22 + 19 files changed, 1702 insertions(+), 49 deletions(-) create mode 100644 key-wallet-manager/tests/common/mod.rs create mode 100644 key-wallet-manager/tests/observed_spent_large_block_stress_test.rs create mode 100644 key-wallet-manager/tests/observed_spent_multi_wallet_test.rs create mode 100644 key-wallet-manager/tests/out_of_order_spend_repro_test.rs create mode 100644 key-wallet/src/tests/observed_spent_outpoints_tests.rs diff --git a/dash-spv/src/sync/filters/batch.rs b/dash-spv/src/sync/filters/batch.rs index 386e0d16e..b389c5a1b 100644 --- a/dash-spv/src/sync/filters/batch.rs +++ b/dash-spv/src/sync/filters/batch.rs @@ -1,7 +1,7 @@ use dashcore::bip158::BlockFilter; use dashcore::ScriptBuf; use key_wallet_manager::{FilterMatchKey, WalletId}; -use std::collections::{BTreeSet, HashMap, HashSet}; +use std::collections::{BTreeMap, HashMap, HashSet}; /// A completed batch of compact block filters ready for verification. /// @@ -24,10 +24,15 @@ pub(super) struct FiltersBatch { pending_blocks: u32, /// Whether rescan has been completed for this batch. rescan_complete: bool, - /// Wallets that were behind for this batch's height range at scan time and - /// therefore need their `synced_height` advanced when the batch commits. - /// Already-synced wallets must not be touched. - scanned_wallets: BTreeSet, + /// Wallets that were behind for this batch's height range at scan time — + /// and therefore need their `synced_height` advanced when the batch + /// commits — each mapped to the wallet's `account_generation` at scan + /// time. Commit refuses to advance a wallet whose generation has changed + /// since the scan: an account added mid-flight means the scan did not test + /// the new account's scripts, so the batch cannot certify coverage for the + /// current account set (dashpay/rust-dashcore#649). Already-synced wallets + /// must not be touched. + scanned_wallets: BTreeMap, /// Cached scriptPubKeys discovered during block processing that still /// need rescan, attributed per wallet so we can rerun matching only /// against the wallet that produced each new script. @@ -49,7 +54,7 @@ impl FiltersBatch { scanned: false, pending_blocks: 0, rescan_complete: false, - scanned_wallets: BTreeSet::new(), + scanned_wallets: BTreeMap::new(), collected_scripts: HashMap::new(), } } @@ -118,13 +123,14 @@ impl FiltersBatch { pub(super) fn take_collected_scripts(&mut self) -> HashMap> { std::mem::take(&mut self.collected_scripts) } - /// Record the set of wallets that were behind for this batch at scan time. - pub(super) fn set_scanned_wallets(&mut self, wallets: BTreeSet) { + /// Record the wallets that were behind for this batch at scan time, each + /// with its `account_generation` snapshot. + pub(super) fn set_scanned_wallets(&mut self, wallets: BTreeMap) { self.scanned_wallets = wallets; } - /// Wallets that were behind at scan time and must have their synced_height - /// advanced when this batch commits. - pub(super) fn scanned_wallets(&self) -> &BTreeSet { + /// Wallets that were behind at scan time (with their generation snapshot) + /// and must have their synced_height advanced when this batch commits. + pub(super) fn scanned_wallets(&self) -> &BTreeMap { &self.scanned_wallets } } diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index 376a80c8c..005ca99dd 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -574,8 +574,26 @@ impl= batch_start { + wallet.update_wallet_synced_height(wallet_id, end); + } } } } @@ -821,15 +839,20 @@ impl = behind.clone(); + // batch. Each wallet's `account_generation` is snapshotted under the + // same read lock as the behind set: commit refuses to advance a wallet + // whose account set changed after this scan + // (dashpay/rust-dashcore#649). + let scanned_wallets: BTreeMap = behind + .iter() + .map(|wallet_id| (*wallet_id, wallet.wallet_account_generation(wallet_id))) + .collect(); + drop(wallet); if let Some(batch) = self.active_batches.get_mut(&batch_start) { batch.set_scanned_wallets(scanned_wallets); @@ -1322,7 +1345,7 @@ mod tests { batch1.set_pending_blocks(0); batch1.mark_scanned(); batch1.mark_rescan_complete(); - batch1.set_scanned_wallets(BTreeSet::from([MOCK_WALLET_ID])); + batch1.set_scanned_wallets(BTreeMap::from([(MOCK_WALLET_ID, 0)])); manager.active_batches.insert(0, batch1); manager.try_commit_batches().await.unwrap(); @@ -1364,7 +1387,7 @@ mod tests { batch.set_pending_blocks(0); batch.mark_scanned(); batch.mark_rescan_complete(); - batch.set_scanned_wallets(BTreeSet::from([wallet_a])); + batch.set_scanned_wallets(BTreeMap::from([(wallet_a, 0)])); manager.active_batches.insert(0, batch); manager.try_commit_batches().await.unwrap(); @@ -1373,6 +1396,286 @@ mod tests { assert_eq!(multi.read().await.wallet_synced_height(&wallet_b), 0); } + /// Contiguity guard (dashpay/rust-dashcore#649): a batch scanned before an + /// account-add rewinds the wallet's checkpoint must NOT clobber the rewound + /// value forward at commit time — otherwise the account-addition rescan is + /// silently cancelled. The wallet stays behind and the tick picks it up. + #[tokio::test] + async fn mid_flight_account_add_does_not_clobber_rescan_floor() { + let wallet_a: WalletId = [0xAA; 32]; + let multi = Arc::new(RwLock::new(MultiMockWallet::new())); + { + let mut w = multi.write().await; + // Already synced up to 4999, so the next batch legitimately starts at 5000. + w.insert_wallet( + wallet_a, + MockWalletState { + synced_height: 4999, + ..MockWalletState::default() + }, + ); + } + let mut manager = create_multi_test_manager(multi.clone()).await; + manager.set_state(SyncState::Syncing); + + // Batch [5000..9999] scanned with wallet_a recorded, ready to commit. + let mut batch = FiltersBatch::new(5000, 9999, HashMap::new()); + batch.set_pending_blocks(0); + batch.mark_scanned(); + batch.mark_rescan_complete(); + batch.set_scanned_wallets(BTreeMap::from([(wallet_a, 0)])); + manager.active_batches.insert(5000, batch); + + // Simulate an account being added mid-flight: the wallet's checkpoint is + // rewound to just below birth, far below this batch's start. + multi.write().await.wallet_mut(&wallet_a).synced_height = 49; + + manager.try_commit_batches().await.unwrap(); + + // committed_height still advances (chain progress), but the wallet's + // checkpoint is NOT dragged forward over the gap it must rescan. + assert_eq!(manager.progress.committed_height(), 9999); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_a), + 49, + "the rewound checkpoint must survive commit — the batch is non-contiguous with it" + ); + // The wallet is still behind, so the next tick rescans it. + assert!( + multi.read().await.wallets_behind(9999).contains(&wallet_a), + "the wallet remains behind and is picked up by the tick rescan" + ); + } + + /// The contiguity guard is transparent in normal operation: contiguous + /// ascending batches advance each wallet's checkpoint exactly as before. + #[tokio::test] + async fn contiguity_guard_permits_normal_advance() { + let wallet_a: WalletId = [0xCC; 32]; + let multi = Arc::new(RwLock::new(MultiMockWallet::new())); + multi.write().await.insert_wallet(wallet_a, MockWalletState::default()); + let mut manager = create_multi_test_manager(multi.clone()).await; + manager.set_state(SyncState::Syncing); + + // First batch starts at 0 = synced_height (0) + ... the wallet is fresh, + // so this batch is contiguous and advances the checkpoint. + let mut batch1 = FiltersBatch::new(0, 4999, HashMap::new()); + batch1.set_pending_blocks(0); + batch1.mark_scanned(); + batch1.mark_rescan_complete(); + batch1.set_scanned_wallets(BTreeMap::from([(wallet_a, 0)])); + manager.active_batches.insert(0, batch1); + + manager.try_commit_batches().await.unwrap(); + assert_eq!(multi.read().await.wallet_synced_height(&wallet_a), 4999); + + // Second batch starts exactly at synced_height + 1 = 5000: still + // contiguous, so it advances too. + let mut batch2 = FiltersBatch::new(5000, 9999, HashMap::new()); + batch2.set_pending_blocks(0); + batch2.mark_scanned(); + batch2.mark_rescan_complete(); + batch2.set_scanned_wallets(BTreeMap::from([(wallet_a, 0)])); + manager.active_batches.insert(5000, batch2); + + manager.try_commit_batches().await.unwrap(); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_a), + 9999, + "contiguous ascending batches advance the checkpoint unchanged by the guard" + ); + } + + /// Marvin QA-001 round 2: two wallets in the SAME batch, only one of them + /// rewound mid-flight (simulating an account add on wallet_a only). The + /// contiguity guard must block wallet_a's advance while still advancing + /// wallet_b normally — a per-wallet leak here would either strand a + /// healthy wallet or silently clobber the rewound one. + #[tokio::test] + async fn contiguity_guard_is_per_wallet_not_batch_wide() { + let wallet_a: WalletId = [0xAA; 32]; + let wallet_b: WalletId = [0xBB; 32]; + let multi = Arc::new(RwLock::new(MultiMockWallet::new())); + { + let mut w = multi.write().await; + w.insert_wallet( + wallet_a, + MockWalletState { + synced_height: 4999, + ..MockWalletState::default() + }, + ); + w.insert_wallet( + wallet_b, + MockWalletState { + synced_height: 4999, + ..MockWalletState::default() + }, + ); + } + let mut manager = create_multi_test_manager(multi.clone()).await; + manager.set_state(SyncState::Syncing); + + // Batch [5000..9999] scanned with BOTH wallets recorded. + let mut batch = FiltersBatch::new(5000, 9999, HashMap::new()); + batch.set_pending_blocks(0); + batch.mark_scanned(); + batch.mark_rescan_complete(); + batch.set_scanned_wallets(BTreeMap::from([(wallet_a, 0), (wallet_b, 0)])); + manager.active_batches.insert(5000, batch); + + // Only wallet_a gets an account added mid-flight (rewound). wallet_b + // is untouched and remains legitimately contiguous with this batch. + multi.write().await.wallet_mut(&wallet_a).synced_height = 49; + + manager.try_commit_batches().await.unwrap(); + + assert_eq!(manager.progress.committed_height(), 9999); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_a), + 49, + "wallet_a's rewound checkpoint must survive commit" + ); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_b), + 9999, + "wallet_b was legitimately contiguous and must still advance despite \ + sharing a batch with a rewound wallet" + ); + } + + /// Generation guard, mid-batch rewind (Codex QA round 3): batch + /// [5000..9999] is scanned while the wallet sits at 9000, then an account + /// add rewinds the checkpoint to 7499 — INSIDE the batch's range. The + /// height-only contiguity check (7500 >= 5000) cannot see anything wrong, + /// but heights 7500..=9999 were scanned without the new account's scripts + /// (and 7500..=9000 were not attributed to this wallet at all), so the + /// commit must not certify them. The `account_generation` snapshot taken + /// at scan time is what blocks it (dashpay/rust-dashcore#649). + #[tokio::test] + async fn account_add_rewind_inside_batch_range_does_not_certify_unscanned_gap() { + let wallet_a: WalletId = [0xAA; 32]; + let multi = Arc::new(RwLock::new(MultiMockWallet::new())); + multi.write().await.insert_wallet( + wallet_a, + MockWalletState { + synced_height: 9000, + ..MockWalletState::default() + }, + ); + let mut manager = create_multi_test_manager(multi.clone()).await; + manager.set_state(SyncState::Syncing); + + // Batch [5000..9999] scanned while the wallet's generation was 0. + let mut batch = FiltersBatch::new(5000, 9999, HashMap::new()); + batch.set_pending_blocks(0); + batch.mark_scanned(); + batch.mark_rescan_complete(); + batch.set_scanned_wallets(BTreeMap::from([(wallet_a, 0)])); + manager.active_batches.insert(5000, batch); + + // Account added mid-flight: checkpoint rewound to birth-1 = 7499, + // generation bumped — both done by the same rewind in key-wallet. + { + let mut w = multi.write().await; + let state = w.wallet_mut(&wallet_a); + state.synced_height = 7499; + state.account_generation = 1; + } + + manager.try_commit_batches().await.unwrap(); + + assert_eq!(manager.progress.committed_height(), 9999); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_a), + 7499, + "a rewind INSIDE the batch range must survive commit: 7500..=9999 were \ + scanned without the new account's scripts" + ); + assert!( + multi.read().await.wallets_behind(9999).contains(&wallet_a), + "the wallet remains behind and is picked up by the tick rescan" + ); + } + + /// Generation guard, height-invisible account add: the wallet still sits + /// at its birth floor (999) when batch [1000..5999] is scanned, an account + /// is added (the rewind moves nothing — the checkpoint is already at the + /// floor), and the batch then commits. No height-based check can detect + /// this: the batch is perfectly contiguous with the unchanged checkpoint, + /// yet it was scanned without the new account's scripts. Only the + /// generation mismatch blocks the advance (dashpay/rust-dashcore#649). + #[tokio::test] + async fn account_add_with_unmoved_checkpoint_still_blocks_commit_advance() { + let wallet_a: WalletId = [0xAA; 32]; + let multi = Arc::new(RwLock::new(MultiMockWallet::new())); + multi.write().await.insert_wallet( + wallet_a, + MockWalletState { + synced_height: 999, + ..MockWalletState::default() + }, + ); + let mut manager = create_multi_test_manager(multi.clone()).await; + manager.set_state(SyncState::Syncing); + + let mut batch = FiltersBatch::new(1000, 5999, HashMap::new()); + batch.set_pending_blocks(0); + batch.mark_scanned(); + batch.mark_rescan_complete(); + batch.set_scanned_wallets(BTreeMap::from([(wallet_a, 0)])); + manager.active_batches.insert(1000, batch); + + // Account added mid-flight; the checkpoint is already at the birth + // floor so ONLY the generation moves. + multi.write().await.wallet_mut(&wallet_a).account_generation = 1; + + manager.try_commit_batches().await.unwrap(); + + assert_eq!(manager.progress.committed_height(), 5999); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_a), + 999, + "an account add that moved no heights must still block the advance — \ + the batch was scanned without the new account's scripts" + ); + } + + /// A generation bump after commit does not retroactively matter, and an + /// unchanged generation lets contiguous batches advance exactly as before: + /// the guard is transparent in normal operation. + #[tokio::test] + async fn generation_guard_transparent_when_generation_unchanged() { + let wallet_a: WalletId = [0xDD; 32]; + let multi = Arc::new(RwLock::new(MultiMockWallet::new())); + multi.write().await.insert_wallet( + wallet_a, + MockWalletState { + // A wallet whose generation is already nonzero (accounts were + // added before this scan): the snapshot captures the CURRENT + // value, so commit still advances. + account_generation: 3, + ..MockWalletState::default() + }, + ); + let mut manager = create_multi_test_manager(multi.clone()).await; + manager.set_state(SyncState::Syncing); + + let mut batch = FiltersBatch::new(0, 4999, HashMap::new()); + batch.set_pending_blocks(0); + batch.mark_scanned(); + batch.mark_rescan_complete(); + batch.set_scanned_wallets(BTreeMap::from([(wallet_a, 3)])); + manager.active_batches.insert(0, batch); + + manager.try_commit_batches().await.unwrap(); + assert_eq!( + multi.read().await.wallet_synced_height(&wallet_a), + 4999, + "matching generation snapshot advances the checkpoint unchanged by the guard" + ); + } + /// `scan_batch` with two wallets at different `synced_height` values: /// only the wallet whose synced_height is below the matching block's /// height should be attributed. @@ -1394,6 +1697,7 @@ mod tests { addresses: vec![address_low.clone()], synced_height: 10, last_processed_height: 10, + account_generation: 0, }, ); // wallet_high is mostly synced: synced_height=50, only sees > 50. @@ -1403,6 +1707,7 @@ mod tests { addresses: vec![address_high.clone()], synced_height: 50, last_processed_height: 50, + account_generation: 0, }, ); } @@ -1576,6 +1881,7 @@ mod tests { addresses: vec![], synced_height: 20, last_processed_height: 20, + account_generation: 0, }, ); w.insert_wallet( @@ -1584,6 +1890,7 @@ mod tests { addresses: vec![], synced_height: 60, last_processed_height: 60, + account_generation: 0, }, ); } @@ -1696,6 +2003,7 @@ mod tests { addresses: vec![address.clone()], synced_height: 0, last_processed_height: 0, + account_generation: 0, }, ); } @@ -1748,6 +2056,7 @@ mod tests { addresses: vec![address.clone()], synced_height: 0, last_processed_height: 0, + account_generation: 0, }, ); } @@ -1793,6 +2102,7 @@ mod tests { addresses: vec![address.clone()], synced_height: 0, last_processed_height: 0, + account_generation: 0, }, ); w.insert_wallet( @@ -1801,6 +2111,7 @@ mod tests { addresses: vec![address.clone()], synced_height: 0, last_processed_height: 0, + account_generation: 0, }, ); } @@ -1933,6 +2244,7 @@ mod tests { addresses: vec![address_a.clone()], synced_height: 0, last_processed_height: 0, + account_generation: 0, }, ); w.insert_wallet( @@ -1941,6 +2253,7 @@ mod tests { addresses: vec![address_b.clone()], synced_height: 0, last_processed_height: 0, + account_generation: 0, }, ); } diff --git a/key-wallet-manager/src/process_block.rs b/key-wallet-manager/src/process_block.rs index b5aabca86..11db8aeef 100644 --- a/key-wallet-manager/src/process_block.rs +++ b/key-wallet-manager/src/process_block.rs @@ -274,6 +274,10 @@ impl WalletInterface for WalletM self.wallet_infos.get(wallet_id).map(|info| info.synced_height()).unwrap_or(0) } + fn wallet_account_generation(&self, wallet_id: &WalletId) -> u64 { + self.wallet_infos.get(wallet_id).map(|info| info.account_generation()).unwrap_or(0) + } + fn update_wallet_synced_height(&mut self, wallet_id: &WalletId, height: CoreBlockHeight) { if let Some(info) = self.wallet_infos.get_mut(wallet_id) { if height > info.synced_height() { diff --git a/key-wallet-manager/src/test_utils/mock_wallet.rs b/key-wallet-manager/src/test_utils/mock_wallet.rs index 21ee92d0c..c81649436 100644 --- a/key-wallet-manager/src/test_utils/mock_wallet.rs +++ b/key-wallet-manager/src/test_utils/mock_wallet.rs @@ -380,6 +380,9 @@ pub struct MockWalletState { pub addresses: Vec
, pub synced_height: CoreBlockHeight, pub last_processed_height: CoreBlockHeight, + /// Mirrors `ManagedWalletInfo::account_generation`: bumped by tests to + /// simulate an account being added to this wallet mid-flight. + pub account_generation: u64, } /// Multi-wallet mock that holds independent state for several wallet IDs, @@ -492,6 +495,10 @@ impl WalletInterface for MultiMockWallet { self.wallets.get(wallet_id).map(|s| s.synced_height).unwrap_or(0) } + fn wallet_account_generation(&self, wallet_id: &WalletId) -> u64 { + self.wallets.get(wallet_id).map(|s| s.account_generation).unwrap_or(0) + } + fn update_wallet_synced_height(&mut self, wallet_id: &WalletId, height: CoreBlockHeight) { if let Some(state) = self.wallets.get_mut(wallet_id) { if height > state.synced_height { diff --git a/key-wallet-manager/src/wallet_interface.rs b/key-wallet-manager/src/wallet_interface.rs index 754ece0cf..3b175d9fe 100644 --- a/key-wallet-manager/src/wallet_interface.rs +++ b/key-wallet-manager/src/wallet_interface.rs @@ -137,6 +137,17 @@ pub trait WalletInterface: Send + Sync + 'static { /// Return the per-wallet committed sync checkpoint, or `0` if unknown. fn wallet_synced_height(&self, wallet_id: &WalletId) -> CoreBlockHeight; + /// Return the generation of one wallet's account set — a counter bumped + /// whenever an account is added to that wallet (`0` if unknown). Filter + /// sync snapshots this per wallet when scanning a range and refuses to + /// advance `wallet_synced_height` at commit time if it changed in between: + /// the scan did not test the added account's scripts, so it cannot certify + /// coverage for the current account set (dashpay/rust-dashcore#649). The + /// default (constant `0`) opts an implementation out of the check. + fn wallet_account_generation(&self, _wallet_id: &WalletId) -> u64 { + 0 + } + /// Advance one wallet's committed sync checkpoint. Implementations must /// only advance forward (a value below the current is silently ignored). fn update_wallet_synced_height(&mut self, wallet_id: &WalletId, height: CoreBlockHeight); diff --git a/key-wallet-manager/tests/common/mod.rs b/key-wallet-manager/tests/common/mod.rs new file mode 100644 index 000000000..7acc7f9b8 --- /dev/null +++ b/key-wallet-manager/tests/common/mod.rs @@ -0,0 +1,87 @@ +//! Shared helpers for the observed-spent-outpoint guard integration tests +//! (dashpay/rust-dashcore#649). +//! +//! Each test binary pulls this in via `mod common;` and uses a subset of the +//! helpers, so unused-item warnings are expected per binary and silenced here. +#![allow(dead_code)] + +use dashcore::blockdata::block::Block; +use dashcore::blockdata::transaction::OutPoint; +use dashcore::{Address, ScriptBuf, Transaction, TxIn, TxOut, Witness}; +use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; +use key_wallet_manager::{WalletId, WalletInterface, WalletManager}; +use std::collections::BTreeSet; + +/// Deliver `block` at `height` to the given `wallets`. +pub async fn process_block_for( + manager: &mut WalletManager, + block: &Block, + height: u32, + wallets: &BTreeSet, +) { + manager.process_block_for_wallets(block, block.block_hash(), height, wallets).await; +} + +/// Deliver `block` at `height` to every wallet the manager holds. +pub async fn process_block_all_wallets( + manager: &mut WalletManager, + block: &Block, + height: u32, +) { + let wallet_ids: BTreeSet = manager.list_wallets().into_iter().copied().collect(); + process_block_for(manager, block, height, &wallet_ids).await; +} + +/// A transaction paying `value` to `address` from one synthetic, unrelated +/// input (`input_seed` makes its txid deterministic and distinct). Only the +/// transaction's own txid/vout matter to the tests, as the coin later spent. +pub fn funding_tx(address: &Address, value: u64, input_seed: u8) -> Transaction { + Transaction { + version: 2, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint::new(dashcore::Txid::from([input_seed; 32]), 0), + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: Witness::new(), + }], + output: vec![TxOut { + value, + script_pubkey: address.script_pubkey(), + }], + special_transaction_payload: None, + } +} + +/// A transaction spending `outpoint` and paying `value` to an unrelated +/// external payee (`ext_id` selects a distinct script). +/// +/// The payee script is built directly (a P2PKH shape seeded by `ext_id`) rather +/// than via `Address::dummy`, so the helper needs no `Network`: the +/// observed-spend logic keys on the input outpoint, never on the payee, so the +/// payee's network is irrelevant to every scenario here. `Transaction::dummy` +/// does not fit this helper — it derives its inputs from an id range and cannot +/// spend a caller-specified `outpoint`, which is the whole point of a spend. +pub fn spend_tx(outpoint: OutPoint, value: u64, ext_id: usize) -> Transaction { + // OP_DUP OP_HASH160 <20-byte push> OP_EQUALVERIFY OP_CHECKSIG — a well-formed + // P2PKH scriptPubKey whose hash160 is seeded by `ext_id` so each payee is + // distinct and never collides with a wallet-owned script. + let mut payee = vec![0x76, 0xa9, 0x14]; + payee.extend_from_slice(&[ext_id as u8; 20]); + payee.extend_from_slice(&[0x88, 0xac]); + Transaction { + version: 2, + lock_time: 0, + input: vec![TxIn { + previous_output: outpoint, + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: Witness::new(), + }], + output: vec![TxOut { + value, + script_pubkey: ScriptBuf::from_bytes(payee), + }], + special_transaction_payload: None, + } +} diff --git a/key-wallet-manager/tests/observed_spent_large_block_stress_test.rs b/key-wallet-manager/tests/observed_spent_large_block_stress_test.rs new file mode 100644 index 000000000..3ca2e170a --- /dev/null +++ b/key-wallet-manager/tests/observed_spent_large_block_stress_test.rs @@ -0,0 +1,136 @@ +//! Adversarial stress test for the observed-spent-outpoint guard +//! (dashpay/rust-dashcore#649) at the *single-block* granularity, as opposed +//! to spreading growth across many small blocks. +//! +//! `process_block_for_wallets` (`key-wallet-manager/src/process_block.rs`) +//! iterates every transaction in `block.txdata` and calls +//! `check_transaction_in_wallets` for EACH ONE, unconditionally — a matched +//! block is delivered whole once ANY of its transactions matches a wallet's +//! compact filter. `ManagedWalletInfo::check_core_transaction` +//! (`key-wallet/src/transaction_checking/wallet_checker.rs`) then records +//! every input of every transaction it sees in block context into +//! `observed_spent_outpoints`, whether or not the transaction is relevant to the +//! wallet — so a single matched block grows the map by one entry per input over +//! the WHOLE block, not just the relevant tx. +//! +//! This test constructs a single `Block` with 5,000 unrelated 3-input +//! transactions plus one wallet-relevant spend, and verifies both correctness +//! (the bug is still caught when the relevant tx is buried in a large noisy +//! block) and the actual growth/cost this produces. + +mod common; + +use common::process_block_all_wallets; +use dashcore::blockdata::block::Block; +use dashcore::blockdata::transaction::OutPoint; +use dashcore::{Network, ScriptBuf, Transaction, TxIn, TxOut, Witness}; +use key_wallet::wallet::initialization::WalletAccountCreationOptions; +use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; +use key_wallet_manager::WalletManager; +use std::time::Instant; + +const NOISE_TXS_PER_BLOCK: usize = 5_000; +const INPUTS_PER_NOISE_TX: usize = 3; + +/// An unrelated 3-input transaction paying an external address, with inputs +/// deterministically derived from `seed` so every one is distinct and none +/// resolves to any outpoint the wallet will ever fund. +fn noise_tx(seed: u32) -> Transaction { + let input = (0..INPUTS_PER_NOISE_TX as u32) + .map(|v| { + let mut bytes = [0u8; 32]; + bytes[..4].copy_from_slice(&seed.to_le_bytes()); + bytes[4] = v as u8; + TxIn { + previous_output: OutPoint::new(dashcore::Txid::from(bytes), v), + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: Witness::new(), + } + }) + .collect(); + Transaction { + version: 2, + lock_time: 0, + input, + output: vec![TxOut { + value: 1_000, + script_pubkey: dashcore::Address::dummy(Network::Testnet, (seed % 90) as usize + 1) + .script_pubkey(), + }], + special_transaction_payload: None, + } +} + +#[tokio::test] +async fn wallet_relevant_spend_buried_in_large_noisy_block_still_caught() { + let mut manager = WalletManager::::new(Network::Testnet); + let wallet_id = manager + .create_wallet_with_random_mnemonic(WalletAccountCreationOptions::Default) + .expect("failed to create wallet"); + let funding_address = + manager.monitored_addresses().first().cloned().expect("wallet must have addresses"); + + let funding_value = 1_000_000u64; + let funding_tx = common::funding_tx(&funding_address, funding_value, 0xAB); + let funding_outpoint = OutPoint::new(funding_tx.txid(), 0); + + let spend_tx = common::spend_tx(funding_outpoint, funding_value - 1_000, 99); + + // One large block: 5,000 unrelated noise transactions with the single + // wallet-relevant spend buried in the middle — this is the realistic + // shape of "a matched block" (dash-spv delivers the whole block once any + // tx in it matches the filter), not 5,000 separate block deliveries. + let mut txdata: Vec = (0..NOISE_TXS_PER_BLOCK as u32).map(noise_tx).collect(); + let mid = txdata.len() / 2; + txdata.insert(mid, spend_tx.clone()); + let expected_inputs_in_spend_block = + NOISE_TXS_PER_BLOCK * INPUTS_PER_NOISE_TX + 1 /* spend_tx's own input */; + + let spend_block = Block::dummy(200, txdata); + + let start = Instant::now(); + process_block_all_wallets(&mut manager, &spend_block, 200).await; + let large_block_elapsed = start.elapsed(); + + let funding_block = Block::dummy(100, vec![funding_tx.clone()]); + process_block_all_wallets(&mut manager, &funding_block, 100).await; + + let utxos_after = manager.wallet_utxos(&wallet_id).expect("wallet must be registered"); + let still_tracked = utxos_after.iter().any(|u| u.outpoint == funding_outpoint); + assert!( + !still_tracked, + "BUG #649 regression: the relevant spend buried among {NOISE_TXS_PER_BLOCK} unrelated \ + transactions in the same block was not correctly reconciled against its \ + out-of-order funding" + ); + + let observed_len = + manager.get_wallet_info(&wallet_id).expect("wallet info").observed_spent_outpoints().len(); + let expected_total = expected_inputs_in_spend_block + 1 /* funding_tx's own input */; + assert_eq!( + observed_len, expected_total, + "a single matched block with {NOISE_TXS_PER_BLOCK} unrelated txs records one \ + observed-spent entry per input in the WHOLE block, not just the relevant tx — \ + set grew to {observed_len} entries from ONE block, confirming growth scales with \ + transactions-per-block, not with block count" + ); + + eprintln!( + "single block with {} txs ({} inputs) processed in {:?} ({:.1} us/input); \ + observed_spent_outpoints now holds {} entries after ONE block", + NOISE_TXS_PER_BLOCK + 1, + expected_inputs_in_spend_block, + large_block_elapsed, + large_block_elapsed.as_micros() as f64 / expected_inputs_in_spend_block as f64, + observed_len, + ); + + // Per-block processing cost, reported as a diagnostic rather than asserted: + // a wall-clock threshold flakes on shared CI runners, while correctness is + // already pinned above. A multi-second time here would flag the un-gated + // recording/removal seam as a production concern for busy blocks. + eprintln!( + "single block with {NOISE_TXS_PER_BLOCK} unrelated txs processed in {large_block_elapsed:?}" + ); +} diff --git a/key-wallet-manager/tests/observed_spent_multi_wallet_test.rs b/key-wallet-manager/tests/observed_spent_multi_wallet_test.rs new file mode 100644 index 000000000..1acbdb5f8 --- /dev/null +++ b/key-wallet-manager/tests/observed_spent_multi_wallet_test.rs @@ -0,0 +1,67 @@ +//! Multi-wallet isolation for the observed-spent-outpoint guard +//! (dashpay/rust-dashcore#649). +//! +//! `observed_spent_outpoints` lives on `ManagedWalletInfo` (per wallet), not on +//! a shared `WalletManager` structure. So one wallet observing a spend for an +//! outpoint must never suppress a *different* wallet's legitimate funding of the +//! same outpoint — even when both wallets, in the same process, happen to match +//! it (e.g. a synthetic/colliding outpoint, or a reused xpub-derived address +//! across two independently imported wallets, which the SDK does not forbid). + +mod common; + +use common::{funding_tx, process_block_for, spend_tx}; +use dashcore::blockdata::block::Block; +use dashcore::blockdata::transaction::OutPoint; +use dashcore::Network; +use key_wallet::wallet::initialization::WalletAccountCreationOptions; +use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; +use key_wallet_manager::{WalletId, WalletManager}; +use std::collections::BTreeSet; + +#[tokio::test] +async fn observed_spends_are_isolated_per_wallet() { + let mut manager = WalletManager::::new(Network::Testnet); + + // Create wallet B first and capture one of its own addresses while it is the + // only wallet, so the funding transaction pays an address B recognizes. + let wallet_b = manager + .create_wallet_with_random_mnemonic(WalletAccountCreationOptions::Default) + .expect("create wallet B"); + let b_address = manager.monitored_addresses().first().cloned().expect("wallet B has addresses"); + + // Then create wallet A, which observes the spend but never funds the coin. + let wallet_a = manager + .create_wallet_with_random_mnemonic(WalletAccountCreationOptions::Default) + .expect("create wallet A"); + + // Funding pays B's address; the outpoint it creates is O. + let funding_value = 1_000_000u64; + let funding = funding_tx(&b_address, funding_value, 0xB0); + let outpoint = OutPoint::new(funding.txid(), 0); + + // A spend of O, delivered to wallet A out of order (before any funding A sees). + let spend = spend_tx(outpoint, funding_value - 1_000, 77); + + let only_a: BTreeSet = [wallet_a].into_iter().collect(); + let only_b: BTreeSet = [wallet_b].into_iter().collect(); + + // Wallet A observes the spend (records O in A's set only). + let spend_block = Block::dummy(200, vec![spend]); + process_block_for(&mut manager, &spend_block, 200, &only_a).await; + + // Wallet B funds O in order — B never saw the spend, so B's set is empty. + let funding_block = Block::dummy(100, vec![funding]); + process_block_for(&mut manager, &funding_block, 100, &only_b).await; + + let b_utxos = manager.wallet_utxos(&wallet_b).expect("wallet B registered"); + assert!( + b_utxos.iter().any(|u| u.outpoint == outpoint), + "wallet B's legitimate funding must NOT be suppressed by wallet A's spend observation" + ); + let a_utxos = manager.wallet_utxos(&wallet_a).expect("wallet A registered"); + assert!( + !a_utxos.iter().any(|u| u.outpoint == outpoint), + "wallet A never funded O and must not track it" + ); +} diff --git a/key-wallet-manager/tests/out_of_order_spend_repro_test.rs b/key-wallet-manager/tests/out_of_order_spend_repro_test.rs new file mode 100644 index 000000000..7eb4bcd02 --- /dev/null +++ b/key-wallet-manager/tests/out_of_order_spend_repro_test.rs @@ -0,0 +1,81 @@ +//! A spend that is processed BEFORE the transaction that created the UTXO it +//! spends (out-of-order block delivery during rescan) must not leave that UTXO +//! permanently in the wallet's tracked set (dashpay/rust-dashcore#649). +//! +//! Guard site: `key-wallet/src/managed_account/managed_core_funds_account.rs` +//! (`update_utxos`). The function guards exactly this ordering ("Check if this +//! outpoint was already spent by a transaction we've seen. This handles +//! out-of-order block processing during rescan...") via +//! `self.is_outpoint_spent(&outpoint)` checked against `self.spent_outpoints`, +//! populated by the same function's spend-processing loop +//! (`self.spent_outpoints.insert(input.previous_output)` for every input of +//! every processed transaction, regardless of whether that input is recognized +//! as one of this account's own UTXOs). +//! +//! During a cold rescan, block heights are processed in essentially arbitrary +//! order (the parallel-filters matching completes in whatever order worker +//! threads finish, not height order). A spend can therefore be processed before +//! the funding transaction that created the coin it spends — the spending +//! transaction shows `sent = 0` because the wallet does not yet recognize the +//! input as its own, and the funding transaction is then inserted as a fresh, +//! spendable UTXO. If the guard fails, `key-wallet` can later select that coin +//! as `require_final_inputs`-eligible funding for an asset-lock transaction the +//! network rejects because the input is already spent. +//! +//! This test reproduces the same ORDER — process the spending block first, then +//! the funding block — deterministically, and checks that the out-of-order +//! guard prevents the funding UTXO from being (re-)inserted once its spend has +//! already been observed. + +mod common; + +use common::{funding_tx, process_block_all_wallets, spend_tx}; +use dashcore::blockdata::block::Block; +use dashcore::blockdata::transaction::OutPoint; +use key_wallet::wallet::initialization::WalletAccountCreationOptions; +use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; +use key_wallet_manager::WalletManager; + +#[tokio::test] +async fn spend_processed_before_its_funding_tx_does_not_leave_utxo_tracked() { + let mut manager = WalletManager::::new(dashcore::Network::Testnet); + let wallet_id = manager + .create_wallet_with_random_mnemonic(WalletAccountCreationOptions::Default) + .expect("failed to create wallet"); + + // The address that will receive the funding output — this account's first + // monitored (external, index 0) address. + let funding_address = + manager.monitored_addresses().first().cloned().expect("wallet must have addresses"); + + // Funding pays 1,000,000 duffs to our own address; only its own txid/vout + // (the coin later spent) matters. + let funding_value = 1_000_000u64; + let funding = funding_tx(&funding_address, funding_value, 0xAB); + let funding_outpoint = OutPoint::new(funding.txid(), 0); + + // Spend the funding UTXO (by its already-known txid) out to an external + // address, so the wallet no longer owns this specific output afterward. + let spend = spend_tx(funding_outpoint, funding_value - 1_000, 99); + + // Out-of-order delivery: the spend's block (height 200) is processed before + // the funding's block (height 100). + let spend_block = Block::dummy(200, vec![spend]); + process_block_all_wallets(&mut manager, &spend_block, 200).await; + + let funding_block = Block::dummy(100, vec![funding]); + process_block_all_wallets(&mut manager, &funding_block, 100).await; + + let utxos_after = manager.wallet_utxos(&wallet_id).expect("wallet must be registered"); + let still_tracked = utxos_after.iter().any(|u| u.outpoint == funding_outpoint); + + assert!( + !still_tracked, + "BUG #649 regression: funding outpoint {funding_outpoint} is still present in the \ + wallet's tracked UTXO set after its spend was processed FIRST (height 200) and its \ + funding transaction was processed SECOND (height 100). `update_utxos`'s own \ + `is_outpoint_spent` guard (managed_core_funds_account.rs) exists to handle this \ + out-of-order case but did not prevent the funding output from being (re-)inserted as \ + tracked/spendable once its spend had already been observed." + ); +} diff --git a/key-wallet/src/managed_account/address_pool.rs b/key-wallet/src/managed_account/address_pool.rs index a421421fb..57c021bed 100644 --- a/key-wallet/src/managed_account/address_pool.rs +++ b/key-wallet/src/managed_account/address_pool.rs @@ -1173,6 +1173,7 @@ impl AddressPool { for idx in indices_to_remove { if let Some(info) = self.addresses.remove(&idx) { self.address_index.remove(&info.address); + self.script_pubkey_index.remove(&info.script_pubkey); pruned += 1; } } diff --git a/key-wallet/src/managed_account/managed_account_ref.rs b/key-wallet/src/managed_account/managed_account_ref.rs index e95e36a7a..b94895b79 100644 --- a/key-wallet/src/managed_account/managed_account_ref.rs +++ b/key-wallet/src/managed_account/managed_account_ref.rs @@ -20,6 +20,8 @@ use crate::transaction_checking::account_checker::AccountMatch; use crate::transaction_checking::transaction_router::TransactionType; use crate::transaction_checking::TransactionContext; use crate::Network; +use dashcore::blockdata::transaction::OutPoint; +use dashcore::prelude::CoreBlockHeight; use dashcore::{Address, ScriptBuf, Transaction, Txid}; use std::collections::BTreeMap; @@ -293,16 +295,43 @@ impl<'a> ManagedAccountRefMut<'a> { /// Funds variants update UTXO state and balance; keys variants update /// only the transaction history. Both are subject to the /// `keep-finalized-transactions` Cargo feature for chainlocked records. + /// + /// This public entry point records with no observed-spend context — the + /// pre-#649 behavior, appropriate for callers driving an account directly + /// without wallet-level block processing. The wallet checker uses + /// `Self::record_transaction_with_observed_spends` instead, which + /// reconciles the record against the wallet-level observed-spent set. pub fn record_transaction( &mut self, tx: &Transaction, account_match: &AccountMatch, context: TransactionContext, transaction_type: TransactionType, + ) -> TransactionRecord { + self.record_transaction_with_observed_spends( + tx, + account_match, + context, + transaction_type, + &BTreeMap::new(), + ) + } + + /// Record a new transaction, reconciling it against `observed_spent` — + /// the wallet-level `observed_spent_outpoints` view + /// (dashpay/rust-dashcore#649); only the funds variant consults it (keys + /// accounts track no UTXOs/output details). + pub(crate) fn record_transaction_with_observed_spends( + &mut self, + tx: &Transaction, + account_match: &AccountMatch, + context: TransactionContext, + transaction_type: TransactionType, + observed_spent: &BTreeMap, ) -> TransactionRecord { match self { ManagedAccountRefMut::Funds(a) => { - a.record_transaction(tx, account_match, context, transaction_type) + a.record_transaction(tx, account_match, context, transaction_type, observed_spent) } ManagedAccountRefMut::Keys(a) => { a.record_transaction(tx, account_match, context, transaction_type) @@ -314,16 +343,41 @@ impl<'a> ManagedAccountRefMut<'a> { /// /// Funds variants additionally refresh UTXO state. Returns the updated /// record only when confirmation status actually changes. + /// + /// This public entry point confirms with no observed-spend context — the + /// pre-#649 behavior, appropriate for callers driving an account directly + /// without wallet-level block processing. The wallet checker uses + /// `Self::confirm_transaction_with_observed_spends` instead. pub fn confirm_transaction( &mut self, tx: &Transaction, account_match: &AccountMatch, context: TransactionContext, transaction_type: TransactionType, + ) -> Option { + self.confirm_transaction_with_observed_spends( + tx, + account_match, + context, + transaction_type, + &BTreeMap::new(), + ) + } + + /// Re-process an existing transaction, reconciling refreshed UTXO state + /// against `observed_spent` — the wallet-level `observed_spent_outpoints` + /// view (dashpay/rust-dashcore#649); only the funds variant consults it. + pub(crate) fn confirm_transaction_with_observed_spends( + &mut self, + tx: &Transaction, + account_match: &AccountMatch, + context: TransactionContext, + transaction_type: TransactionType, + observed_spent: &BTreeMap, ) -> Option { match self { ManagedAccountRefMut::Funds(a) => { - a.confirm_transaction(tx, account_match, context, transaction_type) + a.confirm_transaction(tx, account_match, context, transaction_type, observed_spent) } ManagedAccountRefMut::Keys(a) => { a.confirm_transaction(tx, account_match, context, transaction_type) diff --git a/key-wallet/src/managed_account/managed_core_funds_account.rs b/key-wallet/src/managed_account/managed_core_funds_account.rs index 2418152dc..a52816c18 100644 --- a/key-wallet/src/managed_account/managed_core_funds_account.rs +++ b/key-wallet/src/managed_account/managed_core_funds_account.rs @@ -165,11 +165,15 @@ impl ManagedCoreFundsAccount { } /// Add new UTXOs for received outputs, remove spent ones. + /// + /// Skips any output whose outpoint is already in `observed_spent` — it is + /// spent on-chain (dashpay/rust-dashcore#649), so the record stays consistent. fn update_utxos( &mut self, tx: &Transaction, account_match: &AccountMatch, context: TransactionContext, + observed_spent: &BTreeMap, ) { // Update UTXOs only for spendable account types match self.keys.managed_account_type() { @@ -242,6 +246,18 @@ impl ManagedCoreFundsAccount { continue; } + // #649 spend-first ordering: the spend was observed in an + // earlier-processed block, so this output is genuinely spent + // on-chain even though this account has never seen it before — + // never insert it, so the record built below is born correct. + if observed_spent.contains_key(&outpoint) { + tracing::debug!( + outpoint = %outpoint, + "Skipping UTXO already observed spent in an earlier-processed block (#649)" + ); + continue; + } + // Flag outputs from a "trusted" mempool transaction we created — // one whose inputs all spend our own final UTXOs and which pays // this output back to one of our internal (change) addresses. @@ -327,6 +343,7 @@ impl ManagedCoreFundsAccount { account_match: &AccountMatch, context: TransactionContext, transaction_type: TransactionType, + observed_spent: &BTreeMap, ) -> Option { let txid = tx.txid(); @@ -339,7 +356,13 @@ impl ManagedCoreFundsAccount { if !self.keys.has_transaction(&txid) { // Genuinely new sighting — delegate to record_transaction // (which handles finalize-on-record itself). - let record = self.record_transaction(tx, account_match, context, transaction_type); + let record = self.record_transaction( + tx, + account_match, + context, + transaction_type, + observed_spent, + ); return Some(record); } @@ -378,7 +401,7 @@ impl ManagedCoreFundsAccount { // chainlock catches up. #[cfg(not(feature = "keep-finalized-transactions"))] let drop_now = context.is_chain_locked(); - self.update_utxos(tx, account_match, context); + self.update_utxos(tx, account_match, context, observed_spent); #[cfg(not(feature = "keep-finalized-transactions"))] if drop_now { self.keys.drop_finalized_transaction(&txid); @@ -386,13 +409,21 @@ impl ManagedCoreFundsAccount { record_after } - /// Record a new transaction and update UTXOs for spendable account types + /// Record a new transaction and update UTXOs for spendable account types. + /// + /// `observed_spent` is the wallet-level `observed_spent_outpoints` view + /// (dashpay/rust-dashcore#649), read-only from `check_core_transaction`. + /// It is threaded into [`Self::update_utxos`], which skips inserting a UTXO + /// for any output whose outpoint is already observed spent on-chain, so a + /// coin whose spend was seen in an earlier-processed block is never + /// (re-)tracked as spendable. pub(crate) fn record_transaction( &mut self, tx: &Transaction, account_match: &AccountMatch, context: TransactionContext, transaction_type: TransactionType, + observed_spent: &BTreeMap, ) -> TransactionRecord { let net_amount = account_match.received as i64 - account_match.sent as i64; @@ -423,10 +454,12 @@ impl ManagedCoreFundsAccount { } } - // Use both UTXO-based input details and `account_match.sent` as signals - // that we created this transaction. The UTXO set may be incomplete - // (e.g., partial rescan) so `account_match.sent > 0` catches cases where - // the transaction still spent our funds even without matching UTXOs. + // Marks a transaction that spends our coins. `input_details` (built + // above) and `account_match.sent` (built in `check_transaction_with_index`) + // both derive from `self.utxos.get(&input.previous_output)` on this + // account, with no UTXO mutation between the two lookups, so they + // populate together; keeping both keeps this robust should the two + // call sites ever compute over different UTXO snapshots. let has_inputs = !input_details.is_empty() || account_match.sent > 0; let network = self.keys.network(); @@ -498,7 +531,7 @@ impl ManagedCoreFundsAccount { // feature is on (we want to keep the full record). #[cfg(not(feature = "keep-finalized-transactions"))] let drop_now = context.is_chain_locked(); - self.update_utxos(tx, account_match, context); + self.update_utxos(tx, account_match, context, observed_spent); #[cfg(not(feature = "keep-finalized-transactions"))] if drop_now { self.keys.drop_finalized_transaction(&txid); @@ -860,6 +893,21 @@ impl ManagedAccountTrait for ManagedCoreFundsAccount { } } +/// Rebuild the account-local `spent_outpoints` set from recorded transactions. +/// +/// Every input of every recorded transaction is a spend this account has seen, +/// so its `previous_output` belongs in the derived set. The field is not +/// persisted (`#[serde(skip)]`), so both [`Deserialize`] and the test reload +/// simulation reconstruct it through here to stay in lockstep. +#[cfg(any(feature = "serde", test))] +fn rebuild_spent_outpoints(keys: &ManagedCoreKeysAccount) -> HashSet { + keys.transactions() + .values() + .flat_map(|record| &record.transaction.input) + .map(|input| input.previous_output) + .collect() +} + #[cfg(feature = "serde")] impl<'de> Deserialize<'de> for ManagedCoreFundsAccount { fn deserialize(deserializer: D) -> Result @@ -875,13 +923,7 @@ impl<'de> Deserialize<'de> for ManagedCoreFundsAccount { let helper = Helper::deserialize(deserializer)?; - let spent_outpoints = helper - .keys - .transactions() - .values() - .flat_map(|record| &record.transaction.input) - .map(|input| input.previous_output) - .collect(); + let spent_outpoints = rebuild_spent_outpoints(&helper.keys); Ok(ManagedCoreFundsAccount { keys: helper.keys, diff --git a/key-wallet/src/tests/address_pool_tests.rs b/key-wallet/src/tests/address_pool_tests.rs index 7da10b5cb..473f22730 100644 --- a/key-wallet/src/tests/address_pool_tests.rs +++ b/key-wallet/src/tests/address_pool_tests.rs @@ -197,3 +197,45 @@ fn test_next_unused_multiple_mixed_usage() { assert!(!initial.contains(&next_batch[6])); assert!(!initial.contains(&next_batch[7])); } + +/// `prune_unused` must drop a pruned address from BOTH reverse indexes it +/// maintains — `address_index` and `script_pubkey_index`. A leftover +/// `script_pubkey_index` entry would let a pruned (unwatched) script still +/// resolve to an index and be treated as wallet-relevant. Regression guard for +/// the missing `script_pubkey_index.remove` in `prune_unused` +/// (dashpay/rust-dashcore#649 companion fix). +#[test] +fn prune_unused_clears_script_pubkey_index() { + let base_path = DerivationPath::from(vec![ChildNumber::from_normal_idx(0).unwrap()]); + let gap_limit = 20u32; + let mut pool = AddressPool::new_without_generation( + base_path, + AddressPoolType::External, + gap_limit, + Network::Testnet, + ); + let (key_source, _) = test_key_source(); + + // Generate well beyond the gap limit; none are marked used, so with + // `highest_used == None` everything above `gap_limit - 1` is prunable. + pool.generate_addresses(gap_limit + 10, &key_source, true).unwrap(); + + // Pick an index that will be pruned and capture its address + script. + let victim_index = gap_limit + 5; + let victim = pool.addresses.get(&victim_index).expect("victim address generated").clone(); + assert_eq!(pool.address_index.get(&victim.address), Some(&victim_index)); + assert_eq!(pool.script_pubkey_index.get(&victim.script_pubkey), Some(&victim_index)); + + let pruned = pool.prune_unused(); + assert!(pruned > 0, "addresses beyond the gap limit should be pruned"); + + assert!(!pool.addresses.contains_key(&victim_index), "pruned address removed from pool"); + assert!( + !pool.address_index.contains_key(&victim.address), + "pruned address removed from address_index" + ); + assert!( + !pool.script_pubkey_index.contains_key(&victim.script_pubkey), + "pruned address must also be removed from script_pubkey_index" + ); +} diff --git a/key-wallet/src/tests/mod.rs b/key-wallet/src/tests/mod.rs index 0e356a976..8a91ccf52 100644 --- a/key-wallet/src/tests/mod.rs +++ b/key-wallet/src/tests/mod.rs @@ -22,6 +22,8 @@ mod keep_finalized_transactions_tests; mod managed_account_collection_tests; +mod observed_spent_outpoints_tests; + mod performance_tests; mod provider_key_derivation_tests; diff --git a/key-wallet/src/tests/observed_spent_outpoints_tests.rs b/key-wallet/src/tests/observed_spent_outpoints_tests.rs new file mode 100644 index 000000000..4bcda6e8e --- /dev/null +++ b/key-wallet/src/tests/observed_spent_outpoints_tests.rs @@ -0,0 +1,444 @@ +//! Unit-level coverage for the wallet-level observed-spent-outpoint machinery +//! (dashpay/rust-dashcore#649). +//! +//! The manager-level integration tests (`key-wallet-manager/tests/`) drive the +//! *spend-first* ordering end-to-end through the public `WalletManager` surface: +//! a spend delivered before its funding, reconciled at `update_utxos` insert +//! time. That path leaves several branches of the fix unexercised, because they +//! only trigger on orderings or lifecycle events a black-box manager test cannot +//! reach directly: +//! +//! * the serde adapter for `observed_spent_outpoints` (no manager test +//! serializes a wallet holding entries), +//! * `prune_finalized_observed_spends` and its finality boundary, +//! * the account-add sync rewind for a standalone (from-xpub) account, and +//! * the *born-fully-spent* history-recording path: a funding transaction +//! applied after its spend was already observed must still be recorded in +//! history, while its already-spent output is never (re-)tracked as a UTXO. +//! +//! These white-box tests exercise those branches directly against the +//! `pub(crate)` surface. + +use dashcore::blockdata::transaction::{OutPoint, Transaction}; +use dashcore::bls_sig_utils::BLSSignature; +use dashcore::ephemerealdata::chain_lock::ChainLock; +use dashcore::hashes::Hash; +use dashcore::{BlockHash, TxIn, Txid}; + +use crate::managed_account::managed_account_trait::ManagedAccountTrait; +use crate::test_utils::TestWalletContext; +use crate::transaction_checking::{BlockInfo, TransactionContext}; +use crate::wallet::managed_wallet_info::managed_account_operations::ManagedAccountOperations; +use crate::wallet::ManagedWalletInfo; +use crate::AccountType; + +/// A transaction whose inputs spend exactly `spent`, with no outputs. Enough to +/// drive `record_observed_spends` (reads inputs) and the removal guard. +fn spending_tx(spent: &[OutPoint]) -> Transaction { + Transaction { + version: 1, + lock_time: 0, + input: spent + .iter() + .map(|op| TxIn { + previous_output: *op, + ..Default::default() + }) + .collect(), + output: Vec::new(), + special_transaction_payload: None, + } +} + +/// A syntactically-valid `ChainLock` at `height`; only `block_height` is read by +/// the prune boundary, the signature is never verified here. +fn dummy_chain_lock(height: u32) -> ChainLock { + ChainLock { + block_height: height, + block_hash: BlockHash::from_slice(&[height as u8; 32]).expect("hash"), + signature: BLSSignature::from([0u8; 96]), + } +} + +/// The `observed_spent_outpoints` serde adapter round-trips: it encodes the map +/// as a `(OutPoint, height)` sequence (format-agnostic), and reloads it. An +/// account-less wallet serializes cleanly — the populated-account blocker is +/// `AddressPool::script_pubkey_index` (a non-string JSON map key), not this +/// field — so it isolates the adapter under test. +#[cfg(feature = "serde")] +#[test] +fn observed_spent_outpoints_survive_serde_round_trip() { + let mut info = ManagedWalletInfo::dummy(7); + let op_a = OutPoint::new(Txid::from([0x11; 32]), 0); + let op_b = OutPoint::new(Txid::from([0x22; 32]), 3); + info.record_observed_spends(&spending_tx(&[op_a]), 100); + info.record_observed_spends(&spending_tx(&[op_b]), 250); + assert_eq!(info.observed_spent_outpoints().len(), 2); + + let json = serde_json::to_string(&info).expect("serialize"); + let restored: ManagedWalletInfo = serde_json::from_str(&json).expect("deserialize"); + + assert_eq!(restored.observed_spent_outpoints().get(&op_a), Some(&100)); + assert_eq!(restored.observed_spent_outpoints().get(&op_b), Some(&250)); + assert_eq!(restored.observed_spent_outpoints().len(), 2); + + // An empty map round-trips through the sequence adapter too (empty-seq + // serialize + visitor path), and `#[serde(default)]` still yields an empty + // set rather than erroring. + let empty = ManagedWalletInfo::dummy(8); + let json_empty = serde_json::to_string(&empty).expect("serialize empty"); + let restored_empty: ManagedWalletInfo = + serde_json::from_str(&json_empty).expect("deserialize empty"); + assert!(restored_empty.observed_spent_outpoints().is_empty()); +} + +/// `prune_finalized_observed_spends` is a no-op without a chainlock, and +/// otherwise evicts exactly the entries at or below +/// `min(chainlock height, synced_height)` — the finality boundary — keeping +/// everything above it. +#[test] +fn prune_finalized_observed_spends_respects_finality_boundary() { + let mut info = ManagedWalletInfo::dummy(9); + let op_low = OutPoint::new(Txid::from([0x01; 32]), 0); // height 50 + let op_bnd = OutPoint::new(Txid::from([0x02; 32]), 0); // height 100 (== boundary) + let op_high = OutPoint::new(Txid::from([0x03; 32]), 0); // height 150 + info.record_observed_spends(&spending_tx(&[op_low]), 50); + info.record_observed_spends(&spending_tx(&[op_bnd]), 100); + info.record_observed_spends(&spending_tx(&[op_high]), 150); + + // No chainlock has been applied: there is no provable finality boundary, so + // nothing may be pruned regardless of how high `synced_height` is. + info.metadata.synced_height = 1_000_000; + info.prune_finalized_observed_spends(); + assert_eq!( + info.observed_spent_outpoints().len(), + 3, + "no chainlock => nothing is provably final => no eviction" + ); + + // Boundary = min(chainlock height 200, synced_height 100) = 100. The + // chainlock being higher than the sync checkpoint is the exact case an + // in-progress rescan produces, and it must not over-prune. + info.metadata.synced_height = 100; + info.metadata.last_applied_chain_lock = Some(dummy_chain_lock(200)); + info.prune_finalized_observed_spends(); + + let remaining = info.observed_spent_outpoints(); + assert!(!remaining.contains_key(&op_low), "height 50 <= boundary 100: evicted"); + assert!(!remaining.contains_key(&op_bnd), "height 100 == boundary 100: evicted"); + assert!(remaining.contains_key(&op_high), "height 150 > boundary 100: retained"); + assert_eq!(remaining.len(), 1); +} + +/// Adding a standalone (from-xpub) account rewinds the sync checkpoint below +/// wallet birth, so the new account's coins get filter coverage before pruning +/// can consume the certificate (dashpay/rust-dashcore#649). A wallet still in +/// initial sync (already at/below the floor) is left untouched. +#[test] +fn adding_account_from_xpub_rewinds_sync_checkpoint() { + let mut ctx = TestWalletContext::new_random(); + ctx.managed_wallet.metadata.birth_height = 1_000; + ctx.managed_wallet.metadata.synced_height = 5_000; + let generation_before = ctx.managed_wallet.account_generation(); + + ctx.managed_wallet + .add_managed_account_from_xpub( + AccountType::IdentityTopUp { + registration_index: 7, + }, + ctx.xpub, + ) + .expect("attach standalone IdentityTopUp account"); + + assert_eq!( + ctx.managed_wallet.metadata.synced_height, 999, + "synced_height rewound to birth_height - 1 so the new account is re-scanned from birth" + ); + assert_eq!( + ctx.managed_wallet.account_generation(), + generation_before + 1, + "adding an account bumps the account generation so in-flight scans are invalidated" + ); + + // A wallet whose sync checkpoint is already below the birth floor is left + // as-is: the rewind only ever collapses the certificate, never advances it. + ctx.managed_wallet.metadata.synced_height = 10; + ctx.managed_wallet + .add_managed_account_from_xpub( + AccountType::IdentityTopUp { + registration_index: 8, + }, + ctx.xpub, + ) + .expect("attach second standalone account"); + assert_eq!( + ctx.managed_wallet.metadata.synced_height, 10, + "a still-behind sync checkpoint is not advanced by the rewind" + ); + assert_eq!( + ctx.managed_wallet.account_generation(), + generation_before + 2, + "the generation bumps even when the checkpoint does not move — this is the only \ + signal a commit-time check has for a height-invisible account add" + ); +} + +/// Regression for the "born-fully-spent" history record-loss reported by +/// HashEngineering against #851/#866 (dashpay/rust-dashcore#649, and the +/// #846-class invisible-record family): a funding transaction recovered +/// *after* its spend was already observed — so every one of its +/// wallet-relevant outputs is in `observed_spent_outpoints` before the funding +/// itself is applied — must still be **recorded in history**, not silently +/// dropped. The balance and UTXO set are unaffected (the output is genuinely +/// spent on-chain), so this is purely about not losing the history record. +/// +/// This is the spend-first / out-of-order ordering produced by the committed- +/// range rescan (`track_for_new_scripts`): the forward scan already processed +/// the spend, then the old funding block is re-applied. #649's fix keeps the +/// record: `check_transaction_for_match` classifies relevance by address +/// membership (never gated on spent-status, so the fully-spent funding is still +/// relevant), and `ManagedCoreFundsAccount::record_transaction` unconditionally +/// inserts the record while `update_utxos` skips creating a UTXO for the +/// already-observed-spent output. The received output stays in the record's +/// history (the minimal fix does not rewrite it away — balance correctness comes +/// from the skipped UTXO, not from erasing the receive). This test pins that a +/// born-fully-spent funding tx (1) is surfaced as a new record and (2) lands in +/// the account's transaction history, while balance and UTXOs stay at zero. +#[tokio::test] +async fn born_fully_spent_funding_tx_is_recorded_in_history() { + use dashcore::blockdata::script::ScriptBuf; + use dashcore::TxOut; + + let mut ctx = TestWalletContext::new_random(); + + // Funding F pays our receive address; its single output F:0 is ours. + let funding_value = 1_000_000u64; + let funding = Transaction::dummy(&ctx.receive_address, 0..1, &[funding_value]); + let f_txid = funding.txid(); + let f_outpoint = OutPoint::new(f_txid, 0); + + // Spend S consumes F:0 entirely to an external address (no wallet output), + // so the wallet cannot attribute it to any account when it arrives first. + let external = dashcore::Address::p2pkh( + &dashcore::PublicKey::from_slice(&[0x02; 33]).expect("pubkey"), + dashcore::Network::Testnet, + ); + let spend = Transaction { + version: 2, + lock_time: 0, + input: vec![TxIn { + previous_output: f_outpoint, + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: dashcore::Witness::new(), + }], + output: vec![TxOut { + value: funding_value - 1_000, + script_pubkey: external.script_pubkey(), + }], + special_transaction_payload: None, + }; + + // Spend-first: the spend's block (height 200) is applied before the + // funding's block (height 100), exactly as the committed-range rescan + // re-applies old funding blocks after their spends. + let spend_ctx = TransactionContext::InBlock(BlockInfo::new( + 200, + BlockHash::from_slice(&[2u8; 32]).expect("hash"), + 1_650_000_200, + )); + let spend_res = ctx.check_transaction(&spend, spend_ctx).await; + // The spend pays nothing to us and spends a coin we do not yet own, so it is + // not attributable — but it must still be observed so the funding is + // reconciled against it. + assert!(!spend_res.is_relevant, "external spend of a not-yet-owned coin is unattributable"); + assert_eq!( + ctx.managed_wallet.observed_spent_outpoints().get(&f_outpoint), + Some(&200), + "the spend must be recorded in observed_spent_outpoints" + ); + + // Now the funding block arrives. Every output of F is already observed + // spent, i.e. F is born fully spent. + let fund_ctx = TransactionContext::InBlock(BlockInfo::new( + 100, + BlockHash::from_slice(&[1u8; 32]).expect("hash"), + 1_650_000_100, + )); + let fund_res = ctx.check_transaction(&funding, fund_ctx).await; + + // (1) The funding tx is still relevant and surfaced as a NEW record — this + // is the detection event a consumer persists into history. Before #649's + // record-keeping fix the spend-first path made it come out not-relevant and + // produced no record and no event. + assert!(fund_res.is_relevant, "a fully-spent funding tx paying our address is still relevant"); + assert!( + fund_res.new_records.iter().any(|r| r.txid == f_txid), + "born-fully-spent funding tx must be surfaced as a new record" + ); + + // (2) The record is present in the account's transaction history. + let account = ctx.managed_wallet.first_bip44_managed_account().expect("BIP44 account"); + assert!( + account.transactions().contains_key(&f_txid), + "born-fully-spent funding tx must be recorded in the account's history" + ); + + // Balance and UTXO accounting are unchanged: the output is genuinely spent, + // so it is never inserted as a UTXO and contributes zero to the balance. + assert!(account.utxos.is_empty(), "the already-spent output must not become a UTXO"); + assert_eq!(ctx.managed_wallet.balance.total(), 0, "born-fully-spent tx must not add balance"); + + // History preserves the receive: the minimal #649 fix does not rewrite the + // record. The output is genuinely spent on-chain, so it is never (re-)tracked + // as a UTXO (asserted above) — but the record still shows the wallet received + // it, matching what in-order delivery would record. + let record = account.transactions().get(&f_txid).expect("record present"); + assert_eq!( + record.net_amount, funding_value as i64, + "the received output stays in the record's net_amount; balance/UTXO correctness comes \ + from update_utxos skipping the already-spent coin, not from erasing the receive" + ); + assert!( + record.output_details.iter().any(|d| matches!( + d.role, + crate::managed_account::transaction_record::OutputRole::Received + | crate::managed_account::transaction_record::OutputRole::Change + )), + "the received output detail is preserved in the born-fully-spent record's history" + ); +} + +/// Companion to [`born_fully_spent_funding_tx_is_recorded_in_history`] covering +/// the CoinJoin-style intermediate hop that HashEngineering measured missing: a +/// transaction that both **spends a live wallet coin** and pays its change back +/// to us, where that change output is itself already observed spent. It must be +/// recorded even though its change output never becomes a UTXO and it leaves the +/// balance at zero. +#[tokio::test] +async fn born_fully_spent_intermediate_hop_is_recorded_in_history() { + use dashcore::blockdata::script::ScriptBuf; + use dashcore::TxOut; + + let mut ctx = TestWalletContext::new_random(); + + // A second receive address for the hop's change output. + let hop_addr = ctx + .managed_wallet + .first_bip44_managed_account_mut() + .expect("BIP44 account") + .next_receive_address(Some(&ctx.xpub), true) + .expect("hop address"); + + // C funds our first receive address (a live coin). + let c = Transaction::dummy(&ctx.receive_address, 0..1, &[1_000_000]); + let c_txid = c.txid(); + + // Hop T spends C:0 and pays 990_000 back to our hop address (T:0). + let t = Transaction { + version: 2, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint::new(c_txid, 0), + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: dashcore::Witness::new(), + }], + output: vec![TxOut { + value: 990_000, + script_pubkey: hop_addr.script_pubkey(), + }], + special_transaction_payload: None, + }; + let t_txid = t.txid(); + + // U spends T:0 out to an external address. + let external = dashcore::Address::p2pkh( + &dashcore::PublicKey::from_slice(&[0x02; 33]).expect("pubkey"), + dashcore::Network::Testnet, + ); + let u = Transaction { + version: 2, + lock_time: 0, + input: vec![TxIn { + previous_output: OutPoint::new(t_txid, 0), + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: dashcore::Witness::new(), + }], + output: vec![TxOut { + value: 980_000, + script_pubkey: external.script_pubkey(), + }], + special_transaction_payload: None, + }; + + let blk = |h: u32, b: u8| { + TransactionContext::InBlock(BlockInfo::new( + h, + BlockHash::from_slice(&[b; 32]).expect("hash"), + 1_650_000_000 + h, + )) + }; + + // Order: fund C (live), then U (records T:0 observed spent), then the hop T + // whose own output is already spent. + ctx.check_transaction(&c, blk(1, 1)).await; + let u_res = ctx.check_transaction(&u, blk(3, 3)).await; + assert!(!u_res.is_relevant, "U is an external spend of a not-yet-owned coin"); + let t_res = ctx.check_transaction(&t, blk(2, 2)).await; + + // The hop is relevant (it spends our live coin C) and recorded, even though + // its own output is already observed spent and so never becomes a UTXO. + assert!(t_res.is_relevant, "a hop that spends our live coin is relevant"); + assert!( + t_res.new_records.iter().any(|r| r.txid == t_txid), + "the fully-spent hop must be surfaced as a new record" + ); + + let account = ctx.managed_wallet.first_bip44_managed_account().expect("BIP44 account"); + assert!( + account.transactions().contains_key(&t_txid), + "the fully-spent hop must be recorded in history" + ); + assert!( + account.transactions().contains_key(&c_txid), + "the funding tx must also remain in history" + ); + // Both C and T's outputs are consumed, so no UTXOs and zero balance remain. + assert!(account.utxos.is_empty(), "no UTXO should remain after the hop chain"); + assert_eq!(ctx.managed_wallet.balance.total(), 0, "hop chain nets to zero balance"); +} + +/// Growing the observed-spent map is itself a state modification +/// (dashpay/rust-dashcore#649): the map is persisted wallet state, and a +/// consumer that persists only on reported modifications must not drop a +/// recorded spend across a restart. An irrelevant block spend therefore +/// reports `state_modified`, a byte-identical redelivery (map unchanged) does +/// not, and a mempool spend (never recorded) does not. +#[tokio::test] +async fn recording_observed_spend_reports_state_modified() { + let mut ctx = TestWalletContext::new_random(); + let unrelated = OutPoint::new(Txid::from([0x77; 32]), 1); + let spend = spending_tx(&[unrelated]); + let block = + BlockInfo::new(200, BlockHash::from_slice(&[3u8; 32]).expect("hash"), 1_700_000_000); + + let result = ctx.check_transaction(&spend, TransactionContext::InBlock(block)).await; + assert!(!result.is_relevant, "the spend touches nothing the wallet owns"); + assert!( + result.state_modified, + "recording a new observed spend must surface as a state modification" + ); + assert_eq!(ctx.managed_wallet.observed_spent_outpoints().get(&unrelated), Some(&200)); + + // Same tx at the same height: the map entry is already present with the + // same value, so nothing changed and nothing must be reported. + let result = ctx.check_transaction(&spend, TransactionContext::InBlock(block)).await; + assert!(!result.state_modified, "an unchanged map must not report a modification"); + + // Mempool spends are never recorded, so they modify nothing either. + let mempool_spend = spending_tx(&[OutPoint::new(Txid::from([0x78; 32]), 0)]); + let result = ctx.check_transaction(&mempool_spend, TransactionContext::Mempool).await; + assert!(!result.state_modified, "a mempool spend records nothing"); +} diff --git a/key-wallet/src/transaction_checking/wallet_checker.rs b/key-wallet/src/transaction_checking/wallet_checker.rs index 7ea653924..cb0272f2a 100644 --- a/key-wallet/src/transaction_checking/wallet_checker.rs +++ b/key-wallet/src/transaction_checking/wallet_checker.rs @@ -6,8 +6,6 @@ pub(crate) use super::account_checker::TransactionCheckResult; use super::transaction_context::TransactionContext; use super::transaction_router::TransactionRouter; -#[cfg(test)] -use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; use crate::wallet::managed_wallet_info::ManagedWalletInfo; use crate::{KeySource, Wallet}; @@ -62,6 +60,27 @@ impl WalletTransactionChecker for ManagedWalletInfo { // Check only relevant account types let mut result = self.accounts.check_transaction(tx, &relevant_types); + // #649: remember every spend seen in a block, independent of the + // spending tx's classification or whether the wallet can attribute it. + // Insert-only here (no account mutation), so a spend that IS attributed + // still has its `input_details` built from the live funding UTXO by + // `record_transaction` below. Mempool / IS-lock spends are deliberately + // never recorded — an unconfirmed spend must not invalidate a coin. + let block_height = if update_state { + context.block_info().map(|info| info.height()) + } else { + None + }; + if let Some(height) = block_height { + // The observed-spent map is persisted wallet state: growing it is a + // state modification in its own right, even when the tx is + // otherwise irrelevant — losing an entry across a restart would + // reopen #649 for that coin. + if self.record_observed_spends(tx, height) { + result.state_modified = true; + } + } + if !update_state || !result.is_relevant { return result; } @@ -125,11 +144,12 @@ impl WalletTransactionChecker for ManagedWalletInfo { result.updated_records.push(record.clone()); } } else { - let record = account.record_transaction( + let record = account.record_transaction_with_observed_spends( tx, &account_match, context.clone(), tx_type, + &self.observed_spent_outpoints, ); account.mark_utxos_instant_send(&txid); result.new_records.push(record); @@ -156,15 +176,24 @@ impl WalletTransactionChecker for ManagedWalletInfo { }; if is_new { - let record = - account.record_transaction(tx, &account_match, context.clone(), tx_type); + let record = account.record_transaction_with_observed_spends( + tx, + &account_match, + context.clone(), + tx_type, + &self.observed_spent_outpoints, + ); result.new_records.push(record); result.state_modified = true; } else { let existed_before = account.has_transaction(&tx.txid()); - if let Some(record) = - account.confirm_transaction(tx, &account_match, context.clone(), tx_type) - { + if let Some(record) = account.confirm_transaction_with_observed_spends( + tx, + &account_match, + context.clone(), + tx_type, + &self.observed_spent_outpoints, + ) { result.state_modified = true; if existed_before { result.updated_records.push(record); @@ -241,6 +270,7 @@ impl WalletTransactionChecker for ManagedWalletInfo { mod tests { use super::*; use crate::account::account_type::StandardAccountType; + use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::managed_account::transaction_record::{OutputRole, TransactionDirection}; use crate::test_utils::TestWalletContext; use crate::transaction_checking::BlockInfo; @@ -256,6 +286,7 @@ mod tests { use dashcore::TxOut; use dashcore::{Address, BlockHash, TxIn, Txid}; use dashcore_hashes::Hash; + use std::collections::BTreeMap; /// Test wallet checker with unrelated transaction #[tokio::test] @@ -1375,7 +1406,13 @@ mod tests { let block_context = TransactionContext::InBlock(BlockInfo::new(600, block_hash, 1700000000)); let tx_type = TransactionRouter::classify_transaction(&tx); - let backfilled = account.confirm_transaction(&tx, &account_match, block_context, tx_type); + let backfilled = account.confirm_transaction( + &tx, + &account_match, + block_context, + tx_type, + &BTreeMap::new(), + ); assert!(backfilled.is_some(), "Should return Some when backfilling a missing record"); // Verify the transaction was recorded with block context @@ -1426,7 +1463,13 @@ mod tests { .first_bip44_managed_account_mut() .expect("Should have BIP44 account"); let tx_type = TransactionRouter::classify_transaction(&tx); - let confirmed = account.confirm_transaction(&tx, &account_match, block_context, tx_type); + let confirmed = account.confirm_transaction( + &tx, + &account_match, + block_context, + tx_type, + &BTreeMap::new(), + ); assert!(confirmed.is_some(), "Should return Some when confirming unconfirmed tx"); let record = account.transactions().get(&txid).expect("Should have record"); diff --git a/key-wallet/src/wallet/managed_wallet_info/managed_accounts.rs b/key-wallet/src/wallet/managed_wallet_info/managed_accounts.rs index 40610e6d2..498172e68 100644 --- a/key-wallet/src/wallet/managed_wallet_info/managed_accounts.rs +++ b/key-wallet/src/wallet/managed_wallet_info/managed_accounts.rs @@ -78,6 +78,9 @@ impl ManagedAccountOperations for ManagedWalletInfo { // Insert into the collection self.accounts.insert(managed_account)?; + // A newly added account has no filter coverage, so the sync certificate + // must be invalidated or pruning could reopen #649 for its coins. + self.rewind_sync_checkpoint_for_new_account(); Ok(()) } @@ -110,6 +113,9 @@ impl ManagedAccountOperations for ManagedWalletInfo { // Insert into the collection self.accounts.insert(managed_account)?; + // A newly added account has no filter coverage, so the sync certificate + // must be invalidated or pruning could reopen #649 for its coins. + self.rewind_sync_checkpoint_for_new_account(); Ok(()) } @@ -155,6 +161,9 @@ impl ManagedAccountOperations for ManagedWalletInfo { // Insert into the collection self.accounts.insert(managed_account)?; + // A newly added account has no filter coverage, so the sync certificate + // must be invalidated or pruning could reopen #649 for its coins. + self.rewind_sync_checkpoint_for_new_account(); Ok(()) } @@ -188,6 +197,9 @@ impl ManagedAccountOperations for ManagedWalletInfo { // Insert into the collection self.accounts.insert(managed_account)?; + // A newly added account has no filter coverage, so the sync certificate + // must be invalidated or pruning could reopen #649 for its coins. + self.rewind_sync_checkpoint_for_new_account(); Ok(()) } @@ -234,6 +246,9 @@ impl ManagedAccountOperations for ManagedWalletInfo { // Insert into the collection self.accounts.insert(managed_account)?; + // A newly added account has no filter coverage, so the sync certificate + // must be invalidated or pruning could reopen #649 for its coins. + self.rewind_sync_checkpoint_for_new_account(); Ok(()) } @@ -272,6 +287,9 @@ impl ManagedAccountOperations for ManagedWalletInfo { // Insert into the collection self.accounts.insert(managed_account)?; + // A newly added account has no filter coverage, so the sync certificate + // must be invalidated or pruning could reopen #649 for its coins. + self.rewind_sync_checkpoint_for_new_account(); Ok(()) } } diff --git a/key-wallet/src/wallet/managed_wallet_info/mod.rs b/key-wallet/src/wallet/managed_wallet_info/mod.rs index 62b3d9e71..f687d4953 100644 --- a/key-wallet/src/wallet/managed_wallet_info/mod.rs +++ b/key-wallet/src/wallet/managed_wallet_info/mod.rs @@ -22,11 +22,12 @@ use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::wallet::managed_wallet_info::transaction_building::AccountTypePreference; use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; use crate::{Network, Wallet}; +use dashcore::blockdata::transaction::OutPoint; use dashcore::prelude::CoreBlockHeight; -use dashcore::{Address, Txid}; +use dashcore::{Address, Transaction, Txid}; #[cfg(feature = "serde")] use serde::{Deserialize, Serialize}; -use std::collections::HashSet; +use std::collections::{BTreeMap, HashSet}; /// Information about a managed wallet /// @@ -46,15 +47,189 @@ pub struct ManagedWalletInfo { pub description: Option, /// Wallet metadata pub metadata: WalletMetadata, - /// All managed accounts + /// All managed accounts. + /// + /// Prefer the `add_managed_*` methods to add accounts to a live wallet: + /// they rewind the sync checkpoint so the new account's coins get filter + /// coverage. Inserting directly here without that rewind can reopen + /// dashpay/rust-dashcore#649 for the added account. pub accounts: ManagedAccountCollection, /// Cached wallet core balance - should be updated when accounts change pub balance: WalletCoreBalance, /// Transactions that have received an InstantSend lock. #[cfg_attr(feature = "serde", serde(skip))] pub(crate) instant_send_locks: HashSet, + /// Outpoints observed spent by transactions seen in a processed block, + /// mapped to the height of the block that spent them. + /// + /// This is the wallet-level, classification-independent record of "this + /// coin was consumed on-chain" used to keep out-of-order rescan delivery + /// from resurrecting a spent UTXO (dashpay/rust-dashcore#649). A spend can + /// be delivered before the transaction that funded the coin it spends, and + /// it may be routed away from — or fail to match — the account that owns + /// the coin; recording every block-observed spend here, independent of the + /// spending transaction's classification, lets the funding-side insert be + /// reconciled away whichever order the two blocks arrive in. + /// + /// Membership is bounded-permanent: an entry is retained until its spend + /// height is provably final, then evicted by + /// [`Self::prune_finalized_observed_spends`]. Until then it is only ever + /// added, never removed, and reorg rollback does NOT retract it — treating a + /// coin as spendable again after a spend was observed is the failure this + /// set exists to prevent. Only spends seen in a block (`InBlock` / + /// `InChainLockedBlock`) are recorded; mempool-context spends are never + /// recorded, so an unconfirmed spend can never wrongly invalidate a coin. + /// + /// # Bounded permanence + /// + /// An entry `(outpoint, height)` is removed only when + /// `height <= min(last_applied_chain_lock.block_height, synced_height)` — + /// the finality boundary. At that boundary the spend is chain-locked (it can + /// never be reorged out) and any funding transaction for the outpoint has + /// been delivered (BIP158 filters have no false negatives below + /// `synced_height`) and finalized (promoted into `finalized_txids`, or kept + /// as a chainlocked record), so every redelivery path short-circuits before + /// a coin could be re-inserted — in both `keep-finalized-transactions` + /// configurations and across a reload. No other removal path may be added + /// without a deliberate decision. + /// + /// Eviction is event-driven (chainlock application, sync-checkpoint commit), + /// never age- or recency-based: during an out-of-order rescan `synced_height` + /// is low, so nothing is pruned in exactly the window where #649 ordering + /// hazards live, and the set self-repopulates on any replay since + /// `record_observed_spends` runs unconditionally per checked tx. A naive + /// age/LRU eviction would instead evict the cold entries whose funding tx may + /// still arrive out of order, reopening #649 for that coin. Steady-state size + /// is the above-boundary window only — roughly one block's inputs on a + /// healthy chain. A defensive cap on the deserialized entry count (see the + /// serde adapter) guards against a corrupted or hostile wallet file forcing + /// an unbounded allocation on load. + /// + /// # Persistence + /// + /// The field is serde-serializable so it is preserved wherever a + /// `ManagedWalletInfo` is serialized. Its real value, though, is + /// order-independent correctness *within a single continuous processing run* + /// — which is what the observed #649 symptom needs, since a cold rescan is a + /// full fresh replay in one process. `#[serde(default)]` keeps snapshots + /// written before this field existed loadable, seeding an empty set (a + /// pre-fix wallet gets no retroactive backfill; only spends observed after + /// the field exists are protected). That default applies only to + /// self-describing formats (e.g. JSON) that can detect the absent field; + /// non-self-describing serde formats (bincode-serde, `platform_value::Value`) + /// cannot default a missing field, so consumers using those need a versioned + /// migration to load pre-field snapshots. + #[cfg_attr(feature = "serde", serde(default, with = "observed_spent_outpoints_serde"))] + pub(crate) observed_spent_outpoints: BTreeMap, + /// Generation counter for the wallet's account set, bumped every time an + /// account is added to a live wallet (see + /// [`Self::rewind_sync_checkpoint_for_new_account`]). + /// + /// Filter-sync layers snapshot this when they scan a range for the wallet + /// and compare at commit time: a batch scanned under an older generation + /// must not certify coverage (`synced_height`) for the current account + /// set, because the added account's scripts were not part of the scan + /// (dashpay/rust-dashcore#649). A height-only contiguity check cannot + /// detect this when the account-add rewind lands inside the batch's range + /// — or moves nothing at all because the checkpoint already sat at the + /// birth floor. + /// + /// In-memory only (`serde(skip)`): the snapshots comparing against it are + /// in-flight scan state that does not survive a restart either, and a + /// fresh process restarts both sides at 0. + #[cfg_attr(feature = "serde", serde(skip))] + pub(crate) account_generation: u64, +} + +/// Serde adapter for [`ManagedWalletInfo::observed_spent_outpoints`] that +/// encodes the map as a sequence of `(OutPoint, height)` pairs rather than as a +/// map. +/// +/// This is for format-agnosticism, not a JSON limitation: `OutPoint` serializes +/// to a string in human-readable formats (so it is a fine `serde_json` object +/// key) but to a struct `{ txid, vout }` in non-human-readable / strict-key +/// encoders (bincode's serde layer, `platform_value::Value`), where a struct +/// cannot be a map key. Encoding the map as a sequence of key/value pairs +/// round-trips identically across every serde format. +/// +/// Deserialization applies a defensive cap ([`MAX_OBSERVED_SPENT_OUTPOINTS`]): +/// a corrupted or hostile wallet file declaring an absurd number of entries is +/// rejected rather than driving an unbounded allocation. +#[cfg(feature = "serde")] +mod observed_spent_outpoints_serde { + use super::{BTreeMap, CoreBlockHeight, OutPoint, MAX_OBSERVED_SPENT_OUTPOINTS}; + use core::fmt; + use serde::de::{Error as _, SeqAccess, Visitor}; + use serde::ser::SerializeSeq; + use serde::{Deserializer, Serializer}; + + pub(super) fn serialize( + map: &BTreeMap, + serializer: S, + ) -> Result + where + S: Serializer, + { + let mut seq = serializer.serialize_seq(Some(map.len()))?; + for entry in map { + seq.serialize_element(&entry)?; + } + seq.end() + } + + pub(super) fn deserialize<'de, D>( + deserializer: D, + ) -> Result, D::Error> + where + D: Deserializer<'de>, + { + struct PairsVisitor; + + impl<'de> Visitor<'de> for PairsVisitor { + type Value = BTreeMap; + + fn expecting(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "a sequence of (OutPoint, height) pairs") + } + + fn visit_seq(self, mut seq: A) -> Result + where + A: SeqAccess<'de>, + { + // Stream elements one at a time and cap the count so an + // oversized declared length cannot force an unbounded + // allocation before we notice. + let mut map = BTreeMap::new(); + while let Some((outpoint, height)) = + seq.next_element::<(OutPoint, CoreBlockHeight)>()? + { + if map.len() >= MAX_OBSERVED_SPENT_OUTPOINTS { + return Err(A::Error::custom(format!( + "observed_spent_outpoints exceeds the maximum of {MAX_OBSERVED_SPENT_OUTPOINTS} entries" + ))); + } + map.insert(outpoint, height); + } + Ok(map) + } + } + + deserializer.deserialize_seq(PairsVisitor) + } } +/// Defensive upper bound on the number of [`ManagedWalletInfo::observed_spent_outpoints`] +/// entries accepted when deserializing a wallet. +/// +/// Chosen far above any plausible legitimate size (matched-block spend activity +/// over a realistic rescan — a busy matched block contributes on the order of +/// 10⁴ inputs, and steady state holds only the above-finality-boundary window) +/// so it only ever rejects a corrupted or hostile wallet file, never a real +/// one. At ~40 bytes per entry this caps the load-time allocation for this +/// field at a few tens of MB. +#[cfg(feature = "serde")] +const MAX_OBSERVED_SPENT_OUTPOINTS: usize = 1_000_000; + impl ManagedWalletInfo { /// Create new managed wallet info with network and wallet ID pub fn new(network: Network, wallet_id: [u8; 32]) -> Self { @@ -67,6 +242,8 @@ impl ManagedWalletInfo { accounts: ManagedAccountCollection::new(), balance: WalletCoreBalance::default(), instant_send_locks: HashSet::new(), + observed_spent_outpoints: BTreeMap::new(), + account_generation: 0, } } @@ -81,6 +258,8 @@ impl ManagedWalletInfo { accounts: ManagedAccountCollection::new(), balance: WalletCoreBalance::default(), instant_send_locks: HashSet::new(), + observed_spent_outpoints: BTreeMap::new(), + account_generation: 0, } } @@ -105,6 +284,8 @@ impl ManagedWalletInfo { accounts: ManagedAccountCollection::from_account_collection(&wallet.accounts), balance: WalletCoreBalance::default(), instant_send_locks: HashSet::new(), + observed_spent_outpoints: BTreeMap::new(), + account_generation: 0, } } @@ -136,6 +317,98 @@ impl ManagedWalletInfo { &self.instant_send_locks } + /// Read-only access to the wallet-level observed-spent outpoint set + /// (dashpay/rust-dashcore#649), mapping each on-chain-spent outpoint to the + /// height of the block that spent it. See the field's doc comment for the + /// invariants; exposed for diagnostics and tests. + pub fn observed_spent_outpoints(&self) -> &BTreeMap { + &self.observed_spent_outpoints + } + + /// Record every outpoint `tx` spends into [`Self::observed_spent_outpoints`] + /// at `height`. Insert-only bookkeeping — it never touches account UTXO sets, + /// so it is safe to call before `record_transaction` builds a spend's + /// `input_details` from the still-present funding UTXO. + /// + /// A coinbase spends nothing real (its single input is the null prevout), so + /// it is skipped. Call this only for block-context transactions + /// (`InBlock` / `InChainLockedBlock`); mempool spends must never be recorded + /// (see the field doc — an unconfirmed spend may never be mined). + /// + /// Returns whether the map actually changed (a new outpoint, or an existing + /// one re-observed at a different height). The map is persisted wallet + /// state, so callers must surface a `true` as a state modification — + /// otherwise a consumer that persists only on reported modifications could + /// drop the recorded spend across a restart and reopen #649 for it. + pub(crate) fn record_observed_spends( + &mut self, + tx: &Transaction, + height: CoreBlockHeight, + ) -> bool { + if tx.is_coin_base() { + return false; + } + let mut changed = false; + for input in &tx.input { + changed |= + self.observed_spent_outpoints.insert(input.previous_output, height) != Some(height); + } + changed + } + + /// Evict [`Self::observed_spent_outpoints`] entries at or below the finality + /// boundary `min(last_applied_chain_lock.block_height, synced_height)`. + /// + /// An entry `(outpoint, height)` with `height <= boundary` is safe to + /// forget: the spend at that height is chain-locked (never reorged out) and + /// any funding transaction for the outpoint has been delivered and finalized, + /// so no redelivery path can re-insert the coin (dashpay/rust-dashcore#649). + /// No-op until a chainlock has been applied (`last_applied_chain_lock` is + /// `None`) — without a finality boundary nothing can be proven final. + /// + /// Called on chainlock application and sync-checkpoint commit; not age- or + /// recency-based. + pub(crate) fn prune_finalized_observed_spends(&mut self) { + let Some(chain_lock) = self.metadata.last_applied_chain_lock.as_ref() else { + return; + }; + let boundary = chain_lock.block_height.min(self.metadata.synced_height); + self.observed_spent_outpoints.retain(|_, height| *height > boundary); + } + + /// Invalidate the wallet's sync certificate when an account is added. + /// + /// `synced_height` certifies "every filter at or below this height was + /// matched against the wallet's scripts" — but only for the account set that + /// existed while those filters were scanned. A newly added account has had no + /// filter coverage, so the certificate no longer holds for the current + /// account set, and [`Self::prune_finalized_observed_spends`] must not + /// consume it (dashpay/rust-dashcore#649). Rewinding `synced_height` to just + /// below wallet birth collapses the prune boundary and makes the sync layer + /// detect the wallet as behind and backfill from birth (the dash-spv filter + /// tick). A wallet still in initial sync (`synced_height` already at or below + /// the floor) is left untouched; `birth_height == 0` saturates safely. + /// + /// `last_processed_height` and `last_applied_chain_lock` are intentionally + /// left as-is: they are chain facts, not coverage facts, and keeping the + /// chainlock is what lets the backfilled history arrive born-chainlocked. + fn rewind_sync_checkpoint_for_new_account(&mut self) { + let floor = self.metadata.birth_height.saturating_sub(1); + self.metadata.synced_height = self.metadata.synced_height.min(floor); + // Bump unconditionally — even when the checkpoint was already at or + // below the floor and did not move. An in-flight filter scan snapshot + // taken before this add did not include the new account's scripts, and + // the generation mismatch is the only signal a commit-time check has + // when the rewind is height-invisible (dashpay/rust-dashcore#649). + self.account_generation += 1; + } + + /// Current generation of the wallet's account set; see the + /// [`Self::account_generation`] field doc. Bumped on every account add. + pub fn account_generation(&self) -> u64 { + self.account_generation + } + pub fn next_change_address( &mut self, wallet: &Wallet, diff --git a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs index 559549162..0d4de1335 100644 --- a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs +++ b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs @@ -184,6 +184,16 @@ pub trait WalletInfoInterface: Sized + WalletTransactionChecker + ManagedAccount /// Return the durable wallet sync checkpoint height. fn synced_height(&self) -> CoreBlockHeight; + /// Return the current generation of the wallet's account set — a counter + /// bumped on every account addition. Filter-sync layers snapshot it at + /// scan time and refuse to certify coverage at commit time when it has + /// changed, since the scan did not include the added account's scripts + /// (dashpay/rust-dashcore#649). The default (constant `0`) opts an + /// implementation out of the check. + fn account_generation(&self) -> u64 { + 0 + } + /// Return the highest chainlock that has been applied to this /// wallet, retaining the signing proof. Blocks at or below /// `chain_lock.block_height` are considered chainlock-finalized @@ -303,6 +313,10 @@ impl WalletInfoInterface for ManagedWalletInfo { self.metadata.synced_height } + fn account_generation(&self) -> u64 { + self.account_generation + } + fn last_applied_chain_lock(&self) -> Option<&ChainLock> { self.metadata.last_applied_chain_lock.as_ref() } @@ -343,6 +357,11 @@ impl WalletInfoInterface for ManagedWalletInfo { self.metadata.last_applied_chain_lock = Some(chain_lock); } + // Evict observed-spent entries the promotion above just made final. + // Must run after both the per-account promotion and the metadata + // advance, so the finality boundary reflects this chainlock. + self.prune_finalized_observed_spends(); + ApplyChainLockOutcome { locked_transactions, metadata_advanced: advance, @@ -492,6 +511,9 @@ impl WalletInfoInterface for ManagedWalletInfo { fn update_synced_height(&mut self, current_height: u32) { self.metadata.synced_height = current_height; + // A newly committed checkpoint can lift the finality boundary when the + // chainlock was already ahead of the old synced_height. + self.prune_finalized_observed_spends(); } fn matured_coinbase_records( From c6deac5fb6d692ad007d80f401aa08e689a8111e Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Thu, 6 Aug 2026 13:31:57 +0200 Subject: [PATCH 064/149] fix(dash-spv): fix dashd process port assignation (#923) --- dash-spv/src/test_utils/node.rs | 36 ++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/dash-spv/src/test_utils/node.rs b/dash-spv/src/test_utils/node.rs index 41dcd9b71..861bb82d5 100644 --- a/dash-spv/src/test_utils/node.rs +++ b/dash-spv/src/test_utils/node.rs @@ -16,17 +16,43 @@ use std::time::Duration; use tokio::process::Child; use tokio::time::{sleep, timeout}; -/// Atomic counter for unique port allocation across parallel tests. -/// Starts below the standard Dash regtest ports (19898/19899) to avoid conflicts. -static NEXT_PORT: AtomicU16 = AtomicU16::new(19400); +/// Base of the port range used by tests. Sits above the standard Dash regtest +/// ports (19898/19899) and below the Linux ephemeral range (32768+). +const PORT_RANGE_BASE: u16 = 20000; +/// Ports handed out per test process before wrapping into a neighbour's lane. +/// +/// Sized for the heaviest target: `dashd_masternode` runs 3 controller-only +/// tests (2 ports each) plus 7 full-network ones (controller + 4 masternodes at +/// 2 ports apiece), so ~76 ports in one process. +const PORT_LANE_WIDTH: u16 = 100; +/// Number of lanes carved out of the range +const PORT_LANE_COUNT: u16 = 120; + +const _: () = assert!( + PORT_RANGE_BASE as u32 + PORT_LANE_WIDTH as u32 * PORT_LANE_COUNT as u32 <= 32768, + "range must stay below the Linux ephemeral range" +); const MAX_PORT_ATTEMPTS: usize = 100; +/// Atomic counter for unique port allocation across parallel tests. +/// Seeded lazily from the PID +static NEXT_PORT: AtomicU16 = AtomicU16::new(0); + /// Allocate a unique, available TCP port for test use. pub(super) fn find_available_port() -> u16 { + let _ = NEXT_PORT.compare_exchange( + 0, + PORT_RANGE_BASE + (std::process::id() % PORT_LANE_COUNT as u32) as u16 * PORT_LANE_WIDTH, + Ordering::Relaxed, + Ordering::Relaxed, + ); + for _ in 0..MAX_PORT_ATTEMPTS { - let port = NEXT_PORT.fetch_add(1, Ordering::Relaxed); - assert!(port >= 1024, "port counter overflowed"); + let raw = NEXT_PORT.fetch_add(1, Ordering::Relaxed); + // Keep the counter inside the range no matter how many are allocated. + let port = PORT_RANGE_BASE + + raw.wrapping_sub(PORT_RANGE_BASE) % (PORT_LANE_WIDTH * PORT_LANE_COUNT); if std::net::TcpListener::bind(("127.0.0.1", port)).is_ok() { return port; } From 08bf729de819f52973002f754342fedac14a06db Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Thu, 6 Aug 2026 20:50:38 +0700 Subject: [PATCH 065/149] feat(key-wallet-manager): add a lossless persistence event channel to stop the sync-watermark freeze (#927) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(key-wallet-manager): lossless persistence event channel to stop the sync-watermark freeze The wallet event bus is a single bounded `tokio::broadcast` ring (`DEFAULT_WALLET_EVENT_CAPACITY` = 1000), fanned out fire-and-forget from `process_block.rs`. It feeds two very different consumers: 1. incidental subscribers reached via `subscribe_events()` — dash-spv's `EventHandler` dispatch and the unit tests — for which a dropped event under overload is harmless; and 2. the platform durable-persistence consumer, for which a dropped event is NOT harmless: the dropped `TransactionDetected`/`BlockProcessed` rows never reach disk while the surviving `SyncHeightAdvanced` watermark keeps advancing, so the persisted sync height outruns the rows it implies. The platform `#4069` guard then latches that wallet's watermark into a *permanent* freeze for the rest of the process. In the field this is the mainnet sync that climbs, then "rolls back" to a stale height on every relaunch. Under a heavy historical SPV catch-up a burst larger than the ring between consumer drains overflows it (`RecvError::Lagged`) and trips exactly that freeze. Batching the persistence stores (shipped earlier) raised the burst threshold but a single lag still freezes forever. Fix: give the persistence consumer its own dedicated, unbounded `tokio::mpsc` channel that carries the same event stream, in the same order, with no drops — so it can never observe `Lagged` and its freeze guard never fires. The existing broadcast is kept unchanged for the incidental subscribers (no behavioural change for dash-spv or tests). Why unbounded rather than a bounded back-pressuring channel: several emit sites run inside this manager's `RwLock` write guard (SPV block processing holds `wallet.write().await` across `process_block_for_wallets`), while the persistence consumer needs a `read()` lock on the *same* manager to project each event. A bounded `send().await`/`blocking_send` that parked under the write guard would deadlock the very consumer that must drain it. A non-blocking unbounded enqueue keeps every emit path lock-safe while still guaranteeing losslessness. All emission now flows through a single `emit_event` choke point that pushes to both channels in order. `#4069`-safety: unchanged ordering + zero drops means the persistence consumer sees every row event before the watermark event that implies it, exactly as the non-lagged broadcast path already guaranteed; the durable watermark can never outrun its rows. - lib.rs: add `persistence_sender`/`persistence_receiver` + `emit_event`. - accessors.rs: add `take_persistence_receiver()` (handed to the consumer once). - process_block.rs: route all six emit sites through `emit_event`. - event_tests.rs: prove a 5000-event burst is delivered losslessly and in order on the persistence channel while the bounded broadcast lags. Co-Authored-By: Claude Opus 4.8 * fix(key-wallet-manager): make the persistence channel opt-in and surface a lost consumer Addresses two CodeRabbit findings on the lossless persistence mpsc: - Stability & Availability: `WalletManager::new` unconditionally created and retained `mpsc::unbounded_channel()`, so a manager whose consumer is never installed (only tests call `take_persistence_receiver`) would accumulate every emitted event without bound. Make persistence delivery opt-in: the channel is now created lazily by `take_persistence_receiver` (the send half is `None` until a consumer installs itself), so `emit_event` enqueues nothing when no consumer exists. Unbounded + non-blocking is retained for installed consumers, which is required: several emit sites run inside the manager's RwLock write guard while the consumer needs a read lock to drain, so a bounded blocking send would deadlock, and a bounded try_send that dropped would reintroduce the watermark freeze (#4069). The documented take-before-emit contract means no pre-consumer events are lost. - Data Integrity: `emit_event` ignored `SendError`. If the installed consumer drops its receiver while the manager is still running, durable persistence has silently stopped. Surface it: log once (latched) that the consumer was lost. We deliberately do not halt in-memory advancement — the manager does not own durable state, and on restart the wallet re-scans from the last persisted height, so a lost consumer causes no durable corruption. All key-wallet-manager tests pass, including the lossless-burst persistence test. Co-Authored-By: Claude Opus 4.8 * style(key-wallet-manager): rustfmt the persistence-lost guard in emit_event Collapse the hand-wrapped `&&` condition onto one line to satisfy cargo fmt --check (the pre-commit gate). Formatting only; no behavior change. Co-Authored-By: Claude Opus 4.8 * fix(key-wallet-manager): warn when a persistence consumer installs after emission has begun Installing the persistence consumer after events were already emitted leaves those events permanently absent from the persistence stream with no Lagged-style marker — the exact invisible-loss failure mode the channel exists to eliminate. Latch events_emitted in emit_event and warn from take_persistence_receiver on a late install (the receiver is still returned; delivery is lossless from that point on). Also pin the opt-in contract with tests: the receiver is taken exactly once, a late install delivers only post-install events (no invented replay), and a dropped consumer neither wedges nor panics emission. Co-Authored-By: Claude Fable 5 * refactor(key-wallet-manager): remove the unused public event_sender accessor event_sender() exposed the raw broadcast::Sender, letting downstream code publish a WalletEvent to the lossy broadcast while bypassing the lossless persistence channel — the exact split-brain emit_event exists to prevent. It has zero callers anywhere (this workspace including dash-spv, and the dashpay/platform consumer); subscription goes through subscribe_events(). Removing it makes emit_event the only emission surface structurally, not just by convention. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: bfoss765 <38437574+bfoss765@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 --- key-wallet-manager/src/accessors.rs | 45 ++++++- key-wallet-manager/src/event_tests.rs | 166 ++++++++++++++++++++++++ key-wallet-manager/src/lib.rs | 109 +++++++++++++++- key-wallet-manager/src/process_block.rs | 21 ++- 4 files changed, 329 insertions(+), 12 deletions(-) diff --git a/key-wallet-manager/src/accessors.rs b/key-wallet-manager/src/accessors.rs index db66cb763..edee935b5 100644 --- a/key-wallet-manager/src/accessors.rs +++ b/key-wallet-manager/src/accessors.rs @@ -7,7 +7,7 @@ use key_wallet::wallet::managed_wallet_info::wallet_info_interface::WalletInfoIn use key_wallet::wallet::managed_wallet_info::TransactionRecord; use key_wallet::{Account, Address, Network, Utxo, Wallet}; use std::collections::{BTreeMap, BTreeSet}; -use tokio::sync::broadcast; +use tokio::sync::{broadcast, mpsc}; impl WalletManager { /// Get a wallet by ID @@ -204,9 +204,46 @@ impl WalletManager { self.event_sender.subscribe() } - /// Get a reference to the event sender for emitting events. - pub fn event_sender(&self) -> &broadcast::Sender { - &self.event_sender + /// Install a durable persistence consumer and take its event receiver + /// (dashpay/platform#4069). + /// + /// Persistence delivery is opt-in: the first call **creates** the lossless, + /// unbounded persistence channel, installs the send half on the manager, and + /// returns the receive half; every subsequent call returns `None`. Creating + /// the channel only when a consumer asks for it means a manager that never + /// installs a consumer never accumulates an undrained backlog of events (an + /// unbounded `mpsc` with no reader would otherwise grow without limit). + /// + /// The platform durable consumer calls this before the manager is shared + /// with any producer, then drains the stream losslessly (see the + /// `persistence_sender` field docs). Because installation precedes emission, + /// no events are emitted before the consumer is in place; unlike + /// [`subscribe_events`](Self::subscribe_events) there is no + /// subscribe-before-publish race, and unlike a `broadcast::Receiver` the + /// unbounded `mpsc` never `Lagged`s a row-bearing or watermark event. + pub fn take_persistence_receiver(&mut self) -> Option> { + if self.persistence_sender.is_some() { + // A consumer has already been installed; the receiver is taken once. + return None; + } + // Installing after emission has begun violates the documented contract: + // everything emitted so far reached only the lossy broadcast and is + // permanently absent from the persistence stream, with no + // `Lagged`-style marker to reveal the gap. The receiver is still + // returned (delivery is lossless from this point on), but the gap must + // not be silent — it is exactly the invisible-loss failure mode this + // channel exists to eliminate. + if self.events_emitted.load(std::sync::atomic::Ordering::Relaxed) { + tracing::warn!( + "take_persistence_receiver called after wallet events were already emitted; \ + earlier events are absent from the persistence stream. Install the persistence \ + consumer before the manager is shared with any producer \ + (dashpay/platform#4069)." + ); + } + let (sender, receiver) = mpsc::unbounded_channel(); + self.persistence_sender = Some(sender); + Some(receiver) } /// Return the total monitor revision (structural + per-wallet account revisions). diff --git a/key-wallet-manager/src/event_tests.rs b/key-wallet-manager/src/event_tests.rs index 05d89c2f0..3f8eb4052 100644 --- a/key-wallet-manager/src/event_tests.rs +++ b/key-wallet-manager/src/event_tests.rs @@ -1339,3 +1339,169 @@ async fn test_block_processing_stamps_in_block_position() { "second tx in block.txdata must be stamped position 1" ); } + +// --------------------------------------------------------------------------- +// Lossless persistence channel (dashpay/platform#4069) +// --------------------------------------------------------------------------- + +/// A large burst of watermark events that the durable-persistence consumer +/// does not drain until the very end is delivered **losslessly and in order** +/// over the unbounded persistence channel — the exact property the platform +/// consumer's durable sync watermark relies on. The bounded broadcast, by +/// contrast, drops events (`Lagged`) under the same burst; that drop is the +/// freeze root cause this dedicated channel removes. +/// +/// This models a stalled/slow persistence consumer during a heavy SPV +/// catch-up: neither receiver is drained while `BURST` monotonically +/// increasing `SyncHeightAdvanced` watermarks are emitted. On the unbounded +/// channel every watermark survives, so the watermark can keep advancing to +/// the tip after the stall clears; on the broadcast it lags and the watermark +/// would freeze. +#[tokio::test] +async fn persistence_channel_is_lossless_under_a_large_burst() { + // `BURST` far exceeds the broadcast ring (`DEFAULT_WALLET_EVENT_CAPACITY` + // == 1000), so the bounded broadcast is guaranteed to lag. + const BURST: u32 = 5000; + + let (mut manager, wallet_id, _addr) = setup_manager_with_wallet(); + // Take the lossless persistence receiver AND subscribe a bounded broadcast + // receiver before emitting; neither is drained during the burst. + let mut persistence_rx = + manager.take_persistence_receiver().expect("persistence receiver available once"); + let mut broadcast_rx = manager.subscribe_events(); + + for h in 1..=BURST { + manager.update_wallet_synced_height(&wallet_id, h); + } + + // Persistence channel: every watermark arrived, in order, no gaps. + let mut received: Vec = Vec::with_capacity(BURST as usize); + while let Ok(event) = persistence_rx.try_recv() { + match event { + WalletEvent::SyncHeightAdvanced { + wallet_id: w, + height, + } => { + assert_eq!(w, wallet_id, "watermark for the wrong wallet"); + received.push(height); + } + other => panic!("unexpected event on persistence channel: {other:?}"), + } + } + assert_eq!( + received.len(), + BURST as usize, + "persistence channel dropped events: got {} of {BURST}", + received.len() + ); + assert!( + received.windows(2).all(|w| w[0] + 1 == w[1]), + "persistence channel reordered or gapped the watermark stream" + ); + assert_eq!(received.first().copied(), Some(1)); + assert_eq!(received.last().copied(), Some(BURST), "final watermark must reach the tip"); + + // Broadcast channel: the same burst overflows the bounded ring and drops + // events — demonstrating why the persistence consumer must NOT use it. + let mut broadcast_lagged = false; + let mut broadcast_delivered = 0usize; + loop { + match broadcast_rx.try_recv() { + Ok(_) => broadcast_delivered += 1, + Err(tokio::sync::broadcast::error::TryRecvError::Lagged(_)) => broadcast_lagged = true, + Err(_) => break, + } + } + assert!( + broadcast_lagged, + "the bounded broadcast should have lagged under a {BURST}-event burst" + ); + assert!( + broadcast_delivered < BURST as usize, + "broadcast should have dropped events but delivered all {BURST}" + ); +} + +/// The receiver is taken exactly once: the second call returns `None` and the +/// first receiver keeps working. Platform's manager construction relies on +/// exactly this (`take_persistence_receiver().expect(..)` on a fresh manager). +#[tokio::test] +async fn persistence_receiver_is_taken_exactly_once() { + let (mut manager, wallet_id, _addr) = setup_manager_with_wallet(); + let mut first = manager.take_persistence_receiver().expect("first take must succeed"); + assert!(manager.take_persistence_receiver().is_none(), "second take must return None"); + manager.update_wallet_synced_height(&wallet_id, 1); + assert!( + matches!( + first.try_recv(), + Ok(WalletEvent::SyncHeightAdvanced { + height: 1, + .. + }) + ), + "the first receiver must keep receiving after a second take attempt" + ); +} + +/// Late install (contract violation): events emitted before +/// `take_persistence_receiver` are permanently absent from the persistence +/// stream — the channel is created empty and only delivers from installation +/// onward. The accessor warns (see its docs); this pins the behavioural half: +/// no silent replay is invented. +#[tokio::test] +async fn late_install_delivers_only_post_install_events() { + let (mut manager, wallet_id, _addr) = setup_manager_with_wallet(); + // Emitted BEFORE any consumer exists: reaches only the lossy broadcast. + manager.update_wallet_synced_height(&wallet_id, 7); + + let mut rx = + manager.take_persistence_receiver().expect("late install still returns the receiver"); + manager.update_wallet_synced_height(&wallet_id, 8); + + match rx.try_recv() { + Ok(WalletEvent::SyncHeightAdvanced { + height, + .. + }) => { + assert_eq!(height, 8, "only post-install events may be delivered"); + } + other => panic!("expected the post-install watermark, got {other:?}"), + } + assert!( + rx.try_recv().is_err(), + "the pre-install event must not be replayed into the persistence stream" + ); +} + +/// A consumer that drops its receiver while the manager is running must not +/// wedge or panic the emit paths: in-memory processing continues and the +/// broadcast fan-out still delivers (the lost-consumer anomaly is logged once +/// inside `emit_event`). +#[tokio::test] +async fn dropped_persistence_consumer_does_not_wedge_emission() { + let (mut manager, wallet_id, _addr) = setup_manager_with_wallet(); + let rx = manager.take_persistence_receiver().expect("receiver available once"); + drop(rx); + + let mut broadcast_rx = manager.subscribe_events(); + // Two emissions: the first trips the log-once latch, the second proves + // emission keeps flowing afterwards. + manager.update_wallet_synced_height(&wallet_id, 21); + manager.update_wallet_synced_height(&wallet_id, 22); + + let mut heights = Vec::new(); + while let Ok(event) = broadcast_rx.try_recv() { + if let WalletEvent::SyncHeightAdvanced { + height, + .. + } = event + { + heights.push(height); + } + } + assert_eq!( + heights, + vec![21, 22], + "broadcast delivery must be unaffected by a lost persistence consumer" + ); +} diff --git a/key-wallet-manager/src/lib.rs b/key-wallet-manager/src/lib.rs index b3337e7a3..3ef0a9398 100644 --- a/key-wallet-manager/src/lib.rs +++ b/key-wallet-manager/src/lib.rs @@ -40,7 +40,7 @@ use std::str::FromStr; use dashcore::address::NetworkUnchecked; use key_wallet::wallet::managed_wallet_info::fee::FeeRate; -use tokio::sync::broadcast; +use tokio::sync::{broadcast, mpsc}; /// Default capacity for the wallet event bus. const DEFAULT_WALLET_EVENT_CAPACITY: usize = 1000; @@ -127,8 +127,65 @@ pub struct WalletManager, + /// Lossless, unbounded persistence event channel (dashpay/platform#4069). + /// + /// Carries the *same* event stream, in the *same* order, to the single + /// durable-persistence consumer with **no drops**. The platform consumer + /// projects each event into a persisted changeset; a dropped + /// record/watermark event there lets the durable sync watermark advance + /// past rows that never reached disk, which its `#4069` guard then latches + /// into a permanent freeze. An unbounded `mpsc` can never `Lagged`, so that + /// guard never fires. + /// + /// It is deliberately unbounded rather than a bounded back-pressuring + /// channel: several emit sites run *inside this manager's `RwLock` write + /// guard* (SPV block processing holds `wallet.write().await` across + /// `process_block_for_wallets`), while the consumer needs a `read()` lock + /// on the same manager to project each event. A bounded + /// `send().await`/`blocking_send` that parked under the write guard would + /// therefore deadlock the very consumer that must drain it. A non-blocking + /// unbounded enqueue keeps the emit paths lock-safe. See [`emit_event`]. + /// + /// Persistence delivery is **opt-in**: this is `None` until a durable + /// consumer installs itself by calling [`take_persistence_receiver`], which + /// creates the channel and returns the receive half. Until then + /// [`emit_event`] enqueues nothing, so a `WalletManager` used without a + /// persistence consumer (e.g. a pure wallet-management embedding that never + /// drives block processing, or one that simply never installs a consumer) + /// cannot accumulate an unbounded backlog of undrained events. The + /// documented contract is that the consumer takes the receiver *before* the + /// manager is shared with any producer, so no pre-consumer events are lost. + /// + /// [`take_persistence_receiver`]: WalletManager::take_persistence_receiver + /// [`emit_event`]: WalletManager::emit_event + persistence_sender: Option>, + /// Latches `true` the first time a persistence send fails because the + /// installed consumer dropped its receiver while the manager is still + /// running, so that anomaly is surfaced (logged) exactly once instead of on + /// every subsequent emit. + persistence_consumer_lost: std::sync::atomic::AtomicBool, + /// Latches `true` on the first [`emit_event`] call. Read by + /// [`take_persistence_receiver`] to warn when a persistence consumer + /// installs itself only AFTER events have already been emitted — those + /// events reached only the lossy broadcast and are permanently absent from + /// the persistence stream, with no `Lagged`-style marker to reveal the gap. + /// + /// [`emit_event`]: WalletManager::emit_event + /// [`take_persistence_receiver`]: WalletManager::take_persistence_receiver + events_emitted: std::sync::atomic::AtomicBool, } impl WalletManager { @@ -140,7 +197,55 @@ impl WalletManager { wallet_infos: BTreeMap::new(), structural_revision: 0, event_sender: broadcast::Sender::new(DEFAULT_WALLET_EVENT_CAPACITY), + // Persistence delivery is opt-in; the channel is created lazily when + // a consumer calls `take_persistence_receiver`. See that field's docs. + persistence_sender: None, + persistence_consumer_lost: std::sync::atomic::AtomicBool::new(false), + events_emitted: std::sync::atomic::AtomicBool::new(false), + } + } + + /// Emit a wallet event to BOTH the incidental broadcast fan-out and the + /// lossless persistence channel (dashpay/platform#4069). + /// + /// This is the single emit choke point, so both channels observe events in + /// the same order (the manager is the only producer). The persistence send + /// is a non-blocking, non-dropping `mpsc` enqueue: it is safe to call from + /// the synchronous emit paths that run while this manager's `RwLock` write + /// guard is held, whereas a bounded blocking send there would deadlock the + /// persistence consumer (which needs a `read()` lock to drain). See the + /// `persistence_sender` field docs for the full rationale. + fn emit_event(&self, event: WalletEvent) { + self.events_emitted.store(true, std::sync::atomic::Ordering::Relaxed); + // Lossless path: the persistence consumer must never miss a row-bearing + // or watermark event, or its durable sync height freezes. Only enqueue + // once a consumer has installed itself (opt-in), so a manager with no + // persistence consumer never accumulates an undrained backlog. + if let Some(sender) = &self.persistence_sender { + // `send` fails only after the consumer dropped its receiver. Under + // the documented contract that happens at manager shutdown, when + // there is nothing left to persist to. If it happens *while the + // manager is still running* (a consumer task that exited early), + // durable persistence has silently stopped: surface it loudly, once. + // We do not halt in-memory state advancement — the manager does not + // own durable state, and on restart the wallet re-scans from the + // last persisted height, so a lost consumer causes no durable + // corruption (dashpay/platform#4069). + if sender.send(event.clone()).is_err() + && !self.persistence_consumer_lost.swap(true, std::sync::atomic::Ordering::Relaxed) + { + tracing::error!( + "wallet-manager persistence consumer dropped its receiver while the manager is \ + still running; durable persistence has stopped. In-memory state keeps \ + advancing and is recovered by a re-scan from the last persisted height on \ + restart, so there is no durable corruption (dashpay/platform#4069)." + ); + } } + // Lossy-tolerant fan-out for incidental subscribers (dash-spv + // `EventHandler` dispatch, tests). A `Lagged` drop here never affects + // the durable watermark. + let _ = self.event_sender.send(event); } /// Increment the structural revision for wallet/account additions or removals. diff --git a/key-wallet-manager/src/process_block.rs b/key-wallet-manager/src/process_block.rs index 11db8aeef..b751ada64 100644 --- a/key-wallet-manager/src/process_block.rs +++ b/key-wallet-manager/src/process_block.rs @@ -170,7 +170,7 @@ impl WalletInterface for WalletM account_balances: account_balances.clone(), addresses_derived: project_derived_addresses(for_record), }; - let _ = self.event_sender.send(event); + self.emit_event(event); } // If any derivations were left unattributed (records vector // didn't cover every account that derived), log so the @@ -204,7 +204,7 @@ impl WalletInterface for WalletM balance, account_balances: account_balances.clone(), }; - let _ = self.event_sender.send(event); + self.emit_event(event); } } } @@ -282,7 +282,7 @@ impl WalletInterface for WalletM if let Some(info) = self.wallet_infos.get_mut(wallet_id) { if height > info.synced_height() { info.update_synced_height(height); - let _ = self.event_sender.send(WalletEvent::SyncHeightAdvanced { + self.emit_event(WalletEvent::SyncHeightAdvanced { wallet_id: *wallet_id, height, }); @@ -314,6 +314,12 @@ impl WalletInterface for WalletM } fn apply_chain_lock(&mut self, chain_lock: ChainLock) { + // Collect the events under the `iter_mut` borrow, then emit them once + // the mutable borrow of `self.wallet_infos` has ended. `emit_event` + // takes `&self`, which cannot overlap the live `iter_mut` borrow; the + // BTreeMap iteration order is preserved, so the emit order is + // unchanged. + let mut events = Vec::new(); for (wallet_id, info) in self.wallet_infos.iter_mut() { let outcome = info.apply_chain_lock(chain_lock.clone()); @@ -323,13 +329,16 @@ impl WalletInterface for WalletM // promoted nothing). Replays of the same chainlock (no // metadata advance) are silent. if outcome.metadata_advanced { - let _ = self.event_sender.send(WalletEvent::ChainLockProcessed { + events.push(WalletEvent::ChainLockProcessed { wallet_id: *wallet_id, chain_lock: chain_lock.clone(), locked_transactions: outcome.locked_transactions, }); } } + for event in events { + self.emit_event(event); + } } fn process_instant_send_lock(&mut self, instant_lock: InstantLock) { @@ -361,7 +370,7 @@ impl WalletInterface for WalletM }; let prior = prior_account_balances.remove(&wallet_id).unwrap_or_default(); let account_balances = diff_account_balances(&prior, &info.account_balances()); - let _ = self.event_sender().send(WalletEvent::TransactionInstantLocked { + self.emit_event(WalletEvent::TransactionInstantLocked { wallet_id, txid, instant_lock: instant_lock.clone(), @@ -501,7 +510,7 @@ impl WalletManager { account_balances, addresses_derived, }; - let _ = self.event_sender.send(event); + self.emit_event(event); } } } From dca5b05b83a9a2d6fd7b650e7cce4356d1c2ec29 Mon Sep 17 00:00:00 2001 From: romchornyi Date: Thu, 6 Aug 2026 20:20:23 +0300 Subject: [PATCH 066/149] feat(key-wallet): OP_RETURN outputs, output-order control and change-to-VIN0 (#922) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(key-wallet): add MAYA op return support * fix(key-wallet): validate routed change outputs Estimate routed change using the largest eligible input script and reject configured network mismatches. Add regression coverage for P2WSH sizing and cross-network change routing. * fix(key-wallet): size the dust threshold from the routed change script `change_amount > 546` hard-codes the dust threshold for a 34-byte P2PKH output. With `change_to_first_input` the change script is whatever VIN0 uses, and Dash Core derives dust from the serialized output plus the 148-byte input that would spend it — so a 43-byte output has a 573-duff threshold. A 550-duff routed change output cleared the flat 546 check and could then be rejected as dust by the network. Derive the threshold with `3 * (output_size + 148)`, sized from the same `estimated_change_output_size()` that coin selection already uses, so selection and output construction agree. Hoisted above the point `self.outputs` is moved. Nothing moves for the ordinary path: the formula is exactly 546 for a 34-byte P2PKH output, which `test_dust_threshold_follows_the_change_output_size` pins alongside the 43-byte case. --------- Co-authored-by: Roman <51091564+jeanpierreroma@users.noreply.github.com> --- .../transaction_builder.rs | 534 +++++++++++++++++- 1 file changed, 513 insertions(+), 21 deletions(-) diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 076581ed6..7a86d7f17 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -11,12 +11,13 @@ use crate::wallet::managed_wallet_info::coin_selection::{ use crate::wallet::managed_wallet_info::fee::FeeRate; use crate::{Account, DerivationPath, Signer, Utxo, Wallet}; use core::fmt; +use dashcore::blockdata::opcodes; use dashcore::blockdata::script::{Builder, PushBytes, ScriptBuf}; use dashcore::blockdata::transaction::special_transaction::TransactionPayload; use dashcore::blockdata::transaction::{OutPoint, Transaction}; use dashcore::sighash::{EcdsaSighashType, LegacySighash, SighashCache}; use dashcore::Address; -use dashcore::{TxIn, TxOut}; +use dashcore::{Network, TxIn, TxOut}; use dashcore_hashes::Hash; use secp256k1::ecdsa::Signature; use secp256k1::{Message, PublicKey, Secp256k1}; @@ -25,6 +26,10 @@ use std::cmp::Ordering; /// A transaction with more inputs would exceed the relay standard-size cap (~100 KB at ~148 /// bytes/signed input) and be rejected by the network const MAX_STANDARD_TX_INPUTS: usize = 500; +/// Relay-policy limit for OP_RETURN payloads, matching Dash Core's `-datacarriersize` default. +/// Public so callers can reject an over-long payload before handing over a builder +/// `add_op_return` would consume. +pub const MAX_STANDARD_OP_RETURN_BYTES: usize = 80; /// Calculate varint size for a given number fn varint_size(n: usize) -> usize { @@ -36,6 +41,27 @@ fn varint_size(n: usize) -> usize { } } +fn serialized_output_size(output: &TxOut) -> usize { + serialized_script_output_size(&output.script_pubkey) +} + +fn serialized_script_output_size(script_pubkey: &ScriptBuf) -> usize { + let script_len = script_pubkey.as_bytes().len(); + 8 + varint_size(script_len) + script_len +} + +fn addresses_share_network(left: &Address, right: &Address) -> bool { + // Checked addresses retain their encoded network prefix, not a unique Network value: + // legacy testnet/devnet/regtest addresses intentionally share one prefix. Treat two + // addresses as compatible when at least one supported network accepts both encodings. + [Network::Mainnet, Network::Testnet, Network::Devnet, Network::Regtest].into_iter().any( + |network| { + left.as_unchecked().is_valid_for_network(network) + && right.as_unchecked().is_valid_for_network(network) + }, + ) +} + /// Transaction builder for creating Dash transactions /// /// This builder implements BIP-69 (Lexicographical Indexing of Transaction Inputs and Outputs) @@ -49,6 +75,8 @@ pub struct TransactionBuilder { current_height: u32, selection_strategy: SelectionStrategy, require_final_inputs: bool, + preserve_output_order: bool, + change_to_first_input: bool, /// Special transaction payload for Dash-specific transactions special_payload: Option, /// Reservation set of the funding account, captured by `set_funding`. The @@ -74,6 +102,8 @@ impl TransactionBuilder { current_height: 0, selection_strategy: SelectionStrategy::BranchAndBound, require_final_inputs: false, + preserve_output_order: false, + change_to_first_input: false, special_payload: None, reservations: None, } @@ -135,9 +165,11 @@ impl TransactionBuilder { /// Add an output to a specific address /// - /// Note: Outputs will be sorted according to BIP-69 when the transaction is built: + /// Note: by default outputs are sorted according to BIP-69 when the transaction is built: /// - First by amount (ascending) /// - Then by scriptPubKey (lexicographically) + /// + /// Call [`Self::preserve_output_order`] to keep insertion order instead. pub fn add_output(mut self, address: &Address, amount: u64) -> Self { let script_pubkey = address.script_pubkey(); self.outputs.push(TxOut { @@ -147,6 +179,44 @@ impl TransactionBuilder { self } + /// Add an OP_RETURN output carrying `data` (value 0). + /// + /// Errors if `data` exceeds [`MAX_STANDARD_OP_RETURN_BYTES`]. + pub fn add_op_return(mut self, data: &[u8]) -> Result { + if data.len() > MAX_STANDARD_OP_RETURN_BYTES { + return Err(BuilderError::OpReturnDataTooLarge { + len: data.len(), + max: MAX_STANDARD_OP_RETURN_BYTES, + }); + } + + let push_bytes = + <&PushBytes>::try_from(data).map_err(|_| BuilderError::OpReturnDataTooLarge { + len: data.len(), + max: MAX_STANDARD_OP_RETURN_BYTES, + })?; + self.outputs.push(TxOut { + value: 0, + script_pubkey: Builder::new() + .push_opcode(opcodes::all::OP_RETURN) + .push_slice(push_bytes) + .into_script(), + }); + Ok(self) + } + + /// Preserve outputs in insertion order instead of applying BIP-69 sorting. + pub fn preserve_output_order(mut self) -> Self { + self.preserve_output_order = true; + self + } + + /// Route change to the address of the first selected input (post-sort). + pub fn change_to_first_input(mut self) -> Self { + self.change_to_first_input = true; + self + } + pub fn set_fee_rate(mut self, fee_rate: FeeRate) -> Self { self.fee_rate = fee_rate; self @@ -167,6 +237,53 @@ impl TransactionBuilder { } } + fn should_estimate_change_output(&self) -> bool { + self.change_addr.is_some() || self.change_to_first_input + } + + /// Dust threshold for an output of `output_size` serialized bytes, mirroring Dash Core's + /// `GetDustThreshold`: the output is dust when spending it would cost more than a third of + /// its value at the dust relay fee, i.e. `3 * (output_size + 148)` duffs for the 148-byte + /// P2PKH input that would spend it. + /// + /// The flat 546 this replaces is exactly this formula for a 34-byte P2PKH output, so nothing + /// moves for the ordinary path. It matters for `change_to_first_input`, where change is + /// routed to whatever script VIN0 uses: a 43-byte output has a 573-duff threshold, so a + /// 550-duff change output would have passed a flat 546 check and then been rejected as dust + /// by the network. + fn dust_threshold(output_size: usize) -> u64 { + 3 * (output_size as u64 + 148) + } + + fn estimated_change_output_size(&self) -> usize { + if self.change_to_first_input { + // Coin selection may choose any spendable input, then BIP-69 determines VIN0. + // Reserve the largest eligible routing script so every possible winner is covered. + self.inputs + .iter() + .filter(|utxo| utxo.is_spendable(self.current_height)) + .map(|utxo| serialized_script_output_size(&utxo.address.script_pubkey())) + .max() + .unwrap_or(CHANGE_OUTPUT_SIZE) + } else { + self.change_addr + .as_ref() + .map(|address| serialized_script_output_size(&address.script_pubkey())) + .unwrap_or(0) + } + } + + fn effective_outputs_size(&self) -> usize { + match &self.special_payload { + // The asset-lock burn output is sized at the flat TX_OUTPUT_SIZE rather than its + // real ~11 serialized bytes. The real size is smaller, so this over-estimates and + // over-pays slightly — deliberately kept identical to the pre-OP_RETURN behaviour + // so identity funding fees do not move as a side effect of Maya support. + Some(TransactionPayload::AssetLockPayloadType(_)) => TX_OUTPUT_SIZE, + _ => self.outputs.iter().map(serialized_output_size).sum(), + } + } + /// Calculate the base transaction size excluding inputs /// Based on dashsync/DashSync/shared/Models/Transactions/Base/DSTransaction.m fn calculate_base_size(&self) -> usize { @@ -181,19 +298,21 @@ impl TransactionBuilder { // Add varint for output count size += varint_size( outputs_count - + if self.change_addr.is_some() { + + if self.should_estimate_change_output() { 1 } else { 0 }, ); - // Add outputs size (P2PKH output) - size += outputs_count * TX_OUTPUT_SIZE; + // Add serialized output sizes. A P2PKH output still measures TX_OUTPUT_SIZE; + // non-standard outputs such as OP_RETURN are sized from their real script. + size += self.effective_outputs_size(); - // Add change output if we have a change address - if self.change_addr.is_some() { - size += CHANGE_OUTPUT_SIZE; + // Add change using the actual configured script size, or the largest eligible input + // script when change will be routed to VIN0. + if self.should_estimate_change_output() { + size += self.estimated_change_output_size(); } // Add special payload size if present @@ -306,6 +425,7 @@ impl TransactionBuilder { // estimate doesn't include a phantom (~34-byte) change output. if self.selection_strategy == SelectionStrategy::All { self.change_addr = None; + self.change_to_first_input = false; } // For AssetLock the on-chain spend equals the OP_RETURN burn, which @@ -322,12 +442,8 @@ impl TransactionBuilder { self.inputs.retain(|utxo| utxo.is_confirmed || utxo.is_instantlocked); } - // Matches `calculate_base_size`: a change output is only budgeted with a change address. - let change_output_size = if self.change_addr.is_some() { - CHANGE_OUTPUT_SIZE - } else { - 0 - }; + // Must match `calculate_base_size`, including the conservative VIN0 routing-script size. + let change_output_size = self.estimated_change_output_size(); let selection = CoinSelector::new(self.selection_strategy) .select_coins_with_size( @@ -358,8 +474,12 @@ impl TransactionBuilder { }); } + selected_inputs.sort_by(bip69_input_sorter); + let change_amount = total_input.saturating_sub(total_output).saturating_sub(selection.estimated_fee); + // Computed before `self.outputs` is moved out below. + let change_dust_threshold = Self::dust_threshold(self.estimated_change_output_size()); let mut tx_outputs = match &self.special_payload { Some(TransactionPayload::AssetLockPayloadType(_)) => vec![TxOut { value: total_output, @@ -399,22 +519,39 @@ impl TransactionBuilder { }; credit.value = drained; } - } else if change_amount > 546 { + } else if change_amount > change_dust_threshold { // Add change output if above dust threshold - let Some(change_addr) = self.change_addr else { - return Err(BuilderError::NoChangeAddress); + let change_script_pubkey = if self.change_to_first_input { + let Some(first_input) = selected_inputs.first() else { + return Err(BuilderError::NoInputs); + }; + if self.change_addr.as_ref().is_some_and(|change_addr| { + !addresses_share_network(&first_input.address, change_addr) + }) { + return Err(BuilderError::InvalidData( + "Input-derived change address network does not match configured change address" + .into(), + )); + } + first_input.address.script_pubkey() + } else { + let Some(change_addr) = self.change_addr else { + return Err(BuilderError::NoChangeAddress); + }; + change_addr.script_pubkey() }; + tx_outputs.push(TxOut { value: change_amount, - script_pubkey: change_addr.script_pubkey(), + script_pubkey: change_script_pubkey, }); } - if !matches!(self.special_payload, Some(TransactionPayload::AssetLockPayloadType(_))) { + if !self.preserve_output_order + && !matches!(self.special_payload, Some(TransactionPayload::AssetLockPayloadType(_))) + { tx_outputs.sort_by(bip69_output_sorter); } - - selected_inputs.sort_by(bip69_input_sorter); let tx_inputs: Vec = selected_inputs .iter() .map(|utxo| TxIn { @@ -701,6 +838,11 @@ pub enum BuilderError { count: usize, max: usize, }, + /// OP_RETURN payload exceeds the standard relay-policy size. + OpReturnDataTooLarge { + len: usize, + max: usize, + }, } impl fmt::Display for BuilderError { @@ -727,6 +869,12 @@ impl fmt::Display for BuilderError { } => { write!(f, "Too many inputs for a standard transaction: {count} (max {max})") } + Self::OpReturnDataTooLarge { + len, + max, + } => { + write!(f, "OP_RETURN payload too large: {len} bytes (max {max})") + } } } } @@ -739,10 +887,119 @@ mod tests { use crate::test_utils::TestWalletContext; use crate::Network; use dashcore::blockdata::transaction::special_transaction::asset_lock::AssetLockPayload; + use dashcore::consensus::serialize; use dashcore::{OutPoint, Txid}; use dashcore_hashes::{sha256d, Hash}; use hex; + fn op_return_script(data: &[u8]) -> ScriptBuf { + let push_bytes = <&PushBytes>::try_from(data).expect("test OP_RETURN bytes"); + Builder::new().push_opcode(opcodes::all::OP_RETURN).push_slice(push_bytes).into_script() + } + + fn build_unsigned_legacy(mut builder: TransactionBuilder) -> (Transaction, u64) { + if let Some(TransactionPayload::AssetLockPayloadType(p)) = &builder.special_payload { + assert!(!p.credit_outputs.is_empty(), "legacy helper expects outputs"); + } else if builder.outputs.is_empty() && builder.special_payload.is_none() { + panic!("legacy helper expects outputs"); + } + + if builder.selection_strategy == SelectionStrategy::All { + builder.change_addr = None; + } + + let total_output: u64 = match &builder.special_payload { + Some(TransactionPayload::AssetLockPayloadType(p)) => { + p.credit_outputs.iter().map(|o| o.value).sum() + } + _ => builder.outputs.iter().map(|o| o.value).sum(), + }; + + if builder.require_final_inputs { + builder.inputs.retain(|utxo| utxo.is_confirmed || utxo.is_instantlocked); + } + + let selection = CoinSelector::new(builder.selection_strategy) + .select_coins_with_size( + builder.inputs.iter(), + total_output, + builder.fee_rate, + builder.current_height, + builder.calculate_base_size(), + // The selector's last argument is the change-output size. The legacy path + // budgeted one only when a change address was set — reproduce exactly that, + // so the comparison isolates the output-sizing change. + if builder.change_addr.is_some() { + CHANGE_OUTPUT_SIZE + } else { + 0 + }, + ) + .expect("legacy selection"); + + let mut selected_inputs = selection.selected; + let total_input: u64 = selected_inputs.iter().map(|u| u.value()).sum(); + let change_amount = + total_input.saturating_sub(total_output).saturating_sub(selection.estimated_fee); + + let mut tx_outputs = match &builder.special_payload { + Some(TransactionPayload::AssetLockPayloadType(_)) => vec![TxOut { + value: total_output, + script_pubkey: ScriptBuf::new_op_return(&[]), + }], + _ => builder.outputs, + }; + + if builder.selection_strategy == SelectionStrategy::All { + let [out] = tx_outputs.as_mut_slice() else { + panic!("legacy helper expects a single drain output"); + }; + out.value = total_input.saturating_sub(selection.estimated_fee); + } else if change_amount > 546 { + let change_addr = builder.change_addr.expect("legacy change address"); + tx_outputs.push(TxOut { + value: change_amount, + script_pubkey: change_addr.script_pubkey(), + }); + } + + if !matches!(builder.special_payload, Some(TransactionPayload::AssetLockPayloadType(_))) { + tx_outputs.sort_by(|a, b| match a.value.cmp(&b.value) { + Ordering::Equal => a.script_pubkey.as_bytes().cmp(b.script_pubkey.as_bytes()), + other => other, + }); + } + + selected_inputs.sort_by(|a, b| { + let tx_hash_a = a.outpoint.txid.to_byte_array(); + let tx_hash_b = b.outpoint.txid.to_byte_array(); + + match tx_hash_a.cmp(&tx_hash_b) { + Ordering::Equal => a.outpoint.vout.cmp(&b.outpoint.vout), + other => other, + } + }); + + let tx = Transaction { + version: 3, + lock_time: 0, + input: selected_inputs + .iter() + .map(|utxo| TxIn { + previous_output: utxo.outpoint, + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: dashcore::blockdata::witness::Witness::new(), + }) + .collect(), + output: tx_outputs, + special_transaction_payload: builder.special_payload, + }; + + let total_output: u64 = tx.output.iter().map(|out| out.value).sum(); + (tx, total_input.saturating_sub(total_output)) + } + #[test] fn test_transaction_builder_basic() { let utxo = Utxo::dummy(0, 100000, 100, false, true); @@ -1099,6 +1356,241 @@ mod tests { assert_eq!(tx.output[0].value, 100000); } + #[test] + fn test_maya_deposit_shape_preserves_output_order_and_routes_change_to_first_input() { + let vault = Address::dummy(Network::Testnet, 42); + let memo = vec![0x4d; MAX_STANDARD_OP_RETURN_BYTES]; + let utxos = vec![ + Utxo::dummy(0x02, 80_000, 100, false, true), + Utxo::dummy(0x01, 80_000, 100, false, true), + ]; + + let (tx, fee, _reservation) = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::SmallestFirst) + .set_fee_rate(FeeRate::normal()) + .add_inputs(utxos) + .add_output(&vault, 120_000) + .add_op_return(&memo) + .expect("valid OP_RETURN") + .preserve_output_order() + .change_to_first_input() + .build_unsigned_reserved() + .expect("maya-shaped transaction"); + + assert_eq!(tx.output.len(), 3, "vault + memo + change"); + assert_eq!(tx.output[0].script_pubkey, vault.script_pubkey()); + assert_eq!(tx.output[1].value, 0); + assert!(tx.output[1].script_pubkey.is_op_return()); + assert_eq!(tx.output[1].script_pubkey, op_return_script(&memo)); + assert_eq!( + &tx.output[1].script_pubkey.as_bytes() + [tx.output[1].script_pubkey.as_bytes().len() - memo.len()..], + memo.as_slice() + ); + assert_eq!(tx.output[2].script_pubkey, Address::dummy(Network::Testnet, 1).script_pubkey()); + + // `build_unsigned` leaves every script_sig empty, so the serialized bytes are ~107 + // short per input (41 unsigned vs the 148 a signed P2PKH input costs). Comparing the + // fee against that would pass no matter how badly the estimate under-counted — the + // very regression the precise output sizing exists to prevent. Measure against the + // signed size the builder itself assumes. + const SIGNED_INPUT_SIZE: usize = 148; + const UNSIGNED_INPUT_SIZE: usize = 41; + let signed_size = (serialize(&tx).len() + + tx.input.len() * (SIGNED_INPUT_SIZE - UNSIGNED_INPUT_SIZE)) + as u64; + assert!( + fee >= signed_size, + "fee {fee} must cover the signed size {signed_size} at the 1 duff/byte relay minimum" + ); + } + + #[test] + fn test_change_to_first_input_sizes_larger_routing_script() { + let routed_address = + Address::p2wsh(&Builder::new().push_int(1).into_script(), Network::Testnet); + let mut routed_input = Utxo::dummy(0x01, 100_000, 100, false, true); + routed_input.address = routed_address.clone(); + routed_input.txout.script_pubkey = routed_address.script_pubkey(); + + let builder = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::SmallestFirst) + .set_fee_rate(FeeRate::normal()) + .set_change_address(Address::dummy(Network::Testnet, 13)) + .add_inputs([routed_input]) + .add_output(&Address::dummy(Network::Testnet, 42), 50_000) + .change_to_first_input(); + + let routed_output_size = serialized_script_output_size(&routed_address.script_pubkey()); + assert!(routed_output_size > CHANGE_OUTPUT_SIZE); + assert_eq!(builder.estimated_change_output_size(), routed_output_size); + + let (tx, fee, _reservation) = + builder.build_unsigned_reserved().expect("witness-routed change transaction"); + assert_eq!(tx.output.len(), 2); + assert!(tx + .output + .iter() + .any(|output| output.script_pubkey == routed_address.script_pubkey())); + + const SIGNED_INPUT_SIZE: usize = 148; + const UNSIGNED_INPUT_SIZE: usize = 41; + let signed_size = (serialize(&tx).len() + + tx.input.len() * (SIGNED_INPUT_SIZE - UNSIGNED_INPUT_SIZE)) + as u64; + assert!( + fee >= signed_size, + "fee {fee} must cover the larger routed output at signed size {signed_size}" + ); + } + + #[test] + fn test_change_to_first_input_rejects_configured_network_mismatch() { + let mainnet_address = Address::dummy(Network::Mainnet, 1); + let mut input = Utxo::dummy(0x01, 100_000, 100, false, true); + input.address = mainnet_address.clone(); + input.txout.script_pubkey = mainnet_address.script_pubkey(); + + let result = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::SmallestFirst) + .set_change_address(Address::dummy(Network::Testnet, 13)) + .add_inputs([input]) + .add_output(&Address::dummy(Network::Testnet, 42), 50_000) + .change_to_first_input() + .build_unsigned_reserved(); + + assert!(matches!( + result, + Err(BuilderError::InvalidData(message)) + if message.contains("network does not match") + )); + } + + /// The precise per-output sizing added for OP_RETURN support must not move the fee + /// estimate for any transaction shape that existed before it: P2PKH outputs really are + /// 34 serialized bytes, and the asset-lock burn is deliberately still charged at 34. + #[test] + fn test_base_size_unchanged_for_pre_op_return_shapes() { + fn legacy_base_size(outputs_count: usize, has_change: bool, payload_size: usize) -> usize { + let mut size = 8 + 1; + size += varint_size(outputs_count + usize::from(has_change)); + size += outputs_count * 34; + if has_change { + size += 34; + } + if payload_size > 0 { + size += varint_size(payload_size) + payload_size; + } + size + } + + let plain = TransactionBuilder::new() + .set_change_address(Address::dummy(Network::Testnet, 13)) + .add_output(&Address::dummy(Network::Testnet, 11), 80_000) + .add_output(&Address::dummy(Network::Testnet, 12), 60_000); + assert_eq!(plain.calculate_base_size(), legacy_base_size(2, true, 0)); + + let no_change = + TransactionBuilder::new().add_output(&Address::dummy(Network::Testnet, 11), 80_000); + assert_eq!(no_change.calculate_base_size(), legacy_base_size(1, false, 0)); + + let asset_lock = TransactionBuilder::new() + .set_change_address(Address::dummy(Network::Testnet, 13)) + .set_special_payload(TransactionPayload::AssetLockPayloadType(AssetLockPayload { + version: 1, + credit_outputs: vec![TxOut { + value: 50_000, + script_pubkey: Address::dummy(Network::Testnet, 14).script_pubkey(), + }], + })); + let payload_size = 1 + varint_size(1) + 34; + assert_eq!( + asset_lock.calculate_base_size(), + legacy_base_size(1, true, payload_size), + "asset-lock sizing must stay byte-identical to the pre-OP_RETURN estimate" + ); + } + + /// The dust threshold has to follow the change script, not assume P2PKH. Pins both ends: + /// the P2PKH case must still be exactly the 546 the flat literal used to hard-code, and a + /// larger routed script must demand proportionally more before change is worth creating. + #[test] + fn test_dust_threshold_follows_the_change_output_size() { + assert_eq!( + TransactionBuilder::dust_threshold(CHANGE_OUTPUT_SIZE), + 546, + "P2PKH change must keep the historical 546-duff threshold" + ); + // A 43-byte output (e.g. P2WSH) costs the same 148-byte input to spend, so its threshold + // is 3 * (43 + 148). A 550-duff change output clears 546 but is dust at this size. + assert_eq!(TransactionBuilder::dust_threshold(43), 573); + assert!(550 > TransactionBuilder::dust_threshold(CHANGE_OUTPUT_SIZE)); + assert!(550 < TransactionBuilder::dust_threshold(43)); + } + + /// Pins the boundary itself, not just an under-sized fixture: exactly + /// [`MAX_STANDARD_OP_RETURN_BYTES`] must be accepted and one byte more refused, with the + /// error reporting both sides so a caller can render something useful. + #[test] + fn test_add_op_return_enforces_the_standard_ceiling() { + TransactionBuilder::new() + .add_op_return(&[0x4d; MAX_STANDARD_OP_RETURN_BYTES]) + .expect("a payload at exactly the ceiling is standard and must be accepted"); + + match TransactionBuilder::new().add_op_return(&[0u8; MAX_STANDARD_OP_RETURN_BYTES + 1]) { + Err(BuilderError::OpReturnDataTooLarge { + len, + max, + }) => { + assert_eq!(len, MAX_STANDARD_OP_RETURN_BYTES + 1); + assert_eq!(max, MAX_STANDARD_OP_RETURN_BYTES); + } + _ => panic!("expected oversized OP_RETURN error"), + } + } + + #[test] + fn test_default_ordinary_send_matches_legacy_bytes() { + let inputs = vec![ + Utxo::dummy(0x03, 120_000, 100, false, true), + Utxo::dummy(0x01, 130_000, 100, false, true), + ]; + let destination_a = Address::dummy(Network::Testnet, 11); + let destination_b = Address::dummy(Network::Testnet, 12); + let change = Address::dummy(Network::Testnet, 13); + + let builder = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::SmallestFirst) + .set_fee_rate(FeeRate::normal()) + .set_change_address(change) + .add_inputs(inputs) + .add_output(&destination_a, 80_000) + .add_output(&destination_b, 60_000); + + let (legacy_tx, legacy_fee) = build_unsigned_legacy(builder); + + let (tx, fee, _reservation) = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::SmallestFirst) + .set_fee_rate(FeeRate::normal()) + .set_change_address(Address::dummy(Network::Testnet, 13)) + .add_inputs([ + Utxo::dummy(0x03, 120_000, 100, false, true), + Utxo::dummy(0x01, 130_000, 100, false, true), + ]) + .add_output(&destination_a, 80_000) + .add_output(&destination_b, 60_000) + .build_unsigned_reserved() + .expect("ordinary send"); + + assert_eq!(serialize(&tx), serialize(&legacy_tx)); + assert_eq!(fee, legacy_fee); + } + #[test] fn test_bip69_input_ordering() { // Test that inputs are sorted according to BIP-69 From 2b1f7442667d38b67e081028a535be1be4f968be Mon Sep 17 00:00:00 2001 From: romchornyi Date: Fri, 7 Aug 2026 10:10:39 +0300 Subject: [PATCH 067/149] fix(key-wallet): stop counting a contact's watch-only coins as the wallet's (#926) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `all_funding_accounts` included `dashpay_external_accounts`. A DashPay external account is watch-only by construction — its addresses derive from a *contact's* xpub, so this wallet can observe those outputs but can never sign for them. They are the contact's coins, and this wallet only ever pays into them. Counting them as funding had two visible consequences. The wallet balance did not move after paying a contact: the amount left the standard account and reappeared under the external one, because the payment output landed there. Observed on device — after sending 0.01 DASH, `account_balances=[Standard{idx:0,BIP44}=>0.99587131 DASH, DashpayExternal{idx:0}=>0.01 DASH]`, with the external account climbing across successive payments (0.111 → 0.461 → 0.511). To a user this reads as "my balance is wrong", or worse, "the send did not happen". And `utxos()` fed `get_spendable_utxos()`, so coin selection could pick an output the wallet cannot sign for, failing at signing time with funds apparently available. Receival accounts stay: those addresses derive from our own xpub and a contact pays into them, so the funds are genuinely ours. This does not stop the wallet seeing those transactions. Detection runs through `transaction_checking`, whose `AccountTypeToCheck` list still includes `DashpayExternalAccount`, and the monitored-address / compact-filter set is built from `all_accounts`, which also still includes it. Only balance and UTXO aggregation change. Co-authored-by: Roman <51091564+jeanpierreroma@users.noreply.github.com> --- .../managed_account_collection.rs | 112 +++++++++++++++++- 1 file changed, 108 insertions(+), 4 deletions(-) diff --git a/key-wallet/src/managed_account/managed_account_collection.rs b/key-wallet/src/managed_account/managed_account_collection.rs index 4cf5f459f..803e1db45 100644 --- a/key-wallet/src/managed_account/managed_account_collection.rs +++ b/key-wallet/src/managed_account/managed_account_collection.rs @@ -917,31 +917,51 @@ impl ManagedAccountCollection { accounts } - /// Get all funds-bearing accounts (Standard BIP44/32, CoinJoin, DashPay). + /// Get the accounts whose funds belong to **this** wallet (Standard + /// BIP44/32, CoinJoin, DashPay receival). /// /// Use this from callsites that operate on balance / UTXO state — keys-only /// accounts (identity, asset-lock, provider) don't track those, so iterating /// [`Self::all_accounts`] and filtering via [`ManagedAccountRef::as_funds`] /// in those callsites is just noise. + /// + /// **`dashpay_external_accounts` are deliberately excluded.** A DashPay + /// external account is watch-only by construction: its addresses are derived + /// from a *contact's* xpub, so this wallet can observe those outputs but can + /// never sign for them. They are the contact's coins. Counting them here + /// made the wallet's own balance grow by whatever it had just paid out — + /// send 0.01 DASH to a contact and the total does not move, because the + /// amount leaves the standard account and reappears under the contact's — + /// and put unspendable outputs into `get_spendable_utxos`, where coin + /// selection can pick one and fail at signing time. + /// + /// Receival accounts stay: those addresses derive from *our* xpub and a + /// contact pays into them, so the funds are genuinely ours. + /// + /// Excluding them here does not stop the wallet seeing those transactions. + /// Detection runs through + /// [`transaction_checking`](crate::transaction_checking), whose + /// `AccountTypeToCheck` list still includes `DashpayExternalAccount`, and + /// the monitored-address / compact-filter set is built from + /// [`Self::all_accounts`], which also still includes it. pub fn all_funding_accounts(&self) -> Vec<&ManagedCoreFundsAccount> { let mut accounts = Vec::new(); accounts.extend(self.standard_bip44_accounts.values()); accounts.extend(self.standard_bip32_accounts.values()); accounts.extend(self.coinjoin_accounts.values()); accounts.extend(self.dashpay_receival_accounts.values()); - accounts.extend(self.dashpay_external_accounts.values()); accounts } /// Get all funds-bearing accounts mutably. See [`Self::all_funding_accounts`] - /// for which account types are visited. + /// for which account types are visited — in particular, why DashPay + /// *external* (watch-only) accounts are not among them. pub fn all_funding_accounts_mut(&mut self) -> Vec<&mut ManagedCoreFundsAccount> { let mut accounts = Vec::new(); accounts.extend(self.standard_bip44_accounts.values_mut()); accounts.extend(self.standard_bip32_accounts.values_mut()); accounts.extend(self.coinjoin_accounts.values_mut()); accounts.extend(self.dashpay_receival_accounts.values_mut()); - accounts.extend(self.dashpay_external_accounts.values_mut()); accounts } @@ -1035,3 +1055,87 @@ impl ManagedAccountCollection { self.platform_payment_accounts.get_mut(key) } } + +#[cfg(test)] +mod dashpay_funding_scope_tests { + use super::*; + use crate::account::{Account, AccountType, StandardAccountType}; + use crate::managed_account::ManagedCoreFundsAccount; + use crate::Network; + + fn xpub() -> crate::bip32::ExtendedPubKey { + let seed = [7u8; 32]; + let xpriv = + crate::bip32::ExtendedPrivKey::new_master(Network::Testnet, &seed).expect("master key"); + { + let secp = secp256k1::Secp256k1::new(); + crate::bip32::ExtendedPubKey::from_priv(&secp, &xpriv) + } + } + + fn managed(account_type: AccountType) -> ManagedCoreFundsAccount { + ManagedCoreFundsAccount::from_account(&Account { + parent_wallet_id: Some([1u8; 32]), + account_type, + network: Network::Testnet, + account_xpub: xpub(), + is_watch_only: matches!(account_type, AccountType::DashpayExternalAccount { .. }), + }) + } + + /// A DashPay **external** account is watch-only — its addresses derive from + /// the contact's xpub, so those coins are the contact's and this wallet can + /// never sign for them. Counting them as funding made the wallet's balance + /// absorb whatever it had just paid out, and put unspendable outputs in + /// front of coin selection. + /// + /// A DashPay **receival** account derives from our own xpub and must stay. + #[test] + fn external_dashpay_accounts_are_not_funding_accounts() { + let mut collection = ManagedAccountCollection::default(); + collection + .insert_funds_bearing_account(managed(AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account, + })) + .expect("standard"); + collection + .insert_funds_bearing_account(managed(AccountType::DashpayReceivingFunds { + index: 0, + user_identity_id: [0xAA; 32], + friend_identity_id: [0xBB; 32], + })) + .expect("receival"); + collection + .insert_funds_bearing_account(managed(AccountType::DashpayExternalAccount { + index: 0, + user_identity_id: [0xAA; 32], + friend_identity_id: [0xBB; 32], + })) + .expect("external"); + + let funding: Vec = collection + .all_funding_accounts() + .iter() + .map(|a| a.managed_account_type().to_account_type()) + .collect(); + + assert!( + funding.iter().any(|t| matches!(t, AccountType::DashpayReceivingFunds { .. })), + "receival accounts hold our own coins and must fund" + ); + assert!( + !funding.iter().any(|t| matches!(t, AccountType::DashpayExternalAccount { .. })), + "the contact's watch-only account must not count as this wallet's funds" + ); + assert_eq!( + funding.len(), + collection.all_funding_accounts_mut().len(), + "the mutable view must visit exactly the same accounts" + ); + + // The account still exists and is still watched — only its role in + // balance/UTXO aggregation changed. + assert_eq!(collection.dashpay_external_accounts.len(), 1); + } +} From 09904a68377d9a7954de32fc4101ac0a46c8703a Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Fri, 7 Aug 2026 10:29:49 +0200 Subject: [PATCH 068/149] feat(key-wallet): fund transactions from several account types (#925) * feat(key-wallet): multiple account funding sources * Update key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --------- Co-authored-by: QuantumExplorer Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- .../tests/dashd_sync/tests_transaction.rs | 4 +- key-wallet-ffi/src/transaction.rs | 2 +- key-wallet-manager/src/lib.rs | 4 +- key-wallet/src/managed_account/reservation.rs | 117 ++++--- key-wallet/src/tests/spent_outpoints_tests.rs | 7 +- .../managed_wallet_info/asset_lock_builder.rs | 4 +- .../transaction_builder.rs | 83 +++-- .../transaction_building.rs | 329 +++++++++++++----- 8 files changed, 378 insertions(+), 172 deletions(-) diff --git a/dash-spv/tests/dashd_sync/tests_transaction.rs b/dash-spv/tests/dashd_sync/tests_transaction.rs index 0dfb5d7aa..f0f005225 100644 --- a/dash-spv/tests/dashd_sync/tests_transaction.rs +++ b/dash-spv/tests/dashd_sync/tests_transaction.rs @@ -370,7 +370,7 @@ async fn build_and_sign( TransactionBuilder::new() .set_current_height(height) - .set_funding(funds_account, &account) + .add_funding(funds_account, &account) .add_output(&dest, amount) .build_signed(w, |a| funds_account.address_derivation_path(&a)) .await @@ -605,7 +605,7 @@ async fn test_drain_account_into_another() { let (w, info) = lock.get_wallet_and_info_mut(&wallet_id).expect("wallet"); info.build_and_sign_transaction( w, - AccountTypePreference::BIP32, + &[AccountTypePreference::BIP32], 0, vec![(dest.as_unchecked().clone(), 0)], FeeRate::normal(), diff --git a/key-wallet-ffi/src/transaction.rs b/key-wallet-ffi/src/transaction.rs index 700d2c192..eb09a0201 100644 --- a/key-wallet-ffi/src/transaction.rs +++ b/key-wallet-ffi/src/transaction.rs @@ -137,7 +137,7 @@ pub unsafe extern "C" fn wallet_build_and_sign_transaction( manager .build_and_sign_transaction( &wallet_id, - AccountTypePreference::BIP44, + &[AccountTypePreference::BIP44], account_index, outputs, FeeRate::new(fee_per_kb), diff --git a/key-wallet-manager/src/lib.rs b/key-wallet-manager/src/lib.rs index 3ef0a9398..89002d0ac 100644 --- a/key-wallet-manager/src/lib.rs +++ b/key-wallet-manager/src/lib.rs @@ -693,7 +693,7 @@ impl WalletManager { pub async fn build_and_sign_transaction( &mut self, wallet_id: &WalletId, - source: AccountTypePreference, + sources: &[AccountTypePreference], source_index: u32, outputs: Vec<(Address, u64)>, fee_rate: FeeRate, @@ -705,7 +705,7 @@ impl WalletManager { .ok_or(WalletError::WalletNotFound(*wallet_id))?; managed_wallet - .build_and_sign_transaction(wallet, source, source_index, outputs, fee_rate, strategy) + .build_and_sign_transaction(wallet, sources, source_index, outputs, fee_rate, strategy) .await .map_err(|e| WalletError::TransactionBuild(e.to_string())) } diff --git a/key-wallet/src/managed_account/reservation.rs b/key-wallet/src/managed_account/reservation.rs index 1715c5fd4..d24eba69b 100644 --- a/key-wallet/src/managed_account/reservation.rs +++ b/key-wallet/src/managed_account/reservation.rs @@ -24,6 +24,7 @@ //! why the check must be atomic under this set's mutex (`dashpay/platform#4185`). use std::collections::{HashMap, HashSet}; +use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::{Arc, Mutex, MutexGuard}; use dashcore::blockdata::transaction::OutPoint; @@ -41,20 +42,21 @@ use dashcore::blockdata::transaction::OutPoint; /// the very double-spend this guards against. const RESERVATION_TTL_BLOCKS: u32 = 24; -/// Opaque, per-`reserve`-call identity stamped onto every outpoint that call -/// reserves. +/// Opaque, per-build identity stamped onto every outpoint that build reserves. /// /// A token is the *proof of ownership* a build presents to the reservation /// set's `release_if_owner` so a release only removes the build's own -/// reservation. Each `reserve` call mints a fresh token from a monotonic -/// counter, so two reservations never share one — not even two reservations -/// taken at the same block height, which is why the height (which collides -/// freely) cannot serve as the identity. +/// reservation. Tokens are minted from a process-wide counter, so two builds +/// never share one — not even two reservations taken at the same block height, +/// which is why the height (which collides freely) cannot serve as the +/// identity, and not even across two accounts' sets, which is what lets one +/// build funded from several accounts reserve in each of them under a single +/// token. /// -/// The inner counter is private and there is no public constructor: a token can -/// only originate from a real `reserve` call. That is deliberate — it prevents a -/// caller from forging a token that happens to match another build's ownership -/// and releasing inputs out from under it. +/// The inner counter is private and minting is crate-private: a token can only +/// originate from this crate. That is deliberate — it prevents a caller from +/// forging a token that happens to match another build's ownership and +/// releasing inputs out from under it. /// /// Copy semantics let a build hold its token cheaply across an `.await` (e.g. /// the platform broadcast path in `dashpay/platform#4185`) and present it again @@ -62,6 +64,17 @@ const RESERVATION_TTL_BLOCKS: u32 = 24; #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] pub struct ReservationToken(u64); +impl ReservationToken { + /// Mint a token, unique within this process. + /// + /// (Wraparound would need ~2^64 mints in one process lifetime, which is + /// unreachable in practice.) + pub(crate) fn next() -> Self { + static NEXT_TOKEN: AtomicU64 = AtomicU64::new(0); + Self(NEXT_TOKEN.fetch_add(1, Ordering::Relaxed)) + } +} + /// A single reserved outpoint: when it was reserved (for the TTL backstop) and /// which build owns it (for owner-guarded release). #[derive(Debug, Clone, Copy)] @@ -73,16 +86,10 @@ struct Reservation { owner: ReservationToken, } -/// Mutex-guarded interior: the reservations keyed by outpoint plus the counter -/// that mints the next [`ReservationToken`]. +/// Mutex-guarded interior: the reservations keyed by outpoint. #[derive(Debug, Default)] struct Reserved { entries: HashMap, - /// Monotonically increasing source of unique tokens. Never persisted and - /// only ever incremented, so within a process every issued token is unique. - /// (Wraparound would need ~2^64 reserves in one process lifetime, which is - /// unreachable in practice.) - next_token: u64, } /// Ephemeral, in-memory set of reserved outpoints. Cloning shares the @@ -95,7 +102,7 @@ pub(crate) struct ReservationSet { impl ReservationSet { /// Recovers from a poisoned mutex rather than panicking: the guarded data is - /// a plain map plus a counter with no invariant a partial write could break, + /// a plain map with no invariant a partial write could break, /// and panicking here would strand all later coin selection in a /// long-running node. fn lock(&self) -> MutexGuard<'_, Reserved> { @@ -115,25 +122,26 @@ impl ReservationSet { }); } - /// Reserve `outpoints` as of `current_height`, dropping expired entries - /// first, and return the [`ReservationToken`] stamped onto all of them. + /// Reserve `outpoints` as of `current_height` on behalf of `owner`, + /// dropping expired entries first. /// - /// Every outpoint in a single call shares the one returned token: the caller - /// keeps it and later presents it to [`Self::release_if_owner`] to release - /// only what this call reserved. Re-reserving an outpoint (a later `reserve` - /// naming it again) refreshes its height *and* transfers ownership to the new - /// token — the previous owner's [`Self::release_if_owner`] then becomes a - /// no-op for it, which is precisely the behavior that closes the + /// The owner is the token the reserving build was minted, and it is what the + /// build later presents to [`Self::release_if_owner`] to release only what it + /// reserved — here and, for a build funded from several accounts, in every + /// other account's set it reserved in. Re-reserving an outpoint (a later + /// `reserve` naming it again) refreshes its height *and* transfers ownership + /// to the new owner — the previous owner's [`Self::release_if_owner`] then + /// becomes a no-op for it, which is precisely the behavior that closes the /// release/re-reserve race described in the module docs (`platform#4185`). - pub(crate) fn reserve(&self, outpoints: &[OutPoint], current_height: u32) -> ReservationToken { + pub(crate) fn reserve( + &self, + outpoints: &[OutPoint], + current_height: u32, + owner: ReservationToken, + ) { let mut reserved = self.lock(); Self::sweep(&mut reserved, current_height); - let owner = ReservationToken(reserved.next_token); - // Increment even on an empty `outpoints` slice so a token is never - // reissued; wrapping is documented-unreachable but avoids a debug panic. - reserved.next_token = reserved.next_token.wrapping_add(1); - for outpoint in outpoints { reserved.entries.insert( *outpoint, @@ -143,7 +151,6 @@ impl ReservationSet { }, ); } - owner } /// Return the currently reserved outpoints, dropping expired entries first. @@ -224,7 +231,7 @@ mod tests { let a = outpoint(0x01, 0); let b = outpoint(0x02, 1); - set.reserve(&[a], 100); + set.reserve(&[a], 100, ReservationToken::next()); assert!(set.reserved(100).contains(&a)); assert!(!set.reserved(100).contains(&b)); assert_eq!(set.reserved(100), HashSet::from([a])); @@ -239,7 +246,7 @@ mod tests { let a = outpoint(0x03, 0); // Releasing an unreserved outpoint is a no-op. set.release([&a]); - set.reserve(&[a], 10); + set.reserve(&[a], 10, ReservationToken::next()); set.release([&a]); set.release([&a]); assert!(!set.reserved(10).contains(&a)); @@ -249,7 +256,7 @@ mod tests { fn ttl_reclaims_stale_reservation_by_height() { let set = ReservationSet::default(); let a = outpoint(0x04, 0); - set.reserve(&[a], 100); + set.reserve(&[a], 100, ReservationToken::next()); // The boundary is exclusive, so the entry survives until exactly // `reserved_at + RESERVATION_TTL_BLOCKS`. @@ -262,14 +269,14 @@ mod tests { fn zero_height_disables_sweep() { let set = ReservationSet::default(); let a = outpoint(0x07, 0); - set.reserve(&[a], 0); + set.reserve(&[a], 0, ReservationToken::next()); // Height 0 means the wallet has no processed height yet, so the elapsed // span is unknown and the sweep is suppressed: the reservation stands. assert!(set.reserved(0).contains(&a)); // Once a real height is known the TTL backstop applies again. - set.reserve(&[a], 1); + set.reserve(&[a], 1, ReservationToken::next()); assert!(set.reserved(1 + RESERVATION_TTL_BLOCKS).is_empty()); } @@ -277,10 +284,10 @@ mod tests { fn re_reserving_refreshes_the_ttl_height() { let set = ReservationSet::default(); let a = outpoint(0x06, 0); - set.reserve(&[a], 100); + set.reserve(&[a], 100, ReservationToken::next()); // A later reserve replaces the stored height, so the TTL is measured // from the most recent reservation, not the first. - set.reserve(&[a], 150); + set.reserve(&[a], 150, ReservationToken::next()); assert_eq!(set.reserved(150 + RESERVATION_TTL_BLOCKS - 1), HashSet::from([a])); assert!(set.reserved(150 + RESERVATION_TTL_BLOCKS).is_empty()); @@ -293,18 +300,16 @@ mod tests { let a = outpoint(0x05, 0); // A reservation taken on one handle is visible through the other, which // is what lets a build's reservation outlive the wallet lock. - set.reserve(&[a], 1); + set.reserve(&[a], 1, ReservationToken::next()); assert!(clone.reserved(1).contains(&a)); } #[test] - fn each_reserve_call_mints_a_distinct_token() { - let set = ReservationSet::default(); - // Two reserves at the SAME height must still get different tokens — the - // whole point of not keying ownership on height, which collides. - let token_a = set.reserve(&[outpoint(0x10, 0)], 100); - let token_b = set.reserve(&[outpoint(0x11, 0)], 100); - assert_ne!(token_a, token_b); + fn each_build_mints_a_distinct_token() { + // Two builds at the SAME height must still get different tokens — the + // whole point of not keying ownership on height, which collides. The + // counter is process-wide, so this holds across accounts' sets too. + assert_ne!(ReservationToken::next(), ReservationToken::next()); } #[test] @@ -313,8 +318,9 @@ mod tests { let mine = outpoint(0x20, 0); let theirs = outpoint(0x21, 0); - let my_token = set.reserve(&[mine], 100); - let _their_token = set.reserve(&[theirs], 100); + let my_token = ReservationToken::next(); + set.reserve(&[mine], 100, my_token); + set.reserve(&[theirs], 100, ReservationToken::next()); // Releasing with my token frees only my outpoint; theirs is untouched. set.release_if_owner(&[mine, theirs], my_token); @@ -326,11 +332,12 @@ mod tests { fn release_if_owner_is_a_noop_for_unreserved_or_wrong_token() { let set = ReservationSet::default(); let a = outpoint(0x22, 0); - let stale_token = set.reserve(&[a], 100); + let stale_token = ReservationToken::next(); + set.reserve(&[a], 100, stale_token); // Simulate the outpoint being released and re-reserved by someone else. set.release([&a]); - let _new_token = set.reserve(&[a], 100); + set.reserve(&[a], 100, ReservationToken::next()); // The stale token no longer owns `a`, so its release must not remove it. set.release_if_owner(&[a], stale_token); @@ -354,14 +361,16 @@ mod tests { let x = outpoint(0x30, 0); // Build A reserves X. - let token_a = set.reserve(&[x], 100); + let token_a = ReservationToken::next(); + set.reserve(&[x], 100, token_a); assert!(set.reserved(100).contains(&x)); // TTL sweep reclaims A's reservation mid-await (modeled by advancing the // height past the TTL so the next reserve's sweep drops A's entry)... let swept_height = 100 + RESERVATION_TTL_BLOCKS; // ...and build B re-reserves the very same outpoint under a new token. - let token_b = set.reserve(&[x], swept_height); + let token_b = ReservationToken::next(); + set.reserve(&[x], swept_height, token_b); assert_ne!(token_a, token_b); assert!(set.reserved(swept_height).contains(&x)); diff --git a/key-wallet/src/tests/spent_outpoints_tests.rs b/key-wallet/src/tests/spent_outpoints_tests.rs index 6e80f18fd..88eadc42f 100644 --- a/key-wallet/src/tests/spent_outpoints_tests.rs +++ b/key-wallet/src/tests/spent_outpoints_tests.rs @@ -6,6 +6,7 @@ use dashcore::{BlockHash, TxIn, Txid}; use crate::account::{AccountType, StandardAccountType, TransactionRecord}; use crate::managed_account::managed_account_trait::ManagedAccountTrait; +use crate::managed_account::reservation::ReservationToken; use crate::managed_account::transaction_record::TransactionDirection; use crate::managed_account::ManagedCoreFundsAccount; use crate::test_utils::TestWalletContext; @@ -77,7 +78,7 @@ fn reservations_are_not_persisted() { let account = ManagedCoreFundsAccount::dummy_bip44(); let outpoint = OutPoint::new(Txid::from([0x42; 32]), 0); - account.reservations().reserve(&[outpoint], 0); + account.reservations().reserve(&[outpoint], 0, ReservationToken::next()); assert!(account.reservations().reserved(0).contains(&outpoint)); let json = serde_json::to_string(&account).unwrap(); @@ -96,7 +97,7 @@ async fn processing_a_spend_releases_its_reservation() { let account = ctx.managed_wallet.first_bip44_managed_account_mut().expect("BIP44 account"); assert!(account.utxos.contains_key(&funded)); - account.reservations().reserve(&[funded], 0); + account.reservations().reserve(&[funded], 0, ReservationToken::next()); assert!(account.reservations().reserved(0).contains(&funded)); let spend = spending_tx(&[funded]); @@ -114,7 +115,7 @@ async fn processing_a_spend_releases_its_reservation() { let account = ctx.managed_wallet.first_bip44_managed_account_mut().expect("BIP44 account"); assert!(account.utxos.contains_key(&second_funded)); - account.reservations().reserve(&[second_funded], 0); + account.reservations().reserve(&[second_funded], 0, ReservationToken::next()); assert!(account.reservations().reserved(0).contains(&second_funded)); let block_hash = BlockHash::from_slice(&[7u8; 32]).expect("hash"); diff --git a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs index 26595961d..6aca94b34 100644 --- a/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/asset_lock_builder.rs @@ -319,7 +319,7 @@ impl ManagedWalletInfo { builder = builder.set_selection_strategy(SelectionStrategy::All); } let (transaction, fee, reservation_token) = builder - .set_funding(funds_acc, acc) + .add_funding(funds_acc, acc) .require_final_inputs() .build_signed_reserved(wallet, |addr| funds_acc.address_derivation_path(&addr)) .await?; @@ -446,7 +446,7 @@ impl ManagedWalletInfo { builder = builder.set_selection_strategy(SelectionStrategy::All); } let (transaction, fee, reservation_token) = builder - .set_funding(funds_acc, &acc) + .add_funding(funds_acc, &acc) .require_final_inputs() .build_signed_reserved(signer, |addr| funds_acc.address_derivation_path(&addr)) .await?; diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 7a86d7f17..7c42d2871 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -22,6 +22,7 @@ use dashcore_hashes::Hash; use secp256k1::ecdsa::Signature; use secp256k1::{Message, PublicKey, Secp256k1}; use std::cmp::Ordering; +use std::collections::HashSet; /// A transaction with more inputs would exceed the relay standard-size cap (~100 KB at ~148 /// bytes/signed input) and be rejected by the network @@ -79,10 +80,11 @@ pub struct TransactionBuilder { change_to_first_input: bool, /// Special transaction payload for Dash-specific transactions special_payload: Option, - /// Reservation set of the funding account, captured by `set_funding`. The - /// inputs chosen during assembly are reserved here so a concurrent build - /// skips them until the broadcast transaction is processed. - reservations: Option, + /// Reservation set of each funding account paired with the outpoints that + /// account contributed, captured by `add_funding`. Reservations live on the + /// account that holds the UTXO, so each account reserves its own share of + /// the chosen inputs — all under the one token this build is stamped with. + funding: Vec<(ReservationSet, HashSet)>, } impl Default for TransactionBuilder { @@ -105,7 +107,7 @@ impl TransactionBuilder { preserve_output_order: false, change_to_first_input: false, special_payload: None, - reservations: None, + funding: Vec::new(), } } @@ -130,26 +132,36 @@ impl TransactionBuilder { self } - /// Seed the builder with the funding account's spendable UTXOs, skipping any - /// already reserved by another in-flight build. + /// Add a funding account's spendable UTXOs to the candidate input set, + /// skipping any already reserved by another in-flight build. + /// + /// Call it once per funding account: coin selection then draws from the + /// union of their UTXOs, and the first account supplies the change + /// address. /// /// This call and the later `assemble_unsigned` that reserves the chosen /// inputs must run under one uninterrupted hold of the wallet lock. If two /// builds interleave between here and their reservation, both can observe the /// same UTXO as free and select it, defeating the reservation. The builder - /// must therefore not be held across an `await` between `set_funding` and + /// must therefore not be held across an `await` between `add_funding` and /// `build_signed` or `assemble_unsigned`, since suspending there reopens the /// read-then-reserve window for a concurrent build. - pub fn set_funding(mut self, funds_acc: &mut ManagedCoreFundsAccount, acc: &Account) -> Self { + pub fn add_funding(mut self, funds_acc: &mut ManagedCoreFundsAccount, acc: &Account) -> Self { let reserved = funds_acc.reservations().reserved(self.current_height); - self.inputs = funds_acc + let candidates: Vec = funds_acc .utxos .values() .filter(|utxo| !reserved.contains(&utxo.outpoint)) .cloned() .collect(); - self.reservations = Some(funds_acc.reservations().clone()); - self.change_addr = funds_acc.next_change_address(Some(&acc.account_xpub), true).ok(); + self.funding.push(( + funds_acc.reservations().clone(), + candidates.iter().map(|utxo| utxo.outpoint).collect(), + )); + self.inputs.extend(candidates); + if self.change_addr.is_none() { + self.change_addr = funds_acc.next_change_address(Some(&acc.account_xpub), true).ok(); + } self } @@ -403,13 +415,13 @@ impl TransactionBuilder { /// Select inputs, build the unsigned transaction, and reserve the chosen /// inputs. The optional [`ReservationToken`] is `Some` exactly when a - /// reservation set was attached (via [`set_funding`]) and identifies the + /// reservation set was attached (via [`add_funding`]) and identifies the /// reservation this build just took, so a caller that later abandons the /// build can release *only* its own inputs via /// [`ReservationSet::release_if_owner`]. It is `None` for builds with no /// reservation set (nothing was reserved, nothing to release). /// - /// [`set_funding`]: Self::set_funding + /// [`add_funding`]: Self::add_funding fn assemble_unsigned( mut self, ) -> Result<(Transaction, Vec, Option), BuilderError> { @@ -572,13 +584,22 @@ impl TransactionBuilder { // Reserve the chosen inputs so a concurrent build skips them until the // broadcast transaction is processed back into the wallet (which - // releases the reservation) or the TTL backstop reclaims it. Keep the - // stamped owner token so the caller can release only this reservation - // if the build is later abandoned (see `release_if_owner`). - let reservation_token = self.reservations.as_ref().map(|reservations| { - let outpoints: Vec = - selected_inputs.iter().map(|utxo| utxo.outpoint).collect(); - reservations.reserve(&outpoints, self.current_height) + // releases the reservation) or the TTL backstop reclaims it. Each + // account reserves the inputs it contributed, since its own set is the + // one its coin selection consults. Keep the stamped owner token so the + // caller can release only this reservation if the build is later + // abandoned (see `release_if_owner`). + let reservation_token = (!self.funding.is_empty()).then(|| { + let owner = ReservationToken::next(); + for (reservations, candidates) in &self.funding { + let outpoints: Vec = selected_inputs + .iter() + .map(|utxo| utxo.outpoint) + .filter(|outpoint| candidates.contains(outpoint)) + .collect(); + reservations.reserve(&outpoints, self.current_height, owner); + } + owner }); return Ok((transaction, selected_inputs, reservation_token)); @@ -617,7 +638,9 @@ impl TransactionBuilder { /// `.await` that releases the wallet lock — most importantly the platform /// broadcast path, which reserves inputs, awaits the broadcast, and on /// rejection must release them — and release with - /// [`ManagedCoreFundsAccount::release_reservation_if_owner`]. See + /// [`ManagedCoreFundsAccount::release_reservation_if_owner`] on *every* + /// funding account passed to [`Self::add_funding`]: each account reserves + /// only the inputs it contributed, in its own set. See /// `ReservationSet::release_if_owner` for why owner-guarded release is /// required (`dashpay/platform#4185`). pub fn build_unsigned_reserved( @@ -664,7 +687,7 @@ impl TransactionBuilder { S: TransactionSigner + ?Sized + Sync, P: Fn(Address) -> Option + Send, { - let reservations = self.reservations.clone(); + let funding = self.funding.clone(); let (tx, inputs, reservation) = self.assemble_unsigned()?; let total_input: u64 = inputs.iter().map(|utxo| utxo.value()).sum(); @@ -683,8 +706,10 @@ impl TransactionBuilder { let tx = match signer.sign_tx(tx, inputs, path_resolver).await { Ok(tx) => tx, Err(err) => { - if let (Some(reservations), Some(token)) = (&reservations, reservation) { - reservations.release_if_owner(&reserved, token); + if let Some(token) = reservation { + for (reservations, _) in &funding { + reservations.release_if_owner(&reserved, token); + } } return Err(err); } @@ -1740,10 +1765,10 @@ mod tests { funds.utxos.insert(reserved.outpoint, reserved.clone()); funds.utxos.insert(free.outpoint, free.clone()); - funds.reservations().reserve(&[reserved.outpoint], 200); + funds.reservations().reserve(&[reserved.outpoint], 200, ReservationToken::next()); let builder = - TransactionBuilder::new().set_current_height(200).set_funding(&mut funds, &account); + TransactionBuilder::new().set_current_height(200).add_funding(&mut funds, &account); let candidates: Vec = builder.inputs.iter().map(|utxo| utxo.outpoint).collect(); assert!(!candidates.contains(&reserved.outpoint)); @@ -1764,7 +1789,7 @@ mod tests { let builder = TransactionBuilder::new() .set_current_height(200) .set_fee_rate(FeeRate::normal()) - .set_funding(&mut funds, &account) + .add_funding(&mut funds, &account) .set_change_address(Address::dummy(Network::Testnet, 1)) .add_output(&destination, 500_000); @@ -1792,7 +1817,7 @@ mod tests { let (tx, _, _) = TransactionBuilder::new() .set_current_height(200) .set_fee_rate(FeeRate::normal()) - .set_funding(&mut funds, &account) + .add_funding(&mut funds, &account) .set_change_address(Address::dummy(Network::Testnet, 1)) .add_output(&destination, 500_000) .build_unsigned_reserved() diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index 425d36ac7..63d4f0259 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -4,122 +4,209 @@ use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::signer::Signer; use crate::wallet::managed_wallet_info::coin_selection::SelectionStrategy; use crate::wallet::managed_wallet_info::fee::FeeRate; -use crate::wallet::managed_wallet_info::transaction_builder::{BuilderError, TransactionBuilder}; +use crate::wallet::managed_wallet_info::transaction_builder::{ + BuilderError, TransactionBuilder, TransactionSigner, +}; use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; use crate::wallet::ManagedWalletInfo; -use crate::Wallet; +use crate::{DerivationPath, Wallet}; use dashcore::address::NetworkUnchecked; use dashcore::{Address, Transaction}; +use std::collections::HashMap; /// Account type preference for transaction building -#[derive(Debug, Clone, Copy)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum AccountTypePreference { BIP44, BIP32, CoinJoin, } +impl AccountTypePreference { + /// The account types an empty source list draws from, in order. + /// + /// CoinJoin is deliberately absent: spending mixed outputs alongside + /// transparent ones in the same transaction links them and undoes the + /// mixing + pub const DEFAULT: [Self; 2] = [Self::BIP44, Self::BIP32]; +} + impl ManagedWalletInfo { + /// Build and sign a transaction funded from the given account types at + /// `source_index`, signing with the wallet's own keys. + /// + /// Coin selection draws from the union of those accounts' UTXOs, and the + /// first of them supplies the change address. An empty `sources` means + /// [`AccountTypePreference::DEFAULT`], skipping the types absent at the + /// index; a non-empty one is taken literally and every account named must + /// exist. pub async fn build_and_sign_transaction( &mut self, wallet: &Wallet, - source: AccountTypePreference, + sources: &[AccountTypePreference], source_index: u32, outputs: Vec<(Address, u64)>, fee_rate: FeeRate, strategy: SelectionStrategy, ) -> Result<(Transaction, u64), BuilderError> { - let height = self.last_processed_height(); - - let managed_account = match source { - AccountTypePreference::BIP44 => { - self.accounts.standard_bip44_accounts.get_mut(&source_index) - } - AccountTypePreference::BIP32 => { - self.accounts.standard_bip32_accounts.get_mut(&source_index) - } - AccountTypePreference::CoinJoin => { - self.accounts.coinjoin_accounts.get_mut(&source_index) - } - } - .ok_or_else(|| { - BuilderError::AccountNotFound(format!("managed account {source:?} #{source_index}")) - })?; - let account = match source { - AccountTypePreference::BIP44 => wallet.get_bip44_account(source_index), - AccountTypePreference::BIP32 => wallet.get_bip32_account(source_index), - AccountTypePreference::CoinJoin => wallet.get_coinjoin_account(source_index), - } - .ok_or_else(|| { - BuilderError::AccountNotFound(format!("wallet account {source:?} #{source_index}")) - })?; - - let mut tx_builder = TransactionBuilder::new() - .set_fee_rate(fee_rate) - .set_selection_strategy(strategy) - .set_current_height(height) - .set_funding(managed_account, account); + self.build_and_sign(wallet, sources, source_index, outputs, fee_rate, strategy, wallet) + .await + } - for (address, value) in outputs { - let checked = address.require_network(wallet.network).map_err(|e| { - BuilderError::InvalidData(format!("Output address network mismatch: {}", e)) - })?; - tx_builder = tx_builder.add_output(&checked, value); - } + /// [`Self::build_and_sign_transaction`] with signing delegated to an + /// external [`Signer`], so the private keys never have to be held by this + /// process. + #[allow(clippy::too_many_arguments)] + pub async fn build_and_sign_transaction_with_signer( + &mut self, + wallet: &Wallet, + sources: &[AccountTypePreference], + source_index: u32, + outputs: Vec<(Address, u64)>, + fee_rate: FeeRate, + strategy: SelectionStrategy, + signer: &S, + ) -> Result<(Transaction, u64), BuilderError> { + self.build_and_sign(wallet, sources, source_index, outputs, fee_rate, strategy, signer) + .await + } - tx_builder.build_signed(wallet, |addr| managed_account.address_derivation_path(&addr)).await + /// [`Self::build_and_sign_transaction`] without the signing step, for + /// callers that sign the transaction themselves. Its inputs are reserved + /// just the same, so a build that is never broadcast holds them until the + /// reservation's TTL reclaims them. + pub fn build_unsigned_transaction( + &mut self, + wallet: &Wallet, + sources: &[AccountTypePreference], + source_index: u32, + outputs: Vec<(Address, u64)>, + fee_rate: FeeRate, + strategy: SelectionStrategy, + ) -> Result<(Transaction, u64), BuilderError> { + let (builder, _paths) = + self.funded_builder(wallet, sources, source_index, outputs, fee_rate, strategy)?; + let (transaction, fee, _reservation) = builder.build_unsigned_reserved()?; + Ok((transaction, fee)) } #[allow(clippy::too_many_arguments)] - pub async fn build_and_sign_transaction_with_signer( + async fn build_and_sign( &mut self, wallet: &Wallet, - source: AccountTypePreference, + sources: &[AccountTypePreference], source_index: u32, outputs: Vec<(Address, u64)>, fee_rate: FeeRate, strategy: SelectionStrategy, signer: &S, ) -> Result<(Transaction, u64), BuilderError> { - let height = self.last_processed_height(); + let (builder, paths) = + self.funded_builder(wallet, sources, source_index, outputs, fee_rate, strategy)?; + builder.build_signed(signer, move |addr| paths.get(&addr).cloned()).await + } - let managed_account = match source { - AccountTypePreference::BIP44 => { - self.accounts.standard_bip44_accounts.get_mut(&source_index) - } - AccountTypePreference::BIP32 => { - self.accounts.standard_bip32_accounts.get_mut(&source_index) - } - AccountTypePreference::CoinJoin => { - self.accounts.coinjoin_accounts.get_mut(&source_index) - } - } - .ok_or_else(|| { - BuilderError::AccountNotFound(format!("managed account {source:?} #{source_index}")) - })?; - let account = match source { - AccountTypePreference::BIP44 => wallet.get_bip44_account(source_index), - AccountTypePreference::BIP32 => wallet.get_bip32_account(source_index), - AccountTypePreference::CoinJoin => wallet.get_coinjoin_account(source_index), - } - .ok_or_else(|| { - BuilderError::AccountNotFound(format!("wallet account {source:?} #{source_index}")) - })?; + /// A builder carrying the outputs and the funding of `sources`, ready to be + /// built with or without signing. + fn funded_builder( + &mut self, + wallet: &Wallet, + sources: &[AccountTypePreference], + source_index: u32, + outputs: Vec<(Address, u64)>, + fee_rate: FeeRate, + strategy: SelectionStrategy, + ) -> Result<(TransactionBuilder, HashMap), BuilderError> { + let outputs = outputs + .into_iter() + .map(|(address, value)| { + address.require_network(wallet.network).map(|checked| (checked, value)).map_err( + |e| { + BuilderError::InvalidData(format!("Output address network mismatch: {}", e)) + }, + ) + }) + .collect::, _>>()?; - let mut tx_builder = TransactionBuilder::new() + let height = self.last_processed_height(); + let builder = TransactionBuilder::new() .set_fee_rate(fee_rate) .set_selection_strategy(strategy) - .set_current_height(height) - .set_funding(managed_account, account); + .set_current_height(height); + + let (mut builder, paths) = self.fund(wallet, sources, source_index, builder)?; for (address, value) in outputs { - let checked = address.require_network(wallet.network).map_err(|e| { - BuilderError::InvalidData(format!("Output address network mismatch: {}", e)) - })?; - tx_builder = tx_builder.add_output(&checked, value); + builder = builder.add_output(&address, value); } - tx_builder.build_signed(signer, |addr| managed_account.address_derivation_path(&addr)).await + Ok((builder, paths)) + } + + /// Seed `builder` with the UTXOs of every funding account named by + /// `sources` at `source_index`, returning it alongside the derivation path + /// of each candidate input address, since the inputs can come from + /// different accounts. + fn fund( + &mut self, + wallet: &Wallet, + sources: &[AccountTypePreference], + source_index: u32, + mut builder: TransactionBuilder, + ) -> Result<(TransactionBuilder, HashMap), BuilderError> { + let named_explicitly = !sources.is_empty(); + let preferences = if named_explicitly { + sources + } else { + &AccountTypePreference::DEFAULT + }; + + let mut paths = HashMap::new(); + let mut funded = 0usize; + + for &preference in preferences { + let account = match preference { + AccountTypePreference::BIP44 => wallet.get_bip44_account(source_index), + AccountTypePreference::BIP32 => wallet.get_bip32_account(source_index), + AccountTypePreference::CoinJoin => wallet.get_coinjoin_account(source_index), + }; + let managed_account = match preference { + AccountTypePreference::BIP44 => { + self.accounts.standard_bip44_accounts.get_mut(&source_index) + } + AccountTypePreference::BIP32 => { + self.accounts.standard_bip32_accounts.get_mut(&source_index) + } + AccountTypePreference::CoinJoin => { + self.accounts.coinjoin_accounts.get_mut(&source_index) + } + }; + + let (Some(account), Some(managed_account)) = (account, managed_account) else { + if named_explicitly { + return Err(BuilderError::AccountNotFound(format!( + "account {preference:?} #{source_index}" + ))); + } + continue; + }; + + for utxo in managed_account.utxos.values() { + if let Some(path) = managed_account.address_derivation_path(&utxo.address) { + paths.insert(utxo.address.clone(), path); + } + } + builder = builder.add_funding(managed_account, account); + funded += 1; + } + + if funded == 0 { + return Err(BuilderError::AccountNotFound(format!( + "no funding account of any type at index {source_index}" + ))); + } + + Ok((builder, paths)) } } #[cfg(test)] @@ -539,7 +626,7 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, - AccountTypePreference::BIP44, + &[AccountTypePreference::BIP44], 99, dest_outputs(100_000), FeeRate::normal(), @@ -578,7 +665,7 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, - AccountTypePreference::BIP44, + &[AccountTypePreference::BIP44], 0, dest_outputs(100_000), FeeRate::normal(), @@ -644,7 +731,7 @@ mod tests { let (tx, fee) = info .build_and_sign_transaction_with_signer( &wallet, - AccountTypePreference::BIP44, + &[AccountTypePreference::BIP44], 0, dest_outputs(send_amount), FeeRate::normal(), @@ -685,7 +772,7 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, - AccountTypePreference::BIP44, + &[AccountTypePreference::BIP44], 0, dest_outputs(500_000), FeeRate::normal(), @@ -736,7 +823,7 @@ mod tests { let result = info .build_and_sign_transaction_with_signer( &wallet, - AccountTypePreference::BIP44, + &[AccountTypePreference::BIP44], 0, outputs, FeeRate::normal(), @@ -750,4 +837,88 @@ mod tests { result.err() ); } + + // -- Multi-account funding -- + + use dashcore::{OutPoint, TxOut}; + use std::collections::HashSet; + use test_case::test_case; + + /// Put a confirmed 300k UTXO on a fresh receive address of the account at + /// index 0 and return its outpoint. + fn fund( + wallet: &Wallet, + info: &mut ManagedWalletInfo, + preference: AccountTypePreference, + txid_byte: u8, + ) -> OutPoint { + let (account_xpub, account) = match preference { + AccountTypePreference::BIP32 => ( + wallet.get_bip32_account(0).unwrap().account_xpub, + info.accounts.standard_bip32_accounts.get_mut(&0).unwrap(), + ), + _ => ( + wallet.get_bip44_account(0).unwrap().account_xpub, + info.accounts.standard_bip44_accounts.get_mut(&0).unwrap(), + ), + }; + let address = account.next_receive_address(Some(&account_xpub), true).unwrap(); + let outpoint = OutPoint { + txid: Txid::from_byte_array([txid_byte; 32]), + vout: 0, + }; + account.utxos.insert( + outpoint, + Utxo { + outpoint, + txout: TxOut { + value: 300_000, + script_pubkey: address.script_pubkey(), + }, + address, + height: 1000, + is_coinbase: false, + is_confirmed: true, + is_instantlocked: false, + is_locked: false, + is_trusted: false, + }, + ); + outpoint + } + + /// Neither account covers the 500k target on its own, so the build only + /// succeeds by pooling both — and signing them proves the derivation paths + /// were collected across both accounts. + #[test_case(&[AccountTypePreference::BIP44, AccountTypePreference::BIP32] ; "named explicitly")] + #[test_case(&[] ; "empty list draws from every type")] + #[tokio::test] + async fn funding_pools_across_account_types(sources: &[AccountTypePreference]) { + let (wallet, mut info) = test_wallet_and_info(); + let bip44 = fund(&wallet, &mut info, AccountTypePreference::BIP44, 0x11); + let bip32 = fund(&wallet, &mut info, AccountTypePreference::BIP32, 0x22); + info.update_last_processed_height(1100); + + let (tx, _fee) = info + .build_and_sign_transaction( + &wallet, + sources, + 0, + dest_outputs(500_000), + FeeRate::normal(), + SelectionStrategy::BranchAndBound, + ) + .await + .expect("a 500k send funded by two 300k accounts"); + + let spent: HashSet = tx.input.iter().map(|txin| txin.previous_output).collect(); + assert_eq!(spent, HashSet::from([bip44, bip32])); + + // Each account reserves what it contributed, in its own set, so a build + // funded from only one of them still skips the spent-to-be UTXO. + let bip44_account = info.accounts.standard_bip44_accounts.get(&0).unwrap(); + let bip32_account = info.accounts.standard_bip32_accounts.get(&0).unwrap(); + assert_eq!(bip44_account.reservations().reserved(1100), HashSet::from([bip44])); + assert_eq!(bip32_account.reservations().reserved(1100), HashSet::from([bip32])); + } } From 944e53a503a88daa0d7706d7d7c5f5350213d295 Mon Sep 17 00:00:00 2001 From: Borja Castellano Date: Fri, 7 Aug 2026 14:12:00 +0200 Subject: [PATCH 069/149] feat(key-wallet): dashpay contact account funding (#929) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(key-wallet): skip already used accounts when funding a transaction * feat(key-wallet): fund transactions from DashPay contact accounts Funds received from a DashPay contact (DIP-15) were unreachable from `build_and_sign_transaction`: `AccountTypePreference` only named the standard families, so a contact account could hold a balance the wallet could never spend. `AccountTypePreference::DashpayReceivingFunds` names one contact by its identity pair. A contact may hold funds at several account indices, so the source covers all of them and ignores `source_index` — unlike the standard families, an index alone does not identify a contact account. Only the receiving side is offered: a `DashpayExternalAccount` tracks what we sent *to* the contact and its keys are theirs, so it can never fund a spend of ours. Contact accounts also stay out of the implicit (empty) source list, which cannot guess which contact to spend from. The account lookup now goes through `AccountType`, which both sides of the wallet already key on: `AccountCollection::account_of_type` on the wallet side, and a new `ManagedAccountCollection::funds_account{,_mut}` on the managed side, which addresses any funds-bearing account including the DashPay ones (keyed by identity pair plus index). Contact accounts keep a single address pool with no internal branch, so they cannot supply change: a contact-only build fails with `NoChangeAddress` unless a Standard account comes along to take it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01ToH2xGXqVcxiwMkNYaWkh7 * fix(key-wallet): skipping already used AccountType when funding a transaction --------- Co-authored-by: Claude Opus 5 (1M context) --- .../managed_account_collection.rs | 69 +++- .../src/wallet/managed_wallet_info/mod.rs | 19 + .../transaction_building.rs | 384 +++++++++++++++--- .../wallet_info_interface.rs | 12 +- 4 files changed, 424 insertions(+), 60 deletions(-) diff --git a/key-wallet/src/managed_account/managed_account_collection.rs b/key-wallet/src/managed_account/managed_account_collection.rs index 803e1db45..5df90c72c 100644 --- a/key-wallet/src/managed_account/managed_account_collection.rs +++ b/key-wallet/src/managed_account/managed_account_collection.rs @@ -17,7 +17,7 @@ use crate::managed_account::managed_platform_account::ManagedPlatformAccount; use crate::managed_account::{ManagedCoreFundsAccount, ManagedCoreKeysAccount}; use crate::transaction_checking::account_checker::CoreAccountTypeMatch; use crate::KeySource; -use crate::{Account, AccountCollection}; +use crate::{Account, AccountCollection, AccountType}; #[cfg(feature = "serde")] use serde::{Deserialize, Serialize}; @@ -577,13 +577,76 @@ impl ManagedAccountCollection { )) } + /// Get the spendable account for an exact [`AccountType`] + pub fn funds_account(&self, account_type: &AccountType) -> Option<&ManagedCoreFundsAccount> { + match account_type { + AccountType::Standard { + index, + standard_account_type, + } => match standard_account_type { + crate::account::StandardAccountType::BIP44Account => { + self.standard_bip44_accounts.get(index) + } + crate::account::StandardAccountType::BIP32Account => { + self.standard_bip32_accounts.get(index) + } + }, + AccountType::CoinJoin { + index, + } => self.coinjoin_accounts.get(index), + AccountType::DashpayReceivingFunds { + index, + user_identity_id, + friend_identity_id, + } => self.dashpay_receival_accounts.get(&DashpayAccountKey { + index: *index, + user_identity_id: *user_identity_id, + friend_identity_id: *friend_identity_id, + }), + _ => None, + } + } + + /// Mutable twin of [`Self::funds_account`]. + pub fn funds_account_mut( + &mut self, + account_type: &AccountType, + ) -> Option<&mut ManagedCoreFundsAccount> { + match account_type { + AccountType::Standard { + index, + standard_account_type, + } => match standard_account_type { + crate::account::StandardAccountType::BIP44Account => { + self.standard_bip44_accounts.get_mut(index) + } + crate::account::StandardAccountType::BIP32Account => { + self.standard_bip32_accounts.get_mut(index) + } + }, + AccountType::CoinJoin { + index, + } => self.coinjoin_accounts.get_mut(index), + AccountType::DashpayReceivingFunds { + index, + user_identity_id, + friend_identity_id, + } => self.dashpay_receival_accounts.get_mut(&DashpayAccountKey { + index: *index, + user_identity_id: *user_identity_id, + friend_identity_id: *friend_identity_id, + }), + _ => None, + } + } + /// Get a funds-bearing account by primary index across Standard BIP44, /// Standard BIP32, and CoinJoin accounts. /// /// Returns only [`ManagedCoreFundsAccount`] entries — keys-only accounts /// (identity / asset-lock / provider) are not reachable via this index - /// lookup. Use [`Self::get_by_account_type_match`] or direct field access - /// for those. + /// lookup. Use [`Self::funds_account`], [`Self::get_by_account_type_match`] + /// or direct field access for those. pub fn get(&self, index: u32) -> Option<&ManagedCoreFundsAccount> { if let Some(account) = self.standard_bip44_accounts.get(&index) { return Some(account); diff --git a/key-wallet/src/wallet/managed_wallet_info/mod.rs b/key-wallet/src/wallet/managed_wallet_info/mod.rs index f687d4953..698e3327b 100644 --- a/key-wallet/src/wallet/managed_wallet_info/mod.rs +++ b/key-wallet/src/wallet/managed_wallet_info/mod.rs @@ -451,6 +451,15 @@ impl ManagedWalletInfo { debug_assert!(false, "CoinJoin accounts are spend-only in our current use cases"); None } + // DIP-15 contact accounts have a single address pool and no + // internal (change) branch, so they can never supply change. + AccountTypePreference::DashpayFriendshipReceivingFunds { + .. + } + | AccountTypePreference::DashpayIdentityReceivingFunds { + .. + } + | AccountTypePreference::AllDashpayReceivingFunds => None, }; address @@ -498,6 +507,16 @@ impl ManagedWalletInfo { debug_assert!(false, "CoinJoin accounts are spend-only in our current use cases"); None } + // Addresses of a DIP-15 contact account are handed out through the + // contact flow that knows the identity pair, not through this + // index-addressed helper. + AccountTypePreference::DashpayFriendshipReceivingFunds { + .. + } + | AccountTypePreference::DashpayIdentityReceivingFunds { + .. + } + | AccountTypePreference::AllDashpayReceivingFunds => None, }; address diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs index 63d4f0259..50aa36277 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_building.rs @@ -1,5 +1,6 @@ //! Transaction building functionality for managed wallets +use crate::account::StandardAccountType; use crate::managed_account::managed_account_trait::ManagedAccountTrait; use crate::signer::Signer; use crate::wallet::managed_wallet_info::coin_selection::SelectionStrategy; @@ -9,17 +10,26 @@ use crate::wallet::managed_wallet_info::transaction_builder::{ }; use crate::wallet::managed_wallet_info::wallet_info_interface::WalletInfoInterface; use crate::wallet::ManagedWalletInfo; -use crate::{DerivationPath, Wallet}; +use crate::{AccountType, DerivationPath, Wallet}; +use core::fmt; use dashcore::address::NetworkUnchecked; use dashcore::{Address, Transaction}; -use std::collections::HashMap; +use std::collections::{HashMap, HashSet}; /// Account type preference for transaction building -#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] pub enum AccountTypePreference { BIP44, BIP32, CoinJoin, + DashpayFriendshipReceivingFunds { + user_identity_id: [u8; 32], + friend_identity_id: [u8; 32], + }, + DashpayIdentityReceivingFunds { + user_identity_id: [u8; 32], + }, + AllDashpayReceivingFunds, } impl AccountTypePreference { @@ -27,8 +37,57 @@ impl AccountTypePreference { /// /// CoinJoin is deliberately absent: spending mixed outputs alongside /// transparent ones in the same transaction links them and undoes the - /// mixing + /// mixing. So are DashPay accounts: an index alone does not identify one, + /// and which contact to spend from is not something a default can pick. pub const DEFAULT: [Self; 2] = [Self::BIP44, Self::BIP32]; + + /// The single [`AccountType`] this preference selects at `index`, or `None` + /// for a DashPay source that names a set of accounts rather than one. + pub fn account_type(&self, index: u32) -> Option { + Some(match *self { + Self::BIP44 => AccountType::Standard { + index, + standard_account_type: StandardAccountType::BIP44Account, + }, + Self::BIP32 => AccountType::Standard { + index, + standard_account_type: StandardAccountType::BIP32Account, + }, + Self::CoinJoin => AccountType::CoinJoin { + index, + }, + Self::DashpayFriendshipReceivingFunds { + user_identity_id, + friend_identity_id, + } => AccountType::DashpayReceivingFunds { + index, + user_identity_id, + friend_identity_id, + }, + Self::DashpayIdentityReceivingFunds { + .. + } + | Self::AllDashpayReceivingFunds => return None, + }) + } +} + +impl fmt::Display for AccountTypePreference { + /// Elides the 32-byte identity hashes so error messages stay readable. + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::BIP44 => f.write_str("BIP44"), + Self::BIP32 => f.write_str("BIP32"), + Self::CoinJoin => f.write_str("CoinJoin"), + Self::DashpayFriendshipReceivingFunds { + .. + } => f.write_str("DashpayReceiving{friendship}"), + Self::DashpayIdentityReceivingFunds { + .. + } => f.write_str("DashpayReceiving{identity}"), + Self::AllDashpayReceivingFunds => f.write_str("DashpayReceiving{all}"), + } + } } impl ManagedWalletInfo { @@ -143,10 +202,42 @@ impl ManagedWalletInfo { Ok((builder, paths)) } + /// The accounts a source resolves to: the one at `source_index` for the + /// standard families, and every DashPay receiving account the selector + /// picks for a DashPay source. + fn account_types_for( + &self, + preference: AccountTypePreference, + source_index: u32, + ) -> Vec { + let (identity, friend) = match preference { + AccountTypePreference::AllDashpayReceivingFunds => (None, None), + AccountTypePreference::DashpayIdentityReceivingFunds { + user_identity_id, + } => (Some(user_identity_id), None), + AccountTypePreference::DashpayFriendshipReceivingFunds { + user_identity_id, + friend_identity_id, + } => (Some(user_identity_id), Some(friend_identity_id)), + _ => return preference.account_type(source_index).into_iter().collect(), + }; + + self.accounts + .dashpay_receival_accounts + .keys() + .filter(|key| identity.is_none_or(|id| key.user_identity_id == id)) + .filter(|key| friend.is_none_or(|id| key.friend_identity_id == id)) + .map(|key| AccountType::DashpayReceivingFunds { + index: key.index, + user_identity_id: key.user_identity_id, + friend_identity_id: key.friend_identity_id, + }) + .collect() + } + /// Seed `builder` with the UTXOs of every funding account named by - /// `sources` at `source_index`, returning it alongside the derivation path - /// of each candidate input address, since the inputs can come from - /// different accounts. + /// `sources`, returning it alongside the derivation path of each candidate + /// input address, since the inputs can come from different accounts. fn fund( &mut self, wallet: &Wallet, @@ -162,45 +253,46 @@ impl ManagedWalletInfo { }; let mut paths = HashMap::new(); - let mut funded = 0usize; + let mut funded: HashSet = HashSet::new(); for &preference in preferences { - let account = match preference { - AccountTypePreference::BIP44 => wallet.get_bip44_account(source_index), - AccountTypePreference::BIP32 => wallet.get_bip32_account(source_index), - AccountTypePreference::CoinJoin => wallet.get_coinjoin_account(source_index), - }; - let managed_account = match preference { - AccountTypePreference::BIP44 => { - self.accounts.standard_bip44_accounts.get_mut(&source_index) - } - AccountTypePreference::BIP32 => { - self.accounts.standard_bip32_accounts.get_mut(&source_index) - } - AccountTypePreference::CoinJoin => { - self.accounts.coinjoin_accounts.get_mut(&source_index) - } - }; + let account_types = self.account_types_for(preference, source_index); + if account_types.is_empty() && named_explicitly { + return Err(BuilderError::AccountNotFound(format!("account {preference}"))); + } - let (Some(account), Some(managed_account)) = (account, managed_account) else { - if named_explicitly { - return Err(BuilderError::AccountNotFound(format!( - "account {preference:?} #{source_index}" - ))); + for account_type in account_types { + // Sources overlap: a DashPay source names a set of accounts, + // and another source can name some of the same ones. Funding an + // account twice would offer coin selection every one of its + // UTXOs twice, letting it spend a single output as two inputs. + if funded.contains(&account_type) { + continue; } - continue; - }; - for utxo in managed_account.utxos.values() { - if let Some(path) = managed_account.address_derivation_path(&utxo.address) { - paths.insert(utxo.address.clone(), path); + let account = wallet.accounts.account_of_type(account_type); + let managed_account = self.accounts.funds_account_mut(&account_type); + + let (Some(account), Some(managed_account)) = (account, managed_account) else { + if named_explicitly { + return Err(BuilderError::AccountNotFound(format!( + "account {account_type}" + ))); + } + continue; + }; + + for utxo in managed_account.utxos.values() { + if let Some(path) = managed_account.address_derivation_path(&utxo.address) { + paths.insert(utxo.address.clone(), path); + } } + builder = builder.add_funding(managed_account, account); + funded.insert(account_type); } - builder = builder.add_funding(managed_account, account); - funded += 1; } - if funded == 0 { + if funded.is_empty() { return Err(BuilderError::AccountNotFound(format!( "no funding account of any type at index {source_index}" ))); @@ -840,29 +932,43 @@ mod tests { // -- Multi-account funding -- + use crate::managed_account::managed_account_trait::ManagedAccountTrait; + use crate::wallet::managed_wallet_info::managed_account_operations::ManagedAccountOperations; + use crate::{KeySource, ManagedAccountType}; use dashcore::{OutPoint, TxOut}; use std::collections::HashSet; use test_case::test_case; /// Put a confirmed 300k UTXO on a fresh receive address of the account at - /// index 0 and return its outpoint. + /// `index` and return its outpoint. fn fund( wallet: &Wallet, info: &mut ManagedWalletInfo, preference: AccountTypePreference, + index: u32, txid_byte: u8, ) -> OutPoint { - let (account_xpub, account) = match preference { - AccountTypePreference::BIP32 => ( - wallet.get_bip32_account(0).unwrap().account_xpub, - info.accounts.standard_bip32_accounts.get_mut(&0).unwrap(), - ), - _ => ( - wallet.get_bip44_account(0).unwrap().account_xpub, - info.accounts.standard_bip44_accounts.get_mut(&0).unwrap(), - ), + let account_type = preference.account_type(index).expect("a single account"); + let account_xpub = wallet + .accounts + .account_of_type(account_type) + .unwrap_or_else(|| panic!("wallet account {preference}")) + .account_xpub; + let account = info + .accounts + .funds_account_mut(&account_type) + .unwrap_or_else(|| panic!("managed account {preference}")); + + // A contact account keeps a single pool and so has no receive-address + // helper of its own; its addresses come straight off that pool. + let address = match account.managed_account_type_mut() { + ManagedAccountType::DashpayReceivingFunds { + addresses, + .. + } => addresses.next_unused(&KeySource::Public(account_xpub), true).unwrap(), + _ => account.next_receive_address(Some(&account_xpub), true).unwrap(), }; - let address = account.next_receive_address(Some(&account_xpub), true).unwrap(); + let outpoint = OutPoint { txid: Txid::from_byte_array([txid_byte; 32]), vout: 0, @@ -887,6 +993,35 @@ mod tests { outpoint } + /// Naming one account twice must not offer its UTXOs twice: a 400k target + /// is not met by a single 300k coin, however often its account is named. + #[tokio::test] + async fn naming_an_account_twice_does_not_double_its_funds() { + let (wallet, mut info) = test_wallet_and_info(); + fund(&wallet, &mut info, AccountTypePreference::BIP44, 0, 0x11); + info.update_last_processed_height(1100); + + let result = info + .build_and_sign_transaction( + &wallet, + &[AccountTypePreference::BIP44, AccountTypePreference::BIP44], + 0, + dest_outputs(400_000), + FeeRate::normal(), + SelectionStrategy::BranchAndBound, + ) + .await; + + assert!( + matches!( + result, + Err(BuilderError::InsufficientFunds { .. }) | Err(BuilderError::CoinSelection(_)) + ), + "300k must not cover a 400k target, got: {:?}", + result.map(|(tx, _)| tx.input.len()) + ); + } + /// Neither account covers the 500k target on its own, so the build only /// succeeds by pooling both — and signing them proves the derivation paths /// were collected across both accounts. @@ -895,8 +1030,8 @@ mod tests { #[tokio::test] async fn funding_pools_across_account_types(sources: &[AccountTypePreference]) { let (wallet, mut info) = test_wallet_and_info(); - let bip44 = fund(&wallet, &mut info, AccountTypePreference::BIP44, 0x11); - let bip32 = fund(&wallet, &mut info, AccountTypePreference::BIP32, 0x22); + let bip44 = fund(&wallet, &mut info, AccountTypePreference::BIP44, 0, 0x11); + let bip32 = fund(&wallet, &mut info, AccountTypePreference::BIP32, 0, 0x22); info.update_last_processed_height(1100); let (tx, _fee) = info @@ -921,4 +1056,153 @@ mod tests { assert_eq!(bip44_account.reservations().reserved(1100), HashSet::from([bip44])); assert_eq!(bip32_account.reservations().reserved(1100), HashSet::from([bip32])); } + + // -- DashPay contact accounts -- + + /// A source naming the friendship between our identity `user` and `friend`. + fn friendship(user: u8, friend: u8) -> AccountTypePreference { + AccountTypePreference::DashpayFriendshipReceivingFunds { + user_identity_id: [user; 32], + friend_identity_id: [friend; 32], + } + } + + /// Add that friendship's receiving account at `index` and fund it with 300k. + fn add_funded_friendship( + wallet: &mut Wallet, + info: &mut ManagedWalletInfo, + user: u8, + friend: u8, + index: u32, + txid_byte: u8, + ) -> OutPoint { + let account_type = friendship(user, friend).account_type(index).unwrap(); + wallet.add_account(account_type, None).unwrap(); + info.add_managed_account(wallet, account_type).unwrap(); + fund(wallet, info, friendship(user, friend), index, txid_byte) + } + + /// Funds received from a contact are spendable, across every account index + /// that contact holds funds at — 300k at each of two indices covers a 500k + /// target that neither covers alone. The DIP-15 contact path is 256-bit, so + /// the build only succeeds if it resolved for signing, and a contact account + /// has no internal pool, so BIP44 comes along for change. + #[tokio::test] + async fn dashpay_friendship_funds_a_spend_across_indices() { + let (mut wallet, mut info) = test_wallet_and_info(); + let contact_0 = add_funded_friendship(&mut wallet, &mut info, 0xaa, 0xbb, 0, 0x44); + let contact_1 = add_funded_friendship(&mut wallet, &mut info, 0xaa, 0xbb, 1, 0x55); + info.update_last_processed_height(1100); + + let send_amount = 500_000u64; + let (tx, _fee) = info + .build_and_sign_transaction( + &wallet, + &[friendship(0xaa, 0xbb), AccountTypePreference::BIP44], + 0, + dest_outputs(send_amount), + FeeRate::normal(), + SelectionStrategy::BranchAndBound, + ) + .await + .expect("a 500k send funded by one contact's accounts"); + + let spent: HashSet = tx.input.iter().map(|txin| txin.previous_output).collect(); + assert_eq!(spent, HashSet::from([contact_0, contact_1])); + + let change = tx + .output + .iter() + .find(|out| out.value != send_amount) + .expect("600k in against a 500k target leaves change"); + let bip44 = info.accounts.standard_bip44_accounts.get(&0).unwrap(); + assert!(bip44.managed_account_type().all_script_pubkeys().contains(&change.script_pubkey)); + } + + /// Overlapping DashPay sources must not fund an account twice. The build + /// drains, so every candidate input reaches the transaction and a coin + /// offered twice shows up as two inputs spending the same outpoint — + /// whichever way the sources are combined. + #[test_case( + vec![AccountTypePreference::AllDashpayReceivingFunds, friendship(0xaa, 0xbb)] + ; "all, then a friendship it already covers" + )] + #[test_case( + vec![friendship(0xaa, 0xbb), AccountTypePreference::AllDashpayReceivingFunds] + ; "a friendship, then all" + )] + #[test_case( + vec![ + AccountTypePreference::DashpayIdentityReceivingFunds { + user_identity_id: [0xaa; 32], + }, + friendship(0xaa, 0xbb), + ] + ; "an identity, then one of its friendships" + )] + #[tokio::test] + async fn overlapping_dashpay_sources_fund_each_account_once( + sources: Vec, + ) { + let (mut wallet, mut info) = test_wallet_and_info(); + let first = add_funded_friendship(&mut wallet, &mut info, 0xaa, 0xbb, 0, 0x44); + let second = add_funded_friendship(&mut wallet, &mut info, 0xaa, 0xcc, 0, 0x55); + info.update_last_processed_height(1100); + + let (tx, _fee) = info + .build_and_sign_transaction( + &wallet, + &sources, + 0, + dest_outputs(400_000), + FeeRate::normal(), + SelectionStrategy::All, + ) + .await + .expect("draining the two contacts' accounts"); + + // Draining spends every candidate, so an account funded twice would + // put its coin in here a second time. + let spent: Vec = tx.input.iter().map(|txin| txin.previous_output).collect(); + assert_eq!(spent.len(), 2, "the two contacts' coins, once each: {spent:?}"); + assert_eq!(spent.iter().copied().collect::>(), HashSet::from([first, second])); + } + + /// Two of our identities hold 300k each, so only a source spanning both + /// covers a 500k target. + #[test_case(AccountTypePreference::AllDashpayReceivingFunds, true ; "every identity")] + #[test_case( + AccountTypePreference::DashpayIdentityReceivingFunds { + user_identity_id: [0xaa; 32], + }, + false ; "a single identity" + )] + #[tokio::test] + async fn dashpay_sources_span_the_identities_they_name( + source: AccountTypePreference, + covers_target: bool, + ) { + let (mut wallet, mut info) = test_wallet_and_info(); + add_funded_friendship(&mut wallet, &mut info, 0xaa, 0xbb, 0, 0x44); + add_funded_friendship(&mut wallet, &mut info, 0xcc, 0xdd, 0, 0x55); + info.update_last_processed_height(1100); + + let result = info + .build_and_sign_transaction( + &wallet, + &[source, AccountTypePreference::BIP44], + 0, + dest_outputs(500_000), + FeeRate::normal(), + SelectionStrategy::BranchAndBound, + ) + .await; + + assert_eq!( + result.is_ok(), + covers_target, + "got: {:?}", + result.map(|(tx, _)| tx.input.len()) + ); + } } diff --git a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs index 0d4de1335..db795d200 100644 --- a/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs +++ b/key-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs @@ -129,18 +129,16 @@ pub trait WalletInfoInterface: Sized + WalletTransactionChecker + ManagedAccount .collect() } - /// The funds-bearing account selected by `(preference, index)`, if it exists. + /// The funds-bearing account selected by `(preference, index)`, if it + /// exists. `None` for a DashPay source naming a set of accounts rather than + /// one — those are only reachable through transaction building. fn funds_account( &self, preference: AccountTypePreference, index: u32, ) -> Option<&ManagedCoreFundsAccount> { - let accounts = self.accounts(); - match preference { - AccountTypePreference::BIP44 => accounts.standard_bip44_accounts.get(&index), - AccountTypePreference::BIP32 => accounts.standard_bip32_accounts.get(&index), - AccountTypePreference::CoinJoin => accounts.coinjoin_accounts.get(&index), - } + let account_type = preference.account_type(index)?; + self.accounts().funds_account(&account_type) } /// Balance of the funds-bearing account selected by `(preference, index)`, if it exists. From 6be24078d6705b97c7afb2a874144829fa1a49f5 Mon Sep 17 00:00:00 2001 From: QuantumExplorer Date: Fri, 7 Aug 2026 22:47:16 +0700 Subject: [PATCH 070/149] fix(key-wallet): never offer an outpoint the builder already holds (#931) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `add_funding` appends every unreserved UTXO of a funding account to the candidate set. When the builder was already seeded with one of those UTXOs — `add_inputs` with a caller-chosen outpoint, or an earlier `add_funding` of an overlapping account — the same outpoint became TWO candidates. Coin selection does not deduplicate by outpoint: `SelectionStrategy::All` takes every candidate, so the duplicate was guaranteed to reach the transaction, and the ordinary strategies could pick both copies and double-count them toward the target. Either way the result is a transaction spending one prevout twice, which Core rejects. This is specific to additive funding. The `set_funding` it replaced assigned `self.inputs`, so a pre-seeded outpoint was silently dropped rather than duplicated — an invalid transaction is the strictly worse failure, so fix it where the candidates are built. The account still records every unreserved outpoint it owns that is in the candidate pool, including a pre-seeded one, so that if selection picks it the owning account reserves it. Recording only the UTXOs this call appended would leave a pre-seeded input unreserved and free for a concurrent build to select. Co-authored-by: Claude Opus 5 --- .../transaction_builder.rs | 89 ++++++++++++++++--- 1 file changed, 79 insertions(+), 10 deletions(-) diff --git a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs index 7c42d2871..15d0c92cb 100644 --- a/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs +++ b/key-wallet/src/wallet/managed_wallet_info/transaction_builder.rs @@ -148,16 +148,33 @@ impl TransactionBuilder { /// read-then-reserve window for a concurrent build. pub fn add_funding(mut self, funds_acc: &mut ManagedCoreFundsAccount, acc: &Account) -> Self { let reserved = funds_acc.reservations().reserved(self.current_height); - let candidates: Vec = funds_acc - .utxos - .values() - .filter(|utxo| !reserved.contains(&utxo.outpoint)) - .cloned() - .collect(); - self.funding.push(( - funds_acc.reservations().clone(), - candidates.iter().map(|utxo| utxo.outpoint).collect(), - )); + // An outpoint the builder already holds — seeded by `add_inputs`, or + // offered by an earlier `add_funding` of an overlapping account — must + // not become a second candidate for the SAME outpoint. Coin selection + // does not deduplicate (and `SelectionStrategy::All` takes every + // candidate), so a duplicate would be spent twice in one transaction and + // Core rejects duplicate prevouts. This is additive funding's hazard + // specifically: the `set_funding` it replaced overwrote `inputs`, so a + // pre-seeded outpoint was silently dropped instead of duplicated. + let present: HashSet = self.inputs.iter().map(|utxo| utxo.outpoint).collect(); + let mut candidates: Vec = Vec::new(); + // Every unreserved UTXO this account owns that ends up in the candidate + // pool — including one pre-seeded by `add_inputs` — so that if selection + // picks it, THIS account reserves it. Recording only the UTXOs added + // here would leave a pre-seeded input unreserved and free for a + // concurrent build to select. + let mut owned: HashSet = HashSet::new(); + for utxo in funds_acc.utxos.values() { + if reserved.contains(&utxo.outpoint) { + continue; + } + owned.insert(utxo.outpoint); + if present.contains(&utxo.outpoint) { + continue; + } + candidates.push(utxo.clone()); + } + self.funding.push((funds_acc.reservations().clone(), owned)); self.inputs.extend(candidates); if self.change_addr.is_none() { self.change_addr = funds_acc.next_change_address(Some(&acc.account_xpub), true).ok(); @@ -1775,6 +1792,58 @@ mod tests { assert!(candidates.contains(&free.outpoint)); } + /// A UTXO seeded with `add_inputs` and then offered again by `add_funding` + /// must appear ONCE. Additive funding otherwise pushes a second candidate + /// for the same outpoint, and since coin selection does not deduplicate, + /// `SelectionStrategy::All` spends it twice — a transaction Core rejects + /// for duplicate prevouts. + #[test] + fn add_funding_does_not_duplicate_a_pre_seeded_input() { + let ctx = TestWalletContext::new_random(); + let account = + ctx.wallet.accounts.standard_bip44_accounts.get(&0).expect("BIP44 account").clone(); + + let mut funds = ManagedCoreFundsAccount::dummy_bip44(); + let seeded = Utxo::dummy(0x01, 500_000, 100, false, true); + let other = Utxo::dummy(0x02, 500_000, 100, false, true); + funds.utxos.insert(seeded.outpoint, seeded.clone()); + funds.utxos.insert(other.outpoint, other.clone()); + + let builder = TransactionBuilder::new() + .set_current_height(200) + .set_selection_strategy(SelectionStrategy::All) + .add_inputs(vec![seeded.clone()]) + .add_funding(&mut funds, &account) + .add_output(&ctx.receive_address, 100_000); + + let candidates: Vec = builder.inputs.iter().map(|utxo| utxo.outpoint).collect(); + assert_eq!( + candidates.iter().filter(|op| **op == seeded.outpoint).count(), + 1, + "the pre-seeded outpoint must be offered exactly once, got {candidates:?}" + ); + + let (tx, _fee, token) = builder.build_unsigned_reserved().expect("build"); + let mut prevouts: Vec = tx.input.iter().map(|i| i.previous_output).collect(); + prevouts.sort_by_key(|op| (op.txid.to_byte_array(), op.vout)); + let deduped = { + let mut d = prevouts.clone(); + d.dedup(); + d + }; + assert_eq!(prevouts, deduped, "transaction must not contain duplicate prevouts"); + + // The pre-seeded input still belongs to this account, so a drain that + // spends it must RESERVE it — otherwise a concurrent build would see it + // as free and double-spend it. + assert!(token.is_some(), "a funded build stamps a reservation token"); + let reserved = funds.reservations().reserved(200); + assert!( + reserved.contains(&seeded.outpoint), + "the pre-seeded input must be reserved by the account that owns it" + ); + } + #[tokio::test] async fn build_signed_releases_reservation_on_signing_failure() { let ctx = TestWalletContext::new_random(); From b056d07c61f8618f05082552bbb88072290d57c1 Mon Sep 17 00:00:00 2001 From: Kevin Rombach <35775977+xdustinface@users.noreply.github.com> Date: Sat, 8 Aug 2026 22:30:36 +1000 Subject: [PATCH 071/149] fix(dash-spv): stop the filter rescan loop when the chain anchors above the wallet (#933) * fix(dash-spv): stop the filter rescan loop when the chain anchors above the wallet A wallet whose `synced_height` sits below the chain anchor could never be certified, and the resulting rescan never terminated. On mainnet `hd_wallet_sync_floor` pushes the anchor to the checkpoint at 200000 while the CLI creates wallets with birth height 0, so `synced_height` stayed at 0 for the whole run: 4846 restarts and 51784 batch scans produced zero sync-height advances and burned ~350% CPU indefinitely. Two independent gates jammed, both because neither knew that heights below the anchor are out of scope rather than pending work. `tick` restarted the scan whenever `wallets_behind` reported anyone below the committed frontier, wiping in-flight batches before any could commit. It now compares against the height a restart would actually resume from, which is floored at the wallets' birth heights and the stored headers' start. The commit-time contiguity guard required `synced_height + 1 >= batch_start`, which reads `1 >= 200000` under an anchor and never holds. It now substitutes `scan_floor - 1` for an unreachable `synced_height`. The floor deliberately excludes the `committed_height + 1` term of `scan_start`: tracking the scan frontier would reduce the guard to `batch_start >= batch_start` and delete the height check that `#649` relies on. A genuine gap above the floor still blocks the advance, and the account-generation guard is untouched. The floor comes from the stored header start rather than `hd_wallet_sync_floor`, so `--start-height` is covered too. That path bypasses the network clamp entirely and reproduces the same loop on testnet. Also warn when the anchor exceeds the wallet's birth height. That range is never scanned, and because BIP44 discovery is sequential, usage hidden in it can stop the gap-limit window from advancing and mask later addresses. * test(dash-spv): cover filter sync when the chain anchors above the wallet `hd_wallet_sync_floor` is non-zero only for mainnet, so every existing test ran with a scan floor of 0 where the contiguity guard reads `0 + 1 >= 0` and passes. The suite was structurally unable to reach the stall. `create_anchored_test_manager` stores headers at a non-zero start height instead, which is where the floor is read from. That reproduces the mainnet shape without depending on `Network`, so it runs in normal CI. Four cases fail before the fix: a wallet below the anchor reaching `Synced`, the boundary sweep over `synced_height` in `test_tick_does_not_rescan_when_no_wallets_behind` (199998 included so a fix keyed on `synced_height == 0` does not pass), no re-request of in-flight filters after a tick, and a wallet added at runtime converging. The last one is the case that raising the birth height at construction would not have covered. Two more extend the existing `#649` guard tests and pass both before and after. They pin the behavior a wrong floor would break: a gap between the anchor and the batch must still block certification, and with contiguity satisfied at the floor the generation guard must be the only thing left holding the advance. * test(dash-spv): seed anchored storage in place instead of swapping it in The multi-wallet test built a throwaway manager only to move three storage handles onto another one, which orphaned the storage `FiltersManager::new` had already read its progress from. `seed_anchored_storage` seeds whichever manager it is given, so both wallet types share one path. It also added wallet B before sync ever started, so it never exercised the runtime add its name claimed. B now joins after the frontier is reached, which covers the other half of the fix: a wallet that genuinely needs a rescan must still get one, since a floor that suppressed every restart would pass the other tests just as well. `drain_getcfilters` replaces the two copies of the request-drain destructure. --- dash-spv/src/sync/filters/manager.rs | 315 +++++++++++++++++++++- dash-spv/src/sync/filters/sync_manager.rs | 29 +- 2 files changed, 335 insertions(+), 9 deletions(-) diff --git a/dash-spv/src/sync/filters/manager.rs b/dash-spv/src/sync/filters/manager.rs index 005ca99dd..f01fbcb59 100644 --- a/dash-spv/src/sync/filters/manager.rs +++ b/dash-spv/src/sync/filters/manager.rs @@ -213,6 +213,21 @@ impl