[william-gr]

Adds an HA mode to truenas-appliance-runner. With ha: true, the action boots two TrueNAS appliances inside the runner, set up the way middleware expects an HA pair's hardware to be, and hands them over unconfigured. Licensing, the VIP, a pool and failover are left for the suite to set up, because that setup is what an HA suite tests.

The new ha.sh sits next to appliance.sh and reuses lib.sh and tnctl.py. The single-appliance path is unchanged: ha defaults to false.

What the pair is built from

• Runner: a scale set that attaches the appliance disk twice: {"disks": ["truenas_appliance:nightly", "truenas_appliance:nightly"]}.
• Node identity: SMBIOS manufacturer QEMU, with serials ending _c1 and _c2. Middleware then reports IXKVM, node A or B (utils/hardware/detect.py).
• Heartbeat link: the only device behind a PCIe root port, so the guest names it enp1s0, the name middleware hard-codes for IXKVM. It has its own bridge.
• Data network: a bridge with the runner at 10.231.0.1. dnsmasq gives node A .11 and node B .12, and .10 is left free for the VIP. Docker's FORWARD DROP would block traffic between the nodes, so iptables rules accept it on both bridges.
• Shared disks: LIO fileio LUNs, presented to each node through its own loopback target. That gives LIO two separate initiators, so it tells the nodes' reservations apart. Each node attaches them as scsi-block, so SCSI-3 persistent reservations pass through. fenced needs them, and pool.create needs fenced.
• Unique nodes: both OS disks are clones of one image, so both nodes start with the same /etc/machine-id, and fenced builds its reservation keys from it. Node B gets a new machine-id and hostid over SSH and reboots, before anything is licensed.

Two things that were not obvious

• LIO's state file: after every reservation change, LIO writes <db_root>/pr/aptpl_<serial>. If the write fails, LIO still applies the change but returns LOGICAL UNIT COMMUNICATION FAILURE, which looks like a transport bug. Debian's targetcli sets db_root to /etc/rtslib-fb-target and doesn't create pr/ in it, so the script creates it.
• Root for qemu: the kernel passes PERSISTENT RESERVE OUT through SG_IO only for a process with CAP_SYS_RAWIO. qemu therefore runs under sudo, and its QMP socket and files are handed back to the runner user.

Exports

• TN_ variables:* TN_HOST is node A, the only address a fresh pair answers on. Also TN_HA=true, TN_HA_NODE_A, TN_HA_NODE_B, TN_HA_VIP, TN_HA_NETMASK and TN_HA_GATEWAY.
• truenas-appliance on PATH: gains call <a|b>, ssh <a|b> and console <a|b>.
• No baseline snapshot: the shared disks live outside both VMs, so the pair can't be snapshotted together.

Testing

truenas/webui's ha-e2e branch boots a pair with this branch (ha: true, on a truenas-lab-xlarge runner: 12 vCPU, 24 GB). It then sets up HA over the API and runs one real failover: https://github.com/truenas/webui/actions/runs/38956716308

Stage	Time
Pair booted, password and SSH set, B made unique	2m40s
License, interface with VIP, sync to B	2m03s
Pool, HA settled (A MASTER, B BACKUP)	20s
A becomes passive, then B takes the VIP	33s
A back as BACKUP, no disabled reasons	45s

shellcheck -x appliance.sh ha.sh lib.sh is clean, and the CI step now includes ha.sh.

[bugclerk]

Jira URL: https://ixsystems.atlassian.net/browse/NAS-144589

[github-actions]

LOW — With ha: true, the ssh and disk inputs are silently ignored. ha.sh never reads APPLIANCE_SSH or APPLIANCE_DISK. A caller passing ssh: true gets an empty outputs.ssh and no TN_SSH, even though ssh is in fact enabled on both nodes, at TN_HA_NODE_A:22 / TN_HA_NODE_B:22. This only affects the new opt-in path, but the header comment above could say so: ssh is always on for a pair and reached at the node addresses, and disk doesn't apply. Another option is for ha.sh to die when it is given APPLIANCE_DISK.

[github-actions]

LOW — This header and SHARED="${APPLIANCE_DATA_DISKS:-4}" say a pair has 4 shared disks by default. Through the action, though, APPLIANCE_DATA_DISKS is always set from data-disk-count, whose default is '10'. So every caller of the action gets 10 shared LUNs per node, not 4. That is harmless, since the files are sparse, but the documented default only holds when ha.sh is run directly.

[github-actions]

Three findings: three LOW. No human review needed.

The HA path is opt-in (ha defaults to 'false'), and with ha: false the single-appliance path runs exactly as before. Under set -e, [ "$HA" = true ] && script=ha.sh doesn't abort when $HA is false, so existing callers are unaffected. ha.sh calls the lib.sh helpers with the arguments they expect (needs in pairs, ovmf, vars_qcow2, skip_grub), and CI's shellcheck step now covers it.

• LOW — action.yml:127: with ha: true, the ssh and disk inputs are silently ignored. ssh: true gives an empty outputs.ssh and no TN_SSH, although ssh is on at both nodes' addresses (inline).
• LOW — action.yml:152 (outside the diff, so not inline): for a pair, the closing log line prints baseline 'fresh-install' in s, though no baseline is taken.
• LOW — ha.sh:52: the header documents 4 shared disks by default, but through the action data-disk-count always sets it to 10 (inline).

[github-actions]

Approved. 3 finding(s): 3 LOW.

Nothing blocks and no human review is needed.

[bugclerk]

This PR has been merged and conversations have been locked.
If you would like to discuss more about this issue please use our forums or raise a Jira ticket.