Skip to content

squid: revert nvmeof workaround and introduce new container image - #42

Merged
s0nea merged 3 commits into
squid-19.2.7-cobaltcore-storagefrom
backport/625
Oct 24, 2026
Merged

s0nea merged 3 commits into
squid-19.2.7-cobaltcore-storagefrom
backport/625

Conversation

@s0nea

@s0nea s0nea commented Oct 24, 2026

Copy link
Copy Markdown

Closes cobaltcore-dev/cloud-storage#625

Backport file

  • release-management/backports/<uuid>.md added (or updated) by this PR
  • id in frontmatter matches the filename stem
  • provenance.upstream_prs set (or provenance.type: other with an other: block)

Stage-B risk (filled per release-management/backports/RISK-RUBRIC.md)

  • blast — cosmetic / availability / data-loss
  • conflict — clean / trivial / substantive
  • coverage — strong / partial / weak
  • Risk-notes paragraph in the prose section (required for high band)

Merge

Land this via just merge-backport PR=<N> — do not click the green merge button.
The just recipe constructs the merge commit's Backport-Id trailer.

s0nea and others added 3 commits October 24, 2026 18:09
…#39)"

This reverts commit 6a1ca82, reversing changes made to f973198.

The workaround handed nvmeof gateways a key of the older aes type,
because every squid-line gateway image bundled a librados from before
aes256k and could not authenticate with the aes256k keys cephadm
creates since 19.2.6.

While the workaround was being worked on, ceph-nvmeof published 1.4.29,
the first squid-line image built on a ceph that knows aes256k (19.2.7):

  https://github.com/ceph/ceph-nvmeof/releases/tag/1.4.29
  ceph/ceph-nvmeof#2230

The gateways can use the key cephadm already gives them, so revert the
workaround and move the default image to 1.4.29 instead.

Ref: https://tracker.ceph.com/issues/81778

Signed-off-by: Tatjana Dehler <tatjana.dehler@clyso.com>
On-behalf-of: SAP <t.dehler@sap.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
quay.io/ceph/nvmeof:1.2.5 bundles a librados built from ceph 19.0.0,
which predates the aes256k cephx key type.  Since cephadm creates
aes256k daemon keys (19.2.6 on), that gateway cannot connect at all:

    ERROR state.py:357: Unable to create OMAP, exiting!
    rados.IOError: [errno 5] RADOS I/O error (error connecting to the
    cluster)

nvmeof:1.4.29 is the first image of the squid line built on a ceph that
knows aes256k (19.2.7), so it can use the key cephadm gives it.

Bumped in all three places that name the image: the mgr module, the
cephadm binary constants (still on 1.2.1) and the example listing in
the monitoring docs.

Also update the nvmeof conf template. Say why bdevs_per_cluster has to
stay. Under nvmeof 1.2.5 this was an ordinary tunable. 1.4.x picks its
cluster allocator from whichever of bdevs_per_cluster,
flat_bdevs_per_cluster or cluster_connections is set. In that order.
The template emits only the first. Removing it as redundant would leave
the gateway without an allocator, and it exits at startup with
"Unknown cluster allocator in the config". Add a jinja comment, so the
rendered ceph-nvmeof.conf is unchanged.

Ref: https://tracker.ceph.com/issues/81778

Signed-off-by: Tatjana Dehler <tatjana.dehler@clyso.com>
On-behalf-of: SAP <t.dehler@sap.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Tatjana Dehler <tatjana.dehler@clyso.com>
On-behalf-of: SAP <t.dehler@sap.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@s0nea
s0nea merged commit 68da157 into squid-19.2.7-cobaltcore-storage Oct 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant