Repository navigation
fix(rest): unbreak CSI clone-from-volume and make snapshot restore idempotent - #190
Open
Andrei Kvapil (kvaps) wants to merge 4 commits into
Open
Andrei Kvapil (kvaps) wants to merge 4 commits into
Andrei Kvapil (kvaps) wants to merge 4 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two CSI-facing defects reported from a Cozystack stand running the blockstor backend.
The internal clone-snapshot reap that used to ride here moved to #200, stacked on this one, and the test-harness fixes to #199. This one is the two fixes and what they strictly need: a refactor that splits store-error and body-decode answers from their envelope, and
delete_namespaceson resource-definition and group modify, which the clone path uses.Clone rejected every request golinstor sends
The endpoint declared five fields and decoded with
DisallowUnknownFields, solayer_list, which linstor-csi always sends, was a 400 before any of the handler ran, and every clone-from-volume failed.resource_groupwas the next 400 behind it. Both are now honoured on both clone paths, validated the wayrg modifyvalidates them, the stack stored in canonical case, and the group is checked to exist.external_nameandvolume_passphrasesare refused rather than accepted and dropped.A
layer_liston a source with volumes must match the source's stack as a set: the clone restores the source's bytes and then brings the stack up over them, so an added LUKS layer formats the restored data and a dropped one leaves the target reading bytes it cannot decode. Withoutlayer_list, the target is stamped with the source's data-plane stack, so the target group's stack cannot change what the control plane believes about it. The volume-less shell clone gets the same stamp.Refusals reach linstor-csi as the
[]ApiCallRcgolinstor decodes; python-linstor keeps the CloneStarted object it decodes. The shape is picked by User-Agent. golinstor turns any 404 into a bareNotFoundError, so a 404's cause never reaches the PVC's events.Restore through linstor-csi was refused on its first call
The actual cause of #186. linstor-csi's
VolFromSnapcreates the resource definition itself, restores the volume definitions, then callssnapshot-restore-resource. The definition it created carried no restore marker, so the restore took it for somebody else's and answered 409 on the first call, every time. That is word for word the error in the report.A prepared target is now accepted when its volumes are exactly the snapshot's, it has no live replica, its group still exists, and its layer set matches the source's. The marker is stamped only after all of that, so a refusal leaves an operator's definition untouched. The CLI follows the same LINSTOR sequence (
rd c,s vd restore,s rsc restore). The decisive reads go past the informer cache, since linstor-csi's calls can land on different apiserver replicas.Retries resume instead of failing for good
CSI requires
CreateVolumeto be idempotent. A leftover carrying this operation's marker is resumed, judged by one function both doors and the CLI share: tear-down first, then the parent group, then an abandoned-rollback mark, then the volumes and a live replica. A finished leftover is answered as a replay and is left alone. Only a replica with a disk counts: a diskless or tie-breaker one never makes a leftover finished. An operator's diskless replica on a node the request places onto is refused; the controller's tie-breaker witness there is promoted the way autoplace promotes one. A finished restore still answers as a replay after its snapshot is deleted, judged against the record of the volumes the snapshot held.Two requests on one target are handled explicitly. A request that resumes a leftover marks it adopted before writing anything, and the creator's rollback re-reads that mark and yields instead of deleting what the other request answered 201 for. Both sides fail closed: a rollback that cannot hold or read the handshake deletes nothing and says to retry. A failed placement that already left the definition finished, with every volume and one replica holding data, is kept rather than rolled back on both doors, since a replay may have answered for it; the error names the requested replicas that never landed.
The props blockstor writes on a definition (the restore marker, the record of the snapshot's volumes, the adoption and rollback marks) are refused in clone prop edits,
rd modifyand the CLI property verbs.Testing
Every behavioural change is pinned by a test checked by reverting the change and confirming the named test goes red, and a mutation sweep over every new error return leaves none uncovered.
docs/cli-parity-known-deltas.mdrows 86 and 87 describe the clone refusals and the retry semantics, including one accepted window: a replica an operator adds by hand between the prepared-target check and the marker counts as part of the restore.The placer flake is #198. Locally,
pkg/resthas aserver did not stop within 2sflake in the test harness; the harness PR fixes the related port race but not this one.