Repository navigation
Backport/tentacle 20.2.4 cobaltcore storage/keystone iam policy - #40
Closed
supriti wants to merge 23 commits into
Conversation
Backport-Id: 01a11b0e-da4d-7890-bfeb-43841f3aab2d Backport-PR: #24 Upstream-PRs: ceph#67219, ceph#67548, ceph#66065
Add rate limiting specific to LIST ops, similar to the current rate-limiting (https://docs.ceph.com/en/latest/radosgw/admin/#rate-limit-management) Example usage: ``` ./bin/radosgw-admin ratelimit set --ratelimit-scope=user --uid=<UID> --max_list_ops=2 ./bin/radosgw-admin ratelimit set --ratelimit-scope=user --uid=<UID> --max_delete_ops=2 ./bin/radosgw-admin ratelimit enable --ratelimit-scope=user --uid=<UID> ./bin/radosgw-admin ratelimit get --ratelimit-scope=user --uid=<UID> { "user_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_list_ops": 2, "max_delete_ops": 2, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": true } } pkill -9 radosgw ./bin/radosgw -c ./ceph.conf ... aws --endpoint-url 'http://0:8000' s3 mb s3://bkt aws --endpoint-url 'http://0:8000' s3 cp ./ceph.conf s3://bkt aws --endpoint-url http://0:8000 s3api list-objects-v2 --bucket bkt --prefix 'ceph.conf' --delimiter '/' { "Contents": [ { "Key": "ceph.conf", "LastModified": "2025-07-30T13:59:38+00:00", "ETag": "\"13d11d431ae290134562c019d9e40c0e\"", "Size": 32346, "StorageClass": "STANDARD" } ], "RequestCharged": null } aws --endpoint-url http://0:8000 s3api list-objects-v2 --bucket bkt --prefix 'ceph.conf' --delimiter '/' { "Contents": [ { "Key": "ceph.conf", "LastModified": "2025-07-30T13:59:38+00:00", "ETag": "\"13d11d431ae290134562c019d9e40c0e\"", "Size": 32346, "StorageClass": "STANDARD" } ], "RequestCharged": null } aws --endpoint-url http://0:8000 s3api list-objects-v2 --bucket bkt --prefix 'ceph.conf' --delimiter '/' argument of type 'NoneType' is not iterable tail -F ./out/radosgw.8000.log | grep beast ... beast: 0x7fffbbe09780: [30/Jul/2025:15:44:50.359 +0000] " GET /bkt?list-type=2&delimiter=%2F&prefix=ceph.conf&encoding-type=url HTTP/1.1" 200 535 - "aws-cli/2.15.31 Python/3.9.21 Linux/5.14.0-570.28.1.el9_6.x86_64 source/x86_64.rhel.9 prompt/off command/s3api.list-objects-v2" - latency=0.000999995s beast: 0x7fffbbe09780: [30/Jul/2025:15:44:53.904 +0000] " GET /bkt?list-type=2&delimiter=%2F&prefix=ceph.conf&encoding-type=url HTTP/1.1" 200 535 - "aws-cli/2.15.31 Python/3.9.21 Linux/5.14.0-570.28.1.el9_6.x86_64 source/x86_64.rhel.9 prompt/off command/s3api.list-objects-v2" - latency=0.000999995s vvv beast: 0x7fffbbe09780: [30/Jul/2025:15:44:58.192 +0000] " GET /bkt?list-type=2&delimiter=%2F&prefix=ceph.conf&encoding-type=url HTTP/1.1" 503 228 - "aws-cli/2.15.31 Python/3.9.21 Linux/5.14.0-570.28.1.el9_6.x86_64 source/x86_64.rhel.9 prompt/off command/s3api.list-objects-v2" - latency=0.000000000s beast: 0x7fffbbe09780: [30/Jul/2025:15:44:58.798 +0000] " GET /bkt?list-type=2&delimiter=%2F&prefix=ceph.conf&encoding-type=url HTTP/1.1" 503 228 - "aws-cli/2.15.31 Python/3.9.21 Linux/5.14.0-570.28.1.el9_6.x86_64 source/x86_64.rhel.9 prompt/off command/s3api.list-objects-v2" - latency=0.000999994s beast: 0x7fffbbe09780: [30/Jul/2025:15:44:59.807 +0000] " GET /bkt?list-type=2&delimiter=%2F&prefix=ceph.conf&encoding-type=url HTTP/1.1" 503 228 - "aws-cli/2.15.31 Python/3.9.21 Linux/5.14.0-570.28.1.el9_6.x86_64 source/x86_64.rhel.9 prompt/off command/s3api.list-objects-v2" - latency=0.000000000s s3cmd put ./ceph.conf s3://bkt/1 s3cmd put ./ceph.conf s3://bkt/2 s3cmd put ./ceph.conf s3://bkt/3 s3cmd rm s3://bkt/1 s3cmd rm s3://bkt/2 s3cmd rm s3://bkt/3 delete: 's3://bkt/1' delete: 's3://bkt/2' WARNING: Retrying failed request: /3 (503 (SlowDown)) WARNING: Waiting 3 sec... WARNING: Retrying failed request: /3 (503 (SlowDown)) ^^^ ``` Fixes: https://tracker.ceph.com/issues/72894 Signed-off-by: Mark Kogan <mkogan@ibm.com> Update PendingReleaseNotes Co-authored-by: Yuval Lifshitz <yuvalif@yahoo.com> Signed-off-by: Mark Kogan <31659604+mkogan1@users.noreply.github.com> Update PendingReleaseNotes Co-authored-by: Yuval Lifshitz <yuvalif@yahoo.com> Signed-off-by: Mark Kogan <31659604+mkogan1@users.noreply.github.com> (cherry picked from commit dae572d) Conflicts: src/common/options/rgw.yaml.in - Remove “rgw_redis_connection_pool_size” because tentacle doesn’t have 1ac08d3 src/rgw/rgw_rest_ratelimit.cc - Removed "rgw_sal.h”, “rgw_sal_config.h” and imports because tentacle doesn’t have de90732 and 09383ab
…set to 0 Signed-off-by: morphes1995 <morphes1995@gmail.com> (cherry picked from commit 91e9f00)
When RGW rejects an S3 request with -ERR_RATE_LIMITED, return an HTTP Retry-After header that tells the client how many seconds to wait before retrying. The value is computed from the current token-bucket deficit and replenishment rate, so a polite client that honors the header will retry right around when its tokens recover. The delay is threaded out of the rate limiter through req_state and onto the response in abort_early(). The rate-limit check now returns int64_t seconds (0 = allowed) instead of bool, with a 1-second floor on the rejected path. The seconds-to-recover calculation is factored into a small static helper RateLimiterEntry::compute_delay(limit, needed, interval) so it can be unit-tested directly. Each dimension (ops, bytes) calls it independently and the caller takes the max, leaving room for additional limit dimensions in the future. The user-facing documentation is updated to describe the new header and to suggest proxy-layer remapping for operators who need 429 instead of the existing 503. Tracker: https://tracker.ceph.com/issues/64083 Tracker: https://tracker.ceph.com/issues/67655 Signed-off-by: Lumir Sliva <61183145+lumir-sliva@users.noreply.github.com> (cherry picked from commit 3dae903)
Signed-off-by: Tatjana Dehler <tatjana.dehler@clyso.com>
nodeenv 1.11.0 (released 2026-09-26) no longer leaves the node-env/src directory behind after installing a prebuilt node. The build step runs an unpinned `pip install nodeenv`, so as of today every build with WITH_MGR_DASHBOARD_FRONTEND fails at chown: cannot access '.../node-env/src': No such file or directory ninja: build stopped: subcommand failed. (e.g. ceph-pr-pipeline build 2504). The chown is a workaround for tarball uid mapping in containers and is only needed when the src directory exists, so skip it when there is nothing to fix up. Assisted-by: Claude Fable 5 Signed-off-by: David Galloway <david.galloway@ibm.com> (cherry picked from commit 5aff221) Conflicts: src/pybind/mgr/dashboard/frontend/CMakeLists.txt squid still creates the npm cache directory with "mkdir -p" and has no DEPENDS on setup-virtualenv.sh; kept those lines as they are and took only the guarded chown and its comment
…change (#25) Backport-Id: 01a11a76-84c1-7371-840f-b82c5fbf1a72 Backport-PR: #25 Upstream-PRs: ceph#64762, ceph#66509, ceph#68210
3e885ab ("mon/MonClient: add assertions for monc lock in MonConnection") added a `ceph::mutex& monc_lock` reference member to MonConnection, which makes the explicitly defaulted move assignment operator implicitly deleted. clang (-Wdefaulted-function-deleted, promoted by -Werror) rejects that, so every make check / API build on main fails since that merge: src/mon/MonClient.h:70:18: error: explicitly defaulted move assignment operator is implicitly deleted [-Werror,-Wdefaulted-function-deleted] Nothing move-assigns a MonConnection (pending_cons/active_con only ever move-construct), so spell out what the compiler already decided. Fixes: 3e885ab Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit 79296fc) (cherry picked from commit 3c86b2d)
The cephx upgrade suites added with the CVE-2025-30156 merge are missing the .qa links in X/ms_bind, X/tasks/07-client-auth-keys, X/tasks/08-allowed-ciphers and the release-X counterparts, so the "Check for missing .qa links" workflow (src/script/verify-qa) fails on every PR based on current main. Add the links verify-qa asks for. Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit 62c4d50) (cherry picked from commit 89e6482)
fae0d0d ("mgr/cephadm: rotate keyring for core ceph daemons during upgrade") made _upgrade_daemons() call _rotate_mgr_mon_auth_keys() on every pass. That function indexes mon_daemons[0] unconditionally, and with an empty mon list _detect_need_upgrade() reports no mons needing upgrade, which is taken as "all mons upgraded" -- so it goes on to set auth_allowed_ciphers and rotate mgr keys before crashing with IndexError: list index out of range cephadm/upgrade.py:1795 Since that merge four cephadm unit tests fail deterministically in run_tox_mgr on every PR (test_upgrade_daemons_offline_hosts, test_upgrade_run[False/True], test_do_upgrade_limit_exhausted_marks_complete_without_scope_check); they exercise the upgrade machinery without any mon daemons in the cache. Without any known mon daemons we cannot tell whether the mons have been upgraded and there is no mon keyring to rotate, so skip the rotation for this pass and let the next upgrade pass retry once the cache has mons. Fixes: fae0d0d Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit fa100f0) (cherry picked from commit d742bab)
0753b1d ("common/entity_name: dump type name as string") turned STR_TO_ENTITY_TYPE into a vector of (entity_type_t, std::string) pairs but left get_valid_types_as_str() streaming .first, i.e. the uint8_t type value, so e.g. `ceph-conf --name total.garbage` now says valid types are: , \x01, \x04, \x02, \x10, \x08 instead of "auth, mon, osd, mds, mgr, client", and the src/test/cli/ceph-conf/invalid-args.t cram test fails in run-cli-tests. Stream the name (.second). Fixes: 0753b1d Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit 18e32fa) (cherry picked from commit 36982bb)
The CVE-2025-30156 cephx merge (9633561) added a -t/--key-type option to ceph-authtool, --auth-service-cipher/--auth-allowed-ciphers/ --auth-preferred-cipher options and auth_epoch/auth_*_cipher fields to monmaptool, and the cephx_auth_aes256k mon feature bit, without updating the cram expectations under src/test/cli, so run-cli-tests fails on every PR. Update the ten affected .t files with the output the tools now produce. Fixes: 9633561 Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) Conflicts: src/test/cli/monmaptool/feature-set-unset-list.t The mon feature lists in the expected output are release-specific, so the expectations were rebuilt from this branch's src/mon/mon_types.h (inserting cephx_auth_aes256k(2147483648) before nvmeof_beacon_diff(4294967296) in tentacle's supported/persistent feature sets) instead of taking main's lists. (cherry picked from commit a935139) (cherry picked from commit 7c4745b)
The cephx merge (9633561) left four unused local variables in qa/tasks/ceph.py (key_prune/key_rotate/task) and an f-string without placeholders plus an unused exception binding in qa/tasks/radosbench.py, so run-tox-qa (flake8 --select=F,E9) fails on every PR. No functional change. Fixes: 9633561 Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit 297c44b) (cherry picked from commit 4d90e67)
The OSD keyring rotation changes from the CVE-2025-30156 cephx merge (c9bc85a "cephadm: restart OSD systemd unit after key rotation", 24dc65e "cephadm: Add retries to OSD key rotation") added lines to cephadmlib/daemons/ceph.py that are not formatted the way `tox -e check-black` (black>=23,<24, -l78) expects, so run-tox-cephadm fails on every PR now that the py3 env passes again: would reformat /ceph/src/cephadm/cephadmlib/daemons/ceph.py 1 file would be reformatted, 50 files would be left unchanged. This is the output of `tox -e format-black` (black 23.12.1) on that file; no functional change. Fixes: 24dc65e Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit 6b206bc) (cherry picked from commit 7fb3c92)
TestOSDMap.cc uses stringify() but never includes include/stringify.h; until now it compiled via a transitive include that the CVE-2025-30156 cephx merge removed, so once the MonClient.h build break earlier in the compile is fixed, make check dies here instead: src/test/osd/TestOSDMap.cc: error: use of undeclared identifier 'stringify' main got the include from 0735848 ("test: add missing includes", part of a 22-commit include-hygiene PR that is not worth backporting wholesale), hence this branch-local commit rather than a cherry-pick. Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit 01a873f)
Retarget test-ceph-custom.yml from the squid branches to the tentacle ones: the pull_request trigger now matches 'tentacle-*-cobaltcore-storage', and the workflow_dispatch example points at a tentacle branch. Also remove any leftover ceph_build container before building. A job that dies mid-build, for example when the runner is lost, leaves its container behind. build-with-container.py always names the container ceph_build, so every later run fails right away with "container name ceph_build already in use". Running podman rm -f ceph_build first clears it, and the podman ps line logs what was found. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@clyso.com>
Signed-off-by: Uwe Schwaeke <uwe.schwaeke@clyso.com>
…rade_complete 5eaf4db ("mgr/cephadm: mute key rotation related health warnings during upgrade") brought a hunk of main's _mark_upgrade_complete() to tentacle that belongs to the PG autoscaler feature added on main by da4b361 ("mgr: Control PG autoscaler during upgrades with pg_autoscale_during_upgrade"), which was never backported. So tentacle calls self._unset_noautoscale() and assigns self.upgrade_state.noautoscale_set, neither of which exists here, and run-tox-mgr fails for every tentacle PR: cephadm/upgrade.py: error: "CephadmUpgrade" has no attribute "_unset_noautoscale" [attr-defined] cephadm/upgrade.py: error: "UpgradeState" has no attribute "noautoscale_set" [attr-defined] Nothing on tentacle ever sets noautoscale_set, so the getattr() guard is always False and the block is dead code; drop it rather than backporting the feature to a stable branch. Signed-off-by: David Galloway <david.galloway@ibm.com> Assisted-by: Claude Fable 5 (claude-fable-5) (cherry picked from commit a3f44c1)
…telimit tests The backport of "rgw: add rate limit for LIST & DELETE ops" (512e930, cherry picked from dae572d) carried the qa test hunk unchanged from main, where it still used the old rgwadmin_rest(admin_conn, ...) call form. Tentacle's rgwadmin_rest() takes (endpoint, creds, cmd, ...) and no admin_conn exists in task(), so flake8 reports F821 undefined name 'admin_conn' and run-tox-qa fails. The test cases would also raise NameError under teuthology. Adapt the calls to rgwadmin_rest(endpoint, admin_creds, ...), as main did in a7f557c ("qa/radosgw_admin_rest: replace boto2 with boto3"). The resulting ratelimit test block is identical to main's. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@clyso.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Backport-Id: 01a12f91-ee54-7edc-80b3-610fd82571ba Backport-PR: #26 Upstream-PRs: ceph#71165, ceph#71190
Keystone roles are injected into the IAM policy evaluation environment as "keystone:role" condition keys. This enables bucket policies to grant or deny access based on Keystone roles. It bridges the gap between Openstack RBAC and S3 IAM policy. Document keystone:role in bucket policy and Keystone S3 integration Signed-off-by: Supriti Singh <supriti.singh@clyso.com> Signed-off-by: Marcel Lauhoff <marcel.lauhoff@clyso.com> On-behalf-of: SAP <supriti.singh@clyso.com> On-behalf-of: SAP <marcel.lauhoff@sap.com> (cherry picked from commit 1e6e393)
Expose the Keystone token user UUID as condition key "keystone:userid" so IAM and bucket policies can restrict access by user (e.g. per-user buckets or user-specific allow/deny). On-behalf-of: SAP <supriti.singh@clyso.com> Signed-off-by: Supriti Singh <supriti.singh@clyso.com> (cherry picked from commit b06b7a8)
Signed-off-by: Supriti Singh <supriti.singh@clyso.com>
Author
|
oops wrong branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes cobaltcore-dev/cloud-storage#565
Closes cobaltcore-dev/cloud-storage#566
Backport file
release-management/backports/<uuid>.mdadded (or updated) by this PRidin frontmatter matches the filename stemprovenance.upstream_prsset (orprovenance.type: otherwith another:block)Stage-B risk (filled per
release-management/backports/RISK-RUBRIC.md)blast— cosmetic / availability / data-lossconflict— clean / trivial / substantivecoverage— strong / partial / weakhighband)Merge
Land this via
just merge-backport PR=<N>— do not click the green merge button.The just recipe constructs the merge commit's
Backport-Idtrailer.