Skip to content

feat: Create WebhookInjectorCertRotationStuck alert - #39

Merged
defo89 merged 1 commit into
cloudoperators:mainfrom
yahor-kurachkin:add-webhook-injector-cert-rotation-stuck-alert
Oct 22, 2026
Merged

defo89 merged 1 commit into
cloudoperators:mainfrom
yahor-kurachkin:add-webhook-injector-cert-rotation-stuck-alert

Conversation

@yahor-kurachkin

@yahor-kurachkin yahor-kurachkin commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

feat: Create WebhookInjectorCertRotationStuck alert

Summary

Adds monitoring for webhook-injector certificate rotation failures in controlplane-operations.

The alert detects sustained reconciliation issues — excessive requeues or persistent errors — in the configmap controller running the webhook-injector container.

Alert Logic

Triggers when either condition persists over a 5-minute window:

  • Requeue rate > 0.05 req/s (configmap controller, webhook-injector)
  • Error rate > 0 (same controller/container)

Changes

  • charts/controlplane-operations/alerts/controlplane-remote.yaml — adds the WebhookInjectorCertRotationStuck alert rule with configurable thresholds, diagnostic annotations, and a playbook link

Notes

  • Alert uses result-specific label matchers — consider whether alternating failure modes could delay detection or produce duplicate firings

Copilot AI balanced review requested due to automatic review settings October 21, 2026 12:41
@yahor-kurachkin
yahor-kurachkin requested a review from a team as a code owner October 21, 2026 12:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Result-specific alert identities can delay detection when failure modes alternate and produce duplicate alerts when both persist.

1 open issue
What changed in this PR

Adds WebhookInjectorCertRotationStuck to monitor potential webhook-injector certificate rotation failures.

Changes:

  • Detects sustained excessive requeues or reconciliation errors.
  • Adds configurable alert settings, diagnostic annotations, and a playbook link.
File Description
charts/​controlplane-operations/​alerts/​controlplane-remote.yaml Adds the certificate rotation alert and its configuration.

Review effort: Balanced

Comment thread charts/controlplane-operations/alerts/controlplane-remote.yaml
Signed-off-by: C5421281 <yahor.kurachkin@sap.com>
@yahor-kurachkin
yahor-kurachkin force-pushed the add-webhook-injector-cert-rotation-stuck-alert branch from 9440611 to 1ba31ec Compare October 21, 2026 12:56
@defo89
defo89 merged commit 82588e6 into cloudoperators:main Oct 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants