From 11a190192a786667d5ed8ce49a9c8c6519a606eb Mon Sep 17 00:00:00 2001 From: Ben Date: Thu, 15 Oct 2026 13:07:07 +0300 Subject: [PATCH] feat(kubescape): add vulnerability drill-down tools and bound CVE output An agent could not answer "what are the worst vulnerabilities in my cluster?" without either being lied to or blowing its context window. The only route to any severity information was list_vulnerabilities, which returned 200,793 B (~50k tokens) for a single image, and nothing aggregated across manifests at all, so cluster-scope questions had no answer at any price. The Kubescape storage API already exposes a full aggregation ladder that this provider never used: cluster -> namespace -> workload -> manifest -> CVE, where every level except the last is a handful of integers plus a reference. Two new tools walk it: kubescape_vulnerability_overview Severity totals per namespace from the server-side aggregates, worst first. One call, ~1 KB, from vulnerabilitysummaries (cluster-scoped, one object per namespace). kubescape_list_vulnerable_workloads Workloads ranked by severity, each row carrying the manifest_name that kubescape_list_vulnerabilities takes, read from the summary's spec.vulnerabilitiesRef. One call, ~1.9 KB per workload. Both take ListOptions.ResourceVersion = "fullSpec". That sentinel is load-bearing and not obvious. The storage server strips spec from EVERY list response by default, so severity counters come back zero and vulnerabilitiesRef comes back empty -- not an error, and indistinguishable from a clean cluster. Measured on storage v0.0.298, nginx:1.14.0 has 76 critical CVEs and a default LIST of its summary reports 0. This is the same failure shape as the vulnerability_count bug, one level up, so listing these resources without the sentinel would report a vulnerable cluster as clean. Verified against a live cluster that the typed client transmits it: a wire capture shows resourceVersion=fullSpec on the request, and the returned counts match the values an individual GET reports. Because that correctness rests on a vendor sentinel, the overview also refuses to present an all-zero cluster as good news: it says the result could not be confirmed and points at a per-manifest check. A wrong "no vulnerabilities" is the most expensive answer this provider can give. kubescape_list_vulnerabilities is reshaped to fit the ladder: - severity_summary always describes the WHOLE manifest, even when the array is filtered or truncated, so the aggregate survives a bounded response. - The array is capped by limit (default 20), ordered worst severity first with id as a tiebreak for stable output, and reports total_count, returned_count and truncated so the agent knows it holds partial data. - Records carry only id, severity and fix_state. Measured field shares of the old payload: description 56.1%, data_source 19.5%, fix_versions 7.9%. The description was truncated at 200 chars mid-word, so it was lossy prose that invited reasoning from a fragment; the full text, data source and fix versions are all in get_vulnerability_details. - New severity and fixable_only filters for drill-down. - Grype emits one match per affected package, so the same CVE recurs. With the package fields trimmed away those rows are byte-identical, and duplicates would consume the limit budget: the first 20 records for nginx:1.14.0 held CVE-2017-12424 and CVE-2017-15670 twice each. Matches are now collapsed by CVE id with an affected_artifacts count, and a CVE left unfixed in any package is never reported as fixed. 466 matches become 293 distinct CVEs. Measured against the same image on a live cluster: 3,013 B, against 200,793 B before -- 67x smaller, with a complete and correct severity summary. Also fixes a miscount in that summary: severityCounts had no Negligible bucket, so the 102 Negligible CVEs in nginx:1.14.0 were reported as "Unknown": 102. Severity handling is now driven by one ordered list used for both bucketing and ranking, so a severity cannot be dropped into the wrong bucket again. Relevancy is surfaced where the API provides it -- "relevant" counts CVEs whose code node-agent observed loaded at runtime, e.g. 76 critical of which 30 are reachable. It is emitted only when non-zero: the field is `json:"relevant,omitempty"` upstream, so a zero is indistinguishable from "relevancy was never computed", and node-agent needs a learning period before it reports anything. Rendering absent as 0 would claim nothing is reachable when nobody has looked yet, and the overview's description says the absence is ambiguous in both directions. A relevancy-availability verdict derived from the kubescape.io/status annotation was considered and ruled out: workloads with status=ready were measured with 'relevant' absent. One value from the summaries cannot be trusted. vulnerabilitiesRef reports the WORKLOAD's namespace, but the manifests live in the Kubescape namespace, so following it verbatim yields NotFound -- confirmed on a live cluster, where a GET of docker.io-library-nginx-1.14.0-e34030 in the referenced namespace fails while the same GET in `kubescape` succeeds. Only the name is taken from the ref; the namespace comes from the operator namespace, overridable with kubescape_namespace. Handing the agent a pointer that 404s would defeat the point of the chain. Tests are fake-client based and need no cluster. Note that client-go's fake drops ResourceVersion from recorded actions, so a reactor cannot observe the sentinel; the tests wrap the typed client to record the ListOptions each handler actually passes, which pins the invariant at the real call site. Every response is a typed Out struct, so the SDK derives an output schema for the two new tools. `limit` is a *int on both inputs so that an absent limit (default 20) stays distinguishable from an explicit 0 (no limit). The e2e coverage sweep now drives both new tools. Docs-exempt: no existing doc describes the vulnerability tool surface Signed-off-by: Ben Hirschberg Signed-off-by: Ben --- pkg/kubescape/kubescape.go | 545 ++++++++++++++++++++++++++++--- pkg/kubescape/kubescape_test.go | 548 ++++++++++++++++++++++++++++++++ test/e2e/coverage_test.go | 2 + 3 files changed, 1057 insertions(+), 38 deletions(-) diff --git a/pkg/kubescape/kubescape.go b/pkg/kubescape/kubescape.go index dcadfee..006cec2 100644 --- a/pkg/kubescape/kubescape.go +++ b/pkg/kubescape/kubescape.go @@ -4,6 +4,7 @@ import ( "context" "encoding/json" "fmt" + "sort" "strings" "time" @@ -35,8 +36,70 @@ const ( // Pod labels operatorPodLabel = "app.kubernetes.io/name=kubescape-operator" storagePodLabel = "app.kubernetes.io/name=storage" + + // defaultVulnerabilityLimit bounds kubescape_list_vulnerabilities. A single + // image measured 466 CVEs / 200 KB unbounded, which no agent can consume; + // the severity summary carries the aggregate answer instead. + defaultVulnerabilityLimit = 20 + + // defaultWorkloadLimit bounds kubescape_list_vulnerable_workloads. Each + // workload summary renders small, but a large cluster has many of them. + defaultWorkloadLimit = 20 + + // fixStateFixed is the Grype fix state meaning an upgrade is available. + fixStateFixed = "fixed" + + severityUnknown = "Unknown" ) +// severityOrder lists Grype severities worst first. It is both the ranking used +// to sort results and the set of buckets the severity summary reports, so a +// severity can never be silently dropped into the wrong bucket. +var severityOrder = []string{"Critical", "High", "Medium", "Low", "Negligible", severityUnknown} + +// normaliseSeverity maps a Grype severity onto a known bucket, case-insensitively. +// Anything unrecognised becomes Unknown -- but Negligible is a real severity and +// must not land there: 102 Negligible CVEs were previously counted as Unknown. +func normaliseSeverity(s string) string { + for _, known := range severityOrder { + if strings.EqualFold(s, known) { + return known + } + } + return severityUnknown +} + +func isKnownSeverity(s string) bool { + for _, known := range severityOrder { + if strings.EqualFold(s, known) { + return true + } + } + return false +} + +// severityRank returns the sort position of a severity, worst first. +func severityRank(s string) int { + for i, known := range severityOrder { + if s == known { + return i + } + } + return len(severityOrder) +} + +// limitOrDefault resolves an optional limit argument: absent means def, and +// zero or a negative value means no limit. +func limitOrDefault(limit *int, def int) int { + if limit == nil { + return def + } + if *limit < 0 { + return 0 + } + return *limit +} + // kubescapeErrResult adapts ToolError to an MCP error result. func kubescapeErrResult(toolErr *errors.ToolError) *sdkmcp.CallToolResult { return toolErr.ToMCPResult() @@ -133,9 +196,22 @@ type listVulnerabilityManifestsInput struct { Level string `json:"level" jsonschema:"Type of manifests to list: 'image', 'workload', or 'both' (default: both)"` } +type vulnerabilityOverviewInput struct { + Namespace string `json:"namespace" jsonschema:"Restrict to one namespace (optional; omit for the whole cluster)"` +} + +type listVulnerableWorkloadsInput struct { + Namespace string `json:"namespace" jsonschema:"Restrict to one namespace (optional, defaults to all namespaces)"` + Limit *int `json:"limit" jsonschema:"Maximum workloads to return (default: 20). Use 0 for no limit."` + KubescapeNamespace string `json:"kubescape_namespace" jsonschema:"Namespace the Kubescape operator is installed in, where vulnerability manifests are stored (default: kubescape)"` +} + type listVulnerabilitiesInManifestInput struct { Namespace string `json:"namespace" jsonschema:"Namespace of the manifest (default: kubescape)"` ManifestName string `json:"manifest_name" jsonschema:"Name of the vulnerability manifest"` + Severity string `json:"severity" jsonschema:"Only return CVEs of this severity: 'Critical', 'High', 'Medium', 'Low', 'Negligible' or 'Unknown'. severity_summary still covers the whole manifest."` + FixableOnly bool `json:"fixable_only" jsonschema:"Only return CVEs that have a fix available (default: false)"` + Limit *int `json:"limit" jsonschema:"Maximum CVEs to return (default: 20). Use 0 for no limit -- a single image can hold hundreds of CVEs and overflow the context window."` } type getVulnerabilityDetailsInput struct { @@ -192,30 +268,170 @@ type listVulnerabilityManifestsOutput struct { } type severitySummary struct { - Critical int `json:"Critical"` - High int `json:"High"` - Medium int `json:"Medium"` - Low int `json:"Low"` - Unknown int `json:"Unknown"` + Critical int `json:"Critical"` + High int `json:"High"` + Medium int `json:"Medium"` + Low int `json:"Low"` + Negligible int `json:"Negligible"` + Unknown int `json:"Unknown"` +} + +// bucket returns the counter for a severity already passed through +// normaliseSeverity. Every entry of severityOrder has its own bucket. +func (s *severitySummary) bucket(severity string) *int { + switch severity { + case "Critical": + return &s.Critical + case "High": + return &s.High + case "Medium": + return &s.Medium + case "Low": + return &s.Low + case "Negligible": + return &s.Negligible + default: + return &s.Unknown + } } +// vulnerabilitySummary carries only the fields an agent needs to decide what to +// look at next. description (56% of the old payload) was truncated mid-word and +// is available in full from kubescape_get_vulnerability_details, which is also +// where data_source and fix_versions live. type vulnerabilitySummary struct { - ID string `json:"id"` - Severity string `json:"severity"` - Description string `json:"description"` - DataSource string `json:"data_source"` - FixState string `json:"fix_state,omitempty"` - FixVersions []string `json:"fix_versions,omitempty"` + ID string `json:"id"` + Severity string `json:"severity"` + FixState string `json:"fix_state"` + AffectedArtifacts int `json:"affected_artifacts"` +} + +type vulnerabilityFilters struct { + Severity string `json:"severity"` + FixableOnly bool `json:"fixable_only"` } type listVulnerabilitiesInManifestOutput struct { ManifestName string `json:"manifest_name"` Namespace string `json:"namespace"` - TotalCount int `json:"total_count"` SeveritySummary severitySummary `json:"severity_summary"` + TotalCount int `json:"total_count"` + ReturnedCount int `json:"returned_count"` + Truncated bool `json:"truncated"` + Filters vulnerabilityFilters `json:"filters"` Vulnerabilities []vulnerabilitySummary `json:"vulnerabilities"` } +// severityCount is one severity's count from a server-side summary: every CVE, +// plus those whose code node-agent observed loaded at runtime. +// +// `relevant` is `json:"relevant,omitempty"` upstream, so a zero is +// indistinguishable from "relevancy was never computed" -- node-agent needs a +// learning period before it reports anything. Emitting a zero there would tell +// an agent that no vulnerability is runtime-reachable when the truth may be that +// nobody has looked yet, so the key is omitted unless it holds a real value and +// the tool description spells out that its absence is ambiguous. +// +// There is deliberately no derived "relevancy: available|unavailable" field. +// The obvious signal for one does not work: workloads carrying +// kubescape.io/status=ready were measured with `relevant` absent, so the +// annotation says nothing about whether relevancy was computed. Reporting a +// confident availability verdict from it would be a guess dressed as a fact. +type severityCount struct { + All int64 `json:"all"` + Relevant int64 `json:"relevant,omitempty"` +} + +// severityCounts renders a server-side SeveritySummary for output. +type severityCounts struct { + Critical severityCount `json:"Critical"` + High severityCount `json:"High"` + Medium severityCount `json:"Medium"` + Low severityCount `json:"Low"` + Negligible severityCount `json:"Negligible"` + Unknown severityCount `json:"Unknown"` +} + +func newSeverityCounts(s v1beta1.SeveritySummary) severityCounts { + c := func(v v1beta1.VulnerabilityCounters) severityCount { + return severityCount{All: v.All, Relevant: v.Relevant} + } + return severityCounts{ + Critical: c(s.Critical), + High: c(s.High), + Medium: c(s.Medium), + Low: c(s.Low), + Negligible: c(s.Negligible), + Unknown: c(s.Unknown), + } +} + +func (c *severityCounts) buckets() []*severityCount { + return []*severityCount{&c.Critical, &c.High, &c.Medium, &c.Low, &c.Negligible, &c.Unknown} +} + +// add accumulates other into c. A relevant count that is absent everywhere +// stays absent in the total. +func (c *severityCounts) add(other severityCounts) { + src := other.buckets() + for i, dst := range c.buckets() { + dst.All += src[i].All + dst.Relevant += src[i].Relevant + } +} + +func (c severityCounts) total() int64 { + var t int64 + for _, b := range c.buckets() { + t += b.All + } + return t +} + +// worseThan orders by critical count, then by total, worst first. +func (c severityCounts) worseThan(other severityCounts) bool { + if c.Critical.All != other.Critical.All { + return c.Critical.All > other.Critical.All + } + return c.total() > other.total() +} + +type namespaceVulnerabilities struct { + Namespace string `json:"namespace"` + WorkloadCount int `json:"workload_count"` + Severities severityCounts `json:"severities"` +} + +type vulnerabilityOverviewOutput struct { + Scope string `json:"scope"` + ClusterTotals severityCounts `json:"cluster_totals"` + Namespaces []namespaceVulnerabilities `json:"namespaces"` + NextStep string `json:"next_step"` + Warning string `json:"warning,omitempty"` +} + +type vulnerableWorkload struct { + Namespace string `json:"namespace"` + WorkloadSummary string `json:"workload_summary"` + Workload string `json:"workload,omitempty"` + Container string `json:"container,omitempty"` + Image string `json:"image,omitempty"` + Severities severityCounts `json:"severities"` + ScanStatus string `json:"scan_status"` + ManifestName string `json:"manifest_name,omitempty"` + ManifestNamespace string `json:"manifest_namespace,omitempty"` + WorkloadManifestName string `json:"workload_manifest_name,omitempty"` +} + +type listVulnerableWorkloadsOutput struct { + Namespace string `json:"namespace"` + TotalWorkloads int `json:"total_workloads"` + Returned int `json:"returned"` + Truncated bool `json:"truncated"` + Workloads []vulnerableWorkload `json:"workloads"` + NextStep string `json:"next_step"` +} + type configurationScanSummary struct { Namespace string `json:"namespace"` ManifestName string `json:"manifest_name"` @@ -658,6 +874,182 @@ func (k *KubescapeTool) handleCheckHealth(ctx context.Context, request *sdkmcp.C return mcp.NewToolResultText(string(content)), result, nil } +// storageFullSpec is the sentinel the Kubescape storage API server requires in +// ListOptions.ResourceVersion to return object specs on LIST. +// +// By default the server strips spec from EVERY list response, so severity +// counters come back as zero and vulnerabilitiesRef comes back empty. That is +// not an error and is indistinguishable from a healthy cluster with no +// vulnerabilities -- listing the summary resources without this sentinel +// reports a vulnerable cluster as clean. Measured on storage v0.0.298: +// nginx:1.14.0 has 76 critical CVEs and a default LIST reports 0. +// +// Defined as ResourceVersionFullSpec in +// github.com/kubescape/storage/pkg/apis/softwarecomposition/register.go. +const storageFullSpec = "fullSpec" + +// fullSpecList returns the ListOptions required to read specs from the storage +// API. Use it for every list of an aggregate resource. +func fullSpecList() metav1.ListOptions { + return metav1.ListOptions{ResourceVersion: storageFullSpec} +} + +// handleVulnerabilityOverview reports cluster or namespace vulnerability posture +// from the server-side aggregated summaries. +func (k *KubescapeTool) handleVulnerabilityOverview(ctx context.Context, request *sdkmcp.CallToolRequest, in vulnerabilityOverviewInput) (*sdkmcp.CallToolResult, vulnerabilityOverviewOutput, error) { + if k.initError != nil { + toolErr := errors.NewKubescapeError("vulnerability_overview", k.initError) + return kubescapeErrResult(toolErr), vulnerabilityOverviewOutput{}, nil + } + + namespace := in.Namespace + + // VulnerabilitySummary is cluster-scoped and each object's NAME is a + // namespace, so listing returns one aggregate per namespace. + summaries, err := k.spdxClient.VulnerabilitySummaries(metav1.NamespaceAll).List(ctx, fullSpecList()) + if err != nil { + toolErr := errors.NewKubescapeError("vulnerability_overview", err). + WithContext("namespace", namespace) + return kubescapeErrResult(toolErr), vulnerabilityOverviewOutput{}, nil + } + + clusterTotals := severityCounts{} + namespaces := []namespaceVulnerabilities{} + for _, summary := range summaries.Items { + if namespace != "" && summary.Name != namespace { + continue + } + counts := newSeverityCounts(summary.Spec.Severities) + clusterTotals.add(counts) + namespaces = append(namespaces, namespaceVulnerabilities{ + Namespace: summary.Name, + WorkloadCount: len(summary.Spec.WorkloadVulnerabilitiesObj), + Severities: counts, + }) + } + + // Worst namespace first, so the agent's next call is obvious. + sort.SliceStable(namespaces, func(i, j int) bool { + return namespaces[i].Severities.worseThan(namespaces[j].Severities) + }) + + scope := "cluster" + if namespace != "" { + scope = "namespace" + } + + result := vulnerabilityOverviewOutput{ + Scope: scope, + ClusterTotals: clusterTotals, + Namespaces: namespaces, + NextStep: "call kubescape_list_vulnerable_workloads with a namespace to rank its workloads", + } + + // A cluster with summaries but no counts anywhere is far more likely to mean + // the spec was stripped than that every image is clean. Never present that + // as good news. + if len(namespaces) > 0 && clusterTotals.total() == 0 { + result.Warning = "Every namespace reported zero vulnerabilities, which could not be confirmed as a genuinely clean cluster: " + + "the Kubescape storage API returns zeroed counts when it strips object specs from list responses. " + + "Verify with kubescape_list_vulnerabilities on a specific manifest before concluding there are no vulnerabilities." + } + + content, err := json.MarshalIndent(result, "", " ") + if err != nil { + return mcp.NewToolResultError(fmt.Sprintf("failed to marshal result: %v", err)), vulnerabilityOverviewOutput{}, nil + } + return mcp.NewToolResultText(string(content)), result, nil +} + +// handleListVulnerableWorkloads ranks workloads by vulnerability severity and +// hands back the manifest name needed to drill into each one. +func (k *KubescapeTool) handleListVulnerableWorkloads(ctx context.Context, request *sdkmcp.CallToolRequest, in listVulnerableWorkloadsInput) (*sdkmcp.CallToolResult, listVulnerableWorkloadsOutput, error) { + if k.initError != nil { + toolErr := errors.NewKubescapeError("list_vulnerable_workloads", k.initError) + return kubescapeErrResult(toolErr), listVulnerableWorkloadsOutput{}, nil + } + + namespace := in.Namespace + // Where the VulnerabilityManifest objects live, which is not what the + // summaries' vulnerabilitiesRef reports -- see the comment at its use below. + manifestNamespace := in.KubescapeNamespace + if manifestNamespace == "" { + manifestNamespace = defaultKubescapeNamespace + } + limit := limitOrDefault(in.Limit, defaultWorkloadLimit) + + queryNamespace := metav1.NamespaceAll + if namespace != "" { + queryNamespace = namespace + } + + summaries, err := k.spdxClient.VulnerabilityManifestSummaries(queryNamespace).List(ctx, fullSpecList()) + if err != nil { + toolErr := errors.NewKubescapeError("list_vulnerable_workloads", err). + WithContext("namespace", namespace) + return kubescapeErrResult(toolErr), listVulnerableWorkloadsOutput{}, nil + } + + workloads := []vulnerableWorkload{} + for _, summary := range summaries.Items { + entry := vulnerableWorkload{ + Namespace: summary.Namespace, + WorkloadSummary: summary.Name, + Severities: newSeverityCounts(summary.Spec.Severities), + ScanStatus: summary.Annotations["kubescape.io/status"], + Container: summary.Labels["kubescape.io/workload-container-name"], + Image: summary.Annotations["kubescape.io/image-tag"], + } + if kind, name := summary.Labels["kubescape.io/workload-kind"], summary.Labels["kubescape.io/workload-name"]; kind != "" && name != "" { + entry.Workload = kind + "/" + name + } + // vulnerabilitiesRef points straight at the manifests holding the CVEs: + // `all` is the image-level manifest, `relevant` the workload-filtered + // one. These names are exactly what kubescape_list_vulnerabilities takes. + // + // Only the NAME from the ref is usable. The server fills the ref's + // namespace with the workload's namespace, but the manifests live in the + // Kubescape namespace -- verified on a live cluster, where a GET of + // docker.io-library-nginx-1.14.0-e34030 in the referenced namespace + // returns NotFound while the same GET in `kubescape` succeeds. Passing + // the ref's namespace on would hand the agent a pointer that 404s. + if ref := summary.Spec.Vulnerabilities.ImageVulnerabilitiesObj; ref.Name != "" { + entry.ManifestName = ref.Name + entry.ManifestNamespace = manifestNamespace + } + if ref := summary.Spec.Vulnerabilities.WorkloadVulnerabilitiesObj; ref.Name != "" { + entry.WorkloadManifestName = ref.Name + } + workloads = append(workloads, entry) + } + + sort.SliceStable(workloads, func(i, j int) bool { + return workloads[i].Severities.worseThan(workloads[j].Severities) + }) + + total := len(workloads) + truncated := false + if limit > 0 && total > limit { + workloads = workloads[:limit] + truncated = true + } + + result := listVulnerableWorkloadsOutput{ + Namespace: namespace, + TotalWorkloads: total, + Returned: len(workloads), + Truncated: truncated, + Workloads: workloads, + NextStep: "call kubescape_list_vulnerabilities with a workload's manifest_name to see its CVEs", + } + + content, err := json.MarshalIndent(result, "", " ") + if err != nil { + return mcp.NewToolResultError(fmt.Sprintf("failed to marshal result: %v", err)), listVulnerableWorkloadsOutput{}, nil + } + return mcp.NewToolResultText(string(content)), result, nil +} + // handleListVulnerabilityManifests lists vulnerability manifests at image and workload levels func (k *KubescapeTool) handleListVulnerabilityManifests(ctx context.Context, request *sdkmcp.CallToolRequest, in listVulnerabilityManifestsInput) (*sdkmcp.CallToolResult, listVulnerabilityManifestsOutput, error) { if k.initError != nil { @@ -752,6 +1144,18 @@ func (k *KubescapeTool) handleListVulnerabilitiesInManifest(ctx context.Context, return mcp.NewToolResultError("manifest_name parameter is required"), listVulnerabilitiesInManifestOutput{}, nil } + severityFilter := in.Severity + if severityFilter != "" { + if !isKnownSeverity(severityFilter) { + toolErr := errors.NewKubescapeError("list_vulnerabilities", + fmt.Errorf("invalid severity %q: must be one of %v", severityFilter, severityOrder)) + return kubescapeErrResult(toolErr), listVulnerabilitiesInManifestOutput{}, nil + } + severityFilter = normaliseSeverity(severityFilter) + } + fixableOnly := in.FixableOnly + limit := limitOrDefault(in.Limit, defaultVulnerabilityLimit) + manifest, err := k.spdxClient.VulnerabilityManifests(namespace).Get(ctx, manifestName, metav1.GetOptions{}) if err != nil { toolErr := errors.NewKubescapeError("get_vulnerability_manifest", err). @@ -760,46 +1164,81 @@ func (k *KubescapeTool) handleListVulnerabilitiesInManifest(ctx context.Context, return kubescapeErrResult(toolErr), listVulnerabilitiesInManifestOutput{}, nil } - // Extract vulnerabilities with summary info - vulnerabilities := []vulnerabilitySummary{} + // Count every match by severity first. The summary describes the WHOLE + // manifest even when the returned array is filtered or truncated, so a + // bounded response still carries the aggregate answer. severityCounts := severitySummary{} + // Grype reports one Match per affected package, so the same CVE recurs. + // Collapse them by ID -- without the package fields the rows would be + // indistinguishable and duplicates would consume the limit budget -- while + // keeping the number of affected artifacts visible. + matched := []*vulnerabilitySummary{} + byID := map[string]*vulnerabilitySummary{} for _, match := range manifest.Spec.Payload.Matches { vuln := match.Vulnerability - severity := string(vuln.Severity) - switch severity { - case "Critical": - severityCounts.Critical++ - case "High": - severityCounts.High++ - case "Medium": - severityCounts.Medium++ - case "Low": - severityCounts.Low++ - default: - severityCounts.Unknown++ + severity := normaliseSeverity(string(vuln.Severity)) + *severityCounts.bucket(severity)++ + + if severityFilter != "" && severity != severityFilter { + continue + } + if fixableOnly && vuln.Fix.State != fixStateFixed { + continue } - vulnInfo := vulnerabilitySummary{ - ID: vuln.ID, - Severity: severity, - Description: truncateString(vuln.Description, 200), - DataSource: vuln.DataSource, + if existing, seen := byID[vuln.ID]; seen { + existing.AffectedArtifacts++ + // A CVE unfixed in any package is not actionable as "fixed". + if vuln.Fix.State != fixStateFixed { + existing.FixState = vuln.Fix.State + } + continue } - if vuln.Fix.State != "" { - vulnInfo.FixState = vuln.Fix.State - vulnInfo.FixVersions = vuln.Fix.Versions + entry := &vulnerabilitySummary{ + ID: vuln.ID, + Severity: severity, + FixState: vuln.Fix.State, + AffectedArtifacts: 1, } + byID[vuln.ID] = entry + matched = append(matched, entry) + } - vulnerabilities = append(vulnerabilities, vulnInfo) + // Worst first, so a truncated array is the half that matters. Ties break on + // id to keep responses stable between calls. + sort.SliceStable(matched, func(i, j int) bool { + ri, rj := severityRank(matched[i].Severity), severityRank(matched[j].Severity) + if ri != rj { + return ri < rj + } + return matched[i].ID < matched[j].ID + }) + + totalMatching := len(matched) + truncated := false + if limit > 0 && totalMatching > limit { + matched = matched[:limit] + truncated = true + } + + vulnerabilities := make([]vulnerabilitySummary, 0, len(matched)) + for _, v := range matched { + vulnerabilities = append(vulnerabilities, *v) } result := listVulnerabilitiesInManifestOutput{ ManifestName: manifestName, Namespace: namespace, - TotalCount: len(vulnerabilities), SeveritySummary: severityCounts, + TotalCount: totalMatching, + ReturnedCount: len(vulnerabilities), + Truncated: truncated, + Filters: vulnerabilityFilters{ + Severity: severityFilter, + FixableOnly: fixableOnly, + }, Vulnerabilities: vulnerabilities, } @@ -1266,6 +1705,24 @@ func RegisterTools(s *sdkmcp.Server, kubeconfig string, readOnly bool) { Description: "Check if Kubescape operator is installed and operational. Verifies namespace, operator pods, storage pods, CRDs, and scan data availability.", }, tool.handleCheckHealth) + // Cluster / namespace vulnerability posture -- the cheapest entry point + mcp.AddTool(s, "kubescape", &sdkmcp.Tool{ + Name: "kubescape_vulnerability_overview", + Description: "START HERE for any question about cluster-wide or namespace-wide vulnerabilities. " + + "Returns severity totals per namespace from Kubescape's server-side aggregates in a single cheap call, worst namespace first. " + + "Counts are 'all' plus 'relevant' (the vulnerable code was observed loaded at runtime). " + + "'relevant' is reported only when greater than zero; when it is absent that means EITHER no runtime-relevant CVEs OR that relevancy " + + "has not been computed for those workloads yet, and the two cannot be distinguished here -- so do not report an absent 'relevant' as a measured zero. " + + "Then narrow with kubescape_list_vulnerable_workloads.", + }, tool.handleVulnerabilityOverview) + + // Rank workloads and hand back the manifest to drill into + mcp.AddTool(s, "kubescape", &sdkmcp.Tool{ + Name: "kubescape_list_vulnerable_workloads", + Description: "Rank workloads by vulnerability severity, worst first, and return the 'manifest_name' needed to inspect each one's CVEs. " + + "Use after kubescape_vulnerability_overview to find which workloads matter, then pass a returned manifest_name to kubescape_list_vulnerabilities.", + }, tool.handleListVulnerableWorkloads) + mcp.AddTool(s, "kubescape", &sdkmcp.Tool{ Name: "kubescape_list_vulnerability_manifests", Description: "List vulnerability manifests from Kubescape operator, at image or workload level. " + @@ -1276,8 +1733,10 @@ func RegisterTools(s *sdkmcp.Server, kubeconfig string, readOnly bool) { }, tool.handleListVulnerabilityManifests) mcp.AddTool(s, "kubescape", &sdkmcp.Tool{ - Name: "kubescape_list_vulnerabilities", - Description: "List all CVEs/vulnerabilities found in a specific vulnerability manifest. Returns severity summary and vulnerability details.", + Name: "kubescape_list_vulnerabilities", + Description: "List CVEs in a specific vulnerability manifest. Always returns severity_summary, which counts EVERY CVE in the manifest. " + + "The 'vulnerabilities' array is capped at 'limit' (default 20), ordered worst severity first; check 'truncated' and 'total_count' to see whether more exist. " + + "Each entry carries only id, severity and fix_state -- use kubescape_get_vulnerability_details for the description, data source and fix versions of one CVE.", }, tool.handleListVulnerabilitiesInManifest) mcp.AddTool(s, "kubescape", &sdkmcp.Tool{ @@ -1331,6 +1790,8 @@ func RegisterTools(s *sdkmcp.Server, kubeconfig string, readOnly bool) { // Interfaces for testing - allows mocking the Kubernetes clients type KubescapeToolInterface interface { HandleCheckHealth(ctx context.Context, in checkHealthInput) (*sdkmcp.CallToolResult, HealthCheckResult, error) + HandleVulnerabilityOverview(ctx context.Context, in vulnerabilityOverviewInput) (*sdkmcp.CallToolResult, vulnerabilityOverviewOutput, error) + HandleListVulnerableWorkloads(ctx context.Context, in listVulnerableWorkloadsInput) (*sdkmcp.CallToolResult, listVulnerableWorkloadsOutput, error) HandleListVulnerabilityManifests(ctx context.Context, in listVulnerabilityManifestsInput) (*sdkmcp.CallToolResult, listVulnerabilityManifestsOutput, error) HandleListVulnerabilitiesInManifest(ctx context.Context, in listVulnerabilitiesInManifestInput) (*sdkmcp.CallToolResult, listVulnerabilitiesInManifestOutput, error) HandleGetVulnerabilityDetails(ctx context.Context, in getVulnerabilityDetailsInput) (*sdkmcp.CallToolResult, []v1beta1.Match, error) @@ -1350,6 +1811,14 @@ func (k *KubescapeTool) HandleCheckHealth(ctx context.Context, in checkHealthInp return k.handleCheckHealth(ctx, &sdkmcp.CallToolRequest{}, in) } +func (k *KubescapeTool) HandleVulnerabilityOverview(ctx context.Context, in vulnerabilityOverviewInput) (*sdkmcp.CallToolResult, vulnerabilityOverviewOutput, error) { + return k.handleVulnerabilityOverview(ctx, &sdkmcp.CallToolRequest{}, in) +} + +func (k *KubescapeTool) HandleListVulnerableWorkloads(ctx context.Context, in listVulnerableWorkloadsInput) (*sdkmcp.CallToolResult, listVulnerableWorkloadsOutput, error) { + return k.handleListVulnerableWorkloads(ctx, &sdkmcp.CallToolRequest{}, in) +} + func (k *KubescapeTool) HandleListVulnerabilityManifests(ctx context.Context, in listVulnerabilityManifestsInput) (*sdkmcp.CallToolResult, listVulnerabilityManifestsOutput, error) { return k.handleListVulnerabilityManifests(ctx, &sdkmcp.CallToolRequest{}, in) } diff --git a/pkg/kubescape/kubescape_test.go b/pkg/kubescape/kubescape_test.go index fd9a672..1e87bb2 100644 --- a/pkg/kubescape/kubescape_test.go +++ b/pkg/kubescape/kubescape_test.go @@ -4,10 +4,12 @@ import ( "context" "encoding/json" "errors" + "fmt" "testing" "github.com/kubescape/storage/pkg/apis/softwarecomposition/v1beta1" kubescapefake "github.com/kubescape/storage/pkg/generated/clientset/versioned/fake" + spdxv1beta1 "github.com/kubescape/storage/pkg/generated/clientset/versioned/typed/softwarecomposition/v1beta1" sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -1180,3 +1182,549 @@ func TestHandleListVulnerabilityManifests_OmitsVulnerabilityCount(t *testing.T) // The useful metadata is still there. assert.Contains(t, getResultText(result), "nginx:1.14.0") } + +// --------------------------------------------------------------------------- +// T3: list_vulnerabilities -- bounded output, truthful summary (design 01b) +// --------------------------------------------------------------------------- + +// manifestWithMatches builds a manifest holding n CVEs of the given severity, +// alternating fix state so fixable filtering can be exercised. +func manifestWithMatches(name string, perSeverity map[string]int) *v1beta1.VulnerabilityManifest { + matches := []v1beta1.Match{} + i := 0 + for severity, n := range perSeverity { + for j := 0; j < n; j++ { + fixState := "not-fixed" + if i%2 == 0 { + fixState = "fixed" + } + matches = append(matches, v1beta1.Match{ + Vulnerability: v1beta1.Vulnerability{ + VulnerabilityMetadata: v1beta1.VulnerabilityMetadata{ + ID: fmt.Sprintf("CVE-2024-%s-%04d", severity, j), + Severity: severity, + Description: "a description long enough to matter for payload size, repeated over and over", + DataSource: "https://security-tracker.debian.org/tracker/CVE-2024-0000", + }, + Fix: v1beta1.Fix{State: fixState, Versions: []string{"1.2.3"}}, + }, + }) + i++ + } + } + return &v1beta1.VulnerabilityManifest{ + ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "kubescape"}, + Spec: v1beta1.VulnerabilityManifestSpec{Payload: v1beta1.GrypeDocument{Matches: matches}}, + } +} + +type vulnListResponse struct { + ManifestName string `json:"manifest_name"` + SeveritySummary map[string]int `json:"severity_summary"` + TotalCount int `json:"total_count"` + ReturnedCount int `json:"returned_count"` + Truncated bool `json:"truncated"` + Vulnerabilities []map[string]interface{} `json:"vulnerabilities"` +} + +func intPtr(i int) *int { return &i } + +func listVulns(t *testing.T, manifest *v1beta1.VulnerabilityManifest, in listVulnerabilitiesInManifestInput) vulnListResponse { + t.Helper() + spdxClient := kubescapefake.NewClientset(manifest) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + in.ManifestName = manifest.Name + + result, _, err := tool.HandleListVulnerabilitiesInManifest(context.Background(), in) + require.NoError(t, err) + require.False(t, result.IsError, getResultText(result)) + + var resp vulnListResponse + require.NoError(t, json.Unmarshal([]byte(getResultText(result)), &resp)) + return resp +} + +// The whole point of P4: a manifest with hundreds of CVEs must not return +// hundreds of records. nginx:1.14.0 measured 200,793 B / 466 matches. +func TestHandleListVulnerabilities_BoundsOutputByDefault(t *testing.T) { + m := manifestWithMatches("big", map[string]int{ + "Critical": 76, "High": 133, "Medium": 99, "Low": 56, "Negligible": 102, + }) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + + assert.Equal(t, 466, resp.TotalCount, "total_count must report every CVE, not just the returned ones") + assert.Equal(t, 20, resp.ReturnedCount, "default limit should bound the array") + assert.Len(t, resp.Vulnerabilities, 20) + assert.True(t, resp.Truncated, "the agent must be told it received partial data") +} + +// The severity summary is the cheap answer and must describe the WHOLE manifest +// even when the array is truncated. +func TestHandleListVulnerabilities_SummaryCoversAllMatchesNotJustReturned(t *testing.T) { + m := manifestWithMatches("big", map[string]int{ + "Critical": 76, "High": 133, "Medium": 99, "Low": 56, "Negligible": 102, + }) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + + assert.Equal(t, 76, resp.SeveritySummary["Critical"]) + assert.Equal(t, 133, resp.SeveritySummary["High"]) + assert.Equal(t, 99, resp.SeveritySummary["Medium"]) + assert.Equal(t, 56, resp.SeveritySummary["Low"]) +} + +// Measured live: 102 Negligible CVEs were reported as "Unknown": 102 because +// severityCounts had no Negligible bucket. +func TestHandleListVulnerabilities_CountsNegligibleNotUnknown(t *testing.T) { + m := manifestWithMatches("negl", map[string]int{"Negligible": 102}) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + + assert.Equal(t, 102, resp.SeveritySummary["Negligible"], "Negligible must have its own bucket") + assert.Equal(t, 0, resp.SeveritySummary["Unknown"], "Negligible must not be miscounted as Unknown") +} + +// A genuinely unrecognised severity still lands in Unknown. +func TestHandleListVulnerabilities_UnrecognisedSeverityCountsAsUnknown(t *testing.T) { + m := manifestWithMatches("weird", map[string]int{"Bogus": 3}) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + + assert.Equal(t, 3, resp.SeveritySummary["Unknown"]) +} + +// Per-record fields: description (56.1% of the payload) and data_source (19.5%) +// are dropped; description is truncated mid-word anyway and full text lives in +// get_vulnerability_details. +func TestHandleListVulnerabilities_RecordCarriesOnlyIdSeverityFixState(t *testing.T) { + m := manifestWithMatches("fields", map[string]int{"Critical": 1}) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + require.Len(t, resp.Vulnerabilities, 1) + + entry := resp.Vulnerabilities[0] + assert.ElementsMatch(t, []string{"id", "severity", "fix_state", "affected_artifacts"}, keysOf(entry)) +} + +func keysOf(m map[string]interface{}) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + return out +} + +// Worst-first, so a truncated array is the part that matters. +func TestHandleListVulnerabilities_SortsBySeverityDescending(t *testing.T) { + m := manifestWithMatches("order", map[string]int{ + "Negligible": 5, "Critical": 2, "Medium": 3, "High": 4, "Low": 1, + }) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{Limit: intPtr(6)}) + + got := []string{} + for _, v := range resp.Vulnerabilities { + got = append(got, v["severity"].(string)) + } + assert.Equal(t, []string{"Critical", "Critical", "High", "High", "High", "High"}, got) +} + +func TestHandleListVulnerabilities_FiltersBySeverity(t *testing.T) { + m := manifestWithMatches("filter", map[string]int{"Critical": 3, "Low": 7}) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{Severity: "Critical"}) + + assert.Equal(t, 3, resp.TotalCount, "total_count reflects the filter") + assert.Len(t, resp.Vulnerabilities, 3) + for _, v := range resp.Vulnerabilities { + assert.Equal(t, "Critical", v["severity"]) + } + // The summary still describes the whole manifest, so the agent keeps context. + assert.Equal(t, 7, resp.SeveritySummary["Low"]) +} + +func TestHandleListVulnerabilities_FiltersFixableOnly(t *testing.T) { + m := manifestWithMatches("fixable", map[string]int{"Critical": 10}) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{FixableOnly: true}) + + assert.NotZero(t, len(resp.Vulnerabilities)) + for _, v := range resp.Vulnerabilities { + assert.Equal(t, "fixed", v["fix_state"]) + } + assert.Less(t, resp.TotalCount, 10, "fixable_only must actually exclude the unfixed ones") +} + +// An explicit limit above the match count must not claim truncation. +func TestHandleListVulnerabilities_NotTruncatedWhenLimitExceedsMatches(t *testing.T) { + m := manifestWithMatches("small", map[string]int{"High": 3}) + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{Limit: intPtr(100)}) + + assert.Equal(t, 3, resp.TotalCount) + assert.Equal(t, 3, resp.ReturnedCount) + assert.False(t, resp.Truncated) +} + +func TestHandleListVulnerabilities_RejectsInvalidSeverity(t *testing.T) { + m := manifestWithMatches("bad", map[string]int{"High": 1}) + spdxClient := kubescapefake.NewClientset(m) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + result, _, err := tool.HandleListVulnerabilitiesInManifest(context.Background(), listVulnerabilitiesInManifestInput{ + ManifestName: "bad", + Severity: "VeryBad", + }) + require.NoError(t, err) + assert.True(t, result.IsError) + assert.Contains(t, getResultText(result), "severity") +} + +// --------------------------------------------------------------------------- +// T1/T2: overview and workload ranking (design 01b) +// +// These read the aggregate resources through the storage API. That API strips +// spec from every LIST unless ResourceVersion is the "fullSpec" sentinel, so a +// default LIST returns all-zero counts -- the P0 failure shape at cluster +// scale. The fake clientset ignores ResourceVersion and returns whatever was +// seeded, so it CANNOT reproduce the stripping. The invariant is therefore +// pinned by asserting the outgoing ListOptions, not by observing the response. +// --------------------------------------------------------------------------- + +func counters(all, relevant int64, withRelevant bool) v1beta1.VulnerabilityCounters { + c := v1beta1.VulnerabilityCounters{All: all} + if withRelevant { + c.Relevant = relevant + } + return c +} + +func nsSummary(namespace string, crit, high int64, refs ...string) *v1beta1.VulnerabilitySummary { + objRefs := []v1beta1.VulnerabilitiesObjScope{} + for _, r := range refs { + objRefs = append(objRefs, v1beta1.VulnerabilitiesObjScope{ + Namespace: namespace, Name: r, Kind: "vulnerabilitymanifestsummary", + }) + } + return &v1beta1.VulnerabilitySummary{ + ObjectMeta: metav1.ObjectMeta{Name: namespace}, + Spec: v1beta1.VulnerabilitySummarySpec{ + Severities: v1beta1.SeveritySummary{ + Critical: counters(crit, crit/2, true), + High: counters(high, 0, false), + }, + WorkloadVulnerabilitiesObj: objRefs, + }, + } +} + +func workloadSummary(namespace, name, imageTag, manifestName string, crit, high int64) *v1beta1.VulnerabilityManifestSummary { + return &v1beta1.VulnerabilityManifestSummary{ + ObjectMeta: metav1.ObjectMeta{ + Name: name, + Namespace: namespace, + Annotations: map[string]string{ + "kubescape.io/image-tag": imageTag, + "kubescape.io/status": "ready", + }, + Labels: map[string]string{ + "kubescape.io/workload-kind": "deployment", + "kubescape.io/workload-name": "app", + "kubescape.io/workload-container-name": "main", + }, + }, + Spec: v1beta1.VulnerabilityManifestSummarySpec{ + Severities: v1beta1.SeveritySummary{ + Critical: counters(crit, crit/2, true), + High: counters(high, 0, false), + }, + Vulnerabilities: v1beta1.VulnerabilitiesComponents{ + ImageVulnerabilitiesObj: v1beta1.VulnerabilitiesObjScope{ + // The server reports the WORKLOAD's namespace here, but the + // manifest actually lives in the Kubescape namespace, so + // this value is unusable. Verified on a live cluster: a GET + // in this namespace returns NotFound. + Namespace: namespace, Name: manifestName, Kind: "vulnerabilitymanifests", + }, + }, + }, + } +} + +// The fake clientset's recorded actions drop ResourceVersion entirely, so a +// reactor cannot see it. These thin decorators wrap the typed client and record +// the ListOptions the handler actually passes, which is the thing under test. + +type recordingSpdx struct { + spdxv1beta1.SpdxV1beta1Interface + recorded *[]metav1.ListOptions +} + +func (r recordingSpdx) VulnerabilitySummaries(ns string) spdxv1beta1.VulnerabilitySummaryInterface { + return recordingVulnSummaries{r.SpdxV1beta1Interface.VulnerabilitySummaries(ns), r.recorded} +} + +func (r recordingSpdx) VulnerabilityManifestSummaries(ns string) spdxv1beta1.VulnerabilityManifestSummaryInterface { + return recordingManifestSummaries{r.SpdxV1beta1Interface.VulnerabilityManifestSummaries(ns), r.recorded} +} + +type recordingVulnSummaries struct { + spdxv1beta1.VulnerabilitySummaryInterface + recorded *[]metav1.ListOptions +} + +func (r recordingVulnSummaries) List(ctx context.Context, opts metav1.ListOptions) (*v1beta1.VulnerabilitySummaryList, error) { + *r.recorded = append(*r.recorded, opts) + return r.VulnerabilitySummaryInterface.List(ctx, opts) +} + +type recordingManifestSummaries struct { + spdxv1beta1.VulnerabilityManifestSummaryInterface + recorded *[]metav1.ListOptions +} + +func (r recordingManifestSummaries) List(ctx context.Context, opts metav1.ListOptions) (*v1beta1.VulnerabilityManifestSummaryList, error) { + *r.recorded = append(*r.recorded, opts) + return r.VulnerabilityManifestSummaryInterface.List(ctx, opts) +} + +func recordingClient(c *kubescapefake.Clientset, sink *[]metav1.ListOptions) spdxv1beta1.SpdxV1beta1Interface { + return recordingSpdx{c.SpdxV1beta1(), sink} +} + +// THE LINCHPIN TEST. Without ResourceVersion="fullSpec" the storage server +// returns spec-stripped objects and every count reads zero -- an agent would be +// told a vulnerable cluster is clean. A fake cannot show that, so assert the +// request instead. +func TestHandleVulnerabilityOverview_RequestsFullSpec(t *testing.T) { + spdxClient := kubescapefake.NewClientset(nsSummary("verify-targets", 99, 242, "deployment-vuln-nginx-nginx")) + var seen []metav1.ListOptions + + tool := NewKubescapeToolWithClients(nil, nil, recordingClient(spdxClient, &seen)) + _, _, err := tool.HandleVulnerabilityOverview(context.Background(), vulnerabilityOverviewInput{}) + require.NoError(t, err) + + require.Len(t, seen, 1) + assert.Equal(t, storageFullSpec, seen[0].ResourceVersion, + "must request fullSpec: a default LIST returns all-zero counts and would report a vulnerable cluster as clean") +} + +func TestHandleListVulnerableWorkloads_RequestsFullSpec(t *testing.T) { + spdxClient := kubescapefake.NewClientset( + workloadSummary("verify-targets", "deployment-vuln-nginx-nginx", + "docker.io/library/nginx:1.14.0", "docker.io-library-nginx-1.14.0-e34030", 76, 133)) + var seen []metav1.ListOptions + + tool := NewKubescapeToolWithClients(nil, nil, recordingClient(spdxClient, &seen)) + _, _, err := tool.HandleListVulnerableWorkloads(context.Background(), listVulnerableWorkloadsInput{}) + require.NoError(t, err) + + require.Len(t, seen, 1) + assert.Equal(t, storageFullSpec, seen[0].ResourceVersion) +} + +func TestHandleVulnerabilityOverview_AggregatesNamespaces(t *testing.T) { + spdxClient := kubescapefake.NewClientset( + nsSummary("verify-targets", 99, 242, "a", "b"), + nsSummary("other", 1, 2, "c"), + ) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + result, _, err := tool.HandleVulnerabilityOverview(context.Background(), vulnerabilityOverviewInput{}) + require.NoError(t, err) + require.False(t, result.IsError, getResultText(result)) + + var resp struct { + Scope string `json:"scope"` + ClusterTotals map[string]map[string]interface{} `json:"cluster_totals"` + Namespaces []struct { + Namespace string `json:"namespace"` + WorkloadCount int `json:"workload_count"` + } `json:"namespaces"` + } + require.NoError(t, json.Unmarshal([]byte(getResultText(result)), &resp)) + + assert.Equal(t, "cluster", resp.Scope) + assert.Equal(t, float64(100), resp.ClusterTotals["Critical"]["all"], "cluster total sums namespaces") + require.Len(t, resp.Namespaces, 2) + assert.Equal(t, "verify-targets", resp.Namespaces[0].Namespace, "worst namespace first") + assert.Equal(t, 2, resp.Namespaces[0].WorkloadCount) +} + +// relevant is `json:"relevant,omitempty"`, so a zero is indistinguishable from +// "relevancy not computed". Never render it as a measured zero. +func TestHandleVulnerabilityOverview_OmitsRelevantWhenAbsent(t *testing.T) { + spdxClient := kubescapefake.NewClientset(nsSummary("verify-targets", 99, 242, "a")) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + result, _, err := tool.HandleVulnerabilityOverview(context.Background(), vulnerabilityOverviewInput{}) + require.NoError(t, err) + + var resp struct { + ClusterTotals map[string]map[string]interface{} `json:"cluster_totals"` + } + require.NoError(t, json.Unmarshal([]byte(getResultText(result)), &resp)) + + // High was built with no relevant value at all. + _, present := resp.ClusterTotals["High"]["relevant"] + assert.False(t, present, "an absent relevant count must not be reported as 0") + // Critical had one, so it survives. + assert.Equal(t, float64(49), resp.ClusterTotals["Critical"]["relevant"]) +} + +func TestHandleListVulnerableWorkloads_CarriesManifestPointer(t *testing.T) { + spdxClient := kubescapefake.NewClientset( + workloadSummary("verify-targets", "deployment-vuln-nginx-nginx", + "docker.io/library/nginx:1.14.0", "docker.io-library-nginx-1.14.0-e34030", 76, 133)) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + result, _, err := tool.HandleListVulnerableWorkloads(context.Background(), listVulnerableWorkloadsInput{}) + require.NoError(t, err) + require.False(t, result.IsError, getResultText(result)) + + var resp struct { + Workloads []map[string]interface{} `json:"workloads"` + } + require.NoError(t, json.Unmarshal([]byte(getResultText(result)), &resp)) + require.Len(t, resp.Workloads, 1) + + w := resp.Workloads[0] + // The whole point of the ladder: the next call is handed over, never guessed. + assert.Equal(t, "docker.io-library-nginx-1.14.0-e34030", w["manifest_name"]) + assert.Equal(t, "docker.io/library/nginx:1.14.0", w["image"]) + // The summary was built with the workload's namespace in the ref, which is + // what the real server sends and where the manifest is NOT. Reporting it + // verbatim hands the agent a pointer that 404s. + assert.Equal(t, "kubescape", w["manifest_namespace"], + "manifest_namespace must be where manifests actually live, not the unusable value in vulnerabilitiesRef") +} + +func TestHandleListVulnerableWorkloads_SortsAndLimits(t *testing.T) { + spdxClient := kubescapefake.NewClientset( + workloadSummary("ns", "low-risk", "img:a", "manifest-a", 1, 0), + workloadSummary("ns", "high-risk", "img:b", "manifest-b", 76, 0), + workloadSummary("ns", "mid-risk", "img:c", "manifest-c", 12, 0), + ) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + result, _, err := tool.HandleListVulnerableWorkloads(context.Background(), listVulnerableWorkloadsInput{Limit: intPtr(2)}) + require.NoError(t, err) + + var resp struct { + TotalWorkloads int `json:"total_workloads"` + Returned int `json:"returned"` + Truncated bool `json:"truncated"` + Workloads []map[string]interface{} `json:"workloads"` + } + require.NoError(t, json.Unmarshal([]byte(getResultText(result)), &resp)) + + assert.Equal(t, 3, resp.TotalWorkloads) + assert.Equal(t, 2, resp.Returned) + assert.True(t, resp.Truncated) + require.Len(t, resp.Workloads, 2) + assert.Equal(t, "high-risk", resp.Workloads[0]["workload_summary"]) + assert.Equal(t, "mid-risk", resp.Workloads[1]["workload_summary"]) +} + +// If every namespace reports zero, we cannot tell "clean cluster" from +// "fullSpec stopped working". Say so rather than reporting good news. +func TestHandleVulnerabilityOverview_FlagsAllZeroAsUnverified(t *testing.T) { + spdxClient := kubescapefake.NewClientset(nsSummary("verify-targets", 0, 0, "a")) + tool := NewKubescapeToolWithClients(nil, nil, spdxClient.SpdxV1beta1()) + + result, _, err := tool.HandleVulnerabilityOverview(context.Background(), vulnerabilityOverviewInput{}) + require.NoError(t, err) + + assert.Contains(t, getResultText(result), "could not be confirmed", + "an all-zero result must be reported as unconfirmed, never as a clean cluster") +} + +// Grype emits one Match per affected package, so the same CVE ID recurs. Once +// the package fields are trimmed away those rows are byte-identical and would +// burn the limit budget on duplicates -- measured live, the first 20 records for +// nginx:1.14.0 contained CVE-2017-12424 and CVE-2017-15670 twice each. +func TestHandleListVulnerabilities_DeduplicatesByCVE(t *testing.T) { + dup := func(id, severity, fixState string) v1beta1.Match { + return v1beta1.Match{Vulnerability: v1beta1.Vulnerability{ + VulnerabilityMetadata: v1beta1.VulnerabilityMetadata{ID: id, Severity: severity}, + Fix: v1beta1.Fix{State: fixState}, + }} + } + m := &v1beta1.VulnerabilityManifest{ + ObjectMeta: metav1.ObjectMeta{Name: "dupes", Namespace: "kubescape"}, + Spec: v1beta1.VulnerabilityManifestSpec{Payload: v1beta1.GrypeDocument{Matches: []v1beta1.Match{ + dup("CVE-2017-12424", "Critical", "fixed"), + dup("CVE-2017-12424", "Critical", "fixed"), + dup("CVE-2017-12424", "Critical", "fixed"), + dup("CVE-2020-0001", "High", "not-fixed"), + }}}, + } + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + + // The summary counts matches, matching the numbers Kubescape itself reports. + assert.Equal(t, 3, resp.SeveritySummary["Critical"]) + assert.Equal(t, 1, resp.SeveritySummary["High"]) + + // The array carries distinct CVEs. + require.Len(t, resp.Vulnerabilities, 2) + assert.Equal(t, 2, resp.TotalCount, "total_count counts distinct CVEs in the array") + + assert.Equal(t, "CVE-2017-12424", resp.Vulnerabilities[0]["id"]) + assert.Equal(t, float64(3), resp.Vulnerabilities[0]["affected_artifacts"], + "the collapsed matches must still be visible as a count") + assert.Equal(t, float64(1), resp.Vulnerabilities[1]["affected_artifacts"]) +} + +// A CVE fixed in one package but not another is only actionable if the list says +// so, and "not-fixed" is the safer thing to surface. +func TestHandleListVulnerabilities_DedupeKeepsWorstFixState(t *testing.T) { + mk := func(fixState string) v1beta1.Match { + return v1beta1.Match{Vulnerability: v1beta1.Vulnerability{ + VulnerabilityMetadata: v1beta1.VulnerabilityMetadata{ID: "CVE-2021-1", Severity: "High"}, + Fix: v1beta1.Fix{State: fixState}, + }} + } + m := &v1beta1.VulnerabilityManifest{ + ObjectMeta: metav1.ObjectMeta{Name: "mixed", Namespace: "kubescape"}, + Spec: v1beta1.VulnerabilityManifestSpec{Payload: v1beta1.GrypeDocument{Matches: []v1beta1.Match{mk("fixed"), mk("not-fixed")}}}, + } + + resp := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + + require.Len(t, resp.Vulnerabilities, 1) + assert.Equal(t, "not-fixed", resp.Vulnerabilities[0]["fix_state"], + "a CVE unfixed in any package must not be reported as fixed") +} + +// The typed severity_summary replaces a map keyed by severityOrder. Every +// severity must still own a distinct bucket, or one would be silently merged +// into another -- the bug that counted Negligible as Unknown. +func TestSeveritySummary_EveryOrderedSeverityHasItsOwnBucket(t *testing.T) { + var s severitySummary + seen := map[*int]string{} + for _, severity := range severityOrder { + b := s.bucket(severity) + require.NotNil(t, b) + if other, dup := seen[b]; dup { + t.Fatalf("severities %q and %q share a bucket", other, severity) + } + seen[b] = severity + } +} + +// limit is optional: absent means the default, while an explicit 0 means no +// limit and must not be confused with "not supplied". +func TestHandleListVulnerabilities_LimitAbsentVersusZero(t *testing.T) { + m := manifestWithMatches("limits", map[string]int{"High": 30}) + + defaulted := listVulns(t, m, listVulnerabilitiesInManifestInput{}) + assert.Equal(t, defaultVulnerabilityLimit, defaulted.ReturnedCount) + assert.True(t, defaulted.Truncated) + + unlimited := listVulns(t, m, listVulnerabilitiesInManifestInput{Limit: intPtr(0)}) + assert.Equal(t, 30, unlimited.ReturnedCount) + assert.False(t, unlimited.Truncated) +} diff --git a/test/e2e/coverage_test.go b/test/e2e/coverage_test.go index 5d2bc4e..fd01ad9 100644 --- a/test/e2e/coverage_test.go +++ b/test/e2e/coverage_test.go @@ -261,6 +261,8 @@ func readOnlyToolCases() []toolCase { // kubescape (read-only) - the operator may be absent; a tool error is a // legitimate outcome and is recorded, not failed. {name: "kubescape_check_health"}, + {name: "kubescape_vulnerability_overview"}, + {name: "kubescape_list_vulnerable_workloads"}, {name: "kubescape_list_vulnerability_manifests"}, {name: "kubescape_list_configuration_scans"}, {name: "kubescape_list_application_profiles"},