fix: use DD_SERVICE for cold start spans instead of hardcoded aws.lambda - #855
Closed
joeyzhao2018 wants to merge 1 commit into
Closed
joeyzhao2018 wants to merge 1 commit into
joeyzhao2018 wants to merge 1 commit into
Conversation
Cold start spans (aws.lambda.load, aws.lambda.import) were always created with service="aws.lambda". Use DD_SERVICE when set, falling back to aws.lambda otherwise, so they land in the same service as the function span. This matches what the Lambda extension already does by rewriting aws.lambda spans to DD_SERVICE (datadog-lambda-extension#471), and follows the same change made for the function span in #635. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Contributor
❌ ErrorsYour PR has failed checks. Please review the issues below and take necessary action before merging. 🚦 4 Pipeline jobs failed
Useful? React with 👍 / 👎 This comment will be updated automatically if new data arrives.🔗 Commit SHA: 47d4b26 | Docs | View more details | Give us feedback! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Cold start spans (
aws.lambda.load,aws.lambda.import) were always created withservice="aws.lambda". They now useDD_SERVICEwhen set, and still fall back toaws.lambdawhen it isn't.Why this is safe
library span whose service is
aws.lambdatoDD_SERVICE(datadog-lambda-extension#471,bottlecap/src/traces/trace_processor.rs: "Service name could be incorrectly set to'aws.lambda' in datadog lambda libraries"). With extension +
DD_SERVICE, what reachesthe backend doesn't change. This only fixes the Forwarder / no-extension path. Anything
keyed on
service:aws.lambdafor these spans has already been broken for extensionusers for almost two years.
and datadog-lambda-js#666 ("Remove Service Override from
aws.lambdaspans", Jul 2025,prompted by Tier-1 escalation SLES-2371). The cold start spans were missed, and no
snapshot caught it because integration tests disable cold start tracing.
facet / Cold Start Tracing insight looks for
trace.aws.lambda.loadwith thefunction's
resource_name+service+env(web-uitransform-lambda-entity.ts).With
service:aws.lambdait never matches for non-extension users. Trace-viewhighlighting selects cold start spans by
operation_nameonly, so it's unaffected.aws.lambda.loadis still a top-level span, sotrace.aws.lambda.load.*metrics keepbeing generated, now under the customer's service.
service:aws.lambdafor these spans. I searched web-ui,dogweb, dd-go, public docs and OOTB dashboards. The docs never state the service name.
Our cold-start regression monitoring uses
aws.lambda.enhanced.init_duration, not spans.an API) whose AWS-resource service name is a real node on the service map. Cold start
spans are the function's own in-process init work.
aws.lambdaisn't an entity there,it's a service override (dd-trace-py tags these spans
_dd.base_service). SLES-2371was a customer escalation about exactly that kind of override.
What changes for customers
Only non-extension users with
DD_SERVICEset:aws.lambdatoDD_SERVICE, andaws.lambda.loadshowsup as an operation on their service.
aws.lambdaservice override goes away.service:aws.lambda+operation_name:aws.lambda.loadwould stop matching. We found no evidence of such usage.
Nothing changes when
DD_SERVICEis unset.Testing
DD_SERVICEset and unset.share the same service.
🤖 Generated with Claude Code