Emit OpenTelemetry semantic conventions on HTTP spans by zacharycmontoya · Pull Request #12765 · DataDog/dd-trace-java

zacharycmontoya · 2026-10-06T20:20:04Z

What Does This Do

This PR updates HTTP client and server spans produced by the shared automatic-instrumentation decorators so that their span names, attributes, and error handling follow the OpenTelemetry HTTP semantic conventions when DD_TRACE_OTEL_SEMANTICS_ENABLED=true.

Behavior remains unchanged when OpenTelemetry semantics are disabled.

The latest updates take a dependency on the latest tag registry work in #12731.

Implementation Details

Configuration

Enabling DD_TRACE_OTEL_SEMANTICS_ENABLED overrides the effective values of the following settings:

OTEL_TRACES_EXPORTER=otlp
DD_WRITER_TYPE=OtlpWriter
DD_TRACE_SPAN_ATTRIBUTE_SCHEMA=v0
DD_TRACE_PEER_SERVICE_DEFAULTS_ENABLED=false
DD_HTTP_CLIENT_ERROR_STATUSES=400-599 (in line with the OpenTelemetry specification)

HTTP client changes

When DD_TRACE_OTEL_SEMANTICS_ENABLED=true:

Span resource names become a low-cardinality {method} instead of {method} {path}.
Unknown methods use HTTP as the span name, emit _OTHER as http.request.method, and preserve the original method in http.request.method_original.
URLs emit as url.full instead of http.url, with credentials redacted and query parameters obfuscated.
Destination information uses server.address and server.port, including default ports 80 and 443.
HTTP status codes 400–599 are errors by default (changed from 400-499).
Error responses set error.type to the status code unless a more specific error type already exists.
Legacy peer-host, peer-port, and peer-service attributes are not generated.

HTTP server changes

When DD_TRACE_OTEL_SEMANTICS_ENABLED=true:

Span names become {method} {http.route} when a route is available, or {method} otherwise. Request paths and 404 are no longer used as naming fallbacks.
Unknown methods use HTTP as the span name, emit _OTHER as http.request.method, and preserve the original method in http.request.method_original.
The request URL is split into url.scheme, url.path, and the obfuscated url.query, rather than a singular http.url.
Server identity uses server.address and server.port, honoring forwarded protocol, host, and port headers.
Client information emits as client.address and network.peer.address.
HTTP status codes 500–599 are errors by default (unchanged).
Error responses set error.type to the status code unless a more specific error type already exists.
Inferred-proxy and Axway spans follow the same naming and attribute rules.
Endpoint discovery falls back to url.path when the legacy http.url attribute is absent

Test coverage

Unit and component tests

OtelHttpSemanticsTest covers method normalization, URL credential redaction, query removal, fragment preservation, default ports, and error-type behavior.
HttpDecoratorsOtelSemanticsForkedTest covers client and server decorator output, route naming, status handling, forwarded origins, explicit forwarded ports, and IPv6 hosts.
ConfigTest verifies the forced OpenTelemetry settings, calculated configuration values, and client error-status defaults.
QueryObfuscatorTest verifies sanitized client url.full reconstruction and server/client handling of url.query.
HttpEndpointPostProcessorTest verifies endpoint discovery from url.path.
KnownTagsTest and OtlpTraceJsonCollectorTest verify tag registration and the OpenTelemetry names emitted over OTLP.

Instrumentation tests

The shared HTTP test suites run in a forked JVM with OpenTelemetry semantics enabled for:

HTTP clients:

Apache HTTP Client
Apache HTTP Async Client
Google HTTP Client
HttpURLConnection
JAX-RS Client
OkHttp
Play WSClient

HTTP servers:

Akka-Http Server
JAX-RS Server
Jersey Server
Netty HTTP server
Servlet 3 / Jetty
Play 2.6
Ratpack HTTP server
Spring Web MVC 6
Spring WebFlux
Tomcat
Undertow
Vert.x

System tests

This is being tested by this PR: DataDog/system-tests#7919 . Currently the only test skips across all web servers are for Akka and Play servers, where the http.route needs to be set on the root span. I added this below as follow-up work so the majority of frameworks could be served here without being blocked.

Motivation

When DD_TRACE_OTEL_SEMANTICS_ENABLED=true, the Java tracer should produce HTTP spans that align with the OpenTelemetry semantic conventions instead of emitting a mixture of Datadog and OpenTelemetry naming and attributes. This lets users switch between OpenTelemetry and Datadog automatic instrumentation with reduced updates to dashboards, monitors, or downstream processing.

Additional Notes

This PR is easiest to review commit-by-commit: configuration, tag registration, client behavior, server behavior, client integration coverage, and server integration coverage are separated into focused commits.

Related .NET implementations: HTTP client spans and HTTP server spans.

Follow-up work requires the following:

Spring WebClient: The two HTTP client spans must be consolidated to 1 span, in accordance with the OpenTelemetry HTTP semantic conventions
Akka HTTP and Play: Both frameworks need to apply routing information to the root span's http.route

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket(s): [https://datadoghq.atlassian.net/browse/APMAPI-2038], [https://datadoghq.atlassian.net/browse/APMAPI-2039]

datadog-prod-us1-5 · 2026-10-06T20:37:14Z

🎯 Code Coverage (details)
• Patch Coverage: 71.62%
• Overall Coverage: 47.92% (-11.94%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 53b4e2c | Docs | Give us feedback!}

dd-octo-sts · 2026-10-06T21:38:38Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.94 s	13.91 s	[-0.4%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent	12.92 s	13.00 s	[-1.1%; -0.0%] (maybe better)
startup:petclinic:appsec:Agent	17.12 s	17.02 s	[-0.4%; +1.5%] (no difference)
startup:petclinic:iast:Agent	16.44 s	17.02 s	[-7.8%; +1.0%] (no difference)
startup:petclinic:profiling:Agent	16.74 s	16.74 s	[-1.1%; +1.1%] (no difference)
startup:petclinic:sca:Agent	17.09 s	17.02 s	[-0.5%; +1.3%] (no difference)
startup:petclinic:tracing:Agent	15.74 s	16.10 s	[-6.4%; +1.9%] (no difference)

Commit: d27a8f17 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

dougqh · 2026-10-20T15:22:26Z

  public static final String PATHWAY_HASH = "pathway.hash";

  public static final String HTTP_QUERY = "http.query.string";
+  public static final String HTTP_QUERY_OTEL_NAME = "url.query";


Could this live in KnownTags instead of DDTags? dd-trace-api is the customer-facing jar, so a public constant here becomes a permanent API contract. Customers can compile against it, and removing or renaming it later is a breaking change for them.

The registry already generates KnownTags.URL_QUERY_NAME / URL_QUERY_ID for this tag (internal-api). That covers the three main-code uses: HttpServerDecorator, QueryObfuscator and LambdaAppSecHandler. The ID form also fits the move to ID-based setters you made elsewhere in this PR.

Written by Claude, reviewed by @dougqh

I removed DDTags.HTTP_QUERY_OTEL_NAME and replaced all usages to KnownTags.URL_QUERY_NAME in 18c7ce9

datadog-prod-us1-5

OpenTelemetry HTTP semantics suppresses IAST missing-HSTS detection for eligible HTTPS responses. The shim also introduces attribute-ordering and null-name regressions.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 8a835c3}

- Http client error status codes default to 400-600 (instead of 400-500) - The environment variable DD_TRACE_OTEL_SEMANTICS_ENABLED automatically configures the following - OTEL_TRACES_EXPORTER=otlp - DD_WRITER_TYPE=OtlpWriter - DD_TRACE_SPAN_ATTRIBUTE_SCHEMA=v0 - DD_TRACE_PEER_SERVICE_DEFAULTS_ENABLED=false

http: - http.request.method_original (OTel) - server.address (OTel) - server.port (OTel) http.server: - url.path (OTel) - url.query (OTel) - url.scheme (OTel) mixins: peer: tags: - network.peer.address (OTel) - peer.port (DD) => network.peer.port (OTel) Map peer port to the OTel network attribute Fixup http tag conventions. Squash so we can edit the message

- Apache HTTP client - Apache HTTP Async Client - Google HTTP Client - HttpURLConnection - JAX-RS Client - OkHttp - Play WSClient

- Akka-Http Server - JAX-RS Server - Jersey Server - Jetty - Netty HTTP server - Servlet - Play HTTP server - Ratpack HTTP server - Spring MVC - Spring WebFlux - Tomcat - Undertow - Vert.x

- Restore IP tags collection so it is opt-in. The DD->OTel tag mapping remains - Restore query tags collection so it honors the user's query tag opt-out - Improve OtelHttpSemantics.parsePort performance (especially since this can be influenced by a user) - Stores the OTel http client span's "url.full" URL in a lazy wrapper so redaction and query removal only occur at export time

…t span-kind-neutral

…r.port and server.address while keeping the span direction relationships correct HTTP server spans: - http.server: restore "http.hostname" => OTel "server.address" - http.server: add standalone (OTel-only) "server.port" HTTP client spans: - outbound_peer: restore "peer.hostname" => OTel "server.address" - outbound_peer: restore "peer.port" => OTel "server.port"

… DDTags.HTTP_QUERY_OTEL_NAME, which mimics DDTags.HTTP_QUERY so we have one consistent tag name

…-in changes, including: HttpDecoratorsOtelSemanticsForkedTest.java - Elasticsearch and OpenSearch spans should be unchanged - There must not be duplicate attributes for client destination ports (server.port) - InferredProxySpan should record server.port=443 due to the default https scheme - When OTel semantics are enabled, Forwarded-header extraction should be run and determine client span attributes, even if DD_TRACE_CLIENT_IP_ENABLED=false or AppSec+AI Guard are disabled - When OTel semantics are enabled, url.full (OTel equivalent of http.url) decodes the query before storing and sending to QueryObfuscator. OtelHttpSemanticsTest.java - When OTel semantics are enabled, url.full (OTel equivalent of http.url) decodes the query before storing and sending to QueryObfuscator. HttpClientPathMappingOtelSemanticsForkedTest.java - TRACE_HTTP_CLIENT_PATH_RESOURCE_NAME_MAPPING takes precedence and replaces the http.route placeholder in the OTel HTTP server span name "{http.request.method} {http.route}" LambdaHttpOtelSemanticsForkedTest.java & LambdaAppSecHandlerTest.java - Lambda query tags should continue to be obfuscated GatewayBridgeOtelSemanticsForkedTest.java - API Security must continue to fall back to the http.url span attribute when the server span does not have an http.route span attribute

- QueryObfuscator: Obfuscate both the DD and OTel query string tags as a preventative measure to make sure we do not store an unobfuscated query string on a span. When OTel semantics mode is enabled, the OTel value will be take precedence as the final recorded value (the DD value will be discarded if present). - Ensure that we record the correct origin port when handling forwarded schemes

- Correct obfuscated-query expectations - Fix the IAST missing-HSTS detection by allowing it to also check the "url.scheme" OTel HTTP server tag to confirm HTTPS, since the "http.url" tag is only emitted with DD semantic conventions

- In the BaseDecorator.setPeerPort methods (used by both clients and servers), set the tag via the directionless KnownTags.PEER_PORT_ID key. - In HttpServerDecorator, strip the query and fragment from the URL before setting the OTel attribute "url.path". Also, set the OTel attribute "url.query" if a query string is present - In AxwayHTTPPluginDecorator make sure to honor the query opt-out before setting the OTel attribute "url.query"

…KnownTags.URL_QUERY_NAME (OTel "url.query") in its place

…the url-based tags on http clients and servers, for both Datadog and OTel semantics. Key details include: - HttpServerUrl is created as an abstract class to expose a consistent interface for accessing underlying URL facts from various request objects -- there is no equivalent for client spans as the logic is simple enough that operating on a URL object is sufficient - OtelServerUrl and DatadogServerUrl implement TagExtractor<HttpServerUrl> to provide the "OTEL_SEMANTICS branching" for populating http server span tags - DatadogClientUrl and OtelClientUrl implement TagExtractor<URI> to provide the "OTEL_SEMANTICS branching" for populating http client span tags - The host and hostPort calculations from OtelHttpSemantics.setServerAddressAndPort are migrated to HttpServerUrl.Origin.resolve - Additional logic from OtelHttpSemantics is ported to the new UrlExtractors, so a handful of tests in OtelHttpSemanticsTest.java are simply ported to HttpUrlExtractorsTest.java - HttpUrlExtractorsTest.java provides testing of the TagExtractor implementations

zacharycmontoya · 2026-10-21T01:59:53Z

Update: I've responded to the highlighted bugs from human and automated code reviewers. I've also done one (AI-facilitated) pass to move much of the logic out of the decorators themselves and into TagExtractors -- and the existing tests provided confidence in the newly refactored logic. It's important that these tags can be used for tag-based sampling rules, so I think this TagExtractor implementation satisfies that while improving maintenance.

How much more polish / refinement would you like to see on this PR? I think export-time tag extractors may be out of scope for this PR, but y'all are the experts here 😄

Preserve existing query concatenation for captured URLs with decoded path characters when OTel semantics are disabled. Restore port-based service splitting precedence and skip the OTel-only Host header lookup in that mode. Add regression coverage for all three issues. Each new regression failed before the fixes; all 102 selected query obfuscator and decorator tests now pass, including existing OTel and client path-mapping coverage.

Use string concatenation and trailing substrings when inserting or removing queries while retaining fragments. Verify query obfuscation, HTTP semantics helpers, URL extractors, and OTel decorators with their existing tests and formatting checks.

zacharycmontoya changed the base branch from master to dougqh/tag-registry-otel October 6, 2026 20:21

zacharycmontoya force-pushed the otel-http-semconv branch from 4930519 to b4d1988 Compare October 6, 2026 20:34

zacharycmontoya force-pushed the otel-http-semconv branch 4 times, most recently from f6f083f to 1f7dfff Compare October 6, 2026 21:11

zacharycmontoya force-pushed the otel-http-semconv branch 6 times, most recently from 4891a63 to 8cfde98 Compare October 6, 2026 23:45

dougqh force-pushed the dougqh/tag-registry-otel branch 2 times, most recently from e49a183 to 44e132c Compare October 7, 2026 13:11

zacharycmontoya force-pushed the otel-http-semconv branch 5 times, most recently from 06fde25 to 4788421 Compare October 7, 2026 23:42

Base automatically changed from dougqh/tag-registry-otel to master October 8, 2026 21:33

zacharycmontoya force-pushed the otel-http-semconv branch 4 times, most recently from 984c8d3 to f5b4d45 Compare October 13, 2026 21:36

zacharycmontoya changed the title ~~feat(otel): emit OpenTelemetry semantic conventions on HTTP spans~~ Emit OpenTelemetry semantic conventions on HTTP spans Oct 13, 2026

zacharycmontoya force-pushed the otel-http-semconv branch 3 times, most recently from 6906acf to ed69f54 Compare October 15, 2026 19:53

dougqh reviewed Oct 20, 2026

View reviewed changes

Base automatically changed from dougqh/tag-registry-direction-resolution to master October 20, 2026 17:02

datadog-prod-us1-5 Bot reviewed Oct 20, 2026

View reviewed changes

zacharycmontoya added 17 commits October 20, 2026 16:27

Implement OpenTelemetry HTTP client semantic conventions

ea6f1be

Implement OpenTelemetry HTTP server semantic conventions

61d0240

Test end-to-end OpenTelemetry HTTP client semantics

099041c

- Apache HTTP client - Apache HTTP Async Client - Google HTTP Client - HttpURLConnection - JAX-RS Client - OkHttp - Play WSClient

Test end-to-end OpenTelemetry HTTP server semantics

36f08f9

- Akka-Http Server - JAX-RS Server - Jersey Server - Jetty - Netty HTTP server - Servlet - Play HTTP server - Ratpack HTTP server - Spring MVC - Spring WebFlux - Tomcat - Undertow - Vert.x

PR feedback on tag-conventions: Restore network.client.ip so it is no…

01377cc

…t span-kind-neutral

Polish: Update the obfuscator tag name for OpenTelemetry semantics to…

f0f092d

… DDTags.HTTP_QUERY_OTEL_NAME, which mimics DDTags.HTTP_QUERY so we have one consistent tag name

Polish: Update setters to use IDs rather than tag names

4e1404e

Fix: Implement fixes for the noted regressions

70be501

fix linting issue

5eee031

Apply spotless formatting

1d4d8a0

zacharycmontoya force-pushed the otel-http-semconv branch from 8a835c3 to fd65cad Compare October 20, 2026 23:41

zacharycmontoya added 3 commits October 20, 2026 17:20

PR Feedback: Remove DDTags.HTTP_QUERY_OTEL_NAME (public API) and use …

18c7ce9

…KnownTags.URL_QUERY_NAME (OTel "url.query") in its place

This comment has been minimized.

Sign in to view

mcculls added 2 commits October 21, 2026 15:46

Simplify HTTP URL string assembly

53b4e2c

Use string concatenation and trailing substrings when inserting or removing queries while retaining fragments. Verify query obfuscation, HTTP semantics helpers, URL extractors, and OTel decorators with their existing tests and formatting checks.

mcculls added type: feature Enhancements and improvements comp: core Tracer core labels Oct 21, 2026

Navigation Menu

Repository navigation

Emit OpenTelemetry semantic conventions on HTTP spans - #12765

Emit OpenTelemetry semantic conventions on HTTP spans#12765
zacharycmontoya wants to merge 22 commits into
masterfrom
otel-http-semconv

zacharycmontoya commented Oct 6, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-5 Bot commented Oct 6, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

Uh oh!

dd-octo-sts Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

dougqh Oct 20, 2026

Uh oh!

zacharycmontoya Oct 21, 2026

Uh oh!

datadog-prod-us1-5 Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zacharycmontoya commented Oct 21, 2026

Uh oh!

This comment has been minimized.

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

zacharycmontoya commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Implementation Details

Configuration

HTTP client changes

HTTP server changes

Test coverage

Unit and component tests

Instrumentation tests

System tests

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-prod-us1-5 Bot commented Oct 6, 2026 • edited by datadog-prod-us1-4 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

dougqh Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

zacharycmontoya Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-prod-us1-5 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zacharycmontoya commented Oct 21, 2026

Uh oh!

This comment has been minimized.

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zacharycmontoya commented Oct 6, 2026 •

edited

Loading

datadog-prod-us1-5 Bot commented Oct 6, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

dd-octo-sts Bot commented Oct 6, 2026 •

edited

Loading