Skip to content

Register APM tag constants on their span types (otlp - tag registry) - #12893

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 50 commits into
masterfrom
dougqh/tag-registry-coverage
Oct 20, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 50 commits into
masterfrom
dougqh/tag-registry-coverage

Conversation

@dougqh

@dougqh dougqh commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Registers the APM tag constants in the tag registry, each on the span types it's actually set on, so instrumentation can migrate to setting tags by KnownTags.*_ID and every tag gets a dense-store slot. A ratchet test tracks what's left.

  • A ratchet over the tag constants. TagConstantsRegistryCoverageTest requires every String constant in Tags, InstrumentationTags and DDTags to be registered, or listed by name or pattern (*):
    • NEVER_REGISTERED maps each deliberate exception to its reason: values of a tag (Tags.SPAN_KIND_*), a name prefix, a name pattern, an operation name, names only the serializers write, and unused public API.
    • NOT_YET_REGISTERED_FAMILIES names whole families with a reason, today the CI Visibility constants (Tags.TEST_*, Tags.GIT_*, Tags.CI_*, ...). A family excuses only its unregistered constants, and is stale once none is left.
    • NOT_YET_REGISTERED lists single constants and can only shrink: registering one fails the test until its entry is removed. A new constant must be registered or listed.
  • Placement from the code. For each tag I checked its production set sites (as a constant and as a string literal), its value type, and which spans it lands on, and placed it there: as low in the type hierarchy as it belongs, because a tag on a broad type takes a slot in every subtype's clique.
  • Span types and mixins. Span types model a concept or an extension of one concept. Mixins are for concerns that cut across span types or kinds: product add-ons (Data Streams, Database Monitoring, Exception Replay, the debugger's code origin), inherited sampling, the AWS request tags shared by AWS client and SQS consumer spans, and a messaging system (Kafka, AMQP), which spans producers, consumers and more.
  • Generator: the Java overlay can declare span_types and mixins. Java-only tags (servlet's, Spring's, TIBCO's, Mule's) live in tag-conventions-java.yaml, keeping the shared file language-agnostic. An overlay span type must extend a span type; an overlay mixin must say what it applies to. servlet.path and servlet.context move out of the shared file into a Java servlet.request type.

New span types

  • Messaging: messaging.producer, .consumer, .broker (time in queue), .client (AMQP commands), .poll, with kafka and amqp mixins.
  • RPC: rpc.client, rpc.server.
  • Databases: db.client keeps what any database client sets; db.sql (JDBC, R2DBC), db.cassandra, db.couchbase, db.redis and db.mongo extend it. Database Monitoring's propagation tags are a dbm mixin on db.sql and db.mongo.
  • AWS: aws.client (an http.client) and one type per service: aws.s3, .sqs, .sns, .kinesis, .dynamodb, .sfn, .lambda. One aws.client holding every service's tags would be 67 tags, over the 63-slot cap, though a span never carries two services' tags.
  • Websocket: websocket.receive, .send, .close.
  • Generic: web.handler (framework controller spans), internal (internal spans no other type models).
  • Java overlay: servlet.request, spring.view.render, spring.webflux.handler, spring.webflux.client, tibco.process, mule.action, elasticsearch.rest, elasticsearch.transport, twilio.sdk, jsp.compile, jsp.render, quartz.job, hystrix.command.

Motivation

The registry had 66 tags against about 350 tag constants. The migration to ids needs every tag registered, and the dense store gives a tag a precise slot only when it's on the span types it appears on.

Additional Notes

  • No functional change at runtime beyond more tags having ids. Two unreferenced constants are deleted from internal-api: Tags.DB_HOST and InstrumentationTags.JMS_PRODUCE (whose string is JMSDecorator's operation name).
  • Coverage: every constant outside CI Visibility is registered or deliberately never registered. What remains is the CI Visibility families (about 100 tags on test spans), in a follow-up PR for that team to review. Some tags have no constant and are registered anyway (jsp.requestURL, elasticsearch.action, ...): tags set only by literal are outside this test's reach, and APMLP-2046 proposes auditing them in the instrumentation tests.
  • Slots: with Color known tags into co-occurrence slots in their ids (phase 3a - groundwork - dense tag storage) #12882's coloring the 213 tags use 47 slots, under the cap of 63. The largest clique is elasticsearch.transport, which adds 21 tags to db.client.
  • trace_level holds both process-static facts (the .enabled flags, now including _dd.apm.enabled) and values common to a trace (local-root tags such as the attribute schema version).
  • Name patterns aren't registered. A family like _dd.code_origin.frames.<N>.<field> has unboundedly many names, so it can't have ids ahead of time and can't use the dense store; its members stay custom tags. Frame 0's constants are listed as never registered for that reason, and only _dd.code_origin.type is registered. A general registry could later describe patterns (name, type, OpenTelemetry naming) without ids.
  • Open question: websocket spans copy peer.hostname from the handshake, but outbound_peer declares it for client spans only; left as a TODO in the YAML.
  • Possible follow-up: span types for extensions of a concept mean some parallel types (AWS per service). If reviewers prefer an add-on concept, mutually exclusive mixins could be colored per alternative, so they'd cost no more slots than types.

🤖 Generated with Claude Code

dougqh and others added 30 commits October 9, 2026 08:28
Adds id-keyed setters, getters and removal to TagMap, TagMap.Entry and
spans, so a writer that knows the tag skips the name lookup. Squashed
from the review history of #12715.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
span.setTag(0L, "") or a null value reached TagMap.getAndRemove(long),
which rejects unknown ids, so clearing threw IllegalArgumentException
while every id-keyed setter ignores an unknown id. removeTag(long) now
ignores it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The long-valued setters prechecked with needsIntercept, but when the
interceptor declined the tag they discarded the box it was given and stored
the primitive. Follow the same precheckIntercept -> setBox shape as the
other primitive setters, so TagMap keeps the box.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
peer.port is declared per direction in the registry (#12713), so it has no
direction-free id: BaseDecorator, shared by client and server decorators,
sets it by name, and the set-by-id tests use an unshared int tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Set-path routing is a per-language concern, so it lives in a Java overlay
next to tag-conventions.yaml rather than in the language-agnostic
conventions. The overlay declares the keys that exist only to be routed
(resource.name, error, sampling directives, ...) and lists every tag
TagInterceptor may route. Each listed tag's id carries the INTERCEPTED bit
(bit 1), so a setter called with a constant id can fold the interception
test away. Serial numbers become public so TagInterceptor can switch on
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TagInterceptor switches on the tag's serial rather than its name, so a
known tag is matched under any of its names and the *_OTEL_NAME cases go
away. needsIntercept(long) tests the INTERCEPTED bit first, which folds
away for a constant id. split-by-tags entries resolve to serials at
construction (each direction for a name declared per direction); only
custom tags are still matched by name. A test checks that exactly the
tags carrying the bit have a case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A name setter resolves a known tag's name to its id once and sets it as the
id-keyed setters do; only a custom tag is still handled by name. The
id-keyed setters no longer look up the name to feed the interceptor, so with
a constant id the interception test folds to the INTERCEPTED bit. Entry
paths (builder ledger, prototypes, default tags) route by the entry's id
and precheck before boxing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generated nameOf switched over every tag's serial: 491 bytes of
bytecode, too big for C2 to inline, so each id-keyed set paid two
out-of-line calls into it (the span's unknown-id guard and the entry's
name). Read a NAMES_BY_SERIAL array instead; nameOf is now 23 bytes and
inlines at every caller.

Also adds SetTagBenchmark (span setTag by constant id, non-constant id,
name, and custom name, against a bare and a synchronized TagMap store).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The table is always allocated and never resized, so the split check -- the
only run-time check left on a non-intercepted constant-id set -- is a single
load, with no null or length test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The span's unknown-id guard called nameOf; KnownTagCodec.isKnown answers the
same question from the serial and a generated SERIAL_LIMIT, so it folds
away for a constant id. Also note that serials are not stable across
releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DDSpanContext.setTag calls interceptTag only for intercepted tags. Kept out
of line (over FreqInlineSize, 325 bytes of bytecode), it never brings the
profiled handler bodies into setTag's compiled code, which would push setTag
past InlineSmallCode and stop callers inlining it -- the inlining a constant
id needs to fold its interception test. A test now fails if the switch
shrinks below the limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An overlay tag exists only to be routed, so the generator now marks it
intercepted without a second listing; intercepted: lists only the tags
declared in tag-conventions.yaml that the tracer also routes. Also restore
the doc comment and @Suppress that the overlay helper displaced, read the
serial limit one way, inline isUnknownTag, and drop an unused default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tag declared once per direction (peer.port) has two ids but one
shared Datadog name, and keyOf resolves that name to neither. An entry
keyed by one of those ids hashed by the id while name-based access
hashed the name as a custom tag, so get/remove/containsKey by name
missed it and a later set by name produced a duplicate key.

Encode a SHARED_NAME flag in the reserved id bit 0 so the check folds
for a constant id. TagMap's id-keyed create/get/remove reject such ids,
and DDSpanContext's id-keyed setters ignore them, as they do unknown
ids, until name resolution knows the span's direction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Entry's package-private name factories are now custom-only: they skip the
registry and assert the name is not a known tag. Callers that start from a
name resolve it once -- TagMap.set(String, ...) forwards a known tag to
set(long, ...), while getAndSet, put, Ledger.set, putAll and the public
Entry.create(String, ...) API go through anyEntryFor and friends -- so a
custom tag pays one registry lookup instead of two.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	.github/CODEOWNERS
TagMap.internals() returns a small Internals view whose setKnown(...) skips
the id validation the checked set(long, ...) repeats; DDSpanContext's
id-keyed setters, which have already checked isKeyableById, now use it.
Internals is created per call, so escape analysis removes it.

internals() is marked with a new @restricted(allowedIn = ...) annotation,
and gradle/forbiddenApiFilters/instrumentation.txt bans it in
instrumentation modules, so calling it there fails forbiddenApisMain.
Internals is also where the batched setTagsFrom/TagSink writes will go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It exists to be created, used within the expression, and scalar-replaced;
storing one would force a real allocation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Restricted lives in the new datadog.apicontract package, which had no owner.
Give it the same owner as its sibling datadog.perfcontract.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
peer.port was modeled as one tag per direction: PEER_PORT_INBOUND_ID and
PEER_PORT_OUTBOUND_ID, sharing the Datadog name and marked with a
SHARED_NAME id bit. Two ids under one name kept producing lookup and
supersede bugs: keyOf("peer.port") resolved to neither id, so a value set
by name and one set by id landed in different entries, and tombstones and
getAndSet by name could not find what an id had stored.

The generator now merges a name's per-direction declarations into one tag
with one id (PEER_PORT_ID), and only its OpenTelemetry name depends on the
span's direction: openTelemetryNameOf(id, direction) returns client.port
inbound and server.port outbound. The SHARED_NAME bit, hasSharedName and
isKeyableById go away (bit 0 is left unused), and keyOf("peer.port")
returns PEER_PORT_ID.

KnownTagCodec gains the DIRECTION_* constants and
openTelemetryTagOf(id, direction). Direction-scoped OpenTelemetry names
are still not resolved when written by name.

This is the direction-neutral-id part of #12731's redesign; its
direction-scoped aliases and read-time resolution are left to that PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With peer.port now one direction-neutral tag, its OpenTelemetry name
depends on the span: server.port on a client span, client.port on a
server span. The OTLP JSON and protobuf writers compute the span's
direction once from its span.kind and name each tag for it, so
direction-scoped renames apply too: peer.hostname exports as
server.address on a client span, and network.client.ip as
network.peer.address on a server span.

KnownTagCodec gains directionOf(TagMap) / directionOfKind(String), and
TagMap.EntryReader gains openTelemetryTag(direction), standing in for
the SpanTags helpers #12731 introduces.

Ported from #12731's redesign.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Entry's full constructor and the Internals and Ledger constructors were
private, so javac (targeting Java 8, without nestmates) gave each one a
synthetic access constructor taking the never-loaded class TagMap$1. C2
refuses to inline a call whose signature names an unloaded class
("unloaded signature classes"). So the Internals view the core creates
for each id write could not be scalar-replaced, and every Entry built
through that constructor went through a real call. Making the
constructors package-private removes the accessors and TagMap$1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Several private members were used across TagMap's nested classes, so
javac (targeting Java 8) generated access$ methods for them: Entry's
*2Prim helpers and requireKnownName, TagMap.parentEntryVisible,
TagMap.putEntry and the TagMap.buckets field. Each accessor adds a level
of inlining depth. putEntry's accessor sits on Internals.setKnown, the
core's trusted id-write path. All of these are now package-private.

TagMapSyntheticAccessorTest fails if TagMap or any of its nested classes
gains a synthetic accessor again. It is the prototype for the
@NoSyntheticAccess contract (APMLP-1976).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master now has #12715 (set-by-id, squashed) and #12037 (decorator
SpanPrototype). Conflicts resolve to this branch's newer id-keyed code,
plus:
- getAndSet finds its inherited prior by the entry's id (#12715's last fix);
  the shared-name test for it is dropped, since peer.port now has one id.
- #12037's resolveSplitByTagsPrecedence checks component by id and language
  by name against the serial-indexed split-by-tags table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings the three classes back over jacoco's thresholds: the trusted
setters' Object/float/double overloads, isIntercepted, isKnown's range
edges, canonicalTagName, span-kind direction, and an entry's directional
OpenTelemetry name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build-logic is a separate Gradle build, so the root spotlessApply never
formats it. Three generator lines were wrapped where its Kotlin formatter
keeps them on one line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh requested review from a team as code owners October 15, 2026 19:57
@dougqh
dougqh requested review from cecile75 and sarahchen6 and removed request for a team October 15, 2026 19:57
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T20:01:11.815625Z 20a0148 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The two most actionable issues are: messaging span type declarations (producer, consumer, client) omit peer address/hostname/service tag mixins that RabbitMQ instrumentation already writes, and web.handler spans are excluded from the code-origin tag applies list, causing Spring controller spans to omit _dd.code_origin.type from their resolved tag sets.

Open Bits AI session

🤖 Bits Code Review · Commit 20a0148

Comment thread tag-conventions.yaml
Comment thread tag-conventions.yaml Outdated
Two review findings, both confirmed in the code:

- RabbitChannelInstrumentation calls onPeerConnection on its client,
  producer and consumer decorators. messaging.producer and
  messaging.client now include peer, peer_address and outbound_peer
  (peer.service is computed for client and producer spans, as for
  http.client and rpc.client); messaging.consumer includes peer_address
  and inbound_peer. Consumers also set peer.hostname, which
  outbound_peer declares for client spans only: a TODO, as for
  websocket spans.
- Code origin's entry capture instruments annotated controller methods,
  whose active span is the framework's handler span, so the code_origin
  mixin also applies to web.handler.

Slots unchanged at 47 with #12882's coloring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@sarahchen6 sarahchen6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look reasonable, though I'm not familiar with OTLP tag expectations TBH... Probably worth another check from the SDK Cap team

dougqh and others added 2 commits October 19, 2026 11:28
Clean merge. With the merged coloring, this branch's 212 registered tags
color into 47 slots (SLOT_COUNT 47, cap 63), and no tag shares a slot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dougqh added a commit that referenced this pull request Oct 19, 2026
…registry-civis

Clean merge; the generator now matches #12893's, with no trace of the
reverted slot marker. With the merged coloring the 319 tags color into
63 slots: the CI test span types overflow, and 66 optional tags share
slots (CI Visibility's occasional tags plus _dd.span_links, _dd.svc_src,
error.debug_info_captured and events), on those span types only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh added this pull request to the merge queue Oct 19, 2026
@dd-octo-sts

dd-octo-sts Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-10-19 19:20:24 UTC ℹ️ Start processing command /merge


2026-10-19 19:20:29 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-10-19 20:12:36 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for f80c7d6:

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
  • Any question, go check the FAQ.
Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 19, 2026
@dougqh
dougqh added this pull request to the merge queue Oct 20, 2026
@dd-octo-sts

dd-octo-sts Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-10-20 00:59:42 UTC ℹ️ Start processing command /merge


2026-10-20 00:59:46 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-10-20 02:27:36 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 20, 2026
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit c53ffd3 into master Oct 20, 2026
610 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the dougqh/tag-registry-coverage branch October 20, 2026 02:27
@github-actions github-actions Bot added this to the 1.68.0 milestone Oct 20, 2026
dougqh added a commit that referenced this pull request Oct 20, 2026
- OtelSpanBuilder, OpenTelemetry14DirectionalAttributesTest, and
  KnownTagCodec.directionalKeyOf: master's versions, carrying #12731's
  late fixes (held attributes resolved at setSpanKind, null keys ignored).
- tag-conventions.yaml: #12893 already declares process_id and
  _dd.trace_span_attribute_schema as required local-root trace tags, so
  this branch's duplicate declarations go; _dd.agent_psr joins that group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dougqh added a commit that referenced this pull request Oct 20, 2026
Master's #12893 is this branch's base, squashed, so this branch's versions
of its three files stand; tag-conventions.yaml also takes #12731's later
server.port tag and direction-resolution notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants