Repository navigation
feat(tracing): add failure-check span attribute helper - #555
mohammadatallah-scale wants to merge 9 commits into
Conversation
Offline failure checks read sgp.agent.* attributes the agent declares on its spans. FailureCheckSpec validates the root span declarations on construction and renders them as flat dotted keys holding native JSON values, which the SGP processor ships verbatim as span metadata. output_name, verification_result and escalates write the per-span attributes. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The span store keeps metadata in a ClickHouse JSON column, where a schema object spreads into many dynamic paths whose keys can collide after dot-splitting. FailureCheckSpec still takes a JSON object and validates it, then renders sgp.agent.output_schema as a JSON string. Every other attribute stays a native value. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
output_schema is a mutable dict on a frozen spec, so an edit after construction skipped the checks construction applies and could write NaN or invalid JSON text. attributes() now runs the same check before writing the schema, so an edited schema fails with the error construction raises. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
P3 Refuted: |
adk.tracing.span yields None when trace_id is empty or the span fails to start, so the example reading root.data raised AttributeError on that path. It now guards the span the way the tutorial's acp.py does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only step_budget, stop_reason and prompt_version errors were checked for the attribute they name, so relabelling escalation_channels as quote_arguments still passed. Every root field and VerificationRule now asserts the name its error starts with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
P3 Follow-up: the attribute-naming pins landed in 815dad0, and each label swap above now fails 2 to 11 tests. The non-string key coercion is left open because fixing it changes behavior the scale-gp-beta port must match, so both helpers need it together. |
The depth was an inline number, so a reader could not tell what it controls. It is now a named class constant. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A path like send_email.to.cc was rejected with a message that did not say why. Tool names cannot contain dots, so an entry is a tool and one top-level argument, and the error now says nested argument paths are not supported. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
escalates("") failed with a message about a verification span id, which did
not name the attribute being written. The message now starts with escalates,
as the other validation errors start with their attribute.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
JSON writes every object key as a string, so the encoder quietly turned an
int, bool, float or None key into text and let {1: "a", "1": "b"} collapse
into one entry. Any non-string mapping key in the schema is now rejected, at
construction and again when attributes render.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Important
Pending Mohammad's self-review. This note is removed by a human, not automation.
The problem. Offline failure checks read attributes that an agent declares on its spans. AgentEx agents have to hand-build every key and nothing checks the values (OVE-1375).
The fix. A helper that writes and validates those attributes from one spec. The scale-gp-beta port is scaleapi/sgp-dev-python-staging#10.
dataand the SGP processor sends it as span metadata. The tracing tutorial shows how.Test plan
🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; no new blocking issue was found.Summary
Adds
FailureCheckSpec,VerificationRule, and per-span helpers throughadk.No new actionable issues found. Tests were inspected, not run.
Reviews (4) · Last reviewed commit: "fix(tracing): reject non-string keys in ..." · Reviewed by Greptile