Skip to content

fix(telemetry): align gen_ai.choice system with request model for LiteLLM - #7640

Open
Hugo-DDT wants to merge 1 commit into
google:mainfrom
Hugo-DDT:fix/choice-log-gen-ai-system-from-request-model
Open

Hugo-DDT wants to merge 1 commit into
google:mainfrom
Hugo-DDT:fix/choice-log-gen-ai-system-from-request-model

Conversation

@Hugo-DDT

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:
With ADK_TELEMETRY_SCHEMA_VERSION_OPT_IN=1, gen_ai.choice logs report gen_ai.system=gemini for LiteLLM models such as openai/gpt-4o and groq/llama-3.1-8b-instant, while the span and system/user message logs correctly report openai / groq. Anthropic is unaffected because bare claude-* ids already resolve via the Anthropic pattern.

Root cause: _inference_system_name preferred llm_response.model_version. LiteLLM returns that without a provider prefix (e.g. gpt-4o), so _resolve_gen_ai_system_name falls back to Gemini.

Solution:
When model_version does not itself identify a gen_ai.system provider (no Gemini match, no Anthropic pattern, no <provider>/<model> prefix), fall back to the agent-configured request model id (the same id the span already uses). Added _model_identifies_gen_ai_system to share that check with the resolution rules.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

New test_choice_log_system_name_when_model_version_lacks_provider covers openai/gpt-4o, groq/llama-3.1-8b-instant, anthropic, and gemini when model_version is a bare id.

pytest tests/unittests/telemetry/test_spans.py::test_choice_log_system_name_when_model_version_lacks_provider \
       tests/unittests/telemetry/test_spans.py::test_generate_content_span_system_name_follows_model \
       tests/unittests/telemetry/test_spans.py::test_resolve_gen_ai_system_name \
       tests/unittests/telemetry/test_spans.py::test_resolve_gen_ai_system_name_never_names_a_path_segment
# 24 passed

pytest tests/unittests/telemetry/test_spans.py tests/unittests/telemetry/test_stable_semconv.py
# 190 passed

Red/green: the new openai/groq cases failed on main (gemini vs expected provider) and pass with the fix.

Manual End-to-End (E2E) Tests:

Not run (repro in #7631 uses LiteLLM mock_response; covered by the unit regression above).

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

N/A

…on lacks provider

LiteLLM returns model_version without a provider prefix (e.g. gpt-4o for
openai/gpt-4o). _inference_system_name preferred that bare id, so
gen_ai.choice fell back to gemini while the span correctly used openai/groq.

Prefer the agent-configured model id when model_version does not identify a
provider. Fixes google#7631.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gen_ai.choice log reports gen_ai.system=gemini for OpenAI and Groq models through LiteLlm

2 participants