Skip to content

Add shared profile execution behind Compatibility SPI - #315

Open
qoli wants to merge 53 commits into
huggingface:mainfrom
qoli:codex/upstream-profile-tool-continuations
Open

qoli wants to merge 53 commits into
huggingface:mainfrom
qoli:codex/upstream-profile-tool-continuations

Conversation

@qoli

@qoli qoli commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Scope

Shared execution support for #314's composition API. Depends on #314/#313 and main's #352. Isolated layer comparison.

The 1.0 RequestContext/zero-argument resolver stay public. New fields, overloads, provider/callback/Tool hooks and init(profile:history:) remain Compatibility SPI. A missing profile model still throws the existing explicit error; no default model or constructor-visibility decision is introduced.

  • Per-request model/options/profile resolution, lifecycle preparation and producing-request callbacks.
  • Writable profile/callback history; read-only Tool/dynamic-instructions history. Protected pending entries survive writes without leaking into durable history.
  • Raw public Tools retain identity; execution/native bindings carry canonical history and the selected offered-Tool list.
  • Shared continuation mapping keeps the ordinary transcript + inFlight path. Profile projection only replaces the uniquely matching complete unchanged round; legal removals/reordering remain authoritative.
  • Tool-output identity is owning ToolCalls round + call ID, not global provider call-ID deduplication. Provider fixtures compare the complete committed ToolCalls entry seen by hooks/Tools, including ID and metadata.
  • Shared ToolRoundLimit uses original call-site options.maximumToolRounds, upstream Let callers set how many tool rounds a response takes #331's error type and all completed handoff rounds. MLX/Llama share the same counter. Repeated calls remain allowed (Run a repeated tool-call round instead of stopping #343).

Upstream 1.1 synchronization

Normal history-preserving merges synchronize upstream main 7d43219, including #357 per-request usage accounting, #341 Ollama reasoning, #358 host-scoped Chat Completions reasoning, #300 response-start timeouts, #362 cancellation collection, #364 partial structured snapshots and #366/#367 bounds/reasoning/system-Tool fixes. Earlier #336 final-request usage, #340 HTTPTransport, #297/#325/#331/#343/#352/#354/#356 remain intact. No force-push, new checkout, replacement PR or extra fallback.

Response/snapshot usage remains the final request, including zero when unreported; the internal ledger retains earlier requests across handoffs. Non-streaming profile and ordinary generation now use upstream's request-counting channel, so completed requests are counted before return and retained when generation throws. Streams are created outside an enclosing response's accounting channel, including the profile-dispatch task. Prompt/schema/image entry-point regressions first reproduced 10 accounting issues and 6 stream-leak issues, then passed. Existing handoff totals, original call-site round budgets and complete committed-entry identities remain covered.

Fresh local verification (2026-10-23)

Head cf4ad0d. CI=1 GIT_CONFIG_GLOBAL=/dev/null swift test: 949 tests / 100 suites passed. The separate upstream macro test target also passed 2 tests / 1 suite. Strict formatting/diff checks passed. These are compile/offline tests, not live API, model-weight or Apple Intelligence inference acceptance.

A separate same-checkout scratch run with --traits MLX,Llama,CoreML passed 1115 / 110 on this head; no live weight/inference acceptance is claimed.

Provider follow-ups remain separate: #344 OpenAI, #345 OpenResponses, #346 Anthropic, #347 Gemini, #348 Ollama and #349 native projection/initial context reuse. #346/#347 additionally depend on #312. Public profile construction and the separate ordinary Ollama continuation PR remain outstanding scope decisions.

@qoli qoli mentioned this pull request Oct 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

In-flight Tool history can be lost, wrapper models can recurse during initial dispatch, and one context option is unreachable.

3 open findings
What changed in this PR

Adds Foundation Models 27-compatible dynamic profiles and preserves profile-selected behavior across provider-owned Tool continuations.

Changes:

  • Adds session properties, request/context options, dynamic profiles, and lifecycle callbacks.
  • Re-resolves profiles between Tool rounds and supports model handoffs across compatibility providers.
  • Expands provider, compatibility, lifecycle, and history tests.
File Description
README.md Documents request options.
Sources/​AnyLanguageModel/​ContextOptions.swift Defines request-context options.
Sources/​AnyLanguageModel/​DynamicInstructions.swift Exposes instruction resolution internally.
Sources/​AnyLanguageModel/​DynamicProfile.swift Implements profiles, modifiers, and lifecycle handling.
Sources/​AnyLanguageModel/​GenerationOptions.swift Adds sampling alias and Tool-calling mode.
Sources/​AnyLanguageModel/​LanguageModelSession.swift Integrates profiles, properties, callbacks, and error policies.
Sources/​AnyLanguageModel/​ProfileToolExecution.swift Centralizes Tool execution and model handoffs.
Sources/​AnyLanguageModel/​SessionProperties.swift Implements session-scoped property and history storage.
Sources/​AnyLanguageModel/​Models/​AnthropicLanguageModel.swift Adds profile-aware Tool continuations.
Sources/​AnyLanguageModel/​Models/​GeminiLanguageModel.swift Adds profile-aware Tool continuations.
Sources/​AnyLanguageModel/​Models/​OllamaLanguageModel.swift Adds multi-round profile-aware Tool execution.
Sources/​AnyLanguageModel/​Models/​OpenAILanguageModel.swift Adds profile-aware Chat and Responses continuations.
Sources/​AnyLanguageModel/​Models/​OpenResponsesLanguageModel.swift Adds profile-aware Open Responses continuations.
Sources/​AnyLanguageModel/​Models/​SystemLanguageModel.swift Bridges Tool-calling mode to Foundation Models.
Sources/​AnyLanguageModelMacros/​Plugin.swift Registers the session-property macro.
Sources/​AnyLanguageModelMacros/​SessionPropertyEntryMacro.swift Generates session-property storage accessors.
Tests/​AnyLanguageModelTests/​AnthropicReasoningTests.swift Covers non-streaming Tool continuation reasoning.
Tests/​AnyLanguageModelTests/​APICompatibilityAnyLanguageModelTests.swift Checks compatibility profile APIs.
Tests/​AnyLanguageModelTests/​APICompatibilityFoundationModelsTests.swift Checks Foundation Models API parity.
Tests/​AnyLanguageModelTests/​DynamicProfileTests.swift Tests profile resolution and lifecycle behavior.
Tests/​AnyLanguageModelTests/​FoundationModelsGenerationOptionsTests.swift Tests Tool-mode bridging.
Tests/​AnyLanguageModelTests/​ProviderUsageTests.swift Tests provider continuations and model switching.
Tests/​AnyLanguageModelTests/​RequestOptionsTests.swift Tests request options and serialization.
Tests/​AnyLanguageModelTests/​SessionPropertyTests.swift Tests property storage, history, and observation.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Sources/AnyLanguageModel/ProfileToolExecution.swift
Comment thread Sources/AnyLanguageModel/SessionProperties.swift Outdated
Comment thread Sources/AnyLanguageModel/LanguageModelSession.swift Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Model handoffs can bypass tool-loop limits, while continuation history and typed property defaults have correctness defects.

3 open findings
3 resolved since last review

🧠 Review effort: Balanced

Comment thread Sources/AnyLanguageModel/ProfileToolExecution.swift
Comment thread Sources/AnyLanguageModelMacros/SessionPropertyEntryMacro.swift Outdated
Comment thread Sources/AnyLanguageModel/LanguageModelSession.swift Outdated
qoli added 4 commits October 15, 2026 09:03
…dynamic-profiles

# Conflicts:
#	Sources/AnyLanguageModel/LanguageModelSession.swift
#	Sources/AnyLanguageModel/SessionProperties.swift
#	Sources/AnyLanguageModelMacros/SessionPropertyEntryMacro.swift
#	Tests/AnyLanguageModelTests/APICompatibilityAnyLanguageModelTests.swift
#	Tests/AnyLanguageModelTests/APICompatibilityFoundationModelsTests.swift
#	Tests/AnyLanguageModelTests/SessionPropertyTests.swift

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved history and handoff bugs can corrupt transcripts, break session isolation, and repeat Tool side effects.

5 open findings
3 resolved since last review
Previously missed (1)

In code that hasn't changed since last review

Medium severity Filter instructions from tool execution history bindings

Sources/​AnyLanguageModel/​SessionProperties.swift:104

For an ordinary session with static instructions, withProfileToolExecution passes the full canonical request transcript into this binding. Unlike the unbound history getter, the binding does not filter instructions, so a Tool reading @SessionProperty(\.history) sees instruction entries despite HistoryView's instruction-free contract. Filter instructions when constructing the history binding.

🧠 Review effort: Balanced

Comment thread Sources/AnyLanguageModel/LanguageModelSession.swift
Comment thread Sources/AnyLanguageModel/LanguageModelSession.swift Outdated
Comment thread Sources/AnyLanguageModel/LanguageModelSession.swift
Comment thread Sources/AnyLanguageModel/ProfileToolExecution.swift Outdated
Comment thread Sources/AnyLanguageModel/SessionProperties.swift Outdated
@qoli

qoli commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Updated in 89c0077: the shared profile/model-handoff budget now reads the original call-site options.maximumToolRounds and preserves #331’s ToolRoundLimitExceeded type/payload. Fresh full SwiftPM run: 850 tests in 87 suites passed, including nondefault two-round ordinary/profile budget regressions and shared-counter access. The legacy RequestContext/zero-argument resolver remains public; only the added fields/overloads/hooks stay SPI. The shared wire-message mapper and reused-call-ID/provider encoding changes from the latest review are still pending separately, not claimed fixed here. Latest-head upstream CI is action_required (workflow approval), not a compile/test failure.

@qoli

qoli commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

The remaining shared-continuation work is now implemented in 90c85f1 / 1dcc38c and propagated to the provider PRs.

  • One shared wire-message mapper keeps transcript + inFlight for ordinary sessions. Profiles replace only the matching unchanged round; a removed/reordered projection remains authoritative.
  • ToolOutput identity is scoped to its owning ToolCalls round. Regression cases reuse call_0 in two rounds and across turns and inspect the continuation requests, rather than deduplicating by that ID globally.
  • Protected in-flight history is restored during writable-history edits without leaking it into durable history.
  • The original options.maximumToolRounds counter remains shared across model handoffs; the upstream ToolRoundLimitExceeded payload now retains all earlier completed rounds as well.
  • Integrate dynamic profiles with Anthropic Tool loops #346's ordinary nonstreaming path records outputs without building/encoding unused in-flight messages.
  • Integrate dynamic profiles with Ollama Tool loops #348 retains upstream ChatReply errors and disallowed-tool behavior; lookup is limited to the tools actually offered.

Fresh validation: this runtime head passes 880 tests / 91 suites. Each provider branch was independently tested and pushed. The aggregate draft #317 passes 930 / 92 with default traits and 798 / 83 using a new isolated AsyncHTTPClient scratch build. These are offline CI-mode tests, not live inference. All merges preserve published history.

The constructor visibility and ordinary Ollama nonstreaming PR separation remain outstanding decisions; this push does not claim those two requests are complete.

@qoli

qoli commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Synced the stack normally with upstream 7013d51 (#351 typed bounds, #336 final-request usage and #340 HTTPTransport), preserving the separate PR boundaries.

In #315 2e96883, a profile handoff now carries completed requests in the internal request ledger, leaving Response/Snapshot.usage as the final request, including zero when that final request reports none. It also retains the selected provider's earlier requests so session.usage counts the whole handoff. A streaming/non-streaming regression first reproduced 12 failures, then passed; provider handoff fixtures now assert final-zero versus cumulative session accounting.

All provider branches now use the injected HTTPTransport for their continuation requests. The existing complete ToolCalls entry/metadata, round-scoped call IDs, shared round budget, legal history projections, image snapshot/MIME behavior and native delegate are retained. Ollama's already-published ordinary non-streaming loop remains intact; its new ordinary/profile final-usage regression first failed in all four cases, then passed. No new constructor visibility/default-model choice or separate Ollama PR is bundled here.

Fresh local CI-mode verification: #315 909 tests / 96 suites, plus 1075 / 106 with MLX/Llama/CoreML. Draft aggregate #317 802d496 contains every new implementation head and passed 969 / 97 default, 826 / 87 AsyncHTTPClient and 1135 / 107 MLX/Llama/CoreML with separate scratch paths in the same fixed checkout. Individual PR descriptions list their latest head/results. These are compile/offline checks, not live API/model-weight inference acceptance. The aggregate keeps its original independent 64-line README delta and stays draft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants