Repository navigation
fix(onlion): the embedding breaker must live in the invocation, not the module - #136
Merged
Merged
Conversation
…he module Every log line carries instance.seq = 1 while uptime_s counts a reused container: this platform starts a **fresh isolate per request**, so the module-level breaker added in #135 never outlived the request that tripped it and protected nothing. The 18:12-18:27 window shows the real mechanism: nine dependency_timeout events at ~12 s each inside requests that ran 60-76 s, four concurrent Adds all killed at 120 s, all status 200. What actually ran to the ceiling was one request paying the provider once per fold entry: eight entries x one 12 s timeout. The breaker is now per invocation - after NMG_EMBED_FAIL_FAST_AFTER (default 1) provider-side failure, every later call in the same request fails in microseconds as embedding_suppressed. That truncates 8 x 12 s to one 12 s. Reproduced locally with a stubbed provider that always times out: before, every Add paid the timeout; after, the first Add costs one timeout (395 ms at a 300 ms test budget) and the following seven complete in 8-205 ms with **one provider call in total** instead of eight. The cross-request window is kept: it applies wherever an isolate is reused, and costs nothing when it is not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Doc owner:
docs/decisions/implemented/2026-10-30-embedding-breaker.{md,zh-CN.md,terms.yaml}(amended in this PR if the record needs the scope correction — see below).Evidence from 18:12-18:27: every request logs
instance.seq = 1with containeruptime_s1210-1232, so a fresh isolate per request — the module-level breaker from #135 never outlived its own request. The real cost was inside one request: eight fold entries x one ~12 s provider timeout each = 60-76 s, and four concurrent Adds all died at the 120 s ceiling with status 200.Fix: the breaker trips per invocation (
NMG_EMBED_FAIL_FAST_AFTER, default 1). After one provider-side failure, later calls in the same request fail in microseconds. Local reproduction with a permanently-timing-out provider: one provider call total instead of eight; Adds go from 12 s each to 8-205 ms after the first.Not verified
Not deployed yet. The 12 s truncation is verified only in the local reproduction (fresh-request-per-fetch on Miniflare), which is not the same isolate lifecycle as production; the per-invocation counter is correct on both, but only a live window can show the effect on the 120 s class.