Repository navigation
simplify(search-eval): embed the canonical corpus - #4530
Open
ScriptedAlchemy wants to merge 1 commit into
Open
ScriptedAlchemy wants to merge 1 commit into
ScriptedAlchemy wants to merge 1 commit into
Conversation
|
This was referenced Oct 25, 2026
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 25, 2026 10:24
45a1e69 to
35c547d
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 25, 2026 10:24
f88b6aa to
d5b13e2
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 25, 2026 10:35
35c547d to
33d47ea
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 25, 2026 10:35
d5b13e2 to
46532d2
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 25, 2026 11:27
33d47ea to
0edfed2
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 25, 2026 11:27
46532d2 to
deb6c24
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 25, 2026 19:42
0edfed2 to
b9864dc
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 25, 2026 19:42
deb6c24 to
e6d05d4
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 25, 2026 20:06
b9864dc to
91804a5
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 25, 2026 20:06
e6d05d4 to
e5d734b
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 26, 2026 03:42
d368062 to
9cf7a68
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 26, 2026 03:42
379a571 to
e852784
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 26, 2026 03:51
9cf7a68 to
4f5bbe9
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 26, 2026 03:51
e852784 to
3e31ec1
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 26, 2026 03:59
4f5bbe9 to
fa929ee
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 26, 2026 03:59
3e31ec1 to
e4d884f
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 26, 2026 05:15
fa929ee to
fb0786d
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 26, 2026 05:15
e4d884f to
12bc45d
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 26, 2026 05:34
fb0786d to
25c2f98
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 26, 2026 05:35
12bc45d to
0ffccb3
Compare
ScriptedAlchemy
force-pushed
the
fix/retire-unused-sync-controls
branch
from
October 26, 2026 08:58
25c2f98 to
603f723
Compare
ScriptedAlchemy
force-pushed
the
fix/search-eval-canonical-corpus
branch
from
October 26, 2026 08:58
0ffccb3 to
2e642cb
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge #4478 first, followed by #4526, #4510, #4524, #4489, #4485, #4527, #4528, #4530, #4531, #4529, #4534, and #4541.
This PR is number 9 of 13, based on #4528. Merge #4531 next.
Use merge commits. After merging a parent PR, change its child's base to
masterbefore merging the child. Auto-merge is disabled.Fixes #4449.
Why
The packaged search evaluator embedded a separately maintained copy of its runtime corpus under
crates/tracedecay-query/assets/runtime-root. Fifteen files duplicated existing canonical fixtures; the workload JSON existed only in the mirror. This made source and packaged qualification depend on keeping two editable trees synchronized.Changes
tests/fixtures/search_quality/query-lexical-graph-workload-v1.jsonwithout rewriting it, and delete the 15 redundant corpus copies.evals/agent_adoption, not a new parentevalspackage. Remove the obsolete generated parent BUILD and preserve handwritten rules.The runtime corpus remains embedded. This change makes no binary-size reduction claim.
Exact corpus inventory
Every removed path below had the prefix
crates/tracedecay-query/assets/runtime-root/. The retained canonical path is the listed path at the repository root. All 15 duplicate pairs had zero byte differences, verified with SHA-256 and original Git blob identity; together they remove 149,745 redundant source bytes. The workload relocation also retains its exact original blob,b7e73cd31925006254c4aa7f020d3eab5bbb6337.evals/agent_adoption/fixture/Cargo.locktests/fixtures/context_eval_project/src/auth/login.rstests/fixtures/context_eval_project/src/auth/session.rstests/fixtures/context_eval_project/src/net/retry.rstests/fixtures/context_eval_project/src/storage/config_store.rstests/fixtures/sample.dockerfiletests/fixtures/search_quality/corpus/cargo-slot/src/main.rust.fixturetests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/integration.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/repository.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/research/canonical.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/research/coverage.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/research/error.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/research/time.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/research/watermark.rstests/fixtures/search_quality/corpus/crates/tracedecay-domain/src/session.rstests/fixtures/search_quality/query-lexical-graph-workload-v1.jsonThe embedding list is the runtime consumer; source workload/corpus loading uses the same logical paths. Generated compile inputs, LF handling, and the report diagnostic are updated together. Historical Git fixture packs and distinct/versioned fixtures retain their existing ownership.
Preserved semantic pins
sha256:b40c4c470dc2afae55debd2930d39f9f5d56494afd8a010672fb53bceb0295ec8312618fee8109b16be09e65f45118b4e550fa14587aed48f162f74765da3a35fc41850523ea377dsha256:7f52a7213e6caab00e6a2bfca4a24ccdfe3f37fcd2312fd86da34807a397b846sha256:f8f6e5973a2c07c2e3dc46ec270c75e819f6dc333030f8acabdf1580311577ebAll 67 query definitions, labels, quoted provenance, and train/validation boundaries retain their original bytes. No receipt or semantic identity was re-pinned.
Validation
Canonical Bazel validation on the integrated parent tree:
//evals:agent_adoption_fixtureused a glob that could not cross the existingevals/agent_adoptionpackage boundary. Invocation173f64e8-65b9-4ef4-8f07-4063fda24401; no tests executed.67158e9e-29e7-4c4b-95c0-2ef4b74f06f5.07b62bf5-4f1c-4ff4-b920-1908eb857cf2.--check: all 61 generated BUILD files current, invocation21e4261d-c459-4203-ac18-daec4236a130.1aa066dc-10d5-4d1b-a39a-e17e16e1b8c6.Failresult for conceptual misses. Passing these tests proves preservation and truthful reporting, not a qualification Pass.Stack
Based on #4528 (
fix/retire-unused-sync-controls). This layer contains only the corpus consolidation, generated compile inputs, and focused regression coverage.