Repository navigation
fix(api,common): escalate low-confidence history votes to the local decide tier (carries #1467) - #1503
Open
xerj-org wants to merge 6 commits into
Open
fix(api,common): escalate low-confidence history votes to the local decide tier (carries #1467)#1503xerj-org wants to merge 6 commits into
xerj-org wants to merge 6 commits into
Conversation
Motivation: with #1467's escalation, a split history vote is answered by the local head and written back. The same question asked again is often still split (its own cached answer is one neighbour among several), so it escalates again, and the write-back used a random UUID, so every repeat appended another document voting for the local head's label. Repetition alone could then out-vote the history, human corrections included. Fix: the write-back document id is `decide-<xxh3-128 of the text>`, so re-deciding the same text overwrites its single cached answer. Before: escalating "refund money back check" twice -> 4 docs (2 seeded + 2 identical cached answers). After: 3 docs, the cached one updated in place (new ts). Tests: unit write_back_id_is_one_per_decided_text; e2e (decide-local) re_escalating_the_same_question_keeps_one_cached_answer, verified to fail with the random-id write-back and pass with this change. Files: engine/crates/xerj-api/src/systemone_api.rs (write_back_id, spawn_write_back), engine/crates/xerj-api/tests/decide_flywheel.rs.
…1094) The ladder section said history always wins when it has support; since #1467 a split vote escalates to the armed local head below decisions.escalate_below. Documents the exception, its off switch, the choice-label caveat (the local head is near chance on untrained label vocabularies), the deterministic write-back id, and adds escalate_below to the config block.
# Conflicts: # engine/crates/xerj-common/src/lib.rs
Main added server.auto_port (#1487, 132 -> 133) while this branch added decisions.escalate_below (132 -> 133); git merged the identical "133" lines silently. The true total is 134: updated every quote (config.rs header/comments/test, lib.rs, xerj.default.toml "56 of 134", EXPECTED_SETTINGS in product_experience.rs).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carries #1467 by @todie (their two commits are cherry-picked with authorship intact), plus review fixes. Refs #1094.
What the code does
With the local decide tier armed (
decide-local+XERJ_DECIDE_MODE=local), a history vote whose winning label holds less than the new[decisions] escalate_belowshare (default0.8, range0.0..=1.0,0.0= off) is handed to the local head instead of being answered from history, on bothPOST /_decideandPOST /v1/systemone. The local answer is then written back to[decisions] indexlike every tier-2 answer, so the history keeps growing on split votes instead of freezing once BM25 starts returning neighbours (#1094). With no local tier armed, nothing changes: a split vote still answers from history. The setting count goes from 132 to 133 everywhere it is quoted.What the review added (separate xerj-org commits)
decide-<xxh3-128 of the text>) instead of a random UUID. Before this, a question that kept escalating (its own cached answer is only one neighbour among split ones) appended a new document on every request, so the local head's label gained one more vote each time. Repetition alone could then out-vote the history, human corrections included. Re-deciding the same text now overwrites its single cached answer. New tests: unitwrite_back_id_is_one_per_decided_text, and e2ere_escalating_the_same_question_keeps_one_cached_answer(fails on the carried commits: the second escalation appends a 4th document).escalate_belowin the config block. It also notes that the local head is weak onchoicelabels it was not trained on, where operators should lower or disableescalate_below.Verification
cargo test -p xerj-api --features decide-local --test decide_flywheel --test systemone_local_decide,cargo test -p xerj-api --lib systemone_api,cargo test -p xerj-common,cargo test -p xerj-engine --test product_experiencecargo fmt --all --check,cargo clippy -p xerj-api -p xerj-common -- -D warnings(also with--features decide-local)Note: CI does not build
decide-local, so the escalation e2e tests run only locally (as they were run here).🤖 Generated with Claude Code