Skip to content

fix(api,common): escalate low-confidence history votes to the local decide tier (carries #1467) - #1503

Open
xerj-org wants to merge 6 commits into
mainfrom
carry/todie-1467
Open

xerj-org wants to merge 6 commits into
mainfrom
carry/todie-1467

Conversation

@xerj-org

Copy link
Copy Markdown
Owner

Carries #1467 by @todie (their two commits are cherry-picked with authorship intact), plus review fixes. Refs #1094.

What the code does

With the local decide tier armed (decide-local + XERJ_DECIDE_MODE=local), a history vote whose winning label holds less than the new [decisions] escalate_below share (default 0.8, range 0.0..=1.0, 0.0 = off) is handed to the local head instead of being answered from history, on both POST /_decide and POST /v1/systemone. The local answer is then written back to [decisions] index like every tier-2 answer, so the history keeps growing on split votes instead of freezing once BM25 starts returning neighbours (#1094). With no local tier armed, nothing changes: a split vote still answers from history. The setting count goes from 132 to 133 everywhere it is quoted.

What the review added (separate xerj-org commits)

  • One cached answer per decided text. Write-back documents now get a deterministic id (decide-<xxh3-128 of the text>) instead of a random UUID. Before this, a question that kept escalating (its own cached answer is only one neighbour among split ones) appended a new document on every request, so the local head's label gained one more vote each time. Repetition alone could then out-vote the history, human corrections included. Re-deciding the same text now overwrites its single cached answer. New tests: unit write_back_id_is_one_per_decided_text, and e2e re_escalating_the_same_question_keeps_one_cached_answer (fails on the carried commits: the second escalation appends a 4th document).
  • docs/DECISIONS.md now documents the escalation exception in the ladder, the one-per-text write-back, and escalate_below in the config block. It also notes that the local head is weak on choice labels it was not trained on, where operators should lower or disable escalate_below.
  • The original body said "Closes decision flywheel: write-back freezes once the history index has BM25 support — 12 of 2,000 fill answers cached #1094". This changes it to "Refs decision flywheel: write-back freezes once the history index has BM25 support — 12 of 2,000 fill answers cached #1094": the fix covers only nodes with the local tier armed, and the issue's Banking77 replay gate has not been re-measured.

Verification

  • cargo test -p xerj-api --features decide-local --test decide_flywheel --test systemone_local_decide, cargo test -p xerj-api --lib systemone_api, cargo test -p xerj-common, cargo test -p xerj-engine --test product_experience
  • cargo fmt --all --check, cargo clippy -p xerj-api -p xerj-common -- -D warnings (also with --features decide-local)
  • ES-YAML conformance on a private port with a throwaway data dir: see the review comment for counts.

Note: CI does not build decide-local, so the escalation e2e tests run only locally (as they were run here).

🤖 Generated with Claude Code

todie and others added 4 commits October 16, 2026 00:29
Motivation: with #1467's escalation, a split history vote is answered by
the local head and written back. The same question asked again is often
still split (its own cached answer is one neighbour among several), so it
escalates again, and the write-back used a random UUID, so every repeat
appended another document voting for the local head's label. Repetition
alone could then out-vote the history, human corrections included.

Fix: the write-back document id is `decide-<xxh3-128 of the text>`, so
re-deciding the same text overwrites its single cached answer.

Before: escalating "refund money back check" twice -> 4 docs (2 seeded +
2 identical cached answers). After: 3 docs, the cached one updated in
place (new ts).

Tests: unit write_back_id_is_one_per_decided_text; e2e (decide-local)
re_escalating_the_same_question_keeps_one_cached_answer, verified to fail
with the random-id write-back and pass with this change.

Files: engine/crates/xerj-api/src/systemone_api.rs (write_back_id,
spawn_write_back), engine/crates/xerj-api/tests/decide_flywheel.rs.
…1094)

The ladder section said history always wins when it has support; since
#1467 a split vote escalates to the armed local head below
decisions.escalate_below. Documents the exception, its off switch, the
choice-label caveat (the local head is near chance on untrained label
vocabularies), the deterministic write-back id, and adds escalate_below
to the config block.
@cla-bot cla-bot Bot added the cla-signed label Oct 16, 2026
# Conflicts:
#	engine/crates/xerj-common/src/lib.rs
Main added server.auto_port (#1487, 132 -> 133) while this branch added
decisions.escalate_below (132 -> 133); git merged the identical "133"
lines silently. The true total is 134: updated every quote (config.rs
header/comments/test, lib.rs, xerj.default.toml "56 of 134",
EXPECTED_SETTINGS in product_experience.rs).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

decision flywheel: write-back freezes once the history index has BM25 support — 12 of 2,000 fill answers cached

2 participants