Repository navigation
fix: backspace once per character when replacing wide-char tokens - #487
Closed
aaaxingiiiiiii wants to merge 1 commit into
Closed
aaaxingiiiiiii wants to merge 1 commit into
aaaxingiiiiiii wants to merge 1 commit into
Conversation
calculateReplacement used tokenLength, a display width, as the number of backspaces to send. Shells delete one character per backspace, so a token containing wide characters (CJK) over-deleted: typing `cd 中` and accepting `中文目录/` sent two backspaces for one character and also removed the space before it, producing `cd中文目录/`. Convert the display width to a character count before sending backspaces. Quotes and escape characters are single-width, so the difference between the token text's display width and its character count is exactly the extra width to subtract. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
calculateReplacementsendstokenLengthbackspaces before inserting a suggestion, buttokenLengthis a display width, where a CJK character counts as 2. Shells delete one character per backspace, so tokens that contain wide characters delete too much.For example, type
cd 中and accept the folder中文目录/. Two backspaces are sent for one character, so the space aftercdis also deleted and the line becomescd中文目录/.This change converts the display width to a character count before sending backspaces. Quotes and escape characters are single-width, so the difference between the token text's display width and its character count is exactly the extra width.
The existing test
wide-char divergent uses display width for backspaceexpected the old behavior. I renamed it and changed it to expect one backspace, and added a mixed-width case.Test plan
src/tests/runtime/replacement.test.tspasses.npm run buildandnpm run lintpass.🤖 Generated with Claude Code