Skip to content

chore(qa): run the visual QA suites in every interface locale - #437

Merged
smith-and-web merged 3 commits into
mainfrom
chore/qa-visual-locales
Nov 6, 2026
Merged

smith-and-web merged 3 commits into
mainfrom
chore/qa-visual-locales

Conversation

@smith-and-web

Copy link
Copy Markdown
Owner

Summary

The visual QA suites matched English button text, messages and aria-labels, so a pass started with KINDLING_LOCALE=<tag> stopped at the first English wait (about 12 seconds in). With this change, all five non-English locales complete the full 147-checkpoint matrix.

  • Harness (harness.js): on install, loads the app's own i18n module (same module instance, same locale) and indexes the English catalog. q.L / q.Ls / q.has / q.is map the English copy a suite uses to the running locale. One-placeholder templates such as "Close {name}" are handled recursively. q.template builds prefix/suffix selectors for templated labels such as "Shortcut for {label}" (zh-CN puts the label first).
  • Runner (run.mjs): localize() rewrites English aria-label / title / placeholder selectors in any page code it sends. It handles plain or escaped quotes and asks the page once for labels outside the catalog. tr() covers values the runner types. Button lookup goes through q.is.
  • Suites: about 25 English text checks now use window.__qa.has/is. The "Project: N words" wait uses the catalog key.
  • Docs: a README section "Other interface languages", plus a pointer in the visual-qa skill.

English runs are unchanged. The full matrix reports the same changed checkpoints as before this branch: the suite 07 toolbar row, fixed separately in #436, and the timing-dependent 19-02 export preview.

Results from the first all-locale pass on main (64a068a)

All five non-English locales ran all 147 checkpoints. Axe, Press, overflow and console audits were clean in every locale. Every image differs from the English baselines, as expected. A review of the captures found the defects to fix in follow-ups:

  • Hardcoded English outside the catalog:
    • revision statuses (revisions.ts)
    • export format and starter profile names, and the "Chapter N" heading (exportPrototype.ts)
    • the raw project_type in the copy-references dialog
    • the default review round name (Rust)
  • Formatting: dates and decimals use the OS locale instead of the app locale.
  • Export settings search: matches English keywords only.
  • Assembled strings: .toLowerCase() on type labels breaks German nouns, and hedge templates built by concatenation read badly («Novo(a) capítulo», «Новый(-ая) глава»).
  • Layout: the sidebar All / Planned / Next 5 control overlaps in es, pt-BR and ru.
  • Fonts: Cyrillic and CJK headings fall back from Fraunces to system fonts.

Test plan

  • npm run test:qa-harness: 34 passed, including new tests for the page helpers and Runner.localize / tr
  • npm run qa:visual: English full matrix, 138/147 matched; the 9 changed are the known 07 and 19-02 diffs
  • KINDLING_LOCALE=<tag> npm run qa:visual for zh-CN, es, pt-BR, ru and de: full matrix, audits clean, cleanup succeeded
  • npm run format:check

🤖 Generated with Claude Code

The suites matched English button text, messages and aria-labels, so a
pass under KINDLING_LOCALE stopped at the first English wait. The harness
now loads the app's i18n module and maps the English copy the suites use,
including one-placeholder templates such as "Close {name}", to the running
locale; the runner rewrites English aria-label, title and placeholder
selectors the same way. English runs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@smith-and-web
smith-and-web merged commit 1693dbe into main Nov 6, 2026
15 checks passed
@smith-and-web
smith-and-web deleted the chore/qa-visual-locales branch November 6, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant