refactor(ast): expose positional documentation runs and attachment evidence #1659

Objective

Make Go/Ruby/Lua run recognition preserve member nodes and attachment evidence while maintaining existing row accounting.

Umbrella: #1654 · Planning ID: D04
Completion prerequisites: D02: #1656, D28: #1657

Shared architecture and constraints

Follow the umbrella's shared contracts. AST recognition serves two consumers: exact legacy convention/coverage streaming for root LOC, and richer source documentation for owner lookup and local text. Source attachment, profile eligibility and physical metric ownership are separate. Preserve the legacy candidate view before run selection, original emission timing, and metrics-only early exits.

Documentation fragments, logical documents and convention-qualified owner edges have separate identities. Source offsets refer to the actual parsed buffer and included ranges. The index is caller-owned, immutable and source-bound; lookup validates tree/index identity. Complete text means local decoded/cleaned content with forward source mapping, not inherited or generator-expanded documentation.

Implementation touchpoints

• big-code-analysis-ast/src/roles/ and roles/mod.rs: shared candidate membership and context/projection contract.

• big-code-analysis-ast/src/lang_helpers/doc_text.rs: byte marker classifiers, split from the binding walker.

• big-code-analysis-ast/src/lang_helpers/doc_comment.rs

• big-code-analysis-ast/src/checker/{go,ruby,lua}.rs

• src/metrics/loc/shared.rs

• src/metrics/loc/{go,ruby,lua}.rs

Implementation steps

• Read DocRunOpener/DocRunDirective and candidate rows through the current context-aware helpers while retaining once-per-run decisions.

• Evolve leading_doc_run to report ordered run members, following candidate, opener-based reason, directive members and run boundaries.

• Preserve exact legacy eligibility and coverage through the internal streaming path; atomically adapt LOC callers.

• Keep run finalization at the last member and per-walk caches; reverse scans must not become one scan per comment.

• Distinguish a following real declaration from opener/virtual documentation; preserve evidence for later language owner resolution.

• Coordinate shared binding-file changes with D03; refactor(ast): split doc_comment.rs into the binding walker and the text classifiers #1649 is already merged. Schedule serial landings when the same walker regions are edited.

Tests and edge cases

• Plain Lua -- versus --- run above the same declaration; Ruby YARD-ineligible wrappers versus metric-visible members; directive-bearing Go/Ruby runs with exact fragment order.

• Blank source lines versus blank comment lines; trailing comments that cannot seed a run; multiple comment nodes sharing one row.

• Go directives, Ruby shebang/magic comments, Lua long-comment termination and opener-only blocks.

• A non-target after a run, file EOF, and explicit virtual opener retain precise outcomes.

• Hand-pinned member order and spans plus legacy metric tuples; perturb boundary/opener/directive checks.

• Existing Go/Ruby/Lua depth/width probes and D28 characterization pass.

Use independently specified expected owners, fragment counts/order, byte ranges, text and availability. Pair them with separate legacy metric assertions where relevant. Do not sort/trim observed results, rely only on facade/delegate equality, or accept generated snapshots without an expected-value anchor. Use verbatim source for EOF/CRLF cases. Perturb the decision-bearing expression and confirm the intended assertion fails; use the maintained utils/perturb.py only on tracked, clean targets, recording or stashing real work first. Multi-target cargo runs require --no-fail-fast; include the mutated crate and dependent crates before claiming a unique guard. Keep irreproducible fixtures/manifests in tests, maintained utils or an issue attachment, following .claude/rules/temp-files.md.

Acceptance criteria

• One positional mechanism serves extraction and legacy coverage.
• No lost directive evidence or fabricated owner for an opener-only run.
• Current metrics and amortized run traversal behavior are preserved.

Verification and completion evidence

• Follow current role-census/role-coverage/role-bypass requirements and applicable role-parity-corpus checks. Preserve independent semantic fixtures; refresh .role-coverage-observed.txt when any sampled corpus gitlink changes.

• Put revisions, exact commands, tool versions and durable input links beside parity/coverage claims; host-local evidence paths are not citations.

• Recheck current source and the prerequisite contracts/baseline applicable to this issue; record intentional changes to the premises.

• Run focused production-path tests and affected language siblings. Use package-isolated -p big-code-analysis-ast / -p big-code-analysis feature builds; workspace feature unification is not isolation.

• Run the repository's required checks for the touched surfaces. For implementation changes: make worktree-setup, make pre-commit read from its BCA_GATE: verdict, and relevant test-language membership comparison. Add scaling and chain-audit checks for traversal changes; run make fuzz-check and seed replay for fuzz changes.

• Record exact test/perturbation/probe evidence in a completion comment and update this body. Review unexpected metric drift before snapshot acceptance; follow baseline and integration-submodule discipline if behavior deliberately changes.

• Land public API documentation, re-export/stability review and changelog entries with the originating API change. Keep unrelated CLI/web/Python surfaces outside this issue unless explicitly listed.

Revalidation against main (2026-11-04)

Source baseline inspected: 07822c69211fe851c452ac6c0ebbd99cf92589a5. Relevant landed work: PR #1769, PR #1803, PR #1804. These are implementation updates to the existing acceptance criteria; this review does not mark extraction work complete.

• Go/Ruby/Lua documentation rows and gates are already declared. Run selection stays in doc_comment.rs; marker/directive byte tests live in doc_text.rs; Ruby YARD scope and call forms are in lang_helpers/ruby.rs.

• Lua's DocTarget row is used by its opener decision; Checker::is_doc_target intentionally stays false for plain -- runs. A generic row-forward change would widen legacy documentation incorrectly.

• Ruby member traversal was broadened/shared in PR fix: batch 2026-11-03-2 (8 issues) #1804, but YARD publication scope has its own contract. Reuse structural facts without equating metric body traversal with documentation eligibility.