Skip to content

test(storage): verify authentic historical SQLite upgrades - #655

Merged
N0zoM1z0 merged 9 commits into
mainfrom
test/authentic-storage-migrations
Oct 16, 2026
Merged

N0zoM1z0 merged 9 commits into
mainfrom
test/authentic-storage-migrations

Conversation

@N0zoM1z0

@N0zoM1z0 N0zoM1z0 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Replace SQLite migration tests that subtracted objects from the latest schema with forward upgrades from authoritative migration declarations. A private cfg(test) helper constructs historical prefixes; normal Storage::open performs upgrades.

Add immutable main/WAL snapshots produced by official v0.1.15, v0.1.27 and v0.1.28 executables, with pinned binary/source/dependency provenance. Verify canonical schemas, historical data and FTS, counters, constraints, reopen, subsequent writes and transactional failure rollback. Freeze released migration/provenance digests and document the two historical column-order exceptions.

Explicit fixture regeneration uses fresh private workspaces, committed archive/executable hashes, streaming reads and independent archive/member/executable/decoder limits. Child stdout/stderr use regular files under a 4 MiB kernel file limit. Only zero arguments or exactly --prepare are accepted, before any workspace/network/execution activity. Normal CI runs offline checks in normal and optimized Python. Bare focused-owner selectors now stop at the module boundary while partial selectors retain their behavior.

Closes #519.

Validation

  • All 154 selected storage, historical-migration and cache tests passed on integration with production query-plan changes, including genuine prefix9/prefix11 private invariants; 17 CI planner tests passed.
  • All 17 offline generator tests passed in normal and optimized Python, including the real optimized script entry point, invalid arguments, replaced release assets, log overflow, oversized/lying streams, archive metadata, identity checks and private-workspace retention.
  • All three official executables produced the release anchors. All three retained official archives also passed bounded prepare-only validation without network or execution; fixture bytes remain unchanged.
  • Formatting and relevant all-target/all-feature Clippy passed. Final normal CI run 38939491357 passed on 2f3f6eb. Codex review completed without findings, Greptile reported 5/5, and all review threads are resolved.
  • Rust builds/tests ran serially with one compiler job under a resource guard; the combined 154-test run peaked at 2.09 GiB group RSS with at least 41.52 GiB available. No OOM occurred.

Generate every migration prefix from the authoritative test-only schema owner
and verify normal forward upgrades against fresh schema and data contracts.

Add provenance-bound official release main/WAL fixtures, append-only migration
digests, explicit compatibility review, and transactional rollback coverage.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
@N0zoM1z0
N0zoM1z0 requested a review from morluto as a code owner October 16, 2026 15:58
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T17:28:22.369644Z 2f3f6eb New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The reviewed changes appear safe to merge; no blocking issue was found.

Summary

The PR replaces reconstructed SQLite histories with authoritative migration prefixes and immutable released main/WAL snapshots. The latest changes reject unsupported generator arguments before any work starts.

  • Storage migration tests now start from authentic database histories.
  • Fixture preparation checks pinned releases within fixed resource limits.
  • Focused storage selectors now stop at their module boundary.

Reviews (7) · Last reviewed commit: "fix(testing): reject unknown migration-g..." · Reviewed by Greptile

Comment thread crates/test-suite/fixtures/storage-migrations/generate.py Outdated
Comment thread xtask/src/main.rs Outdated
Comment thread crates/test-suite/src/domains/storage_migrations.rs Outdated
Download, extract, and execute release generators in fresh private workspaces; preserve failed invocation evidence and cover the lifecycle with offline checks.

End bare suite owner filters at the module boundary while preserving partial test selectors. Remove the cross-owner source text ban; retain migration digests, immutable provenance, forward-prefix matrices and transactional failure checks.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
Keep current cache inventory bounds and coverage policy alongside forward historical migration fixtures and reviewed test infrastructure.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 869e2015fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/test-suite/fixtures/storage-migrations/generate.py Outdated
Comment thread crates/test-suite/fixtures/storage-migrations/test_generate.py
Keep checksum, archive member, pinned tag and fixture integrity checks active with Python optimization. Add offline refusal cases and run the suite in normal and optimized modes in the required CI quality lane.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
Resolve shared test owners by preserving both independent evidence suites, genuine prefix9/prefix11 private invariants, and unrelated storage lifecycle checks. Combine owner documentation without retaining duplicated EQP or reverse-built migration fixtures.

Validation:154 selected storage, migration and cache tests,17CI planner tests, and5offline generator tests in both normal and optimized Python passed.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a64779ee68

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/test-suite/fixtures/storage-migrations/generate.py Outdated
Stream downloads and executable extraction through fixed buffers with incremental hashes and independent actual/declared byte limits. Bound checksum responses, archive members, expanded tar, metadata reads and XZ decoder memory before executing a verified release.

Document every preparation bound and retain immutable fixture bytes. Ten offline tests pass in normal and optimized Python, and all three retained official archives pass bounded preparation without network or execution.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
Write child stdout and stderr directly to private regular files under the existing kernel file-size limit, and bound successful stdout readback before JSON parsing. Preserve logs on failure and timeout without captured output pipes.

Twelve offline tests pass in normal and optimized Python, including real child stdout/stderr overflow checks. Immutable release fixtures remain unchanged.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b859a4d92b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/test-suite/fixtures/storage-migrations/generate.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d8ad97179

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/test-suite/fixtures/storage-migrations/generate.py Outdated
Read bounded checked-in provenance before network access and validate its fixed version, tag, commit and digest identities. Require streamed archive and executable hashes to match those trusted pins; endpoint checksum files are consistency checks and cannot update execution authority.

Fourteen offline tests pass in normal and optimized Python, including a jointly replaced archive/checksum negative case. All three retained official archives pass bounded preparation against unchanged committed pins.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd0d7e46d3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/test-suite/fixtures/storage-migrations/generate.py Outdated
Accept only explicit generation or the exact --prepare mode before any workspace, network or executable activity. Exercise the real optimized entry point and invalid/duplicate arguments offline.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
@N0zoM1z0
N0zoM1z0 merged commit d1843c1 into main Oct 16, 2026
11 checks passed
@N0zoM1z0
N0zoM1z0 deleted the test/authentic-storage-migrations branch October 16, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test SQLite upgrades from authentic historical schemas instead of subtracting from latest

1 participant