Repository navigation
fix(cuda): retain quantized scratch through stream completion - #13
Merged
Merged
Conversation
jamesbrink
added a commit
to utensils/mold
that referenced
this pull request
Sep 6, 2026
CUDA generation had four related reliability gaps: unqualified Wan residual caches could collapse or blur output, completed Wan renders could be refused on reuse because retained context memory was charged twice, remote `--offload` never reached engine construction, and the H3 CUDA server suite did not run hermetically in CI. This batch fixes those boundaries together so the slow CUDA CI covers one coherent change. The change refuses step caching on the unqualified Wan 1.3B and dense 14B execution graphs while retaining the qualified A14B pair; derives expert-pair membership from current model paths rather than cached primary-transformer geometry; converts certified tensor-free Wan context attribution into an incremental peak consistently at admission and both worker rechecks; refreshes the current certificate after every synchronized Wan teardown; freezes request-level offload into engine identity, provenance, and durable sequence stages; and runs the complete H3 server library suite with isolated host configuration. It also makes the LTX-2.5 CUDA/ComfyUI qualification harness portable across GPU UUIDs, server profiles, and storage roots, and retains the completed Q4 oracle. The required Candle work landed first in [utensils/candle#13](utensils/candle#13). This PR pins its merged commit `c34de9b7`, which owns quantized CUDA scratch per device and orders cross-thread reuse and destruction with completion events. Validation: - exact-head `./scripts/ci-local.sh contracts docs`: all steps passed - Rust MSRV workspace and feature checks, fmt, generated contracts, workspace Clippy, optional-feature checks, mDNS, and H3 gates passed - the full workspace suite passed everywhere except the pre-existing `mold-db` concurrent-open lock race (322/323 database tests); that single test passed three consecutive clean-target reruns - CUDA forced-local, qualification-record, and private-H3 Clippy gates - hermetic `mold-ai-server --features h3-cuda --lib`: 2,183 passed, 0 failed, 10 ignored with no physical GPU visible - MiniMax H3 conditioner-cache tests: 15 passed - all four H3 CUDA capture binaries under Clippy - Candle source/lock/Nix identity contracts - Candle compute-sanitizer red/green: the old fixture reported a 64 MiB use-after-free and 44 errors; the merged fix passes both delayed cross-thread PTX cases with zero sanitizer errors - three consecutive Wan A14B Q5 73-frame renders under a 21,000 MiB foreign allocation passed in 119.9 s, 117.7 s, and 116.1 s with identical output hashes and a clean context certificate after every run - raising the foreign allocation to 24,000 MiB returned authoritative `infeasible / insufficientvram` - Wan 2.1 dense 14B default render, 33 frames at 832x480: visual pass with cache refusal; the cached control was blurred - Wan 2.1 1.3B default render, 81 frames: visual pass - LTX-2.5 remote forced-offload and GGUF Q4 mechanics rows, plus a ComfyUI GGUF Q4_K_M oracle capture An independent peer review found one stale-certificate issue during UAT; the final implementation replaces the prior certificate with the newest clean measurement and includes a decreasing-sample regression. Closes #1559. Closes #1481. Closes #1361. Closes #1462.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Quantized CUDA matmul scratch was owned by a host-thread-local cache even though
the submitted kernels could outlive that thread. Dropping a worker could free
the scratch asynchronously on a different stream, and another host thread could
reuse the same device allocation before the prior PTDS work completed.
This moves GGUF scratch ownership onto the CUDA device, leases each reusable
slot through the raw FFI submission, records a completion event before releasing
the lease, waits on that event before cross-thread reuse, and synchronizes it
before asynchronous destruction. The ordinary same-thread path retains the
workspace cache.
Validation:
use-after-free with 44 errors; the fixed fixture passes both delayed PTX tests
with zero sanitizer errors.
allocation baseline.
mainafter fix(metal): bound convolution workspaces and check completed commands #12.