Repository navigation
Conversation
Drop `--max-jobs 1 --cores 2` from `python-runtime-deps` and `extra-python-packages`. The cap was introduced in 909f18e for the CUDA Docker image build, where it works around OOM. It was later copied to these check jobs without a recorded OOM there. Hosted runners have 4 cores, and the cap makes the source builds of torch and onnxruntime the bottleneck. In run 42780073869 the CPU `extra-python-packages` job hit the 6 hour job limit, with torch alone taking 154 minutes at `-j2`. The uncapped `lint` job already builds the same torch and onnxruntime concurrently at `-j4` each. It finished in 4h49m without OOM in run 42500297518 on main. The cap on the CUDA Docker image build is kept.
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the Python CI parallelism change.
No open issues 🎉
What changed in this PR
Restores default Nix parallelism in the Python CI check jobs to reduce source-build bottlenecks and avoid timeouts.
Changes:
- Removes
--max-jobs 1 --cores 2frompython-runtime-depsandextra-python-packages. - Keeps the CUDA Docker image build’s resource cap unchanged.
| File | Description |
|---|---|
.github/workflows/build.yml |
Removes explicit parallelism limits from both Python check jobs. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drop
--max-jobs 1 --cores 2frompython-runtime-depsandextra-python-packages.The cap was introduced in 909f18e for the CUDA Docker image build,
where it works around OOM.
It was later copied to these check jobs without a recorded OOM there.
Hosted runners have 4 cores,
and the cap makes the source builds of torch and onnxruntime the bottleneck.
In run 42780073869 the CPU
extra-python-packagesjob hit the 6 hour job limit,with torch alone taking 154 minutes at
-j2.The uncapped
lintjob already builds the same torch and onnxruntime concurrently at-j4each.It finished in 4h49m without OOM in run 42500297518 on main.
The cap on the CUDA Docker image build is kept.