Add wheel tests to CI #1416
Add wheel tests to CI #1416
Conversation
gforsyth
changed the title
pentschev
added
3 - Ready for Review
|
/ok to test |
|
@gforsyth I've triggered CI manually now but FYI going forward you'll need to setup GitHub commit signing to ensure CI runs automatically. |
Adding this now that wheels are available
gforsyth
force-pushed
the
add-wheel-tests
branch
from
3ef9326 to
76f1886
Compare
We want a name like: `dask_cuda_wheel_python_dask_cuda_py312_x86_64.tar.gz` from these environment variables
gforsyth
force-pushed
the
add-wheel-tests
branch
from
9ab6a88 to
4ab1ed0
Compare
|
We don't currently use tags/keyword in the PR titles, so as a nit I just changed its title to contain only the message, this is really just a nit, hope you don't mind Gil. 🙂 |
There was a problem hiding this comment.
Thanks for working on this! Left some suggestions.
Can you please also add a similar entry in https://github.com/rapidsai/dask-cuda/blob/branch-25.02/.github/workflows/test.yaml? You can follow the patterns from other repos, like https://github.com/rapidsai/cudf/blob/branch-25.02/.github/workflows/test.yaml
Those .github/workflows/test.yaml workflow configs define which tests run in 3 scenarios:
- on the target branch (e.g.
branch-25.02) after merges - nightly
- manually triggered in the UI via workflow dispatch
@ me if you want some helping resolving the CI failures.
| # This selects "ARCH=amd64 + the latest supported Python + CUDA". | ||
| matrix_filter: map(select(.ARCH == "amd64")) | max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]) | [.] |
There was a problem hiding this comment.
| # This selects "ARCH=amd64 + the latest supported Python + CUDA". | |
| matrix_filter: map(select(.ARCH == "amd64")) | max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]) | [.] |
I think we should cover the full RAPIDS PR CI matrix here.
Even though the dask-cuda wheels are pure-Python and not CPU architecture or CUDA-version dependent, with tags like this:
dask_cuda-25.2.0a10-py3-none-any.whl
... some of its dependencies do produce different variants for those different environments, and it's possible dask-cuda could interact with them in different ways that we'd want to catch. I'm reminded for example of microsoft/LightGBM#6509 and microsoft/LightGBM#6654, where I found that there was some aarch64-specific issue whose resolution was "load OpenMP as early as possible".
By the way, for PRs that matrix is only 2 jobs. You can find them here:
| @@ -0,0 +1,11 @@ | |||
| #!/bin/bash | |||
| # Copyright (c) 2023, NVIDIA CORPORATION. | |||
There was a problem hiding this comment.
| # Copyright (c) 2023, NVIDIA CORPORATION. | |
| # Copyright (c) 2024, NVIDIA CORPORATION. |
This is a brand new file, so the copyright date should be 2024. In a follow-up PR, could you add the verify-copyright pre-commit hook here to catch and fix stuff like this?
|
|
||
| set -eou pipefail | ||
|
|
||
| RAPIDS_PY_VERSION="312" RAPIDS_PY_WHEEL_NAME="dask-cuda" rapids-download-wheels-from-s3 ./dist |
There was a problem hiding this comment.
| RAPIDS_PY_VERSION="312" RAPIDS_PY_WHEEL_NAME="dask-cuda" rapids-download-wheels-from-s3 ./dist | |
| RAPIDS_PY_WHEEL_NAME="dask-cuda" RAPIDS_PY_WHEEL_PURE="1" rapids-download-wheels-from-s3 python ./dist |
Let's match the way this is spelled in other pure-Python cases like:
jupyterlab-nvdashboard: https://github.com/rapidsai/jupyterlab-nvdashboard/blob/d9746aefaab945241461046fb9d87334339c13a2/ci/test_wheel.shcudf-polars: https://github.com/rapidsai/cudf/blob/a95fbc88f94df24c3418766fbbea5b6633ff2328/ci/test_wheel_cudf_polars.sh#L9
Keeping these scripts looking similar is helpful when we do touch-all-the-repos types of migrations.
| - ucx-py-cu12==0.42.*,>=0.0.0a0 | ||
| - matrix: | ||
| cuda: "11.*" | ||
| cuda_suffixed: "true" | ||
| packages: | ||
| - cudf-cu11==25.2.*,>=0.0.0a0 | ||
| - dask-cudf-cu11==25.2.*,>=0.0.0a0 | ||
| - kvikio-cu11==25.2.*,>=0.0.0a0 |
Adding this now that wheels are available
Resolves #1344