Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the number of CI tests #2020

Open
certik opened this issue May 19, 2024 · 2 comments
Open

Reduce the number of CI tests #2020

certik opened this issue May 19, 2024 · 2 comments

Comments

@certik
Copy link
Contributor

certik commented May 19, 2024

Right now there are way too many CI tests (34), that increase the maintenance burden. For example not all of them are properly "required" in a PR, some are not required, and some other once are required but not present. It's hard to maintain all of them in Settings and we already can't keep up. So we need to fix it by reducing the number of total tests to about 10, which is manageable.

Rather, one should test the representative build on each platform/compiler combination that we want to support. And then on the fastest runner (say Ubuntu 22) test the rest of the cmake options and optional dependencies combinations.

Ideally 10 total tests or something like that. Right now there are 34 tests.

@rikardn
Copy link
Contributor

rikardn commented May 20, 2024

I agree. And having all tests run on GHA would also reduce the maintenance burden. Currently a handful still run on appveyor.

@lkeegan
Copy link
Member

lkeegan commented Sep 18, 2024

I agree with simplifying the CI tests - in the meantime @oliverlee suggested a nice solution to avoid having to update the list of required tests every time the CI is updated, by having a check CI job that passes if all other jobs pass and fails otherwise. Then only the check job needs to be required.
I made a PR here: #2049

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants