Add PR binary graduation to the things spackbot can do #45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
scottwittenburg
force-pushed
the
graduate-pr-binaries
branch
2 times, most recently
from
4287e0f to
c5ecbd3
Compare
scottwittenburg
force-pushed
the
graduate-pr-binaries
branch
9 times, most recently
from
124c808 to
8d37c3f
Compare
scottwittenburg
force-pushed
the
graduate-pr-binaries
branch
from
8d37c3f to
01f30f4
Compare
scottwittenburg
force-pushed
the
graduate-pr-binaries
branch
from
01f30f4 to
13fb2b1
Compare
kwryankrattiger
added a commit
to kwryankrattiger/spackbot
that referenced
this pull request
Sep 30, 2022
Changes migrated from PR spack#45 using the new workers and a slightly more general name for the long running process queue. TODO: Add a pruning task that is occasionally inserted after a copy task but before reindex
Merged
kwryankrattiger
added a commit
to kwryankrattiger/spackbot
that referenced
this pull request
Oct 14, 2022
Changes migrated from PR spack#45 using the new workers and a slightly more general name for the long running process queue. TODO: Add a pruning task that is occasionally inserted after a copy task but before reindex
kwryankrattiger
added a commit
to kwryankrattiger/spackbot
that referenced
this pull request
Oct 21, 2022
Changes migrated from PR spack#45 using the new workers and a slightly more general name for the long running process queue. TODO: Add a pruning task that is occasionally inserted after a copy task but before reindex
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a new handler to spackbot routing to process spack PRs which have been merged to
develop. When these events happen, we want to copy the binaries from the per-PR mirror used by gitlab pipelines run against the PR to a shared PR mirror. The shared PR mirror can then be used by other PR pipelines that come along before thedeveloppipeline (associated with the aforementioned merged PR) finishes populating the main mirror with its own copies of those binaries.Because copying the binaries could take awhile (depending on the number of specs the PR had to rebuild), and the update of the shared PR mirror index could take a long time (depending on the number of binaries in the mirror), this PR uses the
rqmodule to schedule those tasks as jobs to be run by another process in a separate container, freeing up the handler to return quickly.I've tested this using the updated
docker-compose.ymland a webhook I set up on the spack repo to send events on pull requests. Because that's not the same as when a github app sends the requests, I temporarily commented out the installation authentication part for my testing. I'd love a better workflow for testing this part of spackbot without breaking it for everyone else though.I'm currently working on a change to
spack-infrastructureto update thespackbot-spack-iodeployment to reflect the extra containers spackbot will need to support doing PR binary graduation this way. See that PR to see how the deployment looks and how I'm testing it "in production".Remaining TODO: