-
Notifications
You must be signed in to change notification settings - Fork 534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add option to restore cache only #316
Comments
Hello @ForsakenHarmony! Thank you for the suggested idea! |
@ForsakenHarmony likely seen this already, but if not you might find this helpful too to automatically delete PR caches when the PR is closed: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries |
@gwatts yes, I'm doing that, doing it after each commit's CI is done actually, but it's an ugly workaround |
Hello @ForsakenHarmony, if i understood your request correctly you need the "restore cache only" action https://github.com/actions/cache/blob/main/restore/README.md Does it solve the problem? |
We would love this feature too. Our CI pipeline consist of:
When cache is enabled (default in v4) and there is a change in We would like to configure test job to "download only", basically ability to disable the setup-go post hook uploading the cache. |
How do I use |
Hello @lzap, |
Unfortunately, I am having hard time figuring out how setup-go creates the cache key. The SHA sums for my case do not match for some reason: setup-go-Linux-go-1.19.6-2ecd730a3adf2d5167ee1676ce4793359350a5aef4bc40807996837393159830 It does not match:
Are you saying that I need to completely disable go caching and write caching myself (both store and retrieve)? That is exactly what I don’t want to do and the primary reason why I 👍🏻 on this issue. Frankly either a key prefix/suffix or some sort of "lookup-only" flag that would be passed into the cache action would be really cool. I don’t know TS/JS so crossing my fingers this gets picked up sometime. Not a big deal for our project right now, we disabled caching for some actions for now. |
hello @lzap This issue requests new feature "Add option to restore cache only". It won't be implemented because it duplicates the functionality of the existing action: https://github.com/actions/cache/blob/main/restore/README.md You should disable basic caching for setup-go https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs and add a step with restore only action https://github.com/actions/cache/blob/main/restore/README.md As a result the workflow should contain something like this:
does it help? |
Hey thanks for comment, I understand, however, let me argue that the behavior is not documented at the moment. I had hard time figuring out how hash is actually calculated, I had no idea that both files are concatenated and in this order. Maybe this issue could be turned into documentation change. Still I think this could be useful feature for couple of reasons. A single option is much easier to write than additional 5 lines long extra action. It is not just that, please realize that we need this functionality when we have multiple parallel actions which "fight" each other for the initial cache save, so it is not just 5 lines, but 20 lines of copy-pasted YAML for my case (4 actions - build, unit tests, integration tests and linter). It does not have to be restore cache attribute, this could be easily implemented as cache prefix/suffix attribute which would allow us to create separate caches for various actions which interfere witch each other. Finally, when setup-go action authors decide to change the key pattern, it will break others pipelines and it will be pain to figure it out. A consistency is my last argument. And with that, I will, again, thank you for your comment and effort you put into this project. I will try to implement what you suggested now. |
Oh I just learned that it is not just 5 lines, what you pasted is not functional example. I need to figure out path to dependencies that is another action. Also the key does not match what I see. I guess I will keep cache disabled for tests in our case :-( |
Just for googlers, the recommended way of workarounding this problem looks like this (pseudo code, untested). You need to repeat it for every action that needs to fetch cache: - uses: actions/setup-go@v4
with:
go-version: "1.20.x"
cache: false
- name: Get Go environment
run: |
echo "cache=$(go env GOCACHE)" >> $GITHUB_ENV
echo "modcache=$(go env GOMODCACHE)" >> $GITHUB_ENV
- name: Set up cache
uses: actions/cache@v3
with:
path: |
${{ env.cache }}
${{ env.modcache }}
key: setup-go-${{ runner.os }}-go-GOVERSION-${{ hashFiles('go.sum go.mod') }}
restore-keys: |
setup-go-${{ runner.os }}-go- |
Hello @lzap , The creating cache is documened here https://github.com/actions/cache/blob/main/README.md#creating-a-cache-key (it is followed by link 'See creating a cache key` right there https://github.com/actions/cache/blob/main/restore/README.md#inputs For multiple actions probably it is better to utilize matrix feature introduced to simplify parallel runs |
You linked documentation of the cache action, that is not what I mean. This project ( As much as I would love to dig in and find an alternative solution to "copying actions/cache", I don't understand JS/TS stuff. Well, thanks anyway! Cheers. |
Thank you @lzap , i've got your point and now i see why @actions/restore is not the adequate replacement of |
steps:
|
To work around this, you'll need to order the path to match what - uses: actions/setup-go@v4
with:
go-version-file: 'go.mod'
cache: ${{ github.ref == 'refs/heads/main' }} # only update the cache in main.
- name: Prepare for go cache
if: ${{ github.ref != 'refs/heads/main' }}
run: |
echo "GO_CACHE=$(go env GOCACHE)" | tee -a "$GITHUB_ENV"
echo "GO_MODCACHE=$(go env GOMODCACHE)" | tee -a "$GITHUB_ENV"
echo "GO_VERSION=$(go env GOVERSION | tr -d 'go')" | tee -a "$GITHUB_ENV"
- name: Setup read-only cache
if: ${{ github.ref != 'refs/heads/main' }}
uses: actions/cache/restore@v3
with:
path: |
${{ env.GO_MODCACHE }}
${{ env.GO_CACHE }}
key: setup-go-${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('go.sum') }}
restore-keys: |
setup-go-${{ runner.os }}- I noticed some runners will include the |
GitHub cache artifacts are immutable, so the first job to write to the cache "wins" in the sense that all subsequent jobs will start with whatever that job happened to cache. We currently use `setup-go` in the license check action, but this job is very quick (typically a couple minutes) and doesn't really build any artifacts or download modules. Subsequent jobs then need to re-download and re-compile all the provider's modules. This PR disables caching for the license check job. This gives the `prerequisites` job an opportunity to more fully populate the cache. Ideally we would only disable _writing_ to the cache, but there isn't an official way to do that yet (actions/setup-go#316) and manually restoring the cache is fragile.
Description:
It would be nice to have the option to restore only and not save.
Justification:
In bigger repos, it's easy to hit the 10gb cache limit. Not caching in PRs can help with this.
Right now, I have to manually delete caches to help keep the caches on
main
alive.Are you willing to submit a PR?
References
actions/toolkit#1308
Swatinem/rust-cache#95 (has a
save-if
option https://github.com/Swatinem/rust-cache#example-usage)The text was updated successfully, but these errors were encountered: