Migrate away from google.com gcp project k8s-testimages #1523
Comments
k8s-ci-robot
added
wg/k8s-infra
sig/release
|
/milestone v1.21 |
spiffxp
added
the
priority/important-soon
|
I am open to suggestions on where we should move these images. I was thinking instead of a straight rename, take the time to break up the two classes:
Or, if we're just doing a lift-and-shift... then k8s-staging-test-infra |
|
@spiffxp -- Maybe a few different ownership levels to consider here:
In the shared access "tier" would be kubekins-e2e and maybe krte. I have approver on kubekins-e2e and I think this IAM group covers that case: k8s.io/groups/sig-release/groups.yaml Lines 20 to 34 in 35c5108 I think the closest existing staging project would be |
I agree with putting kubekins-e2e somewhere shared / under releng purview. I don't think it should be build-images though. Thinking toward applying policies for build provenance and security audit for the build chain of k8s releases. The kubekins-e2e image is an organically evolved mess, easier to keep it out of that repo than attempt to filter policy on certain image names. I'm thinking of:
My preference for shared repo would be releng, you've started moving some kubernetes job images there already IIRC But ci-images also sounds like a better name, so not a strong preference Hold on migrating kubekins until after code freeze but move on everything else, and see what else makes sense to move to shared from there WDYT? |
|
Ping @BenTheElder and @kubernetes/release-managers for comment |
|
Another bit of followup to consider, setup auto bumping of images used in jobs: kubernetes/test-infra#21137 |
|
I think we should continue to have CI images in a dedicated registry. They're not the same as, say, GCB images (e.g. docker in docker setup, bootstrap.py, you name it). I also think staging registries should map to a single git repo so it's easier to locate the git source for any given image. (And that's the pattern we have right now in k8s.io pretty consistently). CI images should continue to be pushed by automation, which is working well. We don't need to and should not grant humans push access (far less auditable than automation pushing from image sources in public git). Definitely do not move anything in the middle of code freeze, please. This is not a worthwhile diversion from reviewing code changes before freeze. |
|
Opened #1908. |
|
Followup of #1908, add a canary ProwJob that push a test-infra image. Kettle ? |
|
/milestone v1.23 I think it's perfectly acceptable to start pushing images to the staging project @ameukam setup and keep pushing to k8s-testimages for now. Maybe even start switching over some of the non-kubernetes/kubernetes jobs. But let's wait until after v1.22 releases to change images on the high traffic release-blocking / merge-blocking jobs. |
|
all missing repo is done now, waiting approval |
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
/remove-lifecycle stale |
k8s-ci-robot
removed
the
lifecycle/stale
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
k8s-ci-robot
added
lifecycle/rotten
|
/remove-lifecycle rotten |
k8s-ci-robot
removed
the
lifecycle/rotten
|
All of these images will go away when GCR shuts down ... there's currently no owner / intention to migrate to AR and I'm not sure we should. I think we should use this code search for the current list and burn it down until there are no results: It looks like two images still publish to k8s-testimages but I'm not sure we're actually using them at all. |
|
I suspect some of those are the image promoter's test images. I did a quick search and it still has references to gcr.io in the documentation and hardcoded everywhere 😢 I can check what is still current after Cloud Native SecurityCon (in about a week). |
|
We need to finish this ASAP, GCR shutdown => google projects turning down earlier => these images are at risk. I'm in a thread about if we can keep them but I have some OOO coming up and these remain a liability anyhow. |
|
List of images (or 'images') used in https://cs.k8s.io/?q=gcr.io%2Fk8s-testimages&i=nope&files=&excludeFiles=&repos=:
There's also some examples, tests, or docs mentions that aren't as relevant:
|
|
For comparison, the actual list of images in k8s-testimages:
A lot of these were migrated previously or as part of the move off the default build cluster in Prow and shouldn't be in use anymore. |
|
/milestone v1.32 |
|
We should ensure the one we need are pushed to Artifact Registry and drop the rest. /milestone clear |
Part of umbrella issue to migrate away from google.com gcp projects: #1469
At least some of this is part of the umbrella to migrate kubernetes e2e test images/registries to community-owned infrastructure: #1458
We should migrate away from the google.com-owned gcr.io/k8s-testimages repository and instead use a community-owned repository.
k8s-testimages hosts a variety of images that are built from source in kubernetes/test-infra. They fall broadly into two classes:
/wg k8s-infra
/sig release
/sig testing
/area release-eng
EDIT(spiffxp): Went through and exhaustively identified images that need to be migrated from the repo, or can be left behind.
Images that are used and need migration, have already migrated, or are unused and need source deleted:
Images that appear to be unused:
The text was updated successfully, but these errors were encountered: