gitlabci with services parsed wrong #33056

Wurstnase · 2024-12-12T07:10:08Z

Wurstnase
Dec 12, 2024

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

GitLab (renovate/renovate:39@sha256:68af0e4f4375dcedda551b64d6270b79090edad5fdb23c939001bd657c3c78ab)

Please tell us more about your question or problem

Renovate try to parse an image where no image is.

When you have something like this in a gitlab-ci.yml:

my-job:
  services: !reference [.postgres, services]
  script:
    - echo foo

renovate try to find an image .postgres and will fail. It becomes worse, when the bot is behind a proxy and docker.io is not allowed.

Please check this minimal: minimal

Logs (if relevant)

The runner is in a private network and most internet traffic is forbidden.

Most important part is:

"dockerRepository": "library/services"
...
"dockerRepository": "library/.postgres"

Error without a proxy

DEBUG: HEAD https://index.docker.io/v2/library/services/manifests/latest = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=401 retryCount=0, duration=94) (repository=test-repo)
DEBUG: Unauthorized docker lookup (repository=test-repo)
       "registryHost": "https://index.docker.io",
       "dockerRepository": "library/services"
DEBUG: Response code 401 (Unauthorized) (repository=test-repo)
       "err": {
         "name": "HTTPError",
         "code": "ERR_NON_2XX_3XX_RESPONSE",
         "timings": {
           "start": 1733953012628,
           "socket": 1733953012629,
           "lookup": 1733953012629,
           "connect": 1733953012629,
           "secureConnect": 1733953012629,
           "upload": 1733953012629,
           "response": 1733953012722,
           "end": 1733953012722,
           "phases": {
             "wait": 1,
             "dns": 0,
             "tcp": 0,
             "tls": 0,
             "request": 0,
             "firstByte": 93,
             "download": 0,
             "total": 94
           }
         },
         "message": "Response code 401 (Unauthorized)",
         "stack": "HTTPError: Response code 401 (Unauthorized)\n    at Request.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at processTicksAndRejections (node:internal/process/task_queues:105:5)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
             "authorization": "***********",
             "accept": "application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://index.docker.io/v2/library/services/manifests/latest",
           "hostType": "docker",
           "username": "",
           "password": "",
           "method": "HEAD",
           "http2": false
         },
         "response": {
           "statusCode": 401,
           "statusMessage": "Unauthorized",
           "body": "",
           "headers": {
             "content-type": "application/json",
             "docker-distribution-api-version": "registry/2.0",
             "www-authenticate": "Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/services:pull\",error=\"insufficient_scope\"",
             "date": "Wed, 11 Dec 2024 21:36:52 GMT",
             "content-length": "159",
             "strict-transport-security": "max-age=31536000",
             "docker-ratelimit-source": "81.200.196.232"
           },
           "httpVersion": "1.1",
           "retryCount": 0
         }
       }
DEBUG: Could not determine new digest for update. (repository=test-repo)
       "packageName": "services",
       "currentValue": undefined,
       "datasource": "docker",
       "newValue": undefined,
       "bucket": undefined
DEBUG: HEAD https://index.docker.io/v2/library/.postgres/manifests/latest = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=307) (repository=test-repo)
DEBUG: Docker Manifest is unknown (repository=test-repo)
       "err": {
         "name": "HTTPError",
         "code": "ERR_NON_2XX_3XX_RESPONSE",
         "timings": {
           "start": 1733953012633,
           "socket": 1733953012634,
           "lookup": 1733953012634,
           "connect": 1733953012742,
           "secureConnect": 1733953012840,
           "upload": 1733953012840,
           "response": 1733953012940,
           "end": 1733953012940,
           "phases": {
             "wait": 1,
             "dns": 0,
             "tcp": 108,
             "tls": 98,
             "request": 0,
             "firstByte": 100,
             "download": 0,
             "total": 307
           }
         },
         "message": "Response code 404 (Not Found)",
         "stack": "HTTPError: Response code 404 (Not Found)\n    at Request.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at processTicksAndRejections (node:internal/process/task_queues:105:5)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
             "authorization": "***********",
             "accept": "application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://index.docker.io/v2/library/.postgres/manifests/latest",
           "hostType": "docker",
           "username": "",
           "password": "",
           "method": "HEAD",
           "http2": false
         },
         "response": {
           "statusCode": 404,
           "statusMessage": "Not Found",
           "body": "",
           "headers": {
             "content-type": "text/plain; charset=utf-8",
             "docker-distribution-api-version": "registry/2.0",
             "x-content-type-options": "nosniff",
             "date": "Wed, 11 Dec 2024 21:36:52 GMT",
             "content-length": "19",
             "strict-transport-security": "max-age=31536000",
             "docker-ratelimit-source": "81.200.196.234"
           },
           "httpVersion": "1.1",
           "retryCount": 0
         }
       },
       "registryHost": "https://index.docker.io",
       "dockerRepository": "library/.postgres",
       "tag": "latest"
DEBUG: Could not determine new digest for update. (repository=test-repo)
       "packageName": ".postgres",
       "currentValue": undefined,
       "datasource": "docker",
       "newValue": undefined,
       "bucket": undefined

Behind a proxy it was very hard to find. Maybe there is a chance to improve error messages behind a HTTP proxy, which normally has no https.

Logs behind the proxy

DEBUG: GET https://index.docker.io/v2/ = (code=EPROTO, statusCode=-1 retryCount=0, duration=74) (repository=iot-platform/gitlab-ci-cd/renovate/test-provisioner)
 WARN: Host error (repository=test-repo)
       "hostType": undefined,
       "packageName": undefined,
       "err": {
         "name": "RequestError",
         "code": "EPROTO",
         "timings": {
           "start": 1733953091988,
           "socket": 1733953092057,
           "lookup": 1733953092057,
           "connect": 1733953092057,
           "upload": 1733953092060,
           "error": 1733953092062,
           "phases": {"wait": 69, "dns": 0, "tcp": 0, "request": 3, "total": 74}
         },
         "message": "write EPROTO C00C199B9A7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354:\n",
         "stack": "RequestError: write EPROTO C00C199B9A7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354:\n\n    at ClientRequest.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/got/dist/source/core/index.js:970:111)\n    at Object.onceWrapper (node:events:633:26)\n    at ClientRequest.emit (node:events:530:35)\n    at ClientRequest.emit (node:domain:489:12)\n    at ClientRequest.origin.emit (/usr/local/renovate/node_modules/.pnpm/@[email protected]/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)\n    at emitErrorEvent (node:_http_client:103:11)\n    at TLSSocket.socketErrorListener (node:_http_client:506:5)\n    at TLSSocket.emit (node:events:530:35)\n    at TLSSocket.emit (node:domain:489:12)\n    at emitErrorNT (node:internal/streams/destroy:170:8)\n    at emitErrorCloseNT (node:internal/streams/destroy:129:3)\n    at processTicksAndRejections (node:internal/process/task_queues:90:21)\n    at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:87:19)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://index.docker.io/v2/",
           "hostType": "docker",
           "username": "",
           "password": "",
           "method": "GET",
           "http2": false
         }
       }

Answered by rarkins

Dec 12, 2024

Converted to issue: #33057

View full answer

Wurstnase · 2024-12-12T07:22:51Z

github-actions[bot]
bot Dec 12, 2024

Hi there,

A maintainer wants you to find and share a specific log message (packageFiles with updates) from your repository or environment.

If you self-host Renovate: make sure you run Renovate with LOG_LEVEL=debug to get the debug log messages! Next, open the debug-level logs and search for packageFiles with updates. This text marks the start of a structured log message that shows every package file, dependency, and update that Renovate found.

Find the relevant dependency/dependencies in the log message, and copy/paste those parts into this discussion. If you do not know which bits we need, you can copy/paste the full log message.

Read the Renovate docs, Troubleshooting to learn more about getting the docs, and getting the correct type of logs.

Thanks, the Renovate team

1 reply

Wurstnase Dec 12, 2024
Author

DEBUG: packageFiles with updates (repository=iot-platform/gitlab-ci-cd/renovate/test-provisioner, baseBranch=main)
       "config": {
         "gitlabci": [
           {
             "packageFile": ".gitlab-ci.yml",
             "deps": [
               {
                 "depName": ".custom",
                 "replaceString": ".custom",
                 "autoReplaceStringTemplate": "{{depName}}{{#if newValue}}:{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}",
                 "datasource": "docker",
                 "depType": "service-image",
                 "updates": [],
                 "packageName": ".custom",
                 "versioning": "docker",
                 "warnings": []
               },
               {
                 "depName": "services",
                 "replaceString": "services",
                 "autoReplaceStringTemplate": "{{depName}}{{#if newValue}}:{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}",
                 "datasource": "docker",
                 "depType": "service-image",
                 "updates": [],
                 "packageName": "services",
                 "versioning": "docker",
                 "warnings": []
               }
             ]
           }
         ]
       }

rarkins · 2024-12-12T08:26:39Z

rarkins
Dec 12, 2024
Maintainer

The syntax from your post:

my-job:
  services: !reference [.postgres, services]
  script:
    - echo foo

And as you note, we then try to look up a docker package .postgres.

What reasoning should Renovate use to not do that? e.g. if it was without the dot:

my-job:
  services: !reference [postgres, services]
  script:
    - echo foo

Would then it be valid for Renovate to look up postgres and services? Or should everything inside !reference [ ... ] be ignored, etc?

I think we could apply a filter to drop any docker packageName starting with . because it's an invalid image name, but wondering if our filter should be applied much earlier/widely.

1 reply

Wurstnase Dec 12, 2024
Author

everything in the !reference should be ignored. It is a special construct for gitlab.

https://docs.gitlab.com/ee/ci/yaml/yaml_optimization.html#reference-tags

rarkins · 2024-12-12T09:22:52Z

rarkins
Dec 12, 2024
Maintainer

Converted to issue: #33057

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gitlabci with services parsed wrong #33056

{{title}}

Replies: 3 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

gitlabci with services parsed wrong #33056

Wurstnase Dec 12, 2024

How are you running Renovate?

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Please tell us more about your question or problem

Logs (if relevant)

Replies: 3 comments · 2 replies

github-actions[bot] bot Dec 12, 2024

Wurstnase Dec 12, 2024 Author

rarkins Dec 12, 2024 Maintainer

Wurstnase Dec 12, 2024 Author

rarkins Dec 12, 2024 Maintainer

Wurstnase
Dec 12, 2024

Replies: 3 comments 2 replies

github-actions[bot]
bot Dec 12, 2024

Wurstnase Dec 12, 2024
Author

rarkins
Dec 12, 2024
Maintainer

Wurstnase Dec 12, 2024
Author

rarkins
Dec 12, 2024
Maintainer