Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_cloud_events_player_example fails waiting on ksvc. #264

Open
orfeas-k opened this issue Dec 10, 2024 · 2 comments
Open

test_cloud_events_player_example fails waiting on ksvc. #264

orfeas-k opened this issue Dec 10, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@orfeas-k
Copy link
Contributor

orfeas-k commented Dec 10, 2024 •
edited by mvlassis
Loading

Bug Description

After bumping the dependencies in #259, the aforementioned test fails waiting on the ksvc https://github.com/canonical/knative-operators/actions/runs/12257700684/job/34195774643

INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
FAILED

To Reproduce

rerun the PR's CI

Environment

juju 3.6.0, microk8s 1.29

Relevant Log Output

============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-8.3.4, pluggy-1.5.0 -- /home/runner/work/knative-operators/knative-operators/.tox/integration/bin/python
cachedir: .tox/integration/.pytest_cache
rootdir: /home/runner/work/knative-operators/knative-operators
configfile: pyproject.toml
plugins: operator-0.38.0, asyncio-0.21.2, anyio-4.5.2
asyncio: mode=strict
collecting ... collected 7 items

tests/test_bundle.py::test_build_deploy_knative_charms 
-------------------------------- live log setup --------------------------------
INFO     pytest_operator.plugin:plugin.py:766 Connecting to existing model github-pr-c4238-microk8s:knative-test on unspecified cloud
-------------------------------- live log call ---------------------------------
INFO     pytest_operator.plugin:plugin.py:583 Using tmp_path: /home/runner/work/knative-operators/knative-operators/.tox/integration/tmp/pytest/knative-test0
INFO     pytest_operator.plugin:plugin.py:1139 Building charm knative-operator
INFO     pytest_operator.plugin:plugin.py:583 Using tmp_path: /home/runner/work/knative-operators/knative-operators/.tox/integration/tmp/pytest/knative-test0
INFO     pytest_operator.plugin:plugin.py:1139 Building charm knative-serving
INFO     pytest_operator.plugin:plugin.py:583 Using tmp_path: /home/runner/work/knative-operators/knative-operators/.tox/integration/tmp/pytest/knative-test0
INFO     pytest_operator.plugin:plugin.py:1139 Building charm knative-eventing
INFO     pytest_operator.plugin:plugin.py:1144 Built charm knative-serving in 1492.52s
INFO     pytest_operator.plugin:plugin.py:1144 Built charm knative-eventing in 1501.63s
INFO     pytest_operator.plugin:plugin.py:1144 Built charm knative-operator in 1562.73s
INFO     juju.model:model.py:2284 Deploying ch:amd64/focal/istio-pilot-1246
INFO     juju.model:model.py:2284 Deploying ch:amd64/focal/istio-gateway-1298
WARNING  juju.model:model.py:1666 relate is deprecated and will be removed. Use integrate instead.
INFO     juju.model:model.py:3211 Waiting for model:
  istio-pilot/0 [allocating] waiting: installing agent
  istio-ingressgateway/0 [allocating] waiting: installing agent
INFO     juju.model:model.py:3211 Waiting for model:
  istio-pilot/0 [executing] active: 
  istio-ingressgateway/0 [executing] waiting: List of <ops.model.Relation istio-pilot:1> versions not found for apps: istio-pilot
INFO     juju.model:model.py:2284 Deploying local:knative-operator-0
INFO     juju.model:model.py:3211 Waiting for model:
  knative-operator/0 [allocating] waiting: installing agent
INFO     juju.model:model.py:3211 Waiting for model:
  knative-operator/0 [idle] active: 
INFO     juju.model:model.py:2284 Deploying local:knative-serving-0
INFO     juju.model:model.py:2284 Deploying local:knative-eventing-0
INFO     juju.model:model.py:3211 Waiting for model:
  knative-serving/0 [idle] waiting: agent initialising
  knative-eventing/0 [allocating] waiting: installing agent
INFO     juju.model:model.py:3211 Waiting for model:
  knative-eventing/0 [idle] active: 
PASSED
tests/test_bundle.py::test_cloud_events_player_example 
-------------------------------- live log setup --------------------------------
INFO     test_bundle:test_bundle.py:146 Waiting on ksvc to exist
INFO     test_bundle:test_bundle.py:149 Checking for ksvc CRD
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/services.serving.knative.dev "HTTP/1.1 200 OK"
-------------------------------- live log call ---------------------------------
INFO     httpx:_client.py:1038 HTTP Request: POST https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services "HTTP/1.1 201 Created"
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:198 Waiting on ksvc for status to be available
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
INFO     test_bundle:test_bundle.py:191 Checking ksvc status
INFO     httpx:_client.py:1038 HTTP Request: GET https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"
INFO     test_bundle:test_bundle.py:202 Waiting on ksvc with conditions [] to be Ready
FAILED
------------------------------ live log teardown -------------------------------
INFO     httpx:_client.py:1038 HTTP Request: DELETE https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/cloudevents-player "HTTP/1.1 200 OK"

tests/test_bundle.py::test_eventing_custom_image 
-------------------------------- live log setup --------------------------------
WARNING  juju.client.connection:connection.py:490 Receiver: Connection closed, reconnecting
WARNING  juju.client.connection:connection.py:579 RPC: Connection closed, reconnecting
WARNING  juju.client.connection:connection.py:579 RPC: Connection closed, reconnecting
ERROR    asyncio:base_events.py:1707 Task exception was never retrieved
future: <Task finished name='Task-3308' coro=<Connection.reconnect() done, defined at /home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py:697> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 709, in reconnect
    res = await connector(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 800, in _connect_with_login
    await self._connect(endpoints)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 743, in _connect
    result = await task
  File "/usr/lib/python3.8/asyncio/tasks.py", line 619, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 729, in _try_endpoint
    return await self._open(endpoint, cacert)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 330, in _open
    await websockets.connect(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/client.py", line 654, in __await_impl__
    _transport, protocol = await self._create_connection()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1066, in _create_connection_transport
    sock.setblocking(False)
OSError: [Errno 9] Bad file descriptor
ERROR    juju.client.connection:connection.py:587 RPC: Automatic reconnect failed
ERROR    asyncio:base_events.py:1707 Task exception was never retrieved
future: <Task finished name='Task-3311' coro=<Connection.reconnect() done, defined at /home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py:697> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 709, in reconnect
    res = await connector(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 800, in _connect_with_login
    await self._connect(endpoints)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 743, in _connect
    result = await task
  File "/usr/lib/python3.8/asyncio/tasks.py", line 619, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 729, in _try_endpoint
    return await self._open(endpoint, cacert)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 330, in _open
    await websockets.connect(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/client.py", line 654, in __await_impl__
    _transport, protocol = await self._create_connection()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1066, in _create_connection_transport
    sock.setblocking(False)
OSError: [Errno 9] Bad file descriptor
ERROR    juju.client.connection:connection.py:587 RPC: Automatic reconnect failed
ERROR
tests/test_bundle.py::test_serving_custom_image 
-------------------------------- live log setup --------------------------------
WARNING  juju.client.connection:connection.py:579 RPC: Connection closed, reconnecting
ERROR    juju.client.connection:connection.py:587 RPC: Automatic reconnect failed
ERROR
tests/test_bundle.py::test_ksvc_deployment_configs 
-------------------------------- live log call ---------------------------------
WARNING  juju.client.connection:connection.py:579 RPC: Connection closed, reconnecting
ERROR    juju.client.connection:connection.py:587 RPC: Automatic reconnect failed
FAILED
------------------------------ live log teardown -------------------------------
INFO     httpx:_client.py:1038 HTTP Request: DELETE https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/helloworld "HTTP/1.1 404 Not Found"

tests/test_bundle.py::test_ksvc_deployment_configs ERROR
tests/test_bundle.py::test_queue_sidecar_image_config 
-------------------------------- live log setup --------------------------------
WARNING  juju.client.connection:connection.py:579 RPC: Connection closed, reconnecting
ERROR    juju.client.connection:connection.py:587 RPC: Automatic reconnect failed
ERROR
tests/test_bundle.py::test_serving_proxy_config 
-------------------------------- live log call ---------------------------------
WARNING  juju.client.connection:connection.py:579 RPC: Connection closed, reconnecting
ERROR    juju.client.connection:connection.py:587 RPC: Automatic reconnect failed
FAILED
------------------------------ live log teardown -------------------------------
INFO     pytest_operator.plugin:plugin.py:903 Model status:

Model         Controller                Cloud/Region        Version  SLA          Timestamp
knative-test  github-pr-c4238-microk8s  microk8s/localhost  3.6.0    unsupported  14:40:19Z

App                   Version  Status       Scale  Charm             Channel       Rev  Address         Exposed  Message
istio-ingressgateway           active           1  istio-gateway     latest/edge  1298  10.152.183.85   no       
istio-pilot                    maintenance    0/1  istio-pilot       latest/edge  1246  10.152.183.155  no       
knative-eventing               active           1  knative-eventing                  0  10.152.183.153  no       
knative-operator               maintenance    0/1  knative-operator                  0  10.152.183.31   no       
knative-serving                active           1  knative-serving                   0  10.152.183.66   no       

Unit                     Workload  Agent  Address      Ports  Message
istio-ingressgateway/0*  active    idle   10.1.139.75         
istio-pilot/0            unknown   lost                       agent lost, see 'juju show-status-log istio-pilot/0'
knative-eventing/0*      active    idle   10.1.139.80         
knative-operator/0       unknown   lost                       agent lost, see 'juju show-status-log knative-operator/0'
knative-serving/0*       active    idle   10.1.139.79         

INFO     pytest_operator.plugin:plugin.py:909 Juju error logs:


INFO     pytest_operator.plugin:plugin.py:991 Forgetting model main...


==================================== ERRORS ====================================
_________________ ERROR at setup of test_eventing_custom_image _________________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in transfer_data
    message = await self.read_message()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1023, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1098, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1155, in read_frame
    frame = await Frame.read(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/framing.py", line 70, in read
    data = await reader(2)
  File "/usr/lib/python3.8/asyncio/streams.py", line 723, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/lib/python3.8/asyncio/streams.py", line 517, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_setup
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 840, in pytest_runtest_setup
    yield from self._runtest_for(item, "setup")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 875, in pytest_runtest_setup
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_setup
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 160, in pytest_runtest_setup
    item.session._setupstate.setup(item)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 514, in setup
    col.setup()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 1630, in setup
    self._request._fillfixtures()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 697, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 532, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 617, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1091, in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1140, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 898, in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 302, in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 284, in setup
    res = await gen_obj.__anext__()
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 242, in restore_eventing_custom_image_settings
    custom_image_config = (await ops_test.model.applications["knative-eventing"].get_config())[
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/application.py", line 465, in get_config
    return (await app_facade.Get(application=self.name)).config
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
    reply = await f(*args, **kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client20.py", line 381, in Get
    reply = await self.rpc(msg)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
    result = await self.connection.rpc(msg, encoder=TypeEncoder)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 574, in rpc
    await self._ws.send(outgoing)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 628, in send
    await self.ensure_open()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 938, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
_________________ ERROR at setup of test_serving_custom_image __________________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in transfer_data
    message = await self.read_message()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1023, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1098, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1155, in read_frame
    frame = await Frame.read(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/framing.py", line 70, in read
    data = await reader(2)
  File "/usr/lib/python3.8/asyncio/streams.py", line 723, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/lib/python3.8/asyncio/streams.py", line 517, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_setup
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 840, in pytest_runtest_setup
    yield from self._runtest_for(item, "setup")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 875, in pytest_runtest_setup
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_setup
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 160, in pytest_runtest_setup
    item.session._setupstate.setup(item)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 514, in setup
    col.setup()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 1630, in setup
    self._request._fillfixtures()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 697, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 532, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 617, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1091, in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1140, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 898, in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 302, in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 284, in setup
    res = await gen_obj.__anext__()
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 286, in restore_serving_custom_image_settings
    custom_image_config = (await ops_test.model.applications["knative-serving"].get_config())[
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/application.py", line 465, in get_config
    return (await app_facade.Get(application=self.name)).config
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
    reply = await f(*args, **kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client20.py", line 381, in Get
    reply = await self.rpc(msg)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
    result = await self.connection.rpc(msg, encoder=TypeEncoder)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 574, in rpc
    await self._ws.send(outgoing)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 628, in send
    await self.ensure_open()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 929, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
______________ ERROR at teardown of test_ksvc_deployment_configs _______________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/lightkube/core/generic_client.py", line 235, in raise_for_status
    resp.raise_for_status()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/httpx/_models.py", line 763, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://10.1.0.101:16443/apis/serving.knative.dev/v1/namespaces/knative-test/services/helloworld'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 174, in remove_helloworld_example
    lightkube_client.delete(KSVC, "helloworld", namespace=ops_test.model_name)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/lightkube/core/client.py", line 143, in delete
    return self._client.request(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/lightkube/core/generic_client.py", line 311, in request
    return self.handle_response(method, resp, br)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/lightkube/core/generic_client.py", line 251, in handle_response
    self.raise_for_status(resp)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/lightkube/core/generic_client.py", line 237, in raise_for_status
    raise transform_exception(e)
lightkube.core.exceptions.ApiError: services.serving.knative.dev "helloworld" not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 97, in pytest_runtest_teardown
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 100, in pytest_runtest_teardown
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 853, in pytest_runtest_teardown
    yield from self._runtest_for(item, "teardown")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 885, in pytest_runtest_teardown
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 189, in pytest_runtest_teardown
    item.session._setupstate.teardown_exact(nextitem)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 557, in teardown_exact
    raise exceptions[0]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 546, in teardown_exact
    fin()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1032, in finish
    raise exceptions[0]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1021, in finish
    fin()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 907, in _teardown_yield_fixture
    next(it)
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 177, in remove_helloworld_example
    if e.code == 404:
AttributeError: 'ApiError' object has no attribute 'code'
______________ ERROR at setup of test_queue_sidecar_image_config _______________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in transfer_data
    message = await self.read_message()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1023, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1098, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1155, in read_frame
    frame = await Frame.read(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/framing.py", line 70, in read
    data = await reader(2)
  File "/usr/lib/python3.8/asyncio/streams.py", line 723, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/lib/python3.8/asyncio/streams.py", line 517, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_setup
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 840, in pytest_runtest_setup
    yield from self._runtest_for(item, "setup")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 875, in pytest_runtest_setup
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_setup
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 160, in pytest_runtest_setup
    item.session._setupstate.setup(item)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 514, in setup
    col.setup()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 1630, in setup
    self._request._fillfixtures()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 697, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 532, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 617, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1091, in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 1140, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/fixtures.py", line 898, in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 302, in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 284, in setup
    res = await gen_obj.__anext__()
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 399, in restore_queue_sidecar_image_setting
    await ops_test.model.applications["knative-serving"].get_config()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/application.py", line 465, in get_config
    return (await app_facade.Get(application=self.name)).config
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
    reply = await f(*args, **kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client20.py", line 381, in Get
    reply = await self.rpc(msg)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
    result = await self.connection.rpc(msg, encoder=TypeEncoder)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 574, in rpc
    await self._ws.send(outgoing)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 628, in send
    await self.ensure_open()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 929, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
=================================== FAILURES ===================================
_______________________ test_cloud_events_player_example _______________________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 880, in pytest_runtest_call
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 529, in inner
    _loop.run_until_complete(task)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 217, in test_cloud_events_player_example
    wait_for_ready(resource=KSVC, name="cloudevents-player", namespace=ops_test.model_name)
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 189, in wait_for_ready
    for attempt in RETRY_FOR_THREE_MINUTES:
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/tenacity/__init__.py", line 443, in __iter__
    do = self.iter(retry_state=retry_state)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/tenacity/__init__.py", line 418, in exc_check
    raise retry_exc.reraise()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/tenacity/__init__.py", line 185, in reraise
    raise self.last_attempt.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 206, in wait_for_ready
    raise timeout_error
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 206, in wait_for_ready
    raise timeout_error
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 206, in wait_for_ready
    raise timeout_error
  [Previous line repeated 33 more times]
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 199, in wait_for_ready
    raise timeout_error
TimeoutError: Timed out waiting for ksvc to be ready
_________________________ test_ksvc_deployment_configs _________________________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in transfer_data
    message = await self.read_message()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1023, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1098, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1155, in read_frame
    frame = await Frame.read(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/framing.py", line 70, in read
    data = await reader(2)
  File "/usr/lib/python3.8/asyncio/streams.py", line 723, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/lib/python3.8/asyncio/streams.py", line 517, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 880, in pytest_runtest_call
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
Task exception was never retrieved
future: <Task finished name='Task-3310' coro=<Connection._connect.<locals>._try_endpoint() done, defined at /home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py:724> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 729, in _try_endpoint
    return await self._open(endpoint, cacert)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 330, in _open
    await websockets.connect(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/client.py", line 654, in __await_impl__
    _transport, protocol = await self._create_connection()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1066, in _create_connection_transport
    sock.setblocking(False)
OSError: [Errno 9] Bad file descriptor
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 529, in inner
    _loop.run_until_complete(task)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 336, in test_ksvc_deployment_configs
    await ops_test.model.applications["knative-serving"].set_config(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/application.py", line 692, in set_config
    return await app_facade.SetConfigs(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
    reply = await f(*args, **kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client20.py", line 676, in SetConfigs
    reply = await self.rpc(msg)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
    result = await self.connection.rpc(msg, encoder=TypeEncoder)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 574, in rpc
    await self._ws.send(outgoing)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 628, in send
    await self.ensure_open()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 929, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
__________________________ test_serving_proxy_config ___________________________
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in transfer_data
    message = await self.read_message()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1023, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1098, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 1155, in read_frame
    frame = await Frame.read(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/framing.py", line 70, in read
    data = await reader(2)
  File "/usr/lib/python3.8/asyncio/streams.py", line 723, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/lib/python3.8/asyncio/streams.py", line 517, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/capture.py", line 880, in pytest_runtest_call
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 529, in inner
    _loop.run_until_complete(task)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/runner/work/knative-operators/knative-operators/tests/test_bundle.py", line 464, in test_serving_proxy_config
    await ops_test.model.applications["knative-serving"].set_config(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/application.py", line 692, in set_config
    return await app_facade.SetConfigs(
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
    reply = await f(*args, **kwargs)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client20.py", line 676, in SetConfigs
    reply = await self.rpc(msg)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
    result = await self.connection.rpc(msg, encoder=TypeEncoder)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 574, in rpc
    await self._ws.send(outgoing)
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 628, in send
    await self.ensure_open()
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 929, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
=========================== short test summary info ============================
FAILED tests/test_bundle.py::test_cloud_events_player_example - TimeoutError: Timed out waiting for ksvc to be ready
FAILED tests/test_bundle.py::test_ksvc_deployment_configs - websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
FAILED tests/test_bundle.py::test_serving_proxy_config - websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
ERROR tests/test_bundle.py::test_eventing_custom_image - websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
ERROR tests/test_bundle.py::test_serving_custom_image - websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
ERROR tests/test_bundle.py::test_ksvc_deployment_configs - AttributeError: 'ApiError' object has no attribute 'code'
ERROR tests/test_bundle.py::test_queue_sidecar_image_config - websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
============== 3 failed, 1 passed, 4 errors in 2040.62s (0:34:00) ==============
Task was destroyed but it is pending!
task: <Task pending name='Task-3301' coro=<Connection._pinger.<locals>._do_ping() running at /home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py:514> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f66568fc1c0>()]> cb=[create_task_with_handler.<locals>._task_result_exp_handler(task_name='tmp', logger=<Logger juju....ion (WARNING)>)() at /home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/jasyncio.py:101]>
Exception ignored in: <coroutine object Connection._pinger.<locals>._do_ping at 0x7f66571724c0>
Traceback (most recent call last):
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 514, in _do_ping
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client1.py", line 410, in Ping
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 589, in rpc
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 382, in _recv
  File "/home/runner/work/knative-operators/knative-operators/.tox/integration/lib/python3.8/site-packages/juju/utils.py", line 113, in get
  File "/usr/lib/python3.8/asyncio/queues.py", line 165, in get
  File "/usr/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
  File "/usr/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
RuntimeError: Event loop is closed
integration: 2059615 C exit 1 (2041.50 seconds) /home/runner/work/knative-operators/knative-operators> pytest --show-capture=no --log-cli-level=INFO -vvs --tb=native --model knative-test tests/test_bundle.py pid=16220 [tox/execute/api.py:286]
  integration: FAIL code 1 (2059.24=setup[17.75]+cmd[2041.50] seconds)
  evaluation failed :( (2059.42 seconds)

The logs of the action can be found here

Additional Context

No response
@orfeas-k orfeas-k added the bug Something isn't working label Dec 10, 2024
@syncronize-issues-to-jira Syncronize issues to Jira
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6666.

This message was autogenerated

@mvlassis mvlassis mentioned this issue Dec 17, 2024
Open
@NohaIhab
Copy link
Contributor

This is currently blocking the CI in #265 and #259 , and we need to fix it before the 1.10 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@orfeas-k @NohaIhab