Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stacktrace without a cause when using SharedInformerFactory without proper RBAC #3806

Open
wind57 opened this issue Nov 28, 2024 · 1 comment

Comments

@wind57
Copy link
Contributor

wind57 commented Nov 28, 2024

In spring-cloud-kubernetes, we are currently using version 19.0.1 of this library.

We have some code that looks like this:

	@PostConstruct
	void inform() {
		LOG.info(() -> "Kubernetes event-based secrets change detector activated");

		namespaces.forEach(namespace -> {
			SharedIndexInformer<V1Secret> informer;
			SharedInformerFactory factory = new SharedInformerFactory(apiClient);
			factories.add(factory);

			CallGenerator callGenerator = params -> {
				try {
					return coreV1Api.listNamespacedSecretCall(namespace, null, null, null, null, null, null,
							params.resourceVersion, null, null, params.timeoutSeconds, params.watch,
							new KubernetesClientApiCallback());
				}
				catch (Exception e) {
					LOG.error("Exception in secret shared informer");
					if (e instanceof ApiException apiException) {
						LOG.error(apiException.getResponseBody());
					}
					throw new RuntimeException(e);
				}
			};

			informer = factory.sharedIndexInformerFor(callGenerator, V1Secret.class, V1SecretList.class);
			informer.addEventHandler(handler);
			informers.add(informer);
			factory.startAllRegisteredInformers();
		});
	}

This works just fine when RBAC are correctly set-up, in our case :

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: watcher
rules:
  - apiGroups: ["", "extensions", "apps"]
    resources: ["configmaps", "pods", "services", "endpoints", "secrets"]
    verbs: ["get", "list", "watch"]

If I remove secrets from the resources, the API call should fail, which is expected. What is not expected is a lack of a proper message explaining why that happens. At the moment, all we get is:

listing and watching...
2024-11-28T07:58:32.154Z ERROR 1 --- [spring-cloud-kubernetes-configuration-watcher] [dels.V1Secret-1] i.k.c.informer.cache.ReflectorRunnable   : class io.kubernetes.client.openapi.models.V1Secret#Reflec
tor loop failed unexpectedly

io.kubernetes.client.openapi.ApiException: 
        at io.kubernetes.client.openapi.ApiClient.handleResponse(ApiClient.java:989) ~[client-java-api-19.0.1.jar:na]
        at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:905) ~[client-java-api-19.0.1.jar:na]
        at io.kubernetes.client.informer.SharedInformerFactory$1.list(SharedInformerFactory.java:271) ~[client-java-19.0.1.jar:na]
        at io.kubernetes.client.informer.cache.ReflectorRunnable.run(ReflectorRunnable.java:91) ~[client-java-19.0.1.jar:na]
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
        at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]

and that's it (no cause is provided), which makes it close to impossible for users to diagnose the issue.

@brendandburns
Copy link
Contributor

This is printed here:

log.error("{}#Reflector loop failed unexpectedly", watchingApiTypeClass, t);

We can consider improving the error message here.

In the meantime, you can inject your own error handler if you want better error messages, see:

BiConsumer<Class<ApiType>, Throwable> exceptionHandler) {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants