Spring Boot Admin on Kubernetes with DiscoveryClient for Kubernetes not showing any services

[tspoeri]

Spring Boot Admin Server information

• Version: 3.1.4
• Spring Boot version: 3.1.2
• Configured Security: Basic Auth

Client information

• Spring Boot versions: 3.1.2
• Used discovery mechanism: org.springframework.cloud:spring-cloud-starter-kubernetes-discoveryclient:3.0.3

Description

I have microservices deployed on a Kubernetes cluster. The Kubernetes service registry is exposed through Spring Cloud Kubernetes Discovery Server (docker image springcloud/spring-cloud-kubernetes-discoveryserver:3.1.0-RC1). The clients including the SBA are using the dependency org.springframework.cloud:spring-cloud-starter-kubernetes-discoveryclient:3.0.3 to discover the other services.

To verify that the environment works properly, I checked the health endpoints of my microservices where in the details of the "discoveryComposite" data structure all services are properly listed, except for the SBA where the list of services is empty.

With this setup, the Spring Boot Admin UI is not showing any registered services.

I think the problem is, that the de.codecentric.boot.admin.server.cloud.discovery.InstanceDiscoveryListener only uses instances of org.springframework.cloud.client.discovery.DiscoveryClient for resolving the registered services. But since SBA is using webflux libraries, an instance of org.springframework.cloud.client.discovery.ReactiveDiscoveryClient is created by the autoconfiguration, but SBA does not use this client to resolve services.

[erikpetzold]

Hi @tspoeri ,

Spring Boot Admin is supporting spring-cloud-starter-kubernetes-fabric8 and spring-cloud-starter-kubernetes-client which both support k8s service discovery without the spring-cloud-kubernetes-discoveryserver.
See https://docs.spring.io/spring-cloud-kubernetes/reference/discovery-client.html
There are also executable examples in https://github.com/codecentric/spring-boot-admin-discovery-playground/tree/main/kubernetes.

We will have a look at spring-cloud-kubernetes-discoveryserver and spring-cloud-starter-kubernetes-discoveryclient and try to add an example there + implement missing features. At the moment we cannot provide a timeline when this will be fixed. If you can provide a pull request we would be happy to review and merge that.

[erikpetzold]

For anyone interested: Some more background information why the discoveryserver was introduced: https://spring.io/blog/2021/10/26/new-features-for-spring-cloud-kubernetes-in-spring-cloud-2021-0-0-m3

This "why/when should I use this" is missing in the docs: https://docs.spring.io/spring-cloud-kubernetes/reference/spring-cloud-kubernetes-discoveryserver.html

We should support this, most of the features should come from spring cloud itself. Maybe it is really just DiscoveryClient vs. ReactiveDiscoveryClient.

[ouaibsky]

Hey
I confirm we have same issue.

[erikpetzold]

Hi, I extended our discovery playground to contain the spring-cloud-kubernetes-discoveryserver and added a spring-boot-admin using it: https://github.com/codecentric/spring-boot-admin-discovery-playground/tree/main/kubernetes

• start traefik
• start hello world
• start discovery server
• start spring-boot-admin-discoveryclient

This is working fine with reactive and servlet version of spring boot admin.

However I can confirm that I saw the message saying that some reactive version of client is missing during development. But with this the app even failed to start. I guess this was related to the used versions, it has probably gone with updating dependencies to most recent version. So maybe this was fixed in spring cloud.

Parameter 0 of method kubernetesReactiveDiscoveryClientHealthIndicator in org.springframework.cloud.kubernetes.discovery.KubernetesDiscoveryClientAutoConfiguration$Reactive required a bean of type 'org.springframework.cloud.kubernetes.discovery.KubernetesReactiveDiscoveryClient' that could not be found.

Another problem I can reproduce is that the list of applications does not get updated. The spring cloud kubernetes docs talk about KubernetesCatalogWatch. This class is present in fabric8 and kubernetes java client, but not in discoveryclient. So maybe this feature is still missing there. Adding the EnableScheduling annotation seems to have no effect here.
Starting the admin as last application it shows the other ones.

I discovered another bug (#3034) that is relevant if you have separate management port configured (like in the example playground above). Will be included in next release, probably tomorrow.

Maybe you can test with latest versions or provide an example (e.g. based on the discovery playground).

[andreasfritz]

The missing KubernetesCatalogWatch has been added to the main branch of https://github.com/spring-cloud/spring-cloud-kubernetes

KubernetesCatalogWatch in spring-cloud-kubernetes-discovery
https://github.com/spring-cloud/spring-cloud-kubernetes/blob/main/spring-cloud-kubernetes-discovery/src/main/java/org/springframework/cloud/kubernetes/discovery/KubernetesCatalogWatch.java

This issue should be fixed in the next release of spring-cloud-kubernetes

[andreasfritz]

We have updated our discovery playground. The discovery client is now working as expected. New services are discovered and deleted services are removed.

The important changes:

• spring-cloud 2023.0.1
• added property spring.cloud.kubernetes.http.discovery.catalog.watcher.enabled = true in the discovery client
• added env SPRING_CLOUD_KUBERNETES_HTTP_DISCOVERY_CATALOG_WATCHER_ENABLED = TRUE in the discovery server

Documentation:
https://docs.spring.io/spring-cloud-kubernetes/reference/discovery-client.html