catalogue unable to connect to Database

[MohammedAbuibaid]

- Add labels appropriate to the issue
sock-shop demo, catalogue, catalogue-db

- Describe the expected behaviour and the actual behaviour

root@master:~# kubectl get pods -n sock-shop -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
carts-b4d4ffb5c-h27zm 1/1 Running 0 67s 192.168.0.58 master
carts-db-6c6c68b747-9wbrq 1/1 Running 0 68s 192.168.0.51 master
catalogue-5bd97b4988-hbsrk 0/1 Running 0 68s 192.168.0.53 master
catalogue-db-6868f865bb-c7lvd 1/1 Running 0 68s 192.168.0.52 master
front-end-6649c54d45-v7gkh 1/1 Running 0 68s 192.168.0.54 master
orders-7664c64d75-9njmr 1/1 Running 0 68s 192.168.0.56 master
orders-db-659949975f-b6fq9 1/1 Running 0 68s 192.168.0.62 master
payment-dd67fc96f-2897v 0/1 Running 0 67s 192.168.0.55 master
queue-master-5f6d6d4796-549vw 1/1 Running 0 67s 192.168.0.64 master
rabbitmq-5bcbb547d7-sg4hd 2/2 Running 0 67s 192.168.0.60 master
session-db-7cf97f8d4f-4vtvv 1/1 Running 0 67s 192.168.0.59 master
shipping-7f7999ffb7-s8j8c 1/1 Running 0 67s 192.168.0.57 master
user-888469b9f-whzbv 0/1 Running 0 66s 192.168.0.61 master
user-db-6df7444fc-h94tw 1/1 Running 0 66s 192.168.0.63 master

root@master:~# curl 192.168.0.53:80/health
{"health":[{"service":"catalogue","status":"OK","time":"2022-03-21 14:47:04.366461364 +0000 UTC"},{"service":"catalogue-db","status":"err","time":"2022-03-21 14:47:04.366704712 +0000 UTC"}]}

root@master:~# kubectl logs ccatalogue-5bd97b4988-hbsrk -n sock-shop
...
ts=2022-03-21T14:02:29Z caller=main.go:108 Error="Unable to connect to Database" DSN=catalogue_user:default_password@tcp(catalogue-db:3306)/socksdb
ts=2022-03-21T14:02:29Z caller=main.go:136 transport=HTTP port=80
ts=2022-03-21T14:05:31Z caller=logging.go:85 method=Health result=2 took=1.013200827s

- Describe steps to reproduce the problem:

1. An All-in-One K8s cluster is created on Digital Ocean using the following script. setup.sh
2. Run the following manifest to deploy the sock-shop application
   kubectl create -f https://raw.githubusercontent.com/microservices-demo/microservices-demo/master/deploy/kubernetes/complete-demo.yaml

[ChongyangZ]

have you solved the problem? i meet the same question

[ShuaidongGG]

Answer Update!
The reson made this issus happen on my k8s was that i forget to edit the hosts file( it's on /etc/hosts maybe ). so when i add some route rules to hosts file like this 192.168.56.190 master1 and rebuild cluster, it works.

Pre Answer

I meet the same question. When I used kubectl logs catalogue-5bd97b4988-hbsrk -n sock-shop -v=9 to check out what's going on, I noticed a unnormal response body hostIP=10.0.2.15, it's seem a NAT default IP. So I try to drain all pod(sock-shop related) to an alone Node, and it works. The reason I guess that pods like catalogue need database running on locally, and if wanna distributed running maybe every node should running database image or some better sulotion.

[vamshibeju]

Hi Team

i am facing the same issue but the above answer is not clear for me

[vamshibeju]

Below are the logs of the catalogue
LAST SEEN TYPE REASON OBJECT MESSAGE
2m27s Warning Unhealthy pod/catalogue-6479dbb5bd-ss678 Readiness probe failed: Get "http://10.244.1.185:80/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

[vamshibeju]

Error="Unable to connect to Database" DSN=catalogue_user:default_password@tcp(catalogue-db:3306)/socksdb

[vamshibeju]

hi @MohammedAbuibaid @ShuaidongGG

Could you help me to fix the above issue

Thanks in advance

[ShuaidongGG]

hi @MohammedAbuibaid @ShuaidongGG

Could you help me to fix the above issue

Thanks in advance

Same issues maybe point to very different resons. So if you can see the pod "catalogue" not ready, right?
I think the code in "catalogue pod" rely on the "database pod". Your "catalogue pod" can't works while it can't connect to "database pod". I don't realy know how your k8s build, so I can't really help more without more infomation. Maybe it helps : )

[vamshibeju]

hi @MohammedAbuibaid @ShuaidongGG
Could you help me to fix the above issue
Thanks in advance

Same issues maybe point to very different resons. So if you can see the pod "catalogue" not ready, right? I think the code in "catalogue pod" rely on the "database pod". Your "catalogue pod" can't works while it can't connect to "database pod". I don't realy know how your k8s build, so I can't really help more without more infomation. Maybe it helps : )

I have used kubeadm to provision my cluster in aws in ec2 instances of ubuntu os one for master and one for slave
My catalaogue db is working successfully but catalogue deployment
is not working and getting the below error

ls: ["images/WAT.jpg" "images/WAT2.jpg" "images/bit_of_leg_1.jpeg" "images/bit_of_leg_2.jpeg" "images/catsocks.jpg" "images/catsocks2.jpg" "images/classic.jpg" "images/classic2.jpg" "images/colourful_socks.jpg" "images/cross_1.jpeg" "images/cross_2.jpeg" "images/holy_1.jpeg" "images/holy_2.jpeg" "images/holy_3.jpeg" "images/puma_1.jpeg" "images/puma_2.jpeg" "images/rugby_socks.jpg" "images/sock.jpeg" "images/youtube_1.jpeg" "images/youtube_2.jpeg"]
ts=2023-08-30T19:28:15Z caller=main.go:108 Error="Unable to connect to Database" DSN=catalogue_user:default_password@tcp(catalogue-db:3306)/socksdb
ts=2023-08-30T19:28:15Z caller=main.go:136 transport=HTTP port=80
ts=2023-08-30T19:31:17Z caller=logging.go:85 method=Health result=2 took=20.003337999s
ts=2023-08-30T19:31:17Z caller=logging.go:85 method=Health result=2 took=50.004340945s
ts=2023-08-30T19:32:07Z caller=logging.go:85 method=Health result=2 took=50.002380741s
ts=2023-08-30T19:32:07Z caller=logging.go:85 method=Health result=2 took=20.001907332s
ts=2023-08-30T19:32:57Z caller=logging.go:85 method=Health result=2 took=8.462768ms
ts=2023-08-30T19:32:57Z caller=logging.go:85 method=Health result=2 took=30.010821456s
ts=2023-08-30T19:32:57Z caller=logging.go:85 method=Health result=2 took=20.000942237s
ts=2023-08-30T19:32:57Z caller=logging.go:85 method=Health result=2 took=50.002989202s

[ShuaidongGG]

oops, I notice that you wanna show some screenshot but it doesn't really show. Here is what I can see.

So anyway, if you wanna debug errors in you k8s, you should try to login to your "catalaogue" pod and test network connectivity with other nodes and pods. You maybe use the command line to debug, instead of checking your error logs again and again. And How use command line to debug, I think you can google it. Hope it helps. :)

[vamshibeju]

have you solved the problem? i meet the same question

Did you got any solution for this