Control Plane fails to pull information from database pending "migrations finish" command

[lays147]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

3.6.1

Current Behavior

After deploying a new container version based on kong:3.6.1-ubuntu in the hybrid deployment, the control plane starts but apparently is unable to retrieve the configurations in the database since it logs an error that "migration finish" should be run, however it starts successfully and dispatches the update of the Data Plane, with no configuration at all.

Expected Behavior

The control plane should not start whether there's any database error, or it should not dispatch any updates to the Data Plane if there's any database error.

Steps To Reproduce

1. Build a new docker image based on kong:3.6.1-ubuntu
2. Deploy kong in hibrid mode with pending migrations on the database
3. The control plane starts, but logs an error pending "migration finish" but it stays healthy
4. Data plane connects to CP to retrieve the config, and it gets nothing. All proxy routes fails with 404

Anything else?

Debugging more of my logs, I found an issue with the ddtrace plugin configuration that impacted the Data Plane, and Kong didn't show any logs that this error impacted the whole process of configuration the Data Plane.

However, I'll persist with this issue here, because I had the same problem in my staging environment where the ddtrace was not used.

[StarlightIbuki]

This is an expected behavior of Kong. When Kong migrates to the new version, it applies a technique called blue-green deployment, which allows a seamless transition to the new version.
After kong migrations up, the cluster enters a state where DPs of the old version and new version can coexist, and the database should be able to support both of them; the kong migrations finish cleans up the database, which should be called after the migration process is done and old version DPs are no longer running.

[StarlightIbuki]

It is not an error, but a message that should not be ignored by the admin, thus the error level.