custom proxy_access_log still not working in 3.4.*

[ravigunasekara]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

kong/kong-gateway:3.4.3.6

Current Behaviour

docker run fails with nginx: [emerg] unknown log format "custom-json" in /usr/local/kong/nginx-kong.conf

I have set environment variable KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json \ as per the instructions in the issue #12061.

Also custom-json.conf is included in nginx.conf http block.
http { include 'custom-json.conf'; include 'nginx-kong.conf'; }

Below is how custom-json.conf looks like.
log_format custom-json escape=json '{' '"consumer":"$http_x_consumer_username",' '"consumer_id":"$http_x_consumer_id",' '}';

Expected Behaviour

docker-compose run successfully build the image and able to run the container with the new log formatter.

Steps To Reproduce

1. Update your Docker file

FROM kong/kong-gateway:3.4.3.6 
ENV APP_HOME=/app \
    KONG_DATABASE=off \
    KONG_DECLARATIVE_CONFIG=/tmp/kong.yml \
    KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json \
    KONG_ADMIN_ACCESS_LOG=/dev/stdout \
    KONG_PREFIX="/usr/local/kong/"

2. include custom log formatter into the kong.conf

http {
    include 'custom-json.conf';
    include 'nginx-kong.conf';
}

3. For an example your custom-json.conf looks like this (this could be any format)

log_format customer-json escape=json '{'
    '"consumer":"$http_x_consumer_username",'
    '"consumer_id":"$http_x_consumer_id",'
'}';

4. run docker-compose build
5. Failed with nginx: [emerg] unknown log format "customer-json" in /usr/local/kong/nginx-kong.conf

Anything else?

It seems that Kong always expects log_format definitions to be inside the nginx-kong.conf. If we create a new log formatter and add it to the http block inside the kong.conf, Kong fails to fetch it. Normally, nginx accepts custom log formatters via the http block. There is a simple workaround to overcome this issue, but I would like to know the recommended solution.

[ravigunasekara]

I am sharing the workaround I have implemented in case others experience the same issues.
I have modified Dockerfile in a way that it override the current access_log in nginx-kong.conf file that it uses my own log formatter(customer-json).
Added below line to Dockerfile

RUN set -xe && \
    kong prepare -p "${KONG_PATH}" && \
		sed -i "s|access_log /dev/stdout kong_log_format|access_log /dev/stdout customer-json|" ${KONG_PATH}/nginx-kong.conf && \
    true	

Hope this helps :)

[samugi]

Hi @ravigunasekara, thank you for opening this issue.

I couldn't reproduce it in 3.4.3.6, I noticed, however, that you have defined a custom format named custom-json and then configured a slightly different name in your environment variable: KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json, i.e. customer instead of custom. Could that be the problem?

Thank you.

[github-actions]

This issue is marked as stale because it has been open for 14 days with no activity.