[Feature Request]: request/response bodies in HTTP logs should be trimmed

[SpecLad]

Feature Type

• Adding new functionality to Nuclio

• Changing existing functionality in Nuclio

• Removing existing functionality in Nuclio

Problem Description

When the Nuclio dashboard logs an HTTP request, it dumps the full request and response bodies, as seen here:

nuclio/pkg/restful/middleware/middleware.go

Lines 115 to 134 in 04497d2

	logVars := []interface{}{
	"requestMethod", request.Method,
	"requestPath", request.URL,
	"requestHeaders", requestHeaders,
	"requestBody", string(requestBody),
	"responseStatus", responseWrapper.Status(),
	"responseTime", time.Since(requestStartTime).String(),
	}
	// response body is too spammy
	if !common.StringSliceContainsStringPrefix([]string{
	"/api/functions",
	"/api/function_templates",
	"/api/v3io_streams",
	"/kaniko",
	}, strings.TrimSuffix(request.URL.Path, "/")) {
	logVars = append(logVars, "responseBody", responseBodyBuffer.String())
	}
	logger.DebugWithCtx(request.Context(), "Handled request", logVars...)

In our use case, the requests and responses frequently contain large images (up to several megabytes in size), which get dumped to the log in their entirety. This creates several problems:

1. It's hard to visually examine the logs, because the request/response bodies fill up the screen.
2. When the maximum log size is capped, much fewer log entries are retained, because the bodies consume most of the allotted size.

Feature Description

The request and response bodies should be trimmed to a reasonable size (e.g. 1000 characters).

It also wouldn't be unreasonable to just omit them entirely. AFAIK, logging bodies is not common in web servers.

Alternative Solutions

It would be possible to work around this by e.g. postprocessing the logs, but it would be pretty annoying to implement.

Additional Context

No response

[liranbg]

Hi @SpecLad - few questions
why would you include large images as part of nuclio dashboard's? which user flow require you to send nuclio dashboard an image? part of function arbitrary fields (annotation, etc)?
in addition to that, you may increase nuclio dashboard logger level to INFO ignoring such log lines.

[SpecLad]

why would you include large images as part of nuclio dashboard's? which user flow require you to send nuclio dashboard an image? part of function arbitrary fields (annotation, etc)?

I'm just calling a function via /api/function_invocations. My functions usually take an image as input (and some output fairly large blobs, as well).

in addition to that, you may increase nuclio dashboard logger level to INFO ignoring such log lines.

Yeah, but I think that would suppress HTTP logs completely, which I don't want.

[TomerShor]

@SpecLad another workaround would be to invoke your function directly using its invocation URLs, instead of using the function_invocations api that goes through the dashboard.