Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support convertReqRes-style ECS field mapping where req/res data is unavailable as Node http objects #141

Open
pushred opened this issue Feb 24, 2023 · 1 comment
Labels
agent-nodejs Make available for APM Agents project planning. community triage

Comments

@pushred
Copy link

pushred commented Feb 24, 2023

In my case I am running Node in an AWS Lambda environment where there is not a Node server running. Request data is available as APIGatewayProxyEvent or APIGatewayEventRequestContextV2 and responses as APIGatewayProxyResult or APIGatewayProxyStructuredResultV2.

Ideally this context would be gathered automatically by the apm-aws-lambda extension and accessible by ecs-logging-nodejs via the agent. But from what I can tell this isn't the case. This library doesn't log any HTTP request/response data as part of it's APM integration. And I don't see that this API Gateway data is collected by the extension today, assuming it is even available from the AWS Lambda Logs/Telemetry APIs.

As a stopgap for an indeterminate period of time I am using AsyncLocalStorage and a Pino mixin to gather this data from my function handlers and include it in my logs following ECS conventions.

If #68 resolves with a namespace-based solution and APM support remains lacking I would like to continue logging HTTP details on top-level ECS fields for consistency with non-Lambda services. This seems possible if I could create compatible instances of http.IncomingMessage and http.ServiceResponse but Node's HTTP API seems excessively complicated for this case.

Ideally I could explicitly provide the ECS fields, similar to what I do today and how existing req/res support works. Alternately support for Fetch API Request and Response objects.

@github-actions github-actions bot added agent-nodejs Make available for APM Agents project planning. community triage labels Feb 24, 2023
@trentm
Copy link
Member

trentm commented Feb 25, 2023

Some of the API Gateway request and response data is captured in the APM Agent's Lambda instrumentation, e.g. https://github.com/elastic/apm-agent-nodejs/blob/5d419876454681e23d0e5fe42641ef3f32ca8a00/lib/lambda.js#L141-L150
However, not all of it. And that info is in the tracing data, not in log records.

Note in that linked-to code that the variable name is pseudoReq. This is creating an object that looks enough like a http.IncomingMessage for the data extraction that the APM agent's handler is doing. I'm not sure if the convertReqRes handling in the current ecs-logging-nodejs formatters could work similarly.

And yah... supporting Fetch API objects eventually is a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-nodejs Make available for APM Agents project planning. community triage
Projects
None yet
Development

No branches or pull requests

2 participants