Unable to get open telemetry tracing working

[rcdailey]

I apologize for the low-effort / lazy post. I recently made a post over on the open telemetry github discussion forum here:

open-telemetry/opentelemetry-dotnet#4233

I wanted to crosspost here just in case my issue is specific to YARP. Based on an issue I read, I'm worried that something about YARP may be preventing it from working.

I'll add some additional context. The code that is sending the HTTP request, as described in the linked forum discussion above, is located in a class implementing IProxyConfigProvider.

If anyone has advice on what I can do to get this working, it would be greatly appreciated.

[samsp-msft]

Its late on Friday, so I don't have time to knock up a demo.

The best thing to try is dumping the headers on the inbound requests at YARP and destination to see what is being sent. See https://microsoft.github.io/reverse-proxy/articles/diagnosing-yarp-issues.html#using-aspnet-request-logging for how to add middleware for that logging.

ASP.NET inbound, and httpClient outbound will support the w3c headers for spans etc, and possibly add one for what happens at the proxy. To see that in your telemetry you'll want to add the OTEL logging to your YARP instance so it is also sending back its records.

[rcdailey]

Thank you for the response. I just want to clarify that there is no inbound request. This is a fresh new request initiated from YARP to a another service inside (done from inside a class implementing IProxyConfigProvider).

[MihaZupan]

Can you show the code you used when sending that request, and how you configured Otel? What HttpClient are you using?

Off the top of my head, I would think that requests initiated by the proxy (even if there is no incoming request) should also "just work" after you've set up Otel. That applies to requests made for active health checks as well for example.

[rcdailey]

Most of the code you can find in the post I linked in my OP. So I will refer you there, if you don't mind.

The code I'm using to send the request is also explained there; it's a swagger-generated client object. I send it from a class implementing IProxyConfigProvider, specifically sent from the GetConfig() method.

I do not get a traceparent header when I inspect the outgoing request with Fiddler.

[MihaZupan]

Just to double-check, the distributed trace is working for you for other proxy requests (those initiated by YARP itself)? Or not at all?

[rcdailey]

I actually have not tested pass-through with headers yet. So far, I was only focused on this specific use case and getting it working.

[samsp-msft]

If you are making an out of band http request - then the presence of YARP should not matter. Whether that request will have any tracing headers will depend on the environment for the request. If you are making an http request from a timer, or other event not associated with an inbound request, then there will be no context for it to propogate, so it will be empty. If you create an Activity then the request should include the span id headers.
To track it end to end, you'll want to create an ActivitySource as the parent to the activity, and register that with the OTel library with the .AddSource(activitySource.Name) API.

I recently added distinct activities to YARP for distributed tracing, so that YARP has its own ActivitySource which will track:

• The proxy forwarding and if applicable the session affinity lookup
• Active Health check requests to ping destinations to validate their status.

For the latter, I create an activity so that the requests have span-id's.