Custom dispatcher and exclude endpoints

[MDzyga]

Hi,

What should we add or change to make your life better?

I want to migrate ProxyKit to YARP and we implemented:

• custom response dispatcher - we have to cache user and client based on incoming token,
• service to exclude endpoints which can't be forwarded to master instance.

Why is this important to you?

Probably you know how to exclude some endponts, but custom response dispatcher is mandatory because without it we can't cache user and client based on incoming token from master instance.

I added middleware to exclude endpoints but it doesn't work:

endpoints.MapReverseProxy(proxyBuilder =>
{
    proxyBuilder.Use(async (context, next) =>
    {
      var endpoint = context.GetEndpoint();

      var excludedEndpointsProvider = app.ApplicationServices.GetRequiredService<IExcludedEndpointsProvider>();
      if (excludedEndpointsProvider.IsExcluded(endpoint?.DisplayName, false))
      {
        context.Response.StatusCode = StatusCodes.Status203NonAuthoritative;
        await context.Response.WriteAsync("Endpoint excluded.");
        return;
      }

      await next();
    });

    proxyBuilder.UseLoadBalancing();
});

[Tratcher]

Your exclusion example looks ok, why do you say it didn't work? Any request that matched that if wouldn't be proxied.

custom response dispatcher - we have to cache user and client based on incoming token,

I'm not sure what you're asking for here, can you give some examples?

[MDzyga]

I put breakpoint inside Use method and debugger doesn't stop there.

This is my implementation when I use ProxyKit.

appBuilder.RunProxy(async proxyContext =>
{
   proxyContext.Request.EnableBuffering();

   var memoryCache = app.ApplicationServices.GetService<IMemoryCache>();
   var masterUrl = memoryCache.Get<string>(Constants.MASTER_URL_CACHE_KEY);

   var forwardContext = proxyContext.ForwardTo(masterUrl).AddXForwardedHeaders();

   var response = await forwardContext.Send();

   using var scope = app.ApplicationServices.CreateScope();
   var handler = scope.ServiceProvider.GetService<IForwardedResponseDispatcher>();
   if (handler != null)
   {
     var isProcessed = await handler.ProcessResponse(response);
     if (!isProcessed && !response.Headers.Contains(RESPONSE_HEADER_NAME))
     {
       response.Headers.Add(RESPONSE_HEADER_NAME, masterUrl);
     }
  }

  return response;
});

So, when I get response from master instance, I have to process it differently in one case.

My YARP routes configuration:

Routes = new List<RouteConfig>
{
  new()
  {
    RouteId = "api",
    ClusterId = CURRENT_INSTANCE,
    Match = new RouteMatch
    {
      Path = "{**catch-all}",
    }
  }
};

I have 2 kinds of endpoints:

• api/{version}/{controller}
• connect/token

[Tratcher]

I put breakpoint inside Use method and debugger doesn't stop there.

But the request still gets proxied? I agree that doesn't make sense, all proxied requests would go through MapReverseProxy and your middleware.

If you're not using the routing system anyways, try the direct proxying API instead, it's much closer to what you were using in proxykit. https://microsoft.github.io/reverse-proxy/articles/direct-forwarding.html

[MDzyga]

No, request is not proxied.

This is my proxy configuration provider.

public class CustomProxyConfigProvider : IProxyConfigProvider
    {
        private CustomMemoryConfig _config;
        private readonly ILogger<CustomProxyConfigProvider> _logger;

        public CustomProxyConfigProvider(ILogger<CustomProxyConfigProvider> logger)
        {
            _logger = logger;
            _config = new CustomMemoryConfig(null);
        }

        public IProxyConfig GetConfig() => _config;

        public void SetMasterUrl(string masterUrl)
        {
            _logger.LogInformation($"Set master url: '{masterUrl}'.");

            string currentMasterUrl = GetMasterUrl();

            if (currentMasterUrl != masterUrl)
            {
                _logger.LogInformation($"Replace master url: '{currentMasterUrl}' on '{masterUrl}'.");

                var oldConfig = _config;
                _config = new CustomMemoryConfig(masterUrl);
                oldConfig.SignalChange();
            }
        }

        public string GetMasterUrl()
        {
            string masterUrl = null;
            var destinations = _config.Clusters[0].Destinations;
            if (destinations != null && destinations.TryGetValue(Constants.MASTER_URL_CACHE_KEY, out DestinationConfig config))
            {
                masterUrl = config.Address;
            }

            return masterUrl;
        }

        private class CustomMemoryConfig : IProxyConfig
        {
            private readonly CancellationTokenSource _cts = new();
            private const string CURRENT_INSTANCE = "current-instance";

            public CustomMemoryConfig(string masterUrl)
            {
                Routes = new List<RouteConfig>
                {
                    new()
                    {
                        RouteId = "api",
                        ClusterId = CURRENT_INSTANCE,
                        Match = new RouteMatch
                        {
                            Path = "{**catch-all}",
                        }
                    }
                };

                Clusters = new[]
                {
                    new ClusterConfig
                    {
                        ClusterId = CURRENT_INSTANCE,
                        Destinations = !string.IsNullOrEmpty(masterUrl)
                            ? new Dictionary<string, DestinationConfig>(StringComparer.OrdinalIgnoreCase)
                            {
                                {
                                    Constants.MASTER_URL_CACHE_KEY,
                                    new DestinationConfig { Address = masterUrl }
                                }
                            }
                            : new Dictionary<string, DestinationConfig>(),
                        HealthCheck = !string.IsNullOrEmpty(masterUrl)
                            ? new HealthCheckConfig
                            {
                                Active = new ActiveHealthCheckConfig
                                {
                                    Enabled = true,
                                    Interval = TimeSpan.FromMinutes(1),
                                    Timeout = TimeSpan.FromSeconds(10),
                                    Policy = HealthCheckConstants.ActivePolicy.ConsecutiveFailures,
                                    Path = Constants.HEALTH_ENDPOINT
                                }
                            }
                            : null
                    },
                };

                ChangeToken = new CancellationChangeToken(_cts.Token);
            }

            public IReadOnlyList<RouteConfig> Routes { get; }

            public IReadOnlyList<ClusterConfig> Clusters { get; }

            public IChangeToken ChangeToken { get; }

            internal void SignalChange()
            {
                _cts.Cancel();
            }
        }
    }

Destination is set correctly, application logs corrrect url

[Tratcher]

If it's not proxied then it's being handled by a different route. If you enable ASP.NET's Debug logging you should see which route was selected instead.
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-6.0#configure-logging

[MDzyga]

We use .Net 5 but I think it will be the same setting

[MDzyga]

Hi @Tratcher,
I noticed my mistake.

app.UseEndpoints(endpoints =>
{
  endpoints.MapControllers();
  endpoints.UseRequestsProxy();
});

When I commented endpoints.MapControllers(); it works.
Could you tell me how to combine API with ReverseProxy?
If I have master instance (destination), request should be forwarded to it.
If I don't have master instance or request is excluded, request should be executed on current instance.

[Tratcher]

And you know you have a master instance if masterUrl is set? In that case you can make this work by only adding the RouteConfig it masterUrl is set. You'll need to adjust the Order property so it overrides the controllers, int.MinValue should work.

[MDzyga]

Hi @Tratcher,
It's working :)
I have one small problem. When I set master instance, all requests are forwarded to it. Some of them can't be forwarded.

proxyBuilder.Use(async (context, next) =>
{
  string endpoint = context.Request.Path.ToString().ToLowerInvariant();

  if (!UseProxy(proxyBuilder, endpoint))
  {
    return;
  }

  context.Request.EnableBuffering();

  using IServiceScope scope = proxyBuilder.ApplicationServices.CreateScope();
  var proxyConfigurator = scope.ServiceProvider.GetRequiredService<IProxyConfigProvider>() as CustomProxyConfigProvider;

  string masterUrl = proxyConfigurator?.GetMasterUrl();

  await next();

  if (context.Response.StatusCode == StatusCodes.Status200OK && endpoint.Contains(TOKEN_ENDPOINT))
  {
    var tokenResponseProcessor = scope.ServiceProvider.GetRequiredService<IForwardedTokenResponseProcessor>();
    await tokenResponseProcessor.Process(context.Response, context.Request);

   context.Response.Headers.Add(RESPONSE_HEADER_NAME, masterUrl);
 }
});

Do you know how I should confogure YARP to skip these endpoints?
My return returns status code 200 instead of executing endpoint locally.

[Tratcher]

Can you show the rest of startup for context?

The UseProxy check can't go in the proxy middleware, routing has already happened, nothing else will handle the request if you don't here.

To fully customize when a request should be proxied vs handled locally, you'd need a middleware in front of routing that did the UseProxy check. If the check returned true then you'd call the proxy APIs directly, and if falls you'd call next so routing could happen.

Also, it's dangerous to try setting response headers after calling next. If the response has a body that has already started (Response.HasStarted) then setting headers will throw, they've already been sent to the client. This kind of check would need to happen in a response transform.
https://microsoft.github.io/reverse-proxy/articles/transforms.html#addresponsetransform

[samsp-msft]

Triage: Closing as answered, please re-open if you are still facing issues.

[MDzyga]

Hi @Tratcher ,
I rewrote my solution to use direct forwarding but it looks like it doesn't work with API controllers.
Configure

app.UseEndpoints(endpoints =>
{
    endpoints.ConfigureProxy(forwarder);
    endpoints.MapControllers();
    endpoints.MapHealthChecks(Constants.HEALTH_ENDPOINT, new HealthCheckOptions
    {
        Predicate = _ => true,
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });
});

ConfigureProxy

endpoints.Map("{**catch-all}", async httpContext =>
{
    var useProxy = true;
    string requestUrl = httpContext.Request.Path.ToString().ToLowerInvariant();

    using IServiceScope scope = endpoints.ServiceProvider.CreateScope();
    var excludedEndpointsProvider = scope.ServiceProvider.GetRequiredService<IExcludedEndpointsProvider>();
    var memoryCache = scope.ServiceProvider.GetService<IMemoryCache>();
    int? timeoutInSeconds = scope.ServiceProvider.GetRequiredService<IOptions<HttpSettings>>().Value.TimeoutInSeconds;

    if (excludedEndpointsProvider.IsExcluded(requestUrl, false))
    {
        useProxy = false;
    }

    string masterUrl = memoryCache.Get<string>(Constants.MASTER_URL_CACHE_KEY) ?? string.Empty;
    if (string.IsNullOrWhiteSpace(masterUrl))
    {
        useProxy = false;
    }

    var requestConfig = new ForwarderRequestConfig
    {
        ActivityTimeout = timeoutInSeconds.HasValue ? TimeSpan.FromSeconds(timeoutInSeconds.Value) : null,
        AllowResponseBuffering = true,
    };

    var httpClient = new HttpMessageInvoker(new SocketsHttpHandler
    {
        UseProxy = useProxy,
        AllowAutoRedirect = false,
        AutomaticDecompression = DecompressionMethods.None,
        UseCookies = false,
    });

    ForwarderError error = await forwarder.SendAsync(
        httpContext,
        masterUrl,
        httpClient,
        requestConfig,
        new CustomTransformer());

    if (error == ForwarderError.None && useProxy && requestUrl.Contains(TOKEN_ENDPOINT))
    {
        var tokenResponseProcessor = scope.ServiceProvider.GetRequiredService<IForwardedTokenResponseProcessor>();
        await tokenResponseProcessor.Process(httpContext.Response, httpContext.Request);
    }
});

I tried to register direct forwarding before and after registering controllers and health check but I got the same result. Requests weren't forwarded.
I verified it using debugger and on master instance I didn't received any requests.

[Tratcher]

Routing is based on finding the most specific match. You're proxying {**catch-all} which is the least specific possible match. What route pattern do you have on your controllers, and what url are you testing?

Also, I don't think that SocketsHttpHandler.UseProxy field does what you think it does. That's enables/disables the use of a forward proxy for outbound calls, it doesn't enable/disable the reverse proxy call.

[MDzyga]

I have 3 different routings patterns:

• api/v{version}/* - API controllers
• connect/* - Identity Server
• health - Health check

In my scenario I have to forward all requests except endpoints below:

• api/v{version}/settings/*
• api/v{version}/users-migration/local
• api/v{version}/certificate/*
• health/local
• .well-known/openid-configuration

[Tratcher]

    var httpClient = new HttpMessageInvoker(new SocketsHttpHandler
    {
        UseProxy = useProxy,

That useProxy flag is unrelated to YARP and won't provide the filtering you want. Remove this.

I'd suggest you take the 5 above routes that you don't want proxied and define them separately in your route table. The most specific match should handle each request. Everything that doesn't match those will be handled by the {**catch-all} route and get proxied.

[MDzyga]

Did you check your idea on my repository which I prepared to reproduce my case?

[Tratcher]

No, please try it and let me know how it goes.

[MDzyga]

I checked it and it doesn't work (comments below):

• YARP sends always requests even if I didn't set master url - in this case local endpoint should be executed
• it try to send requests only if I remove MapControllers - I don't know how to define route separately in route table

I updated my repository

[MDzyga]

I prepared a repository to reproduce my scenario.
https://github.com/MDzyga/YARP

[karelz]

Team triage / discussion:
@MDzyga it seem we have reached the point when we can't help you effectively.
Unfortunately we cannot go and debug every specific application users submit to us.
We recommend you to learn about the rules of ASP.NET routing. That should help you answer your questions. If you run into product issues that don't adhere to the docs, let us know.

Hope it helps.

[MDzyga]

Hi @karelz,
I understand, your package was mentioned as alternative of ProxyKit package. So, I thought it works similar but it doesn't.
I think the biggest difference is that I have to register YARP in UseEndpoints. If I could register it like middleware, I would be able to configure it like ProxyKit.

[karelz]

You can use YARP's DirectForwarding in middleware -- see the answer above: #1612 (comment)

[MDzyga]

@karelz thank you very much :) I updated my repository and it's working :)
Would you like to add my solution to your examples?

[MDzyga]

@karelz @Tratcher
I have last question, how can I get response body?
I get proper response in Postman but it looks like forwarder doesn't fill it after forwarding.

ForwarderError error = await _forwarder.SendAsync(
      context,
      _masterUrl,
      httpClient,
      requestConfig,
      new CustomTransformer());

using IServiceScope scope = context.RequestServices.CreateScope();
var tokenResponseProcessor = scope.ServiceProvider.GetRequiredService<IForwardedTokenResponseProcessor>();

if (error == ForwarderError.None && requestUri.Contains(TOKEN_ENDPOINT))
{
    await tokenResponseProcessor.Process(context.Response, context.Request);
}

context.Response.Body has 0 lenght and CanRead flag is false

[Tratcher]

The response body is streamed, not buffered. To read it you need to buffer the response. There's an example here:
dotnet/aspnetcore#31146

[MDzyga]

I'm not able to use next without passing context. If I pass context, it will execute local endpoint but it shouldn't.

[MDzyga]

Ok, I fixed it :) I prepared custom TransformResponseAsync and there I used ReadAsByteArrayAsync because in other way I got an exception: The stream was already consumed. It cannot be read again.