External Authz filter is not rewriting Host and does not work for an Ext server behind LB

[juanmolle]

Title: External Authz filter is not rewriting Host and does not work for an Ext server behind LB

Description:
When the filter is doing the request to the http external auth server, it copies all the original request headers ( path is an special case, a prefix is added if it set in config)
If the external service is under a LB, Host is not rewrite and 404 is always returned.
My first though was HOST could be used during auth logic. but grpc is apparently not changing it.
Is this a miss-configuration? do I miss something, or a new config option in the filter to replace it and copy the original host in an x-original-host header or similar?

Repro steps:
Start envoy with config
make a call

curl -XGET -H "Authorization: Bearer token1" http://localhost:8000/ -I

Request to ext auth contains HOST: localhost:8088
server behind LB replay 404

Config:

static_resources:
 listeners:
 - address:
     socket_address:
       address: 0.0.0.0
       port_value: 8000
   filter_chains:
   - filters:
     - name: envoy.filters.network.http_connection_manager
       typed_config:
         "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
         codec_type: AUTO
         stat_prefix: ingress_http
         route_config:
           name: local_route
           virtual_hosts:
           - name: upstream
             domains:
             - "*"
             routes:
             - match:
                 prefix: "/"
               route:
                 host_rewrite_literal: httpbin.org
                 cluster: httpbin.default.svc
         http_filters:
         - name: envoy.filters.http.ext_authz
           typed_config:
             "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
             transport_api_version: V3
             allowed_headers:
               patterns:
                 - exact: authorization
             http_service:
               server_uri:
                 uri: <auth-server>
                 cluster: ext-authz-svc
                 timeout: 5s
         - name: envoy.filters.http.router
           typed_config:
             "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

 clusters:
 - name: httpbin.default.svc
   type: LOGICAL_DNS
   connect_timeout: 5s
   dns_lookup_family: V4_ONLY
   transport_socket:
     name: envoy.transport_sockets.tls
     typed_config:
       '@type': type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
       common_tls_context:
         validation_context:
           trusted_ca:
             filename: /etc/ssl/certs/ca-certificates.crt
           match_typed_subject_alt_names:
             - san_type: DNS
               matcher:
                 exact: httpbin.org
                 ignore_case: true
       sni: httpbin.org
   load_assignment:
     cluster_name: httpbin.default.svc
     endpoints:
       - lb_endpoints:
           - endpoint:
               address:
                 socket_address:
                   address: httpbin.org
                   port_value: 443
   typed_extension_protocol_options:
     envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
       '@type': type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
       common_http_protocol_options:
         idle_timeout: 60s
       auto_config: {}
 - name: ext-authz-svc
   type: LOGICAL_DNS
   connect_timeout: 5s
   dns_lookup_family: V4_ONLY
   transport_socket:
     name: envoy.transport_sockets.tls
     typed_config:
       '@type': type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
       common_tls_context:
         validation_context:
           trusted_ca:
             filename: /etc/ssl/certs/ca-certificates.crt
           match_typed_subject_alt_names:
             - san_type: DNS
               matcher:
                 exact: <auth-server>
                 ignore_case: true
       sni: <auth-server>
   load_assignment:
     cluster_name: ext-authz-svc
     endpoints:
       - lb_endpoints:
           - endpoint:
               hostname: <auth-server>
               address:
                 socket_address:
                   address: <auth-server>
                   port_value: 443
   typed_extension_protocol_options:
     envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
       '@type': type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
       common_http_protocol_options:
         idle_timeout: 60s
       auto_config: {}

[ravenblackx]

@esmet

[juanmolle]

Is this a bug, or a misconfiguration?

[ravenblackx]

@tyxia since @esmet didn't respond (based on CODEOWNERS of ext_authz)

[tyxia]

When clientMatcher is empty, the HOST will NOT be added to client response https://github.com/envoyproxy/envoy/blob/main/source/extensions/filters/common/ext_authz/ext_authz_http_impl.cc#L149-L150

You will need to configure something like allowed_client_headers\ allowed_client_headers_on_success to enable this

[juanmolle]

My problem is that host is the client host and it is not rewrited. in route we have the possibility to set auto_host_rewrite = true. and the request to the upstream will contain the destination host. then is the upstream is behind a LB it will find the route.
but in this case, the filter directly call the auth server (using the client host) and there is no possibility to rewrite_host. then fails with 404

[juanmolle]

@tyxia do I explain correctly the issue?

[juanmolle]

for grpc service there is a config to set the authority for the POST resquest to the server, probably we could add it to the http service case. and eventually copy the original :authority into another header if the server needs to use it.

envoy/api/envoy/config/core/v3/grpc_service.proto

Line 43 in 9f494c0

string authority = 2