Clarification on using hostNetwork: true in nginx ingress

[richard-mck]

Hi there,

I'm looking at the documentation surrounding configuring ingress controllers and wanted to get some clarification on the line:

However, you must set hostNetwork to true for TCP- and UDP-based Services to work.

There's been some discussion around adding this in prior issues:

• Health Endpoint not Available for NGINX Ingress Controller rke#2891
• How to set hostNetwork:true for k8s 1.21 and up rancher-docs#455
• tcp-services-configmap and udp-services-configmap are not applied upon upgrade to k8s version using nginx-0.49.3-rancher1 #35943

I'm currently trying to resolve some issues with routing to rabbitmq running within our cluster and found that upgrading from k8s 1.20 to 1.21 knocked out connectivity. It was restored when I added an additional port config to the nginx-ingress daemonset (extraneous config removed for brevity) :

apiVersion: apps/v1
kind: DaemonSet
spec:
  template:
    spec:
      containers:
        name: controller
        ports:
        - containerPort: 80
          hostPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          hostPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        - containerPort: 5671# <-- Extra port/hostport added here
          hostPort: 5671
          name: amqp-tls
          protocol: TCP

Looking at the docs, the recommended resolution here is to enable hostNetwork: true in the RKE config. However, I'm uncertain if this is a secure choice. A quick search for docs around this yielded two relevant items, one from nginx themselves and another from OWASP.

Quoting OWASP - https://cheatsheetseries.owasp.org/cheatsheets/Kubernetes_Security_Cheat_Sheet.html#continuously-assess-the-privileges-used-by-containers:

The host network or process space should NOT be used - using hostNetwork: true will cause NetworkPolicies to be ignored since the Pod will use its host network

And from nginx - https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#via-the-host-network:

Enabling this option exposes every system daemon to the Ingress-Nginx Controller on any network interface, including the host's loopback. Please evaluate the impact this may have on the security of your system carefully

I'm not an expert in this area so I wanted to validate before proceeding. After all, the change made above does enable connectivity but maybe there's a more appropriate way to do it. Any guidance or clarification on this would be awesome. Please let me know if there's any additional context or information I can provide.

Thanks!