Generated Ingress annotation too long

[frank-bee]

What happened?

This error is thrown when creating an Ingress annotation.

Error syncing to physical cluster: Ingress.extensions "seldon-ingress-api-tenant-a-x-istio-system-x-tenant-a-tenant-a" is invalid: metadata.annotations: Invalid value: "alb.ingress.kubernetes.io/conditions.istio-ingressgateway-x-istio-system-x-tenant-a-tenant-a": name part must be no more than 63 characters

The annotation is as follows

alb.ingress.kubernetes.io/conditions.istio-ingressgateway: >
      [{"field":"http-header","httpHeaderConfig":{"httpHeaderName": "KeyId",
      "values":["..."]}},
      {"field":"http-request-method","httpRequestMethodConfig":{"Values":["GET",
      "POST"]}}]

What did you expect to happen?

The Ingress should be created in the host cluster incl. this annotation

How can we reproduce it (as minimally and precisely as possible)?

Create an ingress with an annotation key longer than allowed

Anything else we need to know?

See also https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set

Host cluster Kubernetes version

$ kubectl version
Client Version: v1.28.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.25.15-eks-4f4795d
WARNING: version difference between client (1.28) and server (1.25) exceeds the supported minor version skew of +/-1

Host cluster Kubernetes distribution

AWS EKS

vlcuster version

$ vcluster --version
vcluster version 0.17.1

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

default

OS and Arch

locally I work with a MAc

[FabianKramm]

@frank-bee thanks a lot for creating this issue! I guess what we can do is to add logic to skip annotations that would exceed the limit so that the ingress is able to sync, but the problem that the condition doesn't apply to the service would remain, so I'm unsure how we would fix this as there seems to be a general Kubernetes limitation on this / bad design by AWS ingress-controller that has problems with long service names

[frank-bee]

Fully agree to what you say, @FabianKramm .
Better do not skip these annotations, because if they are there, probably they are not "optionally" (like in our case).
I think we will try to reduce the characters in these 2 substrings (service , release):

alb.ingress.kubernetes.io/conditions.service-name-x-istio-system-x-vcluster-release-name

[FabianKramm]

@frank-bee yeah that sounds like a short term workaround at least until we find an actual solution