Add Modelcars as initContainer with restartPolicy == Always (optional)

[rhuss]

/kind feature

Describe the solution you'd like

Leverage Kubernetes sidecar containers to avoid race conditions when initializing modelcars.

Since Kubernetes 1.29 it is possible to specify sidecar containers directly as part of the initContainer list to separate them from the main application container(s). This technique works by adding the new field initContainers[].restartPolicy set to Always. This way we can ensure that the modelcar container always starts before the main runtime container starts. Alongside a startupProbe for the modelcar container, this ensures that the modelcar symbolic link is created before the runtime starts (that typically assumes implicitly that the model is already initialized).

Since sidecar containers are still hidden behind a feature flag in Kubernetes this mode should be optional in KServe, too, and be enabled with a config flag (and switched off by default)

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Links to the design documents:

https://docs.google.com/document/d/1Bs4fnP8rhPMaoPoLSYVvuRq-z9vkGPQ0rKbmfH4I7js/edit#bookmark=id.uwj8r3x8uwpp