Add vLLM backend e2e test

[yuzisun]

/kind feature

Describe the solution you'd like
vLLM 0.4.1 now support cpu backend, so we can add the e2e tests without depending on GPUs.
vllm-project/vllm#3993

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Links to the design documents:
[Optional, start with the short-form RFC template to outline your ideas and get early feedback.]
[Required, use the longer-form design doc template to specify and discuss your design in more detail]

[sivanantha321]

/assign