AttributeError: 'Deployment' object has no attribute 'deploy'

[SagyHarpazGong]

I'm trying to run model server using kserve but I'm getting the following error:

Traceback (most recent call last):
  File "/project/e2e_servers/pre_asr_kserve.py", line 215, in <module>
    ModelServer(http_port=8000).start({'PRE-ASR': PreASRServer})
  File "/home/xxx/.conda/envs/my_env/lib/python3.9/site-packages/kserve/model_server.py", line 164, in start
    models[key].deploy()
AttributeError: 'Deployment' object has no attribute 'deploy'

I installed kserve using pip:
pip install kserve==0.12.1

So in my conda env installed:
kserve 0.12.1
ray 2.12.0
starlette 0.36.3

My snippet code:

from kserve import Model, ModelServer
from ray import serve

@serve.deployment(name='PRE-ASR', num_replicas=2)
class PreASRServer(Model):
    def __init__(self):
        self.name = 'PRE-ASR'
        super().__init__(self.name)
        self.model = None
        self.ready = False
        self.load()

    def load(self):
        self.model = load_model()
        self.model.eval()
        self.ready = True

    async def predict(self, payload: Dict, headers: Dict[str, str] = None):
        'some code'

if __name__ == "__main__":
    ModelServer().start({'PRE-ASR': PreASRServer})

Am I doing something wrong?

Please advise

[sivanantha321]

Can you try with ray 2.10 ?

[SagyHarpazGong]

@sivanantha321
Thanks for your response.

I tried basically all ray versions until 2.7.1 and only 2.7.1 or lower are work.
from ray version 2.8 the method "deploy" of the Deployment class is deprecated, this is why I'm getting that error I just wondering why ray 2.12 is one of the dependencies of kserve if it not working.
If I'm installing kserve==0.11 I'm not getting that error but it installing ray 2.4

[sivanantha321]

Thanks for reporting, This will be fixed as part of #3668