model with name <inference service name> does not exist. #3682
Labels
Comments
|
@VikasAbhishek Can you post the response of |
|
|
Have you added Host header ? |
|
Classification: Confidential
No I run the command curl -v http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models<http://$%7bINGRESS_HOST%7d:$%7bINGRESS_PORT%7d/v1/models>.
From: Sivanantham ***@***.***>
Sent: Monday, May 13, 2024 1:16 PM
To: kserve/kserve ***@***.***>
Cc: Vikas Ghunawat Meena ***@***.***>; State change ***@***.***>
Subject: Re: [kserve/kserve] model with name <inference service name> does not exist. (Issue #3682)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
Have you added Host header ?
-
Reply to this email directly, view it on GitHub<#3682 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BHL3AUU3DZ5X6JVLIHGTPK3ZCBVUHAVCNFSM6AAAAABHTSUO6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBWHA3TKMRVGI>.
You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>
::DISCLAIMER::
…________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
|
|
Classification: Confidential
After using host header :
curl -v -H "Host: ${SERVICE_HOSTNAME}" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models<http://$%7bINGRESS_HOST%7d:$%7bINGRESS_PORT%7d/v1/models> * Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
GET /v1/models HTTP/1.1
Host: xgboost-pvc.default.example.com
User-Agent: curl/7.68.0
Accept: */*
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 13
< content-type: application/json
< date: Mon, 13 May 2024 07:38:51 GMT
< server: istio-envoy
< x-envoy-upstream-service-time: 15
<
* Connection #0 to host localhost left intact
From: Sivanantham ***@***.***>
Sent: Monday, May 13, 2024 1:16 PM
To: kserve/kserve ***@***.***>
Cc: Vikas Ghunawat Meena ***@***.***>; State change ***@***.***>
Subject: Re: [kserve/kserve] model with name <inference service name> does not exist. (Issue #3682)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
Have you added Host header ?
-
Reply to this email directly, view it on GitHub<#3682 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BHL3AUU3DZ5X6JVLIHGTPK3ZCBVUHAVCNFSM6AAAAABHTSUO6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBWHA3TKMRVGI>.
You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>
::DISCLAIMER::
…________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
|
|
@VikasAbhishek The response is not available in your comment, anyways you can verify if the model is ready, and can view the model name. Try using this as the model name for inference . If the response is empty, this may mean that the model is not loaded. In that case please, verify the model server logs. |
|
Classification: Confidential
I checked with the curl command that my model is not uploading to the models folder like the sklearn-iris example.
curl -v -H "Host: ${SERVICE_HOSTNAME}" -H "Content-Type: application/json" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models<http://$%7bINGRESS_HOST%7d:$%7bINGRESS_PORT%7d/v1/models>
* Trying 172.18.0.2:30227...
* TCP_NODELAY set
* Connected to 172.18.0.2 (172.18.0.2) port 30227 (#0)
GET /v1/models HTTP/1.1
Host: sklearn-iris.kserve-test.example.com
User-Agent: curl/7.68.0
Accept: */*
Content-Type: application/json
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 27
< content-type: application/json
< date: Mon, 13 May 2024 08:46:34 GMT
< server: istio-envoy
< x-envoy-upstream-service-time: 6
<
* Connection #0 to host 172.18.0.2 left intact
{"models":["sklearn-iris"]}
For sklearn-iris example
curl -v -H "Host: ${SERVICE_HOSTNAME}" -H "Content-Type: application/json" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models<http://$%7bINGRESS_HOST%7d:$%7bINGRESS_PORT%7d/v1/models>
* Trying 172.18.0.2:30227...
* TCP_NODELAY set
* Connected to 172.18.0.2 (172.18.0.2) port 30227 (#0)
GET /v1/models HTTP/1.1
Host: xgboost-pvc.default.example.com
User-Agent: curl/7.68.0
Accept: */*
Content-Type: application/json
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 13
< content-type: application/json
< date: Mon, 13 May 2024 08:47:58 GMT
< server: istio-envoy
< x-envoy-upstream-service-time: 9
<
* Connection #0 to host 172.18.0.2 left intact
{"models":[]}
For custom xgboost model v1/models is empty
Can you guide me how to upload my model in the v1/models for pvc storageUri method.
From: Sivanantham ***@***.***>
Sent: Monday, May 13, 2024 2:07 PM
To: kserve/kserve ***@***.***>
Cc: Vikas Ghunawat Meena ***@***.***>; Mention ***@***.***>
Subject: Re: [kserve/kserve] model with name <inference service name> does not exist. (Issue #3682)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
@VikasAbhishek<https://github.com/VikasAbhishek> The response is not available in your comment, anyways you can verify if the model is ready, and can view the model name. Try using this as the model name for inference
-
Reply to this email directly, view it on GitHub<#3682 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BHL3AUWAF27EYW4CDMLBTLLZCB3URAVCNFSM6AAAAABHTSUO6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBWHE3TMNJWGU>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
::DISCLAIMER::
…________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
|
|
@VikasAbhishek As mentioned earlier, the model is not loaded. Please, verify the model server logs and storage initializer logs. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
/kind bug
What steps did you take and what happened:
I ran the inference service on custom xgboost model that I trained and saved in .joblib extension using the pvc storage option, followed the link
https://kserve.github.io/website/master/modelserving/storage/pvc/pvc/
I have used the port forward and node port to get the ingress host and port.
Inference service and pods are running fine.
after running the curl command I am getting this error:
< HTTP/1.1 404 Not Found
< content-length: 55
< content-type: application/json
< date: Mon, 13 May 2024 06:02:43 GMT
< server: istio-envoy
< x-envoy-upstream-service-time: 16
<
{"error":"Model with name xgboost-pvc does not exist."}
What did you expect to happen:
I expected that the curl command will return the predictions.
What's the InferenceService yaml:
[To help us debug please run
kubectl get isvc $name -n $namespace -o yamland paste the output]apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"serving.kserve.io/v1beta1","kind":"InferenceService","metadata":{"annotations":{},"name":"xgboost-pvc","namespace":"default"},"spec":{"predictor":{"xgboost":{"storageUri":"pvc://task-pv-claim/model/S1-B2-C1_xgboost.joblib"}}}}
creationTimestamp: "2024-05-10T11:16:48Z"
finalizers:
generation: 1
name: xgboost-pvc
namespace: default
resourceVersion: "1804889"
uid: efbe751b-9865-47a3-a84a-501f14bc9618
spec:
predictor:
model:
modelFormat:
name: xgboost
name: ""
resources: {}
storageUri: pvc://task-pv-claim/model/S1-B2-C1_xgboost.joblib
status:
address:
url: http://xgboost-pvc.default.svc.cluster.local
components:
predictor:
address:
url: http://xgboost-pvc-predictor.default.svc.cluster.local
latestCreatedRevision: xgboost-pvc-predictor-00001
latestReadyRevision: xgboost-pvc-predictor-00001
latestRolledoutRevision: xgboost-pvc-predictor-00001
traffic:
percent: 100
revisionName: xgboost-pvc-predictor-00001
url: http://xgboost-pvc-predictor.default.example.com
conditions:
status: "True"
type: IngressReady
severity: Info
status: "True"
type: LatestDeploymentReady
severity: Info
status: "True"
type: PredictorConfigurationReady
status: "True"
type: PredictorReady
severity: Info
status: "True"
type: PredictorRouteReady
status: "True"
type: Ready
severity: Info
status: "True"
type: RoutesReady
modelStatus:
copies:
failedCopies: 0
totalCopies: 1
states:
activeModelState: Loaded
targetModelState: Loaded
transitionStatus: UpToDate
observedGeneration: 1
url: http://xgboost-pvc.default.example.com
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
when I checked the logs of xgboost predictor pod after running the curl command I am getting this:
2024-05-13 05:16:25.231 1 kserve ERROR [model_not_found_handler():113] Exception:
Traceback (most recent call last):
File "/prod_venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/prod_venv/lib/python3.9/site-packages/starlette/routing.py", line 74, in app
response = await func(request)
File "/prod_venv/lib/python3.9/site-packages/fastapi/routing.py", line 299, in app
raise e
File "/prod_venv/lib/python3.9/site-packages/fastapi/routing.py", line 294, in app
raw_response = await run_endpoint_function(
File "/prod_venv/lib/python3.9/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
File "/kserve/kserve/protocol/rest/v1_endpoints.py", line 67, in predict
model_ready = self.dataplane.model_ready(model_name)
File "/kserve/kserve/protocol/dataplane.py", line 213, in model_ready
raise ModelNotFound(model_name)
kserve.errors.ModelNotFound: Model with name xgboost-pvc does not exist.
2024-05-13 05:16:25.232 uvicorn.access INFO: 10.244.2.5:0 1 - "POST /v1/models/xgboost-pvc%3Apredict HTTP/1.1" 404 Not Found
2024-05-13 05:16:25.233 kserve.trace kserve.io.kserve.protocol.rest.v1_endpoints.predict: 0.0017123222351074219
2024-05-13 05:16:25.233 kserve.trace kserve.io.kserve.protocol.rest.v1_endpoints.predict: 0.0017050000001290755
Environment:
Istio Version: 1.19.4
Knative Version: 1.10.1
KServe Version: 0.12.1
Kubeflow version: not installed
Cloud Environment:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
Minikube/Kind version: kind version 0.22.0
Kubernetes version: (use
kubectl version) : Client Version: v1.30.0Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2
OS (e.g. from
/etc/os-release): Distributor ID: UbuntuDescription: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
The text was updated successfully, but these errors were encountered: