Are model objects thread safe? #191
Unanswered
chitralverma
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I was planning to use fastembed as part of an API server. The model is pre-cached on the app level and when a POST request arrives, I use the payload as an input to the model.
I observed that with single requests this works okay, but for multiple requests in parallel, there are some non-deterministic latencies.
Any thoughts/ suggestions on this?
Beta Was this translation helpful? Give feedback.
All reactions