Hybrid search slows down upsert operation

[akset2X]

I get to know that delete is not happening on hybrid index. Deleting the embeddings using its "id" is only deleting the embedding list but the parallely created "scoring.terms" and "scoring" remains untouched. This makes /add and /upsert to be very delayed (I guess so).

path: ./index
writable: True

embeddings:
  path: sentence-transformers/all-MiniLM-L6-v2
  content: True
  hybrid: True

I could see that I have 30k plus embedding count available, using /count API. With hybrid index even though search is faster it looks like /upsert is very slow as the data increases. I thought deleting some data would help speed up the upsert, but deleting the embedding doesn't reduce the file size of "scoring" and "scoring.terms" in the actual file directory.
Or is there anyway to access the files like documents, scoring.terms and scoring and delete something safely?
How can I speed up upsert as the count of documents increase?

[davidmezzetti]

Thank you for the write up. I'll take a look and report back.