disable cache

[Kerushii]

Hi,

I am trying to see if llama embedding is dates aware. The sberts are obviously not, however llama chat is able to derive absolute dates from relative+absolute dates. This gave me hope and I wanted to give llama embedding models a try.
From the look of things my question is cached and the return is not what I expected. May I ask if you have any insight on this?

[Kerushii]

@Dicklesworthstone may I ask your opinion on this?

[Dicklesworthstone]

Not sure what exactly you are expecting in the results. I don't think the third choice "I ate 3 apples on 14 Sept." is likely to ever rank as more semantically similar than "What did I eat yesterday?" given that the query phrase contains the latter as a sub-string. If you're wondering why "I ate 8 apples on 11 July." ranks slightly more relevant than "I ate 3 apples on 14 Sept.", (I'm guessing this is what you mean), then it's a good point. My advice is to try my new endpoint that first filters using simple cosine similarity, and then also computes a battery of additional more sophisticated similarity measures and sorts by Hoeffding's D. I suspect that is likely to produce better results. You can also try a different model-- one that has been fine tuned on date awareness might do better. Hope that helps.

[Kerushii]

Not sure what exactly you are expecting in the results. I don't think the third choice "I ate 3 apples on 14 Sept." is likely to ever rank as more semantically similar than "What did I eat yesterday?" given that the query phrase contains the latter as a sub-string. If you're wondering why "I ate 8 apples on 11 July." ranks slightly more relevant than "I ate 3 apples on 14 Sept.", (I'm guessing this is what you mean), then it's a good point. My advice is to try my new endpoint that first filters using simple cosine similarity, and then also computes a battery of additional more sophisticated similarity measures and sorts by Hoeffding's D. I suspect that is likely to produce better results. You can also try a different model-- one that has been fine tuned on date awareness might do better. Hope that helps.

Thanks for the response and the amazing work. The model was swapped to be llama chat and as far as I concern, it's very dates aware.

May I ask how I should go about this? Do I just go try the new endpoint? If the querry is passed to llama chat directly, it should work fine.

[Dicklesworthstone]

Yes, just try this new endpoint and see if it helps:

POST /advanced_search_stored_embeddings_with_query_string_for_semantic_similarity/: Perform a two-step advanced semantic search. First uses FAISS and cosine similarity to narrow down the most similar strings, then applies additional similarity measures for refined comparison.

There are a bunch of changes to the library (you can see the latest changes to the README from today) so it's probably going to be easiest to just clear it out and clone it from scratch. Or you can do this in one step and just get the new version up and running without any manual intervention:

git clone https://github.com/Dicklesworthstone/llama_embeddings_fastapi_service
cd llama_embeddings_fastapi_service
chmod +x setup_dockerized_app_on_fresh_machine.sh
sudo ./setup_dockerized_app_on_fresh_machine.sh

Let me know how that works for you. I'm very curious to know if there are typical use cases where the more subtle similarity measures actually work better in practice than just simple cosine similarity.

Note that the fact that it works in a chat context doesn't necessarily mean that it will work here. There could be other factors at play in terms of how the chat history is stored and used that are different than just embedding based RAG.

[Kerushii]

Yes, just try this new endpoint and see if it helps:

POST /advanced_search_stored_embeddings_with_query_string_for_semantic_similarity/: Perform a two-step advanced semantic search. First uses FAISS and cosine similarity to narrow down the most similar strings, then applies additional similarity measures for refined comparison.

There are a bunch of changes to the library (you can see the latest changes to the README from today) so it's probably going to be easiest to just clear it out and clone it from scratch. Or you can do this in one step and just get the new version up and running without any manual intervention:

git clone https://github.com/Dicklesworthstone/llama_embeddings_fastapi_service
cd llama_embeddings_fastapi_service
chmod +x setup_dockerized_app_on_fresh_machine.sh
sudo ./setup_dockerized_app_on_fresh_machine.sh

Let me know how that works for you. I'm very curious to know if there are typical use cases where the more subtle similarity measures actually work better in practice than just simple cosine similarity.

Note that the fact that it works in a chat context doesn't necessarily mean that it will work here. There could be other factors at play in terms of how the chat history is stored and used that are different than just embedding based RAG.

May I ask if it's possible to join a localllama related discord chat?
A bunch of devs including exllama dev is there too. I think it would facilitate frequent communications

[Kerushii]

hmm the newest version on baremetal linux is giving me this: ImportError: cannot import name 'field_validator' from 'pydantic' (/home/kaltsit/.local/lib/python3.10/site-packages/pydantic/init.cpython-310-x86_64-linux-gnu.so)

[Dicklesworthstone]

That sounds like a version conflict. I highly recommend using a venv for this: git clone https://github.com/Dicklesworthstone/llama_embeddings_fastapi_service cd llama_embeddings_fastapi_service python3 -m venv venv source venv/bin/activate python3 -m pip install --upgrade pip python3 -m pip install wheel pip install -r requirements.txt python3 llama_2_embeddings_fastapi_server.py

…

On Mon, Sep 25, 2023 at 11:27 PM Teresa ***@***.***> wrote: hmm the newest version on baremetal linux is giving me this: ImportError: cannot import name 'field_validator' from 'pydantic' (/home/kaltsit/.local/lib/python3.10/site-packages/pydantic/*init* .cpython-310-x86_64-linux-gnu.so) — Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILNF3X426GKR5XDMYYW6STX4JDR3ANCNFSM6AAAAAA42SQ5EI> . You are receiving this because you were mentioned.Message ID: <Dicklesworthstone/llama_embeddings_fastapi_service/issues/5/1734769023@ github.com>