The similarity results obtained from the index.faiss file are significantly different from those obtained from previous versions

[Amala-R]

Summary

When using the Java implementation of FAISS and extracting the classes.faiss file, the similarity results obtained are significantly different from previous versions.

Platform

OS: Windows 11

Faiss version: faiss-cpu~=1.7.4

Installed from: pip install faiss-cpu~=1.7.4

Interface:

• Python

Reproduction instructions

• Extracted and created new db
• Saved to local workspace
  db = FAISS.from_documents(docs, get_embeddings())
  db.save_local("./db/", "classes")
• This new created classes.faiss works differently from the previous version. Always returns the same similarity result for all queries. DB data is correctly extracted. If I use old classes.faiss file with the new extracted DB, it works perfectly.

General steps:
Load the Java implementation of FAISS.
Extract the classes.faiss file using the provided tools or methods.
Perform similarity search using the extracted index file.
Compare the similarity results with those obtained from previous versions.

[mlomeli1]

The langchain library advises to install faiss via pip install, however, the faiss library (us) do not support installs via pip (even when it is possible). In order to see whether this problem comes from the core faiss library, I would recommend first to create a clean conda environment and install faiss via conda as per the INSTALL.md therein. Otherwise, please post your issue in the langchain git repo : https://github.com/langchain-ai/langchain