Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate COSINE before Lucene 10 release #13281

Open
benwtrent opened this issue Apr 8, 2024 · 3 comments
Open

Deprecate COSINE before Lucene 10 release #13281

benwtrent opened this issue Apr 8, 2024 · 3 comments

Comments

@benwtrent
Copy link
Member

Description

Over a couple disparate discussions over various PRs/issues, we have tossed around the idea of deprecating COSINE.

To me, this makes sense, we shouldn't have COSINE at all IMO. Either users should normalize before indexing, or use max-inner product.

The question then becomes "What about byte vectors"? It seems to me that users should still use DOT_PRODUCT and MAX_INNER_PRODUCT.

@jmazanec15
Copy link
Contributor

@benwtrent Is the main reason to deprecate to stop enabling users to setup non-optimal configurations? Or are there limitations cosine similarity imposes on implementation/optimization for other distance metrics?

@benwtrent
Copy link
Member Author

@jmazanec15

Mainly because cosine has no benefit over normalizing & using dot_product and maintaining optimized Cosine similarity functions is an unnecessary burden for Lucene.

@jmazanec15
Copy link
Contributor

Thanks @benwtrent that makes sense

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants