feat: Qdrant ANN backend

[Anush008]

Description

This PR adds support for Qdrant to be used as an ANN backend. Currently, the integration is available as a separate package at https://github.com/qdrant/qdrant-txtai.

With the recent addition of PGVector, we hope txtai is now open to provide native support for external vector database providers.

Testing

Tests have been configured using an in-memory and Qdrant server instance.

[davidmezzetti]

Thank you for this PR!

A recent objective of txtai was to add top level support for persisting data to relational databases. This is something I'm willing to maintain as a core component.

A lesson learned by langchain and llama_index is that putting everything into a single project gets messy fast.

I've noticed that the Qdrant txtai plugin hasn't been maintained. Is that component considered deprecated? If so, I'm not opposed to creating a plugin project for this.

[Anush008]

Hi David.
Thanks for taking a look. We saw the PGVector addition and the Qdrant integration needed some updates since a while, so thought of raising this PR.

If so, I'm not opposed to creating a plugin project for this.

Could you explain how that would work?

[davidmezzetti]

I'd basically do what the original qdrant-txtai project does now. It would be it's own project.

I don't have the resources to maintain this as a core piece of the project. It would have to be something that I fix when I get to it, like what qdrant-txtai is now.

[Anush008]

We can contribute to the maintenance efforts if that works for you.

[davidmezzetti]

Thanks. I'll keep this open until I figure out the best way to integrate this. I'd also like similar plugins for other vector databases and systems like Elasticsearch. It will get unwieldy if those are all in the core package.

[davidmezzetti]

It looks like these changes have been pushed into the qdrant-txtai project? If so, I'll close this as that is probably the best place for this to live for now.