It is possible to integrate with Milvus and return similarity scores

[wzs1566]

Dear Developers,

I would like to request the following enhancements:

1. Integrate TaskingAI system with the Milvus database to enable efficient vector similarity search.

2. Allow the database to return similarity scores along with retrieved chunks, enabling flexible filtering based on similarity thresholds rather than a fixed number of chunks.

These improvements would greatly enhance the precision and flexibility of our system's search and retrieval capabilities.

Thank you for your consideration.

Best regards,

[jameszyao]

@wzs1566 thanks for your suggestions. I think we can add a score-based retrieval filter. Please stay tuned :-)