Replace Pinecone with Chroma DB as Default Vector Database

[redcpp]

Description:
This pull allows users to use either the existing Pinecone option or the Chroma DB option. The change sets Chroma DB as the default selection. The proposed changes improve the application's costs and complexity while setting everything up.

Changes:
Updated the chat handler to allow choosing the preferred database.
Modified the code to use Chroma DB as the default selection for database operations.
Added an ingest option for Chroma DB.

Testing:
Manually tested the application to ensure that Chroma DB works correctly and is set as the default option.

Feel free to reach out if you have any questions or require further clarification on any aspect of this pull request.

[myaicreator]

_header: 'GET /api/v1/collections/pdf-test/count HTTP/1.1\r\n' +
'Accept: application/json, text/plain, /\r\n' +
'User-Agent: axios/0.26.1\r\n' +
'Host: localhost:8000\r\n' +
'Connection: close\r\n' +
'\r\n',

throw new Error('Failed to ingest your data');

[redcpp]

Hi @myaicreator!

To connect to Chroma's backend - you either need to connect to a hosted version of Chroma, or run it on your local computer. If you can run docker-compose up -d --build you can run Chroma.

git clone https://github.com/chroma-core/chroma.git
cd chroma
docker-compose up -d --build

Described in depth, here: https://docs.trychroma.com/getting-started

[mayooear]

@redcpp Thanks for the PR. Chroma is a good choice, however the setup isn't easy for beginner users to set up atm. Once they move to a hosted a solution, the transition will be much easier.

[kevinkre]

Looks like you may have missed a file... makechain.ts

[madruga8]

theres a way to connect to this example with chroma db ?

[wail-asad]

I think, it is good choice to use Chroma DB.
you can make choice in the .env, if user want to use Pinecone or Chroma DB.

[wail-asad]

@mayooear @redcpp
I want to create MR for this features : chatting history, multiple libraries, upload pdf to libraries.
due to limitation of the Pinecone, I can't create more than one index.
so this feature depend on this merge request.

[CarmichaelAJ]

Is there any update on this yet?

[JPaulMora]

This is great, will try it soon.