[BUG] Slow retrieval of documents

[jeremylatorre]

Describe the bug

When using RAG, I noticed that performances where very low when trying to retrieve information from documents.
Any lead about what is causing this. Clearly the difference between rag and standard bot is huge at that time.
It might also depend on number of documents.

[statefb]

Could you provide the detail? Please do not ignore the issue template.

[jeremylatorre]

When I use my bot, it takes sometimes 17s before starting to render the content.

The first step [Retrieve Knowledge] looks ok, but after that, cursor stay at the beginning of the line for a very long time before starting to write the response.

This doesn't happen when using the LLM without documents, and it seems to increase with the number of document in the knowledge base.

[statefb]

Could you check the items in postgres? We can use management console to query and the table definition can be reffered here.

[jeremylatorre]

Could it be related to the instantion of a Lambda function? The behavior only appear on the first inference for a new conversation

[statefb]

Have you forked and customized this sample? Large container causes longer cold start. You can check whether the invocation is cold or not by reffering cloudwatch logs.
https://stackoverflow.com/questions/47061146/how-can-i-detect-cold-starts-in-aws-lambda