Playground for Agent/Chat debugging

[anada10]

Create a Playground where App builders (developers) can experiment with agents and run side by side allowing users to run A/B tests for a single agent or Compare multiple agents

Let users A/B test with
Agents

• Model
• System Message
• Tools
• Datasources

Users should also be able to see retrieval stats at a chunk level

• Document name
• Chunk text
• Chunk ID (vector db point id)
• Chunk Rank
• Chunk strategy (semantic, character, message by message etc)
• Similarity score (Cosine or otherwise)
• Token/Character size
• Timestamp created date
• Embedding Model