Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run evaluations on selected datasets to optimise basic RAG pipeline #7629

Open
Tracked by #7625
mrm1001 opened this issue May 2, 2024 · 0 comments
Open
Tracked by #7625

Run evaluations on selected datasets to optimise basic RAG pipeline #7629

mrm1001 opened this issue May 2, 2024 · 0 comments
Assignees
@davidsbatista
Labels
P1 High priority, add to the next sprint

Comments

@mrm1001
Copy link
Member

mrm1001 commented May 2, 2024 •
edited

Use evaluation on the selected datasets to optimise some parameters commonly tweaked in RAG pipelines:

  • top_k
  • chunk_size
  • embedding model

The outcome should be a colab or notebook that can be reviewed by other engineers and devrel.
@mrm1001 mrm1001 mentioned this issue May 2, 2024
Open
@mrm1001 mrm1001 changed the title Get performance metrics on selected datasets Get performance metrics on selected datasets using existing techniques already implemented in Haystack May 2, 2024
@mrm1001 mrm1001 added the P1 High priority, add to the next sprint label May 10, 2024
@mrm1001 mrm1001 changed the title Get performance metrics on selected datasets using existing techniques already implemented in Haystack Run evaluations on selected datasets to optimise basic RAG pipeline May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davidsbatista davidsbatista

Labels
P1 High priority, add to the next sprint
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mrm1001 @davidsbatista