Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sparse search] Add sparse search agent #314

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

w5688414
Copy link
Collaborator

@w5688414 w5688414 commented Jan 29, 2024

  • query expansion (few shot for hard examples)
  • query repetition
  • reciprocal rank fusion
    retriever = CustomSearch(base_url=base_url, outId=outId, key=key)
    examples = [
        {"query": "半监督的论文有哪些?", "data": {"keywords": ["半监督"]}},
        {"query": "请推荐5篇半监督学习的论文", "data": {"keywords": ["半监督学习"], "pageSize": 5}},
    ]
    agent = SparseSearchAgent(llm, retriever, few_shots=examples)

    # query = '"计算机科学" 与 "生物学" 交叉学科论文'
    # query = '影响力最高的 "心理学" 研究论文'
    query = "请推荐5篇半监督学习的论文"
    docs = asyncio.run(agent.run(query))
    for idx, item in enumerate(docs):
        print(str(idx) + ".", item["title"], item["keyword"], item["year"])
    print(len(docs))

@w5688414 w5688414 requested a review from sijunhe January 29, 2024 06:09
@w5688414 w5688414 self-assigned this Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant