Discussions

• 1 You must be logged in to vote

  🙏

  I followed the installation instructions and encountered the following error.

  lldhsds asked May 10, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  OpenCompassData.zip的数据集和目前hf上的对比，每个开源数据集的数据量完全一致吗，还是缩减版的？

  pzfok asked May 10, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  运行configs下面的配置文件（加入import os用于读取环境变量）会报错

  pzfok asked May 9, 2024 in Q&A · Closed · Answered

  

  

  7
• 1 You must be logged in to vote

  🙏

  What is the composition of evaluation data in leaderboard?

  BugApe asked May 9, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  有没有一些能够减少显存的操作，我在推理128k的needlebench时，8张A100仍然显存爆炸[Feature]

  1518630367 asked May 9, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💡

  可以考虑加入几个moe模型吗，如Qwen/Qwen1.5-MoE-A2.7B-Chat

  mdjhacker started May 7, 2024 in Ideas · Closed

  

  

  2
• 1 You must be logged in to vote

  🙏

  Why needlebench summarizer hardcodes 4k in the result demonstration?

  b2r66sun asked May 8, 2024 in Q&A · Closed · Answered

  

  

  3
• 1 You must be logged in to vote

  🙏

  评测qa形式自定义数据集，命令行启动，为什么固定使用SizePartitioner，导致infer的结果都被切分了，能不能不切分啊

  pzfok asked May 7, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  没有找到CreationBench数据集

  zhangfelix asked May 7, 2024 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  可以在windows上部署吗

  wangxun6 asked May 7, 2024 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  CodeFuseEval 这样的评测，怎么构建数据集啊

  luckfu asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Code models score low on humaneval dataset

  PJY-coder asked Apr 22, 2024 in Q&A · Closed · Answered

  

  

  8
• 1 You must be logged in to vote

  🙏

  How to summarize CDME result?

  b2r66sun asked Apr 28, 2024 in Q&A · Closed · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  why does the installed version not match the tag of the repository?

  Galaxy-Husky asked Apr 29, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  如何获取longbench和leval数据集

  kkkjyu started Apr 25, 2024 in Community Task

  

  

  1
• 1 You must be logged in to vote

  🙏

  请问如何确保基于openai api多次测评结果的一致性，似乎没有提供seed参数的选择

  wz0424 asked Apr 16, 2024 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  想知道开发维护人员一般用HuggingFaceCausalLM还是VLLM

  IcyFeather233 asked Apr 12, 2024 in Q&A · Closed · Unanswered

  

  

  5
• 1 You must be logged in to vote

  🙏

  Can't run openai api eval

  kratorado asked Apr 27, 2024 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  Testing a local Hugging Face model - Configuration File Required?

  siftxxx asked Apr 23, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  [bug] MT-Bench only evaluate the second turn.

  notoschord asked Apr 25, 2024 in Q&A · Unanswered

  

  

  8
• 1 You must be logged in to vote

  🙏

  在单个模型对多个partitioned数据集做eval的时候，模型会被重复加载

  IcyFeather233 asked Apr 25, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp-a34b3233.so.1 library.这个报错是哪个地方触发的

  IcyFeather233 asked Apr 25, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  opencompass 和 lm-evaluation-harness 选型对比

  hh1562535601 asked Apr 23, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  修改配置文件，无法测试本地模型，没有报错信息[Bug]

  New-World-2019 asked Jan 15, 2024 in Q&A · Unanswered

  

  

  

  

  12
• 1 You must be logged in to vote

  🙏

  为什么启动示例执行推理的时候一直卡着，几个小时都没进度，环境是notebook t4的环境

  pzfok asked Apr 13, 2024 in Q&A · Unanswered

  

  

  0