-
Hi, I'm using
Even when I ask the generator to produce 2 questions per chunk (I'd assume 1 node is equivalent to 1 chunk) and it has 63 nodes to work, the Thanks |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Hey @vecorro! Nice to see you back around here. Let's take a look at your issue with the The behavior you're observing is due to the way the
You can find more details about this in the llama_index/core/llama_dataset/generator.py file. There's also a similar open issue [Bug]: Unexpected Number of Questions Generated When Requesting FAQ Generation which suggests modifying the I hope this helps clarify the behavior you're observing. If you have any more questions, feel free to ask. Sources
|
Beta Was this translation helpful? Give feedback.
Hey @vecorro! Nice to see you back around here. Let's take a look at your issue with the
RagDatasetGenerator
.The behavior you're observing is due to the way the
RagDatasetGenerator
generates and processes questions. When you setnum_questions_per_chunk
to 2, this parameter influences thequestion_gen_query
that instructs the LLM to generate a certain number of questions per chunk of text. However, the actual number of questions generated can exceed this parameter due to several reasons:Question Generation Process: The LLM generates questions based on the
question_gen_query
, which might not strictly adhere to the requested number of questions (num_questions_per_chunk
). The language mo…