talk to documents - localdocs

[kalle07]

how can i force that the model only get information out of the document and dont halucinate ?!?

[e-novative]

Place a text file in a local directory. use the DISK icon, then 'Add & Remove" to add that directory, give it a name. Make sure the box is checked (green). When you close the window the app will built that reference. Now, do a query. It should say it's referencing that table.

[kalle07]

i know that ... but 19 of 20 models haluzzinate ^^

so in privateGPT the pre-prompt is like: "give only answers that you can find in the document"
and its seems very strict, answers can be: "i cant finde your word" and its true in this case !

[SINAPSA-IC]

You may want to try a more complex Prompt Template, at the expense of speed in the reply...
For instance,

"To answer to any question and to reply to any inquiry you will only use information that is not in your programming at that time, about the subject of the respective question or inquiry."
or:
"To answer to any question and to reply to any inquiry you will only use external information about the subject of the respective question or inquiry."
or:
"To answer to any question and to reply to any inquiry you will not use information that you already know about the subject of the respective question or inquiry."

Worth a try. Some models, though, will ignore this, I don't know why.

[kalle07]

thy ill try

pdfgear is very strict, you know ? ;)

[AgatheBauer]

My experience with RAG in GPT4All is really disappointing. Sometimes it finds something, but I just tried to find a phone Number which is definitely in the dataset, but the model said it couldn't find anything related. I installed SBert. What are your experiences?

[SINAPSA-IC]

Hi.
I do not put much trust in the ability of any LLM to deal with numbers; in some instances, some or all numbers (not many, which makes it even more sinister) were mixed up, or wrong.
Otherwise, if I feed the models PDFs as LocalDocs with a lot of info on a subject, and ask for a presentation, then all models try doing their best.
So, apart from numbers, text derived from info in LocalDocs seems OK to me... however, I do not know how many documents are being parsed and what the structure of the folders' categories should be for smooth operation.

[kalle07]

to find a number search in it with a normal pdf reader ^^ that is very useful : D

pdfgear find numbers ask
"can you find phone numbers in the document? if so, tell me!"
if pdfgear ask "search document say no" and than he tells you

[T100D]

Same experience, I am trying it on sysadmin first and secondline documents and the results are bad. I think this is due to the smaller LLM's and that we do not write them context rich.

When I try questioning documents on tooling like Mindpal or Cognosys the answers are better to almost on par, especially when you configure an agent to handle documents and answers in a curtain way. I see that they use a lower temperature by default (0,3) then in gpt4all (0,7). Using agent's for tasks is the way to go it seems.