LLM Hallucinations on In-Context Data

chef_carol

I know of some benchmarks that LLMs do undergo, but I am no expert whatsoever. I think what I am wondering about is closest to TruthfulQA . The question came up when I heard of combining company data with LLMs by providing the internal, sensitive data in prompts à la RAG (of course, you do not want to fine-tune the model every time, because the data may change daily or even hourly, but instead supply it with the data in prompts). So the question...

Originally asked by Mathy on Stack Exchange (ai) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

How sure can one be about the correctness of the models outputs on these kinds of prompts? I do not want it to hallucinate here whatsoever; that would undermine the whole approach of using LLMs in this context? No way to be 100% sure, e.g. see Large Language Models Cannot Self-Correct Reasoning Yet . There can be some tricks to reduce or detect some hallucinations though. Are there benchmarks specifically made for this task (i.e., providing lots of information in context and then asking for things appearing in this context)? Yes, e.g. PDFTriage: Question Answering over Long, Structured Documents but we are waiting for the permission to release data+code.

Full answer by Franck Dernoncourt: https://ai.stackexchange.com/a/42611 (CC BY-SA 4.0).

Need it done for your business? UphillTech.