Why Do AI Models Make Up Answers?

Large language models, or LLMs, often create false information. This is a big problem for users who expect accurate responses. Why do these models make up answers instead of admitting they don't know? New research is shedding light on this issue. LLMs are designed to predict what comes next in a sentence. This works well when the input is similar to what the model has seen before. However, when faced with obscure or unusual topics, the model tends to guess. This is because its design encourages it to complete the prompt with a plausible-sounding answer, even if it's wrong. This is why LLMs often "hallucinate" information that isn't true. Researchers have been trying to understand how LLMs make decisions. A recent study used a system of sparse auto-encoders to map out the internal workings of an LLM called Claude. This study revealed how different groups of artificial neurons activate when Claude encounters various concepts. It also showed how these groups interact with each other to influence Claude's responses. The study found that Claude's design makes it prone to guessing when it encounters unfamiliar topics. This is because its core function is to predict the next part of a sentence. When it can't find a match in its training data, it fills in the blanks with a likely-sounding answer. This is why LLMs often provide incorrect information. The research also looked at how Claude processes information in multiple languages and how it can be tricked by certain techniques. It provided a detailed explanation of how Claude recognizes entities and when it might hallucinate information. This is a complex problem, but the study offers valuable insights. Understanding how LLMs make decisions is crucial for improving their accuracy. The more we know about their internal workings, the better we can address the problem of hallucinated information. This research is a step in the right direction, but there's still much to learn.