One of the most criticized behaviors of AI-powered chatbots is so-called hallucinating, when the AI convincingly answers a question while providing you with factually incorrect information. Simply said, artificial intelligence is making things up in an attempt to satisfy its user.
Although what might sound like a trivial issue is —in reality, a very complex problem. Actually, we might not be able to completely cure this disease, a concern raised by many experts over the past year.
But are the large language models (LLMs) – the core technology behind all those popular generative AI chatbots like Bing, Bard, or ChatGPT – indeed broken? Andrej Karpathy, the co-founder of OpenAI and former senior director of AI at Tesla thinks the exact opposite. “In some sense, hallucination is all LLMs do. They are dream machines,” he says in a post on X (Twitter).
# On the "hallucination problem"
— Andrej Karpathy (@karpathy) December 9, 2023
I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the…
The Slovakia-born expert on deep neural nets and natural language processing hints we, the users are some sort of directors. We are using our prompts to initiate and steer this dreaming process to a hopefully useful result.
“It's only when the dreams go into deemed factually incorrect territory that we label it a ‘hallucination’,” continues Karpathy, adding that it only looks like a bug: “Hallucination is not a bug, it is LLM's greatest feature.”
At the same time, Karpathy is not hiding from the fact that AI chatbots indeed have issues. Although, what is important here is correctly defining the problem:
“I realize that what people ‘actually’ mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it.”
Former head of Tesla’s Autopilot development admits that “LLM Assistants” have a problem that we should fix. And there are several ways to tackle it, all “active and very interesting areas of research” according to Karpathy.
To solve a problem, you first have to be aware of its existence. And now, a little over a year since the generative AI boom started in 2022, all the researchers and developers are very well aware of the issue. In fact, it’s the commercial success of AI bots that drives the research and improvement of these tools, as the tech giants compete in the race for the best consumer product.
Image: Microsoft Bing AI
5 Comments - Add comment