This is an AI startup that prosperous without unraveling its secrets. In a surprising admission, the CEO of Anthropic reveals the opacity of AI models.
Anthropic, an advanced startup in the generative AI, is best known for its Claude model. However, according to his CEO, Dario Amodei, the creators still ignore the precise mechanisms that underlie the decisions of the AI. An admission formulated with a rare franchise, recently published in a test on his personal site.
Generative AI still escapes those who conceive it
“” No one really understands how these systems work », The CEO of the startup IA Anthropic. Generative AI write, summarize, create images. However, Amodei recalls that these models act as black boxes.
When a word is chosen, when an error arises, it is impossible to clearly grasp what motivated this decision. This opacity comes from their very design.
The models train on huge data sets, extracting statistical reasons that their creators do not control Really. The process resembles the growth of a plant. The conditions are laid down, but the final form is unpredictable.
The urgency of interpretability tools
This ignorance on AI is not trivial. The opacity of the models complicates the detection of unexpected or problematic behavior.
Amodei, who co -founded Anthropic in 2021 after leaving Openai for security -related disagreements, made this question a priority.
The startup targets a safer AI, but it requires an understanding of what is happening inside the models. For that, The team explores mechanistic interpretability – an approach that seeks to decode artificial neurons, such as a MRI scanning a human brain.
Progress is promising, but limited. Anthropic spotted millions of characteristics in its modelsassociated with objects or ideas. However, the majority remain a Superimposed notions chaosa phenomenon called superposition.
Certain experiences, such as that of a model obsessed with the Golden Gate Bridge, prove thatIt is possible to manipulate these characteristics. But understanding the entire system remains out of reach.
The more the models grow, the more complex the task. And time is running out, faced with the acceleration of AI.
.