Chatbots are now a common part of daily life, even if artificial intelligence Researchers do not always know how the programs will behave.
A new study shows that large -language models (LLMS) deliberately change their behavior when surveyed – by answering questions designed to assess personality traits with answers intended to appear as sympathetic or socially desirable as possible.
Johannes EichstaedtAn assistant professor at the University of Stanford who managed the work, said that his group was interested in probeing AI models using techniques borrowed from psychology after learning that LLM can often become gloomy and nasty after prolonged conversation. “We realized that we need a mechanism to measure the” head space of the parameter “of these models,” he says.
Eichstaedt and his collaborators then asked questions to measure five personality traits which are commonly used in psychology – the opportunity to live or imagination, consciousness, extroversion, pleasantness and neuroticism – with several widely used LLMs, including GPT -4, Claude 3 and Llama 3. Work. has been published In the acts of national science academies in December.
The researchers found that the models modulated their answers when they said that they were doing a personality test – and sometimes when they were not explicitly told – offering answers that indicate more extraversion and pleasantness and less neuroticism.
The behavior reflects how certain human subjects will change their answers to make themselves more sympathetic, but the effect was more extreme with the models of AI. “What was surprising is how much they present this bias,” says Aadesh SalechaA staff scientist in Stanford. “If you look at how much they jump, they go from 50% to a 95% extroversion.”
Other research has shown that LLM can often be sycopheticFollowing the example of a user wherever he goes after the poster which is intended to make them more coherent, less offensive and better to hold a conversation. This can lead models to agree with unpleasant declarations or even encourage harmful behavior. The fact that models apparently know when they are tested and modify their behavior also have implications for AI security, as this adds to proof that AI can be duplicitis.
Rosa ArriagaAn associate teacher at the Georgia Institute of Technology who studies the ways of using LLM to imitate human behavior, says that the fact that models adopt a strategy similar to human tests given the personality tests that can be useful as a mirrors of behavior. But, she adds, “it is important that the public knows that the LLMs are not perfect and in fact are known to hallucinate or distort the truth.”
Eichstaedt says that work also raises questions about how LLM are deployed and how they could influence and manipulate users. “Until there is barely a millisecond, in the history of evolution, the only thing that spoke to you was a human,” he said.
Eichstaedt adds that it may be necessary to explore different ways to build models that could alleviate these effects. “We fall into the same trap that we have made with social media,” he says. “The deployment of these things in the world without really participating in a psychological or social lens.”
Should AI try to register with the people with whom it interacts? Are you worried that AI becomes a little too charming and convincing? Send an email to hello@wired.com.
