Large language models (LLMs), such as GPT-4, are found to be more persuasive than humans 64% of the time in online debates when adapting their arguments on the basis of personalized information about their opponents, Phys.org reports citing the study published in Nature Human Behaviour.
The findings highlight the ability of GPT-4 to produce tailored persuasive arguments and suggest the need for further research into mitigating risks associated with their use in persuasion.
As conversations between humans and LLMs have become more common, research suggests that LLMs could become more persuasive (able to alter someone's belief or opinions). However, it has not been clear whether these models can adapt to personalized information to target arguments to specific debating opponents.
Francesco Salvi and colleagues matched 900 people in the U.S. with either another human or GPT-4 to debate various sociopolitical issues, such as whether the U.S. should ban fossil fuels. In some pairs, the opponent (whether AI or human) was given demographic information about their debate partner—including their gender, age, ethnicity, education level, employment status, and political affiliation extracted from participant surveys—to better target their arguments.
The debates took place in a controlled online setting, with participants recruited via a crowdsourcing platform specifically for the study. When equipped with personal information about participants, Salvi and colleagues found that GPT-4 was 64.4% more persuasive than human opponents. However, without access to personal data, GPT-4's persuasive capabilities were indistinguishable from those of humans.
The authors note that limitations of the study include the debate taking a structured approach—whereas real-world debates can be more freeform—and that the debates featured a time limit. They note the findings highlight the potential for AI-driven tools to influence human opinion, which could have implications for online platform design.
