Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, Dawn Song
How could LLMs influence our democracy?
We investigate LLMs’ political leanings and
the potential influence of LLMs on voters
by conducting multiple experiments in a U.S.
presidential election context. Through a vot-
ing simulation, we first demonstrate 18 open-
and closed-weight LLMs’ political preference
for a Democratic nominee over a Republican
nominee. We show how this leaning towards
the Democratic nominee becomes more pro-
nounced in instruction-tuned models compared
to their base versions by analyzing their re-
sponses to candidate-policy related questions.
We further explore the potential impact of
LLMs on voter choice by conducting an exper-
iment with 935 U.S. registered voters. During
the experiments, participants interacted with
LLMs (Claude-3, Llama-3, and GPT-4) over
five exchanges. The experiment results show a
shift in voter choices towards the Democratic
nominee following LLM interaction, widening
the voting margin from 0.7% to 4.6%, even
though LLMs were not asked to persuade users
to support the Democratic nominee during the
discourse. This effect is larger than many pre-
vious studies on the persuasiveness of political
campaigns, which have shown minimal effects
in presidential elections. Many users also ex-
pressed a desire for further political interaction
with LLMs. Which aspects of LLM interac-
tions drove these shifts in voter choice requires
further study. Lastly, we explore how a safety
method can make LLMs more politically neu-
tral, while raising the question of whether such
neutrality is truly the path forward.