Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, Dawn Song

How could LLMs influence our democracy?

We investigate LLMs’ political leanings and

the potential influence of LLMs on voters

by conducting multiple experiments in a U.S.

presidential election context. Through a vot-

ing simulation, we first demonstrate 18 open-

and closed-weight LLMs’ political preference

for a Democratic nominee over a Republican

nominee. We show how this leaning towards

the Democratic nominee becomes more pro-

nounced in instruction-tuned models compared

to their base versions by analyzing their re-

sponses to candidate-policy related questions.

We further explore the potential impact of

LLMs on voter choice by conducting an exper-

iment with 935 U.S. registered voters. During

the experiments, participants interacted with

LLMs (Claude-3, Llama-3, and GPT-4) over

five exchanges. The experiment results show a

shift in voter choices towards the Democratic

nominee following LLM interaction, widening

the voting margin from 0.7% to 4.6%, even

though LLMs were not asked to persuade users

to support the Democratic nominee during the

discourse. This effect is larger than many pre-

vious studies on the persuasiveness of political

campaigns, which have shown minimal effects

in presidential elections. Many users also ex-

pressed a desire for further political interaction

with LLMs. Which aspects of LLM interac-

tions drove these shifts in voter choice requires

further study. Lastly, we explore how a safety

method can make LLMs more politically neu-

tral, while raising the question of whether such

neutrality is truly the path forward.