In a new study, Yale SOM’s Tauhid Zaman and Yen-Shao Chen show how a social media platform can shift users’ positions or increase overall polarization by selectively muting and amplifying posts in ways that appear neutral to an outside observer. Zaman says that the dangers of such “shadow banning” are much more immediate than the concerns that led Congress to force a sale of TikTok.
More clandestine than a straightforward ban from a platform, shadow banning limits the broader visibility of a user’s content without their knowledge. A Facebook or Instagram post that’s been subjected to shadow banning would remain on the original poster’s profile page, but it would appear less, or not at all, in the timelines of other users.
In a new paper co-authored with Yale SOM PhD student Yen-Shao Chen, Zaman takes up this phenomenon—not to determine whether it’s currently happening but instead to lay out exactly how it can be done and how powerful it can be.
For the study, the researchers built a simulation of a real social network, and then succeeded in using shadow banning to shift simulated users’ opinions as well as increasing and decreasing polarization. Even when the goal was to use shadow banning to move collective sentiment to the right or left, Zaman says, the content moderation policy appeared neutral from an outside perspective. That’s because it’s possible, he discovered, to shift opinions by turning down the volume on accounts on both sides of a debate at the same time.
“It’s like a frog sitting in a pot of water; the frog’s relaxing, and suddenly, he’s cooked,” Zaman said. “A network could, in fact, be driving people towards one point of view, but if someone tries to call them out on it—like a regulatory body—they’re going to see the network censoring both sides equally,” Zaman says. “It looks like there’s nothing untoward happening, so they leave the network alone—and suddenly everybody thinks the earth’s flat. That’s what we find you could do with our technique, which is a little scary.”