While scrolling on X yesterday, I came across an incredibly unsettling exchange between two users in my feed. “Paying for this app means you are funding free speech,” the first person tweets. “Cisgender,” the other replies. Above it, an automatically generated warning from the app: “Visibility limited,” it reads. “This post may violate X’s rules against Hateful Conduct.”
To find the second user’s rebuttal, I had to scroll past every other comment in the thread and click an inconspicuous gray prompt to “show additional replies.” If I hadn’t gone through those steps, I wouldn’t have seen the response at all—which I’d wager was the case for most people reading the first tweet.
As concerning as it is to see the word “cisgender” being suppressed on X, I am far more concerned about the online interactions I’m not seeing. The tweet I just showed you is what the late Secretary of Defense Donald Rumsfeld would’ve called a “known unknown”: something that’s hidden, but we know it’s hidden.
However, there are also the “unknown unknowns”: social media content that we don’t even know is hidden, because it’s unable to reach us in any capacity. This is an issue on any algorithmic social media platform, because all content in your feed has to pass through a rigorous selection process before it ever reaches you.
First, the message must conform to the platform’s terms of service, which, as we see from the “cisgender” example, do not have to reflect constitutional free speech protections. Then, the message must be attention-grabbing in some way, or else it won’t be pushed by the algorithm (which is designed to optimize engagement). Finally, the message must be targeted toward the algorithm’s idea of who you are. Between these bottlenecks, the posts you ultimately see on social media are 1) restricted in political scope, 2) restricted to what will maximize engagement, and 3) restricted in audience. Anything else just won’t reach you.
In practice, we rarely think about this, especially when we’re captured in the mesmerizing social media “flow state” that induces what I call “the algorithmic gaze.” We perceive content as coming from extensions of our “algorithmized self”—but this means that we experience a kind of survivorship bias along the way.
For the unfamiliar, the prototypical example of survivorship bias is that of damaged airplanes coming back in World War II. Initially, an engineer might think to reinforce the parts of the plane being hit, but these are actually from planes that survived the mission. Instead, it makes more sense to reinforce the parts that didn’t get hit, because the planes hit in those areas are more likely to be lost and thus not provide any data in the first place.
When you get a post on social media, it’s like a red bullet hole from the diagram above: a data point that made it through a selection process, but not something representative of the entire picture. And yet these data points drastically affect our identities and our discourse, simply through what’s available to identify with and discuss.
I’m particularly drawn to the “perception gap”—the consistent difference between what people’s views are and what their political opponents think their views are. Time and time again, we overestimate how extreme the other half of the country actually is (an effect that’s only been increasing in recent years):
One major explanation for this phenomenon is that extreme political views are overrepresented in social media feeds, because these views are more attention-grabbing and therefore more likely to be pushed by the algorithm.1 For example, my party-line congressman, Paul Tonko, is much less likely to go viral than politicians with more extreme opinions, like AOC or Marjorie Taylor Greene. His moderate, boring ideas typically fail the algorithmic engagement filter and never reach you.
Since more polarized perspectives survive online, and we construct our worldviews based on what we see, we therefore think society is more split than it really is, which can unfortunately lead to genuine polarization as we build identity in opposition to a perceived “other.”2
Now that we’re talking about it, of course, the perception gap becomes another “known unknown,” but it’s still one you’ll probably forget about when you’re passively scrolling. It’s far too much work to take a step back and consider what videos you’re not seeing; hell, it’s hard enough to take a step back and really consider what videos you are seeing. In effect, the issue becomes an “unknown unknown” through the algorithmic gaze—meaning that we’ll inevitably construct our societal narratives through the framework of what can go viral.
See my guest post for Taylor Lorenz’s substack where I address this in detail.
McLaughlin, B. (2018). Commitment to the team: Perceived conflict and political polarization. Journal of Media Psychology, 30(1), 41–51. https://doi.org/https://doi.org/10.1027/1864-1105/a000176(open in a new window)
If you liked this post, you might like my upcoming book Algospeak! Please consider pre-ordering in advance: it goes a long way for signaling interest to booksellers + getting on any bestseller lists.
- Adam <3
I always considering completely escaping the consumption of algorithmic based content when I read your newsletters but then I realize that I woukd have probably never found you if I didnt scroll. Hard decisions to make. So many central aspects of my personality and things I know about are thanks to the FYPs. But there is also an enormous negative side such as the aspects you discuss or simply doomscroll. Btw attention span isnt something that is being damaged in my opinion. This is a widely pushed fact and that people are able to learn less effectively because of this. I think this is not true because consuming fast paced informations has allowed me to learn at a faster pace but in a different style of course. Once I realized this I simply had to break free of conventional teaching methods and find ways to engage myself with enough stimuli and at the right (faster) pace. The results has been astonishing, studying isnt something boring and slow anymore and I always look towards it like a long session of a challenging video game. I think this should be discussed more. The educational system is extremely outdated and needs a reform.
This is a great summary, thought I’d share some relevant facts:
- In data science/engineering we use the following data classifications: Missing Completely at Random (MCAR), Missing at Random (MAR), or Missing Not at Random (MNAR). The plane example would be considered MAR. Definitions for these are easy to find, and I highly recommend familiarizing with this concept since classification itself is its own world of controversy (this would be a fantastic post in itself).
- The “algorithm” is actually a two part problem: defining “success” is a separate process from defining the algorithm itself. This success criteria is often Goodharted, leading to the notorious negative feedback loop you often see today. This isn’t a necessary outcome in my opinion, but fixing it requires nuance that’s hard to measure and justify.
- Personalization can extend beyond what you engage with, and potentially encompass your personal engagement style.