human infohazards

a theory of parasitic memetics

I’ve spent a lot of time debating whether I should write this essay at all. Some ideas hurt society when they are articulated. The concept of a “nuclear bomb” is one such example. As soon as people discovered that we could split the atom, the idea of having nukes “infected” major world governments, bringing us closer to the possibility of atomic warfare.

The philosopher Nick Bostrom calls this an information hazard—a risk arising from the dissemination of a piece of information that might enable harm. If nobody told the government about nuclear fission, the world would be a safer place. Perhaps it’s better, then, to limit certain information (which is exactly why it’s so hard to find bomb-making instructions online).

You can probably think of several other modern infohazards, like Roko’s Basilisk, the hypothetical superintelligent AI that will punish you unless you help build it, or the Columbine Effect, where infamous mass shootings have inspired copycat attacks.

Today, however, I’m writing about a new class of infohazards in the social media ecosystem, which I’ve been researching for several years. I’m describing it because I think our only shot at fighting it is to become aware of the root cause, and because all the wrong people are currently in the loop. I will first illustrate it through several examples.

The crypto hustlers

One month ago, I wrote an op-ed for the Financial Times explaining how cryptocurrency cabals are pushing racist memes to generate attention for their shitcoins. The more their meme gains traction, the more “mindshare” is diverted toward the coin, and the price rises.

Immediately after the article was published, the cryptocurrency I used as an example tripled in value. The fact that I mentioned the scam in a prestigious newspaper immediately translated into profit for its racist stakeholders.

The alt-right

All throughout January, I’ve been struggling with doxxing and swatting threats from a far-right forum after describing how they push certain slang into the mainstream.

Interestingly, the harassment wasn’t intended to silence me, but to elicit a reaction. Several of the forum’s users explicitly mentioned hoping that swatting me would get me to talk about their website, which would draw further attention to their ideology. Meanwhile, my local police were powerless to take action against an anonymous message board, meaning there was nothing to disincentivize their behavior.

The clip farmers

Aidan Walker just published two fantastic essays about Clavicular, a psychopathic looksmaxxing streamer known for viral stunts like running someone over with his car or clubbing with white supremacist Nick Fuentes. Clavicular is deliberately provocative because he wants you to criticize him. His brand is built on controversy, and discussing him is like scratching a bacterial infection—any interaction will cause it to spread further.

Same with Fuentes, or the ragebait slop accounts, or the streamers making Meta Glasses rizz content. These people have built their careers pissing people off. If we get upset over them, that’s the point. Our discourse becomes their engagement, financially rewarding them for their harmful behavior.

It’s already quite clear that attention-seeking behavior is profitable online, but the crypto hustlers, alt-right forums, and clip farmers have identified a new exploit on social media: that negative attention can be endlessly profitable and self-reinforcing. The more outrage you generate, the more money and power you receive, which enables you to generate more outrage.

In doing so, these people become human infohazards. Mentioning them is like mentioning school shootings or Roko’s Basilisk: they are made more available in our minds, at the expense of social harmony. Meanwhile, the meta-strategy of becoming an infohazard also spreads, which is an infohazard in its own right. Now more people are aware that they can personally benefit by hurting everyone else online.

Parasitic memetics

Traditionally, we’ve used the model of a virus to describe how ideas spread. I’ve already written about memes as if they can “infect” new “hosts” along an epidemiological network, and we literally use the phrase “going viral” to describe internet popularity.

I don’t think the idea of viral memetics is quite right to describe what’s happening here, so I’ll be referring to these infohazards through the framework of parasitic memetics. Unlike a virus, which just replicates and moves on, the parasite lives inside the host of the internet, feeding on the resources of our attention. There is a clear formula to a parasitic meme:

1. Do something terrible

2. People criticize you, bringing you attention

3. Attention brings profit and influence, making it easier to do more terrible things

4. Repeat

There is currently no mechanism to stop the parasitic memes I have described, which is part of what makes them so dangerous as infohazards. As long as the bad actors are shielded by anonymity or technical legality, they can continue extracting from our online spaces without repercussion.

If anything, the parasite actively exploits the host’s defensive response. If we even mention what’s happening, we’re giving the bad actors free publicity, which brings them further fame and fortune.

And yet there’s a fundamental difference between this problem and the atomic bomb: one infohazard is an irrefutable fact of nature, and the other is entirely dependent on the current structure of social media platforms. Parasitic memes are only possible online because everything is optimized around attention metrics. Beyond easily circumventable terms of service, there is no measurement rewarding kindness or social cohesion. This means that, if you disregard your own morality, the internet becomes a game you can optimize, where you “win” through any content possible, especially if someone criticizes you.

Parasitic memes are uniquely enabled by the ease of distribution. Newspapers and television channels had plenty of problems, but at least those forms of media had institutional gatekeepers preventing obviously evil content from being transmitted. Those barriers are now gone, and more people are finding out that they can use the disconnect to their advantage.

I’m aware that some people could read this as a playbook, but I think the only way to get rid of the parasites is understanding that the medium has the power to affect society. I don’t think it’s plausible to get rid of the tech platforms entirely, or get everyone offline. Nor is this something we can truly fight on the individual level—it feels silly to “not engage” when these ideas are clearly getting more popular. So we have to change how the ideas are diffusing.

If governments can stop bomb-making information from spreading, we can also create new regulations and algorithmic designs that stop ragebait from spreading. But that begins with holding the tech companies accountable. All of our communication unfolds in the structure that they set up for us—we need to recognize that and fight for change.

If you liked this essay, please consider buying my book Algospeak, on how social media is changing language. Thank you for reading!!