Spam has always been a challenge in the fediverse. If you run a server with open signups, that don’t need admin approval, than you get direct spam, and this has been a recurring problem over the history of the fediverse. This problem has been annoying, but also manageable in some way: a server that suddenly gets targeted by bots and sends spam out to other servers get defederated fairly quickly, with mods from servers coordinating and alerting each other if the spam waves moves to another server that has their signups also open.
What most servers that were serious about running a community moved to is approval-based signups, where a human moderator reads a short application and decides whether to let someone in, usually via a form that asks some questions about who are you and why do you want to join. For the longest time, spam bots could not write convincing text about why it wanted to join a specific server, and so for years this worked well enough.
Enter LLMs.
What moderators across the fediverse are reporting, in the last two to three months, is that the application form has stopped working. Alice, a moderator at lgbtqia.space, writes that over 80% of their instance applications are now AI-generated, and Britt at mstdn.games reports the same ratio. Felix Stalder, who runs the small instance tldr.nettime.org, observed the applications going from 10% to over 90% bot in a matter of weeks. The pattern is consistent across instances of different sizes, which tells you this is the new default open signup layer now, and not just a targeted campaign against a few specific servers.
Stalder’s description of what this feels like from the moderator’s side:
We now have to run a some intuitive, semantic reverse CAPTCHA test in our head to decide whether to hit the accept or reject button. […] It forces us to operate under the assumption that every request is made by a bot. Until proven otherwise.
That inversion is the structural shift, I think. The default has flipped, and very application is presumed bot until a human can convince themselves otherwise, with the cognitive cost of making that judgment falls entirely on an unpaid moderator who has to run a private Turing test in their head for every single signup. This is now the daily experience of running a fediverse instance.
lbgtqia.space moderator Alice:
I’m getting burnt out on all my moderation actions being against fucking AI. Like, I never thought I’d say it, but I miss suspending Nazis and bigots — at least they were real people who would give up after a while — these LLMs just go on and on, and they don’t give a shit if they’re suspended or rejected.
The moderation action has lost its deterrent function. Suspending a Nazi meant something because the Nazi had a self that could be discouraged, and eventually would give up. Suspending an LLM bot means nothing because the bot generates a new signup, and the bot has no self and no feelings. The thing being moderated has no investment in any individual account, and the cost of generating a replacement is effectively zero.
The first obvious problem here is the workload this creates for moderators, which is serious enough on its own. The work is unpaid volunteer labour, done by people because they care, and it is demoralizing in a way that ordinary moderation is not. But I think the workload is actually the expected consequence of more bots, and that nobody will be surprised to hear that more bots means more work.
The bigger problem is that the screening mechanism of approval-based signups has lost discriminative power in both directions at the same time.
The first part is that bots do get through. If an adversary puts in effort into their bots, the output of a bot becomes indistinguishable from a human. LLMs are extremely capable of generating plausible text, after all, and a sign-up form for a fediverse server is screening for plausible text. Server admins are saying that bots are getting through now, even with approval-based systems, and that they can be fairly sophisticated about this, for example by laying dormant for a few weeks making a few benign posts, before switching to spam. This problem is not only spam in the ‘click-here-for-free-dogecoin’ sense, IFTAS reports that the fediverse has been targeted by the Russian botnet Portal Kombat for misinformation and propaganda.
The other part is that humans get rejected. Shlee at aus.social ran a test by signing up for five major reviewed Mastodon instances with a deliberately stylized pitch. Three of five instances denied the application, either because the pitch read as “too LLMy” or because the email was not from a major provider. Shlee’s follow-up observation is that if your anti-LLM detection is “does this string contain an em dash,” you have a bigger problem. But the deeper point there is that the detection regime is now understandably selecting against applications which writing has a Claude mouthfeel, which makes it likely but not guaranteed it is a bot application.
But I think the damage goes further than moderation workload, and into something that is more fundamental to how the fediverse actually works as a social space.
The signup form on a small instance was purely an anti-spam mechanism. For a regional server for a local community, or a server for people with a shared hobby, the signup form is also a social ritual. It let the admin curate who joins their community, and it let the newcomer signal genuine interest. It gives the newcomer a way to tell something about themselves, like why they care about the hobby, or why they to connect with people in their region. The form was a first point of contact, a first simple new social connection between the newcomer and the community they were trying to join.
In an article I wrote in February about where community lives on the open social web, I argued that communities tend to form through encounter, through showing up at the same place at the same time and discovering shared interests through proximity. The signup form was the digital version of that: walking through the door of a community and telling them who you are. Moderators on small instances, as Mud at kolektiva.social describes, often found these texts “heartwarming.” Reading them it was how you met your new members, and a first step to build social connections and helping new people onboard into the community.
LLMs have destroyed the signal capacity of that channel. The application text has been automated to near-zero cost, and the moderator can no longer tell whether the person on the other end actually cares about woodworking or lives in your region or wants to join your community for any reason at all. The old spam problem of the internet was that unwanted messages reached people, which is bad. But this is worse in a specific way: it does not just pollute the content layer, it blocks the community formation layer. An instance that cannot reliably accept new members cannot grow, and an instance whose moderators burns out from reviewing fake applications may just wind down altogether. The biggest threat here is not the spam on the timeline but to the social substrate of community servers.
There are things that can be done to reduce the pressure, like sharing lists of throwaway email domains between servers, or coordinating on common application patterns. These are useful and IFTAS has been continuing to doing important work in this area. But there is a structural limit here, I think. These are pattern-matching defenses working against machines that can generate infinite new patterns. The defender’s coordination cost, getting thousands of independent instances to agree on and implement shared standards, exceeds the attacker’s adaptation cost by a significant margin. And because the attacker’s marginal cost per attempt is effectively zero, the arms race is structurally tilted towards the bots.
I wrote about this coordination problem in more abstract terms last October, in an article about why better moderation systems for the fediverse struggle to gain traction. The core challenge is that the fediverse gives you sovereign control over your own server, and that is a meaningful version of autonomy, but it is limited to your own server. The most challenging parts of moderation do not respect server boundaries, they are cross-network problems, and solving them requires collaboration across servers and softwares. That article looked at tools like FediMod, FIRES and Fediseer, systems designed to help servers share information about bad actors, and observed that these tools are only valuable if many servers participate. If only a few servers participate, the signals are too sparse to be useful. They become valuable once a significant portion of fediverse servers participate. This is the classic coordination problem, where individual servers are not incentivised to participate until the system has critical mass, but it cannot reach critical mass until many individual servers participate. The bot signup crisis is this same coordination problem, but worse, because the thing being coordinated against is not a static set of bad actors but an adaptive adversary that changes its patterns faster than the coordination infrastructure can adapt to them. To get a sense of how hard the baseline coordination problem is: for sharing information about bad actors, the fediverse still to a meaningful degree relies on the #fediblock hashtag, a tool created years ago out of necessity that the network has never managed to replace despite it being clearly inadequate for the current scale. If the fediverse has not been able to build shared infrastructure for known harassers, building shared infrastructure for distinguishing LLM bots from genuine applicants in real time is a significantly harder problem.
What makes this actually hard, and not just difficult in the sense that every spam problem is difficult, is that the fediverse’s architecture is both the thing that makes it worth defending and the thing that prevents it from deploying the defenses that worked elsewhere. Email went through this transition in the 2000s, and the eventually landed on reputation systems, and centralised filtering run by large providers. It worked, but it worked by concentrating trust in a handful of actors who could operate at sufficient scale to make the economics of filtering viable. The fediverse does not have those actors, as well as that the culture of the network, with its goals of decentralisation, make it structurally hard to get buy-in for such systems, with people rightfully being skeptical if this is even desirable.
Previously I wrote about how the fediverse’s current resilience was produced by a specific wave of people who decided to run servers at a specific moment in time, and that the network is not really replacing its operator base. The bot signup crisis adds another layer of pressure on that same operator base. The moderator who burns out from reviewing fake applications. and the admin who cannot justify the time cost of running a server that is now mostly a target for automated abuse, these are the same people whose willingness to run servers is the thing holding the network up. I wrote there that the open socials movement has a tendency to fall into protocol eschatology, the belief that the future success of open protocols is guaranteed because the protocols are good. The bot signup problem is a useful corrective to that thinking, because it is a problem that will not be solved by the protocol being good. It will be solved by specific people doing specific coordination work, or it will not be solved at all.
I don’t have a clear suggestion on the way to move forward in the technical sense. LLM generated spam poses a threats to the wider ecosystem of news and online communications in general, and the response so far has made it clear that people do not yet have a good solution on how to handle it. So maybe the best way forward is to flip the question around, away from LLMs and bots: how do we actually want to build and shape our online social spaces, where humans can build meaningful connections with other humans?
