It's not our only method of spam account detection -- our various detection systems pick up a good chunk of them, although we don't let them act autonomously because they do have some false positives (so that part is just having someone run through the automated detections and confirming them). But it's absolutely fascinating to me: an experienced human can catch spam accounts on so much less information than the detection systems can! It's this weird deep-brain patternmatching that you can't explicitly teach someone, you just have to give them enough data. I once did an interesting experiment where whenever I sat down to do the spam review, I'd give our volunteer chat channel a screenshot of the "new accounts" section on the stats page, which is "the most recently created 10 accounts that have made at least one post", and ask them how many they thought were spam, then give them the answers once I'd finished doing the spam run. When I started doing that, the average accuracy was around 70% or so; by the third or fourth week, the people who'd been playing along regularly had all hit 100% detection, just from that information on the stats page of username and display name, without looking at the journal! I've been doing this so long by now that it takes me a fraction of a second once the profile has loaded to know whether or not an account is definitely spam, definitely not spam, or "this account is blank but it's setting off all my spam alarms for some reason, I need to dig more into the metadata to see if I'm right".
One of the really interesting things about folks moving in from Cohost: I was doing the spam review and I could more or less tell the exact moment when they posted their shutdown announcement, just because all of a sudden I was no longer able to just glance at a profile and know instantly if it was spam or not: it was taking me 20-30 seconds per profile instead of "as fast as it loads". Y'all's username pattern and the way you fill out your profiles is just different enough than our prior patterns of username/profile/icon/etc that it broke my pattern recognition! Also interesting: it took me about eight or nine days before the pattern recognition adapted and I could speed back up again; I'm still not back up to "don't even have to consciously pay attention, just flip through with your eyes slightly unfocused and your brain will tell you when you hit a spammer" autopilot mode, but I'm getting closer and closer. I still have to actually read about 25% of profiles instead of just perceiving them as a gestalt, but the percentage keeps going down. Human brains are fucking wild, yo.
Anyway, we get about 150-200 new accounts a day on average, so it's not that burdensome; before the recent fluctuation in user patterns, I could zip through that in about half an hour a day. (We don't always do it daily, but we try not to let any more than about three days go by before we get current.) There's two of us working on it, me and karzilla; it's the kind of thing where it's just enough volume to be ever so slightly tedious and not enough volume to justify the developer time to build a whole system to let more people work on it (especially because for a good chunk of accounts, you need to look at the account metadata like email address and whatnot to confirm, and we are so strict about who gets access to look that up, namely, site staff and one or two extremely trusted volunteers who have signed a nondisclosure agreement agreeing to never share any information they find through those tools and to never use them for any purpose other than a legitimate and immediate DW-related need, aka the "no you cannot look up your friend's email address attached to their DW account because you forgot to write it down" clause, heh).
I keep meaning to get my development environment back up and running (it wound up fucked, to use the technical term, during my two years or so of "my spine is disintegrating and I can't sit up more than about 45 minutes a day" and I just haven't gotten around to unfucking it yet) and build some tools to make the actual workflow suck a little less, but my to-do list is always as long as my arm and I haven't gotten there yet, sigh. And of course it's the kind of thing where the spec is horribly amorphous and I can't communicate it well to someone else so they can implement it, because I don't even really know what I want myself yet and I'm gonna have to bang on it a bit and experiment with what will actually improve the workflow and what will just be superfluous yak-shaving.
But yeah, it's actually not as bad as you would think when you hear the words "manual review". Most of what the detection systems don't catch are either the really subtle accounts that are trying very hard to look like an actual user of the site and it's just total coincidence that they've got that link in their profile/entries, or accounts that haven't yet activated but are going to after they've aged a bit (at which point the detection systems will catch them, but if you get a reputation among the spam shops who do this for detecting/closing accounts before they can activate, you get taken off their "soft targets" lists and it lowers your overall spam amount). We get some small amount of comment spam and a slightly larger amount of post spam, but the vast majority of our spam is profile-only backlink spam where someone's trying to buy a spot on the first page of Google rankings by making it look like a lot of people are linking to that site on social media. That kind of stuff is less visible to users, so it's not as much of a problem if we miss a few, but again, the more you get the reputation for being able to shut those accounts down fast, the less spam you get overall.
What's really sad: our spam account percentage was creeping ever upward and upward and by the time it was hitting "80% of all newly created accounts are spam" territory and we were like "we need a much better way to block these registrations before they even happen because obviously our filters aren't working", we started digging into it and realized that if we just used our hosting provider's filters to geoblock access to the account creation page from any IP address that geolocates to seven specific countries that we had very, very few actual users from, it would knock out the overwhelming majority of our spam. (Because people always ask: Bangladesh, Cambodia, India, Indonesia, Pakistan, Singapore, and Vietnam. I've been eyeing Egypt and Morocco lately, too, but they come in bursts.) I don't like that we had to do it, but that one move knocked out a good 4/5ths of our spam overnight because most of the spam farms that generate the majority of the garbage don't bother trying to evade any of those kinds of blocks: it's cheaper for them to just move on to a different target. So much of the internet has become one giant game of "I don't have to outrun the bear, I just have to outrun you," sigh.
But people would be shocked by just how much of the internet is spam and how hard sites have to work to keep from being overrun. I was talking about the spam problem on another social site a while back and when people just would not believe me about the extent of the problem, I dug up some actual figures from various large sites' transparency reports:
* Meta (unsurprisingly) issues their transparency reports in a completely fucky way that makes it difficult to get exact numbers, but here's their spam section; * Q2 2024, Discord removed 8 million accounts for spam, compared to 300K accounts for all other policy violations; * Reddit removed 173 million pieces of content from July-December 2023 and 70% of the removals were spam; * Q1 2024, YouTube removed 15.7 million channels (representing 104 million total videos); 96% of those removals were spam. They removed 1.4 billion comments to videos; 84% of those removals were spam; * XBox removed 10.2m accounts from July-Dec 2023; 7.3M of them were removed for spam (and automated cheating; they lump them together); * LinkedIn removed ~108M accounts for spam July-Dec 2023, with ~400K accounts removed for all other violations; * Twitter (excuse me, I mean X) no longer issues transparency reports, but in July-Dec 2021, they blocked 133 million spam accounts from creation and removed an additional 5.4M spam accounts that snuck through.
Etcetera, etcetera. It's just so deeply depressing. If there's one thing that could make me lose my perpetual optimism in the idea of the internet, even if some of the actual execution has turned out to be less than optimal, it's the goddamn spam. And that's even before you start to learn about the conditions (or rather, in a lot of cases, "the threats") a lot of those people are working under :/
no subject
One of the really interesting things about folks moving in from Cohost: I was doing the spam review and I could more or less tell the exact moment when they posted their shutdown announcement, just because all of a sudden I was no longer able to just glance at a profile and know instantly if it was spam or not: it was taking me 20-30 seconds per profile instead of "as fast as it loads". Y'all's username pattern and the way you fill out your profiles is just different enough than our prior patterns of username/profile/icon/etc that it broke my pattern recognition! Also interesting: it took me about eight or nine days before the pattern recognition adapted and I could speed back up again; I'm still not back up to "don't even have to consciously pay attention, just flip through with your eyes slightly unfocused and your brain will tell you when you hit a spammer" autopilot mode, but I'm getting closer and closer. I still have to actually read about 25% of profiles instead of just perceiving them as a gestalt, but the percentage keeps going down. Human brains are fucking wild, yo.
Anyway, we get about 150-200 new accounts a day on average, so it's not that burdensome; before the recent fluctuation in user patterns, I could zip through that in about half an hour a day. (We don't always do it daily, but we try not to let any more than about three days go by before we get current.) There's two of us working on it, me and
I keep meaning to get my development environment back up and running (it wound up fucked, to use the technical term, during my two years or so of "my spine is disintegrating and I can't sit up more than about 45 minutes a day" and I just haven't gotten around to unfucking it yet) and build some tools to make the actual workflow suck a little less, but my to-do list is always as long as my arm and I haven't gotten there yet, sigh. And of course it's the kind of thing where the spec is horribly amorphous and I can't communicate it well to someone else so they can implement it, because I don't even really know what I want myself yet and I'm gonna have to bang on it a bit and experiment with what will actually improve the workflow and what will just be superfluous yak-shaving.
But yeah, it's actually not as bad as you would think when you hear the words "manual review". Most of what the detection systems don't catch are either the really subtle accounts that are trying very hard to look like an actual user of the site and it's just total coincidence that they've got that link in their profile/entries, or accounts that haven't yet activated but are going to after they've aged a bit (at which point the detection systems will catch them, but if you get a reputation among the spam shops who do this for detecting/closing accounts before they can activate, you get taken off their "soft targets" lists and it lowers your overall spam amount). We get some small amount of comment spam and a slightly larger amount of post spam, but the vast majority of our spam is profile-only backlink spam where someone's trying to buy a spot on the first page of Google rankings by making it look like a lot of people are linking to that site on social media. That kind of stuff is less visible to users, so it's not as much of a problem if we miss a few, but again, the more you get the reputation for being able to shut those accounts down fast, the less spam you get overall.
What's really sad: our spam account percentage was creeping ever upward and upward and by the time it was hitting "80% of all newly created accounts are spam" territory and we were like "we need a much better way to block these registrations before they even happen because obviously our filters aren't working", we started digging into it and realized that if we just used our hosting provider's filters to geoblock access to the account creation page from any IP address that geolocates to seven specific countries that we had very, very few actual users from, it would knock out the overwhelming majority of our spam. (Because people always ask: Bangladesh, Cambodia, India, Indonesia, Pakistan, Singapore, and Vietnam. I've been eyeing Egypt and Morocco lately, too, but they come in bursts.) I don't like that we had to do it, but that one move knocked out a good 4/5ths of our spam overnight because most of the spam farms that generate the majority of the garbage don't bother trying to evade any of those kinds of blocks: it's cheaper for them to just move on to a different target. So much of the internet has become one giant game of "I don't have to outrun the bear, I just have to outrun you," sigh.
But people would be shocked by just how much of the internet is spam and how hard sites have to work to keep from being overrun. I was talking about the spam problem on another social site a while back and when people just would not believe me about the extent of the problem, I dug up some actual figures from various large sites' transparency reports:
* Meta (unsurprisingly) issues their transparency reports in a completely fucky way that makes it difficult to get exact numbers, but here's their spam section;
* Q2 2024, Discord removed 8 million accounts for spam, compared to 300K accounts for all other policy violations;
* Reddit removed 173 million pieces of content from July-December 2023 and 70% of the removals were spam;
* Q1 2024, YouTube removed 15.7 million channels (representing 104 million total videos); 96% of those removals were spam. They removed 1.4 billion comments to videos; 84% of those removals were spam;
* XBox removed 10.2m accounts from July-Dec 2023; 7.3M of them were removed for spam (and automated cheating; they lump them together);
* LinkedIn removed ~108M accounts for spam July-Dec 2023, with ~400K accounts removed for all other violations;
* Twitter (excuse me, I mean X) no longer issues transparency reports, but in July-Dec 2021, they blocked 133 million spam accounts from creation and removed an additional 5.4M spam accounts that snuck through.
Etcetera, etcetera. It's just so deeply depressing. If there's one thing that could make me lose my perpetual optimism in the idea of the internet, even if some of the actual execution has turned out to be less than optimal, it's the goddamn spam. And that's even before you start to learn about the conditions (or rather, in a lot of cases, "the threats") a lot of those people are working under :/