The Lizardman and the Black Hat Bobcat

post by Screwtape · 2025-04-06T19:02:01.238Z · LW · GW · 13 comments

Contents

  I.
  II.
  III.
  IV.
  V. 
None
13 comments

There's an irritating circumstance I call The Black Hat Bobcat, or Bobcatting for short. This essay is me attempting to give you a new term, so you have a word for it. Bobcatting is when there's a terrible behavior that comes up often enough to matter, but rarely enough that it vanishes in the noise of other generally positive feedback. 

The alt-text for this comic is illuminating. 

"You can do this one in thirty times and still have 97% positive feedback."

I would like you to contemplate this comic and alt-text as though it were deep wisdom handed down from a sage who lived atop a mountaintop. 

I.

Black Hat Bobcatting is when someone (let's call them Bob) does something obviously lousy, but very infrequently. 

If you're standing right there when the Bobcatting happens, it's generally clear that this is not what is supposed to happen, and sometimes seems pretty likely it's intentional. After all, how exactly do you pack a bobcat in a box and mail it by accident?

One potential answer is that if you do something a hundred times, weird things happen once in a hundred. If you do something a thousand times, there'll be ten one-in-a-hundred weirdnesses. If you do something ten thousand times, things can get very weird indeed. There's a story floating around the rationalist sphere about betting on whether a number is prime or not, and it turns out people mess that one up inside of a couple dozen tries.

Do I think I'd accidentally send someone a bobcat if I sent enough packages? No, I genuinely have a hard time imagining a plausible way for that to happen. I am going to make the extremely confident claim that if I packed office chairs into boxes and mailed them, I would be able to pack ten thousand chairs, thirty a day for a year, and zero people would receive a bobcat from me. 

I can totally imagine sending someone Yu-Gi-Oh cards instead of Magic cards, or sending a letter meant for my cousin to a friend from college. Not bobcats. I'm trying to imagine if it turned out I had sent a bobcat, what would have gone wrong, and my first answer is a bobcat snuck into the factory and sat in the box while I wasn't looking, then I picked up the box to put it on the truck (failing to notice it was twice the weight it should be) and the bobcat slept through that. Even that doesn't fully explain how this happened, because bobcats don't live in my state, and the nearest zoo doesn't have bobcats.

(My second answer is the box gets dropped on someone's porch, bobcats live wherever that person lives, the bobcat for some reason decides to tear into the box and sit down, and the customer doesn't notice the hole. I'm still pretty sure I could do ten thousand boxes with no bobcats, but as the number ramps up I acknowledge weird stuff happens.)

Even if it isn't intentional, you don't want people to pack bobcats in packages that are supposed to contain office chairs or sleeping bags or packs of granola bars. Whether it's an enemy or a malefactor [LW · GW] doesn't matter. Whether they're in conflict with you or just prone to mistakes doesn't matter. If you're Ebay, or even just a person who orders things on Ebay, you don't want unexpected bobcats. "A bobcat only happens 3% of the time" is still an unpleasant amount of bobcatting, and if it’s not intentional it indicates some serious quality control issues.

II.

The Lizardman Constant is a term that references the ~4% of the population which will give bizarre answers to surveys or feedback forms. Coined by Scott Alexander, it derives from a poll on conspiracy theories where approximately 4% of people said they believed shapeshifting lizardmen were running the earth. It’s possible this is because ~4% of the population genuinely believes in something I think is extremely implausible, but it could also be that ~4% of the population hits the wrong button on the survey or likes messing with your data. I think the Lizardman Constant generalizes to any kind of wide net looking for responses. 

I spend a decent chunk of my time poring over surveys. Among others, I have access to the ACX Everywhere surveys and the LessWrong Community Census. Once you get up past a hundred or so responses to any survey, I find if you read them closely you start to notice some. . . let's just call them outliers.

I am an Artificial General Intelligence. Or am I?

 

Firstly, do you know about Hava? Hava nice day, idiot, whoever reads this.

This is not helpful. I mostly tune these out and pay attention only to the ones with actionable critiques or ones I agree with once pointed out. Since there aren't many of these, it's pretty easy. They're also not a big problem, more of a nuisance. It's pretty obvious that calling me an idiot isn't an accident any more than mailing a bobcat instead of an office chair would be. 

I can imagine intending to joke around with someone and having the joke offend them instead. I'm no stranger to counter-signalling, I'm pretty sure people I know well enough to counter-signal friendship take the LessWrong Community Census, I can imagine a friend of mine running the census next year and me thinking it'd be funny and appreciated to put a joking insult in the extra comments field. I mostly think I'd do something better than "Hava nice day, idiot" but acknowledge someone might have misread the room.

I keep coming up with ways one bad experience could be an accident or a genuine mistake. Hanlon's Razor, never attribute to malice that which is adequately explained by stupidity. Bobcatting isn’t ever doing one weirdly bad thing, because we all do make mistakes or have bad days. The thing I’m pointing at is when the number of packages containing bobcats starts becoming a noticeable percentage of all packages, even if it’s still a clear minority. If I took a hundred surveys, I don't think I'd screw up the counter-signalling on three or four of them.

III.

The problem comes when your prior on both sides of weird complaints is about the same. I think of this as dealing with both Lizardmen and Black Hat Bobcats at the same time. 

Imagine you're looking at the feedback for an event with a hundred attendees, and three people complain of deeply unpleasant experiences with the same local organizer. Is that concerning? It takes two people to have an argument. It's not out of my model for three unreasonable people to show up to an event with a hundred attendees and make mountains out of molehills, though intuitively this feels a touch high. Depending on the selection effects for the event, one of them might be what I would euphemistically call eccentric, and there was no actual molehill in the first place.

Some context: I'm presently the ACX Meetup Czar. Handling complaints about meetups is one of the top things I'm supposed to be doing. I have looked at a bunch of feedback forms for events or a series of events that had hundreds of attendees.

Most of my events are for the rationality community. Saying 1% of people who try to show up to a rationalist meetup are a problem to have around isn't an unreasonable guess. But I also coordinate ACX meetups around the world. Twice a year for ACX Everywhere, I wind up with a spreadsheet of a bit under two hundred meetups run by different organizers. Saying 1% of the people who volunteer to run a rationalist meetup are a problem to have around also isn't an unreasonable guess, and the first I hear of it is usually when an attendee complains.

I say "a problem to have around" to cover a very wide and not at all specific ground. Maybe they insist the ACX Everywhere meetup be run on Robert's Rules of Order. Maybe they hit on half of the men who attend. Maybe they make every conversation into a heated diatribe on every bad thing the Catholic Church has ever done. 

None of those specific things have to my knowledge ever happened, but I've gotten reports of things in that rough genre.

(And then sometimes a splinter group goes on to commit enough murder to get on international news, which did happen and is the kind of thing that keeps a Meetup Czar on his toes.)

So take the following scenario: an attendee of an ACX Everywhere has reported that the the organizer went on a rant about how puppies should be ground into paste. I reach out to the organizer and they say nope, that didn't happen, but there was this one attendee who was kinda antagonistic and kept talking about how MI6 was giving them coded instructions. I don't have any contact information for anyone else at the event. What should I think?

I am making up the puppy/MI6 example. That said, this very rough sort of thing does happen, and there's more than one person out there who might be reading this and thinking I'm breaking confidentiality except I got e.g. the intelligence agency wrong.

(And milder examples happen more frequently, and provide fuzzier evidence of whether there's something badly wrong. "Probably this isn't important, but they made a joke about race/sex and it made me kind of uncomfortable." That's much easier to see as a one-off mistake than mailing a bobcat. I still try and make a note of it.)

IV.

Of course, sometimes you do get contacted by more than one person.

One report of a problem, when the subject of the complaint has probably interacted with a hundred people who would have complained to me if there was an issue, doesn't tell me much. If anything, I kind of expect it. By the fifth complaint, that's weird and notable. By the tenth, I'm suspicious. I talk about this process a bit more in Intransitive Trust [LW · GW], but the tilt towards suspicion can happen very quickly.

Ebay presumably expects a few one star reviews. Someone (or more realistically, some teams) in Ebay has a pretty good sense of how many one star reviews is normal for say, an office chair seller. They might sort the sellers with better reviews higher, but not view the expected number of complaints as a reason to take much action. 

If I was on that team, and a customer reported that instead of an office chair the package contained a live bobcat, I would assume the customer was nuts. If 10% of the seller's customers were reporting bobcats, that's different. Depending on the size and reliability of your dataset, you might be able to start spotting weird variance much earlier than 10%, maybe as fast as 3%. I might not be paying attention to all the comments in detail, but if someone sells a thousand office chairs and I get thirty reports of receiving a bobcat (especially if they're private reports instead of where other users could see and copy it) then that's pretty weird. 

I want to point out something significant here: In the XKCD example, 97% of the customers are getting what they ordered.

If Ebay notices the bobcatting and shuts this seller down, then the overwhelming majority isn't going to understand the decision. If Bob the Black Hat was doing something unusually nice for the rest of the customers, like offering a good deal or especially attentive customer service (remember, an intentional Bob knows he can't afford to have the normal amount of negative reviews since he's spending his negative review budget on bobcats) then there's going to be some customers complaining at Ebay. 

"What gives," they might say, "Bob's Emporium was great, he answered my questions really fast and always had the cheapest price." Even if they don't complain to Ebay, they might still grumble to each other. "I miss Bob's Emporium, Ebay is oversensitive to a few bad complaints. I don't understand it. They closed a great store."

Bob has every reason to encourage this line of reasoning. "Yeah, Ebay freaked out over no big deal. I got a few bad reviews, but those reviews were totally bogus." Bob is lying, or maybe not exactly lying just kind of misleading. Remember, Bob's the kind of guy who sent people live bobcats when they expected office chairs. Even in the case where this wasn't deliberate, it still implies a concerning lack of quality control.

But Ebay is closed for him, so maybe Bob goes off to Amazon or Etsy. If he sends his former customers a final email telling them where to find him in the future, their direct experiences suggest it might be worth following him. 

Or, to use a different point of view: if you observe The System (say, Ebay, or the organizers of an event) seems pretty set against your friend Bob, even though Bob's been nice to you and the first two people that come to your mind. It could be because The System is getting worked up about something that's not a big deal. It could also be because Bob's done the equivalent of mailing a bobcat to people you don't really know. Consider whether you would actually notice anything different if Bob had done the bobcat thing. Consider whether you're debating if Bob is just under or over the threshold where The System should take action.

“But he’s always been nice to me” is some evidence that Bob is nice to everyone else! Lots of jerks are jerks to everyone. That said, “nice to most people but terrible to a few” is an archetype that exists.

V. 

Black Hat Bobcats are hard to spot, but thankfully they rarely hunt in packs. That prospect is a nightmare.

Consider the ACX Everywhere scenario. Two sides, about equally likely on priors to do something outrageous. Both claim the other person did something absurd. Except this time, one of them has  three good friends, good enough to report how the other side is being a problem while eliding over anything their friend did. Call the two belligerents Carla and Debbie, where Debbie is the one with friends.

Three friends isn't that hard to have, especially if Debbie's not asking them to lie. Three people who are weird in whatever way Debbie is weird also isn't impossible. Maybe the weirdness is something they bonded over or maybe they taught it to each other. Maybe it's just the same kind of selection effects that mean attendees at an ACX Everywhere meetup are way more likely than the general population to work in software even though Scott doesn't write about Javascript. 

If Debbie and her friends can keep their stories straight and avoid coming to too much detailed inspection, this is going to look like Carla is the problem. That's even more true if the close association isn't apparent from a distance and this looks like reports that aren't talking to each other.

Remember, there's about two hundred ACX meetup organizers. By dint of hard work and effort, I think I've managed to talk to maybe half of them at least on the level of a half hour audio call. I do not remotely think my social radar is good enough to spot any and all issues via a half hour call. Almost no ACX meetup is in the habit of keeping an updated membership list and sharing that to the ACX Meetup Czar. That means an organizer could send me an email cc’ing three other email addresses saying that Carla is advocating grinding puppies into paste, everyone at the local meetup heard it, here's three witnesses. 

Maybe it's true! It's probably true, right, that's sort of the way to bet. Except, if I'm in a slightly paranoid frame of mind, I notice that the concrete evidence I have is someone wrote an email and cc’ed a few other addresses which may or may not correspond 1:1 with actual people and those people may or may not be telling me the whole truth.

(CONSTANT VIGILANCE!)

So if you observe someone who's done something outrageous to you like mail you a live bobcat instead of your nice office chair, and you report it and the seller's friends report you, and the marketplace seems like it's not noticing this obvious problem, c'mon, can't they see the picture of the bobcat and your hospital bills. . .

. . . It's not obvious at scale whether you're part of the Lizardman Constant, or there's a Black Hat Bobcatter on the loose.

13 comments

Comments sorted by top scores.