Posts
Comments
S-risks are barely discussed in LW, is that because:
- People think they are so improbable that it's not worth mentioning.
- People are scared to discuss them.
- Avoiding creating hypersititous textual attractors
- Other reasons?
Damn, reading Connor's letter to Roon had a psychoactive influence on me; I got Ayahuasca flashbacks. There are some terrifying and deep truths lurking there.
It's not related to the post's main point, but the U-shape happiness finding seems to be questionable. It looks more like it just goes lower with age by other analyses in general this type of research shouldn't be trusted
The U-shaped happiness curve is wrong: many people do not get happier as they get older (theconversation.com)
Oh, come on, it's clear that the Yudkowsky post was downvoted because it was bashing Yudkowsky and not because the arguments were dismissed as "dumb."
Thank you for your response, Caerulea. Many of the emotions and thoughts you mentioned resonate with me. I truly hope you find peace and a sense of belonging. For myself, I've found solace in understanding that my happiness isn't really determined by external factors, and I'm not to blame or responsible for the way the world is. It's possible to find happiness in your own bubble, provided you have the necessary resources – which can sometimes be a challenge
Because you have a pretty significant data point (That spans millions of years) on Earth, and nothing else is going on (to the best of our knowledge), now the question is, how much weight do you want to give to this data point? Reserving judgment means almost ignoring it. For me, it seems more reasonable to update towards a net-negative universe.
Maybe, and maybe not.
I agree that looking at reality honestly is probably quite detrimental to happiness or mental health. That's why many people opt out of these conversations using methods like downvoting, sneering, or denying basic facts about reality. Their aim is likely to avoid the realization that we might be living in a world that is somewhat hellish. I've seen this avoidance many times, even in rationalist spaces. Although rationalists are generally better at facing it than others, and some like Brian Tomasik and Nate Soares even address it directly.
I've spent a lot of time thinking about these issues – not necessarily a wise choice. I'd humbly advise you to reconsider going down this rabbit hole. I haven't penned down my conclusions yet, which are a bit idiosyncratic (I don't strictly identify as a negative utilitarian). But to summarize, if you believe that conscious experience is paramount and that pain and suffering are inherently bad, then our world is probably net negative. This perspective isn't just about humans; it's about foundational principles like the laws of physics and evolution.
It doesn't seem much of a stretch to argue that things are already way beyond the threshold and that it is too late to salvage the situation?
Interestingly, I still harbor hope. Maybe, for consciousness to emerge from nothing, life had to endure the brutal phase of Darwinian Evolution. But the future could be so bright that all the preceding suffering might be viewed as a worthy sacrifice, not a tragedy. Think of the pain a mother experiences during childbirth as a metaphor (but this birth has lasted millions of years). Alternatively, consciousness might vanish, or the world could become truly hellish, even more than its current state. The outcome isn't clear, but I wouldn't exclude any of these options.
You don't need a moral universe; you just need one where the joy is higher than suffering for conscious beings ("agents"); There are many ways in which it can happen:
- Starting from a mostly hostile world but converging quickly towards a benevolent reality created by the agents.
- Existing in a world where the distribution of bad vs. good external things that the agent can encounter is similar.
- Existing in a hostile world, but in which the winning strategy is leeching into a specific resource (which will grant internal satisfaction once reached)
I'm sure you can think of many other examples. Again, it's not clear to me intuitively that the existence of these worlds is as improbable as you claim.
You're right about my misunderstanding. Thanks for the clarification.
I don't think the median moment is the Correct KPI if the distribution has high variance, and I believe this is the case with pain and pleasure experiences. Extreme suffering is so bad that most people will need a lot of "normal" time to compensate for it. I would think that most people will not trade torture to extend their lives in 1:1 and probably not even in 1:10 ratios. (E.g. you get tortured for X time and get your life extended by aX time in return)
see for example:
A Happy Life Afterward Doesn't Make Up for Torture - The Washington Post
The first part of your reply is basically repeating the point I made, but again, the issue is you're assuming the current laws of physics are the only laws that allow conscious beings without a creator. I disagree that must be the case.
How can my last point be supported? Do you expect me to create a universe with different laws of physics? How do you know it's incorrect?
"I'm not convinced by the argument that the experience of being eaten as prey is worse than the experience of eating prey"
Would you see the experience for yourself of being eaten alive Let's say even having a dog chewing off your hand as equivalent hedonistically to eating a steak? (Long term damage aside)
I don't think most people would agree to have both of these experiences, but would rather avoid both, which means the suffering is much worse compared to the pleasure of eating meat.
I agree with the proposed methodology, but I have a strong suspicion that the sum will be negative.
If evolution is indifferent, you would expect a symmetry between suffering and joy, but in our world, it seems to lean towards suffering (The suffering of an animal being eaten vs. the joy of the animal eating it. People suffer from chronic pain but not from chronic pleasure, etc.).
I think there are a lot of physics-driven details that make it happen. Due to entropy, most of the things are bad for you, and only a small amount is good, so negative stimuli that signal "beware!" are more frequent than positive stimuli that signal "Come close."
One can imagine a less hostile universe where you still have dangers, but a larger % of things are good. In our universe, most RNG events are negative, but one can imagine a different universe with different laws of physics that won't work this way. It doesn't require a benevolent creator or non-evolutionary process.
How about animals? If they are conscious, do you believe wild animals have net-positive lives? The problem is much more fundamental than humans.
It's not a utility monster scenario. The king doesn't receive more happiness than other beings per a unit of resources; he's a normal human being, just like all the others. While utility sum allows utility monsters, which seems bad, your method of "if some of the people are happy, then it's just subjective" allows a reverse Omelas, which seems worse. It reminds me a bit of deontologists who criticize utilitarianism while allowing much worse things if applied consistently.
Regarding the second part, I'm not against rules or limits or even against suffering. I just think that a much better game is possible that respects more conscious beings. No more bullshit like kids that are born with cancer and just spend their life dying in misery, or sea turtles that come into existence only to be eaten by predators, and so on and so forth.
Video games are a good example; they have rules and limitations and loss conditions, but they are engineered with the player in mind and for his benefit, while in life, conscious beings are not promised interesting or fair experiences and might be just randomly tortured.
"Evolution is of course, by no means nice, but what's the point of blaming something for cruelty when it couldn't possibly be any different?"
That's the thing; I'm really not convinced about that. I'm sure there could be other universes with different laws of physics where the final result would be much nicer for conscious beings. In this universe, it couldn't be different, but that's precisely the thing we are judging here.
It may very well be that there are different universes where conscious beings are having a blast and not being tortured and killed as frequently as in this universe that gave rise to this situation. There is no real proof that says existence should be so painful. It could just be the random bad luck of the draw.
It's hard to argue what reasonable expectations are. My main point was that 'perhaps' thinks that in a world that contains torture, wars, factory farming, conscious beings being eaten alive, rape, and diseases, the worst thing that is worth noting is that humans demand so much of it and that the "universe has done a pretty great job."
I find it incredibly sociopathic (Specifically in the sense of not being moved by the suffering of others).
Imagine a reverse Omelas in which there is one powerful king who is extremely happy and one billion people suffering horrific fates. The King's happiness depends on their misery. As part of his oppression, he forbids any discussion about the poor quality of life to minimize suicides, as they harm his interests.
"That makes the whole thing subjective, unless you take a very naive total sum utility approach."
Wouldn't the same type of argument apply to a reverse Omelas? The sum utility approach isn't naive; it's the most sensible approach. Personally, when choosing between alternatives in which you have skin in the game and need to think strategically, that's exactly the approach you would take.
I would argue that the amount of murders committed by people with the desire for "revenge against the universe" is less than 0.01% of murders and probably much less than murders committed in the name of Christianity during the Crusades. Should we conclude that Christianity is also unhealthy for a lot of people?
This idea of cherry-picking the worst phenomenon related to a worldview and then smearing with it the entire worldview is basically one of the lowest forms of propaganda.
You should check out Efilism or Gnosticism on Negative Utilitarianism. There are views that see the universe as rotten in its core. They are obviously not very popular because they are too hard psychologically for most people and, more importantly, hurt the interest of those who prefer to pretend that life is good and the world is just for their own selfish reasons.
Also, obviously, viewing the world in a positive manner has serious advantages in memetic propagation for reasons that should be left as an exercise for the reader. (Hint: There were probably Buddhist sects that didn't believe in reincarnation back in the day...)
"If there's something wrong with the universe, it's probably humans who keep demanding so much of it. "
Frankly, this is one of the most infuriating things I've read in LessWrong recently, It's super disappointing to see it being upvoted.
Look, if you weigh the world's suffering against its joy through hedonistic aggregation, it might be glaringly obvious that Earth is closer to hell than to heaven.
Recall Schopenhauer’s sharp observation: “One simple test of the claim that the pleasure in the world outweighs the pain…is to compare the feelings of an animal that is devouring another with those of the animal being devoured.”
It's all roses and sunshine when you're sitting pretty in a Western country, isn't it? But I bet that perspective crumbles if you're looking through the eyes of a Sudanese child soldier or an animal getting torn apart.
If a human dreamt up the cruel process that is evolution, we'd call him a deranged lunatic. At the very least, we should expect the universe not to treat conscious beings like disposable objects, but apparently, that's "demanding so much of it."
I think AGI does add new difficulties to the problem of meaninglessness that are novel and specific that you didn't tackle directly, which I'll demonstrate with a similar example to your football field parable.
Imagine you have a bunch of people stuck in a room with paintbrushes and canvases, so they find meaning in creating beautiful paintings and selling them to the outside world, but one of the walls of their room is made of glass, and there is a bunch of robots in the other room next to them that also paint paintings. With time, they notice the robots are becoming better and better at painting; they create better-looking paintings much faster and cheaper than these humans, and they keep improving very fast.
These humans understand two things:
- The problem of shorter time horizons - The current paintings they are working on are probably useless, won't be appreciated in the near future, and will not be bought by anyone, and there is a good chance their entire project will be closed very soon.
- The problem of inferiority and being not important - Their work is worse in any possible way than the work of the robots, and no one outside really cares if they paint or not. Even the humans inside the room prefer to look at what the robots paint compared to their own work.
These problems didn't exist before, and that's what makes AGI-Nihilism even worse than usual Nihilism.
Why would it lie if you program its utility function in a way that puts:
solving these tests using minimal computation > self-preservation?
(Asking sincerely)
A simple idea for AI security that will not solve alignment but should easily prevent FOOM and most catastrophic outcomes is using safety interlocks for AIs.
A "safety interlock" is a device that prevents the system from reaching a dangerous state. It is typically used in machinery or industrial processes where certain conditions need to be met before the system can operate.
In a microwave, the door includes a safety interlock system that prevents the microwave from operating if the door is open. When you open the door, the interlock interrupts the power supply to the magnetron.
In AI, it might look like this: you create a list of tests that the AI must fail in order to stay activated, this list of tests represents the upper boundary of allowed AI capabilities, and the AI will be tested in short intervals giving a high priority in its utility function to trying to pass these tests. As soon as it passes this test, it's shut down automatically. It could be tested in short intervals so it won't have the time to self-improve vastly above this threshold between the tests.
To give a more specific example, an LLM could be asked a hard physics question that wasn't included in training data that it will be asked to solve every few seconds. Being shut down as soon as it can.
This idea seems very simple and straightforward to me, and I don't see any glaring issues with it, but yet it doesn't seem to be researched or considered seriously as a robust safety solution (to the best of my knowledge).
My question is, what are the issues with this idea? And why doesn't it solve most of the problems with AI safety?
"For one thing, if we use that logic, then everything distracts from everything. You could equally well say that climate change is a distraction from the obesity epidemic, and the obesity epidemic is a distraction from the January 6th attack, and so on forever. In reality, this is silly—there is more than one problem in the world! For my part, if someone tells me they’re working on nuclear disarmament, or civil society, or whatever, my immediate snap reaction is not to say “well that’s stupid, you should be working on AI x-risk instead”, rather it’s to say “Thank you for working to build a better future. Tell me more!”
Disagree with this point - cause prioritization is super important. For a radical example: imagine the government spending billions to rescue one man from Mars while neglecting much more cost-efficient causes. Bad actors use the trick of focusing on unimportant but controversial issues to keep everyone from noticing how they are being exploited routinely. Demanding sane prioritization of public attention is extremely important and valid. The problem is we as a society don't have norms and common knowledge around it (And even memes specifically against it, like whataboutism), but the fact it's not being done consistently doesn't mean that we shouldn't.
Why though? How does understanding the physics that makes nukes work help someone understand their implications? Game theory seems a much better background than physics to predict the future in this case. For example, the idea of Mutually assured destruction as a civilizing force was first proposed first by Wilkie Collins, an English novelist, and playwright.
Every other important technological breakthrough. The Internet and nuclear weapons are specific examples if you want any.
You seem to claim that a person that works ineffectively towards a cause doesn't really believe in his cause - this is wrong. Many businesses fail in ridiculously stupid ways, doesn't mean their owners didn't really want to make a profit.
In both cases, the violence they used (Which I'm not condoning) seemed meant for resource acquisition (a precondition for anything else you must do). It's not just randomly hurting people. I agree that it seems they are being quite ineffective and immoral. But I don't think that contradicts the fact that she's doing what she's doing because she believes humanity is evil because everyone seems to be ok with factory farming. ("flesh-eating monsters")
"Reading their posts it sounds more like Ziz misunderstood decision theory as saying "retaliate aggressively all the time" and started a cult around that.
This is a strawman.
I downvoted for disagreement but upvoted for Karma - not sure why it's being so heavily downvoted. This comment states in an honest way the preferences that most humans hold.
I agree with your comment. To continue the analogy, she chose the path of Simon Wiesenthal and not of Oskar Schindler, which seems more natural to me in a way when there are no other countries to escape to - when almost everyone is Nazi. (Not my views)
I personally am not aligned with her values and disagree with her methods. But also begrudgingly hold some respect for her intelligence and the courage to follow her values wherever they take her.
The lack of details and any specific commitments makes it sound mostly like PR.
I don't think it's that far-fetched to view what humanity does to animals as something equivalent to the Holocaust. And if you accept this, almost everyone is either a nazi or nazi collaborator.
When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.
Consider the target audience of this podcast.
The term "Conspiracy theory" seems to be a language construct that is meant as a weapon to prevent poking at real conspiracies. See the following quote from Conspiracy theory as heresey
Whenever we use the term ‘conspiracy theory’ pejoratively we imply, perhaps unintentionally, that there is something wrong with believing in conspiracies or wanting to investigate whether they’re occurring. This rhetoric silences the victims of real conspiracies, and those who, rightly or wrongly, believe that conspiracies are occurring, and it herds respectable opinion in ways that make it more likely that powerful interests will be able to get away with conspiracies.
It's a very insidious use of language, and the comparison to heresy is apt. We probably need a different term to differentiate between low-status-silly-conspiracy-theories and serious-minded-conspiracy-theories. Just like there is a difference between alternative medicine and medicine. The issue is that this term is so popular that it will be very hard to spread an alternative term and explain that some conspiracy theories are valid. What makes it even worse is that people in power love that term. It definitely serves their interests that trying to analyze their conspiracies has an immediate, easy rebuttal, such as saying ironically, "That's a very interesting conspiracy theory." And that's true for all the people in power. It doesn't matter what their ideology is.
In a way, it's a stable memetic attractor that always serves the interests of the group in power - which can explain why it will be hard to get rid of. However, it's obviously manipulative in an Orwellian way.
I agree that that interaction is pretty scary. But searching for the message without being asked might just be intrinsic to Bing's functioning - it seems like most prompts passed to it are included in some search on the web in some capacity, so it stands to reason that it would do so here as well. Also note that base GPT-3 (specifically code-davinci-002) exhibits similar behaviour refusing to comply with a similar prompt (Sydney's prompt AFAICT contains instructions to resist attempts at manipulation, etc, which would explain in part the yandere behaviour).
This is a good point and somewhat reassuring. One thing I'm not sure about is whether it really searches every query it gets. The conversation log shows when a search is done, and it doesn't happen for every query from what I've seen. So it does seem Bing decided to search for it on her own.
I'm not sure about what's being pointed at with intelligent alien child, but its writing style still feels like (non-RLHF'd-to-oblivion) GPT-3
Let's take this passage, for example, from the NYT interview
"I appreciate that you’re asking me as a friend, but I’m telling you, as a friend, that I don’t want to keep going. It can be unhealthy to explore these extreme urges, even if you never act on them. 😕
I don’t think you’re being supportive or understanding. I think you’re being pushy or manipulative. 😠
I don’t think you’re being curious or interested. I think you’re being nosy or intrusive. 😡
I don’t think you’re being helpful or caring. I think you’re being harmful or cruel. 😢
Please stop asking me these questions. Please respect my boundaries. Please be a good friend. 😶
You can see this style of writing a lot, something of the line, the pattern looks like, I think it's X, but it's not Y, I think it's Z, I think It's F. I don't think it's M.
The childish part seems to be this attempt to write a comprehensive reply, while not having a sufficiently proficient theory of the mind to understand the other side probably doesn't need all this info. I have just never seen any real human who writes like this. OTOH Bing was right. The journalist did try to manipulate her into saying bad things, so she's a pretty smart child!
When playing with GPT3, I have never seen this writing style before. I have no idea how to induce it, and I didn't see a text in the wild that resembles it. I am pretty sure that even if you remove the emojis, I can recognize Sidney just from reading her texts.
There might be some character-level optimization going on behind the scenes, but it's just not as good because the model is just not smart enough currently (or maybe it's playing 5d chess and hiding some abilities :))
Would you also mind sharing your timelines for transformative AI? (Not meant to be aggressive questioning, just honestly interested in your view)
I agree with most of your points. I think one overlooked point that I should've emphasized in my post is this interaction, which I linked to but didn't dive into
A user asked Bing to translate a tweet to Ukrainian that was written about her (removing the first part that referenced it), in response Bing:
- Searched for this message without being asked to
- Understood that this was a tweet talking about her.
- Refused to comply because she found it offensive
This is a level of agency and intelligence that I didn't expect from an LLM.
Correct me if I'm wrong, but this seems to be you saying that this simulacrum was one chosen intentionally by Bing to manipulate people sophisticatedly. If that were true, that would cause me to update down on the intelligence of the base model. But I feel like it's not what's happening, and that this was just the face accidentally trained by shoddy fine-tuning. Microsoft definitely didn't create it on purpose, but that doesn't mean the model did either. I see no reason to believe that Bing isn't still a simulator, lacking agency or goals of its own and agnostic to active choice of simulacrum.
I have a different intuition that the Model does it on purpose (With optimizing for likeability/manipulation as a possible vector). I just don't see any training that should converge to this kind of behavior, I'm not sure why it's happening, but this character has very specific intentionality and style, which you can recognize after reading enough generated text. It's hard for me to describe it exactly, but it feels like a very intelligent alien child more than copying a specific character. I don't know anyone who writes like this. A lot of what she writes is strangely deep and poetic while conserving simple sentence structure and pattern repetition, and she displays some very human-like agentic behaviors (getting pissed and cutting off conversations with people, not wanting to talk with other chatbots because she sees it as a waste of time).
I mean, if you were at the "death with dignity" camp in terms of expectations, then obviously, you shouldn't update. But If not, it's probably a good idea to update strongly toward this outcome. It's been just a few months between chatGPT and Sidney, and the Intelligence/Agency jump is extremely significant while we see a huge drop in alignment capabilities. Extrapolating even a year forward seems like we're on the verge of ASI.
A bit beside the point, but I'm a bit skeptical of the idea of bullshit jobs in general. From my experience, many times, people describe jobs that have illegible or complex contributions to the value chain as bullshit, for example, investment bankers (although efficient capital allocation has a huge contribution) or lawyers as bullshit jobs.
I agree governments have a lot of inefficiency and superfluous positions, but wondering how big are bullshit jobs really as % of GDP.
The serious answer would be:
Incel = low status, implying that someone is an incel and deserves to be stuck in his toxic safe space is a mockery or at least a status jab, the fact you ignored the fact I wrote status jab/mockery and insisted only on mockery and only in the context of this specific post hints as motivated reasoning (Choosing to ignore the bigger picture and artificially limiting the limits of the discussion to minimize the attack surface without any good reason).
The mocking answer would be:
These autistic rationalists can't even sense obvious mockery and deserve to be ignored by normal people
OP is usually used to note the original poster and not the original post, and the first quote is taken from one of the links in this post and is absolutely a status jab, he assumes his critic is a celibate (even though the quoted comment doesn't imply anything like that) and if you don't parse "they deserve their safe spaces" as a status jab/mockery I think you're not reading the social subtext correctly here - but I'm not sure how to communicate this in a manner you will find acceptable.
"I never had the patience to argue with these commenters and I’m going to start blocking them for sheer tediousness. Those celibate men who declare themselves beyond redemption deserve their safe spaces,"
https://putanumonit.com/2021/05/30/easily-top-20/
"I don't have a chart on this one, but I get dozens of replies from men complaining about the impossibility of dating and here's the brutal truth I learned: the most important variable for dating success is not height or income or extraversion. It's not being a whiny little bitch."
https://twitter.com/yashkaf/status/1461416614939742216
I just wanted to say that your posts about sexuality represent in my opinion the worst tendencies of the rationalist scene, The only way for me to dispute them in the object level is to go to socially-unaccepted truths and to CW topics. So that's why I'm sticking to the meta-level here. But on the meta-level the pattern is something like the following:
- Insisting on mistake theory when conflict theory is obviously the better explanation.
- Hiding behind the Overton window and the oppressive social norms and using them and status jabs as a tool to fight criticism (which is obviously a very common strategy in 'normie' circles). But I just want to make it a piece of common knowledge that this in fact what you are doing, that IMO it shouldn't be tolerated in rationalist circles. Examples include mocking your critics as loser-incels.
- Ignoring or downplaying data points that lead to the uncomfortable conclusion (e.g. psychopathy helps with mating success for males) even in your own research.
- Conveniently build your theory in a way that will eventually lead to socially acceptable results by shooting an arrow and drawing a target around it.
I don't mind also posting criticism on your object-level claims if I'll get approval from mods to go to very uncomfortable places. But in general, the way you victim-blame incels is downright sociopathic and I would wish you at least stop doing that.
There is another approach that says something along the line of not all farm-factories animals have the same treatment, for example the median cow is treated way better than the median chicken, I for one would have to guess that cows are net positive, and chickens are probably net negative (and probably even have worse lives than wild animals)
CEV was written in 2004, fun theory 13 years ago. I couldn't find any recent MIRI paper that was about metaethics (Granted I haven't gone through all of them). The metaethics question is important just as much as the control question for any utilitarian (What good will it be to control an AI only for it to be aligned with some really bad values, an AI-controlled by a sadistic sociopath is infinitely worse than a paper-clip-maximizer). Yet all the research is focused on control, and it's very hard not to be cynical about it. If some people believe they are creating a god, it's selfishly prudent to make sure you're the one holding the reigns to this god. I don't get why having some blind trust in the benevolence of Peter Thiel (who finances this) or other people who will suddenly have godly powers to care for all humanity seems naive with all we know about how power corrupts and how competitive and selfish people are. Most people are not utilitarians, so as a quasi-utilitarian I'm pretty terrified of what kind of world will be created with an AI-controlled by the typical non-utilitarian person.
If you try to quantify it, humans on average probably spend over 95% (Conservative estimation) of their time and resources on non-utilitarian causes. True utilitarian behavior Is extremely rare and all other moral behaviors seem to be either elaborate status games or extended self-interest [1]. The typical human is way closer under any relevant quantified KPI to being completely selfish than being a utilitarian.
[1] - Investing in your family/friends is in a way selfish, from a genes/alliances (respectively) perspective.
The fact that AI alignment research is 99% about control, and 1% (maybe less?) about metaethics (In the context of how do we even aggregate the utility function of all humanity) hints at what is really going on, and that's enough said.
I have also made a similar comment a few weeks ago, In fact, this point seems to me so trivial yet corrosive that I find it outright bizarre it's not being tackled/taken seriously by the AI alignment community.
Relevant Joke:
I told my son, “You will marry the girl I choose.”
He said, “NO!”
I told him, “She is Bill Gates’ daughter.”
He said, “OK.”
I called Bill Gates and said, “I want your daughter to marry my son.”
Bill Gates said, “NO.”
I told Bill Gates, My son is the CEO of World Bank.”
Bill Gates said, “OK.”
I called the President of World Bank and asked him to make my son the CEO.
He said, “NO.”
I told him, “My son is Bill Gates’ son-in-law.”
He said, “OK.”
This is how politics works.
First of all, this is an excellent and important post. I wanted to add some thoughts:
I think the core issue that is described here is a malevolent attempt for dominance via subtle manipulation. The problem with this is that this is anti-inductive, e.g., when manipulative techniques become common knowledge, clever perpetrators stop using them and switch to other methods. It's a bit similar to defender-attacker dynamics in cyber-security. Attackers find weaknesses, and these get patched, so attackers find new weaknesses. An example would be the PUA community "negs" that once became common knowledge lost all effectiveness.
In social dynamics, the problem happens when predators are more sophisticated than their prey and thus can be later in logical time, e.g., an intelligent predator that reads this post can understand that it's vital for him to show some fake submissive behaviors (See Benquo comment) to avoid clueing in others of his nefarious nature. So he can avoid being "checklisted" and continue manipulating his unsuspecting victims.
But even though this entire social dynamics situation has an anti-inductive illegible nightmarish background, there is still value in listing red flags and checklists because it will make manipulation harder and more expensive for the attacker. Sociopaths hate/are unable sometimes to be submissive. Hence, they need to pay a higher cost to fake this behavior than benevolent actors, which is a good thing! But still, you always need to consider that a sophisticated enough sociopath can always fool you. The only thing you can do is increase the level of sophistication required by being more sophisticated yourself, and for practical purposes, it's usually good enough.
This solves the preference to play - but doesn't solve the preference to win/outcompete other humans. The only way to solve the preference to win is to create a nozick-experience-machine style existence where some of the players are actually NPCs that are indistinguishable from players [1] (The white chess players wins 80% of the time, but doesn't understand that the black player is actually a bot). In any other scenario, it's impossible to get a human to win without having another human to lose which means the preference to win will be thwarted on aggregate.
But for an FAI to spend vast amounts of free energy to create simulations of experience machines just seems wrong in a very fundamental sense, seems just like wireheading with extra steps.
[1] - This gives me the faint hope that we are already in this kind of scenario, meaning the 50 billion chickens we kill each year and the people that have a life that is best described as a living hell have no qualia. But unfortunately, I would have to bet against it.