Posts
Comments
In the medium-term reduced-scarcity future, the answer is: lock them into a VR/experience-machine pod.
edit: sorry, misspoke. In this future, humans are ALREADY mostly in these pods. Criminals or individuals who can't behave in a shared virtual space simply get firewalled into their own sandbox by the AI. Or those behaviors are shadowbanned - the perpetrator experiences them, the victim doesn't.
I nominate NYC, and I assert that LA is an inferior choice for this. Source: John Carpenter/Kurt Russel movies.
In a sufficiently wealthy society we would never kill anyone for their crimes.
In a sufficiently wealthy society, there're far fewer forgivable/tolerable crimes. I'm opposed to the death penalty in current US situation, mostly for knowledge and incentive reasons (too easy to abuse, too hard to be sure). All of the arguments shift in weight by a lot if the situation changes. If the equilibrium shifts significantly so that there are fewer economic reasons for crimes, and fewer economic reasons not to investigate very deeply, and fewer economic reasons not to have good advice and oversight, there may well be a place for it.
This was my thinking as well. On further reflection, and based on OP's response, I realize there IS a balance that's unclear. The list contains some false-positives. This is very likely just by the nature of things - some are trolls, some are pure fantasy, some will have moved on, and only a very few are real threats.
So the harm of making a public, anonymous, accusation and warning is definitely nonzero - it escalates tension for a situation that has passed. The harm of failing to do so in the real cases is also nonzero, but I expect many of the putative victims know they have a stalker or deranged enemy who'd wish them dead, and the information is "just" that this particular avenue has been explored.
That balance is difficult. I philosophically lean toward "open is better than secret, and neither is as good as organized curation and controlled disclosure". Since there's no clear interest by authorities, I'd publish. And probably I'd do so anonymously as I don't want the hassle of having potential murderers know about me.
Can you explore a bit more about why you can't ethically dump it on the internet? From my understanding, this is information you have not broken any laws to obtain, and have made no promises as to confidentiality.
If not true publication, what keeps you from sending it to prosecutors and police? They may or may not act, but that's true no matter who you give it to (and true NOW of you).
People who have a lot of political power or own a lot of capital, are unlikely to be adversely affected if (say) 90% of human labor becomes obsolete and replaced by AI.
That's certainly the hope of the powerful. It's unclear whether there is a tipping point where the 90% decide not to respect the on-paper ownership of capital.
so long as property rights are enforced, and humans retain a monopoly on decisionmaking/political power, such people are not-unlikely to benefit from the economic boost that such automation would bring.
Don't use passive voice for this. Who is enforcing which rights, and how well can they maintain the control? This is a HUGE variable that's hard to control in large-scale social changes.
Specifically, "So, the islanders split into two groups and went to war." is fiction - there's no evidence, and it doesn't seem particularly likely.
Well, there are possible outcomes that make resources per human literally infinite. They're not great either, by my preferences.
In less extreme cases, a lot depends on your definition of "poverty", and the weight you put on relative poverty vs absolute poverty. Already in most parts of the world the literal starvation rate is extremely low. It can get lower, and probably will in a "useful AI" or "aligned AGI" world. A lot of capabilities and technologies have already moved from "wealthy only" to "almost everyone, including technically impoverished people", and this can easily continue.
- There's a wide range of techniques and behaviors that can be called "hypnosis", and an even wider range of what can be called "a real thing, right?". Things in the realm of hypnosis (meditation, guided-meditation, self-hypnosis, daily affirmations, etc. have plenty of anecdotal support from adherents, and not a lot of RCTs or formal proof of who it will work for and who it won't.
- There's a TON of self-help and descriptive writing on the topics of meditation and self-hypnosis. For many people, daily affirmations seem to be somewhat effective in changing their attitude over time. For many, a therapist or guide may be helpful in setting up and framing the hypnosis.
What does "unsafe" mean for this prediction/wager? I don't expect the murder rate to go up very much, nor life expectancy to reverse it's upward trend. "Erosion of rights" is pretty general and needs more specifics to have any idea what changes are relevant.
I think things will get a little tougher and less pleasant for some minorities, both cultural and skin-color. There will be a return of some amount of discrimination and persecution. Probably not as harsh as it was in the 70s-90s, certainly not as bad as earlier than that, but worse than the last decade. It'll probably FEEL terrible, because it was on such a good trend recently, and the reversal (temporary and shallow, I hope) will dash hopes of the direction being strictly monotonic.
This seems like a story that's unsupported by any evidence, and no better than fiction.
They could have fought over resources in a scramble of each against all, but anarchy isn't stable.
This seems most likely, and "stable" isn't a filter in this situation - 1/3 of the population will die, nothing is stable. It wouldn't really be "each against all", but "small (usually family) coalitions against some of the other small-ish coalitions". The optimal size of coalition will be dependend on a lot of factors, including ease of defection and strength of non-economic bonds between members.
- If you could greatly help her at small cost, you should do so.
This needs to be quantified to determine whether or not I agree. In most cases I imagine (and a few I've experienced), I would (and did) kill the animal to end it's suffering and to prevent harm to others if the animal might be subject to death throes or other violent reactions to their fear and pain.
In other cases I imagine, I'd walk away or drive on, without a second thought. Neither the benefit nor the costs are simple, linear, measurable things.
- Her suffering is bad.
I don't have an operational definition of "bad". I prefer less suffering, all else equal. All else is never equal - I don't know what alternatives and what suffering (or reduced joy) any given remediation would require, and only really try to estimate them when faced with a specific case.
For the aggregate case, I don't buy into a simple or linear aggregation of suffering (or of joy or of net value of distinct parts of the universe). I care about myself perhaps two dozen orders of magnitude more than the ant I killed in my kitchen this morning. And I care about a lot of things with a non-additive function - somewhere in the realm of logarithmic. I care about the quarter-million remaining gorillas, but I care about a marginal gorilla much less than 1/250K of that caring.
One challenge I'd have for you / others who feel similar to you, is to try to get more concrete on measures like this, and then to show that they have been declining.
I've given some thought to this over the last few decades, and have yet to find ANY satisfying measures, let alone a good set. I reject the trap of "if it's not objective and quantitative, it's not important" - that's one of the underlying attitudes causing the decline.
I definitely acknowledge that my memory of the last quarter of the previous century is fuzzy and selective, and beyond that is secondhand and not-well-supported. But I also don't deny my own experience that the (tiny subset of humanity) people I am aware of as individuals have gotten much less hopeful and agentic over time. This may well be for reasons of media attention, but that doesn't make it not real.
Do you think that the world is getting worse each year?
Good clarification question! My answer probably isn’t satisfying, though. “It’s complicated” (meaning: multidimensional and not ordinally comparable).
On a lot of metrics, it’s better by far, for most of the distribution. On harder-to-operationally-define dimensions (sense of hope and agency for the 25th through 75th percentile of culturally normal people), it’s quite a bit worse.
would consider the end of any story a loss.
Unfortunately, now you have to solve the fractal-story problem. Is the universe one story, or does each galaxy have it's own? Each planet? Continent? Human? Subpersonal individual goals/plotlines? Each cell?
I feel like you're talking in highly absolutist terms here.
You're correct, and I apologize for that. There are plenty of potential good outcomes where individual autonomy reverses the trend of the last ~70 years. Or where the systemic takeover plateaus at the current level, and the main change is more wealth and options for individuals. Or where AI does in fact enable many/most individual humans to make meaningful decisions and contributions where they don't today.
I mostly want to point out that many disempowerment/dystopia failure scenarios don't require a step-change from AI, just an acceleration of current trends.
Presumably, if The Observer has a truly wide/long view, then destruction of the Solar System, or certainly loss of all CHON-based lifeforms on earth, wouldn't be a problem - there have got to be many other macroscopic lifeforms out there, even if The Great Filter turns out to be "nothing survives the Information Age, so nobody ever detects another lifeform".
Also, you're describing an Actor, not just an Observer. If has the ability to intervene, even if it rarely chooses to do so, that's it's salient feature.
This seems like it would require either very dumb humans, or a straightforward alignment mistake risk failure, to mess up.
I think "very dumb humans" is what we have to work with. Remember, it only requires a small number of imperfectly aligned humans to ignore the warnings (or, indeed, to welcome the world the warnings describe).
a lot of people have strong low-level assumptions here that a world with lots of strong AIs must go haywire.
For myself, it seems clear that the world has ALREADY gone haywire. Individual humans have lost control of most of our lives - we interact with policies, faceless (or friendly but volition-free) workers following procedure, automated systems, etc. These systems are human-implemented, but in most cases too complex to be called human-controlled. Moloch won.
Big corporations are a form of inhuman intelligence, and their software and operations have eaten the world. AI pushes this well past a tipping point. It's probably already irreversable without a major civilizational collapse, but it can still get ... more so.
in worlds where AI systems have strong epistemics without critical large gaps, and can generally be controlled / aligned, things will be fine.
I don't have good working definitions of "controlled/aligned" that would make this true. I don't see any large-scale institutions or groups large and sane enough to have a reasonable CEV, so I don't know what an AI could align with or be controlled by.
In non-trivial settings, (some but not all) structural differences between programs lead to differences in input/output behaviour, even if there is a large domain for which they are behaviourally equivalent.
I think this is a crux (of why we're talking past each other; I don't actually know if we have a substantive disagreement). The post was about detecting "smaller than a lookup table would support" implementations, which implied that the input/output functionally-identical-as-tested were actually tested in the broadest possible domain. I fully agree that "tested" and "potential" input/output pairs are not the same sets, but I assert that, in a black-box situation, it CAN be tested in a very broad set of inputs, so the distinction usually won't matter. That said, nobody has built a pure lookup table anywhere near as complete as it would take to matter (unless the universe or my experience is simulated that way, but I'll never know).
My narrower but stronger point is that "lookup table vs algorithm" is almost never as important as "what specific algorithm" for any question we want to predict about the black box. Oh, and almost all real-world programs are a mix of algorithm and lookup.
might be true if you just care about input and output behaviour
Yes, that is the assumption for "some computable function" or "black box which takes in strings and spits out other strings."
I'm not sure your example (of an AI with a much wider range of possible input/output pairs than the lookup table) fits this underlying distinction. If the input/output sets are truly identical (or even identical for all tests you can think of), then we're back to the "why do we care" question.
i don't exactly disagree with the methodology, but I don't find the "why do we care" very compelling. For most practical purposes, "calculating a function" is only and exactly a very good compression algorithm for the lookup table.
Unless we care about side-effects like heat dissipation or imputed qualia, but those seem like you need to distinguish among different algorithms more than just "lookup table or no".
(I’m using time-sensitive words, even though we are stepping out of the spacetime of our universe for parts of this discussion.)
Maybe use different words, so as not to imply that there is a temporal, causal, or spacial relation.
Many people realize that, conceptually “below” or “before” any “base universe,” there is
I don't realize or accept that. Anything that would be in those categories are inaccessible to our universe, and not knowable or reachable from within. They are literally imaginary.
"all" humans?
The vast majority of actual humans are already dead. The overwhelming majority of currently-living humans should expect 95%+ chance they'll die in under a century.
If immortality is solved, it will only apply to "that distorted thing those humans turn into". Note that this is something the stereotypical Victorian would understand completely - there may be biological similarities with today's humans, but they're culturally a different species.
When humans fall well below marginal utility compared to AIs, will their priorities matter to a system that has made them essentially obsolete?
The point behind my question is "we don't know. If we reason analogously to human institutions (which are made of humans, but not really made or controlled BY individual humans), we have examples in both directions. AIs have less biological drive to care about humans than humans do, but also have more training on human writings and thinking than any individual human does.
My suspicion is that it won't take long (in historical time measure; perhaps only a few decades, but more likely centuries) for a fully-disempowered species to become mostly irrelevant. Humans will be pets, perhaps, or parasites (allowed to live because it's easier than exterminating them). Of course, there are plenty of believable paths that are NOT "computational intelligence eclipses biology in all aspects" - it may hit a wall, it may never develop intent/desire, it may find a way to integrate with biologicals rather than remaining separate, etc. Oh, and it may be fragile enough that it dies out along with humans.
Do we have a good story about why this hasn't already happened to humans? Systems don't actually care about the individuals they comprise, and certainly don't care about the individuals that are neither taxpayers, selectorate, contributors, or customers.
Why do modern economies support so many non-participants? Let alone the marginal and slightly sub-marginal workers, which don't cost much and may have option value or be useful to keep money moving in some way, there are a lot who are clearly a drain on resources.
There are a lot of good reasons to believe that stated human preferences correspond to real human preferences.
Can you name a few? I know of one: I assume that there's some similarity with me in because of similar organic structures doing the preferring. That IS a good reason, but it's not universally compelling or unassailable.
Actually, can you define 'real preferences' in some way that could be falsifiable for humans and observable for AIs?
"Surely the AIs can be trained to say "I want hugs" or "I don't want hugs," just as easily, no?"
Just as easily as humans, I'm sure.
"Safety bar" and "most safety possible" both assume that safety is measurable. Is it?
No real knowledge of the factors involved, other than that it's been tried before, somewhat recently. I wouldn't bet either side of those markets, which means they're reasonable to me.
Note that "plainly illegal means" is poorly-defined in this context. Laws change and get interpreted differently in diferent contexts. Things I'd call shenanigans, the Supreme Court may not, especially if one or two more justices get replaced. There will almost certainly be icky behavior that probably would have been illegal in other decades. There will almost certainly NOT be any convictions of Trump (or the eventual winner).
Edit to add: This is not just Trump. He's old and he knows it. It's the change of societal expectations, and the political technology of control.
We haven't figured it out for humans, and only VERY recently in history has the idea become common that people not kin to you deserve empathy and care. Even so, it's based on vibes and consensus, not metrics or proof. I expect it'll take less than a few decades to start recognizing some person-hood for some AIs.
It'll be interesting to see if the reverse occurs: the AIs that end up making decisions about humans could have some amount of empathy for us, or they may just not care.
Core Framework:
- Endurists: Death as the ultimate evil (life at all costs).
- Serenists: Suffering as the ultimate evil (non-existence over agony).
I don't think I fit into this framework, and I reject the "ultimate evil" concept. Death is currently and generally disprefered. Suffering sucks as well and I'd like to minimize it. The question of which is better/worse depends on the distribution of possible futures, and there are (probably) cases I'd choose death, and (probably) cases I'd choose suffering. Especially in situations where suffering has option value - I can choose death later, when I have more information about the distribution of possible futures, or more information about my own mind and it's reaction to suffering.
I also acknowledge that it's very hard to measure suffering (or joy) in a rigorous comparable way - this means I have very little standing to judge someone on their choice for themselves. And I tend to be a bit assymetric in my advice and preferences-for-others: I recommend endurism to people who aren't fully certain over a fairly long period of time (say, 2 human lifetimes...), and I'm not sure I'd choose it for some levels or kinds of suffering.
I have strong doubts as to the effectiveness and secrecy of a single-headed conspiracy. My version of this is "Most popular, open, discussion or news sources are heavily biased and many participants (including reporters with a byline or chat hosts) have a POV or purpose different than the naive cooperative view."
Whether it's a bot, a sock-puppet (alt account), a thrall (idiot parroting some demagogue), or just a low-value uninformed "real" participant, 90% of everything is crap, and has been since before Ted Sturgeon identified the phenomenon in 1956.
Which part of the question are you asking about? Is it OK to take a 50/50 shot at killing a terrorist or a kid? No, I don't think so. Is it really 50/50? Depends on circumstances, but usually no - there will be known factors that weigh one way or the other, and the most likely outcome is a clean miss, which warns the terrorists that snipers are present.
The narrow topic of the principle of indifference is ... fine, I guess. But it's a pretty minor element of the decision, and in many cases, it's better to collect more data or change the situation than to fall back on a base distribution of unknowns.
On further reflection, I realize I'm assuming a fair bit of hyperbole in the setup. I just don't believe there's more than an infinitesimal chance of actual perpetual torture, and my mind substitutes dust motes in one's eye.
I don't think any amount of discussion is likely to get my mind into a state that takes it seriously enough to actually engage on that level, so I'm bowing out. Thanks for the discussion!
Would you take the gamble, or would you choose non-existence?
Non-existence is a gamble too. You could lose out on billions of years of happiness! Even without that opportunity cost, I assert that most humans' ability to integrate over large timespans is missing, and you're going to get answers that are hard to reconcile with any sane definitions and preferences that don't ALREADY lead most people to suicide.
For me, sign me up for immortality if it's not pretty certain to be torture.
Yes, I mean the current deployed level. News hasn't really covered anything major in the last few years on the topic, and I don't know if it's stagnated or the reporting has just given up.
I think both, by a long shot. I estimate I spend over half my time outside of easy video surveillance (room without a webcam or phone pointed in a useful direction, or outdoors not in easy LOS of a traffic or business cam), and a slightly different half for audio. For neither of these is high-fidelity POV data available at all, as described in the post.
For those times when I AM under surveillance, the quality is low and the integration is minimal. There are legal and technical challenges for anyone to use it against me. And it's VERY easy to find times/place where I'm not being recorded when I choose to.
I avoid terms (and concepts) like "inevitable". There are LOTS of unknowns, and many future paths that go through this, or not. Scenario-based conditionality (what would lead to this, and what would lead elsewhere) seems more fruitful.
Perfect surveillance is the default for electronic intelligence - logging is pretty universal. I think this is likely to continue in future universes where most people are electronic.
I think the answer is "Mu" for universes with no people in them.
I think the likely path is "never perfect, but mostly increasing over time" for universes with no singularity or collapse.
I'd love to more about implications of the CURRENT level of observation. Things that are just now feasible, and the things that are promoting or holding them back. For instance, individual facial recognition got a wave of reporting a few years ago, and I honestly don't know if it just quietly became universal or if the handwringing and protests actually worked to keep it only in very controlled and visible places (like border crossings).
This thought experiment can help us to find situations in nature when similar things have already happened.
It can? Depending on what you mean by "similar", either we can find them without this thought experiment or they don't exist and this doesn't help. Your example is absolutely not similar in the key area of individual continuity.
Have you tried this on humans? humans with dementia? very young humans? Humans who speak a different language? How about Chimpanzees or whales? I'd love to see the truth table of different entities you've tested, how they did on the test, and how you perceive their consciousness.
I think we need to start out by biting some pretty painful bullets:
1) "consciousness" is not operationally defined, and has different meanings in different contexts. Sometimes intentionally, as a bait-and-switch to make moral or empathy arguments rather than more objective uses.
2) For any given usage of a concept of consciousness, there are going to be variations among humans. If we don't acknowledge or believe there ARE differences among humans, then it's either not a real measure or it's so coarse that a lot of non-humans already qualify.
3) if you DO come up with a test of some conceptions of consciousness, it won't change anyone's attitudes, because that's not the dimension they say they care about.
3b) Most people don't actually care about any metric or concrete demonstration of consciousness, they care about their intuition-level empathy, which is far more based on "like me in identifiable ways" than on anything that can be measured and contradicted.
I was excited but skeptical, to hear that you could empirically test anything on the topic. And disappointed but unsurprised that you don't. There's nothing empirical about this - there is zero data you're collecting, measuring, or observing.
It will be an empirical test when you ACTUALLY have and use this teleporter.
Until then, you're just finding new ways to show that our intuitions are not consistent on esoteric and currently-impossible (and therefore irrelevant) topics.
"Them" is fine. It's been used for both plural and indefinite-gender for decades, and VERY commonly for at least 20 years. Artificial and small-scale usage changes, especially if not overwhelmingly better, just cause confusion and annoyance.
Useful write-up. I think it's missing a very important point, which is that "responsibilty" has multiple different uses and meanings, and this ambiguity is sometimes intentional. Most of these are somewhat correlated, but not enough to mix them up safely.
1) Legal responsibility. Who can be compelled to change, or be punished (or who deserves rewards, for positive outcomes).
2) Causal decision responsibility. Whether one made choices that resulted in some consequence.
3) Experiential responsibility. Whether one experiences the situation directly, or only indirectly.
4) Intent responsibility. Whether one believes they have significant influence over the thing.
5) Moral responsibility (a). Whether one is pressured (by self or socially) to do something in the future.
6) Moral responsibility (b). Whether one is blamed (by self or socially) for something in the past.
For me it sounds like you did not mention the whole AI alignment question
True. The question didn't specify anything about it, so I tried to answer based on default assumptions.
You need to be clear who is included in "us". AI is likely to be trained on human understanding of identity and death, which is very much based on generational replacement rather than continuity over centuries. Some humans wish this wasn't so, and hope it won't apply to them, but there's not enough examples (none in truth, few and unrealistic in fiction) to train on or learn from.
It seems likely that if "happy people" ends up in the AI goalset, it'll create new ones that have higher likelihood of being happy than those in the past. Honestly, I'm going to be dead, so my preference doesn't carry much weight, but I think I prefer to imagine tiling the universe with orgasmium more than I do paperclips.
It's FAR more effort to make an existing damaged human (as all are in 2025) happy than just to make a new happy human.
I'm confused. You start with
“But you can’t have a story where everyone is happy and everything is perfect! Stories need conflict!”
And then list a bunch of conflicts. David Mamet's three rules are:
- Who wants what from whom
- What happens if they don’t get it
- Why now
And all of your examples are fair places to answer these. I think we don't actually disagree, but I'm not sure I understand the objection you're responding to. Did someone actually say "there's no tension without dystopia"? If they only said "dystopia is the lazy person's generator of fictional tension", then I kind of agree.
The main good bit of market pricing this would miss is the demand reduction and reallocation caused by the higher prices
True. The main thing the "tax a price increase" misses is that it mutes the supply incentive effects of the price increase. I'd need to understand the elasticities of the two (including the pre-supply incentives for some goods: a decision to store more than current demand BEFORE the emergency gets paid DURING) to really make a recommendation, and it'd likely be specific enough to time and place and product and reason for emergency that "don't get involved at a one-size-fits-all level" is the only thing I really support.
This is a novel (to me) line of thinking, and I'm happy to hear about it! I'm not sure it's feasible, as one of the things the public hates more than price increases during a shortage is higher taxes any time.
That said, the REVERSE of this - slightly raise taxes in normal times, and make emergencies a tax holiday, might really work. This gives room for producers/distributors to raise prices WITHOUT as much impact on the consumers. Gets some of the good bits of market pricing, with less of the bad bits (both limited to the magnitude of the tax change relative to the scarcity-based price change).
I don't think your bet has very much relationship to your post title or the first 2/3 of the body. The metaculus question resolved based on "according to widespread media and historical consensus." which does not require any automation or feasible automation of tasks or occupations, let alone all of them.
It's not clear that "a human which doesn't care about perceived status" is actually human. A lot depends on whether you consider the AIs that populate the solar system after biological intelligence is obsolete to be "descendants" or "replacements" of today's humans.