Group rationality -- bridging the gap in a post-truth world 2016-11-18T13:44:40.118Z


Comment by rosyatrandom on How can there be a godless moral world ? · 2021-06-23T10:54:06.078Z · LW · GW

Before I say anything else, a couple of quotes from Pratchett's Night Watch:

'You haven't killed your wife,' he said. 'Anywhere. There is nowhere, however huge the multiverse is, where Sam Vimes as he is now has murdered Lady Sybil. But the theory is quite clear. It says that if anything could happen without breaking any physical laws, it must happen. But it hasn't.[...]'


"He wanted to go home. He wanted it so much that he trembled at the thought. But if the price of that was selling good men to the night, if the price was filling those graves, if the price was not fighting with every trick he knew … then it was too high.

It wasn’t a decision he was making, he knew that. It happened far below the levels of the brain where decisions were made. It was something built in. There was no universe, anywhere, where a Sam Vimes would give in on this, because if he did then he wouldn’t be Sam Vimes, anymore."

And that, basically, is it. We are who we are, and our minds, our personalities, our very natures, our sense of morality and aesthetics... you cannot simply swap them out without changing who we are. 

Being human, and being you-in-particular, entails certain ways of looking at things; morality is part of that. Is morality part of reality, then? It's part of you, and you are part of reality.

Comment by rosyatrandom on Biotech to make humans afraid of AI · 2021-06-18T10:57:12.073Z · LW · GW

Can we develop a drug that makes people afraid of people who suggest making drugs to make people afraid of something?

Comment by rosyatrandom on Can someone help me understand the arrow of time? · 2021-06-16T09:04:28.659Z · LW · GW

I'd say that the normal temporal dimension we impose on reality is related to, but not the same, as the kind of time that underpins our consciousness.

As you say, memory is a process, not a static snapshot; the act of being sentient cannot be usefully be broken down into a sequence of mind-states based at instances on the timeline.

But perhaps there can be something more like a dynamic snapshot; atomic slices of consciousness that span over normal time, and represent a combined state/process from which 'this moment, this thought, this feeling' can be abstracted. 

There's lots of ways to twist the kaleidoscope and interpret the underling structure, and they're all (of course!) related to each other

Comment by rosyatrandom on Why are young, healthy people eager to take the Covid-19 vaccine? · 2020-11-30T21:24:18.976Z · LW · GW

That wasn't an attack. It was a judgement.

Comment by rosyatrandom on Why are young, healthy people eager to take the Covid-19 vaccine? · 2020-11-23T11:56:31.456Z · LW · GW

they value safety of strangers higher than their own safety, and want to take the vaccine for the sake of all the people at risk in the society.


Quite apart from the actually low personal risk from taking a vaccine, why does this strike you as odd? This is perfectly normal and good human behaviour, and if you don't share it there is probably something quite a bit wrong with you.

Comment by rosyatrandom on Rationalists from the UK -- what are your thoughts on Dominic Cummings? · 2020-11-21T23:01:21.124Z · LW · GW

He's a smug little Tory shit who fucked around and got found out. 

That's my thoughts on him.

Comment by rosyatrandom on The point of a memory palace · 2020-06-24T09:30:10.266Z · LW · GW

That's basically lucid daydreaming, then?

Trying to do that reminded me of something I used to do as a kid: I would watch static on TV, and find myself constructing imagery from it. Usually, it would be like traveling over landscapes, or a rotating/panning view over some entity, and the quality of the visuals would be like line drawings.

The reason I remember that is because my mental visualisations have a very similar quality. After maybe a 'flash' of a fairly detailed scene -- or at least the suggestion of one -- it rapidly devolves into short-lived abstractions, and only where I'm mentally focusing.

Perhaps what I need is to look at some static again and see if it improves visualisation.

And I tried it. Didn't help :-/

Comment by rosyatrandom on The point of a memory palace · 2020-06-22T08:20:28.567Z · LW · GW

I have never taken the idea of attempting a memory palace seriously, as although I have a terrible, terrible, memory, I also am terrible at visualising things.

To me, using a memory palace to solve memory issues is like making a speedboat out of coconut husks to escape a desert island. Or, perhaps, the xkcd regex '2 problems' comic.

Comment by rosyatrandom on Can we always assign, and make sense of, subjective probabilities? · 2020-01-17T11:25:37.361Z · LW · GW

As, basically, an atheist, my response to the question 'Is there an all-powerful god?' is to ask: is that question actually meaningful? Is it akin to asking, 'is there an invisible pink unicorn?', or 'have you stopped beating your wife yet?'. To whit, a mu situation .

There are a lot of different types of question, and probabilities don't seem to mean the same thing across them. Sometimes those questions are based on fuzzy semantics that require interpretation, and may not necessarily correspond to a possible state of affairs.

The possibility of a god existing doesn't equate, to me, to seeing if a possible thing exists or not, but rather whether the set of concepts are in any way possible. This is a question about the very nature of reality, and I'm pretty sure that reality is weird enough that the question falls far short of having any real meaning.

Comment by rosyatrandom on [deleted post] 2020-01-14T14:47:10.059Z

I'm glad someone else thinks so, too. I'd also go so far as to say that our notions of rationality are also largely aesthetic.

Comment by rosyatrandom on What is Success in an Immoral Maze? · 2020-01-10T14:16:34.768Z · LW · GW

Most of civilisation right now seems to be one giant gas-lighting immoral maze, where any effort to point out or mitigate the massive problems we have is sneered at or ignored.

Comment by rosyatrandom on Illness anxiety disorder: how to become more rational? · 2020-01-06T10:27:53.073Z · LW · GW
Yesterday, I managed to make an appointment for an ultrasound. However, I'm broke, and it turns out that the particular doctor is really expensive and has really bad reputation online

This is tangential, but part of the problem here is that your healthcare system is evil.

You should be able to see doctors for worries of this magnitude, and get the treatment/checks/referrals that you need, without this bullshit.

Comment by rosyatrandom on Simulation Argument: Why aren't ancestor simulations outnumbered by transhumans? · 2019-09-30T09:12:28.031Z · LW · GW

It's only a problem if you want it to be a problem.

There doesn't *need* to be anyone doing the interpreting, because all possible representations (and the interpreters/ees within) exist for free. I'm comfortable with that. There's no need to invoke special privilege to make reality more complicated, just because you want it to be. Fundamental reality *should* be simple, on some level, don't you think? The complexity is all internal.

Comment by rosyatrandom on Looking for answers about quantum immortality. · 2019-09-09T09:33:33.654Z · LW · GW

I used to be heavily into this area, and after succumbing somewhat to an 'it all adds up to normality' shoulder-shrugging, my feeling on this is that it's not just the 'environment' that is subject to radical changes, but the mind itself. It might be that there's a kind of mind-state attractor, by which minds tend to move along predictable paths and converge upon weirdness together. All of consciousness may, by different ways of looking at it, be considered as fragments of that endstate.

Comment by rosyatrandom on Peter Thiel/Eric Weinstein Transcript on Growth, Violence, and Stories · 2019-09-02T10:06:29.677Z · LW · GW
Even though they pin-point varies issues in society such as radical leftism

Oh, no, is LessWrong becoming one of those places?

Comment by rosyatrandom on Simulation Argument: Why aren't ancestor simulations outnumbered by transhumans? · 2019-08-29T09:57:05.757Z · LW · GW

I don't believe that that is a necessary assumption at all; the conscious state is still an abstractable representation, and if it maps to a dynamic process that itself can map to a temporally-connected collection of brain-states, then that is just more layers of abstraction.

The Boltzmann Brain could easily be not a brain-state representation, but a conscious-state representation.

Comment by rosyatrandom on Simulation Argument: Why aren't ancestor simulations outnumbered by transhumans? · 2019-08-23T13:37:19.237Z · LW · GW
In fact, why not discard physical reality entirely and rest in the thought of everything existing in abstract math space?

Well, yes, that's kind of the implication here. The minimum reality required to contain everything is, basically, nothing. Any more is entirely superfluous and reducible back to that bedrock.

Well, why not jump from a bridge for fun then? You will continue to exist no matter what you do.

You're talking about quantum immortality/suicide, and it's another corollary. Whether you find it ridiculous or not, I find the idea of an arbitrary 'physical' reality far more absurd.

Comment by rosyatrandom on Simulation Argument: Why aren't ancestor simulations outnumbered by transhumans? · 2019-08-23T09:08:06.492Z · LW · GW
Furthermore, if you're convinced by the simulation argument, why not believe that you're a Boltzmann brain instead using the same line of argument?

Why not both?

Confession: my entire metaphysical worldview has been strongly shaped by reading Greg Egan's Permutation City, so I kind of subscribe to something like the Dust Theory/Max Tegmark's Mathematical Multiverse.

To return to your question: if your mind can be construed as existing within many different contexts, be they simulations, Boltzmann Brains, or boring old meatsacks in cosmoses... does it make any sense to say 'I am in _this_ one'? You're in all of them, so long as those contexts can be said to 'exist'. And what is stopping them from 'existing'?

Comment by rosyatrandom on Quantum Mechanics, Nothing to do with Consciousness · 2018-11-28T03:15:54.696Z · LW · GW

If that sort of order is helpful to developing consciousness somewhere down the line, then that is the link

Comment by rosyatrandom on Boltzmann Brains, Simulations and self refuting hypothesis · 2018-11-28T03:14:12.273Z · LW · GW

Consistency seems to be the only real fallback

Comment by rosyatrandom on Boltzmann Brains, Simulations and self refuting hypothesis · 2018-11-27T06:39:30.331Z · LW · GW

Why is it assumed that we are only in *one* of these options? Does it not make no difference, to the point that you can say we exist in all of them to the extent that they are possible? That a BB may not coherently exist further down its own timestream doesn't matter at all, because temporal contiguity is not necessary.

Comment by rosyatrandom on Quantum Mechanics, Nothing to do with Consciousness · 2018-11-27T06:36:55.398Z · LW · GW

My quick take on it, via the Weak Anthropic Principle: consciousness is likely to be linked to QM, because we find ourselves in a QM-based world. If it's not *required*, odds are that QM-based realities are amenable
to containing conscious entities.

Comment by rosyatrandom on Help Me Refactor Myself I am Lost · 2018-11-09T04:23:33.994Z · LW · GW

> according to some personality tests I am an INTJ I don't know whether this is considered science but from what I've read about the personality type it's literally a copy paste of who am I so I believe in them .

I know this is tangential to your question, but that is _not_ a scientific/rational approach you are taking w.r.t. Myers-Briggs.

Comment by rosyatrandom on The Evil Genie Puzzle · 2018-07-25T09:24:31.742Z · LW · GW

Simple cheat solution:

"the Council of Genies has created new, updated rules which ban any unwanted side-effects for the person who makes the wish or any of their loved one's. "

I would argue that I love everyone, by default, especially people in this kind of cruel situation. Therefore this would count as an unwanted side-effect.

Comment by rosyatrandom on Attention! Financial scam targeting Less Wrong users · 2016-03-03T10:28:33.422Z · LW · GW

I'm not sure whether people thinking I was being serious means I was being too dry, or exactly dry enough :D

Comment by rosyatrandom on Attention! Financial scam targeting Less Wrong users · 2016-03-02T09:31:43.964Z · LW · GW

Surely, as rationalists, we should do a controlled test to determine if these are scams? This will require some blindly chosen users to respond in a variety of different ways, some of whom should go through with the possible scam, and report the results.

EDIT: I think it's time to come clean. No, I am not the scammer, but this post wasn't serious. I'm rather surprised anyone thought it could be, to be honest!

Comment by rosyatrandom on Hedonium's semantic problem · 2015-04-09T14:31:09.866Z · LW · GW

Even if the Boltzmann brain is completely chaotic, internally it contains the same structures/processes as whatever we find meaningful about Napoleon's brain. It is only by external context that we can claim that those things are now meaningful.

For us, that may be a valid distinction -- how can we talk to or interact with the brain? It's essentially in it's own world.

For the Boltzmann!Napoleon, the distinction isn't remotely meaningful. It's in it's own world, and it can't talk to us, interact with us, or know we are here.

Even if the internal processes of the brain are nothing more than randomised chance, it maps to 'real', causal processes in brains in 'valid' ontological contexts.

The question is -- do those contexts/brains exists, and is there any real distinction between the minds produced by Boltmann!Napoleon, Virtual!Napoleon, etc.? I would say yes, and no. Those contexts exist, and we are really discussing one mind that corresponds to all those processes .

As to why I would say that, it's essentially Greg Egan's Dust hypothesis/Max Tegmark's Mathematical Universe thing.

Comment by rosyatrandom on A potentially great improvement to minimum wage laws to handle both economic efficiency as well as poverty concerns · 2011-07-25T22:37:11.076Z · LW · GW

So... what we have here is a kind of minimum and maximum entitlement: x/2 and x, with incentives to work more to get more back. Interesting

Comment by rosyatrandom on Spooky Action at a Distance: The No-Communication Theorem · 2011-04-20T15:45:54.901Z · LW · GW

Another late response from me as I read through this series again:

"I realized why the area under a curve is the anti-derivative, realized how truly beautiful it was"

Would this be that the curve is the rate-of-change of the area (as the curve goes up, so does the area beneath it)?

Comment by rosyatrandom on Feynman Paths · 2011-04-16T18:45:39.707Z · LW · GW

Very late response:

I think that the splitting of the photon's path is pretty much entirely a human construction - the smaller the components it is split into, the more accurate the calculation, and each partition is itself an approximation that can be refined by splitting it up further in exactly the same manner. Essentially, it's a shortcut to doing a path integral over the entire range down to the planck level. Maybe... I'm not sure!

Comment by rosyatrandom on The Generalized Anti-Zombie Principle · 2011-04-15T16:07:52.196Z · LW · GW

Another comment to add a few years later than the original post and hence be pretty useless:

My thoughts are that consciousness (as in the experience of it) is a kind of epiphenomenon:

The sensation is derived from cognitive processes that map isomorphically to an abstract model of consciousness in mindspace (and I do not make any distinction or heirarchy between realspace and mindspace in terms of privileged levels of existence).

It does this because the brain is doing exactly what it feels like consciousness does - integrating various inputs into a representation of self and environment, making plans and telling a consistent story about it all. And the mapping, by being possible, is also real.

Comment by rosyatrandom on Manufacturing prejudice · 2011-04-04T08:21:08.193Z · LW · GW

And then there is Kurt Vonnegut's warning from Mother Night: "We are what we pretend to be, so we must be careful about what we pretend to be."

Comment by rosyatrandom on Theists are wrong; is theism? · 2011-01-20T10:03:43.919Z · LW · GW

Since my metaphysical position is (and I'm going to have to come up with a better term for it) pan-existence, having gods that create and influence things requires that those possibilities where they don't (or where other, similar-but-different gods do) are somehow rendered impossible or unlikely.

Gods being statistically significant requires some metaphysical reason for them to be so simply in order to stop the secular realities dominating, and the arbitrary focus of theistic gods on humanity and our loose morals only serves to make them ever more over-specified and unlikely.

Comment by rosyatrandom on Theists are wrong; is theism? · 2011-01-20T01:22:04.037Z · LW · GW

I'm in the 'everything that can exist does so; we're a fixed point in a cloud of possibilities' camp. I'm also an atheist because I see theism as an extra-ordinarily arbitrary and restrictive constraint on what should or must be true in order for us to exist.

It's simply too narrow and unjustified for me to take seriously, and the fact that its trappings are naive and full of wishful thinking and ulterior motives means I certainly don't.

Comment by rosyatrandom on A sense of logic · 2010-12-11T06:30:38.113Z · LW · GW

Does the phrase, "The stupid; it hurts" feel appropriate? It's as if to understand someone's line of thought you have to mutilate your own thought processes, and it's like hearing a truly terrible joke tenfold.

Comment by rosyatrandom on A sense of logic · 2010-12-10T19:12:44.160Z · LW · GW

It sounds like nausea; like you get when you're intoxicated and the room starts swimming--your faculties try to make sense of what's happening, and fail.

Comment by rosyatrandom on A note on the description complexity of physical theories · 2010-11-09T16:56:41.825Z · LW · GW

I think that while a sleek decoding algorithm and a massive look-up table might be mathematically equivalent, they differ markedly in what sort of process actually carries them out, at least from the POV of an observer on the same 'metaphysical level' as the process. In this case, the look-up table is essentially the program-that-lists-the-results, and the algorithm is the shortest description of how to get them. The equivalence is because, in some kind of sense, process and results imply each other. In my mind, this a bit like some kind of space-like-information and time-like-information equivalence, or as that between a hologram and the surface it's projected from.

In the end, how are we to ever prefer one kind of description over the other? I can only think that it either comes down to some arbitrary aesthetic appreciation of elegance, or maybe some kind of match between the form of description and how it fits in with our POV; our minds can be described in many ways, but only one corresponds directly with how we observe ourselves and reality, and we want any model to describe our minds with as minimal re-framing as possible.

Now, could someone please tell me if what I have just said makes any kind of sense?!

Comment by rosyatrandom on Swords and Armor: A Game Theory Thought Experiment · 2010-10-12T18:00:21.389Z · LW · GW

Yeah, it wasn't that difficult once I worked out how to set it up. I used the table below of Sword vs Armour damages with an index function based on the numbers in the row/column headings. Here's an example:


E2:H5 is the Swords vs Armour table.

O14 is the s1 part of the a4|s1 column label.

A22 is the a2 part of the a2|s4 row label.

Thus, this works out the mitigated attack value of Sword 1 vs Armour 2. This table as a whole worked out the mitigated attacks for columns versus rows. A second table worked out rows vs columns, and the table shown above merely compared the two values.

I agree though, the strategy is complex and I think perhaps in these situations always comes down to how risky/analytical you think the other players going to be, and how you think they think they're going to judge everyone else. And... well, how do you even start doing that? Especially since, most of the time, people will just... stop behaving rationally when faced with this kind of situation.

Comment by rosyatrandom on Swords and Armor: A Game Theory Thought Experiment · 2010-10-12T11:03:37.921Z · LW · GW

You mean this table? :)

(This and the one I made below can be seen properly at , along with the ATT vs DEF tables I worked out the outcomes from)

Hmm. Unless this has gone wrong, the best combo is Sword 1 and Armour 4, with Sword 1/Armour 1 being close). But if you bank on people choosing 1/4, then 1/1 will beat them.

NB: Yes, I made a lot of mistakes and edits to get here, and probably have still made some...

VS        a1         a1    a1    a1    a2    a2    a2    a2    a3    a3    a3    a3    a4    a4    a4    a4
        s1        s2    s3    s4    s1    s2    s3    s4    s1    s2    s3    s4    s1    s2    s3    s4
a1    s1    0.5        0    0    0    1    0    0    0    0    0    0    0    1    1    1    1
a1    s2    1        0.5    1    1    1    0.5    1    1    1    1    1    1    1    1    1    1
a1    s3    1        0    0.5    0    1    1    1    1    0    0    0    0    1    1    1    1
a1    s4    1        0    1    0.5    0    0    0    0    1    1    1    1    0    0    0    0
a2    s1    0        0    0    1    0.5    0    0    1    0    0    0    1    1    1    0    1
a2    s2    1        0.5    0    1    1    0.5    0    1    1    1    1    1    1    1    0    1
a2    s3    1        0    0    1    1    1    0.5    1    0    0    0    0    1    1    1    1
a2    s4    1        0    0    1    0    0    0    0.5    1    1    1    1    0    0    0    0
a3    s1    1        0    1    0    1    0    1    0    0.5    0    1    0    1    0    1    0
a3    s2    1        0    1    0    1    0    1    0    1    0.5    1    0    1    0    1    0
a3    s3    1        0    1    0    1    0    1    0    0    0    0.5    0    1    1    1    0
a3    s4    1        0    1    0    0    0    1    0    1    1    1    0.5    0    0    0    0
a4    s1    0        0    0    1    0    0    0    1    0    0    0    1    0.5    0.5    0    1
a4    s2    0        0    0    1    0    0    0    1    1    1    0    1    0.5    0.5    0    1
a4    s3    0        0    0    1    1    1    0    1    0    0    0    1    1    1    0.5    1
a4    s4    0        0    0    1    0    0    0    1    1    1    1    1    0    0    0    0.5
_    _    10.5    1    6.5    9.5    9.5    4    6.5    9.5    8.5    7.5    8.5    9.5    11    9    7.5    9.5
Comment by rosyatrandom on Swords and Armor: A Game Theory Thought Experiment · 2010-10-12T10:29:56.493Z · LW · GW

That's true; despite having the same damage per minute, the red swords stats are harmed more by armour damage reduction (since if x<y, (x-a)y < x(y-a)).

It should be noted that the Armour Damage stat only affects a Sword's Damage stat, while Dodge is global: Mitigated Damage per minute = (Sword Damage + Armour Damage) Speed (1-Dodge)

Comment by rosyatrandom on Swords and Armor: A Game Theory Thought Experiment · 2010-10-12T09:58:42.580Z · LW · GW
    D    S    D*S        a1        a2        a3        a4    
s1    100    80    8000    6336    6256    6400    6080    6268
s2    80    100    8000    6120    6120    6000    6080    6080
s3    150    50    7500    6210    6035    6500    5700    6111.25
s4    50    180    9000    6156    6426    5400    6840    6205.5
                    6205.5    6209.25    6075    6175    

                Da    -12        -8        -20        0    
                Do    0.1        0.15    0        0.24    

My first instinct was to make the table above, which may or may not be readable here (EDIT: mostly readable, some tab glitches). I first calculated each sword's damage per minute (obviously, in general you want the highest value here), and then worked it out as applied to each armour type.

Here's where it gets... tricky, as you want the sword that maximises damage to all armour types, and the armour that minimises damage from all sword types. Do we look at average values? That could leave you open to being gamed by someone whose choices are poor except against your specific choices.

The best average choice here, by the way, is clearly Sword 1 (Blue) and Armour 3 (Yellow). But Sword 1 is not the best choice against Armours 2 and 4, and Armour 3 is not the best choice against Swords 1 and 3.

Nevertheless, the difference is small enough that I would still go with them; in the end, I think, being optimal like that is still the best strategy, as players with worse choices will be outcompeted. I am niggled by the thought that the optimal armour is not actually optimal against the optimal sword, but I don't think you can really do anything about that

Comment by rosyatrandom on Ugh fields · 2010-10-04T00:17:25.295Z · LW · GW

As someone who just sent the link to my girlfriend, I agree!

Comment by rosyatrandom on Shock Level 5: Big Worlds and Modal Realism · 2010-05-26T13:36:08.754Z · LW · GW

I like the use of the quotient set here. In fact, I would go on to use it more comprehensively: not only does our observer-moment define an equivalence class, but any particular context implementing it does, too. It could be a simulation, or a simulation in a simulation in a (...), a small corner of a more general mathematical system, anything. The point is that for any and every defined part, it too will always be part of a quotient; there will always be an indistinguishability of what's happening below.

As a result of this: does it mean anything to be 'a simulation'?

My own current thinking is that the Born rule - the everydayness of everyday life - is a reflection of how consciousness must function. I am just not entirely sure how yet...

Comment by rosyatrandom on The I-Less Eye · 2010-03-28T21:05:02.040Z · LW · GW


Yes, I see it now. The dead-end copies function as traps, since they stop your participation in the game. As long as you can consciously differentiate your state as a copy or original, this works.

Comment by rosyatrandom on The I-Less Eye · 2010-03-28T19:36:05.552Z · LW · GW

This 0.5^99 figure only appears if each copy bifurcates iteratively.

Rather than

1 becoming 2, becoming 3, becoming 4, ... becoming 100

We'd have

1 becoming 2, becoming 4, becoming 8, ... becoming 2^99

Comment by rosyatrandom on There just has to be something more, you know? · 2010-03-24T02:00:23.556Z · LW · GW

I think a good term for what I believe in might be 'abstractionism'; essentially, I believe in all possible things, and all entities existing in all possible contexts.

From this perspective, matter, mind and mathematics are all the same kind of stuff: patterns. The mind is a pattern than can be abstracted from processes functioning to solve problems which, at a high level, implement our thoughts. Those processes can be performed by brains running in the kinds of universe we are familiar with, which run on ontological frameworks consistent, at least for observable parts, with the mathematics we know and love.

What is it all made of? Just information that can be endlessly traced downward through infinite contextual abstractions. In the end, there are only two definitive aspects: Everything (the kaleidoscopic, crystalline pattern of patterns) and the manner by which elements abstract (which links Nothing to Anything to Everything).

Or, as some of you may recognise it, The Dust Theory. In the end, all other theories require an arbitrary contextual abacadabra.

Comment by rosyatrandom on The AI in a box boxes you · 2010-02-02T15:42:31.782Z · LW · GW

No, but it's there as a baseline.

So in the original scenario above, either:

  • the AI's lying about its capabilities, but has determined regardless that the threat has the best chance of making you release it
  • the AI's lying about its capabilities, but has determined regardless that the threat will make you release it
  • the AI's not lying about its capabilities, and has determined that the threat will make you release it

Of course, if it's failed to convince you before, then unless its capabilities have since improved, it's unlikely that it's telling the truth.

Comment by rosyatrandom on The AI in a box boxes you · 2010-02-02T15:29:05.484Z · LW · GW

If the AI can create a perfect simulation of you and run several million simultaneous copies in something like real time, then it is powerful enough to determine through trial and error exactly what it needs to say to get you to release it.

Comment by rosyatrandom on When does an insight count as evidence? · 2010-01-04T17:02:58.129Z · LW · GW

I think this post makes an excellent point, and brings to light the aspect of Bayesianism that always made me uncomfortable.

Everyone knows we are not really rational agents; we do not compute terribly fast or accurately (as Morendil states), we are often unaware of our underlying motivations and assumptions, and even those we know about are often fuzzy, contradictory and idealistic.

As such, I think we have different ways of reasoning about things, making decisions, assigning preferences, holding and overcoming inconsistencies, etc.. While it is certainly useful to have a science of quantitative rationality, I doubt we think that way at all... and if we tried, we would quickly run into the qualitative, irrational ramparts of our minds.

Perhaps a Fuzzy Bayesianism would be handy: something that can handle uncertainty, ambivalence and apathy in any of its objects. Something where we don't need to put in numbers where numbers would be a lie.

Doing research in biology, I can assure you that the more decimal places of accuracy I see, the more I doubt its reliability.

Comment by rosyatrandom on The Moral Status of Independent Identical Copies · 2009-12-01T12:14:15.258Z · LW · GW

Since morality is subjective, then don't the morals change depending upon what part of this scenario you are in (inside/outside)?

I operate from the perspective (incidentally, I like the term 'modal immortality') that my own continued existence is inevitable; the only thing that changes is the possibility distribution of contexts and ambiguities. By shutting down 99/100 instances, you are more affecting your own experience with the simulations than their's with you (if the last one goes, too, then you can no longer interact with it), especially if, inside a simulation, other external contexts are also possible.