LessWrong 2.0 Reader

I tried to model a best possible confinement strategy in Multilevel AI Boxing. 
I wrote it a few years ago and most ideas will unlikely work for current situation with many instances of chats and open weight models. 
However, the idea of landmines - secret stop words or puzzles which stop AI - may still hold. It is like jail breaking in reverse: unaligned AI finds some secret message which stops it. It could be realized on hardware level, or through anomalous tokens or "philosophical landmines'. 

About archipelago: c2.com, the original wiki tried this. They created a federated wiki but that didn't seem to work. My guess: the volume was too low.

And LW has already all the filtering you need: just subscribe to the people and topics you are interested. There is also the unfinishe reading list.

I get tha  this may not feel like its own community. Within LW this could be done with ongoing open threads about a topic. But tgat requires an organizer and participation.  And we are back at volume.  And at needing good writers. 

(Interesting. FWIW I've recently been thinking that it's a mistake to think of this type of thing--"what to do after the acute risk period is safed"--as being a waste of time / irrelevant; it's actually pretty important, specifically because you want people trying to advance AGI capabilities to have an alternative, actually-good vision of things. A hypothesis I have is that many of them are in a sense genuinely nihilistic/accelerationist; "we can't imagine the world after AGI, so we can't imagine it being good, so it cannot be good, so there is no such thing as a good future, so we cannot be attached to a good future, so we should accelerate because that's just what is happening".)

I also think monopolizing talent enables software companies to make sure those high fixed costs stay nice and high.

If you disagreed with this, is it because you think it is literally false or because you don't agree with the implied argument that software companies are doing this on purpose?

Richness: The model must include all the propositions the agent can meaningfully consider, including those about herself. If the agent can form a proposition “I will do X”, then that belongs in the space of propositions over which she has beliefs and (where appropriate) desirabilities.

I see a potential problem here, depending on what exactly is meant by "can meaningfully consider".

Consider this set up:

You participate in the experiment for seven days. Every day you wake up in a room and can choose between two envelopes. One of them has 100$ the other is empty. Then your memory of this act is erased. At the end of the experiment you get all the money that you've managed to win.

On day one money are assigned to an envelope randomly. However, on all the next days the money are put in the envelope that you didn't pick on the previous day. You do not have any access to random number generators.

Is the model supposed to include credence for proposition "Today the money is in envelope 1" when you wake up participating in such experiment?

Hm, this violates my model of the world.

there are too many AI companies for this deal to work on all of them

Realistically, I think there are like 3-4 labs^[1] that matter, OAI, DM, Anthropic, Meta.

some of these AI companies will have strong kinda-ideological commitments to not doing this

Even if that was true, they will be at the whim of investors who are almost all big tech companies.

this is better done by selling (even at a lower revenue) to anyone who wants an AI SWE than selling just to Oracle.

This is the explicit claim I was making with the WTP argument. I think this is firmly not true, and OpenAI will make more money by selling just to Oracle. What evidence causes you to disagree?

1. ^{^}

   American/Western labs.

I hadn't downvoted this post, but I am not sure why OP is surprised given the first four paragraphs, rather than explaining what the post is about, instead celebrate tree murder and insult their (imagined) audience:

so that no references are needed but those any LW-rationalist is expected to have committed to memory by the time of their first Lighthaven cuddle puddle

I think austerity has a weird relationship with counterfactuals?

It's not clear why anyone would want to claim a self-diagnosis of that, since little about it is 'egosyntonic', as the psychiatrists say.

Since a friend mentioned I might be schizoid, I've been like "...yeah? somewhat? maybe? seems mixed? aren't I just avoidant? but I feel more worried about relating than about being rejected?", though I'm not very motivated to learn a bunch about it. So IDK. But anyway, re/ egosyntonicity:

• Compared to avoidant, schizoid seems vaguely similar, but less pathetic; less needy or cowardly.
• Schizoid has some benefits of "disagreeability, but not as much of an asshole". Thinking for yourself, not being taken in by common modes of being.
• Schizoid is maybe kinda like "I have a really really high bar for who I want to relate to", which is kinda high-status.

I describe myself as a techno-hippy. I was reading Cory Doctorow while waiting for the next chapter of HPMoR. I’ve often wanted a good leftrat community. I feel ya. I have an intuition that Consequentialism is a dangerous philosophy to adopt for optimizers, especially when they get scared. It’s not a huge hop from “saving the world” to “saving people from themselves.”

When I was in my twenties, I’d spend hours a day in forums and on Reddit. I’ve moderated and participated. Now, I’m 47 I just don’t have it in me. Planting a banner works, but it’s a lot of work. 

This is about as close as I get to social media nowadays. It’s too easy to get people angry online, and to use that anger as a weapon. Right now I’m hoping I can guide my son through his teen-age years without him being weaponized. Hopefully kindness and the virtues of Stoicism stand stronger against inversion than Consequentialism.