LessWrong 2.0 Reader

I've always referred to that as the Law of Large Numbers. If there are enough chances, everything possible will happen. For example, it would be very surprising if I won the lottery, but not surprising if someone I don't know won the lottery.

Maybe our concept of a sock or an apple somehow (structurally) resembles a sock or an apple.

I could start writing pairs of sentences like:

REAL WORLD: feet often have socks on them
MY INTUITIVE MODELS: “feet” “often” “have” “socks” “on” “them”

REAL WORLD: socks are usually stretchy
MY INTUITIVE MODELS: “socks” “are” “usually” “stretchy”

(… 7000 more things like that …)

If you take all those things, AND the information that all these things wound up in my intuitive models via the process of my brain doing predictive learning from observations of real-world socks over the course of my life, AND the information that my intuitive models of socks tend to activate when I’m looking at actual real-world socks, and to contribute to me successfully predicting what I see … and you mix all that together … then I think we wind up in a place where saying “my intuitive model of socks has by-and-large pretty good veridical correspondence to actual socks” is perfectly obvious common sense. :)

But what if I'm thinking of the content of your suitcase, and I don't know whether it is a sock or an apple or something else? Surely the part of the model (my brain) which represents/refers to the content of your suitcase does not in any way (structurally or otherwise) resemble a sock, even if the content of your suitcase is indeed identical to a sock.

Right, if I don’t know what’s in your suitcase, then there will be rather little veridical correspondence between my intuitive model of the inside of your suitcase, and the actual inside of your suitcase! :)

(The statement “my intuitive model of socks has by-and-large pretty good veridical correspondence to actual socks” does not mean I have omniscient knowledge of every sock on Earth, or that nothing about socks will ever surprise me, etc.!)

But Scott and [Yvain] are an object in the territory, not parts of a model, so the parts of the model which do represent Scott and Yvain require the existence of some sort of representation relation.

Oh sorry, I thought that was clear from context … when I say “Scott is one part of M”, obviously I mean something more like “[the part of my intuitive world-model that I would describe as Scott] is one part of M”. M is a model, i.e. data structure, stored in the cortex. So everything in M is a part of a model by definition.

Google's red team already knows. They have known about the problem for at least six months and abused the issue successfully in engagements to get very significant access. They're just not really sure what to do because the only solutions they can come up with involve massively disruptive changes.

The Diplomatic Channel is now available once again. [? · GW]

If what you're saying is no set of beliefs perfectly captures reality then I agree.

If what you're saying is don't get attached to your beliefs, I agree. Even rationalists are often anchored to beliefs and preferences much more than is Bayesian optimal. Rationalism provides resistance but not immunity to motivated reasoning.

If you're saying beliefs aren't more or less true and more or less useful, I disagree.

If you're saying we should work on enlightenment before working on AGI x-risk, I disagree.

We may well not have the time.

Is there a name of the thing where an event happens and that specific event is somewhat surprising, but overall you would have expected something in that reference class (or level of surprisingness) to happen?

E.g. It was pretty surprising that Ivanka Trump tweeted about Situational Awareness, but I sort of expect events this surprising to happen.

Oops, I am sorry. We did not intend to take the site down. We ran into an edge-case of our dialogue code that nuked our DB, but we are back up, and the Petrov day celebrations shall continue as planned. Hopefully without nuking the site again, intentionally or unintentionally. We will see.

I know some pretty senior people in security for 2 FAANG companies, and passing acquaintance at others, and currently work in the Security org at a comparable company.  All of them have reporting channels for specific threats, and none (that I know) are ignorant of the range of AI-enabled attacks that are likely in the near future (shockingly many already).   The conversations I've had (regarding products or components I do know pretty well) have convinced me that everything I come up with is already on their radar (though some are of the form "Yeah, that's gonna happen and it's gonna suck.  Current strategy is to watch for it and not talk much about it, in order not to encourage it").

Without disclosing some details, there's probably no way to determine whether your knowledge or theory is something they can update on.  I'm happy to pass on any information, but I can't see why you'd trust me more than more direct employees of the future victims.

"Sorry, you don't have access to this draft"

This is fun! I don't know which place I'm a citizen of, though, it just says "hello citizen"... I feel John Rawls would be pleased...