LessWrong 2.0 Reader

That slides presentation presents me with a "you need access" screen. Is it OK to be public?

I thought this was an interesting take on the Boundaries problem in agent foundations from the perspective of IIT. It is on the amazing Michael Levin's youtube channel: https://www.youtube.com/watch?app=desktop&v=5cXtdZ4blKM

One of the main things that makes it interesting to me is that around 25-30 mins in, ot computationally goes through the main reason why I don't think we will have agentic behaviour from AI in at least a couple of years. GPTs just don't have a high IIT Phi value. How will it find it's own boundaries? How will it find the underlying causal structures that it is part of? Maybe this can be done through external memory but will that be enough or do we need it in the core stack of the scaling-based training loop?

A side note is that, one of the main things that I didn't understand about IIT before was how it really is about looking at meta-substrates or "signals" as Douglas Hofstadter would call them are optimally re-organising themselves to be as predictable for themselves in the future. Yet it does and it integrates really well into ActInf (at least to the extent that I currently understand it.)

Yeah, powering through it.  I've tried adult Fiction and Sci-Fi but he's not interested in it yet - not grokking adult motivations, attitudes and behaviors yet, so feeding him stuff that he enjoys to foster habit of reading.   

I think children can sleep in most places as long as they feel safe. Some parents seem to think that their children can only sleep in tightly controlled environments: Quiet, dark, comfy. But I think that is often a result of training. If the children never sleep in any other environments how can they feel suddenly safe there? Or if the parents or other people are stressed in the other environments, children will notice that something is off and not feel safe and not sleep. But a place with lots of friendly, happy people seems quite safe to me.

I found a photo of two of my kids sleeping "on stage." This table was right next to the stage at my sisters wedding and the music was not quiet for sure.

Submission statement: I mostly finished this a year ago, but held off on posting because I was planning on improving it and writing a corresponding "here's the concepts without the math" post. Might still happen, but now I'm not aiming at a specific timeline.

Things I now want to change:

• Soften the confidence in the vNM axioms, since there's been some good criticisms
• Revamp the whole ontological crisis section to be more general
• Rewrite from academese to easier
• Move proofs to an appendix
• Create some manim videos to illustrate
• Merge with this post [LW · GW]
• Many other things

Still, I hope this is kinda useful for some people.

Edit: Also, there's some issues with the MathJax and dollar signs, I will fix this later.

It's been moved to https://laneless.substack.com/ .

Thank you for the reply.

What kind of questions, analogies, or models are your fellow students responding to your explanations with? Are there any patterns in the specific feedback you've noticed? Are there any particular aspects of Deep Learning or the metaphors or terminology you're using that seem to be the biggest bottlenecks?

My hunch is that maybe you instead look at beginner's introductions to Deep Learning and Neural Networks and see how they go about conveying these concepts. If someone else has done the hard work of figuring out an expedient way to convey the subject matter, why not borrow from them (giving credit, of course)?

Please do get back if you can think of specific examples of the second case and I'll think any books or resources I know of which might be suitable.

 

Straightforward "strategic ignorance" (avoiding to learn things on purpose to avoid related obligations) seems like an obvious moral failure. The practical problem is that once we start judging people for strategic ignorance, it may motivate them to make their strategy indirect. If you can be blamed for not taking a swimming class that your school provided, it motivates you to choose a school that does not provide swimming classes. Or vote for a government that removes swimming classes from schools, because then it's no longer your fault.

This posits a sort of moral obligation to maximally extend your capacity to help others or take care of yourself in a sustainable way.

Yes. Unfortunately, I have only heard this idea in some form from Eliezer Yudkowsky [? · GW] and Jordan Peterson. It seems to be outside the social Overton window.

I suppose the reason is that, socially, we need a definition of "ethics" such that most people kinda reach it. Otherwise we don't get the peer pressure... and might actually get peer pressure against [LW · GW].

Seems like we have two different things here -- what is the right thing to do, and what is the optimal social norm to promote -- and the relation between them is complicated. It feels it would be nice if these two could be the same thing. Promoting the thing that is the right thing to do, sounds like the right thing to do. But that only works if people already agree, or if there is a cult-like situation that can make them agree (to the degree that they become the enforcers of the norm in private; otherwise you just get two competing moralities). In reality, outside of cults, you don't get an agreement on anything.

Another option is to choose the optimal social norm (the thing that realistically can be approved of by the majority) and pretend that this is the right thing to do. I think that's how it works in practice. The problem is what to do about those parts of "doing the right thing" that don't fit into the "optimal norm that can be socially enforced"? If you openly admit that the social norm is actually not the right thing to do, you undermine the social norm. An alternative is to adopt a (logically inconsistent, if you look too closely) position that something things are "good", but some things are "beyond-good" -- good if you choose to do them, but if you refuse to do them, it doesn't make you bad.

So, using the traditional language, saving the drowning child is an obligation for swimmers; and learning to swim is supererogatory. Until it happens that most of the people in your society learn to swim, and then you can switch and make learning to swim an obligation.

An obvious way to do so is put a hazard sign on "probability" and just not use it, not putting resources into figuring out what "probability" SB should name, isn't it?

It's an obvious thing to do when dealing with simularity clusters poorly defined in natural language. Not so much, when we are talking about a logically pinpointed mathematical concept which we know are crucial for epistemology.

(And now I realize a possible point why you're arguing to keep "probability" term for such scenarios well-defined; so that people in ~anthropic settings can tell you their probability estimates and you, being observer, could update on that information.)

It's not just about anthropic scenarios and not just about me being able to understand other people. It's about general truth preserving mechanism of logical and mathematical reasoning. When people just use different definitions - this is annoying but fine. But when they use different definitions without realizing that these definitions are different and, moreover insist that it's you who is making a mistake - then we have an actual disagreement about math which will provide more confusion along the way. Anthropic scenarious are just the ones where this confusion is noticeable.

As for why I believe probability theory to be useful in life despite the fact that sometimes different tools need to be used

What exactly do you mean by "different tools need to be used"? Can you give me an example?

I like this post but I think redwood has varied some on whether control is for getting alignment work out of AIs vs getting generally good-for-humanity work out of them and pushing for a pause once they reach some usefulness/danger threshold (eg well before super intelligence).

[based on my recollection of Buck seminar in MATS 6]