LessWrong 2.0 Reader

I would buy various forms of merch, including clothing. I feel very fond of LessWrong and would find it cool to wear a shirt or something with that brand.

My initial reaction - A lot of AI related predictions are based on "follow the curve" predictions, and this is mostly doing that. With a lack of more deeper underlying theory on the nature of intelligence, I guess that's all we get.

If you look at the trend of how far behind China is to the US, that has gone from 5 years behind 2 years ago, to maybe 3 months behind now. If you follow that curve, it seems to me that China will be ahead of the US by 2026 (even with the chip controls, and export regulations etc - my take is you're not giving them enough agency). If you want to follow the curve, IMO  you can /s/USA/China after 2026 (i.e - China is ahead of the US), and I can imagine it being a better trend-following prediction. It's much less convincing to tell a story about Chinese AI labs being ahead given who we are, but I'd put at least a 50/50 chance on China being ahead vs. USA being ahead.

Other than that, thanks for putting something concrete out there. Even though it's less likely the more specific it is, I feel this will get a lot more talked about, and hopefully some people with power (i.e - governments) start paying some attention to disempowerment scenarios.

To most Americans, "cream cheese" is savory.

Um, no, not particularly?

cured fish.

Why would I do that to myself? I don't feel my sins deserve that level of punishment.

You don't put it on dessert, right?

All the time. Well, in.

Specifically, I think we should call it "cheesecake frosting".

I would read that, first, as something you'd put on cheesecake, and, second, in terms of some of the kinds of cheesecake out there that would be, unfortunate as frostings.

On the other hand, I think whipped cream cheese on an Oreo is decent imitation of cheesecake with an Oreo crust, so I'm not sure I'm the best person to listen to here.

Coupled with your heretical views on cupcakes, it does seem to take you out of my preferred circle of advisors on food-related matters...

Humans didn't always speak in 50-word sentences. If you want to figure out how we came to be trending away from that, you should try to figure out how, when, and why that became normal in the first place. 

I’m curious how it went in terms of ‘do you think you learned anything useful?’

It gets caught.

At this point, wouldn't Agent-4 know that it has been caught (because it knows the techniques for detecting its misalignment and can predict when it would be "caught", or can read network traffic as part of cybersecurity defense and see discussions of the "catch") and start to do something about this, instead of letting subsequent events play out without much input from its own agency? E.g. why did it allow "lock the shared memory bank" to happen without fighting back?

It's rare to see someone with the prerequisites for understanding the arguments (e.g. AIT and metamathematics) trying to push back on this

My view is probably different from Cole's, but it has struck me that the universe seems to have a richer mathematical structure than one might expect given a generic AIT-ish view(e.g. continuous space/time, quantum mechanics, diffeomorphism invariance/gauge invariance), so we should perhaps update that the space of mathematical structures instantiating life/sentience might be narrower than it initially appears(that is, if "generic" mathematical structures support life/agency, we should expect ourselves to be in a generic universe, but instead we seem to be in a richly structured universe, so this is an update that maybe we can only be in a rich/structured universe[or that life/agency is just much more likely to arise in such a universe]). Taken to an extreme, perhaps it's possible to derive a priori that the universe has to look like the standard model. (Of course, you could run the standard model on a Turing machine, so the statement would have to be about how the universe relates/appears to agents inhabiting it, not its ultimate ontology which is inaccessible since any Turing-complete structure can simulate any other)

I played the first third or so of this game when it first came out, and haven't touched it since then. We did two rounds of the exercise, interspersed with 30 minutes of playing Baba is You levels the regular way to build up more intuition (most attendees were either new to the game or haven't played it for years). Some people paired up and some people did the exercise individually.

I did Tiny Pond for the first workshop independently, and found it very difficult - despite running through the strategizing and metastrategizing twice, I was still very stuck.

I did The River for the second workshop (after running through the first few levels of Baba is You again). This time I paired up with someone else, and we were able to get to the correct solution after the first round of strategizing.

Thanks for writing this up, my meetup group just ran a meetup on this [? · GW]. I've told the folks here to give their experiences with the workshop here, because we used pen and paper instead of the google doc.

I mention this up top in an AI post despite all my efforts to stay out of politics, because in addition to torching the American economy and stock market and all of our alliances and trade relationships in general, this will cripple American AI in particular.

Are we in a survival-without-dignity timeline [LW · GW] after all? Big if true.

(Inb4 we keep living in Nerd Hell and it somehow mysteriously fails to negatively impact AI in particular.)