Posts
Comments
Like, I hear you, but that is...also not how they teach gun safety. Like, if there is one fact you know about gun safety, it's that the entire field emphasizes that a gun is inherently dangerous towards anything it is pointed towards.
I largely agree, but think given government hiring timelines, there's no dishonor in staying at a lab doing moderately risk-reducing work until you get a hiring offer with an actual start date. This problem is less bad for the special hiring authorities being used for AI stuff oftentimes, but it's still not ideal.
Oh wow, this is...not what I thought people meant when they say "unconditional love."
In my circles, "conditional love" is about love with lots of threats and demands that the other person change, and if failing to do so, they would be told they're unworthy of being loved by their partner.
As you know from our conversations, I'm largely in the same camp as you on this point.
But one point I'd make incrementally is this: USG folks are also concerned about warning shots of the nature, "The President's Daily Brief ran an article 6 months ago saying warning signs for dangerous thing X would be events W, Y, and Z, and today the PDB had an article saying our intelligence agencies assess that Y and Z have happened due to super secret stuff".
If rationalists want rationalist warning shots to be included, they need to convince relevant government analytic stakeholders of their relevance.
That's probably true if the takeover is to maximize the AI's persistence. You could imagine a misaligned AI that doesn't care about its own persistence -- e.g., an AI that got handed a misformed min() or max() that causes it to kill all humans instrumental to its goal (e.g., min(future_human_global_warming))
The only rational response to AI music generation is live-only music creation within the context of traditional norms that prohibit the use of electricity for productive work.
Unrelatedly, I am co-organizing a kabbalat shabbat at Manifest Conference this year
The fridge horror of this story comes when you realize that the AI timed the call to Biden until it was sure that Biden's advisor had been successfully and fully briefed
Market on 2029 culture war:
I think I'm confused about something in your reference-class argument here. I'm going to say it in a very boring way, on the assumption someone will come by and tell me how I'm wrong shortly: It seems like you're saying that the right reference class is "all possible minds". But isn't the right reference class "minds that look like humans in the 21st century at the (putative) dawn of AI"?
Separately, do we know that we expect all sims to have equal distributions of real to unreal persons as we believe to be the case? I don't think I've ever met someone from lots of countries; why couldn't I be in the ancestor sim where most persons are p-zombies running on simplified scripts unless they're needed to suddenly have detailed interiority "for the plot"?
I find the subscripts distracting, personally, and would prefer something like "Zebras discover fire (60%)" in parentheses
Ok, I'll bite: does that mean that you think China is less likely to invade China if recession? This would be happy news, but is not necessarily consensus view among DC defense wonks (i.e. the fear of someone saying, "We need a Short Victorious War").
I mean, does your Vanguard targeted lifecycle index fund likely invest in equities exposed to AGI growth (conditional on non-doom)?
If you think money still has meaning after AGI and meaningful chance of no-doom, it might actually be optimal to invest in your retirement fund.
I feel like, "adopt a religious/moral stringency or observance" or stopping the same is also a key item here.
I would like to thank you for this post and for encouraging me to get a ball of fairy lights, which provides me with a genuinely unexpected amount of joy whenever I look at it up close.