Posts
Comments
There's a couple problems that even though Sam is surrounded by people like Ilya and Jan warning him of the consequences, he's currently unwilling to change course.
- Staring down AGI ruin is just fundamentally a deeply scary abyss. Doubly so when it would've been Sam himself at fault.
- Like many political leaders, he's unable to let go of power because he believes that he himself is most capable of wielding it, even when it causes them to take actions that are worse than either leader would do independently if they weren't trying to wrest power away from the other.
- More than most, Sam envisions the upside. He's a vegetarian and cares a lot about animal suffering ("someday eating meat is going to get cancelled, and we are going to look back upon it with horror. we have a collective continual duty to find our way to better morals"), seeing the power of AGI to stop that suffering. He talks up the curing cancer and objectively good tech progress a lot. He probably sees it as: heads, I live forever and am the greatest person for bringing utopia, tails, I'm dead anyway like I would be in 50 years.
Ultimately, I think Sama sees the coin flip (but believing utopia AGI is higher chance) and being the ever-optimist, is willing to flip it for the good things because it's too scary to believe that the abyss side would happen.
"Hundred years ago, if you attended school until you were 18, you were considered highly educated. Today, you are considered too stupid to be allowed working at half of the jobs unless you have some kind of university."
You just made me realize how big of an impact university expectations are on fertility twofold. It's obvious that if people are supposed to be 22 now before starting "real adult life", instead of 18, that will affect fertility rates by virtue of being less time to get situated and a shorter fertility window. However, there's a not so obvious double whammy, in that your children will also now be dependents for 4 years longer too! So instead of 18-40 year olds thinking, "i have 22 years to have kids that will just need to make it to 18", we now have to think "i have 18 years to have kids that will need to make it to 22." Timeframe shortened ~20%, dependency burden also increased ~20%.
Upcoming AGI x-risk upweights the simulation hypothesis for me because...
Of all the peoples' lives that exist and have existed, what are the chances I'm living one of the most prosperous lives in all of humanity, only to descend into facing the upcoming rapture of the entire world? Sounds like a video game / choose your adventure from another life...