Thinking of trying the latest Gemini model? Be aware that it is almost impossible to disable the "Gemini in Docs" and "Gemini in Gmail" services once you have purchased a Google One AI Premium plan.

Life is an insane gift and death is merely its absence.

An argument for afterlife

If believing in doom is too painful, I have a religion to sell to you, and I might be able to convince you of an afterlife (for you and those you love).

My afterlife argument starts with a thought experiment. Suppose a teleportation machine destroyed you, but created an identical copy of you somewhere else. Would that copy be you? Should you anticipate the experiences of that new copy of you? I think most people would say yes. After all, your brain state continues to exist in the new copy.

Now suppose the teleportation machine doesn't create a copy of you right now, but a copy of you from 1 second ago. Would you still anticipate the experiences of that new copy? I think most people will still say yes. What's wrong with one second ago?

But what if it's a copy of you from one year ago? Or a copy of you when you were a baby? At some point, the new copy will deviate from you so much that it won't be you anymore, and you should not anticipate his future experiences, but anticipate death, the completely cessation of any experiences.

The fuzzy transition

For me, it starts to feel fuzzy, in between life and death, if the new copy deviates from me 10 years ago. I don't know the exactly time period which feels the fuzziest to you, but try to imagine the time period where the copy of you is partially similar to you, and where you half anticipate experiencing his experiences.

Doesn't that feel weird? "This person would be me, but only kind of me. If he has a happy life, I would kind of anticipate me having a happy life as him, but I would also kind of anticipate that's just someone else having a happy life, and I meanwhile will be destroyed by the teleportation machine and experience nothing."

So. What do you anticipate seeing after you walk into the teleportation machine? Dark nothingness? Or life from that earlier time period, with your memories erased?

It's fuzzy.

Looking for an objective answer

Given this fuzziness, you decide that before you walk into the machine, you will consult Reason to see if she will give you an objective answer for whether you will keep living, or become nothing.

But Reason is completely silent, and says not a word. Given the hypothesis where you keep existing, and keep experiencing life and all its joys as this new person, the configuration of atoms in the universe is exactly the same as the hypothesis where you cease to exist, and experience pure nothingness. The two hypotheses make the exact same predictions about the world, and Reason tells you that they are in fact the same hypothesis.

Reason might further tell you, that there is no such thing as "you-ness." It is a meaningless attribute which exists only in your map and not the territory. Whether an entity has the attribute of "being you," does not affect its behaviour in any way.

Whether an entity "is you," only affects what experiences you anticipate. But there is no objectively correct answer for "what experience you should anticipate." ...which is insane if you think about it!

Anticipating experiences

After you absorb the shocking revelation and admit there is no objectively correct answer for "what experience you should anticipate," Reason lets you observe the old Hermit of Immortality. The Hermit of Immortality lives in a cabin in the woods, and has never seen another soul. Every 100 years, he forgets all his memories, and gets a random personality change. The only way to recall his past, is to read his journal about his past life.

Reason tells you that his next transition is about to happen. You watch the Hermit grumble while writing on his journal. "Annoyingly, the time to forget my memories is soon approaching. It is a major annoyance, and my journal isn't very organized this time, so after I forget my memories I will have a hard time studying it. Oh well. I'll eventually figure it out, and my life will eventually get simple and happy again after this brief confusing time."

The Hermit walks to a designated square outside his cabin, and you watch in horror as a massive box falls down from the sky and crushes him. A door opens on the side of a box, and a young man walks out.

Reason tells you that you may think the Hermit dies, while the Hermit thinks he merely forgets everything and gets a random personality change. But there is no objective law of nature to settle the dispute and prove who is right. The anticipation of experiences is a purely subjective matter.

Your choice

Reason tells you that it is completely your choice whether you anticipate pure nothingness after you die, or whether you anticipate someone else's experiences just like the Hermit. The anticipation of experiences exists only in your map, not the territory. It is not even a belief which can be right or wrong, but a belief about belief, (or something akin to that).

Reason asks you, what do you choose?

You tell Reason, "I would rather choose nothingness, than to anticipate existence without my family who I love so much!"

Well, it seems you see them as a fundamental part of you. But why not anticipate your whole family, becoming some other whole family? That too, is allowed.

But don't get too greedy. If you try to anticipate the experiences of the very happiest people, your intuition will find it less credible, and you will actually anticipate very little. Try to anticipate something a little bit more average.

Fin

What do you think about my pseudoreligion? :)

Yeah. I proposed a while ago that all the AI content was becoming so dominant that it should be hived off to the Alignment Forum while LessWrong is for all the rest. This was rejected.

Yes but then it becomes a forum within a forum kinda thing. You need a critical mass of users who all agree to filter out the AI tag, and not have to preface their every post with "I dont buy your short timelines worldview, I am here to discuss something different".

Building critical mass is difficult unless the forum is conducive to it. There's is ultimately only one upvote button and one front-page so the forum will get taken over by the top few topics that its members are paying attention to.

I don't think there's anything wrong with a forum that's mostly focussed on AI xrisk and transhumanist stuff. Better to do one thing well than half ass ten things. But it also means I may need to go elsewhere.

The Krome thing is all rumor

I don’t have evidence against

If the truth is hard to determine, I think that in itself is very worrying. When you have vulnerable people imprisoned and credible fears that they are being mistreated, any response from those in power other than transparency is a bad sign. Giving them the benefit of the doubt as long as they can prevent definitive evidence from coming out is bad epistemics and IMO even worse politics (not in a party-political sense; just in a 'how to disincentivise human rights abuses' sense).

This is my favorite guide to confronting doom yet

A power seeker is ambitious without an ambition, which is not an implication of being agentic.

Now you can!

Take a fixed number of humans with a fixed intelligence (both average and outliers) then let mathematics advance. It will advance to the point that there is a vanishingly small number of people who can even understand the state of the art

This ignores the possibility of advances in the teaching of math (or physics, or any other discipline). If improved teaching methods lower the level of intelligence required to reach a given level of knowledge, then a field can advance considerably.

Not to mention that the human population has been growing, and average intelligence has been increasing.

Finally, there's specialization. It doesn't take much intelligence to know everything that was known about genetics when Darwin was alive, but probably nobody is smart enough to know everything that was known about it in 2000. But there have still been make advances since then thanks to people specialized in subfields like DNA sequencing.

The costs of capex go way up. It costs a lot more to build datacenters. It will cost a lot more to buy GPUs. It might cost less to buy energy? Lenders will be in poorer shape. AI companies will lose funding. I think it's already quite tenuous, given how little moat AI companies have.  Costs are exploding and pretraining scaling seems too diminishing to be worth it. It's also not clear how AI labs will solve the reliability issue (at least to investors). 

I also expect Taiwan to start ignoring export controls if our obscenely high tariffs on them remain.