LessWrong 2.0 Reader

I can imagine someone several hundred years ago having figured out, purely based on first-principles reasoning, that life is no crisp category at the territory but just a lossy conceptual abstraction. I can imagine them being highly confident in this result because they've derived it for correct reasons and they've verified all the steps that got them there. And I can imagine someone else throwing their hands up and saying "I don't know what mysterious force is behind the phenomenon of life, and I'm pretty sure no one else does, either".

But is this a correct conclusion? I have an option right now to make a civilization out of brains-in-vats in a sandbox simulation similar to our reality but with clear useful distinction on life VS non life. Like, suppose there is a "mob" class.

Like, then, this person there, inside it, who figured out that life and non life is a same thing is wrong in a local useful sense, and correct in a useless global sense (like, everything is code / matter in outer reality).  People inside the simulation who found the actual working thing that is life scientifically, would laugh at them 1000 simulated years later and present it as an example of presumptuousness of philosophers. And i agree with them, it was a misapplication.

If you're interested in doing a PhD in AI in the UK, I recommend applying for the Centres for Doctoral Training (CDTs) in AI such as:

• CDT in Responsible and Trustworthy in-the-world NLP (University of Edinburgh)
• CDT in Practice-Oriented Intelligence (University of Bristol)
• CDT in Fundamentals of AI (University of Oxford)
• CDT in Safe and Trusted AI (King's College London)
• CDT in Statistics and Machine Learning (University of Oxford)

Note that these programs are competitive so the acceptance rate is ~10%.

I just donated $1,000. This is not a minor amount for me, and I almost just donated $10 as suggested in Shoshannah's excellent comment,  but I knew I could donate that much without thought or effort, and I wanted to really put at least some effort into this, after seeing how much obvious effort Oliver and others at Lesswrong have been putting in. 

My decision process was as follows:

First, I dealt with my risk aversion/loss aversion/flinch response to giving large sums of money away. This took a couple minutes, much faster than it used to be thanks to things like my Season of Wealth a couple years ago, but felt like a mildly sharp object jiggling around in my chest until I smoothed it out with reminders of how much money I make these days compared to the relatively poor upbringing I had and the not-particularly-high salary I made for the first ~decade of my adult life. 

Second, I thought of how much I value Lesswrong and Lighthaven existing in the world as a vague thing. Impersonally, not in the ways they have affected me, just like... worlds-with-these-people-doing-this-thing-in-it vs worlds-without. This got me up to roughly 25ish, something vaguely more than double the 10.

Third, I thought about how much value I personally have gained from Lesswrong and Lighthaven. I cannot really put a number on this. It's hard to disentangle the value from all the various sources in the rationality space, and the people who posts on LW and attended Lighthaven events. This ballooned the amount to something extremely hard to measure. Far more than $100, but probably less than 10,000? 

Fourth, I dealt with the flinch-response again. 10,000 is a lot for me. I lost more than that due to FTX's collapse even before the clawback stress started, and that took a bit of time to stop feeling internal jabs over. A few subsections needed dealing with; what if I encounter an emergency and need lots of money? What if my hypothetical future wife or kids do? Would I regret donating then? This bumped me way back down to the 100s range.

Fifth, I thought about how I would feel if I woke up today and instead of reading this post, I read a post saying that they had to shut down Lighthaven, and maybe even LessWrong, due to lack of funding. How much I would regret not having donated money, even if it didn't end up helping. I'm still quite sad that we lost Wytham, and would pay quiet a bit of money to retroactively try to save it if I could. This brought me up to something like $3-500.

Sixth, I dealt with the vague feelings of "hopefully someone out there will donate enough that my contribution will not really matter, so maybe I don't even need to really donate much at all?" This feeling felt bad, and I had a brief chat with my parts, thanking my internal pragmatism before exploring together if this is the sort of person we want to be when other people might need us. After that conversation was over the number had stabilized around the 500 range.

Seventh, I thought about the social signal if I say I donated a lot and how this might encourage others to donate more too, effectively increasing the amount Lesswrong gets, and decided this didn't really affect much. Maybe a minor effect toward increasing, but nothign noticeable.

Eighth, I thought about the impact to the world about Alignment. I felt the black hole there, the potential infinite abyss that I could throw my savings and life into and probably not get any useful effect out of, and spent some time with that before examining it again and feeling like another few hundred may not "make sense" in one direction or the other, but felt better than not doing it.

And ninth, I finally thought about the individuals working at Lightaven that I know. How much do I trust them? How much do I want them to feel supported and motivated and cared for by the community they're contributing so much to?

By the end of that I was around 8-900 and I thought, fuck it, I've made stupider financial decisions than an extra hundred bucks for a fancy T-shirt, and nice round numbers are nice and round.

Thank you all for all you do. I hope this helps.

Recurring option at the main donation link?

The AI time estimates are wildly high IMO, across basically every category. Some parts are also clearly optional (e.g. spending 2 hours reviewing). If you know what you want to research, writing a statement can be much shorter. I have previously applied to ML PhDs in two weeks and gotten an offer. The recommendation letters are the longest and most awkward to request at such notice, but two weeks isn't obviously insane, especially if you have a good relationship with your reference letter writers (many students do things later than is recommended, no reference letter writer in academia will be shocked by this).

If you apply in 2025 December, you would start in 2026 fall. That is a very very long time from now. I think the stupidly long application cycle is pure dysfunction from academia, but you still need to take it into account.

(Also fyi, some UK programs have deadlines in spring if you can get your own funding)

Okay, this at least helps me better understand your position.  Maybe you should have opened with "China Hawks won't do the thing they've explicitly and repeatedly said they are going to do"
 

Couldn't you imagine that you use philosophical reasoning to derive accurate facts about consciousness, which will come with insights about the biological/computational structure of consciousness in the human brain, which will then tell you things about which features are critical / how hard human brains are to simulate / etc.? This would be in the realm of "empirical predictions derived from philosophical reasoning that are theoretically testable but not practically testable".

I think most solutions to consciousness should be like that, although I'd grant that it's not strictly necessary. (Integrated Information Theory might be an example of a theory that's difficult to test even in principle if it were true.)

I think one underused trick for training LLMs is to explicitly "edit" them. That is, suppose they generate some text X in response to prompt Y, and it has some error or is missing something. In that case you can create a text X' that fixes this problem, and do a gradient update to increase log P(X'|Y)/P(X|Y).

For example, if we generate virtual comments in the style of certain LW users, one could either let those users browse the virtual comments that have been created in their style and correct them, or one could let the people who receive the virtual comments edit them to remove misunderstanding or similar.

Read your comment, donated 10$.

I was going to email but I assume others will want to know also so I'll just ask here. What is the best way to donate an amount big enough that it's stupid to pay a Stripe fee, e.g. $10k? Do you accept donations of appreciated assets like stock or cryptocurrency?