LessWrong 2.0 Reader

The root cause may be that there is too much inferential distance [? · GW]

Perhaps, although I generally become more sympathetic to someone's point of view the more I read from them.

And it's part of why I think it's useful to create scenes that operate on different worldview assumptions: it's worth working out the implications of specific beliefs without needing to justify those beliefs each time. 

I used to lean more strongly towards more schools of thought being good, however I've updated slightly on the margin towards believing thinking some schools of thought just end up muddying the waters.

That said, Epoch has done some great research, so I'm overall happy the scene exists. And I think Matthew Barnett is extremely talented, I just think he's unfortunately become confused.

This paradox doesn't occur because a computation trying to prove its own output (and give the opposite output) will have to simulate itself

Due to Löb, if a computation knows that if it finds a proof that it outputs A, then it will output A, then it proves that it outputs A, without any need for recursion. This is why you really shouldn’t output something just because you’ve proved that you will. 

Yeah, from the claim that pi starts with two you can easily prove anything. But I think:

(1) something like logical induction should somewhat help: maybe the agent doesn’t know whether some statement is true and isn’t going to run for long enough to start encounter contradictions.

(2) Omega can also maybe intervene on the agent’s experience/knowledge of more accessible logical statements while leaving other things intact, sort of like making you experience what Eliezer describes here as convincing that 2+2=3: https://www.lesswrong.com/posts/6FmqiAgS8h4EJm86s/how-to-convince-me-that-2-2-3 [LW · GW], and if that’s what it is doing, we should basically ignore our knowledge of maths for the purpose of thinking about logical counterfactuals.

That's not how the math works. Suppose there are 200 activities under the heading of "AI R&D" that each comprise at least 0.1% of the workload. Suppose we reach a point where AI is vastly superhuman at 150 of those activities (which would include any activities that humans are particularly bad at), moderately superhuman at 40 more, and not much better than human (or even worse than human) at the remaining 10. Those 10 activities where AI is not providing much uplift comprise at least 1% of the AI R&D workload, and so progress can be accelerated at most 100x.

This is oversimplified; there is some room for superhuman ability (making excellent choices of experiments to run) can compensate for lack of uplift in other areas (time to code and execute individual experiments). But the fundamental point remains: a complex process can be bottlenecked by its slowest step. Amdahl's Law is not symmetric – a chain can't be as strong as its strongest link.

I claim that it is possible to create a program, which can be interpreted as running uncountably infinite number of simulations

I can't see that such a programme would have to be interpreted as running uncountably infinite number of simulations

My response is to just discard those frameworks, and use something else.

something else that's another philosophical framework, or something else entirely?

All services not running behind AWS, GCP or Azure will be banned from access to the newly branded "Internet 2.0", as they are proven vulnerable to attack from any newer "PhD+ level reasoning/coding ai agent".

See also:

"Critical infrastructure systems often suffer from "patch lag," resulting in software remaining unpatched for extended periods, sometimes years or decades. In many cases, patches cannot be applied in a timely manner because systems must operate without interruption, the software remains outdated because its developer went out of business, or interoperability constraints require specific legacy software." -- Superintelligence Strategy by Dan Hendrycks, Eric Schmidt, Alexandr Wang Mar 2025 https://www.nationalsecurity.ai/chapter/ai-is-pivotal-for-nationaZ<l-security, https://arxiv.org/abs/2503.05628 

"Partner with critical national infrastructure companies (e.g. power utilities) to patch vulnerabilities" -- "An Approach to Technical AGI Safety and Security" Google DeepMind Team, Apr 2025 https://arxiv.org/html/2504.01849v1#S5 

The argument here is incredibly unconvincing and utterly puzzling. Moses is a mythological figure

These vast sweeping claims you're making are not original thoughts that you've gotten from firsthand sources, but rather they are from 18th and 19th century historians. That is, the narrative of gradual improvement over time in what's called Whiggish history. It's very popular among non-historians or amateur historians but 20th century historians were very critical of this view. Experts in the field, the people who are making a career of "looking at historical documents," have largely flipped on this view.

Herbert Butterfield wrote a famous takedown, The Whig Interpretation of History (1931). P. B. M. Blaas felt the style had already passed by 1914, in his seminal work on historiography, Continuity and Anachronism (1978). His term for your idea that people in the past acted on the concept of survival of the fittest before its conceptualization is called Presentism, a form of anachronism, and it's the biggest stumbling block for understanding the people of the past.

David Cannadine, the Dodge Professor of History at Princeton said, "Whig history was, in short, an extremely biased view of the past: eager to hand out moral judgements, and distorted by teleology, anachronism and present-mindedness."

Frederic William Maitland is widely considered the first of a new breed of historians. The answer to Whiggish history was in fact utilizing more data than ever. For him, that meant actually reading as much of English law as possible and understanding it in its own terms, rather than treating it as a more vague process inevitably leading to the present. Contrary to your claim, the firsthand sources in fact shattered Whiggish pretensions.

I'm trying to very politely tell y'all in this thread that this crap is the Newtonian Physics of history. Sure, Edward Gibbon and other ye olde history is a decent starting point, but if that's all you have you're pretty much out of touch with the field.

Does a bacterium "practice survival of the fittest" in a way that matches the expressly Darwinist ideology of Hitler? Of course not. And neither does a Chimp.

The easiest explanation for high measure of biological minds is simulated minds lacking consciousness.

You can acknowledge critics are deluded or self interested whilst also admitting they have some substantial points - this is more in the vein of using that as a justification to ignore all criticism; even valid criticism.