LessWrong 2.0 Reader

Oliver specifically wanted me to include the word "naive" because obviously there are sensible things people could mean by this but they phrase things overly strongly and the Lightcone Team's Autism is Powerful.

Yes I think your equation looks right.

things are almost never greater than the sum of their parts Because Reductionism

Isn't it more like, the value of the sum of the things is greater than the sum of the value of each of the things? That is, $f(a+b)>f\left(a\right)+f\left(b\right)$ (where perhaps $f$ is a utility function). That seems totally normal and not-at-all at odds with Reductionism.

Also, I don't claim there's another major grant maker that's less constrained like this.)

I think the SFF appears less constrained like this

In theory, possibly, but it's not clear how to save the world given such restricted access. See e.g. https://www.lesswrong.com/posts/NojipcrFFMzNx6Grc/sudo-s-shortform?commentId=onKfTrunn2Q2Gc4Pw [LW(p) · GW(p)]

In practice no, because you can't deal with a superintelligence safely. E.g.

• You can't build a computer system that's robust to auto-exfiltration. I mean, maybe you can, but you're taking on a whole bunch more cost, and also hoping you didn't screw up.
• You can't develop this tech without other people stealing it and running it unsafely.
• You can't develop this tech safely at all, because in order to develop it you have to do a lot more than just get a few outputs, you have to, like, debug your code and stuff.
• And so forth. Mainly and so forth.

I'm not sure that AI boxing is a live debate anymore.  People are lining up to give full web access to current limited-but-unknown-capabilities implementations, and there's not much reason to believe there will be any attempt at constraining the use or reach of more advanced versions.

Here's the text with improved grammar:

I think there is one more level at which natural abstraction can occur: the level just "beneath" consciousness.

For example, we can create an LLM that almost perfectly matches my internal voice dialogue's inputs and outputs. For me – internally – there would be no difference if thoughts appearing in my mind were generated by such an LLM, rather than by real biological neurons or even cortical columns. The same applies to the visual cortex and other brain regions.

Such an LLM for thoughts would be no larger than GPT-4 (as I haven't had that many new ideas). In most cases, I can't feel changes in individual neurons and synapses, but only the high-level output of entire brain regions.

I think we can achieve 99 percent behavioral and internal thought mimicry with this approach, but a question arises: what about qualia? However, this question isn't any easier to answer if we choose a much lower level of abstraction.

If we learn that generating qualia requires performing some special mathematical operation F(Observations), we can add this operation to the thought-LLM's outputs. If we have no idea what F(Observations) is, going to a deeper level of abstraction won't reassure us that we've gone deep enough to capture F(O).

Similar ideas here: https://medium.com/@bablulawrence/cognitive-architectures-and-llm-applications-83d6ba1c46cd

For those interested in Chinese philosophy, I'd suggest 韩非子 (Han Feizi), which offers a thoughtful meta-analysis of earlier philosophers like Laozi and contemporaries such as Xunzi, in so far as their thoughts applied to statecraft. (The first Emperor was a big fan of the work.)

This recommendation assumes some basic knowledge of Chinese history. 

For those new to the subject, [Recommendation to come, I am trying to find the English version for a children's book to Chinese Philosophy and History] might be a better starting point.

My recommendations are based on several key considerations:

• Quality of available translations
• Relevance to Western readers
• Focus on state-building as both an entry point and a way to understand how philosophy shaped Chinese governance
• Historical impact on actual governance

Chinese statebuilding is a very relevant lens because how early the Chinese started to concentrate executive power. The legalist school makes sense because the state was able to codify law staffed with court officers instead of relying on customary law enforced by local notables (thou the latter still happened a lot). And for Confucianism, for almost all of its existence, saw the best way to enact their worldly vision was through influencing the Emperor/King and their imperial/monarchial apparatus; Chinese philosophy is very much Chinese Political Philosophy. (Besides things like Xuanxue, ofc.)

While Mozi and Xunzi are often recommended here, I've found they might not be the most accessible entry points. Mozi, while interesting, had relatively limited historical influence compared to other schools of thought - being more well known to lay Chinese audience by his portrayals in historical dramas than his works. It would be somewhat like introducing Western philosophy through a lesser-known PreSocratic philosopher – potentially interesting but perhaps not the most representative starting point.

I've found that Chinese philosophy becomes more accessible when viewed through the lens of practical governance, as this provides concrete examples of how these ideas were implemented. Governance, being a universal concept, offers familiar ground for readers from any background and which ever cardinal direction. Additionally, many philosophical classics about governance have been reinterpreted over time, making them more approachable for modern readers.

TLDR: Chinese philosophy is firmly attached to Chinese history just like Western philosophy to the history of the Church. Gonna know both and let them bounce off each other.

What's the context? 

A bit of a pushback, if I may: confirmation bias/motivated reasoning themselves only arise because of an inherent, deep-seated, [fairly likely] genetically conditioned, if not unconscious sense that:

A. there is, in fact, a single source of ground truth even, if not especially, outside of regular, axiomatic, bottom-up, abstract, formalized representation: be it math [+] or politics [-]

B. it is, in fact, both viable and desirable, to affiliate yourself with any one/number of groups, whose culture/perspective/approach/outlook must fully represent the A: instead of an arbitrarily small, blind-sided to everything else, part of the underlying portion it is most familiar with itself

C. any single point/choice/decision/conclusion/action reached must, in itself, be inherently sensible enough to hold for an arbitrarily significant period of time, without any revision or consideration of the opposite/orthogonal perspective; this one, in turn, might itself stem from an assumption that:

D. the world must be either a [1] static entity, fully representable with an arbitrarily large set of beliefs, attitudes, and considerations; or a [2] dynamic, yet inherently mechanical, following the exact same static laws/rules/patterns in each and every aspect of itself: be it physics or society; these last ones can be safely assumed to be never-changing and, once "understood", always reinterpreted within the exact same light as in the original interpretation of the time

E. whatever the kind of entity it is, any particular snapshot of the linguistic and/or symbolic representation of it is, at every moment, fully capable of describing it, without coming up short within any single aspect of it: an assumption, if you will, that there no "3x+1 Conjectures" the limitations of our cognitive/representative tools in the present would not be able to figure out

Biology-wise, the B might be strong enough to easily overpower, without any of our conscious awareness, the rest of them. Yet even discounting that: the motivated reasoning and the desire to adhere to whatever stance has been reached already themselves stem, fundamentally, from the sheer human arrogance in regarding whatever was [conceived/perceived/assimilated/concluded] as fully sufficient both for what is in the present, as well for what will be yet, going forward. 

That arrogance, in turn, anchors our cognition; which promptly short-circuits itself into whatever Weltanschauung our general A-E'sque attitude of the day lines up to, in attempt to save energy on rather costly and, given A to E, completely wasteful brain cycles. MR/CB is merely an effect of it all.

P.S. Two upticks from me, regardless. The links were much appreciated. Would gladly hear any of your additional thoughts on the matter in a fully-sized post/article/whatever you call it here.

I sent 1k. The sequences changed my life and although I've never been to Lighthaven, everyone says it's an extraordinary place. Lesswrong is too.