LessWrong 2.0 Reader

I think I am not understanding the question this equation is supposed to be answer, as it seems wrong to me.

I think you are considering the case were we draw arrowheads on the lines? So each line is either an "input" or an "output", and we randomly connect inputs only to outputs, never connecting two inputs together or two outputs? With those assumptions I think the probability of only one loop on a shape with N inputs and N outputs (for a total of 2N "puts") is  1/N.

The equation I had ( (N-2)!! / (N-1)!!) is for N "points", which are not pre-assigned into inputs and outputs.

These diagrams explain my logic. On the top row is the "N puts" problem. First panel on the left, we pick a unmatched end (doesn't matter which, by symmetry), the one we picked is the red circle, and we look at the options of what to tie it to, the purple circles. One purple circle is filled with yellow, if we pick that one then we will end up with more than one loop. The probability of picking it randomly is 1/7 (as their are 6 other options). In the next panel we assume we didn't die. By symmetry again it doesn't matter which of the others we connected to, so I just picked the next clockwise. We will follow the loop around. We are now looking to match the newly-red point to another purple. Now their are 5 purples, the yellow is again a "dead end", ensuring more than one loop. We have a 1/5 chance of picking it at random. Continuing like this, we eventually find that the probability of having only one loop is just the probability of not picking badly at any step, (6/7)x(4/5)x(2/3) = (N-2)!! / (N-1)!!.

In the second row I do the same thing for the case where the lines have arrows, instead of 8 ports we have 4 input ports and 4 output ports, and inputs can only be linked to outputs. This changes things, because now each time we make a connection we only reduce the number of options by one at the next step. (Because our new input was never an option as an output). The one-loop chance here comes out as (3/4)x(2/3)x(1/2) = (N-1)! / N! = 1/N. Neither expression seems to match the equations you shared, so either I have gone wrong with my methods or you are answering a different question.

I think there's a difference though between propaganda and the mix of selection effects that decides what gets attention in profit driven mass media news. Actual intentional propaganda efforts exist. But in general what makes news frustrating is the latter, which is a more organic and less centralised effort.

Thanks I enjoyed this.

The main thing that seems wrong to me, similar to some of your other recent posts, is that AI progress seems to mysteriously decelerate around 2030. I predict that things will look much more sci-fi after that point than in your story (if we're still alive).

It has LLM written all over it. For example:

This attitude betrays a misunderstanding of cognitive privilege. Just as a person born into wealth has a head start in life, a person born with high cognitive ability begins the race miles ahead of others. Yet, many in rationalist communities resist this conclusion, likely because it challenges the notion of a purely meritocratic intellect.

"Yet, many in rationalist communities resist this conclusion" — Who? Where? I have never seen anything that fits this. It just comes out of nowhere. And it isn't a "conclusion", it's the observation the article starts from.

"likely because it challenges the notion" — More confabulated speculation.

"of a purely meritocratic intellect" — A what? What is a "meritocratic intellect"? How does cognitive privilege "challenge" this notion?

The implicit assumption that anyone could reason as we do if they simply tried harder.

Never seen this one either. The very opposite has been notably written by Eliezer [LW · GW]. It is commonplace on Lesswrong that while we may to some extent improve our thinking, we are nevertheless cognitively unequal by magnitudes that we know of no way to surmount.

Questions for Reflection

Did the writer prime the LLM with DEI training manuals? Go through it replacing cognitive inequality by race, gender, or income inequality and it would be typical of the genre. In fact, that suggests an alternative hypothesis for the genesis of this article: that the author made just such a translation in the opposite direction.

LessWrong and similar communities value rationality, yet rationalists often overestimate the role of effort and underestimate the role of luck in intellectual ability.

More confabulation.

As AI reshapes our world, it’s time to

Typical LLM tic.

It's all like this. It's a castle in the air, whose nominal author has made no effort to put foundations under it. There is one actual fact in the article, that we have unequal mental abilities. The rest is fog and applause lights.

And speaking of applause lights, while LLM undoubtedly had a hand in writing this article, it is the faults in the thinking and writing that damn it. LLM was merely the tool that facilitated it. People have always been capable of writing such things unaided, parodied by Eliezer [LW · GW]:

I am tempted to give a talk sometime that consists of nothing but applause lights, and see how long it takes for the audience to start laughing:

I am here to propose to you today that we need to balance the risks and opportunities of advanced artificial intelligence. We should avoid the risks and, insofar as it is possible, realize the opportunities. We should not needlessly confront entirely unnecessary dangers. To achieve these goals, we must plan wisely and rationally. We should not act in fear and panic, or give in to technophobia; but neither should we act in blind enthusiasm. We should respect the interests of all parties with a stake in the Singularity. We must try to ensure that the benefits of advanced technologies accrue to as many individuals as possible, rather than being restricted to a few. We must try to avoid, as much as possible, violent conflicts using these technologies; and we must prevent massive destructive capability from falling into the hands of individuals. We should think through these issues before, not after, it is too late to do anything about them . . .

I think the frame of "trying to 'solve the whole' future is aking to gripping too hard" might be relevant for me changing my mind about research directions. But it still doesn't present a positive vision for why one should work on e.g. incremental prosaic methods, so then it's even less clear for me what areas to focus. I had been focusing on "actually solving the problm" in the more safety at all scales or permanent safety versions of the term, but I think even those directions might need to be minimally developed in the minimal superintelligence-assisted future? The article gives some clarity of vision to identifying which research directions might do meaningful "solving", but still not uniquely picking out anything. 

Sorry for moderately ramble-y comment

I have another question: It seems to me that philosophy of mind is valuable for ethical reasons because it attempts to figure out which things have minds that can experience enjoyment and suffering, which has implications for how we should act. Do you disagree?

This article feels like arguing against a statement that was probably never made on Less Wrong.

I even think I remember Yudkowsky saying that individual differences in IQ are unfair, and that in the glorious transhuman future of course everyone should get at least IQ 200, or something like that.

The implicit assumption that anyone could reason as we do if they simply tried harder.

Frustration or dismissal when others fail to grasp concepts we find intuitive.

For me, the frustrating thing is that many of those people who have the sufficiently high intelligence still choose to be irrational. There is a book "What Intelligence Tests Miss: The Psychology of Rational Thought" by Keith Stanovich that used to be popular here, and it is precisely about how intelligence isn't rationality.

Personalized AI tutors could help those with lower cognitive capabilities catch up, but access to these technologies is uneven. If high-quality AI education tools remain expensive or exclusive, the divide will only widen.

Ironically, this part seems like you making the very mistake that you are accusing rationalists of. Suppose that perfect personalized AI tutors are available to everyone, for free. What happens? I think the divide will widen anyway, simply because the more intelligent people will benefit more from the AI tutors.

This is a mistake people frequently make when discussing education, even if we completely ignore the AI or computers in general. Yeah, education sucks for everyone, both for the smart, the average, and the stupid, in a different way for everyone. It could be made much better, and everyone could benefit from that. However, that alone does not imply that optimal education for everyone would make the differences disappear. If I simplify it a lot, if the improved education helped everyone achieve twice as much as they can achieve now, it would be better for everyone, and yet the differences would now be twice as big, not smaller. You would need to argue that there are ways to dramatically improve the education of the stupid, but no comparable ways to dramatically improve the education of the smart, which goes against the usual experience of smart kids being bored at school most of the time.

Recognizing your cognitive privilege is the first step toward engaging with others more fairly and constructively.

I agree, but in the current (or maybe recent) political climate, acknowledging that some people are more intelligent than others would get you in deep trouble.

Intelligence should not be the sole determinant of value or opportunity. If it is largely unearned, then structuring society around cognitive hierarchies is deeply unjust.

Here, the tricky thing is how to decouple "good life" from "making decisions". Capitalism kinda conflates those two things: if you have more money, you can live a more convenient life, and you can also decide the use of more resources. This already assumes some cognitive capabilities -- merely throwing more money at stupid people doesn't necessarily improve their lives; many of them get scammed, spend the money on drugs, etc. Democracy is also built on the assumption that letting stupid people vote creates better outcomes for the stupid people, which again is not necessarily true.

So the problem is how to create some kind of kindergarten environment for the stupid, where they can live as well as possible, but won't be expected to manage the environment. It is a historical experience that not letting people make the decisions often leads to abuse. The problem is that without cognitive abilities, letting people make the decisions also often leads to abuse... so, I guess, it is not obvious how the society should be structured.

What implicit assumptions do you make about others based on your own cognitive experiences?

I don't anymore. I have advanced towards the state of silent despair.

How should the recognition of cognitive privilege shape discussions on AI ethics and policy?

Mostly by realizing that the machines of the future will probably be smarter than all of us? Which again is not a controversial thought in this community. The original plan was to build a Friendly AI, and that still seems to me like a good outcome to strive for, even if I have no idea how to contribute to it.

Fantastic! Finally my paper about "Feedom under Naturalistic Dualism" was accepted in Journal of Neurophilosophy and I wrote this post at EA Forum that you can find interesting. I hope it will be included in the training set too:

https://forum.effectivealtruism.org/posts/5zbmEPdB2wqhyFWdW/naturalistic-dualism [EA · GW]

Probably because he thinks there's a lower chance of it killing everyone if he makes it. And that if it doesn't kill everyone then he'll do a better job managing it than the other lab heads.

This is the belief of basically everyone running a major AGI lab. Obviously all but one of them must be mistaken, but it's natural that they would all share the same delusion.

Fascinating. You caused me to google around and realize "bioshelter" was a sort of an academic trademark for specific people's research proposals from the 1900s.

It doesn't appear to be a closed system, like biosphere2 aspired to be from 1987 to 1991.

The hard part, from my perspective, isn't "growing food with few inputs and little effort through clever designs" (which seems to be what the bioshelter thing is focued on?) but rather "thoroughly avoiding contamination by whatever bioweapons an evil AGI can cook up and try to spread into your safe zone".