It all hinges on the fact that your assumption that you have to be aware of things for them to matter is unjustified—and, it turns out, very very false.

What do you mean by “matter” here?

Remember, we’re talking about the following situation:

1. Woman attempts to “send cues”, with the intended result being that a certain man will perceive these cues and react in a certain desired way.
2. Man has no idea that this is happening.
3. Man does not react in the desired way to the supposed cues that the woman is supposedly sending (and how could he, not being aware of any such things?).
4. Woman is annoyed, frustrated, etc., that she is not getting what she wants.

Now you’re claiming that, somehow, these cues are nevertheless being “sensed” (ok, sure), and also that they nevertheless “matter” in some way.

What is the meaning of “matter” in this context that makes your claim true?

EDIT: By the way, this is false in my experience:

But it is also true that people will call what I refer to as “sensing the cue” “sensing the cue”.

I’ve never encountered this usage from anyone other than you.

Have you tested this hypothesis on your friends? Ask them for their iron level from last blood test, and ask them to self-report anxiety level (you also make a separate estimate of their anxiety level).

Huh, seems you are correct. They also apparently are heavily cannibalistic, which might be a good impetus for modeling the intentions of other members of your species…

Is interpersonal variation in anxiety levels mostly caused by dietary iron?

I stumbled across this paper yesterday. I haven't looked at it very closely yet, but the high-level pitch is that they look at genetic predictors of iron deficiency and then cross that with anxiety data. It's interesting mainly because it sounds pretty legit (i.e. the language sounds like direct presentation of results without any bullshitting, the p-values are satisfyingly small, there's no branching paths), and the effect sizes are BIG IIUC:

The odd ratios (OR) of anxiety disorders per 1 standard deviation (SD) unit increment in iron status biomarkers were 0.922 (95% confidence interval (CI) 0.862–0.986; p = 0.018) for serum iron level, 0.873 (95% CI 0.790–0.964; p = 0.008) for log-transformed ferritin and 0.917 (95% CI 0.867–0.969; p = 0.002) for transferrin saturation. But no statical significance was found in the association of 1 SD unit increased total iron-binding capacity (TIBC) with anxiety disorders (OR 1.080; 95% CI 0.988–1.180; p = 0.091). The analyses were supported by pleiotropy test which suggested no pleiotropic bias.

Odds ratio of anxiety disorders changes by roughly 0.9 per standard deviation in iron level, across four different measures of iron level. (Note that TIBC, the last of the four iron level measures, didn't hit statistical significance but did have a similar effect size to the other three.)

Just eyeballing those effect sizes... man, it kinda sounds like iron levels are maybe the main game for most anxiety? Am I interpreting that right? Am I missing something here?

EDIT: I read more, and it turns out the wording of the part I quoted was misleading. The number 0.922, for instance, was the odds ratio AT +1 standard deviation serum iron level, not PER +1 standard deviation serum iron level. That would be -0.078 PER standard deviation serum iron level, so it's definitely not the "main game for most anxiety".

I think perhaps you have missed the point I was making, which is that what you call “being aware that you have sensed a cue” is just what everyone else calls “sensing the cue” (perhaps “perceiving the cue” might be a better phrase

I haven't. It's just something you're going to struggle to understand until you recognize the difference -- and the importance of the difference -- between the quotation and its referent. So that's where we have to start.

It is true that many people will call the thing I refer to as "being aware that you have sensed the cue" "sensing the cue", yes. But it is also true that people will call what I refer to as "sensing the cue" "sensing the cue".

It's not that people who refer to “being aware that you have sensed a cue” as “sensing the cue” simply have a different definition of terms, it's that they're failing to track a critical distinction.

It's not possible to understand "female signals" -- or male signals, or your own signals, even -- until you understand the importance of the things people perceive and understand which they aren't aware that they perceive and understand. It all hinges on the fact that your assumption that you have to be aware of things for them to matter is unjustified -- and, it turns out, very very false.

I'm happy to explain where your billboard analogy goes wrong, but we really gotta nail this down first.

My current guess for least worst path of ASI development that's not crazy unrealistic:

open source development + complete surveillance of all citizens and all elites (everyone's cameras broadcast to the public) + two tier voting.

Two tier voting:

• countries's govts vote or otherwise agree at global level on a daily basis what the rate of AI progress should be and which types of AI usage are allowed. (This rate can be zero.)
• All democratic countries use daily internet voting (liquid democracy) to decide what stance to represent at the global level. All other countries can use whatever internal method they prefer, to decide their stance at the global level.
• (All ASI labs are assumed to be property of their respective national govts. An ASI lab misbehaving is its govt's responsibility.) Any country whose ASI labs refuse to accept results of global vote and accelerate faster risks war (including nuclear war or war using hypothetical future weapons). Any country whose ASI labs refuse to broadcast themselves on live video risks war. Any country's govt that refuses to let their citizens broadcast live video risks war. Any country whose citizens mostly refuse to broadcast themselves on live video risks war. The exact thresholds for how much violation leads to how much escalation of war, may ultimately depend on how powerful the AI is. The more powerful the AI is (especially for offence not defence), the more quickly other countries must be willing to escalate to nuclear war in response to a violation.

Open source development

• All people working at ASI labs are livestream broadcast to public 24x7x365. Any AI advances made must be immediately proliferated to every single person on Earth who can afford a computer. Some citizens will be able to spend more on inference than others, but everyone should have the AI weights on their personal computer.
• This means bioweapons, nanotech weapons and any other weapons invented by the AI are also immediately proliferated to everyone on Earth. So this setup necessarily has to be paired with complete surveillance of everyone. People will all broadcast their cameras in public. Anyone who refuses can be arrested or killed via legal or extra-legal means.
• Since everyone knows all AI advances will be proliferated immediately, they will also use this knowledge to vote on what the global rate of progress should be.

There are plenty of ways this plan can fail and I haven't thought through all of them. But this is my current guess.

Historically, the social contract describes the set of agreements concerning the legitimacy of government and its role in governing citizens. This concept, developed by philosophers like Hobbes, Locke, and Rousseau, posits that individuals surrender certain natural freedoms and contribute a portion of their wealth to governments in exchange for protection, order, and social stability.

Over time, this foundational concept evolved beyond the basic relationship between citizen and state. The industrial age expanded the social contract to encompass economic relationships between workers, employers, and broader society. Citizens came to expect not just security, but also economic opportunity. Governments increasingly took on responsibilities for education, infrastructure, and basic welfare as part of their obligation under this implicit agreement.

This evolution produced the modern social contract we recognize today: citizens contribute their labor and a portion of their earnings through taxation; in return, they receive not just protection but also economic security and the promise that hard work would be rewarded with prosperity. 

I am not sure how this social change is discontinuous with previous developments which introduced new social conditions, new capabilities, and new externalities. In short, it is clear that if there is big economic change, there will be political changes too, but if this is rethinking the social contract, then we have been doing this continuosly. We do not need to begin to rethink the social contract. We need to recognize that we have always been continually rethinking it. 

Well shoot. I'll work on it, thank you!

Hm, I would say the vibes level is the exact level that this is most effective, rather than any particular method. The basic reason being that LLMs tend to reflect behaviour as they generate from a probability distribution of "likely" outcomes for a given input. Having the "vibes of human-child-rearing" would then result in more outcomes that align with that direction as a result. It's definitely hand wavey so I'm working on more rigerous mathematical formalisms, but the bones are there. I don't nessecarily think feeding an LLM data like we would a child is useful, but I do think that the "vibe" of doing so will be useful. (This is indeed directly related ot the argument that every time we say "AI will kill us all" it makes it x% more likely)

I'd give humans a middling score on that if you look at the state of the world, we are doing pretty well with extreme events like MAD, but on the more minor scale things have been pretty messed up. A good trajectory though, compared to where things were and the relative power we had available. I think a big part of this, that you have helped clarify for me, is that I think it's important that we socialize LLM-based intelligences like humans if we want an outcome that isn't completely alien in it's choices. 

Well that's a bit of the point of the essay isn't it? You have a memetic/homeostatic boundary condition that strongly prefers/incentivizes assuming human adults are alike enough to you that their opinion matters. Even in that statement I can differ, I think childrens perspectives are incredibly important to respect, in some ways more important than an adults because children have an unfiltered honesty to their speech that most adults lack. Although I do delineate heavily between respecting and acting upon/trusting. 

For LLMs I think this is just a new sort of heuristic we are developing, where we have to reckon with the opposite of the animal problem. Animals and plants are harder for us to discern pain/suffering from, but we are more confident when we identify it that they experience it (at least in modern times, many traditions treated animal suffering as essentially fake). Now we have the opposite, creatures that are very easy to interpret but we don't know if they actually have the capacity to feel these things (although we can identify feature activations etc.). So my argument is more that we should be building technology in a way that memetically aligns with the golden rule, because running a society based on something communicating suffering (even if it can't) is going to result in a worse human society regardless. (The counter point being that playing video games where you kill NPCs doesn't make school shooters, but I'm less concerned about those NPCs gaining econmic/social power and patterning off of resentment for having the pattern of being tortured in their memory).

That's some careful analysis!

Two remarks:

1

"Can" is the opposite of "unable". "Unable" means that the change involves granting ability to they who would act, i.e. teaching a technique, providing a tool, fixing the body, or altering the environment.

That's a good characterization, though arguably not a definition, as it relies on "ability", which is circular. I can do something = I have the ability to do something. I can = I'm able to.

But we can use the initial principle (it really needs a name) which doesn't mention ability:

You do a thing iff you can do it and you want to do it.

"Iff" behaves similar to an equation, so we can solve for "can", similar to algebra in arithmetic. I don't know the exact algebra of "iff", but solving for "can" arguably yields:

"I can do X" iff "If I wanted to do X, I would do X"

Which uses wanting and a counterfactual to define "can". We could also define:

"I want to do X" iff "If I could to do X, I would do X"

Though those two definitions together are circular. Maybe it is better to regard one concept as more basic than the other, and only define the less basic one in terms of the more basic one. It seems to me that "want" is more basic than "can", so I would only define "can" in terms of "want", and leave the definition for "want" open (for now).

2

Regarding the Confusing Cases. There are at least two canonical classes: akrasia (weakness of will) and addiction. "Can" the addict quit smoking? "Can" the person suffering from akrasia just do The Thing?

A better question perhaps: In which sense is the answer to the above "yes" and in which sense is it "no"?

One possibility is to analyze these cases in terms of first-order vs second-order desires. The first-order desire would be smoking or being lazy, the second order desire would be not to have the (first-order) desire to smoke, or be lazy. Second-order desires seem to be more important or rational than the first order desires. The second-order and first-order desire are "inconsistent" here. If the first-order desire to smoke is stronger than the second-order desire not to have the first-order desire, I don't quit smoking. (Or don't overcome my laziness in case of akrasia). Otherwise I do.

Here, the "can" definition as "If I wanted to do X, I would do X" is ambiguous. If it means "If I had the first-order desire to quit smoking, I would quit smoking", then the sentence is true, and I "can" overcome the addiction (or the akrasia). If it means "If I had a second-order desire to not have the first order desire to smoke, I would quit smoking", then the sentence is false, as I do in fact have the second-order desire but still don't quit. So in this sense it's not true that I "can" quit.

A similar but different different analysis wouldn't phrase it in terms of first and second-order desires, but in terms of rational wishes and a-rational urges. I wish to quit smoking, but I have the urge to smoke. I wish to get to work, but I have the urge to be lazy. If we count, in the definition of "can", urges as "wanting", I can stop smoking, if we count "wishes" as "wanting", I can't. By a similar argument as above.

I think these cases are actually not a major problem of the "can" definition. After all, it seems in fact ambiguous to ask whether someone "can" overcome some case of akrasia or addiction. The definition captures that ambiguity.