LessWrong 2.0 Reader

Models could incorporate long-term predictions, ensuring decisions align with future sustainability and impact goals.

Researchers implement mechanisms where AI models autonomously recognize misalignment, shut down harmful behaviors, or even self-destruct (via wiping out network weights) for severe cases.

We could build modular AI systems with distinct sub-components focusing on different objectives (e.g., overall goal, ethical considerations, social implications). These agents could check each other's outputs, flagging potential high-risk conflicts or misalignment.

This feels like you either misunderstood the problem, or you respond by circular logic.

The problem with alignment is that we don't know how to do alignment.

Your proposals:

• do alignment with future sustainability
• recognize misalignment
• recognize other agent's misalignment

I repeat: the problem is that we don't know how to "do alignment" (or "recognize misalignment").

*

As an analogy, imagine that someone tells you "I don't know how to swim", and your advice would be:

• keep your head safely above the water
• don't drown
• when swimming together with other people at the same skill level, check each other that you are not drowning

Well, if I knew how to keep my head safely above the water and how not to drown, I wouldn't be asking the question in the first place.

If I had more time, I would have written a shorter post ;) 

What about biological augmentation of intelligence? I think if other avenues are closed, this one can still go pretty far, enough to make society very stratified and incomprehensible and so on. You can imagine biological self-improving intelligences too.

So maybe if you're serious about closing all avenues, it amounts to creating a god that will forever watch over everything and prevent things from becoming too smart. It doesn't seem like such a good idea anymore.

"cannot imagine a study that would convince me that it "didn't work" for me, in the ways that actually matter. The effects on my mind kick in sharply, scale smoothly with dose, decay right in sync with half-life in the body, and are clearly noticeable not just internally for my mood but externally in my speech patterns, reaction speeds, ability to notice things in my surroundings, short term memory, and facial expressions."

The drug actually working would mean that your life is better after 6 years of taking the drug compared to the counterfactual where you took a placebo.

The observations you describe are explained by you simply having a chemical dependency on a drug that you have been on for 6 years.

Why are they more recoverable? Seems like a human who seized power would seek asi advice on how to cement their power

I mentioned samples and expectations for the TLBP because it seems possible (and suggested by the role of degeneracies in SLT) that different samples can correspond to qualitatively different degradations of the model. Cartoon picture : besides the robust circuit X of interest, there are "fragile" circuits A and B, and most samples at a given loss scale degrade either A or B but not both. 

I agree that there is no strong reason to overindex on the Watanabe temperature, which is derived from an idealised situation: global Bayesian inference, degeneracies exactly at the optimal parameters, "relatively finite variance", etc. The scale you propose seems quite natural but I will let LLC-practitioners comment on that.

I found this really useful, thanks! I especially appreciate details like how much time you spent on slides at first, and how much you do now.

In regard to my situation and why I'm presenting you my ideas, I'm an amateur thinker who is wishing to popularise and spread my ideas and is outside the intellectual community so I'm in need of help in spreading my ideas so if you're feeling generous I'd like to ask for help from any of you reading to spread my ideas.

I think you have skipped a few steps here.

First, you need to have some good ideas. They do not necessarily need to be original; popularizing existing good ideas is also a great thing.

Second, you need to get good at explaining things. Write clearly, provide specific examples, etc.

Then, you can write a few articles and people will be happy to share them.

It seems like you think that you are currently at step three. To me it seems like you are still struggling with step one (or maybe step two). I have no idea what are the ideas you want to spread.

Sure. But in an economy with AIs, humans won't be like Bob. They'll be more like Carl the bottom-percentile employee who struggles to get any job at all. Even in today's economy lots of such people exist, so any theoretical argument saying it can't happen has got to be wrong.

And if the argument is quantitative - say, that the unemployment rate won't get too high - then imagine an economy with 100x more AIs than people, where unemployment is only 1% but all people are unemployed. There's no economic principle saying that can't happen.

(Self-review.) I think this pt. 2 is the second most interesting entry in my Whole Dumb Story memoir sequence. (Pt. 1 [LW · GW] deals with more niche psychology stuff than the philosophical malpractice covered here; pt. 3 [LW · GW] is a more of grab-bag of stuff that happened between April 2019 and January 2021; pt. 4 [LW · GW] is the climax. Expect the denouement pt. 5 in mid-2025.)

I feel a lot more at peace having this out there. (If we can't have justice, sanity, or language, at least I got to tell my story about trying to protect them.)

The 8 karma in 97 votes is kind of funny in how nakedly political it is. (I think it was higher before the post got some negative attention on Twitter.)

Given how much prereading and editing effort had already gone into this, it's disappointing that I didn't get the ending right the first time. (I ended up rewriting some of the paragraphs at the end after initial publication [LW(p) · GW(p)] after it didn't land in the comments section the way I wanted it to land.)

Subsection titles would have also been a better choice for such a long piece (which was rectified for the publication of pt.s 3 and 4); I may still yet add them.