LessWrong 2.0 Reader

One thing that seems strangely missing from this discussion is that alignment is in fact, a VERY important CAPABILITY that makes it very much better. But the current discussion of alignment in the general sphere acts like 'alignment' is aligning the AI with the obviously very leftist companies that make it rather than with the user!

Agree with this—we do discuss this very idea at length here [LW · GW] and also reference it throughout the piece.

That alignment is to the left is one of just two things you have to overcome in making conservatives willing to listen. (The other is obviously the level of danger.)

I think this is a good distillation of the key bottlenecks and seems helpful for anyone interacting with lawmakers to keep in mind.

If the untrusted model has a private and un-audited chain of thought (like o1), and that chain of thought is preserved between rounds, the untrusted model may have a very good idea of what its original command was before the auditor replaced the command.

Your "the untrusted model is stateless between queries and only sees the command history and system state" does cover this, but wanted to call out that that assumption might not hold if current architectures are indicative of the future direction of development.

links 11/18/2024: https://roamresearch.com/#/app/srcpublic/page/11-18-2024

• [[links]]
  • https://en.m.wikipedia.org/wiki/Nicholas_Roerich  I like his art; he seems to have led an interesting life
  • https://en.m.wikipedia.org/wiki/Maurice_Quentin_de_La_Tour Rococo portraits
  •  
  • https://www.zach.be/p/yc-is-wrong-about-llms-for-chip-design
    • this doesn't seem to be a coherent argument -- author claims both that LLMs will indeed be used by chip design companies to save on human labor, and that they're not very good and can only replace the more routine parts of a chip designer's work. But...this isn't "YC is wrong, there is not a startup opportunity in using LLMs for chip design!" even if it's true it's just a more measured and realistic picture of what LLMs will be doing in chip design!
      • and they even make sure to shill their own AI-for-chip-hardware startup, which they claim is working on challenges that nobody else is: https://www.normalcomputing.com/
  • https://www.programmablemutter.com/p/how-chaotic-is-trump-ii-going-to speculation about Donald Trump's foreign policy
  • https://en.m.wikipedia.org/wiki/Daniel_Defoe  i want to read his nonfiction
  • https://en.m.wikipedia.org/wiki/Kotex sanitary pads have been around since 1920
  • https://asteriskmag.com/issues/08/the-death-and-life-of-prediction-markets-at-google  the tl;dr: Google's first internal prediction market was intended to go public and stalled as the regulatory environment remained unfriendly; its successor focused on predictions about competitor activity rather than Google's own activity (which is less subject to "office politics" considerations blocking an honest assessment of Google's chances of success etc) and never intended to be public-facing, and is still in active use.
  • https://en.m.wikipedia.org/wiki/Fran%C3%A7oise_d%27Aubign%C3%A9,_Marquise_de_Maintenon
  • https://en.m.wikipedia.org/wiki/Madame_de_Pompadour  patroness of the French Enlightenment.
    • "who was that woman who was an influential mistress to the King of France, politically savvy and queen in all but name"? "which one? this just keeps happening"
  • https://nintil.com/dont-assume this is Jose Luis Ricon life advice, which unsurprisingly tells you to be less afraid of things and talk to people more.
    • shan't.
  • https://en.m.wikipedia.org/wiki/Bill_Monroe
  • https://calisphere.org/item/0bc1137c37161874ded76c71bf982409/  scary guys
  •  

It seems to me like the "more careful philosophy" part presupposes a) that decision-makers use philosophy to guide their decision-making, b) that decision-makers can distinguish more careful philosophy from less careful philosophy, and c) that doing this successfully would result in the correct (LW-style) philosophy winning out. I'm very skeptical of all three.

Counterexample to a): almost no billionaire philanthropy uses philosophy to guide decision-making.

Counterexample to b): it is a hard problem to identify expertise in domains you're not an expert in.

Counterexample to c): from what I understand, in 2014, most of academia did not share EY's and Bostrom's views.

My intuition is to get less excited by single projects (a Double Crux bot) until someone has brought them all together & created momentum behind some kind of "big" agglomeration of people + resources in the "neutrality tools" space. 

I didn't know about all the existing projects and I appreciate the resource! Concrete >> vague in my book, I just didn't actually know much about concrete examples.

Cool. Is this right? For something with a 1/n chance of success I can have a 95% chance of success by making 3n attempts, for large values of n. About what does "large" mean here?

A cool way to measure dishonesty: How many people claim to have completed an impossible five minute task.

This has since been community noted, fairly from my understanding.  

This graph is not about how many people reported completing a task in 5 minutes when that was not true, this graph shows how many people completed the whole task even though it took them more than 5 minutes (which was all the time they were getting paid for).  
 

Yep, Seth has really clearly outlined the strategy and now I can see what I missed on the first reading. Thanks to both of you!

How would you compare your ideas here to Asimov's fictional science of psychohistory? I ask because while reading this post I kept getting flashbacks to Foundation.