LessWrong 2.0 Reader

Hi Aksh, I'm looking to reproduce the CAI process in the HuggingFace tutorial myself. Did you publish your code for this? I'm having a hard time getting LLMSwarm to work so I'm eager to see your setup.

Typo in the linked document:

There is no one is coming to save us.

Both. I do not want to have AI content added to my post without my knowledge or consent.

In fact, thinking further about it, I do not want AI content added to anyone's post without their knowledge or consent, anywhere, not just on LessWrong.

Such content could be seen as just automating what people can do anyway with an LLM open in another window. I've no business trying to stop people doing that. However, someone doing that knows what they are doing. If the stuff pops up automatically amidst the author's original words, will they be so aware of its source and grok that the author had nothing to do with it? I do not think that the proposed discreet "AI-generated" label is enough to make it clear that such content is third-party commentary, for which the author carries no responsibility.

But then, who does carry that responsibility? No-one. An AI's words are news from nowhere. No-one's reputation is put on the line by uttering them. For it is written, the fundamental question of rationality is "What do I think I know and how do I think I know it?" But these AI popovers cannot be questioned.

And also, I do not personally want to be running into any writing that AI had a hand in.

(Oh, hey, you're the one who wrote Please do not use AI to write for you [LW · GW])

I am that person, and continue to be.

links 11/01/2024: https://roamresearch.com/#/app/srcpublic/page/11-01-2024

• https://en.m.wikipedia.org/wiki/Neats_and_scruffies a typology of AI researchers
• https://notes.andymatuschak.org/About_these_notes Andy Matuschak's working notes, mostly about educational technology (but not educational games!)
  • https://notes.andymatuschak.org/zUVBJdPc4kBud5fsLmPFpbw
• https://notes.manjarinarayan.org/ Manjari Narayan's notes, mostly about statistics
• https://www.washingtonpost.com/health/2024/05/06/ultrasound-addiction-treatment/ ultrasound being used as an addiction treatment -- the full study results aren't published yet, but the anecdotes suggest very dramatic effects.
• all drugs for neuropathic pain have poor success rates.
  • https://jamanetwork.com/journals/jamaneurology/fullarticle/2769608#google_vignette
  • https://pmc.ncbi.nlm.nih.gov/articles/PMC6452908/
  • https://pmc.ncbi.nlm.nih.gov/articles/PMC10711341/
• https://pubmed.ncbi.nlm.nih.gov/24291734/ lots of people -- maybe 6-10% of the world population -- have neuropathic pain.
• https://pmc.ncbi.nlm.nih.gov/articles/PMC3201926/ chronic pain generally affects about 20% of adults worldwide.
  • https://www.ncbi.nlm.nih.gov/books/NBK553030/
  • https://pmc.ncbi.nlm.nih.gov/articles/PMC3201926/
  • https://pmc.ncbi.nlm.nih.gov/articles/PMC6676152/
• roughly half of opioid addicts treated with buprenorphine or methadone manage to abstain for 30 days after treatment: https://pubmed.ncbi.nlm.nih.gov/26599131/
• https://www.whitehouse.gov/ondcp/briefing-room/2021/05/28/biden-harris-administration-calls-for-historic-levels-of-funding-to-prevent-and-treat-addiction-and-overdose/ the Biden-Harris administration has allocated $41B to preventing and treating drug addiction; hard to extract from that exactly how much is spent on rehab/treatment vs. anti-drug campaigns or law enforcement
  • https://www.forbes.com/sites/danmunro/2015/04/27/inside-the-35-billion-addiction-treatment-industry/ US addiction treatment spending was estimated at $35B/year back in 2015
• Vampire Weekend's Ezra Koenig:
  • their latest album Only God Was Above Us is wrenching and it's kind of getting to me lately.
    • most of the commentary in interviews is about how Koening, now 40 with a 5-year-old kid, has matured and found peace (though if you listen to the lyrics it's an extremely nihilistic sort of being "at peace" with a terrible world and giving up on trying to change it)
    • nobody is remarking on what I see as pretty explicit themes like:
      • last album's "Harmony Hall" was about a sense of betrayal regarding Ivy-League antisemitism
      • this album is pretty clearly a rejection of the backlash, the Gen-X ("Gen X Cops"), ex-Eastern-Bloc ("Pravda"), or specifically Jewish (in the [[Bari Weiss]]/Tablet-mag vein) "vibe shift".
        • there's a lot of reflection on heritage and generation gaps, there's the sense that someone (his elders? his family?) is pushing him in a direction and he doesn't want to go that way, he thinks it doesn't make sense in his generation, in this era, but he does care enough to be conflicted and to yearn over the pain of people still (mistakenly, he thinks) struggling ("Capricorn").
  • https://en.wikipedia.org/wiki/Ezra_Koenig
  • https://people.com/vampire-weekend-ezra-koenig-finally-feels-adult-exclusive-8625179
  • https://www.theguardian.com/us-news/2016/jun/20/bernie-sanders-vampire-weekend-grizzly-bear-endorsements
  • https://www.theguardian.com/music/2024/mar/23/ezra-koenig-vampire-weekend-interview
  • https://www.thejc.com/life-and-culture/music/vampire-weekend-dont-call-us-white-c3xbezac

My own attempt is much less well written and comprehensive, but I think I hit on some points that theirs misses: https://www.lesswrong.com/posts/NRZfxAJztvx2ES5LG/a-path-to-human-autonomy [LW · GW]

Recently, Raginrayguns [LW · GW] and Philosophy Bear both [presumably] read "Cargo Cult Science" [not necessarily for the first time] on /r/slatestarcodex. I follow both of them, so I looked into it. And TIL that's where "cargo-culting" comes from. He doesn't say why it's wrong, he just waves his hands and says it doesn't work and it's silly. Well, now I feel silly. I've been cargo-culting "cargo-culting". I'm a logical decision theorist. Cargo cults work. If they work unreliably, so do reductionistic methods.

My diet is heavily, heavily oatmeal (out of laziness). Like on most days I just eat oatmeal + peanut butter + walnuts microwaved in water until I get home (and I do ~45 minutes of HIIT or strength training every morning so it's not like I have a very low BMR). Maybe I should track my exact diet and bloodwork over time and report it here.

To doublecheck/clarify: do you feel strongly (or, weakly) that you don't want autogenerated jargon to exist on your posts for people who click the "opt into non-author-endorsed AI content" for that post? Or simply that you don't personally want to be running into it?

(Oh, hey, you're the one who wrote Please do not use AI to write for you [LW · GW])

Your central argument seems to be a metaphor: We caused the Holocene extinction of animals, so godlike AI will kill us.

The problem with metaphorical arguments is that they can be reversed. As humans have become more intelligent, we've started to value animals, created zoos, natural reserves and now even work on the resurrection of extinct animals like mammoths. See more examples of such reversal by Gwern https://gwern.net/modus

Presenting weak arguments is evidence that there are no strong arguments, and this is obvious to outside readers.

The main problem is that we can't predict what superintelligent AI will do, and thus we can't 100 percent prove that it will necessarily kill us. But we shouldn't have to.

Instead, we should show that superintelligence will disempower us and that it may want to kill us for some reasons.

I do think there's something to that idea - physical injury and pain is a very universal and visible experience, whereas mental illness is difficult to parse for those who've never experienced it. I also think there's some sense in which 'treatment' and 'cure' are treated differently for mental and physical illness.

A doctor wouldn't just prescribe painkillers for a broken arm and call it a day because your symptoms have been dealt with; they'd want to actually fix the problem. Depression, on the other hand, doctors seem perfectly fine with merely mitigating the symptoms. Perhaps because that's all they're confident they can do?