I think you could get a lot out of adding a temporary golden dollar sign with amount donated next to our LW names! Upon proof of donation receipt or whatever.

Seems like the lowest hanging fruit for monetizing vanity— benches being usually somewhat of a last resort!

(The benches seem still underpriced to me, given expected amount raised and average donation size in the foreseeable future).

A sketch I'm thinking of: asking people to consume information (a question, in this case) is asking them to do you a favor, so you should do your best to ease this burden, however, also don't be paralyzed so budget some leeway to be less than maximally considerate in this way when you really need to.

what's the best essay on asking for advice?

Going over etiquette and the social contract, perhaps if it's software specific it talks about minimal reproducers, whatever else the author thinks is involved.

Mod note: I'm frontpaging this despite it being the sort of thing we normally wouldn't frontpage (i.e. non-timeless community insider stuff) because it seemed particularly important.

(I will aim to un-frontpage it in a few days so it doesn't show up in the timeless recommendation engine)

I don't think it's necessarily GDPR-related but the names Brian Hood and Jonathan Turley make sense from a legal liability perspective. According to info via ArsTechnica,

Why these names?

We first discovered that ChatGPT choked on the name "Brian Hood" in mid-2023 while writing about his defamation lawsuit. In that lawsuit, the Australian mayor threatened to sue OpenAI after discovering ChatGPT falsely claimed he had been imprisoned for bribery when, in fact, he was a whistleblower who had exposed corporate misconduct.

The case was ultimately resolved in April 2023 when OpenAI agreed to filter out the false statements within Hood's 28-day ultimatum. That is possibly when the first ChatGPT hard-coded name filter appeared.

As for Jonathan Turley, a George Washington University Law School professor and Fox News contributor, 404 Media notes that he wrote about ChatGPT's earlier mishandling of his name in April 2023. The model had fabricated false claims about him, including a non-existent sexual harassment scandal that cited a Washington Post article that never existed. Turley told 404 Media he has not filed lawsuits against OpenAI and said the company never contacted him about the issue.

Interestingly, Jonathan Zittrain is on record saying the Right to be Forgotten is a "bad solution to a real problem" because "the incentives are clearly lopsided [towards removal]".

User throwayian on Hacker News ponders an interesting abuse of this sort of censorship:

I wonder if you could change your name to “April May” and submitted CCPA/GDPR what the result would be..

hi fellow people-who-i-think-have-much-of-the-plot

you two seem, from my perspective as having read a fair amount of content from both, to have a bunch of similar models and goals, but quite different strategies.

on top of both having a firm grip on the core x-risk arguments, you both call out similar dynamics in capabilities orgs capturing will to save the world and turning it into more capabilities progress^[1], you both take issue with somewhat different but i think related parts of openphil's grantmaking process, you both have high p(doom) and not very comfortable timelines, etc.

i suspect if connor explained why he was focusing on the things he is here, that would uncover the relevant difference. my current guess is connor is doing a kind of political alliancebuilding which is colliding with some of habryka's highly active integrity reflexes.

maybe this doesn't change much, these strategies do seem at least somewhat collision-y as implemented so far, but i hope our kind can get along.

1. ^{^}

   e.g. "Turning care into acceleration" from https://www.thecompendium.ai/the-ai-race#these-ideologies-shape-the-playing-field
   
   e.g. https://www.lesswrong.com/posts/h4wXMXneTPDEjJ7nv/a-rocket-interpretability-analogy?commentId=md7QvniMyx3vYqeyD [LW(p) · GW(p)] and lots of calling out Anthropic

Rumors are that 2025 lighthaven is jam packed. If this is the case, and you need money, rudimentary economics suggests only the obvious: raise prices. I know many clients are mission aligned, and there's a reasonable ideological reason to run the joint at or below cost, but I think it's aligned with that spirit if profits from the campus fund the website.

I also want to say in print what I said in person a year ago: you can ask me to do chores on campus to save money, it'd be within my hufflepuff budget. There are good reasons to not go totally "by and for the community" DIY like many say community libraries or soup kitchens, but nudging a little in that direction seems right.

EDIT: I did a mostly symbolic $200 right now, may or may not do more as I do some more calculations and find out my salary at my new job

Why aren't the less wrong books available on amazon anymore, even as print on demand. Wouldn't that be additional revenue?

At first I thought oh no, Connor is planning to Start Over and Do It Right. And people will follow him because Connor is awesome. And then we're more likely to all die, because there isn't time to start over and nobody has a plan to stop progress toward ASI on the current route.

Then I saw the section on Voyager. Great, I thought; Connor is going to make a better way to create language model agents with legible and faithful chains of thought (in structured code). Those seem like our best route to survival, since they're as alignable as we're going to get, and near the default trajectory. Conjecture's previous attempt to make LMA COEMs seemed like a good idea. Hopefully everyone else is hitting walls just as fast, but an improved technique can still beat training for complex thought in ways that obscure the chain of thought.

Then I saw implications that those steps are going to take a lot of time, and the note that of course we die if we go straight to AGI. Oh dang, back to thought #1: Connor will be building perfect AGI while somebody else rushes to get there first, and a bunch of other capable, motivated people are going to follow him instead of trying to align the AGI we are very likely going to get.

The narrow path you reference is narrow indeed. I have not heard a single plan for pause that comes to grips with how difficult it would be to enforce internationally, and the consequences of Russia and China pushing for AGI while we pause. It might be our best chance, but we haven't thought it through.

So I for one really wish Connor and Conjecture would put their considerable talents toward what seems to me like a far better "out:" the possibility that founation model-based AGI can be aligned well enough to follow instructions even without perfect alignment or a perfectly faithful chain of thought.

I realize you currently think this isn't likely to work, but I can't find a single place where this discussion is carried to its conclusion. It really looks like we simply don't know yet. All of the discussions break down into appeals to intuition and frustration with those from the "opposing camp" (of the nicest, most rational sort when they're on LW). And how likely would it have to be to beat the slim chance we can pause ASI development for long enough?

This is obviously a longer discussion, but I'll make just one brief point about why that might be more likely than many assume. You appear to be assuming (I'm sure with a good bit of logic behind it) that we need our AGI to be highly aligned for success - that if network foundation models do weird things sometimes, that will be our doom.

Making a network into real AGI that's reflective, agentic, and learns continuously introduces some new problems. But it also introduces a push toward coherence. Good humans can have nasty thoughts and not act on them or be corrupted by them. A coherent entity might need to be only 51% aligned, not the 99.99% you're shooting for. Particularly if that alignment is strictly toward following instructions, so there's corrigibility and a human in the loop.

Some of this is in Internal independent review for language model agent alignment [AF · GW] and Instruction-following AGI is easier and more likely than value aligned AGI [LW · GW], but I haven't made that coherence point clearly. I think it's another Crux of disagreement on alignment difficulty [LW(p) · GW(p)] that I missed in that writup - and one that hasn't been resolved.

Edit: it seems like a strategy could split the difference by doing what you're describing, but accelerating much faster if you thought agent coherence could take care of some alignment slop.

I for one don't want to die while sticking to principles and saying I told you so when we're close to doom; I want to take our best odds of survival - which seems to include really clarifying which problems we need to solve.

I think there might be a lesswrong editor feature that allows you to edit a post in such a way that the previous version is still accessible. Here’s an example [LW · GW]—there’s a little icon next to the author name that says “This post has major past revisions…”. Does anyone know where that option is? I can’t find it in the editor UI. (Or maybe it was removed? Or it’s only available to mods?) Thanks in advance!