LessWrong 2.0 Reader

Thanks for the great post!

Also it’s California, so there’s some chance this happens, seriously please don’t do it, nothing is so bad that you have to resort to a ballot proposition, choose life

Why are you saying this? In what sense "nothing is so bad"?

The reason why people who have libertarian sensibilities, distrust for government track record in general and specifically for its track record in tech regulation are making exception in this case is the future AI strong potential for catastrophic and existential risks.

So, why people who generally dislike the mechanism and track record of California ballot propositions should not make an exception here as well?

The whole point of all this effort around SB 1047 is that "nothing is so bad" is an incorrect statement.

And especially given that you are correctly saying:

Thus I reiterate the warning: SB 1047 was probably the most well-written, most well-considered and most light touch bill that we were ever going to get. Those who opposed it, and are now embracing the use-case regulatory path as an alternative thinking it will be better for industry and innovation, are going to regret that. If we don’t get back on the compute and frontier model based path, it’s going to get ugly.

There is still time to steer things back in a good direction. In theory, we might even be able to come back with a superior version of the model-based approach, if we all can work together to solve this problem before something far worse fills the void.

But we’ll need to work together, and we’ll need to move fast.

Sure, there is still a bit of time for a normal legislative effort (this time with a close coordination with Newsom, otherwise he will just veto it again), but if you really think that if a normal route fails, the ballot route is still counter-productive, you need to make a much stronger case for that.

Especially given that the ballot measure will probably pass with large margin and flying colors...

I'm not actually proposing this alignment plan.

My alignment plan would be:

Step 1: Create reasonably large datasets about human values that encode what humans value in a lot of situations.

Step 2: Use a reward model to curate the dataset and select the best human data to feed to the AI.

Step 3: Put in the curated data before it can start to conceive of deceptive alignment, to bias it towards human friendliness.

Step 4: Repeat until the training data is done.

I also like the direction of using COT interpretability on LLMs.

IMO, my explanation of why GPT-4 and Claude aren't more useful than they are now (Though I'd argue against Claude/GPT-4 being not that useful, and I'd argue they are already helpful enough to work with as a collaborator, which is why AI use is spiking up quite radically.) is the following 2 things.

1. Reliability issues. I've become of the opinion that it isn't enough to have a capability by itself, and you need an AI that can do something reliably enough to actually use it without you having to look over your shoulder to fix it when it goes wrong.

And to be blunt, GPT-4 and Claude are not reliable enough for most use cases.

2. The issue of not dedicating more to things that let an AI think longer for harder problems like a human. To be clear, o1 is useful progress towards that, and I don't expect this to be too much of a blocker in several years, but this was a reason why they were less useful than people thought.

I gain money in expectation with loans, because I don't expect to have to pay them back. What specific bet are you offering?

Agreed. I basically think the capability ordering we got is probably good for technically solving alignment, but maybe bad for governance/wakeup. Good overall though probably.

I actually no longer fully endorse UDT [LW · GW]. It still seems a better decision theory approach than any other specific approach that I know, but it has a bunch of open problems and I'm not very confident that someone won't eventually find a better approach that replaces it.

To your question, I think if my future self decides to follow (something like) UDT, it won't be because I made a "commitment" to do it, but because my future self wants to follow it, because he thinks it's the right thing to do, according to his best understanding of philosophy and normativity. I'm unsure about this, and the specific objection you have is probably covered under #1 in my list of open questions in the link above.

(And then there's a very different scenario in which UDT gets used in the future, which is that it gets built into AIs, and then they keep using UDT until they decide not to, which if UDT is reflectively consistent would be never. I dis-endorse this even more strongly. [LW · GW])

NetLogo is the easiest entry point. Mesa is gaining support.

This post and the remainder of the sequence were turned into a paper accepted to NeurIPS 2024. Thanks to LTFF for funding the retroactive grant that made the initial work possible, and further grants supporting its development into a published work including new theory and experiments. @Adrià Garriga-alonso [LW · GW] was also very helpful in helping write the paper and interfacing with the review process.

EDTA in toothpaste? It chelates iron and calcium. Binding iron can prevent degradation during storage, so a little bit is often added.

Are you talking about adding a lot more? For what purpose? In situations where you can chelate iron to prevent bacterial growth, you can also just kill bacteria with surfactants. Maybe breaking up certain biofilms held together by Ca? EDTA doesn't seem very effective for that for teeth, but also, chelating agents that could strip Ca from biofilms would also strip Ca from teeth. IIRC, high EDTA concentration was found to cause significant amounts of erosion.

I wouldn't want to eat a lot of EDTA, anyway. Iminodisuccinate seems less likely to have problematic metabolites.

If you have a Substack blog, consider using Datawrapper embeds for inserting tables or charts. For some reason Substack has been somewhat hiding this cool feature as its not visible in the menu and poorly advertised but it lets you create highly customizable charts/tables for your Substack posts - and is also supported on Wordpress or in the form of an HTML embed if you have a self-hosted blog/website. I wrote a brief guideline on how to use it in my blog - and you can see cool Datawrapper embeds in Nate Silver's substack.

Sadly Datawrapper not supported on LessWrong (yet) but LW already has native support for table embeds, so its less relevant.