LessWrong 2.0 Reader

I also talk about a plan to create aligned AI here more here, so go check this comment out too if you want more information:

https://www.lesswrong.com/posts/YyosBAutg4bzScaLu/thoughts-on-ai-is-easy-to-control-by-pope-and-belrose#4yXqCNKmfaHwDSrAZ [LW(p) · GW(p)]

Switch 2020 & 2021. In 2022 it went down three times.

• 2019: site did not go down. See Follow-Up to Petrov Day, 2019 [LW · GW]:
• 2020: site went down. See On Destroying the World [LW · GW].
• 2021: site did not go down. See Petrov Day Retrospective 2021 [LW · GW]
• 2022: site went down three times. See Petrov Day Retrospective 2022 [LW · GW]
• 2023: site did not go down. See Petrov Day Retrospective 2023 [LW · GW]
• 2024: site went down.

A few comments: 

The name of the paper is currently missing... 

Penrose may be a coauthor, but the estimate would really be due to his colleague Stuart Hameroff (who I worked for once), an anesthesiologist who has championed the idea of microtubules as a locus of cognition and consciousness. 

As I said in the other thread, even if microtubules do have a quantum computational role, my own expectation is that they would each only contribute a few "logical qubits" at best, e.g. via phase factors created by quasiparticles propagating around the microtubule cylinder, something which might be topologically protected against room-temperature thermal fluctuations. 

But there are many ideas about their dynamics, so, I am not dogmatic about this scenario. 

Seems like if the scientific community suspected this were true, then at least a few scientists would be trying to develop such a BCI system?

There's no mystery as to whether the scientific community suspects that the Hameroff-Penrose theory is correct. It is a well-known hypothesis in consciousness studies, but it wouldn't have too many avowed adherents beyond its inventors: perhaps some people whose own research program overlaps with it, and some other scattered sympathizers. 

It could be compared to the idea that memories are stored in RNA, another hypothesis that has been around for decades, and which has a pop-culture charisma far beyond its scientific acceptance. 

So it's not a mainstream hypothesis, but it is known enough, and overlaps with a broader world of people working on microtubule physics, biocomputation, quantum biology, and other topics. See what Google Scholar returns for "microtubule biocomputer" and scroll through a few pages. You will find, for example, a group in Germany trying to create artificial microtubule lattices for "network-based biocomputation" (which is about swarms of kinesins walking around the lattice, like mice in a maze looking for the exit), a book on using actin filaments (a relative of the microtubule) for "revolutionary computing systems", and many other varied proposals. 

I don't see anyone specifically trying to hack mouse neurons in order to make novel microtubular deep-learning systems, but that just means you're going to be the godfather of that particular concept (like Petr Vopěnka, whose "principle" started as a joke he didn't believe). 

If you just want to make the tooth surface more negatively charged...a salt of poly(acrylic acid) seems better for that. And I think some toothpastes have that.

Yes, so indexing all generations is absolutely a viable strategy, though like you said, it might be more expensive.
Watermarking by choosing different tokens at a specific temperature might not be as effective (as you touched on), because in order to reverse that, you need the exact input. Even a slight change to the input or the context will shift the probability distribution over the tokens, after all. Which means you can't know if the LLM chose the first or second or third most probable token just by looking at the token.
That being said, something like this can still be used to watermark text: If the LLM has some objective, text-independent criteria for being watermarked (like the "e" before "a" example, or perhaps something more elaborate created using gradient descent), then you can use an LLM's loss function to choose some middle ground between maximizing your independent criteria and minimizing the loss function.
The ideal watermark would put markings into the meaning behind the text, not just the words themselves. No idea how that would happen, but in that way you could watermark an idea, and at that point hacks like "translate to French and back" won't work. Although watermarking the meaning behind text is currently, as far as I know, science fiction.

Maybe breaking up certain biofilms held together by Ca?

Yeah there's a toothpaste on the market called Livfree that claims to work like this.

IIRC, high EDTA concentration was found to cause significant amounts of erosion.

Ok, that sounds bad. Thanks.

ETA: Found an article that explains how Livfree works in more detail:

Tooth surfaces are negatively charged, and so are bacteria; therefore, they should repel each other. However, salivary calcium coats the negative charges on the tooth surface and bacteria, allowing them to get very close (within 10 nm). At this point, van der Waal’s forces (attractive electrostatic forces at small distances) take over, allowing the bacteria to deposit on the tooth surfaces, initiating biofilm formation.10 A unique formulation of EDTA strengthens the negative electronic forces of the tooth, allowing the teeth to repel harmful plaque. This special formulation quickly penetrates through the plaque down to the tooth surface. There, it changes the surface charge back to negative by neutralizing the positively charged calcium ions. This new, stronger negative charge on the tooth surface environment simply allows the plaque and the tooth surface to repel each other. This requires neither an abrasive nor killing the bacteria (Figure 3).

The authors are very positive on this toothpaste, although they don't directly explain why it doesn't cause tooth erosion.

This seems like a fine topic, but FYI I ended up giving it a downvote because I gave up reading part way through, starting skimming, and ironically most of what's in this post turned into information dark matter because I lost faith that I'd gain more from reading it than skimming. I'd have preferred a more condensed post.

Thanks for the great post!

Also it’s California, so there’s some chance this happens, seriously please don’t do it, nothing is so bad that you have to resort to a ballot proposition, choose life

Why are you saying this? In what sense "nothing is so bad"?

The reason why people who have libertarian sensibilities, distrust for government track record in general and specifically for its track record in tech regulation are making exception in this case is the future AI strong potential for catastrophic and existential risks.

So, why people who generally dislike the mechanism and track record of California ballot propositions should not make an exception here as well?

The whole point of all this effort around SB 1047 is that "nothing is so bad" is an incorrect statement.

And especially given that you are correctly saying:

Thus I reiterate the warning: SB 1047 was probably the most well-written, most well-considered and most light touch bill that we were ever going to get. Those who opposed it, and are now embracing the use-case regulatory path as an alternative thinking it will be better for industry and innovation, are going to regret that. If we don’t get back on the compute and frontier model based path, it’s going to get ugly.

There is still time to steer things back in a good direction. In theory, we might even be able to come back with a superior version of the model-based approach, if we all can work together to solve this problem before something far worse fills the void.

But we’ll need to work together, and we’ll need to move fast.

Sure, there is still a bit of time for a normal legislative effort (this time with a close coordination with Newsom, otherwise he will just veto it again), but if you really think that if a normal route fails, the ballot route is still counter-productive, you need to make a much stronger case for that.

Especially given that the ballot measure will probably pass with large margin and flying colors...

That sounds fantastic! Very excited for this.

I'm not actually proposing this alignment plan.

My alignment plan would be:

Step 1: Create reasonably large datasets about human values that encode what humans value in a lot of situations.

Step 2: Use a reward model to curate the dataset and select the best human data to feed to the AI.

Step 3: Put in the curated data before it can start to conceive of deceptive alignment, to bias it towards human friendliness.

Step 4: Repeat until the training data is done.

I also like the direction of using COT interpretability on LLMs.