LessWrong 2.0 Reader

Matt Levine, in response to:

If you lie to board members about other board members in an attempt to gain control over the board, I assert that the board should fire you, pretty much no matter what

 writes:

No! Wrong! Not no matter what! In a normal company with good governance, absolutely. Lying to the board is the main bad thing that the CEO can do, from a certain perspective. But there are definitely some companies — Elon Musk runs like eight of them, but also OpenAI — where, if you lie to board members about other board members in an attempt to gain control over the board, the board members you lie about should probably say “I’m sure that deep down this is our fault, we’re sorry we made you lie about us, we’ll see ourselves out.”

To be clear, I am very sympathetic to the OpenAI board’s confusion. This was not a simple dumb mistake. They did not think “we are the normal board of a normal public company, and we have to supervise our CEO to make sure that he pursues shareholder value effectively.” This was a much weirder and more reasonable mistake. They thought “we are the board of a nonprofit set up to pursue the difficult and risky mission of achieving artificial general intelligence for the benefit of humanity, and we have to supervise our CEO to make sure he does that.” Lying to the board seems quite bad as a matter of, you know, AI misalignment. 

Actually, I haven't seen this article! Thank you very much; it seems very interesting, as do the references cited therein. However, I suppose the distribution from which "filler tokens" (or extra tokens) are drawn might matter, as well as their sequences (that is not just "…", "abcd", or "<pause>", but something more sophisticated might be more useful for a model). It would be very interesting to determine which "filler sequences" are most suitable for hiding computations for specific tasks (this is one of the directions we are working on) and which circuits are responsible for it (if they exist).

Yep I did not cover those here. They are essentially shortcodes for identifying objects/times/locations from context. Related quote:

E.g. "the laptop" can refer to different objects in different contexts, but when used it's usually clear which object is meant. However, how objects get identified does not concern us here - we simply assume that we know names for all objects and use them directly.

("The laptop" is pretty similar to "This laptop".)

(Though "this" can also act as complementizer, as in "This is why I didn't come", though I think in that function it doesn't count as indexical. The section related to complementizers is the "statement connectives" section.)

On a first reading of this summary, I take it as a small positive update on the quality of near-term AI Safety thinking at DeepMind. It treads familiar ground and doesn't raise any major red flags for me.

The two things I would most like to know are:

1. Will DeepMind commit to halting frontier AGI research if they cannot provide a robust safety case for systems that are estimated to have a non-trivial chance of significantly harming humanity?
2. Does the safety team have veto power on development or deployment of systems they deem to be unsafe?

A "yes" to both would be a very positive surprise to me, and would lead me to additionally ask DeepMind to publicly support a global treaty that implements these bare-minimum policies in a way that can be verified and enforced.

A "no" to either would mean this work falls under milling behavior, and will not meaningfully contribute toward keeping humanity safe from DeepMind's own actions.

OK advances in teaching the highest level of physics/math needed for string theory is a big IF. Do you have evidence that is actually happening? I know of two people who tried to learn it and just found it too hard, don't think a better teacher or materials would have helped. The evidence is mixed  but personal accounts certainly suggest that only a very small number of people could get to such a high  level and improved teaching probably wouldn't help much. When we are talking about such extreme skills, people have their plateau or maximum ability level which is mostly fixed.

The human population growing just pushes us along the asymptote faster, rather than changing it. 

To me the data shows that there has been no reliable increase in intelligence in the last ~30 years https://news.northwestern.edu/stories/2023/03/americans-iq-scores-are-lower-in-some-areas-higher-in-one/  Once again it needs to be at the very top level to matter.

Any specialization advantage is already tapped out with string theory and similar. My worldview definitely does not apply to advances in a field like biology as there is lots of experimental data, the tools are improving etc. I would expect advances there without any obvious plateau yet.

Thanks for the link!  It's important to distinguish here between:

(1) support for the movement, 
(2) support for the cause, and
(3) active support for the movement (i.e. attracting other activists to show up at future demonstrations)

Most of the paper focuses on 1, and also on activist's beliefs about the impact of their actions.  I am more interested in 2 and 3.  To be fair, the paper gives some evidence for detrimental impacts on 2 in the Trump example.  It's not clear, however, whether the nature of the cause matters here.  Support for Trump is highly polarized and entangled with culture, whereas global warming (Hallam's cause) and AI risk (PauseAI's) have relatively broad but frustratingly lukewarm public support.  There are also many other factors when looking past short-term onlooker sentiment to the larger question of affecting social change, which the paper readily admits in the Discussion section.  I'd list these points, but they largely overlap with the points I made in my post...though it was interesting to see how much was speculative.  More research is needed.

In any case, I bring up the extreme case to illustrate that the issue is far more nuanced than "regular people get squeamish--net negative!"  This is actually somewhat irrelevant to PauseAI in particular, because most of our actions are around public education and lobbying, and even the protests are legal and non-disruptive.  I've been in two myself and have seen nothing but positive sentiment from onlookers (with the exception of the occasional "good luck with that!" snark).  The hard part with all of these is getting people to show up.  (This last paragraph is not a rebuttal to anything you have said, it's a reminder of context)

This discussion is very interesting to read and I look forward to hearing @Fabien Roger [LW · GW]'s thoughts on your latest comment.

My story was posted before James_Miller's. Does this mean I invented a (sub-sub-)genre of science fiction?

Thinking of trying the latest Gemini model? Be aware that it is almost impossible to disable the "Gemini in Docs" and "Gemini in Gmail" services once you have purchased a Google One AI Premium plan.

Yeah. I proposed a while ago that all the AI content was becoming so dominant that it should be hived off to the Alignment Forum while LessWrong is for all the rest. This was rejected.