LessWrong 2.0 Reader

However, the above is a nitpick. The real issue I have with your comment is that you seem to be criticizing me for not addressing the "capabilities come from not-SGD" threat scenario, when addressing that threat scenario is what this entire post is about.

That's not what I'm criticizing you for. I elaborated a bit more here [EA(p) · GW(p)]; my criticism is that this post sets up a straw version of the SLT argument to knock down, of which assuming it applies narrowly to "spiky" capability gains via SGD is one example.

The actual SLT argument is about a capabilities regime (human-level+), not a specific method for reaching it or how many OOM of optimization power are applied before or after.

The reasons to expect a phase shift in such a regime are because (by definition) a human-level AI is capable of reflection, deception, having insights that no other human has had before (as current humans sometimes do), etc.

Note, I'm not saying that setting up a strawman automatically invalidates all the rest of your claims, nor that you're obligated to address every possible kind of criticism. But I am claiming that you aren't passing the ITT [? · GW] of someone who accepts the original SLT argument (and probably its author).

But it does mean you can't point to support for your claim that evolution provides no evidence for the Pope!SLT, as support for the claim that evolution provides no evidence for the Soares!SLT, and expect that to be convincing to anyone who doesn't already accept that  Pope!SLT == Soares!SLT.

Do you still think your communication was better than the people who thought the line was being towed, and if so then what's your evidence for that?

We are way off topic, but I am actually going to say yes. If someone understands that English uses standing-on-the-right-side-of-a-line as a standard image for obeying rules, then they are also going to understand variants of the same idea. For example, "crossing a line" means breaking rules/norms to a degree that will not be tolerated, as does "stepping out of line". A person who doesn't grok that these are all referring to the same basic metaphor of do-not-cross-line=rule is either not going to understand the other expressions or is going to have to rote-learn them all separately. (And even after rote-learning, they will get confused by less common variants, like "setting foot over the line".) And a person who uses tow not toe the line has obviously not grokked the basic metaphor.

To be clear to other readers, I didn’t recommend making gay marriage illegal.

...

You’re essentially trying to get me to defend the class of “all possible policy proposals”, which is just silly.

Since society can change a lot in 50 years, this issue could very well be,  in 2073, the equivalent of trying to make gay marriage illegal today.

So it's possible that it will simply get laughed out of court and anyone serious, whose opinions matters in the government debt market, will also brush it off.

You pretty much do need to defend the class of 'nearly all possible policy proposals' to credibly account for unknown value and culture drift over 50 years.

To recap:

1. original poster johnswentsworth wrote a piece about people LARPing their jobs rather than attempting to build deeper understanding or models-with-gears
2. aysja added some discussion about people failing to notice that words have referents, as a further psychological exploration of the LARPing idea, and added tow/toe the line as a related phenomenon. They say "LARPing jobs is a bit eerie to me, too, in a similar way. It's like people are towing the line instead of toeing it. Like they're modeling what they're "supposed" to be doing, or something, rather than doing it for reasons."
3. You asked for further clarification
4. I tried using null pointers as an alternative metaphor to get at the same concept.

No one is debating the question of whether learning etymology of words is important and I'm not sure how you got hung up on that idea. And toe/tow the line is just an example of the problem of people failing to load the intended image/concept, while LARPing (and believing?) that they are in fact communicating in the same way as people who do.

Does that help?

What's the logical and theoretical basis for the supposed existence of such a construct?

If there isn't, then the default assumption is that there's no such thing. Much like how we by default assume that there are no porcelain teapots orbiting the sun.

Chrome reports as 117.0.5938.92 (Official Build) (64-bit).

I already described the problem with the first paragraph equation (display equation) on the page.

The second paragraph equation, which can be located by searching for "log-likelihood", also has the problem. In particular, it copies as

\text{PPL}(X) = \exp\left(-\frac{1}{n}\sum_i^n \log p_\theta(x_i|x_{

The third one, locatable via "concept vector v", works correctly:

P_{\perp}(x_i) = x_i - \frac{}{||v||^2}v\,.

There is no fourth paragraph equation on the page.

Let me know if you want me to continue to search for instances of the bug, on other pages.
 

Just to add to Carl Feynman's response, which I thought was good.

Part of the reason these systems are inefficient is because it requires you to (effectively) run gradient descent even at inference, even after training is over. Or you can run the RNN, which is mathematically equivalent but again you can see where the inefficiency comes in: the value at time t=3 is a function of the value at time t=2, which is a function of t=1 and so on, so in order to get the converged value of the activations you have to, in a for loop, compute each timestep one by one.

This is in contrast to a feedforward network like a (normal) convnet or transformer, which can run extremely quickly and in parallel on gpu.

Thanks! 

I think your thinking makes sense, and, if for instance on every timestep you presented a different images in a stereotypically defined sequence, or with a certain correlation structure, you would indeed get information about those correlations in the weights. However, this model was designed to be used in the restricted to settings where you show a single still image for many timesteps until convergence. In that setting, weights give you image features for static images (in a heirarchical manner), and priors for low level features will feed back from activations in higher level areas.

There are extensions to this model that deal with video, where there are explicit spatiotemporal expectations built into the network. you can see one of those networks in this paper: https://arxiv.org/abs/2112.10048

But I've never implemented such a network myself.

This, in so many words was likely one of the biggest factors why Anthropic/OpenAI/Deepmind were so much more successful than any LW person or group at AI safety until maybe 2021 at the earliest, and even then the lead shifted.

A lot of AI safety proposals before deep learning were basically useless because of the feedback loop issue.

I think this is also connected to ambition issues, but even then lack of feedback loops was way worse for LW than they thought it was.

It's also why longtermism should be bounded in practice, and a very severe bound at that.

See also Research as a Stochastic Decision Process