LessWrong 2.0 Reader

Anyway, my prediction is that non-dyslectics do not subvocalize - it's much too slow. You can't read faster than you speak in that case.

Maybe I'm just weird, but I totally do sometimes subvocalize, but incredibly quickly. Almost clipped or overlapping to an extent, in a way that can only really work inside your head? And that way it can go faster than you can physically speak. Why should your mental voice be limited by the limits of physical lips, tongue, and glottis, anyway?

Excellent, thanks!

Much research on deception (Anthropic's recent work, trojans, jailbreaks, etc) is not targeting "real" instrumentally convergent deception reasoning, but learned heuristics.

If you have the slack, I'd be interested in hearing/chatting more about this, as I'm working (or trying to work) on the "real" "scary" forms of deception. (E.g. do you think that this paper [LW · GW] has the same failure mode?)

Anybody know how Fathom Radiant (https://fathomradiant.co/) is doing?

They’ve been working on photonics compute for a long time so I’m curious if people have any knowledge on the timelines they expect it to have practical effects on compute.

Also, Sam Altman and Scott Gray at OpenAI are both investors in Fathom. Not sure when they invested.

I’m guessing it’s still a long-term bet at this point.

OpenAI also hired someone who worked at PsiQuantum recently. My guess is that they are hedging their bets on the compute end and generally looking for opportunities on that side of things. Here’s his bio:

Ben Bartlett I'm currently a quantum computer architect at PsiQuantum working to design a scalable and fault-tolerant photonic quantum computer. I have a PhD in applied physics from Stanford University, where I worked on programmable photonics for quantum information processing and ultra high-speed machine learning. Most of my research sits at the intersection of nanophotonics, quantum physics, and machine learning, and basically consists of me designing little race tracks for photons that trick them into doing useful computations.

For the pretraining-finetuning paradigm, this link is now made much more explicitly in Cross-Task Linearity Emerges in the Pretraining-Finetuning Paradigm; as well as linking to model ensembling through logit averaging. 

YouTube can generate those automatically, or you can rip the .mp4 with an online service (just Google around, there are tons), then pass it to something like Otter.ai

“ taxonomy is not automatically a great category for regular usage.”

This is great, and I love the specific example of trees as a failure to classify a large set into subsets.

Something that’s not exactly the same problem, but rhymes, is that of genre classification for content discovery. Consider Spotify playlists. There are millions of songs, and hundreds of classified genres. Genres are classified much like species/genus taxonomies— two songs share a genre if they share a common ancestor of music. Led Zeppelin and the Beatles are different, but they both derive from traditions of electric guitar which grew out of the blues which… and so on. So we say Led Zeppelin and the Beatles are the same genre, “Rock”. You can do this kind of classification in much greater detail to carve out new genres and subgenres.

However when it comes to utility and discovery, genres underperform. Despite being the same genre, there are few parties which shuffle between the melancholic “Yesterday” and the screaming “Ramble On”. People seek songs which are similar to others in strategy, NOT in tradition. As you said:

“tree is a strategy. Wood is a strategy. Fruit is a strategy. A fish is also a strategy”

Successful user created playlists on Spotify (ones which are public with lots of likes) tend NOT to use genre. The tend to be called something line “rainy tuesday drive home from work”, or “music school nerds playing ALL the notes”. Rather than carve out a subset using genre (a playlist is a subset of music), they define it by strategy.

A failure in taxonomy.

One thing I'd note is that AIs can learn from variables that humans can't learn much from, so I think part of what will make this useful for alignment per se is a model of what happens if one mind has learned from a superset of the variables that another mind has learned from.

There’ll be discussions about how these systems will eventually become dangerous, and safety-concerned groups might even set up testing protocols (“safety evals”).

My impression that safety evals were deemed irrelevant because a powerful enough AGI, being deceptively aligned, would pass all of them anyway. We didn't expect the first general-ish AIs to be so dumb, like how GPT-3 was being so blatant and explicit about lying to the TaskRabbit worker. 

 If this was true, then we could not observe the event „any other coin sequence“ as well since this event is by definition not being tracked. 

When you are tracking event A you are automatically tracking its complement. 

In fact, in order to detect a correspondence between a coin sequence that we have in mind and the actual sequence, our brain has to compare them to decide if there is a match. I can hardly imagine how this comparison could work without observing the specific actual sequence in the first place. That we classify and perceive a specific sequence as „any other sequence“ can be the result of the comparison, but is not its starting point.

Oh sure, you are of course completely correct here. But this doesn't contradict what I'm saying. 

The thing is, we observe a particular outcome and then we see which event(s) it corresponds to. Let's take an example: a series of 3 coin tosses. 

So, in the beginning you have sample space which consist of all the elementary outcomes:

$\{HHH,TTT,HHT,TTH,HTH,THT,THH,HTT\}$

And an event space, some sigma-algebra of the sample space, which depends on your precommitments. Normally, it would look something like this:

$\{\varnothing ,\{HHH,TTT\},\{HHT,TTH,HTH,THT,THH,HTT\},\{HHH,TTT,HHT,TTH,HTH,THT,THH,HTT\left\}\right\}$

Because you are intuitively paying attention to whether there all Heads/Tails in a row. So your event space groups individual outcomes in this particular way, separating the event you are tracking and it's complement.

When a particular combination, say $THH$ is realized in a iteration of the experiment, your mind works like this:

• Outcome $THH$ is realized
• Therefore every event from the event space which includes $THH$ is realized.
• Events $\{HHT,TTH,HTH,THT,THH,HTT\}$ and $\{HHH,TTT,HHT,TTH,HTH,THT,THH,HTT\}$ are realized.
• $P\left(HHTorTTHorHTHorTHTorTHHorHTT\right)=2/3$
• This isn't a rare event and so you are not particularly surprised

So, as you see, you do indeed observe an actual sequence, it's just that observing this sequence isn't necessary an event in itself.