LessWrong 2.0 Reader

Typically I agree with the underlying facts behind her memes! For example I also think AI safety is a pressing issue. If her memes were funny I would instead be writing a post about how awesome it is that Kat Woods is everywhere. My main objection is that I do not like the packaging of the ideas she is spreading. For example the memes are not funny. (See the outline of this post: content, vibes, conduct.) 

You asked for an example of Kat Woods content that aims to convince rather than educate. Here is one recent example. I feel like the packaging of this meme conveys: "all of the objections you might have to the idea of X-risk via AI can actually be easily be debunked, therefore you would be stupid to not believe X-risk via AI". 

In reality, questions regarding likelihood of x-risk via AI are really tricky. Many thoughtful people have thought about these problems at great length and declared them to be hard and full of uncertainty. I feel like this meme doesn't convey this at all. Therefore, I'm not sure whether it is good for peoples' brains to consume this content. I will certainly say it's not good for my brain to consume this content.

I asked a leading question of our "Perspectives" system, and it gave a few hundred hypothetical reasons (here)

I have my own theories about the intentions which I do not feel comfortable discussing, so I'll focus on the practicalities and case studies which show why this complex and difficult to execute:
some hostages have been killed by the IDF during rescue operations, this isn't uncommon, the lone hostage was killed during a French raid in Somalia, consider the Lindt Cafe Siege in Sydney where a pregnant hostage was killed by ricocheting police bullet fire when they finally stormed in, three other hostages and a policeman were injured. This was a lone gunman, I can imagine that the Hamas hostage takers are well organized groups. A hostage during the Gladbeck Crises in Germany were also injured by police fire. 

Kidnapping someone who "knows" the location of some hostages I would guess is highly ineffective for many reasons, Torture is a notoriously inaccurate source of information: hence the propensity for false admissions or telling interrogators what they want to hear. That and I suspect that there is a intentional system of moving around hostages from place to place, and never explicitly sharing locations with others to minimize the risk of locations leaking.

If someone who knows the exact location of a hostage has not been heard from for 24 hours, it is probably a good idea to move to a new location anyway.

Finally there is the incredible danger to the IDF soldiers themselves going into a dynamic environment where they don't know how much resistance they will encounter, being expected to minimize the harm to hostages while almost certainly coming under fire. It's probable suicide.

Thank you. So what do you think the cause of that is, and why do you think that cause exists and will it always exist?

Ah, interesting

Still, even if some parts of the architecture are public, it seems good to keep many details private, details that took the lab months/years to figure out? Seems like a nice moat

Agreed.

If the authors claim that adding randomness in the territory in classical mechanics requires making it more complex, they should also notice that for quantum mechanics, removing the probability from the territory for QM (like Bohmian mechanics) tends to make the the theories more complex. 

Also, QM is not a weird edge case to be discarded at leisure, it is to the best of our knowledge a fundamental aspect of what we call reality. Sidelining it is like arguing "any substance can be divided into arbitrary small portions" -- sure, as far as everyday objects such as a bottle of water are concerned, this is true to some very good approximation, but it will not convince anyone.

Also, I am not sure that for the many world interpretation, the probability of observing spin-up when looking at a mixed state is something which firmly lies in the map. From what I can tell, what is happening in MWI is that the observer will become entangled with the mixed state. From the point of view of the observer, they find themselves either in the state where they observed spin-up and spin-down, but their world model before observation as "I will find myself either in spin-up-world or spin-down-world, and my uncertainty about which of these it will be is subjective" seems to grossly misrepresent their model. They would say "One copy of myself will find itself in spin-down-world, and one in spin-up-world, and if I were to repeat this experiment to establish a frequentist probability, I would find that that the probability of each outcome is given by the coefficient of that part of the wave function to the power of two".

So, in my opinion,

• If a blackjack player wonders if a card placed face-down on the table is an ace, that is uncertainty in their map.
• If someone wonders how a deterministic but chaotic physical system will evolve over time, that is also uncertainty in the map.
• If someone wonders what outcome they are likely to measure in QM, that is (without adding extra epicycles) uncertainty in the territory.
• If someone wonders what the evolution of a large statistical ensemble which is influenced by QM at the microscopic level (such as a real gas) is, that might be mostly uncertainty in the map if looking at very short time scales (where position and momentum uncertainty and the statistical nature of scattering cross-sections would not constrain Laplace's daemon too much), but exists in the territory for most useful time spans.

At a glance, this seems like a high-risk high-reward tactic. I approve of this if this is in fact effective at changing people's minds, and disapprove of this if this is in fact making people cringe and become biased against the ideas she's trying to spread.

My immediate impression was that it's mostly the latter. I agree that these memes seem kind of cringe, especially if they're being spread in communities that are hostile to AI Safety takes and dislike this kind of content (which my cursory familiarity with r/singularity suggests)...

... but glancing at the karma ratings her posts receive, it doesn't seem that she's speaking to a hostile audience using takes that don't land with them. I think most of her posts have at least an average karma rating for a given community, some of them are big hits, and the comments are frequently skeptical but rarely outright derisive. This isn't a major indicator and I've only spent ~5 minutes looking through them, but it does look like it's working.

@KatWoods [LW · GW], do you have any more convincing metrics you're using to evaluate the efficiency of your, ah, propaganda campaign? Genuinely interested in how effective it is.

Seems fine, though this doesn't seem like the central crux. 

Currently:

• Prediction markets are used
• Argument maps tend not to be.

In Firefox, you can hover over the timestamp next to the name of the commenter, right-click (or whatever it is you Mac people do), and select Bookmark Link... to bookmark a comment.

Alternatively, you can click on the timestamp and then use whatever mechanism your OS/Browser allows to bookmark the page you go to as a result of the click.

Some hard problems with anti tampering and their relevance for GPU off-switches

Background on GPU off switches:

It would be nice if a GPU could be limited^[1] or shut down remotely by some central authority such as the U.S government^[2] in case there’s some emergency^[3].

This shortform is mostly replying to ideas like "we'll have a CPU in the H100^[4] which will expect a signed authentication and refuse to run otherwise. And if someone will try to remove the CPU, the H100's anti-tampering mechanism will self-destruct (melt? explode?)".

TL;DR: Getting the self destruction mechanism right isn't really the hard part.

Some hard parts:

1. Noticing if the self destruction mechanism should be used
   1. A (fun?) exercise could be suggesting ways to do this (like "let's put a wire through the box and if it's cut then we'll know someone broke in") and then thinking how you'd subvert those mechanisms. I mainly think doing this 3 times (if you haven't before) could be a fast way to get some of the intuition I'm trying to gesture at
2. Maintenance
   1. If the H100s can never be opened after they're made, then any problem that would have required us to open them up might mean throwing them away. And we want them to be expensive, right?
   2. If the H100s CAN be opened, then what's the allowed workflow?
      1. For example, If they require a key - then what if there's a problem in the mechanism that verifies the key, or in something it depends on, like the power supply? We can't open the box to fix the mechanism that would allow us to open the box
      2. ("breaking cryptography" and "breaking in to the certificate authority" are out of scope for what I'm trying to gesture at)
3. False positives / false negatives
   1. If the air conditioning in the data center breaks, will the H100s get hot^[5], infer they're under attack and self destruct? Will they EXPLODE? Do we want them to only self destruct if something pretty extreme is happening, meaning attackers will have more room to try things?
4. Is the design secret?
   1. If someone gets the full design of our anti-tampering mechanism, will they easily get around it?
   2. If so, are these designs being kept in a state-proof computer? Are they being sent to a private company to build them?
5. Complexity
   1. Whatever we implement here better not have any important bugs. How much bug-free software and hardware do we think we can make? How are we going to test it to be confident it works? Are we okay with building 10,000 when we only think our tests would catch 90% of the important bugs?
6. Tradeoffs here can be exploited in interesting ways
   1. For example, if air conditioning problems will make our H100s self destruct, an attacker might focus on our air conditioning on purpose. After 5 times of having to replace all of our data center - I can imagine an executive saying bye bye to our anti tampering ideas.
      1. I'm trying to gesture at something like "making our anti tampering more and more strict probably isn't a good move, even from a security perspective, unless we have good idea how to deal with problems like this"

In summary, if we build anti-tampering mechanisms in GPUs that an adversaries have easy access to (especially nation states) then I don't expect this will be a significant problem for them to overcome.

(Maybe later - Ideas for off-switches that seem more promising to me)

1. ^{^}

   For example bandwidth, gps location, or something else. the exact limitations are out of scope, I mainly want to discuss being able to make limitations at all

2. ^{^}

   The question of "who should be the authority" is out of scope, I mainly want to enable having some authority at all. If you're interested in this, consider checking out Vitalik's suggestion too, search for "Strategy 2"

3. ^{^}

   Or perhaps this can be used for export controls or some other purpose

4. ^{^}

   I'm using "H100" to refer to the box that contains both the GPU chip and the CPU chip

5. ^{^}

   I'm using "getting hot means being under attack" as a toy example for a "paranoid" condition that someone might suggest building as one of the triggers for our anti-tampering mechanism. Other examples might include "the box shakes", "the power turns off for too long", and so on.