post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by gwern · 2022-06-12T22:14:06.699Z · LW(p) · GW(p)

For instance, quoting from the recent post on list of lethalities:

restricting yourself to doing X will not prevent Facebook AI Research from destroying the world six months later

Compare this to:

restricting yourself to doing X will not prevent another AI lab from destroying the world six months later

One problem with your suggested reformulation is that they do not mean the same thing. Naming FAIR conveys information above and beyond 'another AI lab', which additional information would be hard to sum up, much less put into an equally short phrase: there's reasons it's 'FAIR' which is used in that sentence about catch-up competitors & unilateralist curse, and why various other AI lab names which one might think could be dropped in like 'DeepMind' or 'Sberbank' or 'MILA' or 'BAIR' or 'OpenAI' or 'Nvidia' or 'MSR' would all be wrong in various ways*, but it's hard to even explain. Similarly, there are things like the Google Brain vs DeepMind rivalry which affect how scaling research has been playing out, for the worse, and are relevant to safety, but it would be absurd to try to talk about it in a generic sense or use pseudonyms.

(To try to explain anyway: basically, they're kind of a perpetual second-mover second-best player with a bit of a chip on their shoulder from poaching and scooping - consider Darkforest - with little commitment to AI safety either in the real AI risk sense or AI bias sense and mostly in denial that there is any problem (eg. LeCun: "There is no such thing as Artificial General Intelligence because there is no such thing as General Intelligence."), and get looked down on for being part of Facebook with overly-applied focus, seen as the most ethically-compromised and amoral part of FANG eager to ship tech because of Zuckerberg's conviction that more tech & transparency is always better (at least as long as it increases FB growth), and a historical backstory where FB struggled to hire DL people to begin with despite Zuckerberg strongly believing in DL potential from almost AlexNet, being spurned by the likes of Hassabis, settling for LeCun/Pesenti/etc, because despite access to huge resources & data & industrial application FB/Zuckerberg were already widely hated. This description takes up a whole paragraph, and you probably still don't know a good chunk of what I'm referring to with Darkforest or Zuckerberg NIPS shopping, and if you disagreed, asking why this doesn't characterize OA or GB or Baidu equally well, that would be another set of paragraphs, and so on. But if you already know this, as you should if you are going to discuss these things, then use of 'FAIR' immediately conjures the family of plausible complex-failure disaster scenarios where eg. DM or another top player succeeds in not-instantly-failed AGI, it gets controlled by the DM safety board overruling any commercial pressures, only to trigger Zuck into ordering a replication effort to leapfrog the over-cautious 'dumb fucks' at DM, about which there is zero internal mechanism or oversight to oppose the CEO/controlling shareholder himself nor anyone who cares, which succeeds because the mere fact of AGI success will tend to indicate what the solution is, but then fails.)

* This is one reason I invested so much effort in link-icons on gwern.net, incidentally. Research papers are not published in isolation, and there are lots of 'flavors' and long-standing projects and interests. Much like an expert in a field can often predict who wrote a paper despite blinding (ex ungue leonem), you can often tell which lab wrote a paper even with all affiliation information scrubbed, because they bring with them so much infrastructure, approaches, preferences, and interests. I can predict a remarkable amount of, say, the Gato paper simply from following Decision Transformer research + the title + 'DeepMind' rather than 'Google' or 'OpenAI' or 'BAIR'. (For a natural experiment demonstrating lab flavor, simply compare Gato to its fraternal twin paper, GB's Multi-Game Transformer.) So I find it very useful to have a super-quick icon denoting a Facebook vs OpenAI vs DM vs Nvidia etc paper. You the ordinary reader may not find such annotations that useful, yet, but you will if you git gud.

Replies from: None
comment by [deleted] · 2022-06-13T03:40:14.106Z · LW(p) · GW(p)
comment by Sam Ringer · 2022-06-12T23:56:23.868Z · LW(p) · GW(p)

I think it's good that someone is bringing this up. I think as a community we want to be deliberate and thoughtful with this class of things.

That being said, my read is that the main failure mode with advocacy at the moment isn't "capabilities researchers are having emotional responses to being called out which is making it hard for them to engage seriously with x-risk."
It's "they literally have no idea that anyone thinks what they are doing is bad."

Consider FAIR trying their hardest to open-source capabilities work with OPT. The tone and content of the responses shows overwhelming support for doing something that is, in my worldview, really, really bad.

I would feel much better if these people at least glanced their eyeballs over arguments for not open-source capabilities. Using the names of specific labs surely makes it more likely that the relevant writing ends up in front of them?

Replies from: TrevorWiesinger, None
comment by trevor (TrevorWiesinger) · 2022-06-13T02:52:09.560Z · LW(p) · GW(p)

I've done quite a bit of thinking about this, and I'm pretty familiar with the area.

If a corporation has a brand, and you have no idea how powerful, aggressive, or exploitative that corporation is (e.g. Facebook, Disney, etc), then it's best not to write anything that calls out that brand. If you go on Reddit and write something publically about how awful Dr. Pepper is, then you're entangling yourself into the ongoing conflict between Coca Cola and Pepsi, whether you know about it or not. And if you don't know what you're getting into, or even aren't sure, then you certainly aren't prepared to model the potential consequences.

Replies from: Sam Ringer
comment by Sam Ringer · 2022-06-13T03:50:27.695Z · LW(p) · GW(p)

Ok not sure I understand this. Are you saying "Big corps are both powerful and complicated. Trying to model their response is intractably difficult so under that uncertainty you are better to just steer clear?"

Replies from: TrevorWiesinger
comment by trevor (TrevorWiesinger) · 2022-06-13T04:12:23.522Z · LW(p) · GW(p)

Yes, that's a very good way of putting it. I will be more careful to think about inferential distance [LW · GW] from now on.

comment by [deleted] · 2022-06-13T03:43:42.627Z · LW(p) · GW(p)Replies from: Sam Ringer
comment by Sam Ringer · 2022-06-13T03:55:46.913Z · LW(p) · GW(p)

Yeh so thinking a little more I'm not sure my original comment conveyed everything I was hoping to. I'll add that even if you could get a side of A4 explaining AI x-risk in front of a capabilities researcher at <big_capabilities_lab>, I think they would be much more likely to engage with it if <big_capabilities_lab> is mentioned.

I think arguments will probably be more salient if they include "and you personally, intentionally or not, are entangled with this."

Saying that, I don't have any data about the above. I'm keen to hear any personal experiences anyone else might have in this area.

comment by trevor (TrevorWiesinger) · 2022-06-13T02:47:01.232Z · LW(p) · GW(p)

Facebook AI labs: I wonder what people are saying about us on the internet? Hopefully nothing, obviously. But lets look, because we can.

Facebook AI Labs: *inundated with randomly ordered yudkowsky paragraphs with no context whatsoever*

Facebook AI Labs: Well, at least the problem is contained to a single set of internet forums, and it looks crazy to us, but stranger things have spread to tens of millions of people. We'd better go ask someone if they think these people should be taken care of.

 

Alternatively:

Facebook AI labs: Wait, these are actuals NGOs? And they're saying bad things about AI and Facebook AI Labs? Well, that's clearly a problem to bring to management.

comment by burmesetheater (burmesetheaterwide) · 2022-06-12T18:15:39.813Z · LW(p) · GW(p)

This seems like a case of making a rule to fix a problem that doesn't exist.

Are people harassing individual AI labs or researchers? The tendency for reasonable people who are worried about AI safety should be to not do so, since it predictably won't help the cause and can hurt. So far there does not seem to be any such problem of harassment discernible from background noise.

Naming individual labs and / or researchers is interesting, useful, and keeps things "real." 

Replies from: None
comment by [deleted] · 2022-06-12T18:43:30.849Z · LW(p) · GW(p)Replies from: burmesetheaterwide
comment by burmesetheater (burmesetheaterwide) · 2022-06-12T20:49:38.947Z · LW(p) · GW(p)

So, to be clear, you don't think confidently naming people by first name as destroying the world can be parsed emotionally by them?

Mentions of AI companies / AI personalities on LW will intrinsically tend to be adversarial, even if the author spares a polemic or use of terms like "so and so is working to destroy the world" because misaligned AI destroying the world is clearly THE focus of this community. So it can be argued that to be meaningful, a policy of no names would need to be applied to practically any discussion of AI as even if some AI content is framed positively by the author, the community at large will predictably tend to see it in existential risk terms.

That's one issue. Personally, the calculus seems pretty simple: this well-behaved community and its concerns are largely not taken seriously by "the powers" who will predictably create AGI, there is little sign that these concerns will be taken seriously before reaching AGI, and there is almost no reason to think that humanity will pause to take a break and think "maybe we should put this on hold since we've made no discernible progress toward any alignment solutions" before someone trains and runs an AGI. So a conclusion that could be drawn from this is, we might as well have nice uncensored talks about AI free from petty rules until then.

Replies from: None
comment by [deleted] · 2022-06-13T03:47:22.416Z · LW(p) · GW(p)Replies from: burmesetheaterwide
comment by burmesetheater (burmesetheaterwide) · 2022-06-14T03:43:37.373Z · LW(p) · GW(p)

Anyone can try, this seems way out in a practically invisible part of the tail of obstacles to not being destroyed by AGI, if it's even an obstacle at all. 

Replies from: None
comment by [deleted] · 2022-06-14T14:54:18.050Z · LW(p) · GW(p)
comment by ESRogs · 2022-06-12T21:41:34.521Z · LW(p) · GW(p)

Less antagonist.

Should be "less antagonistic", unless you're talking about the bad guy in a story.

Replies from: None
comment by [deleted] · 2022-06-13T03:44:00.327Z · LW(p) · GW(p)