davekasten's Shortform

A subpoena for what?

"want to pick a war with America" is really strange wording because China's strategic goals are not "win a war against nuclear-armed America", but things like "be able to control its claims in the South China Sea including invading Taiwan without American interference". Likewise Russia doesn't want to "pick a war with the EU" but rather annex Ukraine; if they were stupid enough to want the former they would have just bombed Paris. I don't know whether national security people relate to the phrasing the same way but they do understand this.

Has China has made a statment about starting a war in 2027 or later? Who exactly is the belief that "by-default China will want to pick a war with America in 2027 or later" held by and how confident are you that they hold it? 

I don't think they have stated they'll to to war after 2027. 2027 is the year of their "military modernization" target.

Also of relevance is the wave of resignations from the DC newspaper The Washington Post the past few days over Jeff Bezos suddenly exerting control.

My guess is that we wouldn't actually know with high confidence before (and likely even some time after) things-will-definitely-be-fine.

E.g. 3 months after safe ASI people might still be publishing their alignment takes.  

Endgame strategies from who?

A lot of powerful people would focus on being the ones to control it when it happens, so they'd control the future - and not be subject to some else's control of the future. OpenPhil is about the only org that would think first of the public benefit and not the dangers of other humans controlling it. And not a terribly powerful org, particularly relative to governments.

Random psychologizing explanation that resonates most with me: Claiming to address big problems requires high-status. A low-rank person is allowed to bring up minor issues, but they are not in a position to bring up big issues that might reflect on the status of many high-status people. 

This is a pretty common phenomenon that I've observed. Many people react with strong social slap-down motions if you (for example) call in question whether the net-effect of a whole social community or economic sector is negative, where the underlying cognitive reality seems similar to "you are not high status enough to bring forward this grievance".

There are (at least) two models which could partially explain this:
1) The high-status/high-rank people have that status because they're better at abstract and long-term thinking, and their role is more toward preventing catastrophe rather than nudging toward improvements.  They leave the lesser concerns to the underlings, with the (sometimes correct) belief that it'll come out OK without their involvement.

2) The high-status/high-rank people are rich and powerful enough to be somewahat insulated from most of the prosaic AI risks, while the average member can legitimately be hurt by such things.   So everyone is just focusing on the things most likely to impact themselves.

edit: to clarify, these are two models that do NOT imply the obvious "smarter/more powerful people are correctly worried about the REAL threats, and the average person's concerns are probably unimportant/uninformed".  It's quite possible that this division doesn't tell us much about the relative importance of those different risks.  

Does "highest status" here mean highest expertise in a domain generally agreed by people in that domain, and/or education level, and/or privileged schools, and/or from more economically powerful countries etc? It is also good to note that sometimes the "status" is dynamic, and may or may not imply anything causal with their decision making or choice on priorities.

One scenario is "higher status" might correlates with better resources to achieve those statuses, and a possibility is as a result they haven't experienced or they are not subject to many near-term harms. In other words, it is not really about the difference between "average" and "high status"'s people's intelligence, but more about what kind of world they are exposed to. 

I do think it is good to hear all different perspectives to stay curious/open-minded. 

edit: I just saw Dragon nicely listed two potential reasons, with scenario 2 mentioning something similar with my comment here. But something slightly specific in my thinking, is that these choices made by "average" and "high status" people may or may not be conscious, but rather from the experience from their lives and the world they are exposed to.

I think many mass casualty events would be warning shots, but not all warning shots would be mass casualty events. I think an agentic AI system getting most of the way towards escaping containment or a major fraud being perpetrated by an AI system would both be meaningful warning shots, but wouldn't involve mass casualties.

I do agree with what I think you are pointin at, which is that there is something Orwellian about the "warning shot" language. Like, in many of these scenarios we are talking about large negative consequences, and it seems good to have a word that owns that (in-particular in as much as people are thinking about making warning shots more likely before an irrecoverable catastrophe occurs).

I think a lot of people losing their jobs would probably do the trick, politics-wise.  For most people the crux is "will AIs will be more capable than humans", not "might AIs more capable than humans be dangerous".

Well, what's the alternative? If you think there is something weird enough and suboptimal about essay formats that you are reaching for 'random chance' or 'monkey see monkey do' level explanations, that implies you think there is some much superior format they ought to be using instead. But I can't see what. I think it might be helpful to try to make the case for doing these things via some of the alternatives:

1. a peer-reviewed Nature paper which would be published 2 years from now, maybe, behind a paywall
2. a published book, published 3 years from starting the first draft now, which some people might get around to reading a year or two after that, and dropping halfway through (assuming you finish and didn't burn out writing it)
3. a 1 minute Tiktok video by an AI person with non-supermodel looks
4. a 5-minute heavily-excerpted interview on CNN
5. a 750-word WSJ or NYT op-ed
6. a 10-page Arxiv paper in the standard LaTeX template
7. a Twitter thread of 500 tweets (which can only be read by logged-in users)
8. a Medium post (which can't be read because it is written in a light gray font illegible to anyone over the age of 20. Also, it's paywalled 90% of the time.)
9. a 6 hour Lex Fridman podcast interview, about 4 hours in after Lex has finished his obligatory throatclearing questions (like asking you if aliens exist or the universe is made out of love)
10. interpretive dance in front of the Lincoln Memorial livestreamed on Twitch
11. ...

(I'd also add in Karnofsky's blog post series.)

I think those are the meta because they have just enough space to not only give opinions but to mention reasons for those opinions and expertise/background to support the many unstated judgment calls.

Note that the essays by Altman and Amodei are popular because their positions are central beyond the others because they have not only demonstrable backgrounds in AI but lots of name recognition (we're mostly assuming Altman has bothered learning a lot about how Transformers work even if we don't like him). And that the Gladstone report got itself commissioned by at least a little piece of the government.

A Narrow Path just demonstrates in the text that you and your co-authors have thought deeply about the topic. Shorter essays leave more guesswork on the authors' expertise and depth of consideration.

That "so now what" doesn't sound like a dead end to me. The question of how to mitigate risk when normal risk-mitigation procedures are themselves risky seems like an important one.

I agree that it's not terribly useful beyond identifying someone's fears.  Using almost any taxonomy to specify what the speaker is actually worried about lets you stop saying "infohazard" and start talking about "bad actor misuse of information" or "naive user tricked by partial (but true) information".  These ARE often useful, even though the aggregate term "infohazard" is limited.

I suggest there is a concept distinct enough to warrant the special term, but if it's expansive enough to include secrets, beneficial information that some people prefer others not know, that renders it worthless. 

"Infohazard" ought to be reserved for information that harms the mind that contains it, with spoilers as the most mild examples, SCP-style horrors as the extreme fictional examples.

I think within a bayesian framework where in-general you assume information has positive value, it's useful to have an explicit term when that is not the case. It's a relatively rare occurrence, and as such your usual ways of dealing with information will probably backfire. 

The obvious things to do is to not learn about that information in the first place (i.e. avoid dangerous research), understand and address the causes for why this information is dangerous (because e.g. you can't coordinate on not building dangerous technology), or as a last resort, silo the information and limit the spread of it. 

I do think that it would be useful to have different words that distinguish between "infohazard to the average individual" and "societal infohazard". The first one is really exceedingly rare. The second one is still rare but more common because society has a huge distribution of beliefs and enough crazy people that if information can be used dangerously, there is a non-trivial chance it will. 

Ok, the information's harmful.  You need humans to touch that info anyways to do responsible risk-mitigation.  So now what ?

I think one of the points is that you should now focus on selective [LW · GW] rather than corrective or structural means to figure out who is nonetheless allowed to work on the basis of this information. 

Calling something an infohazard, at least in my thinking, generally implies both that: 

• any attempts to devise galaxy-brained incentive structures [? · GW] that try to get large groups of people to nonetheless react in socially beneficial ways when they access this information are totally doomed and should be scrapped from the beginning.
• you absolutely should not give this information to anyone that you have doubts would handle it well; musings along the lines of "but maybe I can teach/convince them later on what the best way to go about this is" are generally wrong and should also be dismissed.

So what do you do if you nonetheless require that at least some people are keeping track of things? Well, as I said above, you use selective methods instead. More precisely, you carefully curate a very short list of human beings that are responsible people and likely also share your meta views on how dangerous truths ought to be handled [LW · GW], and you do your absolute best to make sure the group never expands beyond those you have already vetted as capable of handling the situation properly.

I suspect this won't get published until November at the earliest, but I am already delightfully pleased with this bit:

Canada geese fly overhead, honking. Your inner northeast Ohioan notices that you are confused; it’s the wrong season for them to migrate this far south, and they’re flying westwards, anyways.

A quick Google discovers that some Canada geese have now established themselves non-migratorily in the Bay Area:

"The Migratory Bird Treaty Act of 1918 banned hunting or the taking of eggs without a permit. These protections, combined with an increase in desirable real estate—parks, golf course and the like—spurred a dramatic turnaround for the species. Canada geese began breeding in the Bay Area—the southern end of their range – in the late 1950s."

You nod, approvingly; this clearly is another part of the East Bay’s well-known, long-term philanthropic commitment to mitigating Acher-Risks.

I agree (minor spoilers below). 

(This is not an endorsement of Jim Caviezel's beliefs, in case anyone somehow missed my point here.)