LessWrong 2.0 Reader

If you think the primary bottleneck to dangerous ASI is not that, but rather something else, then what do you think it is?

So far in this thread I was mostly talking from the perspective of my model(/steelman?) of Abram's argument.

I think the primary bottleneck to dangerous ASI is the ability to develop coherent and correct understandings of arbitrary complex domains and systems

I mostly agree with this.

Still, this doesn't^[1] rule out the possibility of getting an AI that understands (is superintelligent in?) one complex domain (specifically here, whatever is necessary to meaningfully speed up AIS research) (and maybe a few more, as I don't expect the space of possible domains to be that compartmentalizable), but is not superintelligent across all complex domains that would make it dangerous.

It doesn't even have to be a superintelligent reasoner about minds. Babbling up clever and novel mathematical concepts for a human researcher to prune could be sufficient to meaningfully boost AI safety (I don't think we're primarily bottlenecked on mathy stuff but it might help some people and I think that's one thing that Abram would like to see).

1. ^{^}

   Doesn't rule out in itself but perhaps you have some other assumptions that imply it's 1:1, as you say.

I know people who have gotten access to similarly important governmental systems at younger ages. 

Don't worry about it too much. 

If they abuse it,  it'll cost their group lots of political goodwill. (Recursive remove for example)

When it comes to Elon Musk's personal power it's worth speaking about what kind of goals Elon Musk has. At the recent Tesla earnings call, Elon said that deploying FSD for autonomous cars in China is difficult because Chinese law says that the videos Tesla records in China can't leave the US and US laws says that Tesla is not allowed to train AI models in China. In Elon Musk's presentation about what's important for Tesla, FSD is very important. 

If Elon Musk's power would be equal to being a dictator, he would get the US laws changed so that he can train the models in China. The AI policy of the White House is however to be tough on China, and Elon Musk does not have the power to change US policy in the way that would be needed to train FSD in China. He does not have enough power to make his own interests trump the interest of the US as far as Trump sees the interests of the US.

Elon is okay with EV subsidies being lowered even if that's bad for Tesla's bottom line because Elon seems to actually believe in cutting government spending and that's incompatible with having a lot of EV subventions. All the signs we have seen look like Elon actually believes in the goals of DOGE and pursues his activity at DOGE to achieve its stated goals. 

While you might agree or disagree with the stated goals of DOGE, saying that Elon run a coup is silly. Each coup starts by securing power over the monopoly of violence and all the actions of DOGE seem to be focused elsewhere than getting control over the monopoly of violence.

The treasury system seems to be written in COBOL. COBOL isn't user-friendly and if you want to run a bunch of analysis of payment flow you likely don't want to run it on the COBOL mainframe. For that reason, they seem to have decided to export the data out of the COBOL system to another server that's more traditional. I think it's quite plausible that the current COBOL system lacks a "export everything" feature and that the DOGE team needed write access to get the data export to work properly. They probably also wanted to add a feature that makes that data export automatic.

While I haven't looked into the laws myself, there are claims that the way DOGE operates might violate Privacy Act of 1974, Federal Information Security Modernization Act, Federal Records Act and the Administrative Procedure Act. While violating these kinds of laws might be standard operating procedure at intelligence agencies and the Snowden revelations did not lead to anyone being charged over violating privacy laws, the DOGE team might not be so lucky if the next president is a Democrat. It's possible that the whole DOGE team will need pardons. Given the precedent that Biden sat at the end of his term, Trump is likely willing to give them those pardons but it creates bad incentives. Anyone at the DOGE team who has a problem with the way the team operates can be told "If you don't play ball you don't get your pardon".  

Elon has said that he wants to replace the current system by one that uses a blockchain to be tamperproof. That's likely a good idea in principle, if they however spent two months to create their minimal viable product of a new government payment system and then want to move over the whole existing government payment system there's a high likelihood for a few problems. 

You are correct, that should be fixed. That's a straightforward mistake on my part, thank you for pointing it out!

I think it’s 1:1, because I think the primary bottleneck to dangerous ASI is the ability to develop coherent and correct understandings of arbitrary complex domains and systems (further details) [LW · GW], which basically amounts to anti-slop.

If you think the primary bottleneck to dangerous ASI is not that, but rather something else, then what do you think it is? (or it’s fine if you don’t want to state it publicly)

I am not a fan, but it is worth noting that these are the issues that many politicians bring up already, if they're unfamiliar with the more catastrophic risks. Only one missing on there is job loss. So while this choice by OpenAI sucks, it sort of usefully represents a social fact about the policy waters they swim in.

creating task forces of random people apparently selected mostly for personal loyalty, and

Why do you believe that DOGE is mostly selected for personal loyalty? Elon Musk seems to say openly says whatever he wants even if that goes against what Trump said previously. Trump likely would have preferred not to have a public fight about H1B visas but Elon Musk took that fight. Publically tweeting against the 500 billion project was disloyalty to Trump. 

Elon Musk has a history of not really having loyalty to anyone to the point that even his critics talk about how it's likely that he would have a fallout with Trump. True signals of loyalty as signs that there won't be a fallout. 

 Elon Musk is seen as highly skilled at running large organizations by many people. Trump picked Elon for DOGE because he believes that Elon is skilled at accomplishing the goals of DOGE. 

When Elon hired for it he said he seeks: 

"We don’t need more part-time idea generators. We need super high-IQ small-government revolutionaries willing to work 80+ hours per week on unglamorous cost-cutting."

The DOGE team brought their beds to the office to basically work nonstop. While we don't have the exact IQ scores, the background of the people we do have makes it quite plausible that they all have >140 IQ.

If personal loyalty is your main criteria you don't get a bunch of people who never leave the office and work non-stop with high IQs. 

I think the reason for downvoting is that this is an article on a specific topic, and you are trying to derail it into a debate on a generic culture-war topic.

I can sympathize with lesbians who don't want to be picked up by male-bodied people in their lesbian safe spaces, but that would be a different topic for a different thread. (And maybe a different website.)

This is a thread to discuss a specific group of people going around and committing murders. Until recently, this was the only thread on this topic. But there were thousands of threads all over internet about the topic you want to discuss, so it does not make sense to introduce it here.

And all of this will happen far faster than it did in the past, so people won’t get a chance to adapt. If your job gets eliminated by AI, you won’t even have time to reskill for a new job before AI takes that one too.

I propose an alternative to speed as explanation: all previous forms of automation were local. Each factory had to be automated in bespoke fashion one at a time; a person could move from a factory that was automated to any other factory that had not been yet. The automation equipment had to be made some somewhere and then moved to where the automation was happening.

By contrast, AI is global. Every office on earth can be automated at the same time (relative to historical timescales). There's no bottleneck chain where the automation has to be deployed to one locality, after being assembled in a different locality, from parts made in many different localities. The limitations are network bandwidth and available compute, both of which are shared resource pools and complements besides.

No specific link either, but if you know the usual "female brain in a male body" explanation, Ziz kinda has a more nuanced version of this, where each brain hemisphere is a separate personality, so you can have e.g. one male and one female hemisphere in a male body.

(And "if you don't believe an X person when they interpret their own lived experience, that makes you X-phobic" is a standard woke trope.)