LessWrong 2.0 Reader

my most fun talk made a similar claim:

I posted some of my thoughts on their website, might aswell share it on here: 

What I don’t understand is why they would need as much inference compute as a human? Maybe future architecture will make it way more inference cheaply compared to the human brain. And I don’t know how you can compare the human amount of inference and the amount of inference ai needs in order to automate remote work. Also, sample efficiency doesn’t apply to inference since weights are not updated at this stage (yet, some research suggest we should do that) and maybe it’ll end up more sparse (like transformers), in the sense that we can reduce the amount of compute we need for that. I also think you exaggerate how compute bound we are. Suppose we invent a new architecture, new paradigm or just tweak the transformer which makes it so much more sample efficient and also cheaper compute wise, we can just use these automated researchers to make the next generation even more cheaper, and they could be used explicitly at OpenAI or other research labs to speed up ai research, there isn’t really broad deployment needed.

The beings running the tests can skip over a lot of the suffering, and use actors instead of real victims.^[1] Even if actors show telltale signs, they can erase any reasoning you make which detects the inconsistencies. They can even give you fake memories.

Of course, don't be sure that victims are actors. There's just a chance that they are, and that they are judging you.

1. ^{^}

   I mentioned this in the post on Karma Tests. I should've mentioned it in my earlier comment.

They people running the Karma test deserve to lose a lot of Karma for the suffering in this world.

If your hypothesis is true, that's a cruel civilization by my personal standards because of all the suffering in this world.

Yes, that is the same idea. "This is a big pile of speculation that I don't take very seriously, but I feel like if we are being simulated, that's where most simulations of me would be instantiate" Why not take it seriously, if you accept high chance that (1) our reality is a simulation, (2)  we seem on track to creating a paperclip maximizer, (3) weird that I, Robert Miles, would have the personality traits that cause me to be one of the few humans so worried about humanity creating a paperclip maximizer if I'm right about us being on track to probably create one?

this experiment has been done before [EA · GW].

If you have a framing of the AI Doom argument that can cause a consensus of super-forecasters (or AI risk skeptics, or literally any group that has an average pDoom<20%) to change their consensus, I would be exceptionally interested in seeing that demonstrated.

Such an argument would be neither bad nor weak, which is precisely the type of argument I have been hoping to find by writing this post.

> Please notice that your position is extremely non-intuitive to basically everyone. 

Please notice that Manifold both thinks AGI soon and pDoom low.

I think this cumulative argument works:

1. there are dozens of ways AI can prevent a mass extinction event at different stages at its existence.

2. ...

If you make a list of 1000 bad things and I make a list of 1000 good things, I have no reason to think that you are somehow better at making lists than prediction markets or expert surveys.

Imagine a circuit

A --> B --> C
D --> E --> F
C + F --> output

Denoising (restoring) B + E is sufficient to restore behaviour, but B + E are just a "cross-section" (cutting through the circuit at the 2nd layer), not a superset of all components

Predicting AGI/ASI timelines is highly speculative and unviable. Ultimately, there are too many unknowns and complex variables at play. Any timeline must deal with systems and consequences multiple steps out, where tiny initial errors compound dramatically. A range can be somewhat reasonable, a more specific figure less so, and accurately predicting the consequences of the final event when it comes to pass even further improbable. It is simply impractical to come up with an accurate timeline with the knowledge we currently have.

Despite this, timelines are popular – both with the general AI hype crowd and those more informed. People don't seem to penalise incorrect timelines – as evidenced by the many predicted dates we've seen pass without event. Thus, there's little downside to proposing a timeline, even an outrageous one. If it's wrong, it's largely forgotten. If it's right, you're lauded a prophet. The nebulous definitions of "AGI" and "ASI" also offer an out. One can always argue the achieved system doesn't meet their specific definition or point to the AI Effect.

I suppose @gwern [LW · GW]'s fantastic work on The Scaling Hypothesis is evidence of how an accurate prediction can significantly boost credibility and personal notoriety. Proposing timelines gets attention. Anyone noteworthy with a timeline becomes the centre of discussion, especially if their proposal is on the extremes of the spectrum.

The incentives for making timeline predictions seem heavily weighted towards upside, regardless of the actual predictive power or accuracy. Plenty to gain; not much to lose.