Plausible A.I. Takeoff Scenario Short Story 2020-01-01T04:35:41.273Z

Comment by hunterjay on What a 20-year-lead in military tech might look like · 2020-07-30T02:12:24.404Z · LW · GW

I really enjoy'd this writeup! I'd probably even go a little bit on the pessimistic (optimistic?) side, and bet that almost all of this technology would be possible with only a few years of development from today -- though I suppose it might be 20 if development doesn't start/ramp up in earnest.

Comment by hunterjay on Plausible A.I. Takeoff Scenario Short Story · 2020-01-02T01:22:00.130Z · LW · GW

Thanks!

Comment by hunterjay on Plausible A.I. Takeoff Scenario Short Story · 2020-01-01T12:07:21.613Z · LW · GW

That's a good point, I'll write up a brief explanation/disclaimer and put it in as a footnote.

Comment by hunterjay on Plausible A.I. Takeoff Scenario Short Story · 2020-01-01T11:29:42.481Z · LW · GW

Typo corrected, thanks for that.

I agree, it's more likely for the first AGI to begin on a supercomputer at a well-funding institution. If you like, you can imagine that this AGI is not the first, but simply the first not effectively boxed. Maybe its programmer simply implemented a leaked algorithm that was developed and previously run by a large project, but changed the goal and tweaked the safeties.

In any case, it's a story, not a prediction, and I'd defend it as plausible in that context. Any story has a thousand assumptions and events that, in sequence, reduce the probability to infinitesimal. I'm just trying to give a sense of what a takeoff could be like when there is a large hardware overhang and no safety -- both of which have only a small-ish chance of occurring. That in mind, do you have an alternative suggestion for the title?