Conor Sullivan's Shortform

Why doesn't my device automatically warn me when they have that daylight savings time nonsense? (I refuse to learn whatever is the proper sentence structure for talking about DST.) I just want DST to die, but failing that, it would be nice if my device was more interested in me making meetings on time than notifying me about a school shooting or whatever non-news my device shoves in my face. (Okay rant over)

I'm happy to report that I'm now working on a project related to AI safety. The exact details will be announced sometime in the future. In totally unrelated news, I'm less optimistic about alignment than I was before (but still not pessimistic.) What was it that Mark Twain said about man's beliefs and paychecks and AI alignment?

In 2003, a computer worm infected 90% of vulnerable hosts on the entire internet in less than 10 minutes. Wikipedia indentifies such fast spreading malware as a Warhol worm.

↑ comment by Lone Pine (conor-sullivan) · 2022-06-25T16:52:13.811Z · LW(p) · GW(p)

I wish Wikipedia mentioned how the world managed to recover from this bug. It sounds like it should have caused the internet to collapse.

Replies from: anonymousaisafety, conor-sullivan

↑ comment by anonymousaisafety · 2022-06-25T17:31:21.645Z · LW(p) · GW(p)

First, it only targeted Windows machines running an Microsoft SQL Server reachable via the public internet. I would not be surprised if ~70% or more theoretically reachable targets were not infected because they ran some other OS (e.g. Linux) or server software instead (e.g. MySQL). This page makes me think the market share was actually more like 15%, so 85% of servers were not impacted. By not impacted, I mean, "not actively contributing to the spread of the worm". They were however impacted by the denial-of-service caused by traffic from infected servers.

Second, the UDP port (1434) that the worm used could be trivially blocked. I have discussed network hardening in many of my posts. The easiest way to prevent yourself from getting hacked is to not let the hacker send traffic to you -- blocking IP ranges, ports, unneeded Ethernet or IP protocols, and other options available in both network hardware (routers) or software firewalls provides a low cost and highly effective way to do so. This contained the denial-of-service.

Third, the worm's attack only persisted in RAM, so the only thing a host had to do was restart the infected application. Combined with the second point, this would prevent the machine from being reinfected.

This graph^[1] shows the result of wide-spread adoption of filter rules within hours of the attack being detected

1. ^{^}

   https://cseweb.ucsd.edu//~savage/papers/IEEESP03.pdf

Replies from: conor-sullivan

↑ comment by Lone Pine (conor-sullivan) · 2022-06-25T17:49:21.077Z · LW(p) · GW(p)

Thanks, very informative.

↑ comment by Lone Pine (conor-sullivan) · 2022-06-25T17:19:00.852Z · LW(p) · GW(p)

It gets wilder: "As I write this in 2010, Slammer is still out there, nearly eight years after release, still doing the rounds, so to speak. This is at worst an annoyance as, fortunately, Slammer had no destructive payload but it does suggest that there are still unpatched SQL and MSDE installs out there. This is incredible to me but unpatched systems are definitely few and far between."

Two quick points about DOOM:

1. It is not necessary for the mad AI to kill every single human on the planet. The AI only needs to destroy our ability to respond. In order for human civilization to respond to such a threat we need access to technology, infrastructure and functional social structures (such as the military). The AI could defeat us without killing a single person, if it targets industry and infrastructure. (EY's melt-all-GPUs nanotech concept could easily be inverted: melt all rubber tires so no one can get to work, for example.)

2. It is entirely possible that we could see a full Kurzweilian singularity where everything is utopic, and then 20 years later (say) we all drop dead in the same instant. If the AI is both deceptive and patient, it might decide to wait and build up resources, integrate automation (which it can control) into our lives, and manipulate us into thinking everything is okay. Therefore, it is important that we don't drop our vigilance when we enter a seemingly utopic situation. Ideally we will repeatedly prove to ourselves, and teach every new generation of humans to prove to themselves, that the AI is both values-aligned and corrigible.