LessWrong 2.0 Reader

Macroscopic biotech is not software-only singularity (and doesn't depend on it being possible), it's a separate example of a system that is not human industry, that seems sufficient to create a rate of scaling completely implausible for the human economy (illustrating the crux with Ege Erdil's position, which system is relevant for timelines). Software-only singularity operates within an unchanging amount of raw compute, while macroscopic biotech takeoff manufactures new compute much faster than human industry could be instructed to.

The fruit flies are the material substrate, packed with DNA that can specialize the cells to do their part in a larger organism reassembled from these fast-growing and mobile packages of cells, and so precise individual control over the "fruit flies" is unnecessary, only the assembled large biorobots will need to be controllable. The main issue is that even macroscopic biotech might be unable to create efficient compute or fusion plants directly, but they can create integrated factories that manufacture these things in more standard ways by reproducing the whole necessary supply chain starting with raw materials (or whatever is most convenient to transport) on site, plausibly extremely quickly. With enough robots, city-sized factories could be assembled overnight, and then reassembled again an again wholesale, as feedback from trying it out in the real world reveals flaws in design.

I've made a reply formalising this.

I've made a reply formalising this.

I made a reply. You're referring to situation b.

Update based on the replies:

I basically see this as a Markov process.

X(t+1) = P(x(t+1) | x(t), x(t-1), x(t-2), ...) = F(x(t))

where x(t) is a value is sampled from X(t) distribution for all t.

In plain English, given the last value you get a probability distribution for the next value.

In the AI example: Given x(2025), estimate probability distribution X(2030) where x is the AI capability level.

Possibilities

a) x(t+1) value is determined by x(t) value. There is no randomness. No new information is learned from x(t).

b) X(t+1) distribution is conditional on the value of x(t). Learning which value x(t) was sampled from distribution X(t) distribution gives you new information. However you sampled one of those values such that P(x(t+1) | x(t-1), x(t-2), ...) = P(x(t+1) | x(t), x(t-2) ). You got lucky, and the value sampled ensures distribution remains the same.

c) You learned new information and the probability distribution also changed.

a is possible but seems to imply overconfidence to me.

b is possible but seems to imply extraordianry luck to me, especially if it's happening multiple times.

c seems like the most likely situation to me.

This comment seems to be holding up pretty well. Horizon lengths have indeed been increasing and moreover the trend is plausibly superexponential:  https://x.com/DKokotajlo/status/1916520276843782582

Huh, interesting. The sentence you highlighted could also plausibly explain the response about the Wagner group. I found another example and here the prompt includes "## PRE-PROCESSING CHECKLIST (ALWAYS EXECUTE FIRST)", "-TUNISIAN SAUDI BANK", as well as mentions of scanning, validation, identification, etc.

The list of Polish public holidays is still baffling, though. The fact that the response is in Polish is probably due to the web search having access to the user's IP address, but why a list of public holidays?

Basically agree with this, but the caveat here is fruit flies are pretty much pure instinct, and a lot of the nanotech that is imagined is more universalist than that.

But yeah, fruit flies are an interesting case where biotech has managed to get pretty insane doubling times, and if we can pack large amounts of effective compute into very small spaces, this would hugely support something like a software only singularity.

Though I did highlight the possibility of a software-only singularity as the main crux in my post.

Many people are skeptical of nanotech

The best (in my opinion) nanotech skeptical cases are from @Muireall [LW · GW] and @bhauth [LW · GW] below:

https://forum.effectivealtruism.org/posts/oqBJk2Ae3RBegtFfn/my-thoughts-on-nanotechnology-strategy-research-as-an-ea?commentId=WQn4nEH24oFuY7pZy [EA(p) · GW(p)]

https://muireall.space/nanosystems/

https://muireall.space/pdf/considerations.pdf#page=17

https://www.lesswrong.com/posts/FijbeqdovkgAusGgz/grey-goo-is-unlikely [LW · GW]

I think it's unfair to conclude that western philosophy of mind is a train wreck, based on the writings of amateurs like Harth, Graham and Yudkowsky. You need to critique Dennet, Chalmers,Kim, Stawson, etc

There is a similar hypothesis that is testable. Find someone who is illiterate and superstitious today and fund their education upto university level.

Edit: Bonus points if they are selected from an isolated tribal community existing today