LessWrong 2.0 Reader

@Nikola Jurkovic [LW · GW] I'd be interested in timeline estimates for something along the lines of "AI that substantially increases AI R&D". Not exactly sure what the right way to operationalize this is, but something that says "if there is a period of Rapid AI-enabled AI Progress, this is when we think it would occur."

(I don't really love the "95% of fully remote jobs could be automated frame", partially because I don't think it captures many of the specific domains we care about (e.g., AI-enabled R&D, other natsec-relevant capabilities) and partly because I suspect people have pretty different views of how easy/hard remote jobs are. Like, some people think that lots of remote jobs today are basically worthless and could already be automated, whereas others disagree. If the purpose of the forecasting question is to get a sense of how powerful AI will be, the disagreements about "how much do people actually contribute in remote jobs" seems like unnecessary noise.)

(Nitpicks aside, this is cool and I appreciate you running this poll!)

I imagined "FocusMate + TaskRabbit" specifically to address this issue.

Three types of workers I'm imagining here:

• People who are reasonable skilled types, but who are youngish and haven't landed a job yet.
• People who actively like doing this sort of work and are good at it
• People who have trouble getting/keeping a fulltime job for various reasons (which would land them in the "unreliable" sector), but... it's FocusMate/TaskRabbit, they don't need to be reliable all the time, there just needs to be one of them online who responds to you within a few hours, who is at least reasonably competent when they're sitting down and paying attention. 

And then there are reviews (which I somehow UI design to elicit honest reactions, rather than just slapping a 0-5 stars rating which everyone feels obligated to rate "5" all the time unless something was actively wrong"), and they have profiles about what they think they're good at and what others thought they were good at.

(where an expectation is, if you don't have active endorsementss, if you haven't yet been rated you will probably charge a low rate)

Meanwhile if you're actively good and actively reliable, people can "favorite" you and work out deals where you commit to some schedule.

Note I'm not talking about moral weight here, and my point here is that all discussions about counterfactuals (especially human intuitions around counterfactuals) could in principle be executable/actually doable with enough compute and the ability to specify details, so counterfactability/counterfactual worlds isn't special from a philosophical perspective, as it implicitly refers to other real worlds/universes.

Of course, this isn't the only way to do so for a large class of counterfactuals/counterfactual worlds, and sometimes you can run them fully accurately on less compute/data if you can identify simplicities/abstractions that are lossless.

1. Thanks so much for these resources, interesting!!
2. Im not saying my only goal is to combat misinformation, but that logical discourse combats misinformation (as well as hegemonic propaganda) as a matter of course.

I think explicitly computing details in full (as opposed to abstract reasoning about approximate properties) has no bearing on moral weight (degree of being real), but some kind of computational irreducibility forces the simulation of interesting things to get quite close to low level detail in order to figure out most global facts about what's going on there, such as values/culture of people living in a world after significant time passes.

I find myself linking back to this often. I don't still fully endorse quite everything here, but the core messages still seem true even with things seeming further along.

I do think it should likely get updated soon for 2025.

Can't CoT's be what makes RL safe, however? (if you force the reasoner to self-limit under some recursion depth when it senses that the RL agent might be asking for so much that it makes it unsafe)

My girlfriend and I probably wouldn't have got together if not for a conversation at Less Wrong Community Weekend.

One argument I have been making publicly is that I think Ajeya's Bioanchors report greatly overestimated human brain compute. I think a more careful reading of Joe Carlsmith's report that hers was based on supports my own estimates of around 1e15 FLOPs.

Am I getting things mixed up, or isn’t that just exactly Ajeya’s median estimate? Quote from the report: ”Under this definition, my median estimate for human brain computation is ~1e15 FLOP/s.”

https://docs.google.com/document/d/1IJ6Sr-gPeXdSJugFulwIpvavc0atjHGM82QjIfUSBGQ/edit

On this issue specifically, I feel like the bar for what counts as an actually sane and non-dysfunctional organization to the average user of this website is probably way too lofty for 95% of workplaces out there (to be generous!) so it's not even that strange that it would be the case.