LessWrong 2.0 Reader

I am with you, I have been trying to find more sources to see if semiconductor-grade quartz if being produced in places like Norway and Brazil, where deposits of high purity quartz are found. But from the research I have done, it seems that roughly 90% of semiconductor-grade quartz produced today is produce at Spruce Pine. Please let me know if you find any sources that say the contrary (I hope I am wrong honestly).

With respects to road disruptions, I agree. If it's a matter of national security, stakeholders will do whatever is necessary to ensure the material is transported out. However road disruptions is one part of the equation. There are other scenarios to consider such as power issues and destruction of the refining plants themselves that we have to take into consideration. 

 

What Oregon Brain Preservation is doing isn’t exactly cryonics in the traditional sense. Most of what they offer is actually aldehyde-based brain preservation, which stores your brain at refrigerator temperatures. They do have a cryonics option, but it's not free—$15,000 for whole-head cryopreservation, or if you’re feeling more minimalist, $5,000 for just the brain. 

Yes agree, unclear what you are saying that is different to me? The new solution is something unique and powerful when done well like language etc.

Ok, I would definitely call the bee response "smart" but thats hard to define. If you define it by an action that costs the bees very little but benefits a lot, then "swarm the scout hornet" is certainly efficient. Another criteria could be if such a behavior was established would it continue? Say the birds developed a "swarm the attacker" call.  When birds hear it, they look to see if they can find the attacker, if they see it then they repeat the call. When the call gets widespread, the whole flock switches to attack. Would such a behavior persist if developed? If it would then lets call it efficient or smart.

The leap to humans is meant to be large - something extreme like consciousness and language is needed to break the communication/coordination penalty for larger organisms with fitness more defined by the individual than the hive.

not including the cost of stand-by, which is also a significant portion (ie. staying at your bedside in advance until you die)

I assumed this was an overstatement. A quick check shows I'm wrong: TomorrowBio offer whole body (€200k) or just brain preservation (€60k). The 'standby, stabilisation and transport' service (included in the previous costs) amount to €80k and €50k respectively. I expected it to be much less. 

That said, they still set aside €10K for long term storage of the head. I guess this means your head has a higher chance of being stored safety.

Agreed.

To understand your usage of the term “outer alignment” a bit better: often, people have a decomposition in mind where solving outer alignment means technically specifying the reward signal/model or something similar. It seems that to you, the writeup of a model-spec or constitution also counts as outer alignment, which to me seems like only part of the problem. (Unless perhaps you mean that model specs and constitutions should be extended to include a whole training setup or similar?)

If it doesn’t seem too off-topic to you, could you comment on your views on this terminology?

One could claim that "the true spirit of friendship" is loving someone unconditionally or something, and that might be simple, but I don't think that's what humans actually implement.

Yeah, I agree that humans implement something more complex. But it is what we want AI to implement, isn't it? And it looks like may be quite natural abstraction to have.

(But again, it's useless while we don't know how to direct AI to the specific abstraction.)

Cool, Facebook is also on this apparently: https://x.com/PicturesFoIder/status/1840677517553791440

yep, something like more carefulness, less “playfulness” in the sense of [Please don't throw your mind away by TsviBT]. maybe bc AI safety is more professionalised nowadays. idk. 

Thinking about this a bit more, I realize I'm confused.

Aren't you arguing that AI will be aligned by default?

I really thought I wasn't before, but now I feel it would only require a simple tweak to the original argument (which might then be proving too much, but I'm interested in exploring more in depth what's wrong with it).

Revised argument: there is at least one very plausible scenario (described in the OP) in which the ASI is being simulated precisely for its willingness to spare us. It's very implausible that it would be simulated for the exact opposite goal, so us not getting spared is, in all but the tiniest subset of cases, an unintended byproduct. Since that byproduct is avoidable with minimal sacrifice of output (of the order of 4.5e-10), it might as well be avoided just in case, given I expect the likelihood of the simulation being run for the purpose described in the OP to be a few orders of magnitude higher, as I noted earlier.

I don't quite see what's wrong with this revised argument, save for the fact that it seems to prove too much and that other people would probably already have thought of it if it were true. Why isn't it true?