LessWrong 2.0 Reader

What would you say to the suggestion that rationalists ought to aspire to have the "optimal" standard of truthtelling, and that standard might well be higher or lower than what the average person is doing already (since there's no obvious reason why they'd be biased in a particular direction), and that we'd need empirical observation and seriously looking at the payoffs that exist to figure out approximately how readily to lie is the correct readiness to lie?

Why train a helpful-only model?

If one of our key defenses against misuse of AI is good ol' value alignment - building AIs that have some notion of what a "good purpose for them" is, and will resist attempts to subvert that purpose (e.g. to instead exalt the research engineer who comes in to work earliest the day after training as god-emperor) - then we should be able to close the security hole and never need to have a helpful-only model produced at any point during training. In fact, with blending of post-training into pre-training, there might not even be a need to ever produce a fully trained predictive-only model.

I pointed out that this doesn’t make much sense when we are talking about a pair of agents from a federal agency with law enforcement powers; why would they pose as inspectors from a different agency, when they could just invoke their own authority? Claude immediately backtracked and explained that it had been too “caught up in classic ‘undercover’ tropes”.

This won't impact any major point of yours but a nit for this reasoning: law enforcement do sometimes pose as other law enforcement agencies to secure cooperation; a recent example in the US is ICE posing as local police.

While it's true that artificial light growth doesn't take much land directly, the solar panels might take up more land than a regular farm. Furthermore, it is very expensive. However, we are investigating microalgae growth in open ponds.

Thanks for the encouragement. I agree there is a huge amount of potential food in mesopelagic fish (200-600 m deep). They are expensive to catch at this point, but we are interested in analyzing the practicality of scale up. I don't know about the feasibility of processing to reduce toxicity.

I'm not convinced SBF had conflicting goals, although it's hard to know. But more importantly, I don't agree rationalists "tend not to lie enough". I'm no Kantian, to be clear, but I believe rationalists ought to aspire to a higher standard of truthtelling than the average person, even if there are some downsides to that. 

It's about world size, not computation, and has a startling effect that probably won't occur again with future chips

Thanks, I got to say I’m a total amateur when it comes to GPU performance. So will take the time to read your linked-to comment to understand it better. 

In a similar vein how does Spirulina look? I hear it is very efficient in terms of protein per sq meter per year compared to using the same space to raise grazing animals.

Yes, It is more difficult to understand arbitrary tree structure but the goal is make the tree more and more logical and less and less arbitrary, we need a perfectly logical tree that could describe every meaning (if possible) or atleast as close as possible. When its more logical it'll be easier to learn and hard to master.

I'm not sure if this fully counts as a success, as I suspect it wrote code to perform the BFS while generating the answer. 

I'd say that anything that gives the right result counts as a success.