LessWrong 2.0 Reader

Right, and that wouldn’t apply to a model-based RL system that could learn an open-ended model of any aspect of the world and itself, right?

I think your “it is nearly impossible for any computationally tractable optimizer to find any implementation for a sparse/distant reward function” should have some caveat that it only clearly applies to currently-known techniques. In the future there could be better automatic-world-model-builders, and/or future generic techniques to do automatic unsupervised reward-shaping for an arbitrary reward, such that AIs could find out-of-the-box ways to solve hard problems without handholding.

@habryka [LW · GW] this comment has an anomalous amount of karma. It showed up on popular comments, I think, and I'm wondering if people liked the comment when they saw it there which lead to a feedback loop of more eyeballs on the comment, more likes, more eyeball etc. If so, is that the intended behaviour of the popular comments feature? It seems like it shouldn't be.

Let's step back. This thread of the conversation is rooted in this claim: "Let's be honest: all fiction is a form of escapism.". Are we snared in the [Disputing Definitions](https://www.lesswrong.com/posts/7X2j8HAkWdmMoS8PE/disputing-definitions [LW · GW]) trap? To quote from that LW article:

> if the issue arises, both sides should switch to describing the event in unambiguous lower-level constituents, like acoustic vibrations or auditory experiences.  Or each side could designate a new word, like 'alberzle' and 'bargulum', to use for what they respectively used to call 'sound'; and then both sides could use the new words consistently.  That way neither side has to back down or lose face, but they can still communicate.  And of course you should try to keep track, at all times, of some testable proposition that the argument is actually about.

I propose that we recognize several lower-level testable claims, framed as questions. How many people read fiction to ...

1. entertain?
2. distract from an unpleasant reality? 
3. understand the human condition (including society)?
4. think through alternative scenarios?

Now I will connect the conversation to these four points:

* Luke_A_Somers: "Why would I ever want to escape from my wonderful life to go THERE?" See point 2.

* Which points is thomblake referring to? Consider this quote from [The Philosophy of Horror](https://www.kentuckypress.com/9780813136554/the-philosophy-of-horror/). "Whether serious, kitschy, frightening, or ridiculous, horror not only arouses the senses but also raises profound questions about fear, safety, justice, and suffering. From literature and urban legends to film and television, horror's ability to thrill has made it an integral part of modern entertainment." From this, it seems they are emphasizing points 1 and 3.

* JonInstall pulls out the dictionary in the hopes of "settling" the debate. He refers to point 1.

* To add some texture to the discussion: when I read the [embedded story](https://en.wikipedia.org/wiki/Story_within_a_story) [The Tale of the Omegas](https://www.reddit.com/r/singularity/comments/12dmufv/the_most_important_short_story_everyone_here/) in [Life 3.0](https://www.penguinrandomhouse.com/books/530584/life-30-by-max-tegmark/), include me in point 4.

Does this sound about right?

I’m not quite sure what you mean by “deeply painful process.” There is often a segment of any community that resists any change. That’s not to say that it has to be a fight, but community practices have an inertia to them. Sometimes that a shift that’s happens over time.

For instance, when I was a kid (1980s), “gay” was a common pejorative. While there have been plenty of painful events that have happened in the lives of LGBT folk, I don’t think that this was due to some process that is deeply painful, other than people slowly changing their minds over time.

I’ve seen the polyamorous community shift best practices over time. Again though, I don’t think that this is due to some inherently painful process. One could argue that the collective pain that we experience as we’re making mistakes is that process, but I suspect that isn’t what you mean here.

I think that change is generally hard, but it naturally happens over time.

I agree that we do not have an exact model for anything in immunology, unlike physics, and there is a huge amount of uncertainty. But that's different than saying it's not well-understood; we have clear gold-standard methods for determining answers, even if they are very expensive. This stands in stark contrast to AI, where we don't have the ability verify that something works or is safe at all without deploying it, and even that isn't much of a check on its later potential for misuse.

But aside from that, I think your position is agreeing with mine much more than you imply. My understanding is that we have newer predictive models which can give uncertain but fairly accurate answers to many narrow questions. (Older, non-ML methods also exist, but I'm less familiar with them.) In your hypothetical case, I expect that the right experts can absolutely give indicative answers about whether a novel vaccine peptide is likely or unlikely to have cross-reactivity with various immune targets, and the biggest problem is that it's socially unacceptable to assert confidence in anything short of tested and verified case. But the models can get, in the case of the Zhang et al paper above, 70% accurate answers, which can help narrow the problem for drug or vaccine discovery, then they do need to be followed with in vitro tests and trials.

Amateur hour question, if you don't mind: how does your "future of AI teams" compare/contrast with Drexler's CAIS model [LW · GW]?

One issue is going to be filtering.

Strife and conflict is memorable. So you are searching for the least noteworthy examples, the ones that people are least likely to comment on or remember.

I don't know what qualifies as a "community" really. At work I have seen uncontroversial changes come in a few times.

Probably not super helpful/what you're looking for, but one broad category of groups who go from 'doing violence' to 'doing much less or no violence' (often within a short space of time) are resistance organisations that successfully manage the transition, usually after achieving some level of progress. The ANC in South Africa seems like a good example. Sinn Féin in Ireland (established as the political wing of the IRA) is another. 

I find this a very suspect detail, though the base rate of cospiracies is very low.

"He wasn't concerned about safety because I asked him," Jennifer said. "I said, 'Aren't you scared?' And he said, 'No, I ain't scared, but if anything happens to me, it's not suicide.'"

https://abcnews4.com/news/local/if-anything-happens-its-not-suicide-boeing-whistleblowers-prediction-before-death-south-carolina-abc-news-4-2024

To be more explicit about my model, I see communities as a bit like people. And sometimes people do the hard work of changing (especially as they have incentives to) but sometimes they ignore it or blame someone else.

Similarly often communties scapegoat something or someone, or give vague general advice.