LessWrong 2.0 Reader

All the examples of "RL" doing interesting things that look like they involve sparse/distant reward involve enormous amounts of implicit structure of various kinds, like powerful world models.

I guess when you say “powerful world models”, you’re suggesting that model-based RL (e.g. MuZero) is not RL but rather “RL”-in-scare-quotes. Was that your intention?

I’ve always thought of model-based RL is a central subcategory within RL, as opposed to an edge-case.

For those who don't get the joke: benzos are depressants, and will (temporarily) significantly reduce your cognitive function if you take enough to have amnesia.

But Eric Neyman's post suggests that benzos don't significantly reduce performance on some cognitive tasks (e.g. Spelling Bee)

There was an era in a scientific community where they were interested in the "kinds of learning and memory that could happen in de-corticated animals" and they sort of homed in on the basal ganglia (which, to a first approximation "implements habits" (including bad ones like tooth grinding)) as the locus of this "ability to learn despite the absence of stuff you'd think was necessary for your naive theory of first-order subjectively-vivid learning".

(The cerebellum also probably has some "learning contribution" specifically for fine motor skills, but it is somewhat selectively disrupted just by alcohol: hence the stumbling and slurring. I don't know if anyone yet has a clean theory for how the cerebellum's full update loop works. I learned about alcohol/cerebellum interactions because I once taught a friend to juggle at a party, and she learned it, but apparently only because she was drunk. She lost the skill when sober.)

Yeah I actually do cite that piece in the appendix 'GDP as a proxy for welfare [LW · GW]' where I list more literature like this. So yeah, it's not a perfect measure but it's the one we have and 'all models are wrong but some are useful' and GDP is quite a powerful predictor of all kinds of outcomes: 

In a 2016 paper, Jones and Klenow used measures of consumption, leisure, inequality, and mortality, to create a consumption-equivalent welfare measure that allows comparisons across time for a given country, as well as across countries.^[6] 

This measure of human welfare suggests that the true level of welfare of some countries differs markedly from the level that might be suggested by their GDP per capita. For example, France’s GDP per capita is around 60% of US GDP per capita.^[7] However, France has lower inequality, lower mortality, and more leisure time than the US. Thus, on the Jones and Klenow measure of welfare, France’s welfare per person is 92% of US welfare per person.^[8]

Although GDP per capita is distinct from this expanded welfare metric, the correlation between GDP per capita and this expanded welfare metric is very strong at 0.96, though there is substantial variation across countries, and welfare is more dispersed (standard deviation of 1.51 in logs) than is income (standard deviation of 1.27 in logs).^[9]

GDP per capita is also very strongly correlated with the Human Development Index, another expanded welfare metric.^[10] If measures such as these are accurate, this shows that income per head explains most of the observed cross-national variation in welfare. It is a distinct question whether economic growth explains most of the observed variation across individuals in welfare. It is, however, clear that it explains a substantial fraction of the variation across individuals.

Surely asking if anything is safer is only sensible when comparing it to something. Are you comparing it to some implicit expected-if-not RL method of alignment? I don't think we have a commonly shared concept of what that would be. That's why I'm pointing to some explicit alternatives in that post.

Interestingly, many cancer "neoantigens" (for example, MAGEB1) are also expressed in meiotic cells in the testis. This is because they're usually epigenetically suppressed in healthy tissues, but cancer cells have messed up epigenomes. See: https://en.wikipedia.org/wiki/Cancer/testis_antigens

Also, I would disagree that synthesizing long polypeptides is easier than synthesizing long mRNAs. With polypeptides you have 20 amino acids to work with, and some require special treatment (protecting groups on sidechains). With mRNAs the chemistry is much simpler.

I know LW is US/ California heavy, but just as a counter to all the sunscreen advocates here, daily sunscreen use is probably unnecessary, and possibly actively harmful, in winter and/or at northern latitudes. 

There doesn't seem to be much data on using sunscreen when there's no real risk to skin, but you can find a modelling study here:

"There is little biological justification in terms of skin health for applying sunscreen over the 4–6 winter months at latitudes of 45° N and higher (most of Europe, Canada, Hokkaido, Inner Mongolia etc.) whereas year-round sunscreen is advised at latitudes of 30° N (e.g. Southern U.S., Shanghai, North Africa) and lower ... Using products containing UV filters over the winter months at more northerly latitudes could lead to a higher number of people with vitamin D deficiency."

Although most approved sunscreens are generally seen as safe, there are potential systemic health risks from a few products, some proven environmental harms, a potentially increased risk of vit-D deficiency, and some time/financial costs. 

Wait, you know smart people who have NOT, at some point in their life: (1) taken a psychedelic NOR (2) meditated, NOR (3) thought about any of buddhism, jainism, hinduism, taoism, confucianisn, etc???

To be clear to naive readers: psychedelics are, in fact, non-trivially dangerous.

I personally worry I already have "an arguably-unfair and a probably-too-high share" of "shaman genes" and I don't feel I need exogenous sources of weirdness at this point.

But in the SF bay area (and places on the internet memetically downstream from IRL communities there) a lot of that is going around, memetically (in stories about) and perhaps mimetically (via monkey see, monkey do).

The first time you use a serious one you're likely getting a permanent modification to your personality (+0.5 stddev to your Openness?) and arguably/sorta each time you do a new one, or do a higher dose, or whatever, you've committed "1% of a personality suicide" by disrupting some of your most neurologically complex commitments.

To a first approximation my advice is simply "don't do it".

HOWEVER: this latter consideration actually suggests: anyone seriously and truly considering suicide should perhaps take a low dose psychedelic FIRST (with at least two loving tripsitters and due care) since it is also maybe/sorta "suicide" but it leaves a body behind that most people will think is still the same person and so they won't cry very much and so on?

To calibrate this perspective a bit, I also expect that even if cryonics works, it will also cause an unusually large amount of personality shift. A tolerable amount. An amount that leaves behind a personality that similar-enough-to-the-current-one-to-not-have-triggered-a-ship-of-theseus-violation-in-one-modification-cycle. Much more than a stressful day and then bad nightmares and a feeling of regret the next day, but weirder. With cryonics, you might wake up to some effects that are roughly equivalent to "having taken a potion of youthful rejuvenation, and not having the same birthmarks, and also learning that you're separated-by-disjoint-subjective-deaths from LOTS of people you loved when you experienced your first natural death" for example.This is a MUCH BIGGER CHANGE than just having a nightmare and a waking up with a change of heart (and most people don't have nightmares and changes of heart every night (at least: I don't and neither do most people I've asked)).

Remember, every improvement is a change, though not every change is an improvement [LW · GW]. A good "epistemological practice" is sort of a idealized formal praxis for making yourself robust to "learning any true fact" and changing only in GOOD ways from such facts.

A good "axiological practice" (which I don't know of anyone working on except me (and I'm only doing it a tiny bit, not with my full mental budget)) is sort of an idealized formal praxis for making yourself robust to "humanely heartful emotional changes"(?) and changing only in <PROPERTY-NAME-TBD> ways from such events.

(Edited to add: Current best candidate name for this property is: "WISE" but maybe "healthy" works? (It depends on whether the Stoics or Nietzsche were "more objectively correct" maybe? The Stoics, after all, were erased and replaced by Platonism-For-The-Masses (AKA "Christianity") so if you think that "staying implemented in physics forever" is critically important then maybe "GRACEFUL" is the right word? (If someone says "vibe-alicious" or "flowful" or "active" or "strong" or "proud" (focusing on low latency unity achieved via subordination to simply and only power) then they are probably downstream of Heidegger and you should always be ready for them to change sides and submit to metaphorical Nazis, just as Heidegger subordinated himself to actual Nazis without really violating his philosophy at all.)))

I don't think that psychedelics fits neatly into EITHER category. Drugs in general are akin to wireheading, except wireheading is when something reaches into your brain to overload one or more of your positive-value-tracking-modules [? · GW], (as a trivially semantically invalid shortcut to achieving positive value "out there" in the state-of-affairs that your tracking modules are trying to track) but actual humans have LOTS of <thing>-tracking-modules and culture and science barely have any RIGOROUS vocabulary for any them.

Note that many of these neurological <thing>-tracking-modules were evolved.

Also, many of them will probably be "like hands" in terms of AI's ability to model them.

This is part of why AI's should be existentially terrifying to anyone who is spiritually adept.

AI that sees the full set of causal paths to modifying human minds will be "like psychedelic drugs with coherent persistent agendas". Humans have basically zero cognitive security systems. Almost all security systems are culturally mediated, and then (absent complex interventions) lots of the brain stuff freezes in place around the age of puberty, and then other stuff freezes around 25, and so on. This is why we protect children from even TALKING to untrusted adults: they are too plastic and not savvy enough. (A good heuristic for the lowest level of "infohazard" is "anything you wouldn't talk about in front of a six year old".)

Humans are sorta like a bunch of unpatchable computers, exposing "ports" to the "internet", where each of our port numbers is simply a lightly salted semantic hash of an address into some random memory location that stores everything, including our operating system.

Your word for "drugs" and my word for "drugs" don't point to the same memory addresses in the computer's implementing our souls. Also our souls themselves don't even have the same nearby set of "documents" (because we just have different memories n'stuff)... but the word "drugs" is not just one of the ports [LW · GW]... it is a port that deserves a LOT of security hardening.

The bible said ~"thou shalt not suffer a 'pharmakeia' to live" for REASONS.

There are a few issues with that.

1. The cost would probably be a significant fraction of the development of a new monoclonal antibody treatment, making this currently probably limited to billionaires.

2. Personalized drug development on that scale isn't something that can simply be purchased, and if billionaires tried, governments would probably block them, because voters would consider that unfair, and because cancer researchers can't simply be increased in proportion to budgets. There are only so many people with the relevant skills and inclinations.

3. Better methods for development wouldn't just reduce costs, but would also reduce time taken. There would need to be a billionaire, diagnosed with cancer without a good treatment, who would clearly die from it, but not for a couple years.

4. Better methods and understanding wouldn't just reduce costs and time, they'd also reduce risks. Targeting a receptor that's actually important normally but wasn't fully understood could kill someone before the cancer.

This looks cool and I  want to read it in detail, but I'd like to push back a bit against an implicit take that I thought was present here: namely, that GDP takes into account major technological breakthroughs. Let me just quote some text from this article: What Do GDP Growth Curves Really Mean? [LW · GW]
 

More generally: when the price of a good falls a lot, that good is downweighted (proportional to its price drop) in real GDP calculations at end-of-period prices.

… and the way we calculate real GDP in practice is to use prices from a relatively recent year. We even move the reference year forward from time to time, so that it’s always near the end of the period when looking at long-term growth.

Real GDP Mainly Measures The Goods Which Are Revolutionized Least

Now let’s go back to our puzzle about growth since 1960, and electronics in particular.

The cost of a transistor has dropped by a stupidly huge amount since 1960 - I don’t have the data on hand, but let’s be conservative and call it a factor of 10^12 (i.e. a trillion). If we measure in 1960 prices, the transistors on a single modern chip would be worth billions. But instead we measure using recent prices, so the transistors on a single modern chip are worth… about as much as a single modern chip currently costs. And all the world’s transistors in 1960 were worth basically-zero.

1960 real GDP (and 1970 real GDP, and 1980 real GDP, etc) calculated at recent prices is dominated by the things which are expensive today - like real estate, for instance. Things which are cheap today are ignored in hindsight, even if they were a very big deal at the time.

In other words: real GDP growth mostly tracks production of goods which aren’t revolutionized. Goods whose prices drop dramatically are downweighted to near-zero, in hindsight.

When we see slow, mostly-steady real GDP growth curves, that mostly tells us about the slow and steady increase in production of things which haven’t been revolutionized. It tells us approximately-nothing about the huge revolutions in e.g. electronics.