LessWrong 2.0 Reader

I once saw an advert claiming that a pregnancy test was “over 99% accurate”. This inspired me to invent an only-slightly-worse pregnancy test, which is over 98% accurate. My invention is a rock with “NOT PREGNANT” scrawled on it: when applied to a randomly selected human being, it is right more than 98% of the time. It is also cheap, non-invasive, endlessly reusable, perfectly consistent, immediately effective and impossible to apply incorrectly; this massive improvement in cost and convenience is obviously worth the ~1% decrease in accuracy.

This is fantastic work, I'd be very excited to see more work in the vein of auditing games. It seems like the one of the best ways so far to test how useful different techniques for understanding models are

Thanks. I think it's an important point you make; I do have it in mind that traits can have nonlinearities at different "stages", but I hadn't connected that to the personality trait issue. I don't immediately see a very strong+clear argument for personality traits being super exceptional here. Intuitively it makes sense that they're more "complicated" or "involve more volatile forces" or something due to being mental traits, but a bit more clarity would help. In particular, I don't see the argument being able to support yes significant broadsense heritability but very little apparent narrowsense heritability. (Though maybe it can somehow.)

(Also btw I wouldn't exactly call multiplicativity by itself "nonlinearity"! I would just say that after the genomic fan-in there is a nonlinearity. It's linearity as long as there's a latent variable that's a sum of alleles. Indeed, as E.G. pointed out to me, IQ could very well be like this, i.e. IQ-associated traits might be even better predicted by assuming IQ is lognormal (or some other such distribution). Though of course then you can't say that such and such downstream outcome is linear in genes; but the point would be that you can push along the trait by independently pushing on lots of alleles.)

(Also in theory I ought to go through your whole article at some point; but as yet I'm not read in to polygenic scores, as there's already good enough predictors for the MVP, and biotech is the main bottleneck. Though yes, motivating the biotech depends on understanding PGSes.)

edit: uh, well, short answer there totally is, idk if they're the psychedelic states you wanted, but they should do for a lot of relevant purposes, seems pretty hard to match meds though. original longer version:

there's a huge space of psychedelic states, I think the subspace reachable with adding chemicals is a large volume that's hard to get to by walking state space with only external pushes - I doubt the kind of scraping a hole in the wall from a distance you can do with external input can achieve, eg, globally reversing the function of SERT (I think this paper I just found on google may show this - I forget where I first encountered the claim, not double checking it properly now), that MDMA apparently induces! you can probably induce various kinds of serotonin release, though.

but the premise of my argument here in the first place - where you can sometimes overwhelm human output behavior via well crafted input - is that that probably doesn't matter too much. human computational output bitrate seems to be on order ten bits per second across all modalities,^[1] and input bitrate is way above that, so my guess is that update bitrate (forming memories, etc) is much higher than natural output bitrate^[2], probably yeah you can do most of the weird targeted interventions you were previously getting via psychedelics instead from like, getting some emotional/tempo sorts of things to push into the attractor where neurons have similar functionality already. I just doubt you can go all the way to fixing neurological dysfunctions so severe that to even have a hope of doing it from external input, you'd need to be looking for these crazy brain hacking approaches we were talking about.

I guess what we'd need to measure is like, bitrate of self-correction internally within neurons, some FEP thing. not sure off the top of my head quite how to resolve that to something reasonable.

1. ^{^}

   of course, like, actually I'm pretty dang sure you can get way above 10bit/s by outputting more noisy output, but then you get bits that aren't coming from the whole network's integrated state. the 10bps claim feels right for choosing words or similar things like macroscopic choices, but something feels wrong with the claim to me.

2. ^{^}

   some concern I missed counterevidence to this memorization-bandwidth claim from the paper though!

I'm skeptical.

Did the Sakana team publish the code that their scientist agent used to write the compositional regularization paper? The post says

For our choice of workshop, we believe the ICBINB workshop is a highly relevant choice for the purpose of our experiment. As we wrote in the main text, we selected this workshop because of its broader scope, challenging researchers (and our AI Scientist) to tackle diverse research topics that address practical limitations of deep learning, unlike most workshops with a narrow focus on one topic.

This workshop focuses particularly on understanding limitations of deep learning methods applied to real world problems, and encourages participants to study negative experimental outcomes. Some may criticize our choice of a workshop that encourages discussion of “negative results” (implying that papers discussing negative results are failed scientific discoveries), but we disagree, and we believe this is an important topic.

and while it is true that "negative results" are important to report, "we report a negative result because our AI agent put forward a reasonable and interesting hypothesis, competently tested the hypothesis, and found that the hypothesis was false" looks a lot like "our AI agent put forward a reasonable and interesting hypothesis, flailed around trying to implement it, had major implementation problems, and wrote a plausible-sounding paper describing its failure as a fact about the world rather than a fact about its skill level".

The paper has a few places with giant red flags where it seems that the reviewer assumes that there were solid results that the author of the paper was simply not reporting skillfully, for example in section B2

 , 

I favor an alternative hypothesis: the Sakana agent determines where a graph belongs, what would be on the X and Y axis of that graph, what it expects that the graph would look like, and how to generate that graph. It then generates the graph and inserts the caption the graph would show if its hypothesis was correct. The agent has no particular ability to notice that its description doesn't work with the graph.

 

I have a simple model with toy examples of where non-additivity in personality and other domains comes from, see §4.3.3 here [LW · GW].

There are actually atmospheric microbiota, also called aeroplankton, though those are mostly bacteria.

I remember a novel by Stanisław Lem in which he talks about a planet which has an atmosphere with a green tint. It is caused by swarms of insects in the atmosphere doing photosynthesis. I don't know whether that would be realistic, but it's currently not possible on Earth, insofar no insects, nor any other animal, ever developed the ability to do photosynthesis.

Are you expecting the Cost/Productivity ratio of AGI in the future to be roughly the same as it is for the agents Sam is currently proposing? I would expect that as time passes, the capabilities of such agents will vastly increase while they also get cheaper. This seems to generally be the case with technology, and previous technology had no potential means of self-improving on a short timescale. The potential floor for AI "wages" is also incredibly low compared to humans.

It definitely is worth also keeping in mind that AI labor should be much easier to scale than human labor in part because of the hiring issue, but a relatively high(?) price on initial agents isn't enough to update me away from the massive potential it has to undercut labor.

Dogs would be interesting - super smart working dogs might even have a viable labour market, and it seems like the evidence of supercanine IQ would be obvious in a way that's not true of any other species (just given how much exposure most people have to the range of normal canine intelligence).

Sort of analogous to what Loyal is doing for longevity research.

I can do more projects in parallel than I could have before. Which means that I have even more work now... The support and maintenance costs of the code itself are the same, as long as you maintain constant vigilance to make sure nothing bad gets merged. So the costs are moved from development to review. It's a lot easier to produce thousands of lines of slop which then have to be reviewed and loads of suggestions made. It's easy for bad taste to be amplified, which is a real cost that might not be noticed that much.   

There are some evals which work on large codebases (e.g. "fix this bug in django"), but those are the minority, granted. They can help with the scaffolding, though - those tend to be large projects in which a Claude can help find things.

But yeah, large files are ok if you just want to find something, but somewhere under 500 loc seems to be the limit of what will work well. Though you can get round it somewhat by copying the parts to be changed to a different file then copying them back, or other hacks like that...