LessWrong 2.0 Reader

Awesome, thank you! I'm not sure if we're going to correct this; it's a pain in the butt to fix, especially in the YouTube version, and Elizabeth (who has been doing all the editing herself) is sick right now.

who

redundant "who"s in bullets

I think normally "agile" would fulfill the same function (per its etymology), but it's very entangled with agile software engineering.

Very nice piece, and thank you for your service

"cloudy brother of bacteria" should probably be "cloudy broth of bacteria".

Do you understand mathematically what operation we're doing when we say two species or organisms have xx% similar genomes? Each genome is, I guess, several sequences of ATCG, but how do you get a percent similarity for two sequences of different lengths?

This is similar to the answer I got from o1-preview in ChatGPT when I originally asked with OP's post as the text, so that's pleasant to see. (I didn't post anything here because I was unsure and wasn't checking it in enough detail to repost, and so didn't believe in publishing it [LW(p) · GW(p)] without being able to improve it.)

I thought there might be some relationship at first with an appropriate transformation, but when I recalled how Kelly requires both edge and net worth, and the problem of frequency of payoffs, I lost my confidence that there would be any simple elegant relationship beyond a simple 'more information = more returns'. Why indeed would you expect 1 bit of information to be equally valuable for maximizing expected log growth in eg. both a 50:50 shot and a 1,000,000,000:1 shot? (Another way to think of it: suppose you have 1 bit of information on both over the market and you earn the same amount. How many trades would it take before your more informed trade ever made a difference? In the first case, you quickly start earning a return and can compound that immediately; in the second case, you might live a hundred lives without ever once seeing a payoff.)

There the main bottleneck is the iteration of selection, or making synthetic genomes. Going for the most typical genome with the least amount of originality is not a technical challenge in itself, right?

Right.

If you are doing genome synthesis, you aren't frustrated by the rare variant problems as much because you just aren't putting them in in the first place; therefore, there is no need to either identify the specific ones you need to remove from a 'wild' genome nor make highly challenging edits. (This is the 'modal genome' baseline. I believe it has still not been statistically modeled at all.)

While if you are doing iterated embryo selection, you can similarly rely mostly on maximizing the common SNPs, which provide many SDs of possible improvement, and where you have poor statistical guidance on a variant, simply default to trying to select out against them and move towards a quasi-modal genome. (Essentially using rare-variant count as a tiebreaker and slowly washing out all of the rare variants from your embryo-line population. You will probably wind up with a lot in the final ones anyway, but oh well.)

Perhaps this isn’t in scope, but if I were designing a reading list on “lab governance”, I would try to include at least 1-2 perspectives that highlight the limitations of lab governance, criticisms of focusing too much on lab governance, etc.

Specific examples might include criticisms of RSPs, Kelsey’s coverage of the OpenAI NDA stuff, alleged instances of labs or lab CEOs misleading the public/policymakers, and perspectives from folks like Tegmark and Leahy (who generally see a lot of lab governance as safety-washing and probably have less trust in lab CEOs than the median AIS person).

(Perhaps such perspectives get covered in other units, but part of me still feels like it’s pretty important for a lab governance reading list to include some of these more “fundamental” critiques of lab governance. Especially insofar as, broadly speaking, I think a lot of AIS folks were more optimistic about lab governance 1-3 years ago than they are now.)

just read his post.  interesting to see someone have the same train of thought starting out, but then choose different aspects to focus on.

Any non-local behaviour by the neurons shouldn't matter if the firing patterns are replicated.  I think focusing on the complexity required by the replacement neurons is missing the bigger picture.  Unless the contention is that the signals that arrive at the motor neurons have been drastically affected by some other processes, enough so that they overrule some long-held understanding of how neurons operate, they are minor details.

"The third assumption is one you don't talk about, which is that switching the substrate without affecting behavior is possible. This assumption does not hold for physical processes in general; if you change the substrate of a plank of wood that's thrown into a fire, you will get a different process. So the assumption is that computation in the brain is substrate-independent"
Well, this isn't the assumption, it's the conclusion (right or wrong).  It appears from what I can tell is that the substrate is the firing patterns themselves.

I haven't delved too deeply into Penrose's stuff for quite some time.  What I read before doesn't seem to explain how quantum effects are going to influence action potential propagation on a behaviour-altering scale.  It seems like throwing a few teaspoons of water at a tidal wave to try to alter its course.



 

I will revise the post when I get a chance because this is a common interpretation of what I said, which wasn't my intent.   My assertion isn't "if someone or something claims sentience, it must definitely actually be sentient".  Instead we are meant to start with the assumption that the person at the start of the experiment is definitely sentient, and definitely being honest about it.  Then the chain of logic starts from that baseline.

thank you kindly.  I had heard about a general neuron replacement thought experiment before as sort of an open question.  What I was hoping to add here is the specific scenario of this experiment done on someone who begins the experiment as definitively sentient, and they are speaking of their own sentience.  This fills in a few holes and answers a few questions that I think lead us to a conclusion rather than a question