LessWrong 2.0 Reader

So, just found out about this.... https://www.lesswrong.com/posts/aqcoFM99GmtRBCXBr/tapatakt-s-shortform?commentId=AFPi3yHYWhpmNRmwd [LW(p) · GW(p)]

This is exactly why I worry that people studying consciousness / self-awareness / valence / qualia in AI are pursuing inherently infohazardous technology. If you publish how to do such things, someone out there will user that to create the torment nexus.

I have no specific comments on the details, I thought it was interesting and learned from it. I disagree in many places, but any counterpoints I might raise have been raised better elsewhere. 

That said, I hope you're right! That would be great. Every time the world turns out to be better geared towards success than we expect, it's a great thing. Zero downside to that outcome except maybe some wasted effort on what turn out to be non-issues.

But also: the thing about fatal risks is that you need to avoid all of them, every time, or you die. If the fatal risk includes extinction, you need very, very high level confidence in success, against all plausible such risks, or your species will go extinct. The alternative is Russian roulette, which you will eventually lose if you keep on playing. In this kind of context, if there is disagreement among experts, or even well-reasoned arguments by outsiders, on whether an extinction risk is plausible, then the confidence in its implausibility is too low to think we should be ok with it. 

In other words, if I were ever somehow in the position of being next to someone about to create an AGI with what I considered dangerous capabilities, and presented with these categories of reasons and arguments for why they are actually safe, I would take Oliver Cromwell's tack: "I beseech you, in the bowels of Christ, think it possible you may be mistaken." Even if you're right, the arguments you've presented for your position are not strong enough to rely on.

I think gratitude also has value in letting you recognize what is worth maintaining and what has historically shown itself to have lots of opportunities and therefore in the future may have opportunities too.

Only if the punishment for non-punishing has no decay factor. With a decay factor, the punishments due for non-punishing decay with each step, and thus reach zero punishment at some point. I think most people imagine a pretty step decay factor, and might think that it drops to zero after the steps, maybe with an exponential curve so that only the first step is a substantial punishment. Like maybe 1st step is 40% of full, then 2nd is 15%, then 3rd is 3%?

Another option after acknowledging the spiral is giving yourself 30 minutes to do nothing, and doing a Gendlin’s Focusing / IFS parts work with your emotional fatigue during that time. Perhaps laziness is guilt from not doing things in the last day or two. Perhaps it’s a symptom of long-term mental exhaustion. Either way, if you manage to find the time and are skilled enough in it, that’s one of the best ways to at the very least better understand what’s happening. In some cases, if it’s a very recent emotion, you can even become motivated and productive again in half an hour.

I think science can be moved forward by a variety of things. Novel hypotheses is one way. Novel observations is another. Novel tools or technology that enable novel observations is another.

We've picked a lot of the long hanging fruit in novel tools for many research areas. A multimillion dollar laser microscope is a hard bar to beat in biology. CERN is a hard bar to beat in physics. James Webb telescope recently upstaged Hubble, and the result was a rapid surge in exciting new astronomy papers fueled by the new observations enabled by this new tool.

So I don't disagree that the lame incentives for predictable incremental grant proposals are part of the problem, but I'm also pretty confident that that isn't the main source of the slowing of scientific progress.

I don't think "We created a platform that lets you make digital minds feel bad and in the trailer we show you that you can do it, but we are in no way morally responsible if you will actually do it" is a defensible position. Anyway, they don't use this argument, only one about digital substrate.

Hmm, I disagree with Taleb's quote.

I think there are three things important to understanding the universe: empiricism, logic/math, and theorizing. I would say that you can do good empiricism without logical extrapolation or theorizing. You can do math and formal logic without doing empirical fact-finding or theorizing. But you should never trust theorizing that is not based in either logic/math or empiricism.

For empiricism which stood the test of time, look at Cajal's neuron drawings which are still revered by scientists today. Those images alone moved neuroscience forward in an important way.

For logic/math, see Einstein's theories which far preceeded the tests which were able to empirically validate them.

For theorizing without either, see the nonsensical ideas put forth by 'armchair geologists' before actual data had been gathered. Unlike Einstein, they did not support their ideas with math.

I asked Claude Sonnet 3.5.

Nathan : I dislike the term sacrifice since it implies a knowing deliberate choice to give up something, specifically for a ceremonial purpose. I think this intentionality is importantly absent in these race dynamics. Indeed, all the parties realizing the nature of the pattern they are in is often the first step to escaping it.

Claude: You make an excellent point. The term "sacrifice" does indeed carry connotations of intentionality and ceremony that are not accurate for this dynamic. You're right to highlight that the lack of awareness is often a crucial part of the problem. Let's reconsider our options with this in mind:

1. Race-based terms:

   • "Destructive Race Dynamic"
   • "Race to Ruin"
   • "Competitive Decline Spiral"

2. Trap-based terms:

   • "Mutual Depletion Trap"
   • "Collective Erosion Trap"
   • "Competitive Degradation Trap"

3. Equilibrium-based terms:

   • "Negative Equilibrium Dynamic"
   • "Deteriorating Equilibrium Trap"
   • "Suboptimal Stability Trap"

4. Feedback-based terms:

   • "Negative Feedback Loop"
   • "Destructive Feedback Cycle"
   • "Erosive Competition Cycle"

5. Other options:

   • "Collective Value Decay"
   • "Competitive Devaluation Dynamic"
   • "Mutual Loss Spiral"
   • "Unintentional Decline Pattern"

These alternatives avoid the implication of intentional sacrifice while still capturing the essence of the dynamic. They emphasize the unintended, gradual nature of the decline and the trapped state of the participants.

Among these, I find "Competitive Decline Spiral" particularly effective. It captures the competitive nature, the worsening situation over time, and avoids any implication of intentionality. "Mutual Depletion Trap" is another strong contender, emphasizing the trapped nature of the situation and the mutual loss without suggesting deliberate action.

"Unintentional Decline Pattern" might be the most explicit in addressing your concern about avoiding implied intentionality, though it's a bit longer and less punchy than some other options.

a method to induce male stem cells to lose their Y chromosome and duplicate the X chromosome, thus becoming female.

If you had twin fertilised eggs, could you use this method to create almost-genetically-identical opposite-sex twins? Which would supposedly let you isolate the effects of sex on the individual?