LessWrong 2.0 Reader

Sure, electrons are not a fluid; but they’re not really particles either according to quantum mechanics

Sure they are. They're a particle; the word "particle" just doesn't mean quite what you expect it to mean. They can be a fluid: that's why we can talk about electron gases and electron fluids; just not a classical fluid.

What they are not, though, is a "substance" in the way chemists (modern or phlogiston-era) think of the term. Identifying electrons (or heat, or free energy, or etc. etc.) with phlogiston feels akin to saying Democritus discovered the periodic table. Is there any useful insight that informed later generations working with more data and better models? Absolutely! But we have three steps (phlogiston, compositional chemistry, quantum mechanics), each of which captures more of the reality than the step before. Strictly more, I think, in terms of predictive capability. Is there any empirical question the phlogiston theorists got right that compositional chemistry did not? AFAIK, no, but it's a real question and I'd like to know if I'm wrong here. But if I'm right, it means whatever useful model content is being added by understanding electrons has only an aesthetic connection to phlogiston at best, rather than re-introducing a once-known-but-ignored (or forgotten) truth.

We can only value lives at $10 million when we have limited opportunities to make that trade, or we’d go bankrupt.

I'm suspicious of the implication that we have many such opportunities. But a quick check suggests says it's very dependent on guesses as to how many lives are saved be treatments.

I did a crude check for lives saved by cancer treatments. Optimistic estimates suggest that lives are being saved at less than $1 million per life. Robin Hanson's writings have implied that the average medical treatments is orders of magnitude less effective than that.

So, just found out about this.... https://www.lesswrong.com/posts/aqcoFM99GmtRBCXBr/tapatakt-s-shortform?commentId=AFPi3yHYWhpmNRmwd [LW(p) · GW(p)]

This is exactly why I worry that people studying consciousness / self-awareness / valence / qualia in AI are pursuing inherently infohazardous technology. If you publish how to do such things, someone out there will user that to create the torment nexus.

I have no specific comments on the details, I thought it was interesting and learned from it. I disagree in many places, but any counterpoints I might raise have been raised better elsewhere. 

That said, I hope you're right! That would be great. Every time the world turns out to be better geared towards success than we expect, it's a great thing. Zero downside to that outcome except maybe some wasted effort on what turn out to be non-issues.

But also: the thing about fatal risks is that you need to avoid all of them, every time, or you die. If the fatal risk includes extinction, you need very, very high level confidence in success, against all plausible such risks, or your species will go extinct. The alternative is Russian roulette, which you will eventually lose if you keep on playing. In this kind of context, if there is disagreement among experts, or even well-reasoned arguments by outsiders, on whether an extinction risk is plausible, then the confidence in its implausibility is too low to think we should be ok with it. 

In other words, if I were ever somehow in the position of being next to someone about to create an AGI with what I considered dangerous capabilities, and presented with these categories of reasons and arguments for why they are actually safe, I would take Oliver Cromwell's tack: "I beseech you, in the bowels of Christ, think it possible you may be mistaken." Even if you're right, the arguments you've presented for your position are not strong enough to rely on.

I think gratitude also has value in letting you recognize what is worth maintaining and what has historically shown itself to have lots of opportunities and therefore in the future may have opportunities too.

Only if the punishment for non-punishing has no decay factor. With a decay factor, the punishments due for non-punishing decay with each step, and thus reach zero punishment at some point. I think most people imagine a pretty step decay factor, and might think that it drops to zero after the steps, maybe with an exponential curve so that only the first step is a substantial punishment. Like maybe 1st step is 40% of full, then 2nd is 15%, then 3rd is 3%?

Another option after acknowledging the spiral is giving yourself 30 minutes to do nothing, and doing a Gendlin’s Focusing / IFS parts work with your emotional fatigue during that time. Perhaps laziness is guilt from not doing things in the last day or two. Perhaps it’s a symptom of long-term mental exhaustion. Either way, if you manage to find the time and are skilled enough in it, that’s one of the best ways to at the very least better understand what’s happening. In some cases, if it’s a very recent emotion, you can even become motivated and productive again in half an hour.

I think science can be moved forward by a variety of things. Novel hypotheses is one way. Novel observations is another. Novel tools or technology that enable novel observations is another.

We've picked a lot of the long hanging fruit in novel tools for many research areas. A multimillion dollar laser microscope is a hard bar to beat in biology. CERN is a hard bar to beat in physics. James Webb telescope recently upstaged Hubble, and the result was a rapid surge in exciting new astronomy papers fueled by the new observations enabled by this new tool.

So I don't disagree that the lame incentives for predictable incremental grant proposals are part of the problem, but I'm also pretty confident that that isn't the main source of the slowing of scientific progress.

I don't think "We created a platform that lets you make digital minds feel bad and in the trailer we show you that you can do it, but we are in no way morally responsible if you will actually do it" is a defensible position. Anyway, they don't use this argument, only one about digital substrate.

Hmm, I disagree with Taleb's quote.

I think there are three things important to understanding the universe: empiricism, logic/math, and theorizing. I would say that you can do good empiricism without logical extrapolation or theorizing. You can do math and formal logic without doing empirical fact-finding or theorizing. But you should never trust theorizing that is not based in either logic/math or empiricism.

For empiricism which stood the test of time, look at Cajal's neuron drawings which are still revered by scientists today. Those images alone moved neuroscience forward in an important way.

For logic/math, see Einstein's theories which far preceeded the tests which were able to empirically validate them.

For theorizing without either, see the nonsensical ideas put forth by 'armchair geologists' before actual data had been gathered. Unlike Einstein, they did not support their ideas with math.