LessWrong 2.0 Reader

(I've since found https://www.lesswrong.com/rationality, [? · GW] which does the job.)

As with OP, I strongly recommend Aaronson, who explains why waterfalls aren't doing computation in ways that refute the rock example you discuss: https://www.scottaaronson.com/papers/philos.pdf

I agree with that. I think that the general ick that I get from the dialogue is the presumption and general tone of Bryce. Thanks for clarifying!

You seem to fundamentally misunderstand computation, in ways similar to Searle. I can't engage deeply, but recommend Scott Aaronson's primer on computational complexity: https://www.scottaaronson.com/papers/philos.pdf

You seem deeply confused about computation, in ways similar to Searle et al. I cannot engage deeply on this at present, but recommend Aaronson's primer on the topic: https://www.scottaaronson.com/papers/philos.pdf

How did safety engineering get invented for different disciplines, and how do their invention relate to engineering and theory?

Inspired by davidad's tweets: 1, 2, 3

It seems commonsense that a deeper (theoretical) understanding helps both engineering as well as safety engineering. Which one do you think does theory help more? And which development helped grow theory research more?

My intuition is that:

1. First we started building something by trial-and-error, empirical results.
2. We formulated some safety best practices. But they are all heuristics from the trial-and-error.
3. Then we started gaining theoretical understanding of what we are doing.
4. Only then do we become able to advance "safety engineering".
5. At the same time, we also get much better at building that thing - much better at engineering.

How well does this mesh with real-life? In the bridges' case, safety engineering was invented separately, well after we understood how to build bridges - and well after we built a lot of bridges. The pioneers in safety engineering oft have formal math background. This seems to match the intuition above.

That said -

• We did build a lot of bridges, and a lot of them failed, before safety engineering came about. And how much did theories for safety engineering help with bridge capability?
• Did the field advance by novel theory works? Or was it more about the application of existing theories?
• Related to that question is: did safety engineering require an entirely different set of theories that have little to do with bridge capability? (This seems obviously true to me: for example, environmental wear-and-tear and the process of metal rusting does not affect capability, but we need to understand them for safety.)

Italy outright criminalizes surrogacy. Rarely do we see such extreme moral confusion, or such clear cases of civilizational suicide.

Your periodical reminder that the Italian government meets within walking distance from the Vatican.

The linked article frames this as "the Pope obviously hates homosexual couples and doesn't want them to have children", but I think this quite misses the point. The Catholic Church has been advocating against IVF since forever, and they don't care if most surrogacy-seekers are heterosexual couples. According to doctrine it's morally equivalent to abortion, period.

I'm actually quite surprised that no one else outside Rome has tried to fully ban surrogacy on religious grounds. Is no other government Catholic enough?

One way to "see less of" something you hate is to stop it from being produced, and that may be seen as a better solution than basically averting your eyes from it. Rallying mobs to get whoever produced it fired has proven to be quite effective. 

1. Value learning converges to full alignment by construction: since a value learning AI basically starts with the propositions:
   a) as an AI, I should act fully aligned to human values
   b) I do not fully understand what human value are, or how to act fully aligned to them, so in order to be able to do this I need to learn more about human values and how to act fully aligned to them, by applying approximately Bayesian learning to this problem
   c) Here are some Bayesian priors about what human values are, and how to act fully aligned to them: <insert initialization information here>…
2. As usual for a Bayesian learning problem, as long as the Bayesian priors 1 c) are not completely screwed up as a place to start from, this will converge. Thus there is a region of convergence to full alignment.
3. LLMs have a very large amount of detailed information about what human values are and how to act aligned to them. Thus they provide a very detailed set of Bayesian priors for 1 c).
4. Also, training an LLM is a fairly good approximation to Bayesian learning. Thus (with suitable additions to enable online learning) they provide one possible implementation for the Bayesian learning process required by 1 b). For example, one could apply fine-tuning to the LLM to incorporate new information, and/or periodically retrain the LLM based on the training set plus new information the AI has gathered during the value learning process.

Happened on this song on Tiny Desk: Paperclip Maximizer (by Rosie Tucker, from an album titled "Utopia Now!").

Paperclip maximizer
Single minded if you mind at all
A paragon of puritanical panoptical persistence
Everybody envies your resolve
Paperclip maximizer
Mining for a better way
No ontological contention
Tends your content generation
Every sorrow makes a link in the chain

[...]

And the shareholders meet gruesome ends
But the cosmos expands
So the market survives
All the better to bear all your office supplies
And the space they require was once occupied
By the sun
On your hair
And the curve
Of your thighs
Horizon of sighs
Destroyer of worlds