LessWrong 2.0 Reader

For my money, the nice properties that human and AI systems have that matter for alignment is IMO not the properties from Shard Theory, but rather several other properties that mattered:

1. Alignment generalizes further than capabilities because of verifying being easier to generate, as well as learning values being easier than having a lot of other real world capabilities.

2. It's looking like the values of humans are far, far simpler than a lot of evopsych literature and Yudkowsky thought, and related to this, values are less fragile than people thought 15-20 years ago, in the sense that values generalize far better OOD than people used to think 15-20 years ago.

3. The brain and DL AIs, while not the same thing, are doing reasonably similar things such that we can transport a lot of AI insights into neuroscience/human brain insights, and vice versa.

4. One of those lessons is the bitter lesson from Sutton applies to human values and morals, which cashes out into the fact that the data matter much more than the algorithm when predicting it's values, especially OOD generalization of values, and thus controlling the data is basically equivalent to controlling the values.

This is an idea and NOT a recommendation. Unintended consequences abound.

Have you thought about sorting into groups based on carefully-selected categories? For example, econ/social sciences vs quant background with extra whiteboard space, a separate group for new arrivals who didn't do the readings from the other weeks (as their perspectives will have less overlap), a separate group for people who deliberately took a bunch of notes and made a concise list vs a more casual easygoing group, etc?

I'd be interested in reading much more about this. Energy and akrasia as it's popularly called here continue to be my biggest life challenges. High fiber diet seems to help, and high novelty seems to help.

Why does the video show up so tiny?

So like... Quadratic voting? https://en.m.wikipedia.org/wiki/Quadratic_voting

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

This writeup is great. Very simple. Beat by beat. Motion by motion. The character of the writing makes me feel like anything was possible, and history was a series of accidents, which I think is a "true feeling" about history.

So far I've not had this issue (i.e, ...that one thing will come back to bite me a few days later...) with my obligations but have had it with some private projects. Infrequently referencing my obligations in to-do list format allows them to stress me adequately, as they occur first on my mind and aren't detracted from by the gravity of the other tasks I have, which would be easily visible and "present" if on a nearby to-do list.

Having a complex task system containing both job-related obligations and private work tasks somewhat deprioritizes the job-related obligations for me ^[1], relative to how I believe most people might prioritize their job-related obligations, and the near absence of my multiple to-do lists has allowed me, of late, to "calibrate my stress" (the levels of stress I mention here are not severe).

Do you find that you're able to mentally keep track of things better than before...

I am better mentally able to keep track of job-related obligations (which, for the time being and as far as I've surmised, are more important than my private projects / tasks) but less able to remember private project tasks. The magnitude of a task that comes to mind naturally is felt more strongly than if I had proceeded through the same task from a list. I've been tackling tasks that cause me higher levels of stress sooner.

See my comment. The problem with the post is revealed in the fourth sentence:

To demonstrate how chaos theory imposes some limits on the skill of an arbitrary intelligence, I will also look at a game: pinball.

Note that predicting a ball is not at all the same thing as skill in manipulating a ball. It's just a giant non sequitur being slipped in before he begins the math. Which is why he is 100% wrong when he concludes

This is not a problem that is solvable by applying more cognitive effort.

It totally is solvable. The 'cognitive effort' here is 'git gud at pinball, scrub, and stop making excuses for losing', and as he admits in the footnote he didn't include in the LW version, in real life, when adequately incentivized to win rather than find excuses involving 'well, chaos theory shows you can't predict ball bounces more than n bounces out', pinball pros learn how to win and rack up high scores despite 'muh chaos'.

And that is why I don't believe your anecdotal survey responses imply anything good. I think that several or all of those cases, if we were able to investigate them adequately, would turn out to be similar to this pinball essay: a lot of browbeating intimidation-by-math, possibly completely valid insofar as it went, but ultimately, proving an irrelevant claim and the problem in fact soluble.

I am not sure whether this is the answer you're looking for, but I think it's true and could be de-confusing, and others have given the standard/practical answer already.

You can try running a program which computes Bayesian updates to determine what happens when this program is passed as input an 'observation' to which it assigns probability 0. Two possible outcomes (of many, dependent on the exact program) that come to mind:

• The program returns a 'cannot divide by 0' error upon attempting to compute the observation's update.
• The program updates on the observation in a way which rules out the entirety of its probability-space, as it was all premised on the non-0 possibilities. The next time the program tries to update on a new observation, it fails to find priors about that observation.

Bayes' theorem is an algorithm which is used because it happens to help predict the world, rather than something with metaphysical status.

We could also imagine very-different (mathematical)-worlds where prediction is not needed/useful, or, maybe, where the world is so differently-structured that Bayes' theorem is not predictive.