LessWrong 2.0 Reader

This was an interesting read and points to a simple truth that I think is often forgotten: Newton's first law applies to basically everything in life, not just physical systems. The "resets" you describe are definitely valid but by no means a comprehensive list of "opposing" forces that can help drive you in the other direction to reverse your momentum (in a positive way). The two other main ones that I believe are missing, yet fundamental are: 

- Diet: the food we eat affects our mental/emotional tendencies to procrastinate vs get things done through pretty intricate biological + neuroscientific mechanisms
- Exercise: similar to diet, but perhaps harder to get started 

All these can be thought of as different "forces" that can influence our momentum in one way or the other. Side note: it would be interesting to develop some sort of grounded pyschological theory on how different external stimuli affect our mindspace. Some of it is covered in the Vedas (https://en.wikipedia.org/wiki/Gu%E1%B9%87a). 

Thank you for the answer. I notice I feel somewhat confused, and that I regard the notion of "real values" with some suspicion I can't quite put my finger on. Regardless, an attempted definition follows.

Let a subject observation set be a complete specification of a subject and it's past and current environment, from the subject's own subjectively accessible perspective. The elements of a subject observation set are observations/experiences observed/experienced by its subject.

Let O be the set of all subject observation sets.

Let a subject observation set class be a subset of O such that all it's elements specify subjects that belong to an intuitive "kind of subject": e.g. humans, cats, parasitoid wasps.

Let V be the set of all (subject_observation_set, subject_reward_value) tuples. Note that all possible utility functions of all possible subjects can be defined as subsets of V, and that 
V = O x ℝ.

Let "real human values"  be the subset of V such that all subject_observation_set elements belong to the human subject observation set class.^[1]
 

... this above definition feels pretty underwhelming, and I suspect that I would endorse a pretty small subset of "real human values" as defined above as actually good.

1. ^{^}

   Let the reader feel free take the political decision of restricting the subject observation set class that defines "real human values" to sane humans.

On second thought, I agree that gazing at the cosmos is not a fair comparison: rather, I would compare mechanistic interpretability to the early experiments of the Dutch microbiologist van Leeuwenhoek as he first looked at protozoa and bacteria under a microscope.. They weren't the most accurate or informative experiments in the large scheme of things, but they were necessary for others to develop a more sophisticated understanding of biology.

It's very likely that the field of mechanistic interpretability will grow beyond simply examining weights in a model, to higher order understandings of the computational flow within a model (gradient descent and itself data were mentioned in this thread)--I agree that simply examining weights/activations is not a sufficient paradigm for understanding neural computation--but it is a start.
 

This is a bug and we're looking into it. It appears to be specific to Safari on iOS (Chrome on iOS is a Safari skin); it doesn't affect desktop browsers, Android/Chrome, or Android/Firefox, which is why we didn't notice earlier. This most likely started with a change on desktop where clicking on a post (without modifiers) opens when you press the mouse button, rather than when you release it.

Hey folks. Since no one RSVPd I probably won't bother heading to Meadow Park tomorrow. However, feel free to reach out to me if you ever want to meet up and hang out.

I am in New York until Tuesday. DM me if you are in the area and want to meet up and talk about LW, how to use AI for research/thinking/writing, or broader rationality community things.

Currently lots of free time Saturday and Monday. 

Roon: Unfortunately, I don’t think building nice AI products today or making them widely available matters very much. Minor improvements in DAU or usability especially doesn’t matter. Close to 100% of the fruits of AI are in the future, from self-improving superintelligence [ASI].

Every model until then is a minor demo/pitch deck to hopefully help raise capital for ever larger datacenters. People need to look at the accelerating arc of recent progress and remember that core algorithmic and step-change progress towards self-improvement is what matters.

One argument has been that products are a steady path towards generality / general intelligence. Not sure that’s true.

Looks like a deleted tweet...

Too close to truth, so that a presumed OpenAI employee is not supposed to articulate it that explicitly?

Video games also have potential legal advantages over IQ tests for companies. You could argue that "we only hire people good at video games to get people who fit our corporate culture of liking video games" but that argument doesn't work as well for IQ tests.

IANAL but unless you work for a videogame company(or a close analogue like chess.com) , I think this is just false. If your job is cognitively demanding, having IQ tests (or things like IQ tests with a mildly plausible veneer) probably won't get you in legal trouble^[1], whereas I think employment lawyers would have a field day if you install culture fit questions with extreme disparate impact, especially when it's hard to directly link the games to job performance. 

1. ^{^}

   The US Army has something like an IQ test.  So does the US Postal Service. So does the NFL. I've also personally worked in a fairly large tech company (not one of the top ones, before I moved to the Bay Area) that had ~IQ tests as one of the entrance criteria.  AFAIK there has never been any uproar about it. 

And it is important to notice that o1 is an attempt to use tons of inference as a tool, to work around its G (and other) limitations, rather than an increase in G or knowledge.

This is a rather strange statement.

o1 is basically a "System 2" addition (in terms of "Thinking, fast and slow") on top of a super-strong GPT-4o "System 1". As far as "System 1" entities go, GPT-4-level systems seem to me to be rather superior to the "System 1" "fast thinking" components of a human being^[1].

It seems to be the case that the "System 2" part is a significant component of G of a human, and it seems to be the case that o1 does represent a "System 2" addition on top of a GPT-4-level "System 1". So it seems appropriate to attribute an increase of G to this addition (given that this addition does increase its general problem-solving capabilities).

Basically, "System 2" thinking still seems to be a general capability to reason and deliberate, and not a particular skill or tool.

The post is mostly arguing that desires can shift around as we learn and think, which I agree with. But a couple parts of the post (including the title and some of the subheadings) seems to suggest something more than that: it’s not just desire-shifts, but desire convergence, towards a thing called “our values”.

(In other words, if I say that “I’m learning about blah”, then I’m strongly implying that there’s a fact-of-the-matter about blah, and that my beliefs about blah are approximately-monotonically converging towards that fact-of-the-matter. Right?)

Do you think there’s a thing (“human values”) to which desires gradually converge, via the kinds of processes described in this post? (I don’t, see §2.7 here [? · GW].)