LessWrong 2.0 Reader

I used to have an idea for a karma/reputation system: repeatedly recalculate karma weighted by the karma of the upvoters and downvoters on a comment (then normalize to avoid hyperinflation) until a fixed point is reached.

I feel like this is vaguely somehow related to:

• AlphaGoZero
• Humans Consulting HCH [LW · GW]
• Wealth in markets

In our case I'm not worried about when they wake up in the morning, but about going to sleep, especially at naptime. A crib is boring and conducive to sleep, but there are a lot of interesting things to play with around the room.

ok. We take our son anyway out of the bet as soon as he wakes up. He sleeps long enough already by himself.

I mistyped a bit with the use of "relationships". Yes, names and faces both trigger social recognition, but I meant to make the point that they operate in significantly different ways in the brain, and facial recognition is tuned to processing a lot of emotional and social cues that we aren't tuned to from text. I have tons of social associations with people's physical forms that are beyond simply their character.

(ChatGPT helped me write this comment.)

Here's a revised sketch 

A few notes:

• I use Scalable Oversight to refer to both Alignment and Control 
• I'm confused whether weak to strong learning is a restatement of scalable oversight, ELK, or its own thing, so I ignore it 
• I don't explicitly include easy-to-hard, I think OOD basically covers it
• taxonomies and abstractions are brittle and can be counterproductive

Scalable Oversight Taxonomy

• Scalable Oversight
  • Scalable Alignment
    • Benchmarks / Tasks
      • Sandwiching Experiments (human amateurs + model, gt from human experts)
      • Weak models supervising Strong models
    • Approaches
      • Debate
      • Recursive reward modeling
      • (Solution to Eliciting Latent Knowledge) + Narrow Elicitation
        • (Note - I think assumes more then prior scalable oversight ideas that there will be base model with adequate knowledge, such that the hard part is extracting the knowledge rather than teaching the model)
        • Eliciting Latent Knowledge
          • Approaches
            • Contrast Consistent Search
            • Confidence
            • Intermediate Probing
            • "Speed Prior"
            • "Simplicity Prior"
            • Concept Extrapolation - learn all salient generalizations, use expensive supervision to select correct one
            • IID Mechanistic Anomaly Detection + expensive supervision on anomalies
          • Subclasses
            • Measurement Tampering Detection
              • Approaches
                • OOD Mechanistic Anomaly Detection
                  • In distribution
                  • Out of Distribution (likely? requires multiple measurment structure)
                • Concept Extrapolation
                  • train diverse probes on untrusted data, select probe that predicts positive measurements less frequently
        • Narrow Elicitation
          • ...
  • Scalable Control
    • Weak Review
    • Untrusted Rephrase or whatever
    • Coup probes
    • MAD (Review all anomalies)
• Trojans
  • ...
  • MAD (maybe?)
• Adversarial Examples
  • ...
  • MAD (maybe?)
• Natural Mechanism Distinction
  • MAD
• Spurious Correlate Detection / Resolution
  • Concept Extrapolation

I did find it and we sent him an email, hope he reads it and joins :)

Under physicalist epiphenomenalism (which is the standard approach to the mind-matter relation), the mind is super-impressed on reality, perfectly synchronized, and parallel to it.

Under dualist epiphenomenalism, that might be true. Physicalism has it either that consciousness is non existent rather than causally idle (eliminitavism), or identical to physical brain states (and therefore sharing their causal powers).

Understanding why some physical systems make an emergent consciousness appear (the so called “hard problem of consciousness”) or finding a procedure that quantify the intensity of consciousness emerging from a physical system (the so called “pretty hard” problem of consciousness) is impossible:

You could have given a reason why.

If this were true that the concept of „indexical sample space“ does not capture the thirder position, how do you explain that it produces exactly the same probabilities that thirders entertain? Operating with indexicals is a necessary condition (and motivation) for Thirdism, which means assuming indexical sample spaces when it comes to the mathematical formalization of arguments in terms of probability theory. To my knowledge no relevant thirder literature denies that. And within the thirder model, these probabilities indeed hold true. If we assume Monday and Tuesday to be mutually exclusive, than this is mathematically the case. Math is not a judge of our assumptions here, it is merely the executive organ which in this case produces thirder probabilities. The point at issue is whether the theoretical assumptions of the thirder model fit reality and probabilities could be transfered into the real world. Thirders say yes, speaking of regular probabilities, halfers say no speaking of irregular, „weighted“ probabilities.

Yeah fair point. I do think labs have some some nonzero amount of responsibility to be proactive about what others believe about their commitments. I agree it doesn't extend to 'rebut every random rumor'.

I think I'm mostly right, but using a somewhat confused frame. 

It makes more sense to think of MAD approaches as detecting all abnormal reasons (including deceptive alignment) by default, and then if we get that working we'll try to decrease false anomalies by doing something like comparing the least common ancestor of the measurements in a novel mechanism to the least common ancestor of the measurements on trusted mechanisms.