Posts

Classifying specification problems as variants of Goodhart's Law 2019-08-19T20:40:29.499Z · score: 70 (15 votes)
Modeling AGI Safety Frameworks with Causal Influence Diagrams 2019-06-21T12:50:08.233Z · score: 47 (13 votes)
Thoughts on Human Models 2019-02-21T09:10:43.943Z · score: 124 (35 votes)
Cambridge UK Meetup Saturday 12 February 2011-02-02T14:20:02.418Z · score: 6 (7 votes)

Comments

Comment by xrchz on Reframing Impact · 2019-10-22T14:53:51.965Z · score: 1 (1 votes) · LW · GW

One misgiving I have about the illustrated format is that it's less accessible than text. I hope the authors of work in this format keep the needs of a wide variety of readers in mind.

Comment by xrchz on Creating Environments to Design and Test Embedded Agents · 2019-08-23T10:46:30.360Z · score: 1 (1 votes) · LW · GW
the objective of agent-designers is to have the agent collect as many agents as possible

Typo: should say "dollars"?

Comment by xrchz on Open question: are minimal circuits daemon-free? · 2019-08-05T14:01:21.436Z · score: 1 (1 votes) · LW · GW
if the daemon is obfuscated, there is no efficient procedure which takes the daemon circuit as input and produces a smaller circuit that still solves the problem.
So we can't find any efficient constructive argument. That rules out most of the obvious strategies.

I don't think the procedure needs to be efficient to solve the problem, since we only care about existence of a smaller circuit (not an efficient way to produce it).

Comment by xrchz on Open question: are minimal circuits daemon-free? · 2019-08-05T13:50:30.922Z · score: 1 (1 votes) · LW · GW
I don't think this question has much intrinsic importance, because almost all realistic learning procedures involve a strong simplicity prior (e.g. weight sharing in neural networks).

Does this mean you do not expect daemons to occur in practice because they are too complicated?

Comment by xrchz on Selection vs Control · 2019-07-30T08:30:15.687Z · score: 13 (4 votes) · LW · GW

Thanks for a great post! I have a small confusion/nit regarding natural selection. Despite its name, I don't think it's a good exemplar of a selection process. Going through the features of a selection process from the start of the post:

  • can directly instantiate any element of the search space. No: natural selection can only make local modifications to previously instantiated points. But you already dealt with this local search issue in Choices Don't Change Later Choices.
  • gets direct feedback on the quality of each element. Yes.
  • quality of element does not depend on previous choices. No, the evaluation of an element in natural selection depends a great deal on previous choices because they usually make up important parts of its environment. I think this is the thrust of the claim that natural selection is online (which I agree with).
  • only the final output matters. No? From the perspective of natural selection, I think the quality of the current output is what matters.

I'd love to know why natural selection seemed obvious as an example of a selection process, since it did not to me due to its poor score on the checklist above.

Comment by xrchz on Boeing 737 MAX MCAS as an agent corrigibility failure · 2019-03-17T09:43:33.173Z · score: 14 (7 votes) · LW · GW

I like this post because it pushes us to be more precise about what we mean by corrigibility. Nice example.

Comment by xrchz on Logical Updatelessness as a Robust Delegation Problem · 2017-10-29T04:49:08.364Z · score: 10 (3 votes) · LW · GW

Nice post! Do you have a link to an explanation of what counterfactual mugging is and why it's a good thing?

For subagent alignment problems, is there an interesting distinction to be drawn between the limited agent being able to understand the process by which the more powerful agent becomes powerful, versus not even understanding that? (What would it mean to "understand the process"? I suppose it means being able to validate certain relevant facts about the process though not enough to know exactly what results from it.)

Comment by xrchz on New Pascal's Mugging idea for potential solution · 2016-08-05T23:20:27.444Z · score: 2 (2 votes) · LW · GW

More specifically, it seems that your c must include information about how to interpret the X bits. Right? So it seems slightly wrong to say "R is the largest number that can be specified in X bits of information" as long as c stays fixed. c might grow as the specification scheme changes.

Alternatively, you might just be wrong in thinking that 30 bits are enough to specify 3^^^^3. If c indicates that the number of additional universes is specified by a standard binary-encoded number, 30 bits only gets you about a billion.

Comment by xrchz on Superintelligence Reading Group - Section 1: Past Developments and Present Capabilities · 2014-10-03T20:53:56.066Z · score: 2 (2 votes) · LW · GW

They're not yet close to being taken over by AI, but there has been research on automating all of the above. Some possibly relevant keywords: automated theorem proving, and program synthesis.

Comment by xrchz on Superintelligence Reading Group - Section 1: Past Developments and Present Capabilities · 2014-10-03T20:53:04.551Z · score: 0 (0 votes) · LW · GW

They're not yet close to being taken over by AI, but there has been research on automating all of the above. Some possibly relevant keywords: automated theorem proving, and program synthesis.

Comment by xrchz on Collecting expressions of interest in a rationality conference in August · 2013-05-25T15:40:58.682Z · score: 3 (3 votes) · LW · GW

I'd go, presuming no other important commitments on the dates.

Comment by xrchz on Sequence translations: Seeking feedback/collaboration · 2012-07-10T07:53:26.605Z · score: 1 (1 votes) · LW · GW

I know someone who might be interested in doing some translations into Japanese. If you add that language we might have a go.

Comment by xrchz on New Singularity.org · 2012-07-06T14:53:18.016Z · score: 0 (0 votes) · LW · GW

Is the site down now? Is there a separate site to monitor its uptime status?

Comment by xrchz on Help! Name suggestions needed for Rationality-Inst! · 2012-01-31T03:33:47.262Z · score: 0 (0 votes) · LW · GW
  • Humanity's Potential
  • Sense Trust
  • Reach
  • Brightness Released
  • Grow Beyond
  • Coherent Life
Comment by xrchz on Help! Name suggestions needed for Rationality-Inst! · 2012-01-31T02:50:08.972Z · score: 0 (0 votes) · LW · GW

names don't have to be words already

Comment by xrchz on Help! Name suggestions needed for Rationality-Inst! · 2012-01-31T02:04:33.148Z · score: 1 (3 votes) · LW · GW
  • Whatever Works
  • Dreamshine
  • Attainment
  • Veridian
  • Reasound
  • Minds' Arts
  • Melodine
  • Sunshine Army
  • Embrace Sense
Comment by xrchz on Help! Name suggestions needed for Rationality-Inst! · 2012-01-30T01:33:47.106Z · score: 0 (0 votes) · LW · GW
  • Subliminal Processing
  • Wordless Thought
  • Ratikata
  • LessWrong Dojo
  • Think Academy
Comment by xrchz on Help! Name suggestions needed for Rationality-Inst! · 2012-01-29T23:22:01.487Z · score: 3 (5 votes) · LW · GW
  • (Centre for) Art of Reason
  • Art of Winning
  • Act Better
  • Right on Target (or Target on Right)
  • Evidence Based
  • Mindtweak
  • Mindhack
  • Effective Agency
  • Winning Institute
  • Inspired Thought
  • Achieve
  • Decision Training
  • Better Model
  • Perfect Thought
  • Motivated Action
Comment by xrchz on Meetup : Cambridge UK · 2012-01-27T00:17:09.234Z · score: 1 (1 votes) · LW · GW

Wow I didn't realize this was happening! I'm super busy tomorrow but will try to come for at least some of it. When was the last meetup?

Comment by xrchz on Cambridge UK Meetup Saturday 12 February · 2011-02-11T23:13:25.386Z · score: 0 (0 votes) · LW · GW

can you bring the paperclip?

Comment by xrchz on Cambridge UK Meetup Saturday 12 February · 2011-02-11T23:11:53.320Z · score: 0 (0 votes) · LW · GW

I'll be there.

Comment by xrchz on Cambridge UK Meetup Saturday 12 February · 2011-02-03T10:05:33.957Z · score: 0 (0 votes) · LW · GW

What about Friday 11th?

Comment by xrchz on Newcomb's Problem and Regret of Rationality · 2009-11-28T23:03:40.779Z · score: 0 (0 votes) · LW · GW

The new argument is about whether there might be inherently unpredictable things. If not, then your picking a box based on the outcome of a "quantum event" shouldn't make Omega any less physically plausible,

Comment by xrchz on Newcomb's Problem and Regret of Rationality · 2009-11-27T10:35:55.603Z · score: 0 (0 votes) · LW · GW

As I understand it, probabilities are observed because there are observers in two different amplitude blobs of configuration space (to use the language of the quantum physics sequence) but "the one we are in" appears to be random to us. And mathematically I think quantum mechanics is the same under this view in which there is no "inherent, physical" randomness (so it would still be the best formalism we have for predicting things).

Could you say what "physical randomness" could be if we don't allow reference to quantum mechanics? (i.e. is that the only example? and more to the point, does the notion make any sense?)

Comment by xrchz on Newcomb's Problem and Regret of Rationality · 2009-11-26T06:54:36.860Z · score: 0 (0 votes) · LW · GW

What's wrong with Omega predicting a "quantum event"? "50% chance" is not an objective statement, and it may well be that Omega can predict quantum events. (If not, can you explain why not, or refer me to an explanation?)

Comment by xrchz on Newcomb's Problem and Regret of Rationality · 2009-11-22T09:17:03.042Z · score: 1 (1 votes) · LW · GW

Would that make you a supersuperintelligence? Since I presume by "picking randomly" you mean randomly to Omega, in other words Omega cannot find and process enough information to predict you well.

Otherwise what does "picking randomly" mean?

Comment by xrchz on Issues, Bugs, and Requested Features · 2009-11-07T08:55:11.182Z · score: 2 (2 votes) · LW · GW

Update: the welcome post has a small explanation of voting and karma.

Comment by xrchz on Cached Thoughts · 2009-11-03T22:18:34.284Z · score: 0 (2 votes) · LW · GW

It's a good guess that the actual majority of human cognition consists of cache lookups.

Sounds consistent with Jeff Hawkins's memory prediction framework

Comment by xrchz on Searching for Bayes-Structure · 2009-11-03T00:45:17.529Z · score: 3 (2 votes) · LW · GW

Thanks for the reply. I want to follow a related issue now.

So are all natural processes computable (as far as we know)?

I want to know whether the question above makes sense, as well as its answer (if it does make sense).

I have trouble interpreting the question because I understand computability to be about effectively enumerating subsets of the natural numbers, but I don't find the correspondence between numbers and nature trivial. I believe there is a correspondence, but I don't understand how correspondence works. Is there something I should read or think about to ease my confusion? (I hope it's not impenetrable nonsense to both believe something and not know what it means.)

Comment by xrchz on Outside the Laboratory · 2009-11-02T21:08:50.028Z · score: 0 (0 votes) · LW · GW

I tentatively believe in real numbers and differential equations because physics requires (though I also hold out hope that e.g. holographic physics or some other discrete view may enable me to go digital again). However, I don't believe that the real numbers in physics are really made of Dedekind cuts, or any other sort of infinite set.

Shouldn't you add probability theory to the list [physics, differential equations]? Only because probabilities are usually taken to be real numbers. I'm curious what you think of real numbers... how would you construct them? I guess it must be some way that looks a limit of finite processes operating on finite sets, right?

Comment by xrchz on Searching for Bayes-Structure · 2009-11-02T07:27:51.859Z · score: 2 (1 votes) · LW · GW

I don't quite see the trivial observation yet - can you explain a little further?

Comment by xrchz on Timeless Control · 2009-11-01T21:24:08.250Z · score: 1 (1 votes) · LW · GW

I think when people say that apples and choices are illusions, they might mean that they are patterns recognizable by people but not fundamental: if some system couldn't recognize an apple (perhaps only because it never had any reason to form the concept) but did have a model of the amplitude distribution of the universe, it would get along just fine (actually it would probably just have different high-level concepts).

Comment by xrchz on Configurations and Amplitude · 2009-11-01T21:01:24.404Z · score: 2 (2 votes) · LW · GW

You can't explain yourself? I followed your link. It looks like part of why half-silvered mirrors "work" for the purpose of seeing someone without them seeing you is that one side is kept brightly lit while the spying side is kept dark. I think "beam-splitter" is possibly a more accurate term for my question, which I looked up and found

Another design is the use of a half-silvered mirror. This is a plate of glass with a thin coating of aluminum (usually deposited from aluminum vapor) with the thickness of the aluminum coating such that part, typically half, of light incident at a 45 degree angle is transmitted, and the remainder reflected.

(Wikipedia) Of course, this doesn't actually explain anything - why should there be a thickness of aluminum such that part of the light is reflected while the remainder is transmitted?

Would a beam-splitter still work if the silvered and non-silvered parts were much larger (i.e. a chunky block pattern)? If you fired a single photon at that would it still make sense to calculate amplitude as you do in this post (considering the two outward paths and multiplying one by i, the other by 1)? Perhaps the distance between a silvered part and a non-silvered part needs to be close to the wavelength of the photon?

Comment by xrchz on Configurations and Amplitude · 2009-11-01T20:31:45.621Z · score: 1 (1 votes) · LW · GW

I have a question similar to Nate's. How does a half-silvered mirror work? More specifically, what is it about light or about half-silvered mirrors that means there are two paths for a photon out of a half-silvered mirror (compared to a full mirror, for example)? My guess at the moment is that the answer might start "light doesn't actually travel in straight lines..."...

Comment by xrchz on The Born Probabilities · 2009-11-01T08:28:40.294Z · score: 0 (0 votes) · LW · GW

Eliezer gave a simpler answer to my question: "yes". (I'm still not sure what yours means.)

Back to Peter's question. What makes you say decoherence doesn't happen on the Planck time scale? Can you explain that further?

Comment by xrchz on Entangled Photons · 2009-11-01T06:05:14.164Z · score: 5 (5 votes) · LW · GW

Does the "world" in "many worlds" refer to the same thing as "blob" in this post?

Comment by xrchz on The Born Probabilities · 2009-11-01T05:14:46.733Z · score: 0 (0 votes) · LW · GW

Then I'm still unclear about what a world is. Care to explain?

Comment by xrchz on Mind Projection Fallacy · 2009-11-01T02:11:21.023Z · score: 0 (0 votes) · LW · GW

Is "the world is full of people" an example of the mind-projection fallacy? (Compare to "we can both recognize the pattern 'person' at a high-level in our multi-leveled models of levelless reality")

Comment by xrchz on The Born Probabilities · 2009-10-31T23:46:58.291Z · score: 1 (1 votes) · LW · GW

I'd also love to know the answer to Peter's question... A similar question is whether we should expect all worlds to eventually become mangled (assuming the "mangled worlds" model). I understand "world" to mean "somewhat isolated blob of amplitude in an amplitude distribution" - is that right?

Comment by xrchz on Identity Isn't In Specific Atoms · 2009-10-31T21:52:34.198Z · score: 0 (0 votes) · LW · GW

Do you take the real universe to be the (single) point in the universal QM configuration space, along with the single complex value of the universal wavefunction?

No, the universe is an (evolving) amplitude distribution over configuration space.

I'm not what "superposition state" means, but my guess is that the answer to "Are these superposition states not fundamental?" is "Yes they are".

Comment by xrchz on The Generalized Anti-Zombie Principle · 2009-10-31T12:24:15.335Z · score: 1 (1 votes) · LW · GW

It makes sense to humans (modelers), who can recognize hands, to say "this hand is implemented using quarks", and "that hand is implemented using sand (which, incidentally, is implemented using quarks)". But when we say "quarks fully describe a hand" I think part of the meaning is an acknowledgment that reducing to quarks gets you closer to the territory. (Hands are only in our maps.)

Comment by xrchz on Rational Me or We? · 2009-10-31T11:18:23.246Z · score: 1 (1 votes) · LW · GW

For Polymath the kind of desired result of collaboration is clear to me: a (new) (dis-) proof of a mathematical statement.

What is the kind of desired result of collaborating rationalists?

From the talk about prediction markets it seems that "accurate predictions" might be one answer. But predictions of what? Would we need to aggregate our values to decide what we want to predict?

The phrase in Robin's post was "join together to believe truth", so perhaps the desired result is more true beliefs (in more heads)? Did you envision making things that are more likely to be true more visible, so that they become defaults? In other words, caching the results of truth-seeking so they can be easily shared by more people?

Comment by xrchz on Reductive Reference · 2009-10-29T21:00:42.962Z · score: 1 (1 votes) · LW · GW

Indeed, what would it mean for reality to be multi-leveled? (This would address tcpkac's comment too.)

Comment by xrchz on Angry Atoms · 2009-10-29T07:09:42.013Z · score: 0 (0 votes) · LW · GW

an algorithm is a model in our mind to describe the similarities of those physical systems implementing it

a number is a model like that as well, right? (may be relevant to the comments below)

Comment by xrchz on Reductionism · 2009-10-28T21:26:52.332Z · score: 0 (2 votes) · LW · GW

Even if there is only one type of thing 'x', our reality (which is, above all, dynamic) seems to require a relationship and interaction between 'x' and ' ~x'. I'd say, logically, reality needs at least two kinds of things.

Logic can only compel models.

You seem to be saying "Let x denote the universe. ~x is then a valid term. So ~x must denote something that isn't x, thus there are two things!" There are surface problems with this such as that x may not be of type boolean, and that you're just assuming every term denotes something. But the important problem is simpler: we can use logic to deduce things about our models, but logic doesn't touch reality itself (apart from the part of reality that is us).

What do you mean by "reality is dynamic"? Have you read Timeless Physics?

Comment by xrchz on Reductionism · 2009-10-28T11:42:02.297Z · score: 4 (4 votes) · LW · GW

But the way physics really works, as far as we can tell, is that there is only the most basic level - the elementary particle fields and fundamental forces.

To clarify (actually, to push this further): there is only one thing (the universe) - because surely breaking the thing down into parts (such as objects) which in turn lets you notice relations between parts (which in turn lets you see time, for example) -- surely all that is stuff done by modelers of reality and not by reality itself? I'm trying to say that the universe isn't pre-parsed (if that makes any sense...)

Comment by xrchz on Issues, Bugs, and Requested Features · 2009-10-26T22:18:26.114Z · score: 4 (4 votes) · LW · GW

Is there a page for "how to use this website" somewhere that I've missed? For the most part, it is intuitive. But I got a bit worried when I clicked "Report" on some spam and it asked me "Are you sure?". No I'm not sure - I'm just guessing what "Report" means and what it does...

I'd also be interested in knowing how Karma works, who (if anyone) is notified about my comments, what Voting does, etc... Just a general overview of how the website works. And if this information isn't all in one place already maybe it should be.

Comment by xrchz on The Cluster Structure of Thingspace · 2009-10-26T21:59:02.972Z · score: -1 (1 votes) · LW · GW

What are the dimensions of thingspace?

Are "number of sides", "IQ", "age", and "font" all dimensions?

And what are the points in thingspace? It sounds like they include anything that is somewhat "mother" and anything that is somewhat "robin". (And I should think thingspace is a point in thingspace too.)

I think this post makes some good points, the main one, for me, being that words are centers of (indefinitely extending) clusters rather than boundaries of sets. But I think the notion of thingspace rests on shaky foundations: it assumes the world is broken down into things and those things have attributes.

We don't all share the same thingspace do we?

Comment by xrchz on Where are we? · 2009-10-26T21:08:02.581Z · score: 0 (0 votes) · LW · GW

Canberra, Australia, too.

Comment by xrchz on The Parable of the Dagger · 2009-10-26T20:29:42.353Z · score: 6 (6 votes) · LW · GW

It's not the law of the excluded middle that's the problem, it's the jester's assumption that the entire statement "either this ..., or this..., but not both" is true. The jester reasons correctly under his assumptions, but fails to realize that he still has to discharge those assumptions before reaching reality.