LessWrong 2.0 Reader

This is especially useful when pursuing several subgoals in parallel, as forward-checking a combination of moves is combinatorially costly--better to have the agent's parallel actions constrained to nice parts of the space.

If I were a singleton AGI, but not such a Jupiter brain that I could deal with the combinatorial explosion of directly jointly-optimizing every motion of every robot, I would presumably set up an internal “free market” with spot-prices for iron ore and robot-hours and everything else. Then I would iteratively cycle through all my decision-points and see if there are ways to “make money” locally, and then update virtual “prices” accordingly.

In fact, I think there’s probably a theorem that says that the optimal solution of a complex resource allocation problem is isomorphic to a system where things have prices. (Something to do with Lagrange multipliers? Shrug.)

(Fun fact: In the human world, propagating prices within firms—e.g. if the couch is taking up 4m² of the 4000m² floor space at the warehouse, then that couch is “charged” 0.1% of the warehouse upkeep costs, etc.—is very rarely done but leads directly to much better decisions and massive overall profit increases! See here [LW(p) · GW(p)].)  

Externalities are not an issue in this virtual “economy” because I can “privatize” everything—e.g. I can invent fungible allowances to pollute the atmosphere in thus-and-such way etc. This is all just a calculation trick happening in my own head, so there aren’t coordination problems or redistribution concerns or information asymmetries or anything like that. Since I understand everything (even if I can’t juggle it all in my head simultaneously), I’ll notice if there’s some relevant new unpriced externality and promptly give it a price.

So then (this conception of) corrigibility would correspond to something like “abiding by this particular system of (virtual) property rights”. (Including all the weird “property rights” like purchasing allowances to emit noise or heat or run conscious minds or whatever, and including participating in the enterprise of discovering new unpriced externalities.) Do you agree?

EDIT 3/5/24: In the comments for Counting arguments provide no evidence for AI doom, Evan Hubinger agreed that one cannot validly make counting arguments over functions. However, he also claimed that his counting arguments "always" have been counting parameterizations, and/or actually having to do with the Solomonoff prior over bitstrings.

As one of Evan's co-authors on the mesa-optimization paper from 2019 I can confirm this. I don't recall ever thinking seriously about a counting argument over functions. 

Who’s track record of AI predictions would you like to see evaluated?

Whoever has the best track record :)

Yes. Current AI policy is like people in a crowded room fighting over who gets to hold a bomb. It's more important to defuse the bomb than it is to prevent someone you dislike from holding it.

That said, we're currently not near any satisfactory solutions to corrigibility. And I do think it would be better for the world if were easier (by some combination of technical factors and societal factors) to build AI that works for the good of all humanity than to build equally-smart AI that follows the orders of a single person. So yes, we should focus research and policy effort toward making that happen, if we can.

And if we were in that world already, then I agree releasing all the technical details of an AI that follows the orders of a single person would be bad.

There are 3 people. Each person announces an integer. The smallest unique integer wins: e.g. if your opponents both pick 1, you win with any number. If all 3 pick the same number, the winner is picked randomly

Question: what’s the Nash equilibrium?

(I assume this is meant to be natural numbers or positive ints? Otherwise I don't think there is a nash equilibrium.)

Good question!

I have gained a lot of emotion handling skill. This lets me be calmer and kinder to my wife and my son and other people. It also means I suffer less because I can more easily detect negative thoughts and feelings and (to some extent) disengage from them rather than feed them.

I am also slowly getting better at actively cultivating positive/happy/pleasant mind states.

It seems like you're assuming people won't build AGI if they don't have reliable ways to control it, or else that sovereign (uncontrolled) AGI would be likely the be friendly to humanity. Both seem unlikely at this point, to me. It's hard to tell when your alignment plan is good enough, and humans are foolishly optimistic about new projects, so they'll probably build AGI with or without a solid alignment plan.

So I'd say any and all solutions to corrigibility/control should be published.

Also, almost any solution to alignment in general could probably be used for control as well. And probably would be. See

https://www.lesswrong.com/posts/7NvKrqoQgJkZJmcuD/instruction-following-agi-is-easier-and-more-likely-than [LW · GW]

And

https://www.lesswrong.com/posts/587AsXewhzcFBDesH/intent-alignment-as-a-stepping-stone-to-value-alignment [LW · GW]

pro-DEI-requirements in clinical trials, so that drug companies have to have a certain number of various groups in their trial population

If this is motivated by accounting for biological diversity that might translate into different responses to a drug, then this feels like a very non-central case of DEI at best and I would expect it's not what a majority of people think of when they hear/think "DEI".

Yeah, I also guess that something in this direction is plausibly right.

perhaps nothingness actually contains a superposition of all logically possible states, models, and systems, with their probability weighted by the inverse of their complexity.

I think the relevant question here is why we should expect their probability to be weighted by the inverse of their complexity. Is there any abstract theoretical argument for this? In other words, we need to find an a priori justification for this type of Ockham's razor.

Here is one such attempt: Any possible world can be described as a long logical conjunction of "basic" facts. By the principle of indifference, assume any basic fact has the same a priori probability (perhaps even probability 0.5, equal to its own negation), and that they are a priori independent. Then longer conjunctions will have lower probability. But longer conjunctions also describe more complex possible worlds. So simpler possible worlds are more likely.

Though it's not clear whether this really works. Any conjunction completely describing a possible world would also need to include a statement "... and no other basic facts are true", which is itself a quantified statement, not a basic fact. Otherwise all conjunctive descriptions of possible worlds would be equally long.

Nope, same position. In an ideal world he would be prevented from hurting animals (currently, paying people to torture them and strip their bodies of their flesh to bring to him). Current world would also be better without any animal torture payers / doers.

Also, it doesn't seem much like a clarification, versus a reframing. Previously they wrote "Animals suffer, but I don't much care"; now they wrote, "I don't kick dogs, but I have destroyed a wasp's nest in my garage"

(Later edit): Btw, I agree with your other comment that the "original comment has been so strongly disagree-voted because of the framing". In fact, even when people use the original framing but then say this situation is bad, it's often still controversial, because people don't like to think about what's being done to animals in direct terms. Not sure if it's more or less controversial for someone to use a non-euphemistic framing, and then say they're okay with this.