LessWrong 2.0 Reader

I think for this specific example the superior is wrong because realistically we can form an expectation of the distribution of those factors. Just because we don't know doesn't mean it's actually necessarily a gaussian - some factors, like the Coriolis force, are systematic. If the distribution was "a ring of 1 m around the aimed point" then you would know for sure you won't hit the terrorist that way, but have no clue whether you'll hit the kid.

Also, even if the distribution was gaussian, if it's broad enough the difference in probability between hitting the terrorist and hitting the kid may simply be too small to matter.

Exactly! I've also noticed there are so many ideas and theories out there, relative to the available resources to evaluate them and find the best to work on.

A lot of good ideas which I feel deserve a ton of further investigation, seem to be barely talked about after they're introduced. E.g. off the top of my head,

• Why Don't We Just... Shoggoth+Face+Paraphraser? [LW · GW]
• Applying superintelligence without collusion [LW · GW]
• Self-Other Overlap [LW · GW]
• Multi-objective homeostasis [LW · GW]
• MONA: Managed Myopia with Approval Feedback [LW · GW]

My opinion is that there isn't enough funds and manpower. I have an idea [LW · GW] on increasing that, which ironically also got ignored, yay!

Interesting, thanks!

I guess one could object that in you're even more clueless sniper example, applying the POI between Hit and Not Hit is just as arbitrary as applying it between, e.g., Hit, Hit on his right, and Hit on his left. This is what Greaves (2016) -- and maybe others? -- called the "problem of multiple partitions". In my original scenario, people might argue that there isn't such a problem and that there is only one sensible way to apply POI. So it'd be ok to apply it in my case and not in yours. 

I don't know what to make of this objection, though. I'm not sure it makes sense. It feels a bit arbitrary to say "we can apply POI but only when there is one way of applying it that clearly seems more sensible". Maybe this problem of multiple partitions is a reason to reject POI altogether (in situations of what Greaves call "complex cluelessness" at least, like in my sniper example).

No one with the money has offered to fund it yet. I'm not even sure they're aware this is happening.

Um, this seems bad. I feel like I should do something, but I don't personally have that kind of money to throw around. @habryka [LW · GW], is this the LTFF's job??

Can you elaborate further on how Gato is proof that just supplementing the training data is sufficient? I looked on youtube and can't find any videos of task switching.

This story is moved me a lot, and I am giving it a substantial vote,

But... I do still really wish this line...

And they turn away and go back to their work—all except for one, who brushes past the grasshopper and whispers “Meet me outside at dusk and I’ll bring you food. We can preserve the law and still forgive the deviation.”

Came where it originally was located, significantly later in the post, after these sections:

The ants start to receive dozens of requests for food, then hundreds—and while many are fraudulent, enough are real that they are moved to act. In order to set incentives correctly, the ants decide to only give food to those who can prove that they lost their own food supplies through no fault of their own, and set up a system for vetting claims.

This works well for a time—but as fraudsters grow more sophisticated, the ants’ bureaucratic requirements grow more onerous. In order to meet them, other creatures start to deposit their food in large group storehouses which can handle the administrative overhead. But now the food supply is exposed to systemic risk if the leaders of those storehouses make poor decisions, whether from carelessness or greed.

One year several storehouses fail; in trying to fill the shortfall, the ants almost run out of food for themselves. To avoid that ever happening again, they set up stringent regulations and oversight of permissible storehouses, funded by taxes levied throughout the year. At first this takes only a small proportion of their labor—but as their regulatory apparatus inevitably grows, they need to oversee more and more aspects of the ecosystem, and are called upon to right more and more injustices. Eventually the ants—originally the most productive of all creatures—stop producing any food of their own, so busy are they in tending to the system they’ve created.

...

“And therefore, to reduce risks from centralization, and to limit our own power, we can’t give you any food”, the ants conclude. And they turn away and go back to their work, with a quiet sense of satisfaction that they’ve given such legible and defensible reasons for focusing on their own problems and keeping all the food for themselves.

In it's original positioning, the "We can preserve the law and still forgive the deviation" line felt like the climax of a mini-arc. Previously Richard had said he moved it for two reasons [LW(p) · GW(p)]:

Firstly, if the grasshopper was just unlucky, then there's no "deviation" to forgive—it makes sense only if the grasshopper was culpable. Secondly, the earlier parts are about individuals, and the latter parts are about systems—it felt more compelling to go straight from "centralized government" to "locust war" than going via an individual act of kindness.

But, what I find particularly meaningful about it is that even when giant systems have evolved that are turning grotesque with their burueacratic weight... and when maybe those systems are necessary or inevitable or something... individuals can still choose to just... do a nice thing, when they have local knowledge suggesting it's the right thing to do in this case. (And meanwhile, I had no problem backfilling and appropriate history for this particular snippet. I can imagine a grasshopper who slacked off a bit, and also got a bit unlucky, and also seems to have earnestly learned and will make better choices next time)

What do you think is the ideal use-case for steering? Or is it not needed

Note that this story has recieved a beautifully-animated video adaptation by Rational Animations!
 

Terrific!

Oops, yes.