LessWrong 2.0 Reader

To the extent that Tegmark is concerned about exfohazards (he doesn't seem to be very concerned AFAICT (?)), he would probably say that more powerful and yet more interpretable architectures are net positive.

Is this a massive exfohazard? Should this have been published?

Yikes, I'm not even comfortable maximizing my own CEV.

What do you think of this post by Tammy?

Where is the longer version of this? I do want to read it. :)

Well perhaps I should write it :)

Specifically, what is it about the human ancestral environment that made us irrational, and why wouldn't RL environments for AI cause the same or perhaps a different set of irrationalities?

Mostly that thing where we had a lying vs lie-detecting arms race and the liars mostly won by believing their own lies and that's how we have things like overconfidence bias and self-serving bias and a whole bunch of other biases. I think Yudkowsky and/or Hanson has written about this.

Unless we do a very stupid thing like reading the AI's thoughts and RL-punish wrongthink, this seems very unlikely to happen.

If we give the AI no reason to self-deceive, the natural instrumentally convergent incentive is to not self-deceive, so it won't self-deceive.

Again, though, I'm not super confident in this. Deep deception [LW · GW] or similar could really screw us over.

Also, how does RL fit into QACI? Can you point me to where this is discussed?

I have no idea how Tammy plans to "train" the inner-aligned singleton on which QACI is implemented, but I think it will be closer to RL than SL in the ways that matter here.

I'm so happy someone came up with this!

Military nerds correct me if I'm wrong but I think the answer might be the following. I'm not a pilot etc etc.

Stealth can be a bit of a misleading term. F35 aren't actually 'stealth aircraft' - they are low-observable aircraft. You can detect F35s with longwave radar just fine.

The problem isn't knowing that there is a F35 but to get a weapon -grade lock on it. This is much harder and your grainy gpt-interpreted photo isn't close to enough for a missile I think. You mentioned this already as a possibility.

Preventing a weapons grade radar or IR lock is what the F35's stealth architecture is about. A fighter carrier air to air missile might have a range of upwards of a 100km or more but a weapons grade lock can typically only obtained at 30 km for a F35

Still I'd say your proposal is worth pondering. The Ukrainians famously have pioneered something similar for audio which is used to detect missiles & drones entering Ukrainian airspace.

I'm pretty sure I heard Alan Watts say something like that, at least in one direction (lower levels of organization -> higher levels). "The conflict/disorder at the lower level of the Cosmos is required for cooperation/harmony on the higher level."

Or maybe the Ultimate Good in the eyes of God is the epic sequence of: dead matter -> RNA world -> protocells -> ... -> hairless apes throwing rocks at each other and chasing gazelles -> weirdoes trying to accomplish the impossible task of raising the sanity waterline and carrying the world through the Big Filter of AI Doom -> deep utopia/galaxy lit with consciousness/The Goddess of Everything Else [LW · GW] finale.

"So God can’t make the atoms be arranged one way and the humans be arranged another contradictory way."

But couldn't he have made a different sort of thing than humans, which were less prone to evil? Like, it seems to me that he didn't need to make us evolve through the process of natural selection, such that species were always in competition, status was a big deal, fighting over mates commonplace, etc. I do expect that there's quite a bit of convergence in the space of possible minds—even if one is selecting them from the set of "all possible atomic configurations of minds"—but I would still guess that not all of those are as prone to "evil" as us. I.e., if the laws of physics were held constant, I would think you could get less evil things than us out of it, and probably worlds which were overall more favorable to life (fewer natural disasters, etc.). But perhaps this is even more evidence that God only cares about the laws of physics? Since we seem much more like an afterthought than a priority?   

Asking ChatGPT to criticize an article also produces good suggestions often.

It certainly is possible! In more decision-theoritic terms, I'd describe this as "it sure would suck if agents in my reference class just optimized for their own happiness; it seems like the instrumental thing for agents in my reference class to do is maximize for everyone's happiness". Which is probly correct!

But as per my post, I'd describe this position as "not intrinsically altruistic" — you're optimizing for everyone's happiness because "it sure would sure if agents in my reference class didn't do that", not because you intrinsically value that everyone be happy, regardless of reasoning about agents and reference classes and veils of ignorance.