TsviBT's Shortform

The Berkeley Genomics Project is fundraising for the next forty days and forty nights at Manifund: https://manifund.org/projects/human-intelligence-amplification--berkeley-genomics-project

An important thing that the AGI alignment field never understood:

Reflective stability. Everyone thinks it's about, like, getting guarantees, or something. Or about rationality and optimality and decision theory, or something. Or about how we should understand ideal agency, or something.

But what I think people haven't understood is

1. If a mind is highly capable, it has a source of knowledge.
2. The source of knowledge involves deep change.
3. Lots of deep change implies lots of strong forces (goal-pursuits) operating on everything.
4. If there's lots of strong goal-pursuits operating on everything, nothing (properties, architectures, constraints, data formats, conceptual schemes, ...) sticks around unless it has to stick around.
5. So if you want something to stick around (such as the property "this machine doesn't kill all humans") you have to know what sort of thing can stick around / what sort of context makes things stick around, even when there are strong goal-pursuits around, which is a specific thing to know because most things don't stick around.
6. The elements that stick around and help determine the mind's goal-pursuits have to do so in a way that positively makes them stick around (reflective stability of goals).

There's exceptions and nuances and possible escape routes. And the older Yudkowsky-led research about decision theory and tiling and reflective probability is relevant. But this basic argument is in some sense simpler (less advanced, but also more radical ("at the root")) than those essays. The response to the failure of those essays can't just be to "try something else about alignment"; the basic problem is still there and has to be addressed.

(related elaboration: https://tsvibt.blogspot.com/2023/01/a-strong-mind-continues-its-trajectory.html https://tsvibt.blogspot.com/2023/01/the-voyage-of-novelty.html )

Are people fundamentally good? Are they practically good? If you make one person God-emperor of the lightcone, is the result something we'd like?

I just want to make a couple remarks.

• Conjecture: Generally, on balance, over longer time scales good shards express themselves more than bad ones. Or rather, what we call good ones tend to be ones whose effects accumulate more.
• Example: Nearly all people have a shard, quite deeply stuck through the core of their mind, which points at communing with others.
  • Communing means: speaking with; standing shoulder to shoulder with, looking at the same thing; understanding and being understood; lifting the same object that one alone couldn't lift.
  • The other has to be truly external and truly a peer. Being a truly external true peer means they have unboundedness, infinite creativity, self- and pair-reflectivity and hence diagonalizability / anti-inductiveness. They must also have a measure of authority over their future. So this shard (albeit subtly and perhaps defeasibly) points at non-perfect subjugation of all others, and democracy. (Would an immortalized Genghis Khan, having conquered everything, after 1000 years, continue to wish to see in the world only always-fallow other minds? I'm unsure. What would really happen in that scenario?)
  • An aspect of communing is, to an extent, melting into an interpersonal alloy. Thought patterns are quasi-copied back and forth, leaving their imprints on each other and each other leaving their imprints on the thought patterns; stances are suggested back and forth; interoperability develops; multi-person skills develop; eyes are shared. By strong default this cannot be stopped from being transitive. Thus elements, including multi-person elements, spread, binding everyone into everyone, in the long run.
  • God--the future or ideal collectivity of humane minds--is the extrapolation of primordial searching for shared intentionality. That primordial searching almost universally always continues, at least to some extent, to exert itself. The Ner Tamid is saying: God (or specifically, the Shekhinah) is the one direction that we move in; God will be omnipotent, and is of course omnibenevolent. To say things more concretely (though less accurately), people in some sense want to get along, and all else equal they keep searching and learning more how to get along and keep implicitly rewriting themselves to get along more; of course this process could be corrupted / disrupted / prevented, but it's the default.
• Example: On the longest time scales, love increases, hatred decreases.
  • Given more information about someone, your capacity for having {commune, love, compassion, kindness, cooperation} for/with them increases more than your capacity for {hatred, adversariality} towards them increases.
  • You can perfectly well hate someone who you don't know much about.
  • How much more can you hate someone by knowing more about them? Certainly you can learn things which make you hate them more. But if you kept learning even more, would you still be able to hate them?
  • You can be quite adversarial towards someone without knowing them. Everyone can meet in combat in the arena of convergent instrumental subgoals.
  • The exception to this conjecture:
    • It is possible to become extremely more adversarial towards someone by knowing much more about them--so you can pessimize against their values.
    • However, there is a strange sort of silver crack: Because love and cooperation are compatible with unbounded creativity, love and cooperation are unbounded. Therefore, to "keep up" with the love of your unbounded love, adversariality would need access to the unbounded expression of your values, in order to pessimize against them. But this seems to imply that the adversary has to continually give you more and more love, in order to access your values at each stage. Not sure what to make of this. (The outcome would still be very bad, but it's strange.)
  • It's harder to feel compassion towards someone you don't know much about; towards someone you do know much about, compassion is the easiest thing in the world to feel, if you try for a moment.
  • (This is just another example of how Buddhists are bad: faceless compassion is annihilation, not compassion. Yeah I know you like annihilation, but it's bad.)
  • Kindness and cooperation requires information, and (for humans) can increase without bound with more information.
  • Ender: It's impossible or almost impossible to understand someone without loving them.

Thanks to @JuliaHP [LW · GW] for a related conversation.

It is still the case that some people don't sign up for cryonics simply because it takes work to figure out the process / financing. If you do sign up, it would therefore be a public service to write about the process.

If there are some skilled/smart/motivated/curious ML people seeing this, who want to work on something really cool and/or that could massively help the world, I hope you'll consider reaching out to Tabula.

https://www.lesswrong.com/posts/SsLkxCxmkbBudLHQr/ [? · GW]

For its performances, current AI can pick up to 2 of 3 from:

• Interesting (generates outputs that are novel and useful)
• Superhuman (outperforms humans)
• General (reflective of understanding that is genuinely applicable cross-domain)

AlphaFold's outputs are interesting and superhuman, but not general. Likewise other Alphas.

LLM outputs are a mix. There's a large swath of things that it can do superhumanly, e.g. generating sentences really fast or various kinds of search. Search is, we could say, weakly novel in a sense; LLMs are superhumanly fast at doing a form of search which is not very reflective of general understanding. Quickly generating poems with words that all start with the letter "m" or very quickly and accurately answering stereotyped questions like analogies is superhuman, and reflects a weak sort of generality, but is not interesting.

ImageGen is superhuman and a little interesting, but not really general.

Many architectures + training setups constitute substantive generality (can be applied to many datasets), and produce interesting output (models). However, considered as general training setups (i.e., to be applied to several contexts), they are subhuman.

Recommendation for gippities as research assistants: Treat them roughly like you'd treat RationalWiki, i.e. extremely shit at summaries / glosses / inferences, quite good at citing stuff and fairly good at finding random stuff, some of which is relevant.

Protip: You can prevent itchy skin from being itchy for hours by running it under very hot water for 5-30 seconds. (Don't burn yourself; I use tap water with some cold water, and turn down the cold water until it seems really hot.)

(These are 100% unscientific, just uncritical subjective impressions for fun. CQ = cognitive capacity quotient, like generally good at thinky stuff)

• Overeat a bit, like 10% more than is satisfying: -4 CQ points for a couple hours.
• Overeat a lot, like >80% more than is satisfying: -9 CQ points for 20 hours.
• Sleep deprived a little, like stay up really late but without sleep debt: +5 CQ points.
• Sleep debt, like a couple days not enough sleep: -11 CQ points.
• Major sleep debt, like several days not enough sleep: -20 CQ points.
• Oversleep a lot, like 11 hours: +6 CQ points.
• Ice cream (without having eaten ice cream in the past week): +5 CQ points.
• Being outside: +4 CQ points.
• Being in a car: -8 CQ points.
• Walking in the hills: +9 CQ points.
• Walking specifically up a steep hill: -5 CQ points.
• Too much podcasts: -8 CQ points for an hour.
• Background music: -6 to -2 CQ points.
• Kinda too hot: -3 CQ points.
• Kinda too cold: +2 CQ points.

(stimulants not listed because they tend to pull the features of CQ apart; less good at real thinking, more good at relatively rote thinking and doing stuff)

"The Future Loves You: How and Why We Should Abolish Death" by Dr Ariel Zeleznikow-Johnston is now available to buy. I haven't read it, but I expect it to be a definitive anti-deathist monograph. https://www.amazon.com/Future-Loves-You-Should-Abolish-ebook/dp/B0CW9KTX76

The description (copied from Amazon):

A brilliant young neuroscientist explains how to preserve our minds indefinitely, enabling future generations to choose to revive us

Just as surgeons once believed pain was good for their patients, some argue today that death brings meaning to life. But given humans rarely live beyond a century – even while certain whales can thrive for over two hundred years – it’s hard not to see our biological limits as profoundly unfair. No wonder then that most people nearing death wish they still had more time.

Yet, with ever-advancing science, will the ends of our lives always loom so close? For from ventilators to brain implants, modern medicine has been blurring what it means to die. In a lucid synthesis of current neuroscientific thinking, Zeleznikow-Johnston explains that death is no longer the loss of heartbeat or breath, but of personal identity – that the core of our identities is our minds, and that our minds are encoded in the structure of our brains. On this basis, he explores how recently invented brain preservation techniques now offer us all the chance of preserving our minds to enable our future revival.

Whether they fought for justice or cured diseases, we are grateful to those of our ancestors who helped craft a kinder world – yet they cannot enjoy the fruits of the civilization they helped build. But if we work together to create a better future for our own descendants, we may even have the chance to live in it. Because, should we succeed, then just maybe, the future will love us enough to bring us back and share their world with us.

Discourse Wormholes.

In complex or contentious discussions, the central or top-level topic is often altered or replaced. We're all familiar from experience with this phenomenon. Topologically this is sort of like a wormhole:

Imagine two copies of $R^3$ minus the open unit ball, glued together along the unit spheres. Imagine enclosing the origin with a sphere of radius 2. This is a topological separation: The origin is separated from the rest of your space, the copy of $R^3$ that you're standing in. But, what's contained in the enclosure is an entire world just as large; therefore, the origin is not really contained, merely separated. One could walk through the enclosure, and pass through the unit ball boundary, and then proceed back out through the unit ball boundary into the other alternative copy of $R^3$.

You come to a crux of the issue, or you come to a clash of discourse norms or background assumptions; and then you bloop, where now that is the primary motive or top-level criterion for the conversation.

This has pluses and minuses. You are finding out what the conversation really wanted to be, finding what you most care about here, finding out what the two of most ought to fight about / where you can best learn from each other / the highest leverage ideas to mix. On the other hand, you lose some coherence; there is disorientation; it's harder to build up a case, integrate information into single nodes for comparison; and it's harder to follow. [More theory could be done here.]

How to orient to this? Is there a way to use software to get more of the pluses and fewer of the minuses, e.g. in order to have better debates [LW · GW]? E.g. by providing orienting structure with signposts and reminders but without clumsy artificial rigid restrictions on the transference of salience?

Say a "deathist" is someone who says "death is net good (gives meaning to life, is natural and therefore good, allows change in society, etc.)" and a "lifeist" ("anti-deathist") is someone who says "death is net bad (life is good, people should not have to involuntarily die, I want me and my loved ones to live)". There are clearly people who go deathist -> lifeist, as that's most lifeists (if nothing else, as an older kid they would have uttered deathism, as the predominant ideology). One might also argue that young kids are naturally lifeist, and therefore most people have gone lifeist -> deathist once. Are there people who have gone deathist -> lifeist -> deathist? Are there people who were raised lifeist and then went deathist?

The standard way to measure compute is FLOPS. Besides other problems, this measure has two major flaws: First, no one cares exactly how many FLOPS you have; we want to know the order of magnitude without having to incant "ten high". Second, it sounds cute, even though it's going to kill us.

I propose an alternative: Digital Orders Of Magnitude (per Second), or DOOM(S).

(Speculative) It seems like biotech VC is doing poorly, and this stems from the fact that it's a lot of work to discriminate good from bad prospects for the biology itself. (As opposed to, say, ability of a team to execute a business, or how much of a market there is, etc.) If this is true, have some people tried making a biotech VC firm that employs bio experts--like, say, PhD dropouts--to do deep background on startups?