Open Thread February 2019

I’m not very new, but I’ve been mostly lurking, so I think I’ll introduce myself. (Note: 1k words.)

Basic information: high school student, fairly socially clueless. Probably similar to how most people here were as teenagers (if not now) - smart, nerdy, a bit of a loner, etc. I’m saying this because I think it’s relevant enough in limited ways. (My age is relevant to the life plans I describe. The social cluelessness tells you that you should ignore any odd signals I send between the lines, because I didn’t intend to send them. Is there a conversational code, similar in type to Crocker’s rules, that says “I will send as much important information explicitly as possible, please err on the side of ignoring implicit signals”?)

My first introduction to the rationalist community was through Scott Alexander. A few years ago, I was in an online discussion about gender, it turned to tolerance, and somebody linked I Can Tolerate Anything Except The Outgroup. I got hooked on SSC’s clarity and novel ideas, and eventually that led me to LW. I’ve read HPMOR, the Sequences, and so on. It’s taken a while for it to sink in, but it’s had a large effect on my views. Prior to this, I was a fairly typical young atheist nerd, so the changes aren’t very drastic, but I often find myself using the ideas I’ve gotten through the Sequences and the mindset of analytic truthseeking. The object-level belief changes I’ve had are the obvious ones: many worlds, cryonics, intelligence explosion, effective altruism, etc. I’m a humanist transhumanist reductionist materialist atheist, like almost everyone else here. That’s a cluster-membership description, not tribe-membership. Language doesn’t make it easy to Keep Your Identity Small.

I’ve always wanted to go into a career in STEM. I’ve loved mathematics from a young age, and I’ve done pretty well at it too. I started taking university calculus in middle school, and upper-division university mathematics in high school. (I’m not saying that to brag - that wouldn’t even be effective here anyway - but to show that I’m not just a one-in-ten “good at math” person who thinks very highly of themselves. I think I’m at the one-in-ten-thousand level, but I don’t have high confidence in that estimate.) A few months ago, I decided that the best way to achieve my goals was to work in the field of AI risk research. I think I can make progress in that field, and AI risk is probably the most important field in history, so it’s the best choice. (Humanity needs to solve AI risk soon. My 50% estimate for the Singularity is 2040-2060, and the default is we all die. But you’ve heard this before.) I aim to work at MIRI, or a similar organization if that doesn’t work out. It’s a rather high goal I’ve set for myself, but if I can’t have immodest ambitions here, where can I have them? I’ve been accepted to a (roughly) top 10 university for Math/CS, and I read somewhere (80k Hours?) that’s the rough talent level necessary to do good AI risk research, so I don’t think it’s an impossible goal.

My biggest failure point is my inability to carry out goals. That’s what my inner Murphy says would cause my failure to get into MIRI and do good work. That’s probably the most important thing I’m currently trying to get out of LW - the Hammertime sequence looks promising. If anyone has any good recommendations for people who can’t remember to focus, I’d love them.

In fact, any recommendations would be greatly appreciated. Such as: what would you say to someone who’s going into college? What would you say to someone who wants to work in AI risk?

I’m currently working through the MIRI research guide, starting with Halmos’s Naive Set Theory. If anyone else is doing this and would like a study partner, we should study together.

I’ve read some things about AI risk, both through the popularizations available on LW/SSC and through a couple of papers. I’ve had a few ideas already. My Outside View is sane, and I know there’s a very low chance that I’ve seen something that everyone else missed. Should I write a post on LW about it anyway?

To give an example, here’s the idea on my mind right now: it’s probably not possible to encode all our values explicitly into an AI. The obvious solution is to build into it the ability to learn values. This means it’ll start in a state of “moral ignorance”, and learn what it “should want to do” by looking at people. I’m not saying it’ll copy people’s actions, I’m saying its actions will have to be somehow entangled with what humans are and do. Information theory and so on. The crucial point: before it “opens its eyes”, this AI is not a classical consequentialist, right? Classical consequentialists have a ranking over worlds that doesn’t vary by each world. This AI’s terminal goals change depending on which possible world it’s in! I want to explore the implications - are there pitfalls? Does this help us solve problems? What is the best way to build this kind of agent? Should we even build it this way or not? I also want to formalize this kind of agent. It seems very similar to UDT in a sense, so perhaps it’s a simple extension of UDT. But there’s probably complications, and it’s worth turning the fuzzy ideas into math.

Some questions I have: does this seem like a possibly fruitful direction to look? Has someone already done something like this? What advice do you have for someone trying to do what I’m doing? Is there a really good AI risk paper that I could look at and try to mimic in terms of “this is how you formalize things, these are the sorts of questions you need to answer, etc.”? Is there anyone who’d be interested in mentoring a young person who’s interested in the field? (Connotation clarification: communicating, giving advice, kind of a back-and-forth maybe? I don’t know what’s okay to ask for and what’s not, because I’m a young clueless person. But I’m really interested and motivated, and Asking For Help is important, so I’ll put this out there and hope people interpret it charitably.)

How much actual time do you spend when you read a paper deeply?

I am currently reading the Minimum Entropy Production Principle paper, dedicating some time to it every day. I am about 3 hours deep now, and only half finished. I am not going as far as to follow all the derivations in detail, only taking the time to put the arguments in the context of what I already know. I expect it will take more than one reading to internalize.

Sometimes I worry this is so slow I am spending my time poorly, but I haven't any idea how it goes for more serious people, so I thought I would ask.

It was suggested [LW · GW] that alien AI, if it wants to be visible, could create an artificial quasar, and a groups of such quasars could be visible as clearly artificial object on the distances of at least several billion light years. In this comment I will look on the ways how such quasar could be created with surprisingly small efforts and estimate need time for it.

Let's assume that aliens don't have any magical technology to move stars or to convert energy in matter.

In that case, they could create a quasar by directing many normal stars to the center of the galaxy: falling stars will increase the accretion rate in the central black hole and thus its luminosity, and the aliens could regulate the rate and types of falling stars to make changes in the quasar luminosity.

But how to move stars? One idea is that if aliens could change a trajectory of a star slightly, it will eventually pass near another star, make a "gravitational manoeuvre" and will fall to the center of the galaxy. Falling to the center of the galaxy would probably require tens of millions years (based on Sun's orbital period of 250 mln years).

But how make small changes in the trajectory of a star? One idea is to impact the star with large comets. It is not difficult, as remote Oort cloud objects (or wandering small planets, as they are not part of already established orbital movement of the star) need only small perturbations to start falling down on the central star, which could be done via nuclear explosions or smaller impacts by smaller astreoids.

The impacts with comets will have very small effects on the star's trajectory. For example, Pluto's mass is 100 million times less than Sun's mass and impact with Pluto-size object will probably change Sun's trajectory only on 1 mm/sec, which will turn into 1 billion km difference in 20 million years. Close flyby by stars are very rare, so it may take tens of million of years of very complex space billiard to organise need flyby.

All this suggests that creating an artificial quasar is possible, but it may take up to 100 million years in a typical galaxy; changing the galaxy's luminosity by tiling it with Dyson Spheres could be probably done much quicker, in a less than 1 million years. Thus, creating artificial quasars as beacons makes sense only if the difference in 100 mln years is not substantial.

I've been writing a simulism essay that strives to resolve a paradox of subjectivity-measure concentration by rolling over a few inconvenient priors about physics towards a halfway plausible conception of naturally occuring gods. I think it's kind of good, but I've been planning on posting it on April 1st because of the very obvious bias that has been leading my hand towards humanity's favourite deus ex machina ("The reason the universe is weird is that a very great big person did it" (to which I answer, "But a great big person, once such beings exist, totally would do it!"))

It will only be funny if it's posted in a context where people might take it halfway seriously, but I'm not sure it's appropriate to post it to lesswrong. If people upvote it, it will still be here on April 2nd, and that might be kind of embarrassing. I'm not sure where to put it.

Summary: It's weird that anthropic measure seems to be concentrated in humans and absent from rock or water or hydrogen (We each have only one data point in favour of that seeming, though). It's plausible that a treaty-agency between mutually alien species would optimise the abundance of life. If universes turn out to be permeable under superintelligence (very conceivable IMO), and if untapped energy turns out to be more common than pre-existing entropy then the treaty-agency could spread through the universe and make more of it alive than not, and if this has occurred, it explains our measure concentration weirdness, and possibly the doomsday weirdness ("if the future will contain more people than the past, it's weird that we're in the past") as well.

Its many predications also include: Either entropy has no subjectivity (I'd have no explanation for this, although it seems slightly intuitive), or perpetual computers (life that produces no heat) within a universe that contains some seeds of entropy already are somehow realisable under superintelligence (o_o;;;;,, Would bet we can refute that already. It might be fun to see if we can figure out a method a superintelligent set of cells in a conway's gol universe could contain a section of randomly initialised cells that it does not know the state of. My current guess is we'd be able to prove that there is no method that works in 90% of possible cases)

Apropos of the negative utilitarianism question posted recently, has anyone read any pessimism? I picked up The Conspiracy Against the Human Race: A Contrivance of Horror relatively recently. It is a survey, written by Thomas Ligotti, who is a horror and weird fiction writer.

It is gloriously grim. I recommend against it if you are in a sensitive place, however.

(I'm beginning to think that if "natural sciences" tell us _how_ things happen but not _why_ they do... then in order to know how I should ask why.)