Posts
Comments
"That sounds really interesting. If a company working on the problem was paying a lot, I would consider jumping ship."
The Chinese stated preferences here closely track Western revealed preferences. Americans are more likely to dismiss AI risk post-hoc in order to justify making more money, whereas it seems that Chinese people are less likely to sacrifice their epistemic integrity in order to feel like a Good Guy, Hire people, and pay them money!
So that's how you draft scissor statements >:)
Yes. Too many cached decisions.
How did you choose the salary range?
My understanding here is that while this is true, it will discourage the 5%, who will just go work for FAANG and donate money to someone worse (or someone overwhelmed with work), simultaneously losing any chance at a meaningful job. The point being that yes, it's good to donate, but if everyone donates (since that is the default rat race route), noone will do the important work.
No! If everyone donates, there will be enough money to pay direct workers high salaries. I know this goes contra to the image of the selfless, noble Effective Altruist, but if you want shit to get done you should pay people lots of money to do it.
A human life, with all its joys and all its pains, adding up over the course of decades, is worth far more than your brain’s feelings of comfort or discomfort with a plan.
In case you're wondering why Rationalists Don't Win, it's because they're too busy lying to themselves about what their true preferences are.
Good point. Strong-downvoted the post.
Ok, sick. I largely agree with you btw (about the hamster wheel being corrosive). If I came off as agressive, fyi, I liked the spirit of your post a lot, and I strong-upvoted it.
Yes, selfish agents want to not get turned into paperclips. But they have other goals too. You can prefer alignment be solved, while not wanting to dedicate your mind, body, and soul to waging a jihad against it. Where can Charlie effectively donate, say, 10% of his salary to best mitigate x-risk? Not MIRI (according to MIRI).
You don't need that many resources.
True, if we're talking solely about alignment. If we're talking about the larger space of, as you put them, "maximize objective function"-type goals, then there's plenty of demand for resources. Let's say I wanna do (actually effective) longevity research. Since the competition for grant money is (like most things) Goodharted and broken, and because I don't have enough biology credentials, I'm gonna need to self-fund in order to buy lab materials and grad student slaves.
If you have EA-like goals, you have a "maximize objective function"-type goal. It's in the same shape as "become as rich as possible" or "make the world as horrible as possible." Basically, the conventional path is highly highly unlikely to get you all the way there. In this case, you probably want to get into the
- Get skills+resources
- Use skills+resources to do impact
- Repeat
Loop.
I'm in a similar situation to yours. (I'm currently in the "Bob" stage of the Alice -> Bob -> Charlie pipeline.) How do you propose I, and those like us, go about doing step 1 without entering the "software/finance hamster wheel"? Are we supposed to found a dozen start-ups until one of them works? Are we supposed to find and exploit some massive inefficiency in crypto pricing, all by ourselves? Please, do tell.
One of the problems here is that, as of right now, there isn't much of a middle path between "Stay at Google and do nothing" and "quit your job to do alignment work full-time". Then there's the issue of status-seeking vs. altruism as a case of revealed vs. stated preferences. If there was a way to make $750k a year and save the world, people would be all over that. I, personally, would be all over that.
But there isn't. If we use johnswentworth as an optimistic case, those who would go into independent AI alignment work full-time would make about $90k per year. Of course, anyone that complains about the prospect of 'only' making 90k would be derided as a snot-nosed asshole, because we live in a world where hundreds of millions of people subsist upon less than two dollars a day. However, people's internal calculus chugs on just the same, and Charlie decides to stay at Google.
I endorse Rohin Shah's response to that post.
You might think "well, obviously the superintelligent AI system is going to care about things, maybe it's technically an assumption but surely that's a fine assumption". I think on balance I agree, but it doesn't seem nearly so obvious to me, and seems to depend on how exactly the agent is built. For example, it's plausible to me that superintelligent expert systems would not be accurately described as "caring about things", and I don't think it was a priori obvious that expert systems wouldn't lead to AGI. Similarly, it seems at best questionable whether GPT-3 can be accurately described as "caring about things".
Even humans have a huge range of intellectual capacity, and someone who is good at math may not be good at say, writing a novel. So the idea of "general intelligence" is pretty weak from the outset, and it's certainly not a binary value that you either have or have not.
I find point no. 4 weak.
- Unaligned AGI will try to do something horrible to humans (not out of maliciousness, necessarily, we could just be collateral damage), and will not display sufficiently convergent behavior to have anything resembling our values.
I worry that when people reason about utility functions, they're relying upon the availability heuristic. When people try to picture "a random utility function", they're heavily biased in favor of the kind of utility functions they're familiar with, like paperclip-maximization, prediction error minimization, or corporate profit-optimization.
How do we know that a random sample from utility-function-space looks anything like the utility functions we're familiar with? We don't. I wrote a very short story to this effect. If you can retroactively fit a utility function to any sequence of actions, what predictive power do we gain by including utility functions into our models of AGI?
https://en.wikipedia.org/wiki/Brain_stimulation_reward
https://doi.org/10.1126/science.140.3565.394
https://sci-hub.hkvisa.net/10.1126/science.140.3565.394
In our earliest work with a single lever it was noted that while the subject would lever-press at a steady rate for stimulation to various brain sites, the current could be turned off entirely and he would continue lever-pressing at the same rate (for as many as 2000 responses) until told to stop.
It is of interest that the introduction of an attractive tray of food produced no break in responding, although the subject had been without food for 7 hours, was noted to glance repeatedly at the tray, and later indicated that he knew he could have stopped to eat if he wished. Even under these conditions he continued to respond without change in rate after the current was turned off, until finally instructed to stop, at which point he ate heartily.
It's to push our understanding of some crazy niche field and push it further beyond. The widespread recognition is nice, I'm sure, but I don't think most PhD graduates' goals are this.
Insufficiently Hansonpilled.
It's time for me to shill for Big Pharma again.
I spent years trying to come up with a mental strategy that would reliably generate willpower, all to no avail. To my knowledge, there is one (1) way to cut through this Gordian knot, and that way is amphetamines.
- It is easy to get a psychiatrist to prescribe you amphetamines.[1]
- If you request generic drugs over brand-name, they're pretty inexpensive.
- Amphetamines greatly reduce one's appetite; Adderall was originally sold as a diet pill. For the first time in my life, I could empathize with those people who just forget to eat.
- Health risk is minimal.[1:1] In fact, on account of the caloric restriction[2], you could make a solid case that amphetamine supplementation actually increases your expected lifespan.
- In my experience (and many others'), amphetamines reliably inhibit akrasia.
- People on this site often conclude that akrasia is their primary bottleneck, and then try to solve that problem with a more elaborate to-do list system. I've been there, and it's ineffective. Dextroamphetamine is effective.
Hamming distance assumes that the strings you're comparing are the same length, but "Yavin" is shorter than "Yarvin". Levenshtein distance is the smallest number of insertions, deletions, or character substitutions required to get from string A to string B, while Hamming distance only counts char-substitutions.
Epistemic status: Shaky
That's not how it works. People bred chihuahuas from wolves, and the blue whale evolved from a goat-sized land mammal. If the additive genes that control spinal length just upregulate some pre-existing developmental process, it's not too much of a leap to assume that the additive genes that control intelligence just turn up a similar developmental knob. That would imply that intelligence-increasing mutations are common (being less complex), and that intelligence could be selected for like any other trait.
Within humans, intelligence-related genes behave additively. That is, the predicted IQ of a child of a 95 IQ mother and a 115 IQ father follows a normal distribution centered at 105. Since "general intelligence" exists to some degree in octopuses, we're amplifying an existing trait, rather than creating one de novo. If intelligence-related genes behave additively in octopuses as well (and we should assume that they do), then breeding octopi for intelligence should be as straightforward as breeding cows for milk.
Well, "Yarvin" is closer in Levenshtein distance, plus you're acting coy, so I'm updating towards "Yarvin" 😎
Some Yavin guy posted some good stuff. Then I found out he had his own blog. That was pretty good.
Is "Yavin" supposed to be "Yarvin" or "Yvain"? Both are quite plausible.
This is a Moldbug argument, but:
Markets aren't totally efficient, but they are the most adequate part of our civilization. For-profit corporations usually follow the same general structure, so the best form of governance looks like a joint-stock company.
- The "electrode in the reward center" setup has been proven to work in humans, whereas jhanas may not tranfer over Neuralink.
- Deep brain stimulation is FDA-approved in humans, meaning less (though nonzero) regulatory fuckery will be required.
- Happiness is not pleasure; wanting is not liking. We are after reinforcement.
Judging by the fox domestication experiment, intelligence could emerge way sooner than 50 generations. The fox experiment used a population size on the order of 1e2, and selected the top 20% most docile foxes to act as the breeding pool for the next generation. You could accelerate this process by increasing the population size, while selecting a smaller fraction of intelligent octopuses.
Phantoms in the Brain by V.S. Ramachandran is a fun read, and will teach you a thing or two about neuropsychology.
Code by Simon Singh is a nice, low-effort intro to cryptography.
- Eco-terrorism, animal rights terrorism, educational terrorism (!), and many other forms of terrorism are morally justified and should be practiced by advocates of those causes.
Ah yes, the old "let's bomb DeepMind" approach to AI safety.
- The virtually universal practice of assigning a permanent name at birth (which only exists so that governments can more easily tax us) has caused a species-wide shift towards a more narcissistic and egotistical mode of being.
Neither of us are using our government-assigned names right now. Has the internet-culture norm of creating pseudonymous usernames countered this shift?
- Lab-grown human meat is the only food item which can be ethically consumed (it is just as wrong to eat plants as it is to eat animals). Overcoming the aversion to lab-grown meat and cannibalism will be challenging; we need to think outside of the box.
Until such technology is developed, all EAs are advised to subsist upon bukkake.
- Money is, in fact, the root of all evil and we will never be free until all monetary systems are abolished.
Money allows people to express their preferences more than they otherwise could. People's revealed preferences are invariably more evil than their stated preferences. Money reveals evil, but it does not create it.
- We will never stop fighting about race. The only way out of this racial hellscape is to simply breed race away by tweaking dating app algorithms to increase the number of interracial matings.
First, this will never work, because people have strong racial preferences. Second, people will still fight about racial features even if everyone is mixed. Just look at Brazil.
Exquisitely based
I'm glad Demis is somewhat reasonable. How tf can we pull the fire alarm? A petition?
Eliezer replied to a comment of mine recently, coming out in favor of going down the human augmentation path. I also think genetically engineered Von Neumann babies are too far off to be realistic.
If we can really crack human motivation, I expect possible productivity gains of maybe one or two OOM.
Picture a pair of reseachers, one of whom controls an electrode wired to the pleasure centers of the other. Imagine they have free access to methamphetamine and LSD.
You don't need to be a genius to make this happen.
Great! If I recall correctly, you wanted genetically optimized kids to be gestated and trained.
I suspect that akrasia is a much bigger problem than most people think, and to be truly effective, one must outsource part of their reward function. There could be massive gains.
What do you think about the setup I outlined, where a pair of reseachers exist such that one controls an electrode embedded in the other's reward center? Think Focus from Vinge's A Deepness In The Sky.
Fine. What do you think about the human-augmentation cluster of strategies? I recall you thought along very similar lines circa ~2001.
I'm not well calibrated on sub 1% probabilities. Yeah, the odds are low.
There are other classes of Hail Mary. Picture a pair of reseachers, one of whom controls an electrode wired to the pleasure centers of the other. Imagine they have free access to methamphetamine and LSD. I don't think research output is anywhere near where it could be.
The post assumes that there are no ethics-violating strategies that will work. I understand that people can just-world-fallacy their way into thinking that they will be saved if only they sacrifice their deontology. What I'm saying is that deontology-violating strategies should be adopted if they offer, say, +1e-5 odds of success.
These are all strategies to buy time, so that alignment efforts may have more exposure to miracle-risk.
I can imagine a plausible scenario in which WW3 is a great thing, because both sides brick each other's datacenters and bomb each other's semiconductor fabs. Also, all the tech talent will be spent trying to hack the other side and will not be spent training bigger and bigger language models.
-
Coordination (cartelization) so that AI capabilities are not a race to the bottom
-
Coordination to indefinitely halt semiconductor supply chains
-
Coordination to shun and sanction those who research AI capabilities (compare: coordination against embyronic human gene editing)
-
Coordination to deliberately turn Moore's Law back a few years (yes, I'm serious)
Dignity exists within human minds. If human-descended minds go extinct, dignity doesn't matter. Nature grades us upon what happens, not how hard we try. There is no goal I hold greater than the preservation of humanity.