Posts
Comments
How do we do this without falling into the Crab Bucket problem AKA Heckler's Veto, which is definitely a thing that exists and is exacerbated by these concerns in EA-land? "Don't do risky things" equivocates into "don't do things".
A medieval peasant would very much disagree with that sentence, if they were suddenly thrust into a modern grocery store. I think they would say the physical reality around them changed to a pretty magical-seeming degree.
They would still understand the concept of paying money for food. The grocery store is pretty amazing but it's fundamentally the same transaction as the village market. I think the burden of proof is on people claiming that money will be 'done away with' because 'post-scarcity', when there will always be economic scarcity. It might take an hour of explanation and emotional adjustment for a time-displaced peasant to understand the gist of the store, but it's part of a clear incremental evolution of stores over time.
- They think a friendly-AGI-run society would have some use for money, conflict, etc. I'd say the onus is on them to explain why we would need those things in such a society.
I think a basically friendly society is one that exists at all and is reasonably okay (at least somewhat clearly better) compared to the current one. I don't see why economic transactions, conflicts of all sorts, etc wouldn't still happen, assuming the lack of existentially-destructive ones that would preclude the existence of such a hypothetical society. I can see the nature of money changing, but not the fundamentals of there being trades.
I don't think AI can just decide to do away with conflicts via unilateral fiat without an enormous amount of multipolar effort, in what I would consider a friendly society not ran by a world dictator. Like, I predict it would be quite likely terrible to have an ASI with such disproportionate power that it is able to do that, given it could/would be co-opted by power-seekers.
I also think that trying to change things too fast or 'do away with problems' is itself something trending along the spectrum of unfriendliness from the perspective of a lot of humans. I don't think the Poof Into Utopia After FOOM model makes sense, that you have one shot to send a singleton rocket into gravity with the right values or forever hold your peace. This thing itself would be an unfriendly agent to have such totalizing power and make things go Poof without clear democratic deliberation and consent. This seems like one of the planks of SIAI ideology that seems clearly wrong to me, now, though not indubitably so. There seems to be a desire to make everything right and obtain unlimited power to do so, and this seems intolerant of a diversity of values.
This seems to be a combo of the absurdity heuristic and trying to "psychoanalyze your way to the truth". Just because something sounds kind of like some elements of some religions, does not make it automatically false.
I am perfectly happy to point out the ways people around here obviously use Singularitarianism as a (semi-)religion, sometimes, as part of the functional purpose of the memetic package. Not allowing such social observations would be epistemically distortive. I am not saying it isn't also other things, nor am I saying it's bad to have religion, except that problems tend to arise. I think I am in this thread, on these questions, coming with more of a Hansonian/outside view perspective than the AI zookeeper/nanny/fully automated luxury gay space communism one.
Nitpicking a particular topic of interest to me:
Power/money/being-the-head-of-OpenAI doesn't do anything post-singularity.
It obviously does?
I am very confused why people make claims in this genre. "When the Singularity happens, this (money, conflict, the problems I'm experiencing) won't be a problem anymore."
This mostly strikes me as magical, far-mode thinking. It's like people have an afterlife-shaped hole after losing religion. The specific, real reality in front of you won't magically suddenly change after an Intelligence Explosion and assuming we're alive in some coherent state. Money and power are very, very likely to still exist afterwards, just in a different state that makes sense as a transformation of the current world.
I will keep harping on that more people should try starting (public benefit) corporations instead of nonprofits. At least, give it five minutes' thought. Especially if handwaves impact markets something something. This should be in their Overton Window, but it might not be because they automatically assume "doing good => charity => nonprofit". Corporations are the standard procedure for how effective helpful things are done in the world; they are RLHF'd by the need to acquire profit by providing real value to customers, reducing surfacce area for bullshitting. I am not an expert here by any means, but I'm noticing the fact that I can go on Clerky or Stripe Atlas and spend a couple hours spinning up an organization, versus, well, I haven't actually gone through with trying to incorporate a nonprofit, but the process seems at least 10x more painful based on reading a book on it and with how many people seek fiscal sponsorship. I'm pretty surprised this schlep isn't talked about more. Having to rely on fiscal sponsorship seems pretty obviously terrible to me, and I hadn't even considered the information-distortive effects here. I would not be caught dead being financially enmeshed with the EVF umbrella of orgs after FTX. From my naive perspective, the castle could have easily been a separate business entity with EVF having at least majority control?
(I just realized I'm on LessWrong and not EA Forum, and could have leaned harder into capitalismpunk without losing as many social points.)
The wifi hacking also immediately struck me as reminiscent of paranoid psychosis. Though a significant amount of psychosis-like things are apparently downstream of childhood trauma, including sexual abuse, but I forget the numbers on this.
I've worried about it's sustainability, but do you think it's been a good path for you?
Cutting out bird and seafood products (ameliatarianism) is definitely more sustainable for me. I'm very confused why you would think it's less sustainable than, uh, 'cold turkey' veganism. "Just avoid chicken/eggs" (since I don't like seafood or the other types of bird meat products) is way easier than "avoid all meat, also milk, also cheese".
Similar for me. I was very suspicious at first that the first message was a Scam and if I clicked I would blow up the website or something tricksy. Then with the second message I thought it might be customized to test my chosen virtue, "resisting social pressure", so I didn't click it.
"You’ve never experienced bliss, and so you’re frantically trying to patch everything up and pin it all together and screw the universe up so that it's fixed." - Alan Watts
Did you actually bet the money?
People in MIRI/CFAR/LessWrong ~actively resisted the idea of a marketing push optimized more along dimensions of mass persuadability, for better or worse. One reason is that there is inertia once you've built a mass movement with MoPs who can't dialogue like on this site. My straw model is they think "we just need to produce technical insights and communicate them" and other comms work is an opportunity cost or creates/incentivizes some kind of epistemic trap.
And this is the exact type of statement where I would expect this coincidence to pop up. Both are a reasonable number of days for bureaucracy to take, the large discrepancy between them is required for the complaint in the tweet to happen in the first place, and I would expect the number of days to be very specific rather than a round number.
I agree with this. Although I will note people are claiming it actually took 56 (IIRC) days for them to get back to him.
I think it's an entirely sensible inference in the world where it is true that it was intentional, albeit a highly reductive description of the actual psychological reality of someone who holds those beliefs enough to output such symbols. In that world, he could also be paying lipservice to pander or be trolling.
group homes (in adulthood as a long term situation)
People living together in group homes (as extended families) used to be the norm? The weird thing is how isolated and individualist we've become. I would argue that group houses where individual adults join up together are preserving some aspect of traditional social arrangement where people live closely, but maybe you would argue that this is not the same as an extended family or the lifelong kinship networks of a village.
Has the "AI Safety"-washing grift accelerated? https://www.lakera.ai/momentum
Also, when I actually think about what it would be like to have a Plan That I Truly Believe In, it doesn't really seem hard to get it on the desk of The Important Stakeholders (or someone who knows someone who talks to The Important Stakeholders).
I think you can scream a Good Plan from the rooftops and often few will listen. See: Covid in January-February 2020.
Math camps are not sufficient to solve civilization's problems.
Fascinating. Way, way more examples and empirical treatment of rituals would help me understand your case better.
Me, sitting on a throne, as your entirely benevolent world dictator. Oh, how did I get there? Someone posted on LessWrong and I followed their blueprint!
The story had me for most of it but I'm pretty disappointed by the very end. Protagonist seems to have unequivocally become the bad guy at that point. I don't think I would be confident enough in my understanding of consciousness to do that until I spent a lot more time understanding the situation, assuming it wasn't a self-defense emergency by that point. I get being angry but remaining in reflexively unthinking action hero mode rather than administering a battery of Turing Tests just seems wildly uncalled for. Maybe it's supposed to read as morally ambiguous, but like many readers here I am biased towards the Drangerian perspective and find the dilemma unsympathetic.
One possible approach to fixing this is to try to get wayyyy more empirical, and try to produce proof-of-concept implementations of various adversaries we are worried we might face in the future. My analogy would be, there’s a world of difference between speculating about the bogey monster and producing a grainy photo of the bogey monster; the second can at least maaaaaybe be discussed with skeptical people, whereas the first cannot (productively) be.
Anyway, that’s a long-winded way of saying, it seemed to me that it might be useful to implement a treacherous mesa-optimizer in a toy grid-world, so I did. Here is the colab. Below I give a brief recap of the high-level results.
I'm not saying this isn't helpful work worth the tradeoff, but this sounds like very early stage gain of function research? Have you thought about the risks of this line of investigation turning into that? :)
You mentioned a specific Amazon price point that I found surprisingly low and is cheap enough to meaningfully affect strategic plans. The cheapest pasta I've seen is on WebstaurantStore but the shipping doubles the cost to $2/lb approximately. Did you have a link for the $200/year product(s)?
What's the ~$200 food item you saw that would last a person a year, can you link?
(Edit: I see a 30-day container for about $200. It seems to me that $200 buys a month's worth, not a year's worth?)
I consider it collectively important that alignment researchers and their +1s survive, as well as other x-risk researchers and probably other cause areas.
So if there's a 1% yearly risk of nuclear apocalypse
Some think the number is much higher than priors due to current events. You're also not factoring in that that yearly percentage adds up, and a lot of preparations are a one-off action that benefits future you (assuming you don't dig into your backup food).
It felt to me like there's too much for my taste. My impression was that you guys were optimizing for it being about AI content, somewhat related to the % of people involved at Lightcone coworking being AI researchers vs other subjects.
Interesting summary and interpretation of a speech outlining Putin's intentions, "The End of Western Hegemony is INEVITABLE":
This is a reproduction of my live Twitter summary/translation of Vladimir Putin's speech:
I wish every single person in the West would listen to Putin's speech. Obviously, that won't happen so let me summarise as a professional translator for 10+ years. He states, as he has done from the outset, what his intentions and complaints are in the plainest terms possible.
Setting aside his brief comments on the recent "referendums", he spends most of his speech discussing the West. His primary complaint isn't NATO expansion, which gets only a cursory mention. The West is greedy and seeks to enslave and colonise other nations, like Russia.
The West uses the power of finance and technology to enforce its will on other nations. To collect what he calls the "hegemon's tax". To this end the West destabilises countries, creates terrorist enclaves and most of all seeks to deprive other countries of sovereignty.
It is this "avarice" and desire to preserve its power that is the reason for the "hybrid war" the collective West is "waging on Russia". They want us to be a "colony". They do not want us to be free, they want Russians to be a mob of soulless slaves - direct quote.
The rules-based order the West goes on about is "nonsense". Who made these rules? Who agreed to them? Russia is an ancient country and civilization and we will not play by these "rigged" rules. The West has no moral authority to challenge the referendums because it has violated the borders of other countries. Western elites are "totalitarian, despotic and apartheidistic" - direct quote. They are racist against Russia and other countries and nations. "Russophobia is racism". They discriminate by calling themselves the "civilised world".
They colonised, started the global slave trade, genocided native Americans, pillaged India and Africa, forced China to buy opium through war. We, on the other hand, are proud that we "led" the anti-colonial movement that helped countries develop to reduce poverty and inequality.
They are Russophobic (they hate us) because we didn't allow our country to be pillaged by creating a strong CENTRALISED (emphasis his) state based on Christianity, Islam, Judaism and Buddhism. They have been trying to destabilise our country since the 17th century in the Times of Trouble. Eventually, they managed to "get their hands on our riches" at the end of the 20th century. They called us friends and partners while pumping out trillions of dollars (his irony game is strong today).
We remember this. We didn't forget. The West claims to bring freedom and democracy to other countries but it's the exact opposite of the truth. The unipolar world is anti-democratic by its very nature. It is a lie. They used nuclear weapons, creating a precedent. They flattened German cities without "any military need to do so". There was no need for this except to scare us and the rest of the world. Korea, Vietnam. To this day they "occupy" Japan, South Korea and Germany and other countries while cynically calling them "allies".
The West has surveillance over the leaders of these nations who "swallow these insults like the slaves they are".
He then talks about bioweapon research (haven't heard about them for a while) and human experiments "including in Ukraine".
The US rules the world by the power of the fist. Any country which seeks to challenge Western hegemony becomes an enemy. Their neocolonialism is cloaked in lies like "containment" of Russia, China and Iran. The concept of truth has been destroyed with fakes and extreme propaganda (irony game still strong).
You cannot feed your people with printed dollars and social media. You need food and energy. But Western elites have no desire to find a solution to the food and energy crises *they* (emphasis his) created.
They solved the problems at the start of 20c with WW1 and the US established dominance of the world via the dollar as a result of WW2. In the 80s they had another crisis they solved by "plundering our country". Now they want to solve their problems by "breaking Russia".
Russia "understands its responsibility to the international community" and will "do everything to cool the heads of these neocolonials who are destined to fail".
They're crazy. I want to speak to all Russian citizens, do we want to replace mum and dad with parent 1 and 2?
They invented genders and claim you can "transition". Do we want this for our children?
We have a different vision.
They have abandoned religion and embraced Satanism - direct quote.
The world is going through a revolutionary transformation. A multipolar world offers nations freedom to develop as they wish and they make up the majority of the world.
We have many like-minded friends in Western countries. We see and appreciate their support. They are forming liberation, anti-colonial movements as we speak - direct quote. These will only grow.
We are fighting for a fair world for our country. The idea of exceptionalism is criminal and we must turn this shameful page. The breaking of the West's hegemony is INEVITABLE (emphasis his).
There is no going back. We are fighting for our "great (as in big), historic Russia". Our values are (irony game crescendo): love of our fellow man, compassion and mercy.
Truth is with us, Russia is with us.
That's the end of the speech. As I said from day 1, the purpose of what Putin is doing in Ukraine is to throw the West off its pedestal. This isn't about NATO or Ukraine, this is the big play to replace the current world order.
when aggression with conventional weapons greatly endangers Russia's existence
Putin could interpret an attack on its newly annexed territories as "greatly endangering Russia's existence". He seems to be generating rhetoric in that direction.
Russia's state faces an existential threat.
The implication is that attacks on the territories it is annexing are interpretable as an existential threat.
This post explains more, I don't have any other info: https://www.lesswrong.com/posts/F7RgpHHDpZYBjZGia/high-schoolers-can-apply-to-the-atlas-fellowship-usd50k
A related concept: https://twitter.com/mnovendstern/status/1495911334860693507
I wrote this before it was cool!
Have you looked at the Atlas Fellowship, btw?
The presence of pathogens in an environment (including the so-called Covid-19 coronavirus [53]) isn't the prime factor whether people become ill.
Oh, huh. How does that explain the Black Plague, smallpox, ebola, HIV?
"...What do you do with this impossible challenge?
First, we assume that you don't actually say "That's impossible!" and give up a la Luke Skywalker. You haven't run away.
Why not? Maybe you've learned to override the reflex of running away. Or maybe they're going to shoot your daughter if you fail. We suppose that you want to win, not try—that something is at stake that matters to you, even if it's just your own pride. (Pride is an underrated sin.)
Will you call upon the virtue of tsuyoku naritai? But even if you become stronger day by day, growing instead of fading, you may not be strong enough to do the impossible. You could go into the AI Box experiment once, and then do it again, and try to do better the second time. Will that get you to the point of winning? Not for a long time, maybe; and sometimes a single failure isn't acceptable.
(Though even to say this much—to visualize yourself doing better on a second try—is to begin to bind yourself to the problem, to do more than just stand in awe of it. How, specifically, could you do better on one AI-Box Experiment than the previous?—and not by luck, but by skill?)
Will you call upon the virtue isshokenmei? But a desperate effort may not be enough to win. Especially if that desperation is only putting more effort into the avenues you already know, the modes of trying you can already imagine. A problem looks impossible when your brain's query returns no lines of solution leading to it. What good is a desperate effort along any of those lines?
Make an extraordinary effort? Leave your comfort zone—try non-default ways of doing things—even, try to think creatively? But you can imagine the one coming back and saying, "I tried to leave my comfort zone, and I think I succeeded at that! I brainstormed for five minutes—and came up with all sorts of wacky creative ideas! But I don't think any of them are good enough. The other guy can just keep saying 'No', no matter what I do."
And now we finally reply: "Shut up and do the impossible!"
As we recall from Trying to Try, setting out to make an effort is distinct from setting out to win. That's the problem with saying, "Make an extraordinary effort." You can succeed at the goal of "making an extraordinary effort" without succeeding at the goal of getting out of the Box.
"But!" says the one. "But, SUCCEED is not a primitive action! Not all challenges are fair—sometimes you just can't win! How am I supposed to choose to be out of the Box? The other guy can just keep on saying 'No'!"
True. Now shut up and do the impossible.
Your goal is not to do better, to try desperately, or even to try extraordinarily. Your goal is to get out of the box."
lol no one here said "obtain some Bitcoin"
https://www.statmuse.com/money/ask/bitcoin+price+may+15+2013
Noting that the more real, second-order disaster resulting from Chernobyl may have been less usage of nuclear power (assuming that had an influence on antinuclear sentiment). Likewise, I'm guessing the Challenger disaster had a negative influence on the U.S. space program. Covid lockdowns also have this quality of not tracking the cost-benefit of their continuation. Human reactions to disasters can be worse than the disasters themselves, especially if the costs of those reactions are hidden. I don't know how this translates to AI safety but it merits thought.
Dropping this paper here as what I know to be the canonical text on this subject.
https://longtermrisk.org/files/Multiverse-wide-Cooperation-via-Correlated-Decision-Making.pdf
“He has half the deed done who has made a beginning.”
– Horace
This happens intergenerationally as parents forget to alert their children to the actual reasons for things. Having observed this happen with millenials, I am scared of what we are all collectively missing because older generations literally just forgot to tell us.
What do you think we are missing?
A crucial consideration for why destroying restaurant business is good: factory farming.
Hey Ron, I am working on my own version of this (inspired by this Sequence), and would love to get your advice! Right now I am focusing on crowdfunding via dominant assurance contracts on Ethereum.
How did you / would you verify that someone did something? What are specific examples of that happening for different actions? What kinds of evidence can be provided? I have a fuzzy sense of what this looks like right now. The closest sites I can think of just off the top of my head that involve verification are CommitLock (which I made a successful $1000 commitment contract on to get myself to do swim lessons) and DietBet, which requires a photo of your scale (it also has that 'split the pot' feature you mentioned, which I am pretty excited for).
I am very interested in practicing steelmanning/Ideological Turing Test with people of any skill level. I have only done it once conversationally and it felt great. I'm sure we can find things to disagree about. You can book a call here.
I’ve mentioned previously that I’ve been digging into a pocket of human knowledge in pursuit of explanations for the success of the traditional Chinese businessman. The hope I have is that some of these explanations are directly applicable to my practice.
Here’s my current bet: I think one can get better at trial and error, and that the body of work around instrumental rationality hold some clues as to how you can get better.
I’ve argued that the successful Chinese businessmen are probably the ones who are better at trial and error than the lousier ones; I posited that perhaps they needed less cycles to learn the right lessons to make their businesses work.
I think the body of research around instrumental rationality tell us how they do so. I’m thankful that Jonathan Baron has written a fairly good overview of the field, with his fourth edition of Thinking and Deciding. And I think both Ray Dalio’s and Nicholas Nassem Taleb’s writings have explored the implications of some of these ideas. If I were to summarise the rough thrust of these books:
Don’t do trial and error where error is catastrophic.
Don’t repeat the same trials over and over again (aka don’t repeat the same mistakes over and over again).
Increase the number of trials you can do in your life. Decrease the length and cost of each trial.
In fields with optionality (i.e. your downside is capped but your upside is large) the more trials you take, and the more cheap each trial costs, the more likely you’ll eventually win. Or, as Taleb says: “randomness is good when you have optionality.”
Write down your lessons and approaches from your previous successful trials, so you may generalise them to more situations (Principles, chapter 5)
Systematically identify the factor that gives positive evidence, and vary that to maximise the expected size of the impact (Thinking and Deciding, chapter 7)
Actively look for disconfirming evidence when you’ve found an approach that seems to work. (Thinking and Deciding, chapter 7, Principles, chapter 3).
https://commoncog.com/blog/chinese-businessmen-superstition-doesnt-count/
Don’t do trial and error where error is catastrophic.
Wearing a mask in a pandemic. Not putting ALL of your money on a roulette wheel. Not balancing on a tightrope without a net between two skyscrapers unless you have extensive training. Not posting about controversial things without much upside. Not posting photos of meat you cooked to Instagram if you want to have good acclaim in 200 years when eating meat is outlawed. Not building AI because it's cool. Falling in love with people who don't reciprocate.
The unknown unknown risk that hasn't been considered yet. Not having enough slack dedicated to detecting this.
Don’t repeat the same trials over and over again (aka don’t repeat the same mistakes over and over again).
If you've gone on OkCupid for the past 7 years and still haven't got a date from it, maybe try a different strategy. If messaging potential tenants on a 3rd-party site doesn't work, try texting them. If asking questions on Yahoo Answers doesn't get good answers, try a different site.
Increase the number of trials you can do in your life. Decrease the length and cost of each trial.
Talk to 10x the number of people; message using templates and/or simple one-liners. Invest with Other People's Money if asymmetric upside. Write something for 5 minutes using Most Dangerous Writing App then post to 5 subreddits. Posting ideas on Twitter instead of Facebook, rationality content on LessWrong Shortform instead of longform. Yoda Timers. If running for the purpose of a runner's high mood boost, try running 5 times that day as fast as possible. Optimizing standard processes for speed.
In fields with optionality (i.e. your downside is capped but your upside is large) the more trials you take, and the more cheap each trial costs, the more likely you’ll eventually win. Or, as Taleb says: “randomness is good when you have optionality.”
Posting content to 10x the people 10x faster generally has huge upside (YMMV). Programming open-source something useful and sharing it.
Write down your lessons and approaches from your previous successful trials, so you may generalise them to more situations (Principles, chapter 5)
Roam is good for this, perhaps SuperMemo. Posting things to social media and coming up with examples of the rules is also a good way of learning content. cough
Systematically identify the factor that gives positive evidence, and vary that to maximise the expected size of the impact (Thinking and Deciding, chapter 7
Did messaging or posting to X different places work? Try 2X, 5X, etc. 1 to N after successfully going 0 to 1.
Actively look for disconfirming evidence when you’ve found an approach that seems to work. (Thinking and Deciding, chapter 7, Principles, chapter 3).
Stating assumptions strongly and clearly so they are disconfirmable, then setting a Yoda Timer to seek counter-examples of the generalization.
Any updates on this in the past six months?
Mati, would you be interested in having a friendly and open (anti-)debate on here (as a new post) about the value of open information, both for life extension purposes and else (such as Facebook group moderation)? I really support the idea of lifelogging for various purposes such as life extension but have a strong disagreement with the general stance of universal access to information as more-or-less always being a public good.
Sure thing. What would you recommend for learning management?
(I count that as an answer to my other recent question too.)
Warning: TVTropes links
When should I outsource something I'm bad at vs leveling up at that skill?
How would you instruct a virtual assistant to help you with scheduling your day/week/etc?
Great post! It's like the "what if an alien took control of you" exercise but feels more playful and game-y. I started a Google doc to plan the month of April from Gurgeh's perspective.
See also: Outside.
Why does CHAI exclude people who don't have a near-perfect GPA? This doesn't seem like a good way to maximize the amount of alignment work being done. High GPA won't save the world and in fact selects for obedience to authority and years of status competition, leading to poor mental health to do work in, decreasing the total amount of cognitive resources being thrown at the problem.
(Hypothesis 1: "Yes, this is first-order bad but the second-order effect is we have one institutionally prestigious organization, and we need to say we have selective GPA in order to fit in and retain that prestige." [Translator's Note: "We must work with evil in order to do good." (The evil being colleges and grades and most of the economic system.)])
(Hypothesis 2: "GPA is the most convenient way we found to select for intelligence and conscientiousness, and those are the traits we need the most.")
(Hypothesis 3: "The university just literally requires us to do this or we'll be shut down.")
Won't somebody think of the grad students!
No, there has barely been any testing. I think it's more like 200-1000 cases.