Posts
Comments
Correlation looks good all the way except for the three massive outliers at the top of the rankings.
Shorter version:
"How to get people to take ideas seriously without serious risk they will go insane along the way" is a very important problem. In retrospect, CFAR should have had this as an explicit priority from the start.
Relatedly, there's an awkward cursor line in the top-right box for optimism bias.
I believe the corporations in Moral Mazes were mostly in the manufacturing sector. (Your second point applies, though, as a decent explanation for why American manufacturing has been increasingly outcompeted in the last few decades.)
Okay, so another necessary condition for being downstream from an optimizer is being causally downstream. I'm sure there are other conditions, but the claim still feels like an important addition to the conversation.
I'm still skeptical of Circling, but this is exactly the sort of post I want to encourage in general: trying to explain something many readers are skeptical of, while staying within this site's epistemic standards and going only one inferential step out.
Either (1) or (2) (and some other possibilities) would satisfy my prediction. My prediction is just that, however we do things in 2029, it won't be by handing each merchant the keys to our entire credit account.
Dammit, dammit, dammit, I meant to condition these all on no human extinction and no superintelligence. Commenting rather than editing because I forget if the time of an edit is visible, and I want it to be clear I didn't update this based on information from the 2020s.
Okay then, how about higher education as a fraction of GDP?
As with last decade, I'm most confident about boring things, though less optimistic than I'd like to be.
Fewer than 1 billion people (combatants + civilians) will die in wars in the 2020s: 95%
The United States of America will still exist under its current Constitution (with or without new Amendments) and with all of its current states (with or without new states) as of 1/1/30: 93%
Fewer than 10 million people (combatants + civilians) will die in wars in the 2020s: 85%
The median rent per unit in the United States will increase faster than inflation in the 2020s: 80%
The National Popular Vote Interstate Compact will not go into force by 1/1/30: 75%
Human-driven cars will still be street-legal in all major US cities as of 1/1/30: 75%
As of 1/1/30, customers will not make purchases by giving each merchant full access to a non-transaction-specific numeric string (i.e. credit cards as they are today): 70%
Conditional on Pew Research Center releasing a survey on the topic after 1/1/28, their most recent survey by 1/1/30 will show that 60% or fewer of American adults identify as Christian: 70%
Conditional on Pew Research Center releasing a survey on the topic after 1/1/28, their most recent survey by 1/1/30 will show that 33% or more of American adults identify as religiously unaffiliated: 70%
More than half of American adults will use a Facebook product at least once per day in 2029: 60%
Real-time 24-hour news networks will still exist as of 1/1/30, and will average more than 1 million average daily viewers (in the USA) in 2029: 50%
The largest not-currently-existing US tech company (by market cap as of 1/1/30) will not have its primary HQ in the Bay Area: 35%
A song generated entirely by an AI will make one of the Billboard charts: 25%
California will put a new state constitution to a statewide vote: 10%
Re (7), there's a laughable amount of conjunction on even the first prediction in the chain.
Re: higher education bubble, do you also predict that tuition increases will not outpace inflation?
The Rubik's Cube one strikes me as much more feasible than the other AI predictions. Look at the dexterity improvements of Boston Dynamics over the last decade and apply that to the current robotic hands, and I think there's a better than 70% chance you get a Rubik's Cube-spinning, chopsticks-using robotic hand by 2030.
De-aging is different from a fully CGI character.
My most testable prediction (that the top movies/books/albums would make up a much lower fraction of overall revenue) was completely wrong across all media.
Thank maths for videoconferencing, enabling working from home (at least occasionally) for every major tech company.
Wolf Warrior 2 did $874 million in China alone; China's rapidly growing domestic market won this prediction singlehandedly.
3.6 trillion in 2018 (17.7% of GDP), so we don't even need to argue about adjusting for inflation to judge these. Thanks Obamacare!
Outcomes:
1. No.
2. Yes.
3. Hell no.
So close on Brexit, but just missed the deadline.
I hope you paid out on your bets.
Not sure whether it's been fully crossed, but it's close.
By 2015 we had a CGI-on-top-of-body-double Paul Walker and audiences weren't sure when the clips of him were real ones. Rogue One had full-CGI Tarkin and Leia, though those were uncanny for some viewers (and successful for others). Can't think of another fully CGI human example.
(No, non-human humanoids still don't count, as impressive as Thanos was.)
Well, your criticism was correct.
(Though some other trends have obviously reversed- streaming music ate album and single sales, which were increasing rapidly in the iTunes era of the 2000s.)
Following up: I was wrong about my most testable prediction. The biggest media hits in the USA are getting proportionally larger, not smaller, though this may be mediated by streaming/ebooks taking away from the traditional outlets.
(If you find more complete sources for any of these, let me know. I restricted to the US because the international market is growing so rapidly it would skew any trends.)
Music: This is obviously confounded by the switch from buying physical albums to streaming music, but in any case, it looks as if I was wrong: the top albums have sold comparable numbers of copies (after averaging out by 5-year increments) since 2005, while the total number of album sales has plummeted. (Maybe people are only buying albums for the most popular artists and massively diversifying their streaming music, but in any case I would have antipredicted the top artist album sales staying constant.)
Books: Total revenue for trade books has stayed remarkably consistent at about $15 billion per year for the past five years; I didn't find first-half-of-decade results as easily. Top books by print copies might be misleading, but they're easy to find retrospectively using Publishers Weekly lists like this one. And they've been increasing since 2014, though 2013 had the massive outlier of the Fifty Shades series (sigh). Another loss for the theory.
Movies: Domestic box office has been growing slowly, and the biggest domestic hits have been growing rapidly. Essentially, Disney is eating the movie theater market with their big franchises.
And more broadly/vaguely, the US social media landscape looks less like a land of ten thousand subcultures and more like a land of fewer than ten megacultures, each fairly defined by their politics and united on their morality and aesthetics.
I honestly think, in retrospect, that the linchpin of early CFAR's standard of good shared epistemics was probably Critch.
In practice, one fork gets an advantage, more people switch over to work on it, and that form decisively wins with almost all the original contributors. Just like Bitcoin.
I mistrusted ialdabaoth from the start, though it's worth saying that I judged him to be a dangerous manipulator and probable abuser from in-person interactions long before the accusations came out, so it's not just his LessWrong content.
In any case, I found it impossible to argue on his own terms (not because he'd make decent counterarguments, but because he'd try to corrupt the frame of the conversation instead of making counterarguments). So instead I did things like write this post as a direct rebuttal to something he'd written (maybe on LessWrong, maybe on Facebook) about how honesty and consent were fake concepts used to disguise the power differentials of popularity (which ultimately culminates in an implied "high status people sometimes get away with bad behavior X, so don't condemn me when I do X".)
I've always preferred the advice to Keep Your Identity Meta. I'm not an X-believer, I'm a person who processes evidence in way Y.
Yes, that's still an identity, and it's still about a belief complex, but it's much more compatible with actually changing one's mind.
(Y = "with intellectual honesty no matter what" helped me to leave my religion in a principled fashion, even before I came across this community.)
Also, yes rationalists do more curating of good advice than invention of it, just as we do with philosophy. But there's a huge value-add in sorting out the good advice in a domain from the bad advice, and this I think the community does in a more cross-domain way than I see elsewhere.
Randomized controlled trials have many moving parts, removing any of which makes them worse than useless.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
Even lots of leading rationalist organizations are led by people who haven't put particular effort into anything you could call rationalist self-help! That's really surprising!
Indeed, I'm surprised to read that, because for the leading Berkeley rationalist organizations (MIRI, CFAR, CEA) I can think of at least one person in the top part of their org chart whom I personally know has done a rationalist self-help push for at least a couple of months before taking said role. (In two of those cases it's the top person.)
Can you say what organizations you're thinking of?
The LW team is encouraging authors to review their own posts, so:
In retrospect, I think this post set out to do a small thing, and did it well. This isn't a grand concept or a vast inferential distance, it's just a reframe that I think is valuable for many people to try for themselves.
I still bring up this concept quite a lot when I'm trying to help people through their psychological troubles, and when I'm reflecting on my own.
I don't know whether the post belongs in the Best of 2018, but I'm proud of it.
I'm surprised nobody has yet replied that the two examples are both products of significant optimizers with relevant optimization targets, and that the naive definition seems to work with one modification:
A system is downstream from an optimizer of some objective function to the extent that that objective function attains much higher values than would be attained if the system didn't exist, or were doing some other random thing.
I support this post being included in the Best-of-2018 Review.
It does a good job of starting with a straightforward concept, and explaining it clearly and vividly (a SlateStarScott special). And then it goes on to apply the concept to another phenomenon (ethical philosophy) and make more sense of an oft-observed phenomenon (the moral revulsion to both branches of thought experiments, sometimes by the same individual).
I don't recommend this post for the Best-of-2018 Review.
It's an exploration of a fascinating idea, but it'skind of messy and unusually difficult to understand (in the later sections). Moreover, the author isn't even sure whether it's a good concept or one that will be abused, and in addition worries about it becoming a popularized/bastardized concept in a wider circle. (Compare what happened to "virtue signaling".)
I think this explanation is mostly true, but the participants may have been more confident of the relative degree of fishiness than the absolute degree. A single unknown variable affecting all of the studies (in this case, the degree of truly invisible fudging in the field) ought to regress each individual probability away from extremes somewhat.
Insofar as the AI Alignment Forum is part of the Best-of-2018 Review, this post deserves to be included. It's the friendliest explanation to MIRI's research agenda (as of 2018) that currently exists.
I support the inclusion of this post in the Best-of-2018 Review.
It's a thorough explicit explanation of a core concept in group epistemology, allowing aspects of social reality to better add up to normality, and so it's extremely relevant to this community.
I would weakly support this post's inclusion in the Best-of-2018 Review. It's a solid exposition of an important topic, though not a topic that is core to this community.
I would not include this in the Best-of-2018 Review.
While it's good and well-researched, it's more or less a footnote to the Slate Star Codex post linked above. (I think there's an argument for back-porting old SSC posts to LW with Scott's consent, and if that were done I'd have nominated several of those.)
"Caring less" singles out one thing in such a way as to indicate that you might actually care zero about it or care in the opposite direction from your audience.
Analogously, I embarrassed myself when watching the last World Cup with coworkers; when asked who I was cheering for (since the USA didn't qualify), I responded "I just hope it's not Russia". This did not sit well at all with the Russian coworker who overheard. They wouldn't have been upset if I'd instead expressed a positive interest in whoever was playing Russia at the time.
I wish John Nerst could be convinced to crosspost here.
This post should be included in the Best-of-2018 compilation.
This is not only a good post, but one which cuts to the core of what this community is about. This site began not as a discussion of topics X, Y, and Z, but as a discussion of how to be... less wrong than the world around you (even/especially your own ingroup), and the difficulties this entails. Uncompromising honesty and self-skepticism are hard, and even though the best parts are a distillation of other parts of the Sequences, people need to be reminded more often than they need to be instructed.
Update: with the very newest version of AlphaStar, Deepmind won a series of showmatches with Serral (the 2018 world champion, who plays Zerg), with 4 wins and 1 loss. The resulting algorithm is impressively polished at early and mid-game economy and battles, enough so to take down top players, but my original assessment of it still looks good to me.
In particular, AlphaStar still had serious problems with building placement; moreover, it still showed signs of not having mastered scouting, reactive defense, and late-game strategy (especially for Zerg), and thus failed to respond adequately to a player capable of going that far down the game tree.
The game it lost was the one it played as Terran. While it played better than it had in the summer matches, it still failed to wall off its bases, and more crucially it only built units that would help crush an early Zerg attack, not units that would beat a later Zerg army. Even when it started losing armies to Serral's powerful late-game units, it continued building the same units until it lost completely. This looks, again, like the AlphaStar Zerg agents never figured out late-game strategies, so the Terran never had to learn to counter them.
AlphaStar played 3 of the 5 games as Protoss, the race it learned most effectively as seen in the summer matches. (I'm pretty sure it was intentional on DeepMind's part to play most of the games as AlphaStar's preferred race..) These games were won with fantastic economic production and impeccable unit control (especially with multiple simultaneous Disruptor attacks, which are incredibly difficult for humans to control perfectly in the heat of the battle), which overcame noticeable flaws: leaving holes between the buildings so that Zerg units could come in and kill workers, and failing to build the right units against Serral's army (and thereby losing one army entirely before barely winning with one final push).
It's hard to learn much from the one game AlphaStar played as Zerg, since there it went for a very polished early attack that narrowly succeeded; it looked to me as if Serral got cocky after seeing the attack coming, and he could have defended easily from that position had he played it safer.
In summary, my claim that DeepMind was throwing in the towel was wrong; they came back with a more polished version that was able to beat the world champion 4 out of 5 times (though 2 of those victories were very narrow, and the loss was not). But the other analyses I made in the post are claims I still stand behind when applied to this version: a major advance for reinforcement learning, but still clearly lacking any real advance in causal reasoning.
What do you think about doing this for 2017 and years previous?
Is there a word for problems where, as they get worse, the exactly wrong response becomes more intuitively appealing?
For example, I'm thinking of the following chain (sorry for a political example, this is typically a political phenomenon):
resistance to new construction (using the ability of local boards to block projects)
causes skyrocketing rent
which together mean that the rare properties allowed to be developed get bid up to where they can only become high-end housing
which leads to anger at rich developers for building "luxury housing"
which leads to further resistance to new construction
and so on until you get San Francisco
The other explanation I've heard bandied about is a polygenic version of sickle-cell anemia (where being heterozygous for the allele protects you from malaria but being homozygous gives you an awful disease).
In this model, there are a bunch of alleles that all push the phenotype in roughly the same direction, and having some of them is good, but past some threshold fitness starts rapidly declining.
(Further speculation: the optimum threshold is higher in the environment of civilization than in the ancestral environment, so these genes are experiencing massive positive selection over the last 10,000 years.)
This isn't a causal explanation, but it differs from the tower-foundation model in claiming that there's not a separate weakness to go looking for, just an optimum that's being surpassed- especially when two people near the optimum have children.
The NTSB report was released last week, showing that Uber's engineering was doing some things very wrong (with specifics that had not been reported before). Self-driving programs shouldn't go on public roads with that kind of system, even with a driver ready to take over.
Exactly. It seems like you need something beyond present imitation learning and deep reinforcement learning to efficiently learn strategies whose individual components don't benefit you, but which have a major effect if assembled perfectly together.
(I mean, don't underestimate gradient descent with huge numbers of trials - the genetic version did evolve a complicated eye in such a way that every step was a fitness improvement; but the final model has a literal blind spot that could have been avoided if it were engineered in another way.)
I just finished Story of Your Life, and I disagree with your attempt to retcon it as a physically possible story; I think the evidence is clear that Chiang intended it to be understood the way that most people understand it, and that he in fact was misunderstanding* the interaction between the variational principle and causality.
[Spoilers ahead.]
Firstly, it is a really plausible misunderstanding. If you don't deeply grok how decoherence works, then the principle of least action can really seem like it requires information to behave in nonsequential ways. (But in fact the Schrodinger equation exhibits causality, and variational principles are just the result of amplitudes nearly cancelling out for all paths whose action is not a local extremum.) Chiang is a gifted amateur in physics, so this misconception is not that unlikely.
Secondly, if your interpretation were his intended one, he could have done any number of things to suggest it! Louise directly tells the reader that this story is being told from the night of her daughter's conception, and that she now sees both future and past but does not act to change it. This is shown in the story to be the point of Heptapod B, and Chiang works at length to try and justify how understanding this language lets one understand (but not alter) the future. There's nothing else in the story to hint that Louise is an unreliable narrator.
If your interpretation were Chiang's, he would have to be intentionally misdirecting the audience to a degree you only see from authors like Nabokov, and not leaving any clue except for flawed science, which is common enough for dramatic license in science fiction that it really can't count as a clue. I doubt Chiang is doing that.
And finally, the whole thematic import of the story is ruined by your interpretation; the poignance comes from Louise's existentialist fatalism, giving life to the lines of the tragic play she is cast in, which is only meaningful insofar as she's telling the truth about perceiving forwards and backwards. (It's a variant on the Trafalmadorians, though interestingly Chiang hadn't yet read Slaughterhouse Five when he wrote this.) It's just a more boring story the way you see it!
*Possibly intentionally misunderstanding; I can imagine a science fiction author pestering their physics friend with "I've got a great story idea using this concept", the friend saying "It doesn't work that way", and the author asking "But if I pretend it does work that way, exactly how much physics knowledge would it take someone to be annoyed by it?"