AlphaStar: Impressive for RL progress, not for AGI progress 2019-11-02T01:50:27.208Z · score: 99 (46 votes)
orthonormal's Shortform 2019-10-31T05:24:47.692Z · score: 9 (1 votes)
Fuzzy Boundaries, Real Concepts 2018-05-07T03:39:33.033Z · score: 62 (16 votes)
Roleplaying As Yourself 2018-01-06T06:48:03.510Z · score: 86 (32 votes)
The Loudest Alarm Is Probably False 2018-01-02T16:38:05.748Z · score: 179 (71 votes)
Value Learning for Irrational Toy Models 2017-05-15T20:55:05.000Z · score: 0 (0 votes)
HCH as a measure of manipulation 2017-03-11T03:02:53.000Z · score: 1 (1 votes)
Censoring out-of-domain representations 2017-02-01T04:09:51.000Z · score: 2 (2 votes)
Vector-Valued Reinforcement Learning 2016-11-01T00:21:55.000Z · score: 2 (2 votes)
Cooperative Inverse Reinforcement Learning vs. Irrational Human Preferences 2016-06-18T00:55:10.000Z · score: 2 (2 votes)
Proof Length and Logical Counterfactuals Revisited 2016-02-10T18:56:38.000Z · score: 3 (3 votes)
Obstacle to modal optimality when you're being modalized 2015-08-29T20:41:59.000Z · score: 3 (3 votes)
A simple model of the Löbstacle 2015-06-11T16:23:22.000Z · score: 2 (2 votes)
Agent Simulates Predictor using Second-Level Oracles 2015-06-06T22:08:37.000Z · score: 2 (2 votes)
Agents that can predict their Newcomb predictor 2015-05-19T10:17:08.000Z · score: 1 (1 votes)
Modal Bargaining Agents 2015-04-16T22:19:03.000Z · score: 3 (3 votes)
[Clearing out my Drafts folder] Rationality and Decision Theory Curriculum Idea 2015-03-23T22:54:51.241Z · score: 6 (7 votes)
An Introduction to Löb's Theorem in MIRI Research 2015-03-23T22:22:26.908Z · score: 16 (17 votes)
Welcome, new contributors! 2015-03-23T21:53:20.000Z · score: 4 (4 votes)
A toy model of a corrigibility problem 2015-03-22T19:33:02.000Z · score: 4 (4 votes)
New forum for MIRI research: Intelligent Agent Foundations Forum 2015-03-20T00:35:07.071Z · score: 36 (37 votes)
Forum Digest: Updateless Decision Theory 2015-03-20T00:22:06.000Z · score: 5 (5 votes)
Meta- the goals of this forum 2015-03-10T20:16:47.000Z · score: 3 (3 votes)
Proposal: Modeling goal stability in machine learning 2015-03-03T01:31:36.000Z · score: 1 (1 votes)
An Introduction to Löb's Theorem in MIRI Research 2015-01-22T20:35:50.000Z · score: 2 (2 votes)
Robust Cooperation in the Prisoner's Dilemma 2013-06-07T08:30:25.557Z · score: 73 (71 votes)
Compromise: Send Meta Discussions to the Unofficial LessWrong Subreddit 2013-04-23T01:37:31.762Z · score: -2 (18 votes)
Welcome to Less Wrong! (5th thread, March 2013) 2013-04-01T16:19:17.933Z · score: 27 (28 votes)
Robin Hanson's Cryonics Hour 2013-03-29T17:20:23.897Z · score: 29 (34 votes)
Does My Vote Matter? 2012-11-05T01:23:52.009Z · score: 19 (37 votes)
Decision Theories, Part 3.75: Hang On, I Think This Works After All 2012-09-06T16:23:37.670Z · score: 23 (24 votes)
Decision Theories, Part 3.5: Halt, Melt and Catch Fire 2012-08-26T22:40:20.388Z · score: 31 (32 votes)
Posts I'd Like To Write (Includes Poll) 2012-05-26T21:25:31.019Z · score: 14 (15 votes)
Timeless physics breaks T-Rex's mind [LINK] 2012-04-23T19:16:07.064Z · score: 22 (29 votes)
Decision Theories: A Semi-Formal Analysis, Part III 2012-04-14T19:34:38.716Z · score: 23 (28 votes)
Decision Theories: A Semi-Formal Analysis, Part II 2012-04-06T18:59:35.787Z · score: 16 (19 votes)
Decision Theories: A Semi-Formal Analysis, Part I 2012-03-24T16:01:33.295Z · score: 23 (25 votes)
Suggestions for naming a class of decision theories 2012-03-17T17:22:54.160Z · score: 5 (8 votes)
Decision Theories: A Less Wrong Primer 2012-03-13T23:31:51.795Z · score: 72 (76 votes)
Baconmas: The holiday for the sciences 2012-01-05T18:51:10.606Z · score: 5 (5 votes)
Advice Request: Baconmas Website 2012-01-01T19:25:40.308Z · score: 11 (11 votes)
[LINK] "Prediction Audits" for Nate Silver, Dave Weigel 2011-12-30T21:07:50.916Z · score: 12 (13 votes)
Welcome to Less Wrong! (2012) 2011-12-26T22:57:21.157Z · score: 26 (27 votes)
Improving My Writing Style 2011-10-11T16:14:40.907Z · score: 6 (9 votes)
Decision Theory Paradox: Answer Key 2011-09-05T23:13:33.256Z · score: 6 (6 votes)
Consequentialism Need Not Be Nearsighted 2011-09-02T07:37:08.154Z · score: 55 (55 votes)
Decision Theory Paradox: PD with Three Implies Chaos? 2011-08-27T19:22:15.046Z · score: 19 (29 votes)
Why are certain trends so precisely exponential? 2011-08-06T17:38:42.140Z · score: 16 (17 votes)
Nature: Red, in Truth and Qualia 2011-05-29T23:50:28.495Z · score: 44 (38 votes)
A Study of Scarlet: The Conscious Mental Graph 2011-05-27T20:13:26.876Z · score: 29 (34 votes)


Comment by orthonormal on 2018 Review: Voting Results! · 2020-01-26T19:53:02.372Z · score: 2 (1 votes) · LW · GW

Correlation looks good all the way except for the three massive outliers at the top of the rankings.

Comment by orthonormal on Reality-Revealing and Reality-Masking Puzzles · 2020-01-18T20:09:20.852Z · score: 9 (6 votes) · LW · GW

Shorter version:

"How to get people to take ideas seriously without serious risk they will go insane along the way" is a very important problem. In retrospect, CFAR should have had this as an explicit priority from the start.

Comment by orthonormal on What cognitive biases feel like from the inside · 2020-01-12T23:14:01.239Z · score: 2 (1 votes) · LW · GW

Relatedly, there's an awkward cursor line in the top-right box for optimism bias.

Comment by orthonormal on Stripping Away the Protections · 2020-01-12T23:07:19.658Z · score: 4 (2 votes) · LW · GW

I believe the corporations in Moral Mazes were mostly in the manufacturing sector. (Your second point applies, though, as a decent explanation for why American manufacturing has been increasingly outcompeted in the last few decades.)

Comment by orthonormal on Bottle Caps Aren't Optimisers · 2020-01-12T22:57:50.618Z · score: 2 (1 votes) · LW · GW

Okay, so another necessary condition for being downstream from an optimizer is being causally downstream. I'm sure there are other conditions, but the claim still feels like an important addition to the conversation.

Comment by orthonormal on Circling as Cousin to Rationality · 2020-01-04T21:44:02.320Z · score: 16 (7 votes) · LW · GW

I'm still skeptical of Circling, but this is exactly the sort of post I want to encourage in general: trying to explain something many readers are skeptical of, while staying within this site's epistemic standards and going only one inferential step out.

Comment by orthonormal on 2020's Prediction Thread · 2020-01-01T18:26:54.670Z · score: 2 (1 votes) · LW · GW

Either (1) or (2) (and some other possibilities) would satisfy my prediction. My prediction is just that, however we do things in 2029, it won't be by handing each merchant the keys to our entire credit account.

Comment by orthonormal on 2020's Prediction Thread · 2020-01-01T04:08:13.943Z · score: 2 (1 votes) · LW · GW

Dammit, dammit, dammit, I meant to condition these all on no human extinction and no superintelligence. Commenting rather than editing because I forget if the time of an edit is visible, and I want it to be clear I didn't update this based on information from the 2020s.

Comment by orthonormal on 2020's Prediction Thread · 2020-01-01T00:06:05.241Z · score: 2 (1 votes) · LW · GW

Okay then, how about higher education as a fraction of GDP?

Comment by orthonormal on 2020's Prediction Thread · 2019-12-31T21:24:36.387Z · score: 2 (1 votes) · LW · GW

As with last decade, I'm most confident about boring things, though less optimistic than I'd like to be.

Fewer than 1 billion people (combatants + civilians) will die in wars in the 2020s: 95%

The United States of America will still exist under its current Constitution (with or without new Amendments) and with all of its current states (with or without new states) as of 1/1/30: 93%

Fewer than 10 million people (combatants + civilians) will die in wars in the 2020s: 85%

The median rent per unit in the United States will increase faster than inflation in the 2020s: 80%

The National Popular Vote Interstate Compact will not go into force by 1/1/30: 75%

Human-driven cars will still be street-legal in all major US cities as of 1/1/30: 75%

As of 1/1/30, customers will not make purchases by giving each merchant full access to a non-transaction-specific numeric string (i.e. credit cards as they are today): 70%

Conditional on Pew Research Center releasing a survey on the topic after 1/1/28, their most recent survey by 1/1/30 will show that 60% or fewer of American adults identify as Christian: 70%

Conditional on Pew Research Center releasing a survey on the topic after 1/1/28, their most recent survey by 1/1/30 will show that 33% or more of American adults identify as religiously unaffiliated: 70%

More than half of American adults will use a Facebook product at least once per day in 2029: 60%

Real-time 24-hour news networks will still exist as of 1/1/30, and will average more than 1 million average daily viewers (in the USA) in 2029: 50%

The largest not-currently-existing US tech company (by market cap as of 1/1/30) will not have its primary HQ in the Bay Area: 35%

A song generated entirely by an AI will make one of the Billboard charts: 25%

California will put a new state constitution to a statewide vote: 10%

Comment by orthonormal on 2020's Prediction Thread · 2019-12-31T20:25:35.045Z · score: 9 (5 votes) · LW · GW

Re (7), there's a laughable amount of conjunction on even the first prediction in the chain.

Comment by orthonormal on 2020's Prediction Thread · 2019-12-31T20:22:41.438Z · score: 2 (1 votes) · LW · GW

Re: higher education bubble, do you also predict that tuition increases will not outpace inflation?

Comment by orthonormal on 2020's Prediction Thread · 2019-12-31T20:21:17.667Z · score: 3 (2 votes) · LW · GW

The Rubik's Cube one strikes me as much more feasible than the other AI predictions. Look at the dexterity improvements of Boston Dynamics over the last decade and apply that to the current robotic hands, and I think there's a better than 70% chance you get a Rubik's Cube-spinning, chopsticks-using robotic hand by 2030.

Comment by orthonormal on 2010s Predictions Review · 2019-12-31T20:14:46.643Z · score: 2 (1 votes) · LW · GW

De-aging is different from a fully CGI character.

Comment by orthonormal on 2010s Predictions Review · 2019-12-31T20:14:19.585Z · score: 8 (4 votes) · LW · GW

My most testable prediction (that the top movies/books/albums would make up a much lower fraction of overall revenue) was completely wrong across all media.

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T20:10:50.602Z · score: 4 (2 votes) · LW · GW

Thank maths for videoconferencing, enabling working from home (at least occasionally) for every major tech company.

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T20:07:37.493Z · score: 6 (4 votes) · LW · GW

Wolf Warrior 2 did $874 million in China alone; China's rapidly growing domestic market won this prediction singlehandedly.

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:49:28.866Z · score: 7 (3 votes) · LW · GW

3.6 trillion in 2018 (17.7% of GDP), so we don't even need to argue about adjusting for inflation to judge these. Thanks Obamacare!

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:43:29.486Z · score: 14 (4 votes) · LW · GW


1. No.

2. Yes.

3. Hell no.

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:37:14.806Z · score: 5 (3 votes) · LW · GW

So close on Brexit, but just missed the deadline.

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:33:56.934Z · score: 7 (3 votes) · LW · GW

I hope you paid out on your bets.

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:31:56.428Z · score: 3 (2 votes) · LW · GW

Not sure whether it's been fully crossed, but it's close.

By 2015 we had a CGI-on-top-of-body-double Paul Walker and audiences weren't sure when the clips of him were real ones. Rogue One had full-CGI Tarkin and Leia, though those were uncanny for some viewers (and successful for others). Can't think of another fully CGI human example.

(No, non-human humanoids still don't count, as impressive as Thanos was.)

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:22:24.689Z · score: 3 (2 votes) · LW · GW

Well, your criticism was correct.

(Though some other trends have obviously reversed- streaming music ate album and single sales, which were increasing rapidly in the iTunes era of the 2000s.)

Comment by orthonormal on New Year's Predictions Thread · 2019-12-31T19:21:02.147Z · score: 7 (3 votes) · LW · GW

Following up: I was wrong about my most testable prediction. The biggest media hits in the USA are getting proportionally larger, not smaller, though this may be mediated by streaming/ebooks taking away from the traditional outlets.

(If you find more complete sources for any of these, let me know. I restricted to the US because the international market is growing so rapidly it would skew any trends.)

Music: This is obviously confounded by the switch from buying physical albums to streaming music, but in any case, it looks as if I was wrong: the top albums have sold comparable numbers of copies (after averaging out by 5-year increments) since 2005, while the total number of album sales has plummeted. (Maybe people are only buying albums for the most popular artists and massively diversifying their streaming music, but in any case I would have antipredicted the top artist album sales staying constant.)

Books: Total revenue for trade books has stayed remarkably consistent at about $15 billion per year for the past five years; I didn't find first-half-of-decade results as easily. Top books by print copies might be misleading, but they're easy to find retrospectively using Publishers Weekly lists like this one. And they've been increasing since 2014, though 2013 had the massive outlier of the Fifty Shades series (sigh). Another loss for the theory.

Movies: Domestic box office has been growing slowly, and the biggest domestic hits have been growing rapidly. Essentially, Disney is eating the movie theater market with their big franchises.

And more broadly/vaguely, the US social media landscape looks less like a land of ten thousand subcultures and more like a land of fewer than ten megacultures, each fairly defined by their politics and united on their morality and aesthetics.

Comment by orthonormal on We run the Center for Applied Rationality, AMA · 2019-12-20T23:59:38.114Z · score: 15 (12 votes) · LW · GW

I honestly think, in retrospect, that the linchpin of early CFAR's standard of good shared epistemics was probably Critch.

Comment by orthonormal on Your Price for Joining · 2019-12-17T01:58:01.983Z · score: 5 (2 votes) · LW · GW

In practice, one fork gets an advantage, more people switch over to work on it, and that form decisively wins with almost all the original contributors. Just like Bitcoin.

Comment by orthonormal on ialdabaoth is banned · 2019-12-15T08:33:54.746Z · score: 13 (3 votes) · LW · GW

I mistrusted ialdabaoth from the start, though it's worth saying that I judged him to be a dangerous manipulator and probable abuser from in-person interactions long before the accusations came out, so it's not just his LessWrong content.

In any case, I found it impossible to argue on his own terms (not because he'd make decent counterarguments, but because he'd try to corrupt the frame of the conversation instead of making counterarguments). So instead I did things like write this post as a direct rebuttal to something he'd written (maybe on LessWrong, maybe on Facebook) about how honesty and consent were fake concepts used to disguise the power differentials of popularity (which ultimately culminates in an implied "high status people sometimes get away with bad behavior X, so don't condemn me when I do X".)

Comment by orthonormal on The Actionable Version of "Keep Your Identity Small" · 2019-12-15T00:14:55.088Z · score: 14 (6 votes) · LW · GW

I've always preferred the advice to Keep Your Identity Meta. I'm not an X-believer, I'm a person who processes evidence in way Y.

Yes, that's still an identity, and it's still about a belief complex, but it's much more compatible with actually changing one's mind.

(Y = "with intellectual honesty no matter what" helped me to leave my religion in a principled fashion, even before I came across this community.)

Comment by orthonormal on Is Rationalist Self-Improvement Real? · 2019-12-14T23:47:27.627Z · score: 22 (6 votes) · LW · GW

Also, yes rationalists do more curating of good advice than invention of it, just as we do with philosophy. But there's a huge value-add in sorting out the good advice in a domain from the bad advice, and this I think the community does in a more cross-domain way than I see elsewhere.

Comment by orthonormal on Is Rationalist Self-Improvement Real? · 2019-12-14T23:44:01.230Z · score: 13 (4 votes) · LW · GW
Randomized controlled trials have many moving parts, removing any of which makes them worse than useless.

I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.

Comment by orthonormal on Is Rationalist Self-Improvement Real? · 2019-12-14T23:22:15.638Z · score: 28 (7 votes) · LW · GW
Even lots of leading rationalist organizations are led by people who haven't put particular effort into anything you could call rationalist self-help! That's really surprising!

Indeed, I'm surprised to read that, because for the leading Berkeley rationalist organizations (MIRI, CFAR, CEA) I can think of at least one person in the top part of their org chart whom I personally know has done a rationalist self-help push for at least a couple of months before taking said role. (In two of those cases it's the top person.)

Can you say what organizations you're thinking of?

Comment by orthonormal on The Loudest Alarm Is Probably False · 2019-12-12T03:56:16.621Z · score: 13 (7 votes) · LW · GW

The LW team is encouraging authors to review their own posts, so:

In retrospect, I think this post set out to do a small thing, and did it well. This isn't a grand concept or a vast inferential distance, it's just a reframe that I think is valuable for many people to try for themselves.

I still bring up this concept quite a lot when I'm trying to help people through their psychological troubles, and when I'm reflecting on my own.

I don't know whether the post belongs in the Best of 2018, but I'm proud of it.

Comment by orthonormal on Bottle Caps Aren't Optimisers · 2019-12-07T23:09:35.310Z · score: 11 (6 votes) · LW · GW

I'm surprised nobody has yet replied that the two examples are both products of significant optimizers with relevant optimization targets, and that the naive definition seems to work with one modification:

A system is downstream from an optimizer of some objective function to the extent that that objective function attains much higher values than would be attained if the system didn't exist, or were doing some other random thing.

Comment by orthonormal on The Tails Coming Apart As Metaphor For Life · 2019-12-07T23:04:18.893Z · score: 8 (4 votes) · LW · GW

I support this post being included in the Best-of-2018 Review.

It does a good job of starting with a straightforward concept, and explaining it clearly and vividly (a SlateStarScott special). And then it goes on to apply the concept to another phenomenon (ethical philosophy) and make more sense of an oft-observed phenomenon (the moral revulsion to both branches of thought experiments, sometimes by the same individual).

Comment by orthonormal on Meta-Honesty: Firming Up Honesty Around Its Edge-Cases · 2019-12-07T22:53:37.189Z · score: 7 (3 votes) · LW · GW

I don't recommend this post for the Best-of-2018 Review.

It's an exploration of a fascinating idea, but it'skind of messy and unusually difficult to understand (in the later sections). Moreover, the author isn't even sure whether it's a good concept or one that will be abused, and in addition worries about it becoming a popularized/bastardized concept in a wider circle. (Compare what happened to "virtue signaling".)

Comment by orthonormal on Unknown Knowns · 2019-12-07T22:26:16.706Z · score: 4 (2 votes) · LW · GW

I think this explanation is mostly true, but the participants may have been more confident of the relative degree of fishiness than the absolute degree. A single unknown variable affecting all of the studies (in this case, the degree of truly invisible fudging in the field) ought to regress each individual probability away from extremes somewhat.

Comment by orthonormal on Embedded Agents · 2019-12-07T22:19:29.146Z · score: 14 (4 votes) · LW · GW

Insofar as the AI Alignment Forum is part of the Best-of-2018 Review, this post deserves to be included. It's the friendliest explanation to MIRI's research agenda (as of 2018) that currently exists.

Comment by orthonormal on The Costly Coordination Mechanism of Common Knowledge · 2019-12-07T22:16:56.397Z · score: 4 (2 votes) · LW · GW

I support the inclusion of this post in the Best-of-2018 Review.

It's a thorough explicit explanation of a core concept in group epistemology, allowing aspects of social reality to better add up to normality, and so it's extremely relevant to this community.

Comment by orthonormal on A voting theory primer for rationalists · 2019-12-07T22:13:28.075Z · score: 2 (1 votes) · LW · GW

I would weakly support this post's inclusion in the Best-of-2018 Review. It's a solid exposition of an important topic, though not a topic that is core to this community.

Comment by orthonormal on Historical mathematicians exhibit a birth order effect too · 2019-12-07T22:11:44.848Z · score: 2 (1 votes) · LW · GW

I would not include this in the Best-of-2018 Review.

While it's good and well-researched, it's more or less a footnote to the Slate Star Codex post linked above. (I think there's an argument for back-porting old SSC posts to LW with Scott's consent, and if that were done I'd have nominated several of those.)

Comment by orthonormal on Caring less · 2019-12-07T22:06:36.836Z · score: 4 (2 votes) · LW · GW

"Caring less" singles out one thing in such a way as to indicate that you might actually care zero about it or care in the opposite direction from your audience.

Analogously, I embarrassed myself when watching the last World Cup with coworkers; when asked who I was cheering for (since the USA didn't qualify), I responded "I just hope it's not Russia". This did not sit well at all with the Russian coworker who overheard. They wouldn't have been upset if I'd instead expressed a positive interest in whoever was playing Russia at the time.

Comment by orthonormal on Decoupling vs Contextualising Norms · 2019-12-07T22:00:13.717Z · score: 6 (3 votes) · LW · GW

I wish John Nerst could be convinced to crosspost here.

Comment by orthonormal on Local Validity as a Key to Sanity and Civilization · 2019-12-07T21:58:42.734Z · score: 16 (5 votes) · LW · GW

This post should be included in the Best-of-2018 compilation.

This is not only a good post, but one which cuts to the core of what this community is about. This site began not as a discussion of topics X, Y, and Z, but as a discussion of how to be... less wrong than the world around you (even/especially your own ingroup), and the difficulties this entails. Uncompromising honesty and self-skepticism are hard, and even though the best parts are a distillation of other parts of the Sequences, people need to be reminded more often than they need to be instructed.

Comment by orthonormal on AlphaStar: Impressive for RL progress, not for AGI progress · 2019-11-23T09:34:49.136Z · score: 23 (6 votes) · LW · GW

Update: with the very newest version of AlphaStar, Deepmind won a series of showmatches with Serral (the 2018 world champion, who plays Zerg), with 4 wins and 1 loss. The resulting algorithm is impressively polished at early and mid-game economy and battles, enough so to take down top players, but my original assessment of it still looks good to me.

In particular, AlphaStar still had serious problems with building placement; moreover, it still showed signs of not having mastered scouting, reactive defense, and late-game strategy (especially for Zerg), and thus failed to respond adequately to a player capable of going that far down the game tree.

The game it lost was the one it played as Terran. While it played better than it had in the summer matches, it still failed to wall off its bases, and more crucially it only built units that would help crush an early Zerg attack, not units that would beat a later Zerg army. Even when it started losing armies to Serral's powerful late-game units, it continued building the same units until it lost completely. This looks, again, like the AlphaStar Zerg agents never figured out late-game strategies, so the Terran never had to learn to counter them.

AlphaStar played 3 of the 5 games as Protoss, the race it learned most effectively as seen in the summer matches. (I'm pretty sure it was intentional on DeepMind's part to play most of the games as AlphaStar's preferred race..) These games were won with fantastic economic production and impeccable unit control (especially with multiple simultaneous Disruptor attacks, which are incredibly difficult for humans to control perfectly in the heat of the battle), which overcame noticeable flaws: leaving holes between the buildings so that Zerg units could come in and kill workers, and failing to build the right units against Serral's army (and thereby losing one army entirely before barely winning with one final push).

It's hard to learn much from the one game AlphaStar played as Zerg, since there it went for a very polished early attack that narrowly succeeded; it looked to me as if Serral got cocky after seeing the attack coming, and he could have defended easily from that position had he played it safer.

In summary, my claim that DeepMind was throwing in the towel was wrong; they came back with a more polished version that was able to beat the world champion 4 out of 5 times (though 2 of those victories were very narrow, and the loss was not). But the other analyses I made in the post are claims I still stand behind when applied to this version: a major advance for reinforcement learning, but still clearly lacking any real advance in causal reasoning.

Comment by orthonormal on The LessWrong 2018 Review · 2019-11-22T20:53:18.497Z · score: 2 (3 votes) · LW · GW

What do you think about doing this for 2017 and years previous?

Comment by orthonormal on orthonormal's Shortform · 2019-11-17T21:04:18.954Z · score: 4 (3 votes) · LW · GW

Is there a word for problems where, as they get worse, the exactly wrong response becomes more intuitively appealing?

For example, I'm thinking of the following chain (sorry for a political example, this is typically a political phenomenon):

resistance to new construction (using the ability of local boards to block projects)

causes skyrocketing rent

which together mean that the rare properties allowed to be developed get bid up to where they can only become high-end housing

which leads to anger at rich developers for building "luxury housing"

which leads to further resistance to new construction

and so on until you get San Francisco

Comment by orthonormal on Autism And Intelligence: Much More Than You Wanted To Know · 2019-11-14T06:39:20.278Z · score: 13 (8 votes) · LW · GW

The other explanation I've heard bandied about is a polygenic version of sickle-cell anemia (where being heterozygous for the allele protects you from malaria but being homozygous gives you an awful disease).

In this model, there are a bunch of alleles that all push the phenotype in roughly the same direction, and having some of them is good, but past some threshold fitness starts rapidly declining.

(Further speculation: the optimum threshold is higher in the environment of civilization than in the ancestral environment, so these genes are experiencing massive positive selection over the last 10,000 years.)

This isn't a causal explanation, but it differs from the tower-foundation model in claiming that there's not a separate weakness to go looking for, just an optimum that's being surpassed- especially when two people near the optimum have children.

Comment by orthonormal on Experiments and Consent · 2019-11-11T02:49:59.496Z · score: 4 (2 votes) · LW · GW

The NTSB report was released last week, showing that Uber's engineering was doing some things very wrong (with specifics that had not been reported before). Self-driving programs shouldn't go on public roads with that kind of system, even with a driver ready to take over.

Comment by orthonormal on AlphaStar: Impressive for RL progress, not for AGI progress · 2019-11-10T07:29:32.005Z · score: 2 (1 votes) · LW · GW

Exactly. It seems like you need something beyond present imitation learning and deep reinforcement learning to efficiently learn strategies whose individual components don't benefit you, but which have a major effect if assembled perfectly together.

(I mean, don't underestimate gradient descent with huge numbers of trials - the genetic version did evolve a complicated eye in such a way that every step was a fitness improvement; but the final model has a literal blind spot that could have been avoided if it were engineered in another way.)

Comment by orthonormal on Recent updates to (2016-2017) · 2019-11-10T06:30:35.565Z · score: 2 (1 votes) · LW · GW

I just finished Story of Your Life, and I disagree with your attempt to retcon it as a physically possible story; I think the evidence is clear that Chiang intended it to be understood the way that most people understand it, and that he in fact was misunderstanding* the interaction between the variational principle and causality.

[Spoilers ahead.]

Firstly, it is a really plausible misunderstanding. If you don't deeply grok how decoherence works, then the principle of least action can really seem like it requires information to behave in nonsequential ways. (But in fact the Schrodinger equation exhibits causality, and variational principles are just the result of amplitudes nearly cancelling out for all paths whose action is not a local extremum.) Chiang is a gifted amateur in physics, so this misconception is not that unlikely.

Secondly, if your interpretation were his intended one, he could have done any number of things to suggest it! Louise directly tells the reader that this story is being told from the night of her daughter's conception, and that she now sees both future and past but does not act to change it. This is shown in the story to be the point of Heptapod B, and Chiang works at length to try and justify how understanding this language lets one understand (but not alter) the future. There's nothing else in the story to hint that Louise is an unreliable narrator.

If your interpretation were Chiang's, he would have to be intentionally misdirecting the audience to a degree you only see from authors like Nabokov, and not leaving any clue except for flawed science, which is common enough for dramatic license in science fiction that it really can't count as a clue. I doubt Chiang is doing that.

And finally, the whole thematic import of the story is ruined by your interpretation; the poignance comes from Louise's existentialist fatalism, giving life to the lines of the tragic play she is cast in, which is only meaningful insofar as she's telling the truth about perceiving forwards and backwards. (It's a variant on the Trafalmadorians, though interestingly Chiang hadn't yet read Slaughterhouse Five when he wrote this.) It's just a more boring story the way you see it!

*Possibly intentionally misunderstanding; I can imagine a science fiction author pestering their physics friend with "I've got a great story idea using this concept", the friend saying "It doesn't work that way", and the author asking "But if I pretend it does work that way, exactly how much physics knowledge would it take someone to be annoyed by it?"