Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-23T02:04:37.688Z · score: 2 (1 votes) · LW · GW

I also might've expected some people to wonder, given their state interpretation, how come I'm not worried about stuff I mentioned in the whitelisting post anymore

I don't read everything that you write, and when I do read things there seems to be some amount of dropout that occurs resulting in me missing certain clauses (not just in long posts by you, even while proofreading the introduction section of a friend's paper draft!) that I don't notice until quizzed in detail -- I suspect this is partially due to me applying lossy compression that preserves my first guess about the gist of a paragraph, and maybe partially due to literal saccades while reading. The solution is repetition and redundancy: for example, I assume that you tried to do that in your quotes after the phrase "Let's go through some of my past comments about this", but only the quote

[R]elative reachability requires solution of several difficult ontological problems which may not have anything close to a simple core, including both a sensible world state representation and a perfect distance metric

implies to me that we're moving away from a state-based way of thinking, and it doesn't directly say anything about AUP.

Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-23T02:01:21.010Z · score: 2 (1 votes) · LW · GW

This isn't a full response, but it seems to me that Vika is largely talking about problems she percieves with impact measures in general, as defined by "measures of how much impact things have on the world", and is thinking of AUP as an element of this class (as would I, had I not read this comment). Reasons to think this include:

  • A perception of your research as primarily being the development of AUP, and of this post as being research for that development and exposition.
  • The introduction of AUP being in a post titled "Towards a New Impact Measure".

If AUP is not in fact about restricting an agent's impact on the world (or, in other words, on the state of the world), then I would describe it as something other than an "impact measure", since that term is primarily used by people using the way of thinking you denounce (and I believe was invented that way: it seems to have morphed from 'side effects', which strongly suggests effects on parts of the world, according to my quick looking-over of the relevant section of Concrete Problems in AI Safety). Perhaps "optimisation regularisation technique" would be better, although I don't presume to understand your way of thinking about it.

Comment by danielfilan on Value Learning is only Asymptotically Safe · 2019-04-16T07:01:43.130Z · score: 4 (2 votes) · LW · GW

I sort of object to titling this post "Value Learning is only Asymptotically Safe" when the actual point you make is that we don't yet have concrete optimality results for value learning other than asymptotic safety.

We should definitely be writing down sets assumptions from which we can derive formal results about the expected behavior of an agent, but is there anything to aim for that is stronger than asymptotic safety?

In the case of value learning, given the generous assumption that "we somehow figured out how to design an agent which understood what constituted observational evidence of humanity’s reflectively-endorsed utility function", it seems like you should be able to get a PAC-type bound, where by time , the agent is only -suboptimal with probability , where is increasing in but decreasing in -- see results on PAC bounds for Bayesian learning, which I haven't actually looked at. This gives you bounds stronger than asymptotic optimality for value leraning. Sadly, if you want your agent to actually behave well in general environments, you probably won't get results better than asymptotic optimality, but if you're happy to restrict yourself to MDPs, you probably can.

Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-11T18:20:34.307Z · score: 9 (4 votes) · LW · GW

Unlike humans, AI systems would be able to cause butterfly effects on purpose, and could channel their impact through butterfly effects if they are not penalized.

Indeed - a point I think is illustrated by the Chaotic Hurricanes test case. I'm probably most excited about methods that would use transparency techniques to determine when a system is deliberately optimising for a part of the world (e.g. the members of the long-term future population) that we don't want it to care about, but this has a major drawback of perhaps requiring multiple philosophical advances into the meaning of reference in cognition and a greater understanding of what optimisation is.

Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-11T18:10:38.046Z · score: 2 (1 votes) · LW · GW

No.

Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-11T18:10:16.742Z · score: 2 (1 votes) · LW · GW

I prefer the phrase 'impact regularisation', but indeed that was a slip of the mind.

Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-10T23:07:26.475Z · score: 4 (2 votes) · LW · GW

I think that under the worldview of this concern, the distribution of reward functions effectively defines a representation that, if too different from the one humans care about, will either mean that no realistic impact is possible in the real world or be ineffective at penalising unwanted negative impacts.

Comment by danielfilan on Best reasons for pessimism about impact of impact measures? · 2019-04-10T20:45:16.851Z · score: 17 (8 votes) · LW · GW

My concern is similar to Wei Dai's: it seems to me that at a fundamental physical level, any plan involving turning on a computer that does important stuff will make pretty big changes to the world's trajectory in phase space. Heat dissipation will cause atmospheric particles to change their location and momentum, future weather patterns will be different, people will do things at different times (e.g. because they're waiting for a computer program to run, or because the computer is designed to change the flow of traffic through a city), meet different people, and have different children. As a result, it seems hard for me to understand how impact measures could work in the real world without a choice of representation very close to the representation humans use to determine the value of different worlds. I suspect that this will need input from humans similar to what value learning approaches might need, and that once it's done one could just do value learning and dispense with the need for impact measures. That being said, this is more of an impression than a belief - I can't quite convince myself that no good method of impact regularisation exists, and some other competent people seem to disagre ewith me.

Comment by danielfilan on DanielFilan's Shortform Feed · 2019-03-25T23:41:11.929Z · score: 7 (4 votes) · LW · GW

I made this post with the intent to write a comment, but the process of writing the comment out made it less persuasive to me. The planning fallacy?

DanielFilan's Shortform Feed

2019-03-25T23:32:38.314Z · score: 19 (5 votes)
Comment by danielfilan on The Amish, and Strategic Norms around Technology · 2019-03-25T20:45:23.927Z · score: 5 (3 votes) · LW · GW

While [at MAPLE], there were limits on technology that meant, after 9pm, you basically had two choices: read, or go to bed. The choices were strongly reinforced by the social and physical environment. And this made it much easier to make choices they endorsed.

Important context for this is that morning chanting started at 4:40 am, so going to sleep at 9 pm was a more endorsable choice than it might appear.

Comment by danielfilan on Privacy · 2019-03-17T01:23:48.202Z · score: 7 (4 votes) · LW · GW

Do you think that thoughts are too incentivised or not incentivised enough on the margin, for the purpose of epistemically sound thinking? If they're too incentivised, have you considered dampening LWs karma system? If they're not incentivised enough, what makes you believe that legalising blackmail will worsen the epistemic quality of thoughts?

Comment by danielfilan on Blackmailers are privateers in the war on hypocrisy · 2019-03-15T20:54:07.042Z · score: 4 (2 votes) · LW · GW

I think this is the OB post Benquo is quoting from, but accidentally forgot to include the link.

Comment by danielfilan on Blackmailers are privateers in the war on hypocrisy · 2019-03-14T19:08:09.060Z · score: 7 (6 votes) · LW · GW

blackmail-in-practice is often about leveraging the norm enforcement of a different community than the target's, exploiting differences in norms between groups

I'm confused about how you would know this - it seems that by nature, most blackmail-in-practice is going to be unobserved by the wider public, leaving only failed blackmail attempts (which I expect to be systematically different than average since they failed) or your own likely-unrepresentative experiences (if you have any at all).

Comment by danielfilan on Understanding information cascades · 2019-03-14T07:19:31.783Z · score: 15 (4 votes) · LW · GW

This paper looks at the dynamics of information flows in social networks using multi-agent reinforcement learning. I haven't read it, but am impressed by the work of the second author. Abstract:

We model the spread of news as a social learning game on a network. Agents can either endorse or oppose a claim made in a piece of news, which itself may be either true or false. Agents base their decision on a private signal and their neighbors' past actions. Given these inputs, agents follow strategies derived via multi-agent deep reinforcement learning and receive utility from acting in accordance with the veracity of claims. Our framework yields strategies with agent utility close to a theoretical, Bayes optimal benchmark, while remaining flexible to model re-specification. Optimized strategies allow agents to correctly identify most false claims, when all agents receive unbiased private signals. However, an adversary's attempt to spread fake news by targeting a subset of agents with a biased private signal can be successful. Even more so when the adversary has information about agents' network position or private signal. When agents are aware of the presence of an adversary they re-optimize their strategies in the training stage and the adversary's attack is less effective. Hence, exposing agents to the possibility of fake news can be an effective way to curtail the spread of fake news in social networks. Our results also highlight that information about the users' private beliefs and their social network structure can be extremely valuable to adversaries and should be well protected.

Comment by danielfilan on Formalising continuous info cascades? [Info-cascade series] · 2019-03-14T07:11:56.897Z · score: 10 (3 votes) · LW · GW

A relevant result is Aumann's agreement theorem, and offshoots where two Bayesians repeating their probability judgements back and forth will converge on a common belief. Although note that that belief isn't always the one they would have in the case that they both knew all their observations - supposing we both privately flip coins, and state our probabilities that we got the same result, we'll spend all day saying 50% without actually learning the answer - nevertheless you shouldn't expect probabilities to badly asymptote in expectation.

This makes me think that you'll want to think about bounded-rational models where people can only recurse 3 times, or something. [ETA: or models where some participants in the discourse are adversarial, as in this paper].

Comment by danielfilan on Formalising continuous info cascades? [Info-cascade series] · 2019-03-14T07:04:05.480Z · score: 4 (2 votes) · LW · GW

I think that the rewrite mentioned was actually made, and the post as stands is right.

(Although in this case it's weird to call it an information cascade - in the situation described in the post, people don't have any reason to think that a +50 karma post is any better than a +10 karma post, so information isn't really cascading, just karma).

Comment by danielfilan on A defense on QI · 2019-03-08T05:40:59.137Z · score: 15 (6 votes) · LW · GW

For what it's worth, I'm familiar with the philosophy surrounding the many worlds interpretation (MWI, and while I can't vouch for all the argumentation in that paper, I think that (a) quantum immortality is not a consequence of the MWI, and (b) this paper offers a valid argument for (a).

On a more meta point, I think the strategy of "hear something I disagree with -> look for a debunking" isn't likely to lead you to truth - if you were wrong, how would this strategy help you find out? You could carefully check both the argument you disagree with and the debunking, seeing which is flawed or finding a valid synthesis of both, but from the tone of your post I imagine you finding something that counts as a 'debunking' and not pursuing the matter further. I think it would be more wise to think carefully about the claims in the article, look for counterarguments, think carefully about those, and come to your own conclusions (where perhaps the 'thinking' involves discussing the issues with a friend, or on LessWrong or a similar forum). If you can't make heads or tails of the issue, but think you can identify experts who can, then one other option would be to defer to expert consensus. Sadly, in this case, I can't find a poll of experts, but looking at the Wikipedia page on Quantum Suicide and Immortality only quotes two experts (Max Tegmark and David Deutsch), neither of whom agree that quantum immortality works. As such, I suspect that belief in quantum immortality is very uncommon among experts, since otherwise I'd expect to see an expert quoted in the Wikipedia article supporting the view that quantum immortality is real.

Comment by danielfilan on Karma-Change Notifications · 2019-03-05T19:50:52.048Z · score: 20 (7 votes) · LW · GW

I don't like the idea that LW will tell me my daily karma change but only if it's good news.

Comment by danielfilan on How much funding and researchers were in AI, and AI Safety, in 2018? · 2019-03-04T05:22:11.976Z · score: 10 (2 votes) · LW · GW

I notice that in the case of AI safety, it's probably possible to just literally count the researchers by hand.

I think this is probably not true for the average LW reader, or even the average person who's kind of interested in AI alignment, since many orgs are sort of opaque about how many people work there and what team people are on. For example my guess is that most people don't know how many interns CHAI takes, or how many new PhD students we get in a given year, and similarly, I'm not even confident that I could name everybody in OpenAI's safety team without someone to catch my errors.

I assume for "broader work on AI" it'd be necessary to either consult some kind of research that already had them counted, since there's just way too much stuff going on.

Seems correct to me.

Comment by danielfilan on How much funding and researchers were in AI, and AI Safety, in 2018? · 2019-03-03T23:52:27.640Z · score: 9 (5 votes) · LW · GW

By my quick mental count, CHAI's Berkeley branch had something like the equivalent of 8 to 11 researchers focussing on AI alignment in 2018. Kind of tricky to count because we had new PhD students coming in in August, as well as some interns over the summer (some of whom stayed on for longer periods).

Comment by danielfilan on LW Update 2019-01-03 – New All-Posts Page, Author hover-previews and new post-item · 2019-03-02T08:27:20.947Z · score: 3 (2 votes) · LW · GW

I get the same thing in firefox.

Comment by danielfilan on If a "Kickstarter for Inadequate Equlibria" was built, do you have a concrete inadequate equilibrium to fix? · 2019-02-22T20:11:45.317Z · score: 12 (6 votes) · LW · GW
  • Paying people to contribute to open-source software projects. Sometimes, I'm using a software package, wish it had a certain feature, but don't care enough to fix it myself or to pay somebody else to fix it. It would be nice if there were a kickstarter-type mechanism to make things like this happen. A similar example would be working to make reciprocity (a dating site many people in my social circle use) better. [In fact, I'm surprised that it doesn't happen more on Kickstarter. But you don't want exactly that platform, since project proposal should probably be on the consumer side.]
  • Getting community trials of various potential nootropics to happen, e.g. Paul Christiano's proposal of replicating a study of the cognitive benefits of creatine among vegetarians.
Comment by danielfilan on How could "Kickstarter for Inadequate Equilibria" be used for evil or turn out to be net-negative? · 2019-02-22T18:44:13.747Z · score: 8 (5 votes) · LW · GW

A couple of thoughts:

  • When the mechanism is new, and especially if it's somewhat complicated, you could imagine many people using it and accidentally committing to things that they didn't quite realise they were signing up for, or otherwise becoming over-committed.
  • In a recent 80,000 hours podcast, Glen Weyl makes the point that some of his proposed mechanisms for changing how voting and public goods provision are designed under a self-interest assumption, which might not hold true for people in the context of e.g. voting, where they're used to considering the good of the nation. As such, if this type of system were naively designed and initially deployed at significant scale, you could imagine weird problems like over-investment in public goods.
Comment by danielfilan on Avoiding Jargon Confusion · 2019-02-18T21:39:30.793Z · score: 4 (3 votes) · LW · GW

Some people try to fight this sort of thing by punishing people whenever they misuse words, and... I dunno man I just don't think that fight is winnable. Or at least, it seems like we should aim to things up so that we have to spend less energy on that fight in the first place.

I agree that it's good to avoid fights by making good behaviour more easy and natural, but man, I feel very depressed by the idea that the rationality community can't coordinate on the norm that one should use words precisely and clearly to preserve useful meanings. Which is to say, I think that the fight could maybe be winnable (at least in certain contexts), but also that if it isn't it's worth spending more time thinking about how we could have failed so badly.

Robin Hanson on Lumpiness of AI Services

2019-02-17T23:08:36.165Z · score: 16 (6 votes)
Comment by danielfilan on Test Cases for Impact Regularisation Methods · 2019-02-07T20:03:10.850Z · score: 4 (2 votes) · LW · GW

This post is extremely well done.

Thanks!

Wouldn’t most measures with a stepwise inaction baseline pass?

I think not, because given stepwise inaction, the supervisor will issue a high-impact task, and the AI system will just ignore it due to being inactive. Therefore, the actual rollout of the supervisor issuing a high-impact task and the system completing it should be high impact relative to that baseline. Or at least that's my current thinking, I've regularly found myself changing my mind about what systems actually do in these test cases.

Test Cases for Impact Regularisation Methods

2019-02-06T21:50:00.760Z · score: 62 (18 votes)
Comment by danielfilan on What is a reasonable outside view for the fate of social movements? · 2019-01-12T21:48:53.888Z · score: 11 (3 votes) · LW · GW

Look at the time-stamp: you're getting a random number from the 26th of December, not a fresh random number.

Does freeze-dried mussel powder have good stuff that vegan diets don't?

2019-01-12T03:39:19.047Z · score: 17 (4 votes)
Comment by danielfilan on In what ways are holidays good? · 2018-12-28T20:32:31.485Z · score: 4 (2 votes) · LW · GW

I thought my response to this mostly deserved its own thread here. Regarding the financial aspect, I can afford to travel, but it impacts my finances enough that I want to be careful to make good travel decisions, whatever that means.

Comment by danielfilan on In what ways are holidays good? · 2018-12-28T20:00:47.308Z · score: 4 (2 votes) · LW · GW

Since it seems relevant to know where I'm coming from: I've been on holidays before, but almost always with my family, which I assume isn't as good as going on a holiday that you largely control the itinerary of with company that you choose. Sometimes, I get the impression that holidays/travel/tourism can have benefits that aren't obvious to me, like learning new things about different cultures or relaxing. If this is true, then trying out going on holidays without aiming for these benefits might be misleading about how good holidays can be.

Comment by danielfilan on In what ways are holidays good? · 2018-12-28T02:53:29.899Z · score: 3 (4 votes) · LW · GW

What's fun about vacations though? Is it just that other places have more fun things than the place I live has and/or I'm now used to my local fun things?

How much money should you be willing to spend on vacations: as much as fun is worth to you.

This might be an even more naive question than those in my post, but how does/should one figure out how much fun is worth to them? In practice I just sort of use my gut intuitions, but I go on holidays rarely enough, and they involve large enough sums of money, that I don't have reliable gut intuitions. Do you just develop those intuitions by spending money on a TV and a holiday and see which one you like more?

Comment by danielfilan on In what ways are holidays good? · 2018-12-28T02:50:13.793Z · score: 4 (3 votes) · LW · GW

I deliberately excluded signalling value, since usually signalling activities have a pretext of usefulness, or are founded on that pretext, and I'd like to understand it more.

Almost everyone has some [travel] memories that they'd love to share.

Why do they want to share travel memories more than other memories?

And people who have travelled a lot are seen as more adventurous.

Adventure does seem like a function of travel that's hard to otherwise satisfy.

Comment by danielfilan on In what ways are holidays good? · 2018-12-28T02:47:05.876Z · score: 4 (3 votes) · LW · GW

What types of learnings? Why are holidays effective at causing them to happen? What should I do on holidays to get more of them?

Comment by danielfilan on 2018 AI Alignment Literature Review and Charity Comparison · 2018-12-28T02:45:53.001Z · score: 9 (5 votes) · LW · GW

On the theme of 'what about my other contributions', here are two with my name on them that I'd point to as similarly important to the one that was included:

In what ways are holidays good?

2018-12-28T00:42:06.849Z · score: 22 (6 votes)
Comment by danielfilan on Bounded rationality abounds in models, not explicitly defined · 2018-12-12T21:34:28.413Z · score: 5 (3 votes) · LW · GW

I guess I like the hierarchical planning-type view that our 'available action sets' can vary in time, and that one of them can be 'try to think of more possible actions'. Of course, not only do you need to specify the hierarchical structure here, you also need to model the dynamics of action discovery, which is a pretty daunting task.

Comment by danielfilan on Bounded rationality abounds in models, not explicitly defined · 2018-12-11T20:25:06.433Z · score: 7 (5 votes) · LW · GW

I think that I'm more optimistic about action set restriction than you are. In particular, I view the available action set as a fact about what actions the human is considering and choosing between, rather than a statement of what things are physically possible for the human to do. In this sense, action set restriction seems to me to be a vital part of the story of human bounded rationality, although clearly not the entire story (since we need to know why the action set is restricted in the way that it is).

Comment by danielfilan on A Checks and Balances Approach to Improving Institutional Decision-Making (Rough Draft) · 2018-12-03T19:56:09.438Z · score: 6 (5 votes) · LW · GW

Two things I don't quite get about your proposal: How does the committee determine if the reasoning was motivated? Why not just have the committee make the decision in the first place?

Comment by danielfilan on Act of Charity · 2018-11-17T22:04:09.140Z · score: 6 (5 votes) · LW · GW

Unless you are married and doing it missionary-style with intent to make babies, it is possible you are violating a sodomy law, or perhaps an obscenity statute.

In the USA, sodomy laws are unconstitutional.

Comment by danielfilan on Rationality Is Not Systematized Winning · 2018-11-14T07:59:38.553Z · score: 10 (3 votes) · LW · GW

Why should I change any of my actions from the societal default?

If you invest in index funds you'll probably be richer than if you invest in other things. [EDIT: well, this is only true modulo tax concerns, but grokking the EMH is still very relevant to investing] That's advice that you can get from other sources, but that I got from the rationality community, that would be useful to me even if I wasn't trying to save the world.

A separate point is that I think contact with the rationality community got me to consider whether 'it made sense to get'/'I really wanted' things out of my life that I hadn't previously considered e.g. that I wanted to be an effective altruist and help save the world. I do think that this sort of counts as 'winning', although it's stretching the definition.

Comment by danielfilan on Future directions for ambitious value learning · 2018-11-13T19:40:24.377Z · score: 5 (3 votes) · LW · GW

One of the most perplexing parts of the impossibility theorem is that we can’t distinguish between fully rational and fully anti-rational behavior, yet we humans seem to do this easily.

Why does it seem to you that humans do this easily? If I saw two people running businesses and was told that one person was optimising for profit and the other was anti-optimising for negative profit, not only would I not anticipate being able to tell which was which, I would be pretty suspicious of the claim that there was any relevant difference between the two.

Comment by danielfilan on Kelly bettors · 2018-11-13T01:31:13.558Z · score: 7 (2 votes) · LW · GW

I actually wrote this post almost exactly two years ago, and have no idea why it just got cross-posted to LessWrong. I mainly like my post because it covers how the Kelly criterion sort-of applies to markets where you have to predict a bunch of things but don't know what you're actually going to learn. It's also a more theoretical take on the subject. [EDIT: oh, also it goes through the proof of equivalence between a market of Kelly bettors and Bayesian updating, which is kind of nice and an interesting parallel to logical induction]

Kelly bettors

2018-11-13T00:40:01.074Z · score: 23 (7 votes)
Comment by danielfilan on When does rationality-as-search have nontrivial implications? · 2018-11-09T22:07:46.198Z · score: 1 (1 votes) · LW · GW

Yes, it's true that the theorem doesn't show that there's anything exciting that's interestingly different from a universal mixture, just that AFAIK we can't disprove that, and the theorem forces us to come up with a non-trivial notion of 'interestingly different' if we want to.

Comment by danielfilan on When does rationality-as-search have nontrivial implications? · 2018-11-09T06:13:32.700Z · score: 3 (2 votes) · LW · GW

Although it's also worth noting that as per Theorem 16 of the above paper, not all universally dominant enumerable semimeasures are versions of the Solomonoff prior, so there's the possibility that the Solomonoff prior only does well by finding a good non-Solomonoff distribution and mimicking that.

Comment by danielfilan on Open Thread November 2018 · 2018-11-09T00:36:26.447Z · score: 6 (4 votes) · LW · GW

[I'm a grad student at CHAI, but I am not officially speaking on behalf of CHAI or making any promises on anybody's behalf]

If you reached out to a grad student at CHAI or one of our staff, I strongly suspect that we would at least screen the idea for sanity checking, and if it passed that test, I predict that we would seriously consider what to do with it and how dangerous it was.

Comment by danielfilan on When does rationality-as-search have nontrivial implications? · 2018-11-08T19:42:58.317Z · score: 4 (3 votes) · LW · GW

I think that this is a slightly wrong account of the case for Solomonoff induction. The claim is not just that Solomonoff induction predicts computable environments better than computable predictors, but rather that the Solomonoff prior is an enumerable semimeasure that is also a mixture over every enumerable semimeasure, and therefore predicts computable environments at least as well as any other enumerable semimeasure. So, using your notation, . It still fails as a theory of embedded agency, since it only predicts computable environments, but it's not true that we must only compare it to prediction strategies strictly weaker than itself. The paper (Non-)Equivalence of Universal Priors has a decent discussion of this.

Comment by danielfilan on What is ambitious value learning? · 2018-11-01T22:26:20.409Z · score: 5 (3 votes) · LW · GW

Also, if you define the state to be the entire history, you lose ergodicity assumptions that are needed to prove that algorithms can learn well.

Comment by danielfilan on Raemon's Shortform Feed · 2018-10-31T05:21:12.526Z · score: 1 (1 votes) · LW · GW

I think authors generally are more rewarded by comments than by upvotes.

Curious if you've done some sort of survey on this. My own feelings are that I care less about the average comment on one of my posts than 10 karma, and I care less about that than I do about a really very good comment (which might intuitively be worth like 30 karma) (but maybe I'm not provoking the right comments?). In general, I don't have an intuitive sense that comments are all that important except for the info value when reading, and I guess the 'people care about me' value as an incentive to write. I do like the idea of the thing I wrote being woven into the way people think, but I don't feel like comments are the best way for that to happen.

Comment by danielfilan on Thoughts on short timelines · 2018-10-29T17:55:43.048Z · score: 4 (2 votes) · LW · GW

See also AI Impact’s discontinuous progress investigation. They actually consider new land speed records set by jet-propelled vehicles one of the few cases of (moderate) discontinuities that they’ve found so far. To me that doesn’t feel analogous in terms of the necessary magnitude of the discontinuity, though.

I'm kind of surprised that the post doesn't mention the other, larger discontinuities that they've found: nuclear weapons, high-temperature superconduction, and building height.

Plus, it has been argued that the next AI winter is well on its way, i.e. we actually start to see a decline, not a further increase, of interest in AI.

Metaculus has the closest thing to a prediction market on this topic that I'm aware of, which is worth looking at.

Unfortunately, interpreting expert opinion is tricky. On the one hand, in some surveys machine learning researchers put non-negligible probability on “human-level intelligence” (whatever that means) in 10 years. On the other hand, my impression from interacting with the community is that the predominant opinion is still to confidently dismiss a short timeline scenario, to the point of not even seriously engaging with it.

The linked survey is the most comprehensive survey that I'm aware of, and it points to the ML community collectively putting ~10% chance on HLAI in 10 years. I think that if I thought that one should defer to expert opinion, I would put a lot of weight on this survey and very little on the interactions that the author of this piece has had. That being said, the survey also (in my view) shows that the ML community is not that great at prediction.


All in all, my main disagreement with this post is about the level of progress that we've seen and are likely to see. It seems like ML has been steadily gaining a bunch of relevant capacities, and that the field has a lot of researchers capable of bringing the field forward both through incremental and fundamental research. The author implicitly thinks that this is nowhere near enough for AGI in 10 years, my broad judgement is that it makes that achievement not unthinkable, but it's hard to fully lay out the relevant reasons for that judgement.

Comment by danielfilan on [Beta] Post-Read-Status on Lessestwrong · 2018-10-27T02:31:52.662Z · score: 3 (2 votes) · LW · GW

Thought: it basically looks like the default thing where links that you've already clicked are a different colour.

Comment by danielfilan on Schools Proliferating Without Practicioners · 2018-10-26T23:28:57.541Z · score: 9 (5 votes) · LW · GW

Would church sermons work better, or worse, as a podcast?

Note that "sermon podcasts" are definitely a thing. See this article on why they're bad, and this article on why and how to do it.

Comment by danielfilan on Schools Proliferating Without Practicioners · 2018-10-26T22:30:38.996Z · score: 9 (2 votes) · LW · GW

I agree that it pays to be precise, which is why I was asking if you believed that statement, rather than asserting that you did. I guess I'd like to hear what proposition you're claiming - is "X" meant to stand in for "atheism/secularism" there? Atheism is almost precise (although I start wondering whether simulation hypotheses technically count, which is why I included the "as depicted in typical religions" bit), but I at least could map "secularism" to a variety of claims, some of which I accept and some of which I reject. I also still don't know what you mean by "unproductive" - if almost everybody I interact with is an atheist, and therefore I don't feel the need to convince them of atheism, does that mean that I believe atheism is unproductive? (Again, this is a question, not me claiming that your answer to the question will be "yes")

Comment by danielfilan on Schools Proliferating Without Practicioners · 2018-10-26T20:42:04.749Z · score: 9 (7 votes) · LW · GW

“Either false or unproductive” is exactly how I’d describe most rationalists’ (and certainly that of most of the ones in visible/influential online spaces) attitude toward atheism/secularism/etc.

This really surprises me. Do you mean to say that if you asked 20 randomly-selected high-karma LW users whether God as depicted in typical religions exist, at least 10 would say "yes"? If so, I strongly disagree, based on my experience hanging out and living with rationalists in the Bay Area, and would love to bet with you. (You might be right about SSC commenters, I'll snobbishly declare them "not real rationalists" by default)

Bottle Caps Aren't Optimisers

2018-08-31T18:30:01.108Z · score: 53 (21 votes)

Mechanistic Transparency for Machine Learning

2018-07-11T00:34:46.846Z · score: 50 (17 votes)

Research internship position at CHAI

2018-01-16T06:25:49.922Z · score: 25 (8 votes)

Insights from 'The Strategy of Conflict'

2018-01-04T05:05:43.091Z · score: 73 (27 votes)

Meetup : Canberra: Guilt

2015-07-27T09:39:18.923Z · score: 1 (2 votes)

Meetup : Canberra: The Efficient Market Hypothesis

2015-07-13T04:01:59.618Z · score: 1 (2 votes)

Meetup : Canberra: More Zendo!

2015-05-27T13:13:50.539Z · score: 1 (2 votes)

Meetup : Canberra: Deep Learning

2015-05-17T21:34:09.597Z · score: 1 (2 votes)

Meetup : Canberra: Putting Induction Into Practice

2015-04-28T14:40:55.876Z · score: 1 (2 votes)

Meetup : Canberra: Intro to Solomonoff induction

2015-04-19T10:58:17.933Z · score: 1 (2 votes)

Meetup : Canberra: A Sequence Post You Disagreed With + Discussion

2015-04-06T10:38:21.824Z · score: 1 (2 votes)

Meetup : Canberra HPMOR Wrap Party!

2015-03-08T22:56:53.578Z · score: 1 (2 votes)

Meetup : Canberra: Technology to help achieve goals

2015-02-17T09:37:41.334Z · score: 1 (2 votes)

Meetup : Canberra Less Wrong Meet Up - Favourite Sequence Post + Discussion

2015-02-05T05:49:29.620Z · score: 1 (2 votes)

Meetup : Canberra: the Hedonic Treadmill

2015-01-15T04:02:44.807Z · score: 1 (2 votes)

Meetup : Canberra: End of year party

2014-12-03T11:49:07.022Z · score: 1 (2 votes)

Meetup : Canberra: Liar's Dice!

2014-11-13T12:36:06.912Z · score: 1 (2 votes)

Meetup : Canberra: Econ 101 and its Discontents

2014-10-29T12:11:42.638Z · score: 1 (2 votes)

Meetup : Canberra: Would I Lie To You?

2014-10-15T13:44:23.453Z · score: 1 (2 votes)

Meetup : Canberra: Contrarianism

2014-10-02T11:53:37.350Z · score: 1 (2 votes)

Meetup : Canberra: More rationalist fun and games!

2014-09-15T01:47:58.425Z · score: 1 (2 votes)

Meetup : Canberra: Akrasia-busters!

2014-08-27T02:47:14.264Z · score: 1 (2 votes)

Meetup : Canberra: Cooking for LessWrongers

2014-08-13T14:12:54.548Z · score: 1 (2 votes)

Meetup : Canberra: Effective Altruism

2014-08-01T03:39:53.433Z · score: 1 (2 votes)

Meetup : Canberra: Intro to Anthropic Reasoning

2014-07-16T13:10:40.109Z · score: 1 (2 votes)

Meetup : Canberra: Paranoid Debating

2014-07-01T09:52:26.939Z · score: 1 (2 votes)

Meetup : Canberra: Many Worlds + Paranoid Debating

2014-06-17T13:44:22.361Z · score: 1 (2 votes)

Meetup : Canberra: Decision Theory

2014-05-26T14:44:31.621Z · score: 1 (2 votes)

[LINK] Scott Aaronson on Integrated Information Theory

2014-05-22T08:40:40.065Z · score: 22 (23 votes)

Meetup : Canberra: Rationalist Fun and Games!

2014-05-01T12:44:58.481Z · score: 0 (3 votes)

Meetup : Canberra: Life Hacks Part 2

2014-04-14T01:11:27.419Z · score: 0 (1 votes)

Meetup : Canberra Meetup: Life hacks part 1

2014-03-31T07:28:32.358Z · score: 0 (1 votes)

Meetup : Canberra: Meta-meetup + meditation

2014-03-07T01:04:58.151Z · score: 3 (4 votes)

Meetup : Second Canberra Meetup - Paranoid Debating

2014-02-19T04:00:42.751Z · score: 1 (2 votes)