Posts

AXRP Episode 10 - AI’s Future and Impacts with Katja Grace 2021-07-23T22:10:14.624Z
Handicapping competitive games 2021-07-22T03:00:00.498Z
CGP Grey on the difficulty of knowing what's true [audio] 2021-07-13T20:40:13.506Z
A second example of conditional orthogonality in finite factored sets 2021-07-07T01:40:01.504Z
A simple example of conditional orthogonality in finite factored sets 2021-07-06T00:36:40.264Z
AXRP Episode 9 - Finite Factored Sets with Scott Garrabrant 2021-06-24T22:10:12.645Z
Up-to-date advice about what to do upon getting COVID? 2021-06-19T02:37:10.940Z
AXRP Episode 8 - Assistance Games with Dylan Hadfield-Menell 2021-06-08T23:20:11.985Z
AXRP Episode 7.5 - Forecasting Transformative AI from Biological Anchors with Ajeya Cotra 2021-05-28T00:20:10.801Z
AXRP Episode 7 - Side Effects with Victoria Krakovna 2021-05-14T03:50:11.757Z
Challenge: know everything that the best go bot knows about go 2021-05-11T05:10:01.163Z
AXRP Episode 6 - Debate and Imitative Generalization with Beth Barnes 2021-04-08T21:20:12.891Z
AXRP Episode 5 - Infra-Bayesianism with Vanessa Kosoy 2021-03-10T04:30:10.304Z
Privacy vs proof of character 2021-02-28T02:03:31.009Z
AXRP Episode 4 - Risks from Learned Optimization with Evan Hubinger 2021-02-18T00:03:17.572Z
AXRP Episode 3 - Negotiable Reinforcement Learning with Andrew Critch 2020-12-29T20:45:23.435Z
AXRP Episode 2 - Learning Human Biases with Rohin Shah 2020-12-29T20:43:28.190Z
AXRP Episode 1 - Adversarial Policies with Adam Gleave 2020-12-29T20:41:51.578Z
Cognitive mistakes I've made about COVID-19 2020-12-27T00:50:05.212Z
Announcing AXRP, the AI X-risk Research Podcast 2020-12-23T20:00:00.841Z
Security Mindset and Takeoff Speeds 2020-10-27T03:20:02.014Z
Robin Hanson on whether governments can squash COVID-19 2020-03-19T18:23:57.574Z
Should we all be more hygenic in normal times? 2020-03-17T06:14:23.093Z
Did any US politician react appropriately to COVID-19 early on? 2020-03-17T06:12:31.523Z
An Analytic Perspective on AI Alignment 2020-03-01T04:10:02.546Z
How has the cost of clothing insulation changed since 1970 in the USA? 2020-01-12T23:31:56.430Z
Do you get value out of contentless comments? 2019-11-21T21:57:36.359Z
What empirical work has been done that bears on the 'freebit picture' of free will? 2019-10-04T23:11:27.328Z
A Personal Rationality Wishlist 2019-08-27T03:40:00.669Z
Verification and Transparency 2019-08-08T01:50:00.935Z
DanielFilan's Shortform Feed 2019-03-25T23:32:38.314Z
Robin Hanson on Lumpiness of AI Services 2019-02-17T23:08:36.165Z
Test Cases for Impact Regularisation Methods 2019-02-06T21:50:00.760Z
Does freeze-dried mussel powder have good stuff that vegan diets don't? 2019-01-12T03:39:19.047Z
In what ways are holidays good? 2018-12-28T00:42:06.849Z
Kelly bettors 2018-11-13T00:40:01.074Z
Bottle Caps Aren't Optimisers 2018-08-31T18:30:01.108Z
Mechanistic Transparency for Machine Learning 2018-07-11T00:34:46.846Z
Research internship position at CHAI 2018-01-16T06:25:49.922Z
Insights from 'The Strategy of Conflict' 2018-01-04T05:05:43.091Z
Meetup : Canberra: Guilt 2015-07-27T09:39:18.923Z
Meetup : Canberra: The Efficient Market Hypothesis 2015-07-13T04:01:59.618Z
Meetup : Canberra: More Zendo! 2015-05-27T13:13:50.539Z
Meetup : Canberra: Deep Learning 2015-05-17T21:34:09.597Z
Meetup : Canberra: Putting Induction Into Practice 2015-04-28T14:40:55.876Z
Meetup : Canberra: Intro to Solomonoff induction 2015-04-19T10:58:17.933Z
Meetup : Canberra: A Sequence Post You Disagreed With + Discussion 2015-04-06T10:38:21.824Z
Meetup : Canberra HPMOR Wrap Party! 2015-03-08T22:56:53.578Z
Meetup : Canberra: Technology to help achieve goals 2015-02-17T09:37:41.334Z
Meetup : Canberra Less Wrong Meet Up - Favourite Sequence Post + Discussion 2015-02-05T05:49:29.620Z

Comments

Comment by DanielFilan on Delta Strain: Fact Dump and Some Policy Takeaways · 2021-07-28T04:25:24.934Z · LW · GW

if interval is 4 instead of 5.5 days, this would mean that reported R of 7 would turn into R of 9^(4 / 5.5) = 4.

I think this 9 should be a 7?

Comment by DanielFilan on Black ravens and red herrings · 2021-07-27T20:24:21.216Z · LW · GW

Note that if you are a Solomonoff inductor, seeing a black raven doesn't always increase your credence that all ravens are black: see this paper.

Comment by DanielFilan on Handicapping competitive games · 2021-07-22T07:24:38.038Z · LW · GW

Imo this is better modelled as splitting players into a team in the taxonomy of this post, giving the weaker side a computational advantage. But it points to an awkwardness in the formalism.

Comment by DanielFilan on Handicapping competitive games · 2021-07-22T06:50:11.085Z · LW · GW

This argument would be more compelling to me if komi weren't already used - given that you already have to factor that number in, it doesn't seem like such a big deal to use a different number instead.

Comment by DanielFilan on CGP Grey on the difficulty of knowing what's true [audio] · 2021-07-13T22:25:04.064Z · LW · GW

post now has correct timestamps for the non-ad version for what I mean.

Comment by DanielFilan on CGP Grey on the difficulty of knowing what's true [audio] · 2021-07-13T22:22:13.235Z · LW · GW

oops, forgot that I didn't have to deal with ads

Comment by DanielFilan on Finite Factored Sets: Orthogonality and Time · 2021-07-07T23:40:55.820Z · LW · GW

OK I think this is a typo, from the proof of prop 10 where you deal with condition 5:

Thus .

I think this should be .

Comment by DanielFilan on Finite Factored Sets: Orthogonality and Time · 2021-07-07T23:09:02.249Z · LW · GW

From def 16:

... if for all

Should I take this to mean "if for all and "?

[EDIT: no, I shouldn't, since and are both subsets of ]

Comment by DanielFilan on A simple example of conditional orthogonality in finite factored sets · 2021-07-06T19:38:37.511Z · LW · GW

Seems right. I still think it's funky that X_1 and X_2 are conditionally non-orthogonal even when the range of the variables is unbounded.

Comment by DanielFilan on AXRP Episode 9 - Finite Factored Sets with Scott Garrabrant · 2021-07-05T22:53:00.146Z · LW · GW

I'm glad to hear that the podcast is useful for people :)

Comment by DanielFilan on rohinmshah's Shortform · 2021-06-29T07:53:21.418Z · LW · GW

My best guess is that rationalists aren't that sane, especially when they've been locked up for a while and are scared and socially rewarding others being scared.

Comment by DanielFilan on rohinmshah's Shortform · 2021-06-29T07:51:27.605Z · LW · GW

TBH I think what made the uCOVID tax work was that once you did some math, it was super hard to justify levels that would imply anything like the existing risk-avoidance behaviour. So the "active ingredient" was probably just getting people to put numbers on the cost-benefit analysis.

[context note: I proposed the EH uCOVID tax]

Comment by DanielFilan on rohinmshah's Shortform · 2021-06-29T07:48:21.407Z · LW · GW

I feel like Noah's argument implies that states won't incur any costs to reduce CO2 emissions, which is wrong. IMO, the argument for a Pigouvian tax in this context is that for a given amount of CO2 reduction that you want, the tax is a cheaper way of getting it than e.g. regulating which technologies people can or can't use.

Comment by DanielFilan on rohinmshah's Shortform · 2021-06-29T07:44:14.017Z · LW · GW

Another way costs are nonlinear in uCOVIDs is if you think you'll probably get COVID.

Comment by DanielFilan on Knowledge is not just mutual information · 2021-06-10T18:26:36.929Z · LW · GW

Seems like maybe the solution should perhaps be that you should only take 'the system' to be the 'controllable' physical variables, or those variables that are relevant for 'consequential' behaviour? Hopefully if one can provide good definitions for these, it will provide a foundation for saying what the abstractions should be that let us distinguish between 'high-level' and 'low-level' behaviour.

Comment by DanielFilan on Survey on AI existential risk scenarios · 2021-06-08T21:31:38.314Z · LW · GW

As a respondent, I remember being unsure whether I should include those catastrophes.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-06-03T18:25:56.446Z · LW · GW

Ah, understood. I think this is basically covered by talking about what the go bot knows at various points in time, a la this comment - it seems pretty sensible to me to talk about knowledge as a property of the actual computation rather than the algorithm as a whole. But from your response there it seems that you think that this sense isn't really well-defined.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-06-03T18:09:05.196Z · LW · GW

This is correct, altho I'm specifically interested in the case of go AI because I think it's important to understand neural networks that 'plan', as well as those that merely 'perceive' (the latter being the main focus of most interpretability work, with some notable exceptions).

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-06-03T18:06:36.858Z · LW · GW

OP is a fine way to refer to me, I was just confused since I didn't think my post indicated that my desire was to efficiently program a go bot.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-06-03T18:04:10.916Z · LW · GW

I guess by 'learner' you mean the human, rather than the learned model? If so, then I guess your transparency/explanation/knowledge-extraction method could be learner-specific, and still succeed at the above challenge.

Comment by DanielFilan on Feed the spinoff heuristic! · 2021-06-02T00:55:05.875Z · LW · GW

This is no longer true.

Comment by DanielFilan on Curated conversations with brilliant rationalists · 2021-06-01T16:36:20.523Z · LW · GW

FWIW I find it taking more than 1x for native speakers, but I think never longer than 2.5x for anybody.

Comment by DanielFilan on AXRP Episode 7 - Side Effects with Victoria Krakovna · 2021-05-14T19:18:09.095Z · LW · GW

And also thanks for your kind words :)

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T19:07:29.904Z · LW · GW

I suppose this gets back to OP's desire to program a Go Bot in the most efficient manner possible.

If by "OP" you mean me, that's not really my desire (altho that would be nice).

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T19:06:32.630Z · LW · GW

Actually, hmm. My thoughts are not really in equilibrium here.

Comment by DanielFilan on AXRP Episode 7 - Side Effects with Victoria Krakovna · 2021-05-14T19:05:39.938Z · LW · GW

Not sure what the actual sentence you wanted to write was. "are not absolutely necessary" maybe?

You're quite right, let me fix that.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:33:31.621Z · LW · GW

(Also: such a rewrite would be a combination of 'what I really meant' and 'what the comments made me realize I should have really meant')

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:31:55.069Z · LW · GW

OK, the parenthetical helped me understand where you're coming from. I think a re-write of this post should (in part) make clear that I think a massive heroic effort would be necessary to make this happen, but sometimes massive heroic efforts work, and I have no special private info that makes it seem more plausible than it looks a priori.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:29:25.979Z · LW · GW

In the parent, is your objection that the trained AlphaZero-like model plausibly knows nothing at all?

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:28:19.939Z · LW · GW

Suppose you have a computer program that gets two neural networks, simulates a game of go between them, determines the winner, and uses the outcome to modify the neural networks. It seems to me that this program has a model of the 'go world', i.e. a simulator, and from that model you can fairly easily extract the rules and winning condition. Do you think that this is a model but not a mental model, or that it's too exact to count as a model, or something else?

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:23:33.051Z · LW · GW

I feel like it's pretty relevant that AlphaGo is the worst super-human go bot, and I don't think better bots have this behaviour.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:21:29.211Z · LW · GW

(Though KataGo's network isn't a pure CNN and does some global things too; I forget the details.)

The 'global' things seem to be pooling operations that compute channel-wise means and maxes. Paper link.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:16:03.356Z · LW · GW

I don't think joseki are the main reason why professional go players spend so much time studying, unless you define "studying" more narrowly than I would.

This is also my understanding.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:12:43.412Z · LW · GW

One axis along which I'd like clarification is whether you want a form of explanation which is learner agnostic or learner specific?

I don't know what you mean by "learner agnostic" or "learner specific". Could you explain?

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:12:05.173Z · LW · GW

Here's a second operationalization of "know." You're allowed to train up using all the computerized help you want. But then, to prove your ability, you have to perfectly predict the output of the Go program on a set of randomly generated board positions, using only the power of your own brain.

I was thinking more of propositional knowledge (well, actually belief, but it doesn't seem like this was a sticking point with anybody). A corollary of this is that you would be able to do this second operationalization, but possibly with the aid of a computer program that you wrote yourself that wasn't just a copy of the original program. This constraint is slightly ambiguous but I think it shouldn't be too problematic in practice.

Did you have a different operationalization in mind?

The actual thing I had in mind was "come up with a satisfactory operationalization".

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:06:53.774Z · LW · GW

I guess it seems to me that you're claiming that the referent AI isn't doing any mirror-modelling, but I don't know why you'd strongly believe this. It seems false about algorithms that use Monte Carlo Tree Search as KataGo does (altho another thread indicates that smart people disagree with me about this), but even for pure neural network models, I'm not sure why one would be confident that it's false.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-14T18:01:48.440Z · LW · GW

I think I basically agree with all of this.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T22:39:42.577Z · LW · GW

I think you're engaging in a category error when you posit that, for example, a neural network actually knows anything at all.

Why do you believe that?

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T19:26:33.121Z · LW · GW

I think there's some communication failure where people are very skeptical of this for reasons that they think are obvious given what they're saying, but which are not obvious to me. Can people tell me which subset of the below claims they agree with, if any? Also if you come up with slight variants that you agree with that would be appreciated.

  1. It is approximately impossible to succeed at this challenge.
  2. It is possible to be confident that advanced AGI systems will not pose an existential threat without being able to succeed at this challenge.
  3. It is not obvious what it means to succeed at this challenge.
  4. It will probably not be obvious what it means to succeed at this challenge at any point in the next 10 years, even if a bunch of people try to work on it.
  5. We do not currently know what it means for a go bot to know something in operational terms.
  6. At no point in the next 10 years could one be confident that one knew everything a go bot knew, because we won't be confident about what it means for a go bot to know something.
  7. You couldn't know everything a go bot knows without essentially being that go bot.

[EDIT: 8. One should not issue a challenge to know everything a go bot knows without having a good definition of what it means for a go bot to know things.]

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T19:25:23.165Z · LW · GW

I'd also be happy with an inexact description of what the bot will do in response to specified strategies that captured all the relevant details.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T19:04:05.978Z · LW · GW

I think that it isn't clear what constitutes "fully understanding" an algorithm.

That seems right.

Another obstacle to full understanding is memory. Suppose your go bot has memorized a huge list of "if you are in such and such situation move here" type rules.

I think there's reason to believe that SGD doesn't do exactly this (nets that memorize random data have different learning curves than normal nets iirc?), and better reason to think it's possible to train a top go bot that doesn't do this.

There is not in general a way to compute what an algorithm does without running it.

Yes, but luckily you don't have to do this for all algorithms, just the best go bot. Also as mentioned, I think you probably get to use a computer program for help, as long as you've written that computer program.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T18:56:06.409Z · LW · GW

I think it's probably possible to develop better transparency tools than we currently have to extract knowledge from AIs, or make their cognition more understandable.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T17:33:23.456Z · LW · GW

Hmmm. It does seem like I should probably rewrite this post. But to clarify things in the meantime:

  • it's not obvious to me that this is a realistic target, and I'd be surprised if it took fewer than 10 person-years to achieve.
  • I do think the knowledge should 'cover' all the athlete's ingrained instincts in your example, but I think the propositions are allowed to look like "it's a good idea to do x in case y".
Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T16:53:50.970Z · LW · GW

On that definition, how does one train an AlphaZero-like algorithm without knowing the rules of the game and win condition?

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T16:48:48.187Z · LW · GW

I'm not familiar with chess bots, but I would be surprised if one could be confident that chess GMs know everything that chess bots know.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T07:25:29.019Z · LW · GW

Perhaps the bot knows different things at different times and your job is to figure out (a) what it always knows and (b) a way to quickly find out everything it knows at a certain point in time.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T07:23:54.285Z · LW · GW

Also it certainly knows the rules of go and the win condition.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T07:23:09.417Z · LW · GW

But once you let it do more computation, then it doesn't have to know anything at all, right? Like, maybe the best go bot is, "Train an AlphaZero-like algorithm for a million years, and then use it to play."

I would say that bot knows what the trained AlphaZero-like model knows.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T07:21:25.902Z · LW · GW

I roughly know what it means, by virtue of knowing what it means to know stuff. But I think I mention that one of the parts is operationalizing better what it means for a model to know things.

Comment by DanielFilan on Challenge: know everything that the best go bot knows about go · 2021-05-11T06:55:59.549Z · LW · GW

Maybe it nearly suffices to get a go professional to know everything about go that the bot does? I bet they could.