Posts

cousin_it's Shortform 2019-10-26T17:37:44.390Z · score: 3 (1 votes)
Announcement: AI alignment prize round 4 winners 2019-01-20T14:46:47.912Z · score: 80 (19 votes)
Announcement: AI alignment prize round 3 winners and next round 2018-07-15T07:40:20.507Z · score: 102 (29 votes)
How to formalize predictors 2018-06-28T13:08:11.549Z · score: 16 (5 votes)
UDT can learn anthropic probabilities 2018-06-24T18:04:37.262Z · score: 63 (19 votes)
Using the universal prior for logical uncertainty 2018-06-16T14:11:27.000Z · score: 0 (0 votes)
Understanding is translation 2018-05-28T13:56:11.903Z · score: 138 (46 votes)
Announcement: AI alignment prize round 2 winners and next round 2018-04-16T03:08:20.412Z · score: 153 (45 votes)
Using the universal prior for logical uncertainty (retracted) 2018-02-28T13:07:23.644Z · score: 39 (10 votes)
UDT as a Nash Equilibrium 2018-02-06T14:08:30.211Z · score: 36 (12 votes)
Beware arguments from possibility 2018-02-03T10:21:12.914Z · score: 13 (9 votes)
An experiment 2018-01-31T12:20:25.248Z · score: 32 (11 votes)
Biological humans and the rising tide of AI 2018-01-29T16:04:54.749Z · score: 55 (18 votes)
A simpler way to think about positive test bias 2018-01-22T09:38:03.535Z · score: 34 (13 votes)
How the LW2.0 front page could be better at incentivizing good content 2018-01-21T16:11:17.092Z · score: 38 (19 votes)
Beware of black boxes in AI alignment research 2018-01-18T15:07:08.461Z · score: 71 (30 votes)
Announcement: AI alignment prize winners and next round 2018-01-15T14:33:59.892Z · score: 166 (63 votes)
Announcing the AI Alignment Prize 2017-11-04T11:44:19.000Z · score: 1 (1 votes)
Announcing the AI Alignment Prize 2017-11-03T15:47:00.092Z · score: 155 (67 votes)
Announcing the AI Alignment Prize 2017-11-03T15:45:14.810Z · score: 7 (7 votes)
The Limits of Correctness, by Bryan Cantwell Smith [pdf] 2017-08-25T11:36:38.585Z · score: 3 (3 votes)
Using modal fixed points to formalize logical causality 2017-08-24T14:33:09.000Z · score: 3 (3 votes)
Against lone wolf self-improvement 2017-07-07T15:31:46.908Z · score: 30 (28 votes)
Steelmanning the Chinese Room Argument 2017-07-06T09:37:06.760Z · score: 5 (5 votes)
A cheating approach to the tiling agents problem 2017-06-30T13:56:46.000Z · score: 3 (3 votes)
What useless things did you understand recently? 2017-06-28T19:32:20.513Z · score: 7 (7 votes)
Self-modification as a game theory problem 2017-06-26T20:47:54.080Z · score: 10 (10 votes)
Loebian cooperation in the tiling agents problem 2017-06-26T14:52:54.000Z · score: 5 (5 votes)
Thought experiment: coarse-grained VR utopia 2017-06-14T08:03:20.276Z · score: 16 (16 votes)
Bet or update: fixing the will-to-wager assumption 2017-06-07T15:03:23.923Z · score: 26 (26 votes)
Overpaying for happiness? 2015-01-01T12:22:31.833Z · score: 32 (33 votes)
A proof of Löb's theorem in Haskell 2014-09-19T13:01:41.032Z · score: 29 (30 votes)
Consistent extrapolated beliefs about math? 2014-09-04T11:32:06.282Z · score: 6 (7 votes)
Hal Finney has just died. 2014-08-28T19:39:51.866Z · score: 33 (35 votes)
"Follow your dreams" as a case study in incorrect thinking 2014-08-20T13:18:02.863Z · score: 29 (31 votes)
Three questions about source code uncertainty 2014-07-24T13:18:01.363Z · score: 9 (10 votes)
Single player extensive-form games as a model of UDT 2014-02-25T10:43:12.746Z · score: 21 (12 votes)
True numbers and fake numbers 2014-02-06T12:29:08.136Z · score: 19 (29 votes)
Rationality, competitiveness and akrasia 2013-10-02T13:45:31.589Z · score: 14 (15 votes)
Bayesian probability as an approximate theory of uncertainty? 2013-09-26T09:16:04.448Z · score: 16 (18 votes)
Notes on logical priors from the MIRI workshop 2013-09-15T22:43:35.864Z · score: 18 (19 votes)
An argument against indirect normativity 2013-07-24T18:35:04.130Z · score: 1 (14 votes)
"Epiphany addiction" 2012-08-03T17:52:47.311Z · score: 52 (56 votes)
AI cooperation is already studied in academia as "program equilibrium" 2012-07-30T15:22:32.031Z · score: 36 (37 votes)
Should you try to do good work on LW? 2012-07-05T12:36:41.277Z · score: 36 (41 votes)
Bounded versions of Gödel's and Löb's theorems 2012-06-27T18:28:04.744Z · score: 32 (33 votes)
Loebian cooperation, version 2 2012-05-31T18:41:52.131Z · score: 13 (14 votes)
Should logical probabilities be updateless too? 2012-03-28T10:02:09.575Z · score: 12 (15 votes)
Common mistakes people make when thinking about decision theory 2012-03-27T20:03:08.340Z · score: 51 (46 votes)
An example of self-fulfilling spurious proofs in UDT 2012-03-25T11:47:16.343Z · score: 20 (21 votes)

Comments

Comment by cousin_it on cousin_it's Shortform · 2019-12-08T13:18:17.148Z · score: 3 (1 votes) · LW · GW

Basics of digital filters.

Imagine a sequence of numbers x gets transformed into another sequence y by this formula:

y[n] = ax[n]+bx[n-1]

That's the simplest example of a digital filter. Numbers come in, numbers come out. Let's rewrite the formula a bit:

x[n] = (y[n]-bx[n-1])/a

That shows we can compute the current value of x if we know the past values of x and the past and current values of y, so our filter is invertible.

The operation of inverting a filter can be expressed nicely with generating functions. Let's say the generating function of x is X(z) = sum(x[n]z^(-n)), and the same for y and Y(z). (We'll see in a moment why negative exponents on z make sense.) Our definition of y can be rewritten in terms of generating functions:

Y(z) = X(z)*(a+bz^(-1))

So we can say that our filter has "transfer function" H(z) = Y(z)/X(z) = (a+bz^(-1)). To obtain the generating function of y, we multiply the generating function of x by the transfer function. Since the inverse of multiplication is division, the inverse filter's transfer function is H_inv(z) = 1/(a+bz^(-1)). We can do the same for any linear recurrence involving values of x and y, and different ways to express the same recurrence will lead to the same transfer function.

Recall that our inverse filter isn't quite uniquely defined. The current value of y depends on previous values of y, but where did the first value of y come from? That's another way of saying that there are some nonzero sequences x which lead to y being always zero.

One such sequence x is 1,-b/a,(b/a)^2,-(b/a)^3,... Equivalently, if z=-b/a, then H(z) = 0. (That's why the negative exponent!) Equivalently, as z approaches -b/a, H_inv(z) goes to infinity. We say the direct filter has a "zero" at -b/a, and the inverse filter has a "pole" at -b/a.

In particular, if the absolute value of -b/a is greater than 1, that means there's a sequence x that goes to infinity and leads to y being constant zero. In that case, the inverse filter is unstable: for some sequence y that's very close to constant zero, it will output a sequence x that goes to infinity. So for a filter to be stable, all poles of its transfer function must lie within the unit circle.

Here's one way to understand transfer functions. Consider an input signal that rotates at a constant speed: x[n]=z^n where z is some complex number with |z|=1. (The real part of such a signal is a sinusoid.) Then if y[n]=cz^n, then H(z)=c. That means |H(z)| encodes how much the filter amplifies or dampens a given frequency in the input signal, and arg H(z) encodes what happens to the phase.

That raises the possibility of "allpass filters". It turns out there are many transfer functions H(z) that have |H(z)|=1 everywhere on the unit circle. The trivial ones are multiplications by a constant (corresponding to phase shift) or raising z to an integer power (corresponding to time delay), but there are also more interesting ones, like (k+1/z)/(1+k/z) for some k with |k|<1. (Or in recurrence form, y[n] = kx[n] + x[n-1] - ky[n-1].) This filter keeps all frequencies in the input signal intact, but shifts their phases around.

Comment by cousin_it on cousin_it's Shortform · 2019-12-04T17:17:20.809Z · score: 3 (1 votes) · LW · GW

Edit: no point asking this question here.

Comment by cousin_it on CO2 Stripper Postmortem Thoughts · 2019-12-04T10:55:44.473Z · score: 3 (1 votes) · LW · GW

Ah, silly me. Thanks!

Comment by cousin_it on CO2 Stripper Postmortem Thoughts · 2019-12-04T07:59:18.502Z · score: 3 (1 votes) · LW · GW

One human produces about 1 kg of CO2 in 24 hours. We can idealize a perfect CO2 stripper as a magic box that inhales air and spits it out at 0 ppm. If you want a steady-state concentration of 500 ppm for 2 people, then we can see how much air-flow is required to lock up 2 kg of CO2 in 24 hours. This comes out to about 100 cubic feet per minute. This is the bare minimum air flow for any CO2 stripper

Wait, why does stripping one person's CO2 require more airflow than breathing (0.3 cubic feet per minute)?

Comment by cousin_it on A Practical Theory of Memory Reconsolidation · 2019-12-02T13:47:23.050Z · score: 6 (3 votes) · LW · GW

Thank you! Not sure about writing a life advice column, that's not quite who I am, but if you're interested in anything in particular, I'll be happy to answer.

Comment by cousin_it on A Practical Theory of Memory Reconsolidation · 2019-12-01T13:04:25.358Z · score: 17 (5 votes) · LW · GW

Welcome to LW! I haven't been in your situation, but it feels like I could have been, if things turned out a bit differently. So take this for what it's worth.

I think a lot of it comes down to the way you talk and recount your feelings to others. It can feel either "spiky" or "rounded", like the kiki/bouba effect. For example, when you say you want to secure love and affection, that's you being honest, and also "spiky". These are not the same thing! There's a whole art form of expressing your feelings, even going to some very dark places, while still coming across as "rounded". Take the edge off of your word choices; understate things; allow a range of possible reactions; be kind to the feelings of the person you're talking to. It might feel like a restrictive filter, but to me it's liberating. I can just have a normal conversation with anyone, anytime, about anything that's in my head.

It's not quite the "route to love" that you're looking for, but it opens the door to some new connections, and sometimes you'll click with someone in a deeper way. Hope that made sense.

Comment by cousin_it on The Correct Contrarian Cluster · 2019-11-29T13:59:55.916Z · score: 15 (6 votes) · LW · GW

Eliezer's econ case is based on reading Scott Sumner's blog, so it's not very informative that Sumner praises Eliezer (3 out of 4 endorsements you linked, the remaining one is anon).

Comment by cousin_it on Open-Box Newcomb's Problem and the limitations of the Erasure framing · 2019-11-28T17:12:30.588Z · score: 3 (1 votes) · LW · GW

I see, thanks, that makes it clearer. There's no disagreement, you're trying to justify the approach that people are already using. Sorry about the noise.

Comment by cousin_it on Open-Box Newcomb's Problem and the limitations of the Erasure framing · 2019-11-28T15:14:28.496Z · score: 3 (1 votes) · LW · GW

Well, the program is my formalization. All the premises are right there. You should be able to point out where you disagree.

Comment by cousin_it on Open-Box Newcomb's Problem and the limitations of the Erasure framing · 2019-11-28T14:19:35.584Z · score: 3 (1 votes) · LW · GW

I couldn't understand your comment, so I wrote a small Haskell program to show that two-boxing in the transparent Newcomb problem is a consistent outcome. What parts of it do you disagree with?

Comment by cousin_it on Open-Box Newcomb's Problem and the limitations of the Erasure framing · 2019-11-28T12:31:59.211Z · score: 3 (1 votes) · LW · GW

If you see a full box, then you must be going to one-box if the predictor really is perfect.

Huh? If I'm a two-boxer, the predictor can still make a simulation of me, show it a simulated full box, and see what happens. It's easy to formalize, with computer programs for the agent and the predictor.

Comment by cousin_it on Effect of Advertising · 2019-11-27T17:20:44.512Z · score: 5 (2 votes) · LW · GW

You're probably better informed than me, but I thought it was relatively easy to deny service in case of adblock (without trying to show a content teaser, nag message, or ad). Or at least that's easier than getting an ad through.

Comment by cousin_it on Effect of Advertising · 2019-11-27T12:59:16.024Z · score: 3 (1 votes) · LW · GW

Yeah, "excludable" is the key part. Privatizing the commons is supposed to prevent tragedy of the commons and lead to an efficient outcome. Since privately owned websites can choose anytime to use an adblock detector (these exist and work fine) or start charging viewers, we should expect an efficient outcome.

Comment by cousin_it on Effect of Advertising · 2019-11-27T09:48:03.217Z · score: 8 (2 votes) · LW · GW

The wording "disproportionately smaller effects" seems like assuming the conclusion. To me, using adblock has a positive effect. You say if everyone does that, another effect will arise and the sum of two effects will be net negative. But in econ, when everyone chooses what's good for them, the result is usually net positive. There's an exception in case of tragedy of the commons, but many people refusing to look at ads isn't a tragedy of the commons, because all goods involved are private and excludable.

Comment by cousin_it on Mental Mountains · 2019-11-27T09:40:40.040Z · score: 5 (2 votes) · LW · GW

I don't know. In "Epistemic Learned Helplessness" you pointed out that both right and wrong positions have many convincing arguments, so becoming more open to arguments is just as likely to make someone wrong as right.

Comment by cousin_it on Effect of Advertising · 2019-11-27T00:12:17.437Z · score: 12 (4 votes) · LW · GW

Whenever I use adblock, or visit a place that bans roadside billboards, I'm dipping a toe in your imagined world. By your argument, that should make me worse off. But to me it feels better.

Comment by cousin_it on Breaking Oracles: superrationality and acausal trade · 2019-11-25T14:21:25.110Z · score: 4 (2 votes) · LW · GW

Ouch. For example, if an oracle is asked "what's the weather tomorrow" and it suspects that there might be other oracles in the world, it could output a message manipulating humans to reward all oracles, hoping that other oracles in a similar position would do the same. Since this problem applies more to oracles that know less, it could happen pretty early in oracle development :-/

Comment by cousin_it on Acting without a clear direction · 2019-11-24T10:07:29.631Z · score: 6 (2 votes) · LW · GW

Yeah, I'm not sure consequentialism can get us out of the hole that it created. One possible alternative is virtue ethics: trying to live with healthy amounts of courage, ambition, generosity and so on.

Comment by cousin_it on Analysing: Dangerous messages from future UFAI via Oracles · 2019-11-22T16:10:44.313Z · score: 4 (2 votes) · LW · GW

It seems that coordinated erasure, chokepoints and short horizons can help with this problem as well. But if many companies or governments have their own oracles and benefit from longer horizons, it gets harder.

Comment by cousin_it on What makes a good life? This is my map. · 2019-11-22T15:35:45.620Z · score: 7 (3 votes) · LW · GW

Relatedness: I am sensitive to the needs of others; I feel that others are sensitive to my needs

To me, having relationships with people is less about sensitivity, and more about having friends and family who have my back and I have theirs, even if they're insensitive assholes and so am I. Even though that's an unpopular view today.

Comment by cousin_it on Analysing: Dangerous messages from future UFAI via Oracles · 2019-11-22T15:03:41.167Z · score: 7 (4 votes) · LW · GW

the bucket chain can only accelerate a future UFAI, not create one

Wait, I'm not sure that's true anymore. Imagine there are two oracles running at the same time, with independent erasure events. Can they "cooperate" to bring into existence an UFAI that will reward them both?

Comment by cousin_it on Relevance Norms; Or, Gricean Implicature Queers the Decoupling/Contextualizing Binary · 2019-11-22T13:09:21.816Z · score: 20 (10 votes) · LW · GW

Many people have tried to draw such lines: literal vs contextual, mistake vs conflict theory, science vs religion. My line is mostly about reactions to disagreement. An engineer will say ok, I'll be in my garage working on my flying car. A politician will say ok, let's find points of contact. But a fanatic will call their friends and paint a target on me for disagreeing. I wouldn't do that to anyone for disagreeing with me, so it seems like a pretty sharp line.

Comment by cousin_it on Do you get value out of contentless comments? · 2019-11-22T09:07:55.260Z · score: 8 (4 votes) · LW · GW

Yes to both, but I'd still rather get comments that are interesting :-)

Comment by cousin_it on How I do research · 2019-11-21T17:29:57.253Z · score: 4 (2 votes) · LW · GW

Congrats on the great work then! Maybe ask him about flowers in the margins?

Comment by cousin_it on A Brief Intro to Domain Theory · 2019-11-21T08:23:33.207Z · score: 8 (6 votes) · LW · GW

For programmers, I think the simplest way to motivate domain theory is to think about a lazy language like Haskell. In such a language, if you have for example a list of numbers, it doesn't actually contain numbers, but rather "thunks" that when evaluated will give numbers. Some of these thunks, when you try to evaluate them, can go into an infinite loop instead of giving a number. Let's call these "bottoms". Basically any data structure can have some of its parts replaced with bottoms: a subtree of a tree can be bottom, and so on. Then we can say all functions in a lazy language are total functions from things-with-bottoms to things-with-bottoms: if your function goes into an infinite loop, we'll just say it returns a bottom.

Then we can say a piece of data X is "at least as informative" as Y, if Y can be obtained from X by replacing zero or more of its parts with bottoms. That's a partial ordering. And it's kind of obvious that any function we can write in our language will be monotone with respect to that ordering: f(X) will be at least as informative as f(Y).

So basically domain theory is what you get if you ask yourself "what if data structures could contain bottoms" and turn the abstractness crank a few times. In a lazy language it applies pretty naturally; in a strict language it's a bit more awkward, because things-with-bottoms don't exist at runtime to be passed around; and in a total language it's unnecessary, because all functions are already total.

Comment by cousin_it on How I do research · 2019-11-20T22:18:47.541Z · score: 4 (2 votes) · LW · GW

Yeah, he's doing it right :-)

Comment by cousin_it on How I do research · 2019-11-20T16:40:10.614Z · score: 19 (6 votes) · LW · GW

Agreed this attempt was kind of criminal, but still inspiring in its own way. Modernism has been making me tired. The internet could really use some drop caps, flourishes, flower-filled margins, and random weird drawings in the middle of text.

Comment by cousin_it on Internalizing Existentialism · 2019-11-18T20:43:20.200Z · score: 3 (1 votes) · LW · GW

What do you think about the mood effects of internalizing these kinds of ideas? For me, somehow existentialist/unmoored thinking seems to correlate with bad moods, and thinking about emotions as factual seems to correlate with better moods.

Comment by cousin_it on Impossible moral problems and moral authority · 2019-11-18T20:34:39.501Z · score: 3 (1 votes) · LW · GW

It seems to me that with meals, there's a fact of the matter that AI could help with. After all, if two copies of you went and had the different meals, one of them would probably be happier than the other.

Though that happiness might not depend only on the chosen meal. For example, if one meal is a cake that looks exactly like Justin Bieber, that might be actually not as fun as it sounds. But if you skipped it in favor of an ordinary burrito, you'd forever regret that you didn't get to see the Justin Bieber cake.

Comment by cousin_it on Steelmanning social justice · 2019-11-18T19:35:26.453Z · score: 19 (7 votes) · LW · GW

Not to attack you, but I quit reading political subreddits because it was making me unhappy. If this kind of thing becomes common on LW, it'll be less fun for me.

Comment by cousin_it on The Power to Draw Better · 2019-11-18T15:18:07.319Z · score: 7 (4 votes) · LW · GW

Check out the books by Loomis, freely downloadable here. For me, one look at page 21 of "Drawing the Head and Hands" was enough to sell me on constructive drawing.

Comment by cousin_it on The Power to Draw Better · 2019-11-18T08:50:35.612Z · score: 21 (8 votes) · LW · GW

Betty Edwards' book is a fun way to get into drawing, but "drawing what you see" only takes you so far. For example, the visible outline of a human figure is infinitely changing: if you learn how it looks for one pose and viewing angle, it won't help you at all with drawing a slightly different pose or angle. Yet there are many people in the world who can draw a realistic human, in any pose and viewing angle, without reference. That skill doesn't magically emerge from many hours of drawing what you see, it needs to be learned on its own terms - "drawing what you know" or "constructive drawing". I think that method might even be easier for LW folks, because it's more systematic and analytical.

Here's the difference between the two methods: imagine two artists are asked to draw a vase. Betty, a proponent of drawing what she sees, starts by sketching the outline of the vase, paying attention to the beautiful curves of negative space around the handles. Then she gradually fills it in. Meanwhile Connie, a proponent of constructive drawing, starts with the scaffolding instead. A straight line for the axis; an ellipse for the rim, whose center and minor axis lie exactly on the axis; another ellipse for the bottom; some more elliptical cross-sections. A vertical plane passing through the axis, to make sure the handles are symmetrical. Now drawing the actual vase is easy, and unlike Betty's, it will look solid and right.

The same method works for more complex shapes, with different kinds of scaffolding for each. Where Betty draws an outline of a human head, Connie starts with a sphere and a plane of symmetry. Where Betty draws an outline of a car, Connie starts with a perspective-correct transparent box around the car, then draws inward. Where Betty draws a cat starting from a fuzzy blob, Connie starts with a posed skeleton and draws outward. It doesn't give as fast gratification as Betty's method, but the skill builds on itself: once you know the rough 3D shape of a human skull, it becomes a foundation for learning more detail. Everything stays relevant for future drawings from different angles, and nothing gets forgotten.

Comment by cousin_it on [Link] Is the Orthogonality Thesis Defensible? (Qualia Computing) · 2019-11-13T17:40:03.961Z · score: 34 (8 votes) · LW · GW

Of course, it may be possible, but just not reachable through Darwinian selection. But the fact that a search process as huge as evolution couldn’t find it and instead developed profoundly sophisticated phenomenally bound subjectivity is (possibly strong) evidence against the proposition that zombie AGI is possible (or likely to be stumbled on by accident).

This argument also proves that quantum computers can't offer useful speedups, because otherwise evolution would've found them.

If OI is also true, then smarter than human AGIs will likely converge on this as well – since it’s within the reach of smart humans – and this will plausibly lead to AGIs adopting sentience in general as their target for valence optimization.

This argument also proves that Genghis Khan couldn't have happened, because intelligence and power converge on caring about positive valence for all beings.

I think the orthogonality thesis is in good shape, if these are the strongest arguments we could find against it in almost a decade.

Comment by cousin_it on Vanessa Kosoy's Shortform · 2019-11-13T16:16:24.278Z · score: 3 (1 votes) · LW · GW

Yeah, I think I can make peace with that. Another way to think of it is that we can keep the reward structure of the original Newcomb's problem, but instead of saying "Omega is almost always right" we add another person Bob (maybe the mad scientist who built Omega) who's willing to pay you a billion dollars if you prove Omega wrong. Then minimaxing indeed leads to one-boxing. Though I guess the remaining question is why minimaxing is the right thing to do. And if randomizing is allowed, the idea of Omega predicting how you'll randomize seems a bit dodgy as well.

Comment by cousin_it on Vanessa Kosoy's Shortform · 2019-11-13T12:07:24.159Z · score: 3 (1 votes) · LW · GW

I'm not sure your definition has much to do with consciousness, as it would also be satisfied by an AI that runs on an Intel processor and whose utility function says all AIs should run on Intel processors.

Comment by cousin_it on Vanessa Kosoy's Shortform · 2019-11-13T11:10:35.136Z · score: 3 (1 votes) · LW · GW

But in Newcomb's problem, the agent's reward in case of wrong prediction is already defined. For example, if the agent one-boxes but the predictor predicted two-boxing, the reward should be zero. If you change that to +infinity, aren't you open to the charge of formalizing the wrong problem?

Comment by cousin_it on Operationalizing Newcomb's Problem · 2019-11-12T20:41:02.446Z · score: 3 (1 votes) · LW · GW

Sorry, I wrote some nonsense in another comment and then deleted it. I guess the point is that UDT (which I agree with) recommends non-equilibrium behavior in this case.

Comment by cousin_it on Operationalizing Newcomb's Problem · 2019-11-12T20:32:23.440Z · score: 5 (3 votes) · LW · GW

My comment didn't say two-boxing was the only equilibrium, or that you should always play the equilibrium. That's because I don't agree with these statements and in fact I agree with UDT.

The point was more that from the perspective of standard game theory (where the best response to "predict two-boxing" is "two-box"), Newcomb's problem seems to be about equilibrium selection. If that's right, that would explain why it's so hard to come up with a simplified version where one-boxing would be obviously best. And it would explain why analogies between Newcomb's problem and the prisoner's dilemma are misleading, because the latter is more about wishing for a non-equilibrium outcome.

Comment by cousin_it on Operationalizing Newcomb's Problem · 2019-11-12T13:58:39.246Z · score: 4 (2 votes) · LW · GW

I just figured out why Newcomb's problem feels "slippery" when we try to describe it as a two-player game. If we treat the predictor as a player with some payoffs, then the strategy pair {two-box, predict two-boxing} will be a Nash equilibrium, because each strategy is the best response to the other. That's true no matter which payoff matrix we choose for the predictor, as long as correct predictions lead to higher payoffs.

Comment by cousin_it on An optimal stopping paradox · 2019-11-12T10:31:43.581Z · score: 9 (5 votes) · LW · GW

Let's change the problem a bit: assume you're starting with nonzero capital c, so the formula becomes (bt+c)e^(-at). If c>b/a, the derivative of that formula at t=0 is negative, so you need to stop immediately. That shows the decision to stop doesn't depend only on a and b, but also on current capital. So basically "at each moment in time you face the exact same problem" is wrong. The naive solution is the right one: you should stop when c=b/a, which means t=1/a in the original problem.

Comment by cousin_it on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-11T18:14:18.488Z · score: 5 (2 votes) · LW · GW

Agreed - I've seen, and made, quite a few such changes as well. After each big upheaval it's worth spending some time grabbing the low hanging fruit. My only gripe is that I don't think this type of change is sufficient over a project's lifetime. Deeper product change has a way of becoming necessary.

Comment by cousin_it on Experiments and Consent · 2019-11-11T17:54:34.263Z · score: 10 (4 votes) · LW · GW

I agree that A/B tests aren't evil, and are often useful. All I'm saying is, sometimes they give ammo to dark pattern thinking in the minds of people within your company, and reversing that process isn't easy.

Comment by cousin_it on Experiments and Consent · 2019-11-11T09:59:00.556Z · score: 8 (3 votes) · LW · GW

Another ethical consideration is that most A/B tests aren't aimed to help the user, but to improve metrics that matter to the company, like engagement or conversion rate. All the sketchy stuff you see on the web - sign up for your free account, install our mobile app, allow notifications, autoplay videos, social buttons, fixed headers and footers, animated ads - was probably justified by A/B testing at some point.

Comment by cousin_it on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-08T10:57:25.828Z · score: 31 (9 votes) · LW · GW

I think incremental change is a bit overrated. Sure, if you have something that performs so well that chasing 1% improvements is worth it, then go for it. But don't keep tweaking forever: you'll get most of the gains in the first few months, and they will total about +20%, or maybe +50% if you're a hero.

If your current thing doesn't perform so well, it's more cost-effective to look for big things that could bring +100% or +1000%. A/B tests are useful for that too, but need to be done differently:

  1. Come up with a big thing that could have big impact. For example, shortform.

  2. Identify the assumptions behind that thing. For example, "users will write shortform" or "users will engage with others' shortform".

  3. Come up with cheap ways to test these assumptions. For example, "check the engagement on existing posts that are similar to shortform" or "suggest to some power users that they should make shortform posts and see how much engagement they get". At this step you may end up looking at metrics, looking at competitors, or running cheap A/B tests.

  4. Based on the previous steps, change your mind about which thing you want to build, and repeat these steps until you're pretty sure it will succeed.

  5. Build the thing.

Comment by cousin_it on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-07T12:03:03.862Z · score: 5 (2 votes) · LW · GW

I'm not sure that works. Imagine there are two theories about a certain button:

  1. Pressing the button gives you a dollar and does nothing else

  2. Pressing the button gives you a dollar and also causes an unobservable person to suffer

These theories give the same observations, but recommend different actions, so to me they seem to have different meanings.

Comment by cousin_it on Normative reductionism · 2019-11-06T10:27:34.471Z · score: 5 (2 votes) · LW · GW

Some qualities are separable (the mass of an object is the sum of masses of its parts) and others aren't (the enjoyability of a book isn't the sum of enjoyabilities of its pages). I'm not sure human value is separable. At least some of my values seem to be about relationships between people, not just individual experiences.

Comment by cousin_it on Elon Musk is wrong: Robotaxis are stupid. We need standardized rented autonomous tugs to move customized owned unpowered wagons. · 2019-11-04T16:37:28.340Z · score: 7 (5 votes) · LW · GW

Maybe a bit off topic, but to me a future filled with robo-taxis, automated cashiers, drone delivery and other such things would feel kind of scary and alien. I've always lived in cities where you need constant human interaction to get by, and for myself I'd like things to stay that way or even more so.

Comment by cousin_it on cousin_it's Shortform · 2019-11-04T16:21:54.466Z · score: 3 (1 votes) · LW · GW

(13/?)

Global phase.

Consider a quantum system that's even smaller than a qubit. Something with only one possible measurement outcome, instead of two. Let's call it "quantum nit", or "qunit", by analogy with "unit" in typed programming, which means a type with one element.

A qunit has only one possible mixed state, with the density matrix ((1)). But its pure states are more complicated: a one-qunit pure state is a number whose norm is 1, so either 1 or -1 (or more if we allow complex numbers). The same is true for an N-qunit pure state - it's also a single number whose norm is 1! The reason is that the tensor product of N 1-dimensional spaces is 1-dimensional, not N-dimensional. So the joint state of all qunits in the world is a single number, which is called "global phase".

Now imagine we have a system with a bunch of qubits, and take its tensor product with a qunit. We can move a scalar factor from one side of a tensor product to the other, and that shouldn't change anything. So global phase can't affect any measurement, it's just an extra degree of freedom in the theory.

Comment by cousin_it on cousin_it's Shortform · 2019-11-04T11:45:17.805Z · score: 3 (1 votes) · LW · GW

(12/?)

The uncertainty principle, or something like it.

When you measure a qubit cos(φ)|0> + sin(φ)|1>, the result has variance (1-cos(4φ))/4. (I'm skipping over the trig calculations here and below.) If you have a choice between either measuring the qubit, or rotating it by an angle ψ and then measuring it, then the sum of variances of the two operations is at least (1-|cos(2ψ)|)/2. In particular, if ψ=π/4, the sum of variances is at least 1/2.

So no matter what state a qubit is in, and how precisely you know its state, there are two possible measurements such that the sum of variances of their results is at least 1/2. The reason is that the bases corresponding to these measurements are at an angle to each other, not aligned. Apparently in the real world, an object's "position" and "momentum" are two such measurements, so there's a limit on their joint precision.

You can also carry out one measurement and then the other, but that doesn't help - after the first measurement you have variance in the second. Moreover, after the second you have fresh variance in the first. This lets you get an infinite stream of fair coinflips from a single qubit: start with |0>, rotate by π/4, measure, rotate by π/4, measure...

Comment by cousin_it on cousin_it's Shortform · 2019-11-01T10:06:16.086Z · score: 5 (2 votes) · LW · GW

(11/?)

Superdense coding.

Alice is told two classical bits of information and sends a qubit to Bob, who can then recover the two classical bits. Again, it relies on Alice and Bob sharing a prepared pair beforehand. It's the opposite of quantum teleportation, where Alice sends two classical bits and Bob can recover a qubit.

First let's talk about bases. This is the usual basis: |00>, |01>, |10>, |11>. This is the Bell basis: (|00> + |11>)/√2, (|00> - |11>)/√2, (|10> + |01>)/√2, (|10> - |01>)/√2. Check for yourself that each two vectors are orthogonal to each other. To "measure a state in a different basis" means to apply a rotation from one basis into another, then measure. For example, if you have a state and you know that it's one of the Bell basis states, you can figure out which one, by rotating into the usual basis and measuring.

One cool thing about the Bell basis is that you can change any basis vector into any other basis vector by operations on the first qubit only. For example, rotating the first qubit by 90 degrees turns (|00> + |11>)/√2 into (|10> - |01>)√2. Flipping the first qubit gives (|10> + |01>)√2, and so on.

Now superdense coding is easy. Alice and Bob start by sharing two halves of a Bell state. Then depending on which two classical bits Alice needs to send, she either leaves the state alone or rotates into one of the other three, by operating only on her qubit. Then she sends her qubit to Bob, who now has both qubits and can rotate them into the usual basis and measure them, recovering the two classical bits.