Acausal Trade and the Ultimatum Game 2021-09-05T05:36:28.171Z
The halting problem is overstated 2021-08-16T05:26:06.034Z
Combining the best of Georgian and Harberger taxes 2021-08-12T05:55:46.424Z
Analyzing Punishment as Preventation 2021-07-14T15:12:09.071Z
Reasons for Punishment 2021-07-12T14:53:37.974Z
Are bread crusts healthier? 2021-06-18T15:12:59.527Z
What is your coronavirus prediction for Israel? 2021-06-17T19:10:49.697Z
Can someone help me understand the arrow of time? 2021-06-15T17:26:29.113Z
Doomsday, Sampling Assumptions, and Bayes 2021-06-14T12:41:24.259Z
On making fictional miracles seem plausible 2021-05-31T11:32:59.114Z
Could MMRPGs be used to test economic theories? 2021-05-06T14:26:20.114Z
AIDungeon 3.1 2021-03-01T05:56:34.795Z
Don't encourage prisoners dilemmas 2021-02-16T06:33:36.501Z
Heuristic: Replace "No Evidence" with "No Reason" 2021-02-15T21:25:21.468Z


Comment by Yair Halberstadt (yair-halberstadt) on Acausal Trade and the Ultimatum Game · 2021-09-10T14:07:05.309Z · LW · GW

If you lived in a world where people accepted that, you would be offered 1 cent every time. The point of acausal Trade and timeless decision theory is that other agents will act differently depending on how you would act in a counterfactual.

Comment by Yair Halberstadt (yair-halberstadt) on Acausal Trade and the Ultimatum Game · 2021-09-05T12:12:01.268Z · LW · GW

I would say:

a) the chances are the other player will have roughly similiar commitments to you. b) 50/50 is a shelling point that works well for an acausal Trade. So when choosing my precommitments I know that anything too high is likely to be rejected, anything to low is a waste, and 50/50 is likely to be about what most people will choose - as evidenced from the fact that I'm happy to choose it.

Comment by Yair Halberstadt (yair-halberstadt) on Acausal Trade and the Ultimatum Game · 2021-09-05T12:08:08.635Z · LW · GW

That seems reasonable yes.

Comment by Yair Halberstadt (yair-halberstadt) on Altruism Under Extreme Uncertainty · 2021-08-27T11:26:40.270Z · LW · GW

Is it worth acting when you're comparing a 0.051% chance of doing good to a 0.049% chance of doing harm? Maybe, but it's far from a clean argument. Primum non nocere (first, do no harm) matters too.

I would like to see a utilitarian argument for why that is the case. To me it seems like you could completely change the best course of action by simply changing your definition of the default action to take.

Comment by Yair Halberstadt (yair-halberstadt) on A Better Web is Coming · 2021-08-22T03:39:28.538Z · LW · GW

Didn't Stack Overflow (and in general the stackexchange network) already solve both these problems pretty well? If you're just looking for knowledge creation on a particular topic why wouldn't you use their system?

Comment by Yair Halberstadt (yair-halberstadt) on The halting problem is overstated · 2021-08-16T17:32:54.551Z · LW · GW

That's fair.

I think a better way of putting my point is, can we express the particular behaviors I'm interested in proving, and only have the compiler care about those, rather than checking everything it might be interested in.

Comment by Yair Halberstadt (yair-halberstadt) on Why must plausibilities be represented using real numbers? · 2021-08-16T16:43:26.062Z · LW · GW

I think the point is that

a) The reals are most convenient for representing plausibility.

b) Any results that hold under the transitive and universal comparability laws will still apply if we represent plausibility with reals, so you might as well use them.

Comment by Yair Halberstadt (yair-halberstadt) on The halting problem is overstated · 2021-08-16T14:29:48.148Z · LW · GW

A program has necessary complexity proportional to how complex it is to even express the task it needs to carry out, and implementational complexity, related to the difficulty of actually implementing that in practice. The total complexity is the sum of the two.

For example Conways game of life is extremely simple to state, and it should be relatively simple to express the relationship between the current state and the next to a compiler. There is a wealth of literature on how to actually implement it efficiently however, which is far from simple.

By having the compiler prove that your actual code matches the specification you provide, you can safely make the implementation as complex as you like whilst knowing it still matches the spec. For problems with low implementational complexity, that's not a problem that needs solving, but for those with high implementational complexity it is.

Think of it like unit tests - unit tests can often take far longer to write than the program itself, but once you have them you can much more confidently change the program knowing you want adversely affect the behavior you're interested in. Similarly a good spec of a program can take a long time to write, but in our theoretical language, you will now be able to safely change the program knowing the compiler will catch you with certainty if you make any mistakes.

Comment by Yair Halberstadt (yair-halberstadt) on The halting problem is overstated · 2021-08-16T11:37:24.564Z · LW · GW

Nothing stops me from writing A+B when what I wanted was A*B.

That's perfectly true, but if express at a high level my requirements, it would be theoretically possible for a compiler to tell me that using A*B will not achieve those requirements, halting problem notwithstanding.

Comment by Yair Halberstadt (yair-halberstadt) on The halting problem is overstated · 2021-08-16T11:34:26.750Z · LW · GW

I agree with everything you just said, but in all of these cases the halting problem isn't relevant, which is my key focus here.

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T14:01:10.834Z · LW · GW

To be honest I mostly agree (although you would only lose your skyscraper if somebody was willing to pay 1 dollars more than you are willing to pay, not 1 dollar more than you currently are paying).

I think this setup has some really interesting theoretical properties, and I think is a good start in working out a better taxation system. The ideal system would look very similar to this but somehow preserve stability. I would like to try and dig further to find that system.

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T11:58:24.093Z · LW · GW

How would we determine the total value of the current property?

I don't mind using manual assessment for this, since:

a) the exact value doesn't matter all that much. b) it only needs to be done when you want to drop your bid, and the previous owner claims you haven't owned it long enough. So in most cases it won't be necessary.

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T09:16:11.094Z · LW · GW

it just needs to be more than what the competitor is currently paying

It needs to be higher than what the competitor is willing to pay since they can always counter offer.

You could also require that a bid to take over a piece of land stands for at least a length of time equal to K * (total value of current  property / bid rental price)  for some K, meaning that a takeover would have to be serious before you risked doing this, as you stand to have to pay an amount of money equal to K * total value of current  property if nobody outbids you.

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T08:29:43.460Z · LW · GW

Currently we have the opposite problem. For example I went to a school in a random suburb in not particularly exciting buildings a couple of hundred years old. They were marked as cultural heritage and so it was forbidden to knock them down and rebuild even though the school desperately needed more space. So some random people can prevent an extremely successful school from expanding because they've decided they like the building.

Under this proposal they would have to pay to stop that happening, and you'd suddenly discover they value the cultural heritage of the building far less than they claimed.

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T08:22:40.462Z · LW · GW

I would argue that if people across the world are not willing to pool enough money together to save the Colosseum, then for all their claims that culture is important to them, it clearly just really isn't. I'm reminded of

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T07:12:16.078Z · LW · GW

I also I expect in practice that kind of thing wouldn't be super common, because the tit for tat nature is such its just not really worth it for anyone to get into such a war. It also would look extremely bad for the company.

So I reckon this isn't going to be very common to start off with, and even some conservative laws to increase the risk of doing this would be enough to stop it occurring basically ever. Sure there'll be that one case everybody talks about, but it wont actually happen in practice.

Comment by Yair Halberstadt (yair-halberstadt) on Combining the best of Georgian and Harberger taxes · 2021-08-12T06:53:57.708Z · LW · GW

That's an interesting point I hadn't considered at all - malicious destruction. The obvious answer is to "make that illegal", which is a bit vague, but works well enough for things like insider trading.

An alternative would be to make a bid last a minimum length of time. So if I offer a ridiculous temporary price for the factory in order to destroy it, I'm going to have to pay that for an entire year, which will in most cases stop that being worth it.

I definitely need to think about that more!

Comment by Yair Halberstadt (yair-halberstadt) on For God's sake, Google it. · 2021-08-11T04:21:36.551Z · LW · GW

I generally agree with the sentiment, but in some cases I've found asking people for help is more effective since

a) they can give you specific assistance relative to your situation.

b) you can ask them follow up questions if you get stuck.

For example I live in an area where the developer built 50 identical houses. I'd much rather ask the neighbors how they did a DIY project than google, because I know their experience is directly relevant to me.

Comment by Yair Halberstadt (yair-halberstadt) on Who decides what should be worked on? · 2021-08-02T05:57:16.783Z · LW · GW

Whilst I mostly agree, I think this is overstated. 

For example at Google, whilst a lot of the technical decisions are made lower down, the strategic decisions about which markets to really focus on are made at much higher levels. Around a quarter of Googles workforce is in cloud - an area where they're still losing money! A decision to make that kind of investment in such a competitive market is driven by grand master plans at the C level, not by organic decision making further down.

At the scale of the startup, even the most fantastic product wont make money if it doesn't have a suitable target audience. The job of the high level employees at the startup is to identify which target audience makes the most economic sense to target, and tell the lower level employees to focus on the needs of those customers over the needs of others. They certainly shouldn't micromanage how this is done, but it's important that everyone on the company is clear on what the strategic focus is, and what's peripheral.

Comment by Yair Halberstadt (yair-halberstadt) on DeepMind: Generally capable agents emerge from open-ended play · 2021-07-27T16:36:35.478Z · LW · GW

They're also working with raw RGB input here too.

Comment by Yair Halberstadt (yair-halberstadt) on Is the argument that AI is an xrisk valid? · 2021-07-20T04:57:13.891Z · LW · GW

Without getting into the details of the paper, this seems to be contradicted by evidence from people.

Humans are clearly generally intelligent, and out of anyone else's control.

Human variability is obviously only a tiny fraction of the possible range of intelligences, given how closely related we all are both genetically and environmentally. Yet human goals have huge ranges, including plenty that include X-risk.


  • Kill all inferior races (Hitler)
  • Solve Fermat's last theorem (Andrew Wiles)
  • Enter Nirvana (A budhist)
  • Win the lottery at any cost (Danyal Hussein)

So I would be deeply suspicious of any claim that a general intelligence would be limited in the range of its potential goals. I would similarly be deeply suspicious of any claim that the goals wouldn't be stable, or at least stable enough to carry out - Andrew Wiles, Hitler, and the Budhist are all capable of carrying out their very different goals over long time periods. I would want to understand why the argument doesn't apply to them, but does to artificial intelligences.

Comment by Yair Halberstadt (yair-halberstadt) on Analyzing Punishment as Preventation · 2021-07-15T16:10:27.175Z · LW · GW

I think certainly all of these may have weak effects, but I think it's also clear that none of them significantly influence sentencing in most criminal justice systems.

Comment by Yair Halberstadt (yair-halberstadt) on Reasons for Punishment · 2021-07-14T11:59:42.770Z · LW · GW

Great point, will do.

Comment by Yair Halberstadt (yair-halberstadt) on Reasons for Punishment · 2021-07-13T03:32:28.293Z · LW · GW

Thanks for your suggestions. Do you see precommitment as different to my Discouragement? If so where would the o be apply but not the other?

Comment by Yair Halberstadt (yair-halberstadt) on How can there be a godless moral world ? · 2021-06-21T19:06:28.916Z · LW · GW

I think that the argument that morality can only exist with god depends on a logical fallacy.

The first step is to say that morality is defined as whatever god says. You then say that morality cannot exist without god, because the definition of morality is what god says.

Of course this argument is transparently meaningless. I could equally well say that morality is defined as whatever a monkey typing at a typewriter comes up with and make the same argument.

The fallacy only works because the word morality already has meaning. It means that gut feeling I have that something is good or bad. Then somebody comes along and uses that exact same word, but defines it differently - as what god says. Since they both happen to be reasonably well correlated I don't necessarily notice this switch. Then I'm convinced that morality is defined as what god says, and so there can't be morality without god.

If we were strict we would say: don't define what god does as moral. Make up a new word for it e.g. 'godral', and allow morality to keep it's classic meaning. At that point it becomes clear that we need to prove that 'godral' = 'moral', which is far from obvious.

The Euthyphro dilemma is a good search term to find a lot more on this topic.

Comment by Yair Halberstadt (yair-halberstadt) on Are bread crusts healthier? · 2021-06-21T13:55:04.964Z · LW · GW

It was a soft roll with a pretty soft crust. Personally I eat pretty much everything, crust included, but I still have this childhood association of crust as the not nice part of the bread even though I personally don't mind it (I would still say I prefer the inner parts of bread over the crust though).

Comment by Yair Halberstadt (yair-halberstadt) on Can someone help me understand the arrow of time? · 2021-06-15T18:05:16.982Z · LW · GW

I don't think this really addresses the substance of the question. I understand well the concept that we can imagine things which are illusions in general. I have specific mechanics level questions about how it applies to the psychological arrow of time.

Comment by Yair Halberstadt (yair-halberstadt) on Doomsday, Sampling Assumptions, and Bayes · 2021-06-15T03:13:52.301Z · LW · GW

That was meant to be the chance of P2, not P1. Fixed now, thanks!

Comment by Yair Halberstadt (yair-halberstadt) on Core Pathways of Aging · 2021-05-31T05:39:27.237Z · LW · GW

This seems relevant:

approximately 25% of the NMR genome was represented by transposon-derived repeats, which is lower than in other mammals (40% in human, 37% in mouse, and 35% in rat genomes)

However it's just a 1/3 or so reduction compared to similar mammals, so on its own that doesn't explain much. But it suggests a possible lead. 

Comment by Yair Halberstadt (yair-halberstadt) on Vaccine Rollout as Wheeled-Luggage Problem · 2021-05-16T11:00:24.685Z · LW · GW

An alternative hypothesis is this:

Producers can make enough vaccine for early trials, but don't manage to scale up to produce millions of doses until December. The FDA takes the opportunity to gather data on safety and efficacy until that time. However, they don't want to look like they're issuing emergency approval just because the vaccine's ready to go. They want it to look like it's because they have reached satisfaction with the safety and efficacy data. So that's the line they put out to the public. Yet in reality, they give approval right around the time when producers, distributors, and vaccination sites are ready to go.

If that was correct we'd expect that FDA approval process tends to vary by how quickly manufacturing capacity can be ramped up. The key criticism I have of the FDA is not allowing challenge trials. If you are correct and that was intentional to give time for capacity to ramp up, we would expect to see drugs which can be ramped up more quickly use challenge trials - but that's not something we've seen. If anything this was a ridiculously fast approval, for a process that usually takes years.

Furthermore I find it highly unlikely that producers would ramp up production just as quickly before results of trials are in as afterwards. So by not allowing challenge trials, less was invested in ramping up, so the ramp up itself (if it was a critical bottleneck) would have taken longer.

and we can more obviously see how politicians and manufacturers would both stand to benefit from an efficient rather than an inefficient process

The overwhelming evidence I've ever seen is that politicians and government orgs are highly inneficient. My prior on them being efficient here is extremely low.

Comment by Yair Halberstadt (yair-halberstadt) on Vaccine Rollout as Wheeled-Luggage Problem · 2021-05-16T04:42:37.795Z · LW · GW

I disagree with point 3.

Given that we can show governance was definitely a critical bottleneck (production proceeded as soon as governance allowed), why is the burden of proof on us to show that no other bottlenecks happened to also be critical. My prior would be that it is unlikely that 2 bottlenecks happened to be about the same time. Not crazy unlikely, but in the 20/30% range.

In other words, bad governance was definitely an issue and must be fixed. Maybe there's some other issues that also have to be fixed, in which case, sure, bring me some evidence of them and we'll work on solving them.

Comment by Yair Halberstadt (yair-halberstadt) on Vaccine Rollout as Wheeled-Luggage Problem · 2021-05-15T18:41:22.902Z · LW · GW

Sure anything's possible - I haven't seen any evidence that they did that, nor do I think they ever claimed they did that, nor does their behaviour match that strongly. E.g. they still haven't authorized AstraZenica for God knows what reason, but it's currently being produced at a very decent rate.

Comment by Yair Halberstadt (yair-halberstadt) on Vaccine Rollout as Wheeled-Luggage Problem · 2021-05-14T08:04:44.211Z · LW · GW

The Crawford Standard simply states that it's difficult to assert something could have been invented earlier but wasn't, it doesn't tell you why it couldn't have been invented.

The why could be lack of enabling technology, time taken to ramp up production, or regulation.

Given we know that regulation was the limiting factor here, there's no reason to assume other things were also a limiting factor - i.e. we know that vaccines were produced as soon as they were approved. What's the chance that all the other limitations (e.g. ramp up time) happened to take exactly (or close to) the same amount of time?

Comment by Yair Halberstadt (yair-halberstadt) on Vaccine Rollout as Wheeled-Luggage Problem · 2021-05-14T07:59:51.707Z · LW · GW

This doesn't seem right to me - if it takes 8 months minimum to turn over a factory to produce a new vaccine, how come there's a reasonable (if not high) rate of production within a few weeks of each new vaccine coming on board? Had factories already started turning over for each individual vaccine just in case 8 months ago?

Comment by Yair Halberstadt (yair-halberstadt) on What is the strongest argument you know for antirealism? · 2021-05-12T14:30:53.928Z · LW · GW

If you think nothing is "valuable in itself" / "objectively valuable", why do you think so?

I think that's the wrong way round. If you want to claim things have some property, then you have to put forward evidence they do. My strongest argument that things do not objectively have value is, why on earth would you think they do?

It's also clear that this discussion is fruitless. The only way to make progress will be to give some sort of definition for "objective value" at which point this will degenerate into an argument about semantics.

Comment by Yair Halberstadt (yair-halberstadt) on Challenge: know everything that the best go bot knows about go · 2021-05-11T16:50:15.170Z · LW · GW

To me a good definition for this is:

Get to a stage where you can write a computer program which can match the best AI at Go, where the program does no training (or equivalent) and you do no training (or equivalent) in the process of writing the software.

I.E. write a classical computer program that uses the techniques of the Neural Network based program to match it at Go.

Comment by Yair Halberstadt (yair-halberstadt) on Could MMRPGs be used to test economic theories? · 2021-05-07T03:08:47.900Z · LW · GW

Games which give players constant rewards tend to be more addictive, so there's an incentive to make it easy to create money easily, without producing a usable good for example.

Comment by Yair Halberstadt (yair-halberstadt) on Could MMRPGs be used to test economic theories? · 2021-05-06T15:07:47.063Z · LW · GW

Some great points here:

If one used a sufficiently addictive game, players would pay to play it.

My worry with that is that will remove the incentive to make it a good testbed for economic theories of that will make the game less addictive.

Comment by Yair Halberstadt (yair-halberstadt) on April 15, 2040 · 2021-05-05T12:49:36.789Z · LW · GW

"Ok, how about you sign it, and then I get a different assistant to help me with my taxes?"

"That won't work because in order to sign the agreement, I must sign and attach a copy of your tax return for this year."

Speculating about some of the technical details:

How could AI identity work? You can't use some hash on the AI because that would eliminate it's ability to learn. So how could you have identity across a commitment - i.e. this A.I. will have the same signature if and only if it has not been modified to break it's previous commitments.

Comment by Yair Halberstadt (yair-halberstadt) on Rising rents and appropriate responses · 2021-04-19T14:33:14.278Z · LW · GW

The density will grow, and since individual construction companies don't have an incentive to care about urban planning, just their own projects, you will get ugly neighborhoods and questionable infrastructure.

This is only true when construction companies are building small projects. If they're building large projects, then insofar as people value infrastructure, and will pay extra for it, the construction companies will have an incentive to build infrastructure.

Currently construction companies have an incentive to buy large plots of land, and build nice neighborhoods with good infrastructure as they can charge more per unit. So if that's not happening it's because it's too difficult to buy land.

In which case we need to increase land liquidity. 

Robin Hanson likes the idea of Harberger taxes to solve this: The idea is everyone needs to declare a selling price for their property. In order not to encourage inflated selling prices making the market illiquid, you get charged property tax based on your stated price.

An alternative might be the Georgist idea of taxing land value according to its full rent. This means that holding onto land is expensive and it's only profitable to hold onto it if you can use it more valuably than the next guy - there's no money to be made speculating on it. This should help both increase liquidity, and reduce land prices. I enjoyed the recent book review on this at

Comment by Yair Halberstadt (yair-halberstadt) on Don't encourage prisoners dilemmas · 2021-02-16T17:21:43.270Z · LW · GW

At least in the US, donations to political parties, political campaigns, political action committees, etc, are already not tax deductable (which I take is what you mean by "tax back").

Turns out it's the same in the UK. That's embarrassing! However I was just as much talking about political charities which aren't technically a political party.

My initial reaction is that I don't want that result, I want both organizations out there making their cases stronger, not weaker. But I could be talked out of this.

I think I agree with this insofar as political charities tend to work by disseminating the strongest argument for their case, and letting the correctest side win out. I think in practice that's not what they're doing - it's more about how they can use the political system to achieve their aims, at which point I think it's back to a prisoners dilemma.

I'm also not sure why the Against Unicorns Foundation wouldn't be a legally valid charity. It might not be a very popular one, but an argument can certainly be made that destroying unicorns prevents them from suffering, and preventing animal suffering is generally regarded as good and specifically listed as a charitable purpose in section 501c3.

As I said, I think the thing to do is look at where the money was actually spent. If it was spent protecting a unicorn conservation area, I'm pretty certain destroying a unicorn conservation area would not be a valid charity.

However I think you make some great points! Definitely have to think about them.