Comment by Daniel V on Why has the replication crisis affected RCT-studies but not observational studies? · 2021-09-04T01:42:42.407Z · LW · GW

One reason is that experiments are designed to be repeatable. So we repeat them or versions of them. Observational studies are really just sophisticated analyses of something that has happened, so one may more successfully appeal to "this time/context is different."

Another reason is that a lot of times observational studies have proprietary/secret data. Publicly-available datasets are less susceptible to this excuse but sometimes those are paired with proprietary data to develop the research.

I was literally in a seminar this week where Person A was explaining their analysis of proprietary data and found a substitution effect. Person B said they had their own paper with a different proprietary dataset and found a complementarity effect. Without fraud, with perfectly reproducible code, and with utter robustness to alternative specifications, these effects could still coexist. But good luck testing robustness beyond the robustness tests done by the original author(s), good luck testing if the analysis is reproducible, and good luck testing whether any of the data is fraudulent...without the data.

I'm not even advocating for open data, just explaining that the trade secret and "uniqueness" differential make observational studies less conducive to that kind of scrutiny. For what it's worth, reviewers/authors tend to demand/provide more specification robustness tests for observational than experimental data. Some of that has to do with the judgment calls inherent in trying to handle endogeneity issues in observational research (experiments can appeal to random assignment) and covering bases with respect to those "so many more [researcher] degrees of freedom."

A third reason is simply that the crisis struck effects tested with low-powered studies. Because of implementing a statistical significance filter, the published effects were therefore more likely to be overestimates of the true effects. The N of observational studies mitigates this source of overestimation but runs the risk of finding practically-trivial effects (so sometimes you'll see justifications for how the effect size is actually practically meaningful).

Comment by Daniel V on The Death of Behavioral Economics · 2021-08-24T13:27:57.896Z · LW · GW

Academic here, it's (1). Loss aversion is so popular that people think it underpins everything. Although loss aversion doesn't show up in every dataset, it does show up ( - even the "second paper" shared by Kaj just says it appears sometimes. But does that mean it explains all these other findings? No! But some reviewers or authors think "isn't that just loss aversion?" and it seems authors take the easy route to publication (or just aren't well-read enough) instead of probing the psychological source of their findings more seriously. For example, loss aversion was the classic explanation for the endowment effect, but research in the last couple decades has generated results that loss aversion cannot really explain and that other theories readily explain, yet LA is sometimes still cited as the explanation the authors endorse.

Comment by Daniel V on Traps of Formalization in Deconfusion · 2021-08-06T02:36:58.680Z · LW · GW

Andrew Gelman distinguishes these. He believes in symbolic formalization but doesn't get Judea Pearl's DAGs. I personally find visualization to be very useful, not always in DAGs, sometimes simply in expected patterns. I also have found equational models to be useful on different occasions. Other times I find analytical models extremely difficult to follow because the explication is too dense. They are certainly distinct types of formalization, but none is free of the potential risks of non-deconfusing that adamShimi lays out.

Comment by Daniel V on Do we have a term for the issue with quantifying policy effect Scott Alexander stumbled on multiple times? · 2021-07-29T15:40:24.496Z · LW · GW

johnswentworth makes the great point that "some effects tend to dwarf all others, so it is critical to catch every last one" assumes that we can't identify the big effects early. If people are looking around with open eyes, they're not so unable to pick up the relevant stuff first. 

What yhoiseth's framing gets right is that big effects are sometimes not salient, even for people with open eyes. And especially when effects are hard to directly observe or estimate with certainty because they're indirect in nature (like substitution effects), not only are they low on salience for affective reasons, they're low on salience because they don't benefit from a "big is relevant" heuristic (or "open eyes") since the effect size is unknown. That is, rather than effect size and salience having a positive correlation...because effects are often not known with certainty, even among people with open eyes, effect size and salience can have a negative correlation, necessitating getting to the bottom of the salience barrel to identify the big ones. I am unsure how the relative frequencies of "open eyes will see big effects" vs. "open eyes can still struggle to see big effects" compare though.

For example, Scott mentions side effects, addictiveness, pain relief, War on Drugs, high driving, and less drunk driving as effects. The idea is that the first four are rather small effects and the last two are rather large effects, but the error bars are small on the first four and huge on the last two. The first four are also highly salient (obvious, personalized, affectively-charged) despite being small, and the last two are not as salient despite being large (they would be salient if we knew their effect sizes; effect size matters for salience. But because we lack precision, we're left with how salient these are on the basis of being non-obvious, de-personalized, and coldly econometric). If you were to plot these with effect size on X and salience on Y, you'd get a negative correlation if you were omniscient and able to include the last two effects in your dataset (per yhoiseth). But for Scott and typical discussants, the last two effects are missing data so you're left with the weak positive correlation among the first four effects. At least until someone annoyingly but helpfully tells you it is time to go looking. But again, johnswentworth is also right that the actual correlation is frequently positive, so this isn't always a problem.

Aside: it also, technically, depends on what counts as "open eyes." I figure Scott and friends are pretty dialed in, so I take their "missing" the big effects as evidence for "open eyes can still struggle to see big effects." But I suppose an economist whose spent their career studying substitution effects might think Scott et al. were approaching the problem with blindfolds on, duh, substitution is definitely the biggie here.

Comment by Daniel V on Delta Strain: Fact Dump and Some Policy Takeaways · 2021-07-28T14:53:56.602Z · LW · GW

We do have some evidence about which world we’re in. There are studies which find pretty big differences in level of antibody titer produced by the vaccinated, and in some cases where they have almost no antibodies it’s pretty clear that this means immune responsiveness is going to be at fault when they get sick. And I think there are studies finding correlation between titer and effectiveness. Both of these point toward innateness. But we also know that it has to be true that for many of those with low levels of antibodies, a larger dose will push them over the edge. There is also slight evidence from the Israel numbers, which give effectivenesses that vary some over time, that there’s a serious behavioral/environmental component. 


So, we're in both worlds. VE is a function both of immune response and viral load exposure. Which one is relatively dominant may be important for behavioral implications (I agree with you!), but this doesn't have to be an either or. "Breakthrough" cases can have multiple input factors. Even the "innate" world comes with the question of whether the vaccine stochastically increases titers across the board or stochastically increases titers only among a susceptible type of person (is it a single distribution or a mixture distribution?). But once we think we're living in a world where both matter (actually, I wonder to what extent this community endorses this POV or if generally LW thinks it is an either or situation?), and once we obtain a ton more info, the behavioral recommendations can be either really complicated and theoretically optimal but impossible to follow, or they can be simpler and sub-optimal but implementable. We see this with vaccinateds masking - the distribution of titers pretty much indicates vaccinateds will take care of wild type just fine, so the CDC cuts the mask recommendations (despite there being some variation). The titers vs. Delta are not quite so great, so the CDC re-implements the mask recommendations (despite there being some variation). The world is messy. 

Comment by Daniel V on What does knowing the heritability of a trait tell me in practice? · 2021-07-26T23:09:20.717Z · LW · GW

Heritability is an explanation. It's R-squared. As useful as "percent of variance explained" can be in some situations, and as useless in others.

Comment by Daniel V on We Still Don't Know If Masks Work · 2021-07-05T15:15:15.004Z · LW · GW

This analysis of the data leads to a seeming contradiction. Within a given region, increased mask usage is correlated with lower growth rates (the 25% claimed effectiveness), but when comparing across regions masks seem to be ineffective. Depending on the causal story you want to tell, either of these claims could be true.

It is possible that people wear masks most in places where COVID is most transmissible. This would explain why masks don't appear effective when comparing across regions.

However it is also possible that the same factors causes both mask wearing to increase and transmissibility to decrease. For instance, if people wear masks in response to an observed spike in cases, then the population immunity caused by the spike will make masks appear to be effective even if are not.


This is an excellent element of this post, and I am very pleased to see it! These two possibilities are actually testable in a model that statistically controls (i.e., includes) transmissibility. Notice that in the first possibility, mask wearing is a (partial) mediator for the effect of transmissibility on cases. In the second, mask wearing and transmissibility are commonly-caused by something else(s). Either way, controlling the transmissibility path improves our estimate of the mask wearing path.
[Aside: what if your statistical control was "insufficient" and mask wearing still correlates with the error term in your model? Then your causal estimate loses some causal interpretability, but this critique is virtually impossible to completely address short of randomized experimentation. Still, you try your best and transmissibility here is the biggie that should account not just for itself but also soak up any common causes you might expect to be in the error term.]

This is why beginning-of-period transmissibility factors are included in the model to estimate end-of-period transmissibility results. It's possible that mask wearing has a negative effect on cases but that it's more than offset by the big positive direct effect of transmissibility on cases. If you're interested in the correlation between mask wearing and cases, you'll be disappointed by the net positive effect, which is of course not interpretable as causal. If you're interested in the causal effect of mask wearing on cases, you'll be encouraged by the negative effect and hope to find ways to increase mask wearing besides the epidemic just getting so much worse that people start to wear masks. This is the first possibility. This is also how the second possibility looks statistically - we can still get our estimate of the effect of mask wearing, which ultimately is the focus of the investigation. But whether the first or second possibility is "true" may be relevant not so much for estimating the effect of mask wearing (it's statistically the same - control for transmissibility), but for getting a wider understanding of the world (does transmissibility cause mask wearing or does something else cause both?, which is not the focus of the study).

I also want to note that the endogeneity critiques of observational (vs. experimental) methods are legit, but there is a lot that can be (and is) done to draw "more causal" conclusions from observational data than mere correlation, and experiments can have their own internal validity concerns, so both approaches are useful for learning about the world.

Comment by Daniel V on Internal Information Cascades · 2021-06-26T13:22:15.513Z · LW · GW

This reminds me of the garden of forking paths a la Andrew Gelman. Good post with helpful suggestions.

Comment by Daniel V on The Point of Trade · 2021-06-24T14:38:21.267Z · LW · GW

I agree with you that it's hard to understand and that a lot of times people use the term inexactly to mean something else, but neither of these is a reason to avoid using the term, especially in the exact context where it is easiest to illustrate and can be used correctly.


Comment by Daniel V on The Point of Trade · 2021-06-23T11:55:57.195Z · LW · GW

All that just to get to the point of trade being to leverage comparative advantage into unlocking more value in the economy, which is the actual textbook reason for trade. Don't get me wrong, this was an enjoyable read and illustrates that heterogeneous preferences is an appealing but inadequate answer to why we trade. The post also gets at various reasons comparative advantage might come about (nice), but without actually naming the broad concept that umbrellas these things together to explain the benefit of trade. Comparative advantage, whether it exists because you (or the organization, or country, or whatever entity is the unit of analysis) have task-switching costs, are closer to the means of production, are inherently better at the task, have accumulated more experience with the task, or (looking to the comments now) have accumulated more capital devoted to the task via path dependencies, is the reason for trade. The point of trade is to unlock the added value that resides in making more efficient use of scarce resources.

Comment by Daniel V on Alcohol, health, and the ruthless logic of the Asian flush · 2021-06-16T00:46:49.912Z · LW · GW

I was joking ;) But the distinction between prophylaxis and treatment I think is useful because even if "it doesn't work" as one or both, it could work for the other and still be helpful.

Comment by Daniel V on Often, enemies really are innately evil. · 2021-06-07T15:32:19.326Z · LW · GW

Evil is when you "know that some things are damaging to someone else, gain no tangible value from doing them (or even expect that their life would be worse off!), know it is not a virtuous act, and do the harmful acts anyway without expecting future good to come from it."

The first failure point is "gain no tangible value." Imagine any prototypically evil character, maybe a person who is bullying once, maybe a chronic bully, maybe Dr. Evil, maybe Satan. Each of these gains some subjective value from their actions, if not "tangible" value. Either "tangible" is critical here, in which case you have way too narrow a definition of value, or it's not, in which case it is clear that these people are selfish and pretty legible.

What makes them evil is that their value system is so out of whack that they are evil (please just live with the circularity for now, I'm not trying to propose that as a formal definition). So the person who is bullying once and then learning it doesn't fulfill them that much...they may have done a bad, or even evil, thing, but they aren't evil! Same of the chronic bully - if they had a bad home life and are coping poorly, their innate value system may still be programmable to avoid evil acts. Dr. Evil is much closer to chronic evil right up until Goldmember (I can't believe I'm really going with this), when we find out he is a victim of circumstance, which anyone seemingly can be with enough compassion. Satan, well yeah, he's evil.

No one likes or endorses bullying, but you need a definition of evil that has validity, and yours is debatable. But even if accepted, it hardly encompasses a lot of people. You could actually stand to loosen the definition of evil, but you quickly run into selfishness. Construct definition is step one here. And it'll probably carry value judgments (see: virtue).

Comment by Daniel V on Alcohol, health, and the ruthless logic of the Asian flush · 2021-06-04T19:44:22.508Z · LW · GW

Your discussion would suggest that disulfiram might not work at curing alcoholism but could be a useful prophylactic. Lace the drinking water with it and people will avoid alcohol or stop earlier! What could go wrong?

Comment by Daniel V on Summaries of uncertain priors · 2021-06-03T03:14:38.349Z · LW · GW

Aleatory and epistemic uncertainty often get wrapped up together so these estimates are not always proper probabilities nor measuresof confidence. You're separating them, good for you!

Comment by Daniel V on Let's Rename Ourselves The "Metacognitive Movement" · 2021-04-24T00:35:54.687Z · LW · GW

There once was a time when people who were obsessed with knowledge (and the appropriate action flowing therefrom) were called scientists. Now they are just adherents to scientism, and the rest of us have to pick up terms to describe our taking the mantle. Where the "rationalist" community seems to be is at the intersection of metacognition and "rational" (whatever that means :P ). Neither describes the movement entirely on its own, but with their powers combined...

Interesting post, thanks.

Comment by Daniel V on Rising rents and appropriate responses · 2021-04-19T01:45:24.255Z · LW · GW

Basic standard microeconomics (supply and demand) is a pretty strong model, so you're doing great! 

What you're missing is formalizing the value or disvalue being pursued or created by the system.
Right up until "If you do literally nothing at all," the discussion was about prices and quantity, but then suddenly we care about aesthetics and infrastructure. Did you know that people would also pay for that, too? This might lead to things like some neighborhoods being more valuable than others and accordingly commanding higher prices for otherwise similar accommodations.

If a lot of people want to move to the city because the opportunity is so vast and they aren't as concerned about aesthetics, developers would develop accordingly. If instead many of these people are pickier, well, developers would be too. This sounds bad because that means we can't guarantee other people live according to our preferences, but it's actually good because it's demand and supply meeting up. Where things go awry is when these market exchanges create externalities that should be internalized by the market participants. If bare wires a strewn across the streets and children are being electrocuted every day, maybe we need a government to enforce some basic regulatory code to take care of that (because in this hypothetical, I guess the neighborhood is populated by selfish singles and the children come from elsewhere to play in these oh so attractive streets, so the problem won't get fixed otherwise).

Yes, inventing a generic government can cover the really bad results (if they occur) from this market arrangement. The risk with this is that people may then seek to enforce their preferences through this government rather than letting the market handle it. That might be fine. Or it might be inefficient, maybe even unjust. "I think apartment complexes should have at least a one-car garage or two parking spaces per unit; I'm also super benevolent so developers can mix and match" leads to an absurd result when the would-be tenants just grin and bear it despite their preference for taking the available public transit or use their bikes. It just becomes a value-suck, raising prices and/or lowering supply, achieving one (foolish) objective to the neglect of the many (important) others.

I come from a mountain town - space is scarce. The government decided it would be more efficient, kinda neat in town, and better for tax revenue to implement onerous housing regulations but exempt mixed-use (residential on top, commercial on bottom, and you know it, parking in the back) from some (not all) of those regs. We got a lot more mixed use. The housing filled up since we had a shortage already. The commercial did not, wasting resources and space. This also cratered commercial rent prices, but the new building owners don't seem to cry about it. Turns out the developers were building residential space; commercial rent would just be gravy since the commercial space was just to get the desired regulatory structure applied. That's how valuable the residential space was.

I can tell you what experts aren't disagreeing on.

Comment by Daniel V on Coincidences are Improbable · 2021-02-24T19:48:03.349Z · LW · GW

Oh yeah, definitely agree!

Comment by Daniel V on Coincidences are Improbable · 2021-02-24T19:09:51.740Z · LW · GW

The two "direct" causal links are the only ones we would really call "causal" regarding A and B.

But I am a big fan of "correlation implies causation." It might not be between A and B specifically, but it means we've been able to detect something happening.

Sometimes even non-effects, when theory is strong enough, can indicate causation (though then the usual course of action is to control one of the paths to get an effect that you can talk about and publish). For example, you are about to eat an allergen, which you know causes side effects for you with p=1. You take Benadryl beforehand and have no side effects. There is no "effect" there (post state = pre state), but you can feel pretty sure Benadryl had a suppressing action on the allergen's effects (and then you would follow-up with experiments where you ate the allergen without Benadryl or took the Benadryl without eating the allergen to see the positive and negative effects separately).

Comment by Daniel V on The Median is Less than the Average · 2021-02-15T16:20:44.399Z · LW · GW

OP's claim is that intelligence is positively skewed. Counter-points are "most brains are slightly worse"  (Donald Hobson) and "you oversample the high-intelligence people, so your claim is biased because of availability" (Ericf).

Both of these counter-points agree with, rather than disagree with, lsusr's point. Most brains are slightly worse implies positive skew and to the extent that lsusr oversamples high-intelligence people, they are underestimating how positively skewed intelligence is yet still conclude it is positively skewed (caveat: as Donald Hobson says, the measurement approach can be really important here, but for the sake of argument let's say lsusr is talking about latent intelligence, and our measures just need to catch up with the theory).

Ericf also makes another interesting point- "variation in low intelligence is less identifiable than variation in high intelligence," 160 vs. 130 IQ people will act differently, but 40 vs. 70 IQ people won't so much, or at least the IQ test is better at delineating on the high end than low end. I am no expert on the measurement of intelligence, but this point probably shouldn't just be taken at face value- for example, individuals with Down's syndrome consistently have IQs less than 70 and getting below 70 is rare, as expected since IQ is designed to be Gaussian. But the implication of that is that as rare (and therefore difficult to dig into) as low IQs are, high IQs are...equally rare (and therefore difficult to dig into).

I agree that OP's claim should also be subjected to scrutiny -simply saying intelligence is positively skewed doesn't make it so- but I also don't find the present set of counter-points either that contradictory or that convincing either. Just my two cents.


Comment by Daniel V on Build Your Number Sense · 2021-01-27T21:54:53.600Z · LW · GW

FWIW, number sense is definitely a thing in psychology.

Comment by Daniel V on How likely is it that SARS-CoV-2 originated in a laboratory? · 2021-01-25T22:48:49.835Z · LW · GW
  1. Escaped/circulated earlier than officially reported.
  2. False positives.
Comment by Daniel V on Covid 1/14: To Launch a Thousand Shipments · 2021-01-15T18:29:29.386Z · LW · GW

FDA Dr. Peter Marks's reply either indicates his own misunderstanding or that something is wrong with the FDA report! In Table 15 of the FDA's Moderna report, they report efficacy "in Participants Who Only Received One Dose" (emphasis added and the N's are correctly not the full trial's N). 80% (95% CI: 55%, 93%) is a nice round number to tell people, but also we assess two-dose efficacy only after 14 days anyway, so the truly comparable number is 92% (95% CI: 69%, 99%). 

Now if there are other reasons we shouldn't trust those numbers, I'd love to see them. They caveat it with it's not necessarily 80+% effective forever since they only observed single-dosers for a median of 28 days, and the N is definitely lower but still 1000 per group (which is why the confidence intervals are wide). But that gives us pretty high confidence that 14 days after the first dose, the vaccine is effective enough to warrant JABS IN ARMS!

Comment by Daniel V on Covid 1/14: To Launch a Thousand Shipments · 2021-01-15T18:12:35.825Z · LW · GW

It's between-subjects, these aren't real probabilities for individuals. But from a Bayesian standpoint it gives you useful base rates with which to assess risk.

Comment by Daniel V on Collider bias as a cognitive blindspot? · 2020-12-30T17:54:36.941Z · LW · GW

One way to "rewire" your brain is to wire in a quick check- how does selection/stratification/conditioning matter here?

But perhaps most important is to think causally. Sure, you can open up associations, but, theoretically, do they make sense? Why would obesity, conditional on having cardiovascular disease, reduce mortality? Addressing why rather than leaping to a bivariate causal conclusion is important. This is why scientists look for mechanisms and mechanism-implicating boundary conditions.

Comment by Daniel V on Covid 12/17: The First Dose · 2020-12-17T20:45:44.850Z · LW · GW

I'm having trouble with it too and I think Zvi misinterpreted it as well- the far right column is the VE.

Comment by Daniel V on Beware Experiments Without Evaluation · 2020-11-15T17:42:27.594Z · LW · GW

Indeed, these aren't controlled experiments at all, but sometimes they are also not policy-sneaking. Sometimes they are just using the phrase "experimenting with" in place of "trying out" to frame policy-implementation. At that point, the decision has already been made to try (not necessarily to assess whether trying is a good idea, it's already been endorsed as such), and presumably the conditions for going back to the original version are: 1) It leads to obviously-bad results on the criteria "management" was looking at to motivate the change in the first place or 2) It leads to complaints among the underlings.

The degree of skepticism, then, really just depends on your prior for whether the change will be effective, just like anything else. Whether there should have been more robust discussion depends either on the polarity of those priors (imagine a boardroom where someone raises the change and no one really objects vs. one where another person suggests forming an exploratory committee to discuss it further), or on whether you believe more people should have been included in the discussion ("you changed the bull pen without asking any of the bulls?!"). It has little to do with the fact that it was labeled an experiment, since again, it's likely being used as business-speak rather than as a premeditated ploy. I would love to have data on that though- do people who specifically refer to experimentation when they could just use a simpler word tend to use it innocuously or in a sneaky way?

Comment by Daniel V on The US Already Has A Wealth Tax · 2020-08-21T15:38:45.103Z · LW · GW

^Not always true, but true often enough that it definitely bears mentioning.

If you invest in an asset that you expect to have a 0% real return and therefore hand you an after-tax real loss, and then you complain the tax system is handing you an after-tax real loss, there's something wrong there- is it with the tax system?

Comment by Daniel V on Criticism of some popular LW articles · 2020-07-20T15:00:04.914Z · LW · GW

To the contrary, I think the criticism of post 2 is very on point. But Zvi and I are looking at two different parts: Zvi's looking at the logic/begging the question part, and I'm looking at the critique. In thought experiments, we can take imagined exogenous changes to be exogenous even though in the real world they'd be endogenous (i.e., we can take them as events rather than outcomes). Later, we can relax that assumption; the endogeneity problem is important for understanding whether the conclusions extend to the real world, but it is not important for understanding what the conclusions are within the thought experiment. So I agree with Zvi that the logic isn't really an issue here.

However, I do believe this is a bad example (/weak post, Sorry Elizabeth) precisely for the reason AllAmericanBreakfast pointed out- it frames basic economics knowledge as a new insight. Admittedly, the EconLog post that was linked to doesn't discuss comparative advantage either, but that's because it's really just about the "flight to safety" in 2008 where capital has to go somewhere, so it goes to the safest haven- even if that place is on fire, at least it's not on fire next to a ticking time bomb. But, if you really want to talk about the "benefit not from absolute skill or value at a thing, but by being better at it than anyone else" then you can just consult microeconomics 101 (literally) and read up on absolute vs. comparative advantage. And then a better example of it is what you would find in the textbook (ha, probably Mankiw's) of English cloth vs. Portuguese wine, which clearly illustrates the concepts.

Or, maybe Elizabeth really wasn't referring to comparative advantage and more specifically to "when a superlative is applied in a context and the context is later lost." This might seemingly apply better to the USD (we think of it as a safe haven because we used to think of it as a safe haven), but again the USD is not an apt example here because the context isn't lost, it just changed (e.g., suppose the USD scores a 10/10 at being a currency and things change and now it's a terrible 3/10 but it's still better than all the rest). The Tallest Pygmy derives its tension from that fact that you think you've found someone "tall" but it's just among the pygmies you're sampling. The Tallest Pygmy, then, is best understood as getting stuck in a valley at a local, but not global, minimum (gradient descent). Or peaking at a local, but not global, maximum. Sometimes you are fine with local maxima, but if you are optimizing for global maxima, then obviously this creates a problem. May as well go with a classic example instead, which clearly illustrates sampling bias (statistics).

You see this in the academic literature as well where people refer to concepts as "effects." I think it is a good idea to be skeptical of those findings- not that they are fake, just that more clarity could be gained from understanding the core concept that generates the effect. Elizabeth's example is not great for comparative advantage, nor for gradient descent/sampling bias. The USD in 2008 is a "lesser of two evils effect," or really not an effect at all- if you have a choice between 10%, 9%, and 8% returns at equal risk, you choose 10%; if a regime change occurs that makes you choose between 5%, 4.5%, and 4%, you choose 5%. It's worse than before, but it's the best around.

LessWrong is a great community to be in, but AllAmericanBreakfast is correct that many posts stumble upon "new" insights that are really just symptomatic of not having done enough research, particularly when it comes to economics. And that's okay in this forum, we're all trying to figure this stuff out!

Comment by Daniel V on Can Covid-19 spread by surface transmission? · 2020-06-11T00:28:23.665Z · LW · GW

It can *survive* on surfaces for a long enough period of time for it to seem possible. But I think viral load is important and it might not practically be a serious vector for most people and most surfaces, particularly those that are commonly disinfected. It's amazing how our knowledge went from symptomatic/surface? to pre-symptomatic/aerosol? transmission.

Comment by Daniel V on How to validate research ideas? · 2020-06-05T13:15:15.770Z · LW · GW

Also a PhD here - read, read, read. You need to know what's been done to see what the gaps are and how your project would fit in. You will also build up that intuition.

Sure, it's also helpful to be able to bounce ideas around your network, but the less well-formed the idea is, the more likely it is to go to friends who aren't just going to shoot you down or for it to get the benefit of the doubt as "early-stage." You need to get the idea formed to the point where someone can feel comfortable pointing out issues, which will take independent research. You also see that here at LW, where ideas/points are usually more than a paragraph long.

Comment by Daniel V on How do you use face masks? · 2020-02-13T17:29:41.012Z · LW · GW

Here's a broader article with some pointers on masks.

Comment by Daniel V on Suspiciously balanced evidence · 2020-02-12T23:28:20.947Z · LW · GW

Good explanation #3- we perceive probabilities differently from their objective values (i.e., a question of calibration). Our responses to questions will be both a function of our "underlying" subjective probabilities and also the mapping of that to the response format. In the link, for example, responding with (p) 10% to 90% feels like being from (w(p)) 20% to 70% sure.

Comment by Daniel V on Using vector fields to visualise preferences and make them consistent · 2020-01-29T16:41:15.045Z · LW · GW

Charlie Steiner, right, it's not doable for, say, all products on (or could be on) the market, but it is certainly doable among the products in a person's consideration set. If we posit that they would make a choice among 4, then eliciting binary preferences might - but also might not - faithfully reflect how preferences look in the 4 set. So to MichaelA's point, if preferences are context-dependent, then you need to identify appropriate contexts, or reasonable situations.

Context-dependent preferences present a big problem because "true" context-less preferences...maybe don't exist. At the very least, we can make sure we're eliciting preferences in an ecologically-valid way.

Binary choices are useful, but when they lead to inconsistencies, one should wonder whether it's because preferences are inconsistent or whether it's an elicitation thing. If people really would choose between A and B and not consider C or D, then ranking A and B is the relevant question. If people would consider A, B, C, and D (or at least pick between A and B in the context of C and D) then ranking all four (or at least ranking A and B in the context of C and D) is the relevant question.

Comment by Daniel V on Using vector fields to visualise preferences and make them consistent · 2020-01-28T22:09:39.044Z · LW · GW

Very neat post.

Intransitive preferences can be found from a series of binary choices, but if you force a ranking among the full set, you won't have intransitive preferences (i.e., you can write out a gradient). This also means the elicitation procedure affects your inferences about the vectors. It would seem that circular preferences "fit," but really they could just be fitting the (preferences | elicitation method) rather than "unconditional" ("core" + "irrationality," whatever irrationality means) preferences. Preferences are also not independent of "irrelevant" alternatives as perceived attribute levels are evaluated contextually (that's necessarily irrational?).

One implication I see here is that 0 vectors are points with no inclination to switch or having "no desire." These would be useful model falsification points (e.g., Figure 7 implies that people don't care about sportiness at all conditional on weight being "right"). But they would also only seem to correspond to ideal points or "ideal configuration" points. Without data on what the agent wants and only on what they are being offered ("I want a sporty car, but not too sporty; Car A is closest, but still not quite right, too bad"), you'll be fitting the wrong hill to run up.

Comment by Daniel V on Applications of Economic Models to Physiology? · 2019-12-11T17:50:00.136Z · LW · GW

Related: Fungus arbitrage

What is the difference between a generic "signal" and a "price signal"? What is a "price" in physiology? I think it would be interesting to see what insights an economic perspective of physiology would provide, but the constructs need to be defined pretty clearly so analogies can be drawn.

Another question is which basic assumptions embraced in economics can reasonably apply to the units of analysis in physiology (cells, etc.). Economists already have a hard enough time validating assumptions for humans.

Comment by Daniel V on Randomness vs. Ignorance · 2019-11-08T02:53:21.093Z · LW · GW

This is aleatory (inherent randomness) vs. epistemic (knowledge) uncertainty. You can parse this as uncertainty inherent in the parameters vs. uncertainty inherent in your estimates of the parameters / the parameterization of the model.

This is a very important distinction that has received treatment in the prediction literature but, indeed, is not applied enough to interpreting others' predictions among laypeople.