Open thread, December 7-13, 2015

I asked Steve Hsu (an expert) "How long do you think it will probably take for someone to create babies who will grow up to be significantly smarter than any non-genetically engineered human has ever been? Is the answer closer to 10 or 30 years?"

He said it might be technologically possible in 10 years but " who will have the guts to try it? There could easily be a decade or two lag between when it first becomes possible and when it is actually attempted."

In, say, five years someone should start a transhumanist dating service that matches people who want to genetically enhance the intelligence of their future children. Although this is certainly risky, my view is that the Fermi paradox implies we are in great danger and so should take the chance to increase the odds that we figure out a way through the great filter.

Gwern has written an article for Wired, allegedly revealing the true identity of Satoshi Nakamoto:

http://www.wired.com/2015/12/bitcoins-creator-satoshi-nakamoto-is-probably-this-unknown-australian-genius/

Bug report: the antikibitzer's toggle button (which appears at the top right of the browser window's content area) doesn't work correctly for me (on recent Firefox on Windows) because the loop that attempts to identify the antikibitzer stylesheet fails. It fails because an earlier stylesheet in the list (actually, the very first) has a null href.

A simple fix is to change the obvious line in antikibitzer.js to this:

if (document.styleSheets[i].href && document.styleSheets[i].href.indexOf("antikibitzer") > 0)

but I make no guarantee that this is the fix the author of the code would prefer.

Some uncomfortable questions I've asked myself lately:

• Could you without intentionally listening to music for 30 days?

• I recall being taught to argue towards the predetermined point of view in schools and extra-curriculum activities like debating. Is that counterproductive or suboptimal?

• Listening back to a recording I made of a therapy session when I was quite mentally ill, I feel amazed at just how much I have improved. I am appalled by the mode of thought of that young person. What impression do the people around me have that they won't discuss openly?

• Aren't storm water drain explorer's potentially mapping out critical infrastructure which may be targetted more easily by terrorists? One way I see these things going is commercial drain tours. That way there would be a legitimised presence there and perhaps enhanced security.

• something to be asked of academia

• Imagine a person was abused for a large part of their childhood and is subsequently traumatised and mentally ill, then, upon regaining greater functioning as an adult decides to extort their abusive parents for money with the threat of exposing them while still counting on inheritence, instead of simply going to the authorities and approaching a legal settlement (expecting that will cut of any pleasant relations). Are there actions unconscionable? What would you do in their situation?

• If I went straight to a family member without preparing them in advance would they consent to my cryonics application? to support a cryonics application?

• Do most people really think like this?

• The rate at which I come up with ideas that I feel are worthwhile business ventures is unmanageable. So, I’ll take a leaf out of the EA Ventures method webpage by asking: what are three existing organizations that are doing similar things and why aren’t you joining them?

From Omnilibrium:

• Should fundamental science be funded by international agencies?

• Integration of Muslim Immigrants: US vs. Europe

• What's more important Laffer Curve or Income Inequality?

• Should the US radically increase spending on Voice of America?

Paradox at the heart of mathematics makes physics problem unanswerable

Gödel’s incompleteness theorems are connected to unsolvable calculations in quantum physics.

Undecidability of the Spectral Gap (full version) by Toby Cubitt, David Perez-Garcia, Michael M. Wolf

We show that the spectral gap problem is undecidable. Specifically, we construct families of translationally-invariant, nearest-neighbour Hamiltonians on a 2D square lattice of d-level quantum systems (d constant), for which determining whether the system is gapped or gapless is an undecidable problem. This is true even with the promise that each Hamiltonian is either gapped or gapless in the strongest sense: it is promised to either have continuous spectrum above the ground state in the thermodynamic limit, or its spectral gap is lower-bounded by a constant in the thermodynamic limit. Moreover, this constant can be taken equal to the local interaction strength of the Hamiltonian. This implies that it is logically impossible to say in general whether a quantum many-body model is gapped or gapless. Our results imply that for any consistent, recursive axiomatisation of mathematics, there exist specific Hamiltonians for which the presence or absence of a spectral gap is independent of the axioms. These results have a number of important implications for condensed matter and many-body quantum theory.

‘Outsiders’ Crack 50-Year-Old Math Problem

Extreme Self-Tracking

Man has himself MRIed twice a week for a year and a half, plus tracking a lot about his life. The data mining is still going on, but at least it's been shown that (probably) people's connectomes change pretty rapidly.

I'm also posting this to the media thread because I'm not sure where it's more likely to be seen.

Interesting article on vox (not a new one, but it's the first time I've seen it and I thought I'd share; apologies if it's been featured here before) on 'how politics makes us stupid': http://www.vox.com/2014/4/6/5556462/brain-dead-how-politics-makes-us-stupid

tl;dr: The smarter you are, the less likely you are to change your mind on certain issues when presented with new information, even when the new information is very clearly, simply, and unambiguously against your point of view.

World's first anti-ageing drug could see humans live to 120

Anyone know anything about this?

The drug is metformin, currently used for Type 2 diabetes.

Please, not another bias! An evolutionary take on behavioural economics by Jason Collins

So, I want to take you to a Wikipedia page that I first saw when someone tweeted that they had found “the best page on the internet”. The “List of cognitive biases” was up to 165 entries on the day I took this snapshot, and it contains most of your behavioural science favourites … the availability heuristic, confirmation bias, the decoy effect – a favourite of marketers, the endowment effect and so on ….

But this page, to me, points to what I see as a fundamental problem with behavioural economics.

Let me draw an analogy with the history of astronomy. In 1500, the dominant model of the universe involved the sun, planets and stars orbiting around the earth.

Since that wasn’t what was actually happening, there was a huge list of deviations from this model. We have the Venus effect, where Venus appears in the evening and morning and never crosses the night sky. We have the Jupiter bias, where it moves across the night sky, but then suddenly starts going the other way.

Putting all the biases in the orbits of the planets and sun together, we end up with a picture of the orbits that looks something like this picture – epicycles on epicycles.

But instead of this model of biases, deviations and epicycles, what about an alternative model?

The earth and the planets orbit the sun.

Of course, it’s not quite as simple as this picture – the orbits of the planets around the sun are elliptical, not circular. But, essentially, by adopting this new model of how the solar system worked, a large collection of “biases” was able to become a coherent theory.

Behavioural economics has some similarities to the state of astronomy in 1500 – it is still at the collection of deviation stage. There aren’t 165 human biases. There are 165 deviations from the wrong model.

So what is this unifying theory? I suggest the first place to look is evolutionary biology. Human minds are the product of evolution, shaped by millions of years of natural selection.

Hey everyone,

This is my first post!

This is what I've been wondering lately:

Who is the best sales person in the world? Who knows?

‘Sales competitions’ generally refers to ‘in-house’ competitions established by managers to motivate their sales people to compete against one another.

Recently I began thinking about the prospects for a ‘world sales tournament’ of sorts:

Successful sales people have lots of money. But sales is derided, whether it be in real estate, ‘charity mugger’ fundraisers, or even the people doing tenders for defence contracts.

What if we could could take their money, convert it to prestige, and take a smooth commision on the whole thing?

Sales tournaments! The World Series of Sales! The Sales Olympics. Major League Sales. Who says sales people ought to pay an entry fee anyway (except, perhaps to get a manageable number of entrants). If there are companies out there with products or services to sell, getting the best, most competitive sales people in the game to sell them is a highly desirable service itself. In return for product to sell, said product and service companies could sponsor the competition.

Sales people have difficulty switching industries, despite their highly tuned sales skills. Product and industry knowledge is easy to pick-up, but soft skills are tougher to gain. Never-the-less, recruiters are reluctant to pick up sales people from other industries - the numbers don’t always make sense. Someone working for a luxury car dealership may have huge sales numbers, but a luxury hand bag dealer might not be able to translate the numbers over. Having a high sales ranking, in a similar way that programmers are ranked in coding competition websites, could make for a highly desirable piece of career capital.

An online ‘quick and dirty’ version could be coded for email marketers and telemarketers that could be conducted in a distributed fashion in the likeness of coding competitions. But, a large scale, potentially TV-friendly could be much more profitable.

There’s an EA aspect to this too. Rationalisation of sales human resources may make Effective Altruism fundraising more quantified. And, with the birth of this idea here - the earliest competitions, or the non-profit ones could be ‘selling’ those GiveWell recommended charities as options for prospective donors. Major League Fundraising/Philanthropy! The same could be said about politics - if this become ubiquitous, voters could try to adjust their interpretations of the policy-offerings of politicans by their sales ability.

I reckon there could even be ‘team sales’. People might barrack for particular sales team’s their affiliated with - Say the Farmers Marketing Cooperatve of California (made up name) may consist of 10 members, but be supported by hundreds of farmers. Then, when it comes to a competition, say to raise money for Development Media International, one of GiveWell’s charities, people would support them like a sports team. Imagine that, people caring as much about charity as sports teams, or their online gaming leagues! In a sense, this is the gamification of sales.

If you have read this till here, you are the kind of person I want to help me build this. Please do get in touch, preferably publically as a comment here and privately (yes, twice! Once for people to know who’s involved, and twice for contact details which you may prefer not to publicly disclose) with a contact email address so I can keep everyone involved in the loop and we can decide upon a work cycle.

Equity split for founding team, including myself will be by negotiation among all of us. I forsee an equal split of total equity - an equal partnerships, for those involved.

I'm from Baltimore, MD. We have a Baltimore meetup coming up Jan 3 and a Washington DC meetup this Sun Dec 13. So why do the two meetups listed in my "Nearest Meetups" sidebar include only a meetup in San Antonio for Dec 13 and a meetup in Durham NC for Sep 17 2026 (!)?

If the usual short, intermittent gazes of a conversation are replaced by gazes of longer duration, the target interprets this as meaning that the communication is less important than the personal relationship between two people...Traffic police wear reflecting, mirrored glasses to help reduce the possibility of an argument; irate or nervous drivers can be put off a confrontation if they not only cannot see the eyes of the policeman but are also forced to see their own eyes. They experience objective self-awareness, seeing themselves as objects and not seeing those they are engaging with.

I'm thinking about people's capacity for emotional healing-- I believe this is possible because people have a base state to aim at, even if it's a slow and somewhat indirect process. My question is whether it would be possible to build something like this into an AI, since I assume an AI (even if not in a society of AIs) could either have mistakes built into its structure or make mistakes when changing itself.

Facebook has open-sourced its AI hardware.

When I go to CFAR web page, my browser complains about the certificate. Anyone else having this problem?

Curious: Are there any (currently active) readers who are in Idaho, Eastern Washington, or Eastern Oregon?

This article discusses FAI, mentioning Bostrom, EY etc. Its interesting to see how the problem is approached as it goes more mainstream, and in this particular case a novel approach to FAI is articulated: whole brain emulation (or biologically inspired neural nets) ... on acid!

The idea is that the WBE will be too at-one-with-the-universe to want to harm anyone.

Its easy to laugh at this. But I think there's also a real worry that someone might actually try to build an AI with hopelessly inadequate guarantees of safety.

Having said that, perhaps the idea is not quite as crazy as it sounds. If WBE comes first, then some form of psudo-drug based behavioural conditioning is better than nothing for AI control, although I would have thought that modifying oxytocin to increase empathy would be the obvious strategy: digital MDMA, not LSD.

Tangentially, there seems to be a perception some people have that taking LSD causes prosocial values (or, at least, what they believe to be prosocial values), but there seems to be a real danger of confusing the direction of causality here - hippies do acid, and hippies hold certain values, but the causal direction is surely: hippy values -> become hippy -> take acid, not take acid -> become hippy. Of course, perhaps acid might make the hippy values stronger, but that could be because the experience is interpreted within the structure of your pre-existing values. I have heard some (atypical?) neoreactionaries plan an acid trip, for spiritual reasons, and their values certainly appear different from hippy values. Of course, both the neoreactionaries and the hippies believe that they hold prosocial values, they just differ on what prosocial values are. Perhaps their terminal values are not so different, but they have very different models of the world?

To briefly go back to the original point, I think the author is conflating two things - just because 'can we program an AI to hallucinate?' is an interesting question (at least to some people), does not mean that it is an actually sensible proposition for FAI control. Conversely, just because this idea can trigger the absurdity heuristic, does not mean that 'AI behavioural modification with drugs' is an entirely useless idea.

So I’m attempting to adopt practices that will bring me closer to generally strategic behavior. I am also interested specifically in strategic/efficient studying. To that end I would like as much of an info dump as possible on the topic on failure.

This can include avoiding failure, preparing for failure even when avoiding it, how to notice when you are failing, and perhaps how to fail gracefully (as possible). I realize there is overlap/confusion here; I was simply rattling of primers for you to consider.

Please err on the side of inclusivity. I am not starting from a state of complete ignorance (the sequences can be turned towards my concerns handily), but best be safe.

Thank you for your help! :)

Edited for clarification, I hope.

Also for starts: What sorts of questions could someone ask to learn the most about failure?

Is anyone aware of a healthy diet plan that comes with pre-packaged meals?

It is not always possible to measure and calibrate expertise. Even when the opportunity exists, few professional disciplines make use of it. Very few experts have had any formal training or calibration for the provision of judgements. That should change. Perhaps a rationality startup could emerge to provide professional development across a range of professions. Until then, it’s important to know who to trust, how to trust them and when to trust them. To the lay person, graphs are intimidating. Atmospheric science is notoriously complex. Expert judgement is a ‘next best’ option, then perhaps what is socially normative and marketable. To the lay person, how can expert judgement be interpreted? Who even gets counted as an expert. We frequently hear about a ‘scientific consensus’ but also here from seemingly erudite ‘skeptics’, who use graphs that are compelling but uninterpretable in the broader context of all the information around. It looks like the naive algorithm for evaluating evidence, while a naive Bayesian conclusion, is not particularly efficient in some important cases: ...the claim of expert status is evaluated based on the professional characteristics and track record of the person. Their qualifications, experience, publications and professional standing are relevant. In the latter case, expert status may be evaluated by testing an expert’s judgements against independent evidence, matching predictions with outcomes observed subsequently – ACERA This has real world consequences: In Australian Federal courts, opinion evidence is admissible if it assists ‘…the trier of fact in understanding the testimony, or determining a fact in issue’ (ALRC 1985; p. 739-740). Scientific validity is established through falsifiability, peer review, acknowledged error rates, general acceptance of ideas and valid methods (Preston 2003). Science is presumed to act as a check on bias or prejudice. For instance, Cooke (1991) argued that if individuals follow the scientific method scrupulously, then they may arrive at results that have a claim on rationality and impartiality. The same thinking permeates current views of expert judgments in risk analysis. In this review, we will explore the view that scientists are advocates of a particular scientific view and spend their time trying to convince others of their position. In adversarial legal systems, potential expert witnesses are selected overwhelmingly for their credentials and for the strength of their support for the lawyer's viewpoint (Shuman et al. 1993; in Freckelton 1995). Lawyers search for appropriate attributes in an expert and develop strategies to maximize their chance of winning a case. Success often depends on the plausibility or self-confidence of the expert, rather than the expert's professional competence (ALRC 2000) ACERA I feel climate change is a good example of a thing which is allegedly highly important but extremely complex, where deferral to experts is probably prudent. However, knowing how to relate to expert evidence is then important, particularly if you, like me, are unsure about what to do with all the expertise floating around awhile action is sluggish, prompting me to wonder – what’s going on here? So why is a structured approach to expert interpretation useful in general. Let my friends from ACERA tell it When expert opinion is required, ideally, there would be a pool of people available, appropriately trained, with extensive experience and sound normative skills. However, this pool is sometimes small or non-existent. Often, it is composed of people with conflicting opinions, values and motivations. The positive qualities of experts noted above have limitations that are often overlooked. Expertise is limited to technical domains that are narrower than most practical problems. Expert performance does not transfer to other disciplines, even those that seem quite similar. As we will see below, most experts are overconfident. It can be difficult to have a clear picture of what an expert knows, because experts themselves have difficulty knowing the limits of their own expertise (Freudenburg 1999, Ayyab 1999). Furthermore, the self-serving nature of judgments may bias advice (Krinitzsky 1993). This section outlines definitions of experts in several fields and evaluates the importance of defining expertise. After doing some research, I’ve come up with a few notes. They are not in my own words, because they are written about adequately by others: Who is an expert? According to Hart (1986), attributes that characterize an expert include effectiveness (they use knowledge to solve problems with an acceptable rate of success), efficiency (they solve problems quickly) and awareness of limitations (they are willing to say when they cannot solve a problem). Other critical questions may include (Hart 1986, Walton 1997): Is the expert credible? Are they personally reliable? Do they have a special or conflicting interest? Is their assertion consistent with other experts and with their prior advice? What evidence is their opinion based on? How much experience do they have in the domain of the question at hand? Studies of experts and expertise provide a few generalizations (see Feltovitch et al. 2006). Experts organise knowledge effectively, have superior recall of information and have improved abilities to abstract knowledge to new situations, compared to lay people. They perform the basic operations of their discipline efficiently, and are able to think critically about data and methods in their domain. Usually, attaining expertise requires both study and practical experience. -ACERA Should we trust experts? Among experts in ecology: 'No consistent relationships were observed between performance and years of experience, publication record or self-assessment of expertise.'. >Expert responses were found to be overconfident, specifying 80% confidence intervals that captured the truth only 49–65% of the time. In contrast, student 80% intervals captured the truth 76% of the time, displaying close to perfect calibration. Best estimates from experts were on average more accurate than those from students. The best students outperformed the worst experts.

Expert status is a poor guide to good performance. In the absence of training and information on past performance, simple averages of expert responses provide a robust counter to individual variation in performance. -ACERA

Part 2: 'Which experts to trust', 'Limitations' and 'Practice'

**Part 1: IS EXPERT OPINION A WASTE OF TIME? is available here

which experts to trust

Now for an application:

Tetlock (2005) showed, based on 20 years of longitudinal research on several hundred political experts, the single most powerful predictor of forecasting skill had little to do with what experts think, and more to do with how they think: he called it ‘cognitive style’. As well as being a predictor of forecasting accuracy, ‘cognitive style’ is also a strong correlate of overconfidence. Cognitive style is determined in large part by personality traits, which he characterized as hedgehogs and foxes.

Hedgehogs know “one big thing, and under the banner of parsimony [work] to expand the explanatory power of that big thing to ‘cover’ new cases”; the foxes know “many little things and [are] content to improvise ad hoc solutions” (p. 20-21). Cognitive style correlates with personality measures such as‘openness’ and ‘need for closure’. Tetlock’s work showed consistently greater overconfidence in ‘hedgehog’ experts than in ‘fox’ experts.

An obvious example of a category of hedgehog that springs to mind are ideologues – everyone from Anarchocapitalists to Bayesians to ‘materialists’ to ‘ordinary people’ to ‘Agnostics’ (just trying my best to insult the most number of people here, cause not enough people recognise they’re as guilty of the things they might be looking for in others). The motivated reasoning bias springs to mind (after prompting by the ACERA paper).

Commiserate:

Demographic variables had little bearing on accuracy. Years of experience also failed to predict accuracy. Left vs Right (the Ideology factor), Institutional vs Realist (the Realpolitik factor) and Doomster vs Boomster (the Optimism factor) also failed to predict accuracy. The consistent result across the spectrum of content domains, was that hedgehogs were: a) less well calibrated, b) had poorer discrimination, c) were more overconfident and d) were less likely to update their beliefs (were poorer Bayesians). Foxes outperformed all groups on each of the four criteria, and the middling groups fell in between in predictable ways.

important limitations to this research line

However, there have been few, if any, studies of their efficacy in realistic biosecurity settings. In particular, relatively little attention has been paid to the reliability or accuracy of conceptual models. Few of the formal techniques for elicitation, calibration or verification have been tested in conditions typical of biosecurity risk analysis. There is an opportunity to evaluate the most promising methods, with a view to implementing some of the most effective procedures in routine risk analyses.

Practice

For any smart cookies out there, you must wonder – well, what can you do to get the most out of experts? It’s too much information to tell you about here, but I recommend ACERA’s

• Elicitation tool, user manual and

• Elicitation tool, process manual

Academic delphi style groups outperform baseline groups by around 50%. Could professional delphi groups be formed to profit from stock and prediction markets?

and

• UPenn’s Delphi decision aid And this Real time Delphi tool

Sincerely, Carlos Larity

Self-appointed claimant as the resident expert expert

*Penned in order to get my karma back at around 11. I aimed to have a net zero karma – balancing more controversial stuff with popular content, just enough to not discredit myself. Though, I just found out that I actually can’t upvote when my karma is zero, and I need ’11 more’ to do so.

And, just a fun additional link on reddit for you to check out. Remember not to trust experts, particularly not experts in experts...or hedgehogs!

**Part 1: IS EXPERT OPINION A WASTE OF TIME?

Part 2 on 'Which experts to trust', 'Limitations' and 'Practice' available here

It is not always possible to measure and calibrate expertise. Even when the opportunity exists, few professional disciplines make use of it. Very few experts have had any formal training or calibration for the provision of judgements. Perhaps a rationality startup could emerge to provide professional development across a range of professions. Until then, it’s important to know who to trust, how to trust them and when to trust them.

To the lay person, graphs are intimidating. Atmospheric science is notoriously complex. Expert judgement is a ‘next best’ option, then perhaps what is socially normative and marketable.

To the lay person, how can expert judgement be interpreted? Who even gets counted as an expert. We frequently hear about a ‘scientific consensus’ but also here from seemingly erudite ‘skeptics’, who use graphs that are compelling but uninterpretable in the broader context of all the information around.

It looks like the naive algorithm for evaluating evidenc, while a naive Bayesian conclusion, is not particularly efficient in some important cases:

...the claim of expert status is evaluated based on the professional characteristics and track record of the person. Their qualifications, experience, publications and professional standing are relevant. In the latter case, expert status may be evaluated by testing an expert’s judgements against independent evidence, matching predictions with outcomes observed subsequently – ACERA

This has real world consequences:

In Australian Federal courts, opinion evidence is admissible if it assists ‘…the trier of fact in understanding the testimony, or determining a fact in issue’ (ALRC 1985; p. 739-740). Scientific validity is established through falsifiability, peer review, acknowledged error rates, general acceptance of ideas and valid methods (Preston 2003). Science is presumed to act as a check on bias or prejudice. For instance, Cooke (1991) argued that if individuals follow the scientific method scrupulously, then they may arrive at results that have a claim on rationality and impartiality. The same thinking permeates current views of expert judgments in risk analysis. In this review, we will explore the view that scientists are advocates of a particular scientific view and spend their time trying to convince others of their position. In adversarial legal systems, potential expert witnesses are selected overwhelmingly for their credentials and for the strength of their support for the lawyer's viewpoint (Shuman et al. 1993; in Freckelton 1995). Lawyers search for appropriate attributes in an expert and develop strategies to maximize their chance of winning a case. Success often depends on the plausibility or self-confidence of the expert, rather than the expert's professional competence (ALRC 2000) ACERA

I feel climate change is a good example of a thing which is allegedly highly important but extremely complex, where deferral to experts is probably prudent. However, knowing how to relate to expert evidence is then important, particularly if you, like me, are unsure about what to do with all the expertise floating around awhile action is sluggish, prompting me to wonder – what’s going on here?

So why is a structured approach to expert interpretation useful in general. Let my friends from ACERA tell it

When expert opinion is required, ideally, there would be a pool of people available, appropriately trained, with extensive experience and sound normative skills. However, this pool is sometimes small or non-existent. Often, it is composed of people with conflicting opinions, values and motivations. The positive qualities of experts noted above have limitations that are often overlooked. Expertise is limited to technical domains that are narrower than most practical problems. Expert performance does not transfer to other disciplines, even those that seem quite similar. As we will see below, most experts are overconfident. It can be difficult to have a clear picture of what an expert knows, because experts themselves have difficulty knowing the limits of their own expertise (Freudenburg 1999, Ayyab 1999). Furthermore, the self-serving nature of judgments may bias advice (Krinitzsky 1993). This section outlines definitions of experts in several fields and evaluates the importance of defining expertise.

After doing some research, I’ve come up with a few notes. They are not in my own words, because they are written about adequately by others:

Who is an expert?

According to Hart (1986), attributes that characterize an expert include effectiveness (they use knowledge to solve problems with an acceptable rate of success), efficiency (they solve problems quickly) and awareness of limitations (they are willing to say when they cannot solve a problem). Other critical questions may include (Hart 1986, Walton 1997): Is the expert credible? Are they personally reliable? Do they have a special or conflicting interest? Is their assertion consistent with other experts and with their prior advice? What evidence is their opinion based on? How much experience do they have in the domain of the question at hand?

Studies of experts and expertise provide a few generalizations (see Feltovitch et al. 2006). Experts organise knowledge effectively, have superior recall of information and have improved abilities to abstract knowledge to new situations, compared to lay people. They perform the basic operations of their discipline efficiently, and are able to think critically about data and methods in their domain. Usually, attaining expertise requires both study and practical experience.

-ACERA

Should we trust experts?

Among experts in ecology: 'No consistent relationships were observed between performance and years of experience, publication record or self-assessment of expertise.'.

Expert responses were found to be overconfident, specifying 80% confidence intervals that captured the truth only 49–65% of the time. In contrast, student 80% intervals captured the truth 76% of the time, displaying close to perfect calibration. Best estimates from experts were on average more accurate than those from students. The best students outperformed the worst experts.

Expert status is a poor guide to good performance. In the absence of training and information on past performance, simple averages of expert responses provide a robust counter to individual variation in performance.

-ACERA

My impression is that we shouldn’t naively take the judgements of experts to be simply superior to amateurs/lay people. As counterintuitive as it is: experts are more accurate than amateurs, but amateurs are more precise.

Expert overconfidence is one thing. Expert underperformance relative to simple equations is another:

One of the earliest and most confronting studies was by Meehl (1954). He explored clinical versus statistical prediction and demonstrated that many diagnostic judgments of experienced clinical psychologists were worse than the diagnostic judgments produced by a simple formula based on few variables. The implication was that any reasonably competent individual could out-do the clinical judgments of an expert psychologist by asking a standard series of questions and applying a standardised algorithm to the responses. Dawes et al. (1989) reviewed hundreds of studies that followed this tradition, and with the exception of a few medical diagnoses, the results were replicated in a vast array of domains including financial investment, sports forecasting, recidivism or violence in criminals and clinical pathology

My impression after interviewing dozens of academics from various health related fields is that career advancement among these researchers pertains more to signalling the work is being done that actually doing the work. Thiel discredits this arrangement in Zero to One as dysfunctional.

Why can't seasoned politicians handle a windbag millionaire?

Is it because he is a caricature of them?