Understanding information cascades

This paper looks at the dynamics of information flows in social networks using multi-agent reinforcement learning. I haven't read it, but am impressed by the work of the second author. Abstract:

We model the spread of news as a social learning game on a network. Agents can either endorse or oppose a claim made in a piece of news, which itself may be either true or false. Agents base their decision on a private signal and their neighbors' past actions. Given these inputs, agents follow strategies derived via multi-agent deep reinforcement learning and receive utility from acting in accordance with the veracity of claims. Our framework yields strategies with agent utility close to a theoretical, Bayes optimal benchmark, while remaining flexible to model re-specification. Optimized strategies allow agents to correctly identify most false claims, when all agents receive unbiased private signals. However, an adversary's attempt to spread fake news by targeting a subset of agents with a biased private signal can be successful. Even more so when the adversary has information about agents' network position or private signal. When agents are aware of the presence of an adversary they re-optimize their strategies in the training stage and the adversary's attack is less effective. Hence, exposing agents to the possibility of fake news can be an effective way to curtail the spread of fake news in social networks. Our results also highlight that information about the users' private beliefs and their social network structure can be extremely valuable to adversaries and should be well protected.

Have you read the backreaction blog where Sabine Hossenfelder details much the same phenomenon in high-energy physics? She claims that the prevailing groupthink ended up believing into String Theory without a shred of evidence for (only some vague hints), and so far with every single prediction of it refuted?

UPDATE.

We (jacobjacob and Ben Pace) have finally settled on the allocation of the $800 bounty for this question. All the motivations are summarised in this comment, together with links to the relevant prize-winning answer/comment.

We will also post individual notices with motivations next to each comment for ease of discussing them.

We'll PM all prize winners to sort out logistical details of payment.

Main post

David Manheim [LW(p) · GW(p)] (answer and additional points made in discussion) $150

This answer offers relevant and robust evidence about the role of info-cascades in forecasting environments, together with a discussion of its interpretation.

Jan Kulveit [LW(p) · GW(p)] (answer and additional comment below) $200

This answer seems to offer a learnt summary of the relevance of network science (which offers a complementary perspective on the phenomenon to the microeconomic literature linked by other commenters), which not implausibly took Jan at least an order of magnitude less time to compile than it would have taken us. (For example, the seemingly simple fact of using a different Google scholar keyword than "information cascade" might have taken several hours to realise for a non-expert.) It also attempts to apply these to the case of forecasting (despite Jan's limited knowledge of the domain), which is a task that would likely have been even harder to do without deep experience of the field.

Pablo (1 and 2) [LW(p) · GW(p)] $100

These answers compile a useful summary of the literature (we learnt a lot from going through on of the papers linked), and it attaches handy links to everything, which is a task which is on the one hand very helpful to other people, and on the other tedious and without many marginal benefits for the writer, and so likely to be under-incentivised.

Michael McLaren [LW(p) · GW(p)] $50

This answer:

• It offers a novel mechanism which is relevant to the context of intellectual progress, and ties it in with literature cited in the OP
• Rather than just linking the paper, it distills a technical paper, which is a valuable service that is usually underfunded (academic institutions comparatively incentivise novel and surprising insights)

Ways of responding

David Manheim [LW(p) · GW(p)] $50

This answer offers a practical example of a cascade-like phenomenon, which is both generally applicable and has real economic consequences. Also, the fact that it comes with a game to understand and practice responding is rare and potentially quite valuable (I (jacobjacob) am of the opinion that deliberate practice is currently a neglected virtue in the rationality/EA spheres).

rossry [LW(p) · GW(p)] $250

This answer does several important things.

• It references existing (and novel) work in economics and mechanism design, which might have been time-consuming to discover otherwise
• It distills a technical paper, which is a valuable service that is usually underfunded (academic institutions comparatively incentivise novel and surprising insights)
• The insights provided are quite action-guiding, and caused me (jacobjacob) to have ideas for how one can experiment with new kinds of forecasting tournaments that use a threshold-mechanism to change participant incentives

Is it necessarily a good idea to break up the topic into so many separate questions before having a general discussion post about it first? I would imagine that people might have comments which were related to several of the different questions, but now the discussion is going to get fragmented over many places.

E.g. if someone knows about a historical info cascade in academia and how people failed to deal with that, then that example falls under two different questions. So then the answer with that example either has to be be split into two or to be posted in an essentially similar form on both pages, neither of which is good for keeping the entire context of the discussion in one place.

This is an update on the timeline for paying out the bounties on this question. They will be awarded for work done before May 13th, but we're delayed by another few weeks in deciding on the allocation. Apologies!

Nissen et al 2016 ("Publication bias and the canonization of false facts") give a simple model for how publication bias in academic research can have a similar effect to the "information cascades" described in the OP. False scientific claims are likely to be falsified by an experiment, but will sometimes be found to be true. Positive results supporting a claim may be more likely to be published than negative results against the claim. The authors' model assumes that the credence of the scientific community in the claim is determined by the number of published positive and negative results, and that new studies will be done to repeatedly test the claim until the credence becomes sufficiently close to 0 or 1. The publication bias favoring false results can overpower the odds against getting a positive result in any given experimental replication and lead to false claims becoming canonized as fact with a non-negligible probability.

The mechanism here differs in a sense from the "information cascade" examples in the OP and on the Wikipedia page in that the false claim is being repeatedly tested with new experiments. However, I think it could be seen as fundamentally the same as the citation bias example of Greenberg 2009 in the OP, if we think of the scientific community rather than an individual scientist as being the actor. In the Greenberg 2009 example, the problem is that individual scientists tend only to cite positive findings; in the Nissen et al model, the scientific community tends to only publish positive findings. (Of course, this second problem feeds into the first.)

See this post [LW · GW] for a good, simple mathematical description of the discrete version of the phenomenon.