Why sigmoids are so hard to predict

post by Stuart_Armstrong · 2021-03-18T18:21:51.203Z · LW · GW · 7 comments

Contents

  General sigmoids
    Differential equations
    The dampening term
    Different assumptions, different dampening
    Estimating the maximum value
None
7 comments

tl;dr: Predicting the behaviour of a sigmoid-like process is not fitting the parameters of a logistic curve. Instead, it's trying to estimate the strength of the dampening term - a term that might be actually invisible in the initial data.

A logistic curve is a function that starts out growing like an exponential, but then diminishing returns hits in, and the curve plateaus:

The logistic curve has a specific equation $f\left(x\right)=\frac{L}{1+e^{-k(x_0-x)}}$; it has a turning point at $x_0$ (where it goes from accelerating growth to decelerating growth) and an (asymptotic) maximum at $L$.

The broader category of "curves that look like logistic curves" are sigmoids, and they appear all over the place - population growth, virus spreading, and so on.

I've seen a few examples and arguments as to why it's so hard to predict the turning points and maximums of a sigmoid; I wrote one [LW · GW] myself.

But all these arguments, including my own, didn't really satisfy me. The fact is, a logistic curve has three free parameters, so three perfect samples will pin it down entirely. So the arguments then turn on how a little bit of noise can make these parameters very uncertain.

And those argument are correct: small uncertainties in the data do imply very different outcomes. I illustrated this by plotting the standard logistic curve with turning point at $0$ and maximum at $1$, in blue, versus the exponential function $0.51e^{0.69x}$ (which has no turning points and no maximum), in purple:

Then all the arguments and demonstrations are more formal ways of saying "hey, these curves look similar, they're hard to tell apart". If the noise is independent, however, then you will eventually be able to distinguish the two curves if you get enough data points.

General sigmoids

I then started thinking about non-independent noise, and other similar ideas about how these curves would behave in the real world. But that was missing the key question: how do we know that the purple curve is not a sigmoid? It starts off as a pure exponential, but nothing stops me from completing it like this:

The purple curve is not a logistic curve, but it's a perfectly respectable sigmoid. The wikipedia sigmoid article lists a variety of sigmoid functions such as "Logistic functions", "Arctangent functions", "Gudermannian function", "Error functions", "Generalised logistic functions", "Smoothstep functions", and the wonderfully vague but huge category of "Some algebraic functions".

If we want to predict turning points in sigmoid-like functions, we have to do better than trying to fit the best logistic function. We need to ask where the behaviour of the sigmoids derives from.

Differential equations

The differential equation for the logistic curve is:

$$f^{\prime }\left(x\right)=k(1-f(x\left)/L\right)f\left(x\right).$$

When $f\left(x\right)$ is small, this is $f^{\prime }\left(x\right)\approx kf\left(x\right)$. The equation $f^{\prime }\left(x\right)=kf\left(x\right)$ has solution $C{e}^{kx}$ for some constant $C$.

This describes the initial period of exponential growth. More precisely, if we look in a range on which $1-f\left(x\right)$ doesn't vary much, $f\left(x\right)$ will be approximately $C{e}^{kMx}$ where $M=1-f\left(x\right)\approx 1$.

This shows why it's so hard to predict the maximum and turning point of the logistic curve from early data:

• If $f\left(x_1\right)$ and $f\left(x_2\right)$ are small compared to $L$, then, on the interval $[x_1,x_2]$, we are trying to detect the logistic curve shifting from the exponential curve of type $C_1{e}^{k{M}_{1}x}$ to one of type $C_2{e}^{k{M}_{2}x}$, with $M_1=1-f\left(x_1\right)/L$ and $M_2=1-f\left(x_2\right)/L$.

So what we're trying to do, from the potentially noisy data, is pick up the difference between growing as an exponential of exponent $k{M}_1$ (which is very close to $k$) and of exponent $k{M}_2$ (which is also very close to $k$). So we're trying to pick up the difference between two exponentials of very similar coefficients, at the very beginning of their ranges.

But, as I said, if there was no noise, we could detect that from three data points. So let's look again at the differential equation.

The dampening term

The $f\left(x\right)$ operates as growth term, causing the curve to grow exponentially. This means that the growth something is proportional to its current size. This works well for modelling disease spread in its early phases: the amount of people that will be infected is proportional to the amount of people already infected, as these go about and meet their friends and acquaintances.

The $(1-f(x\left)/L\right)$ acts as a dampening term. When $f\left(x\right)$ gets too large, the function can no longer grow as it once did. To go back to the diseases example, suppose that everyone has an equal number of friends, selected randomly from the total population $L$, and that they meet these friends equally often, in equally dangerous situations. Then if $f\left(x\right)/L$ is small, you can consider that every friend they meet will be uninfected, so they can pass on the disease, potentially, to all of them.

But now see what happens if $f\left(x\right)/L$ is larger. If $f\left(x\right)=L/2$ (half the population is infected), then $1-f\left(x\right)/L=1/2$. So if someone infected meets their friends, they will find that only half of them are uninfected. So the growth rate, relative to the infected population, is half of what it was initially. And as $f\left(x\right)$ grows, $1-f\left(x\right)/L$ shrinks rapidly, so the growth rate falls fast.

Different assumptions, different dampening

In the description above, I assumed:

suppose that everyone has an equal number of friends, selected randomly from the total population $L$, and that they meet these friends equally often, in equally dangerous situations.

These assumptions are very unrealistic. People have different numbers of friends; these friends are likely to be friends of each other; people spend different amounts of time with different people, and, during a pandemic, people's behaviour changes - and different people's behaviours change differently. There is a time-changeable network of connections, that cannot be easily reduced to "N infected people are likely to infect kN other people".

So to model a more realistic pandemic - or any sigmoidic process - we want a dampening term that looks more like $1-H\left(f\right(x\left)\right)$, where $H$ is some function that is small when $f\left(x\right)$ is small, and tends to $1$ as $f\left(x\right)$ grows towards $L$.

I chose a few random possible $H$s, to see how things would go:

• $H\left(f\right(x\left)\right)=f\left(x\right)$: the standard logistic curve (blue curve, $L=1$).
• $H\left(f\right(x\left)\right)=\left(f\right(x)/0.5{)}^2$: less initial dampening (purple curve, $L=0.5$).
• $H\left(f\right(x\left)\right)=\sqrt{f\left(x\right)/2}$: more initial dampening (green curve, $L=2$).
• $H\left(f\right(x\left)\right)=max[0,3f(x)-1]$: no initial dampening, which suddenly turns on later (orange curve, $L=2/3$).
• $H\left(f\right(x\left)\right)=0$: no dampening, full exponential (red curve, infinite $L$).

For all these $H$s, I made the assumption that $f(-5)=f^{\prime }(-5)=1/1000$: the values of $f$ and its derivative were $1/1000$ at $x=-5$ (hence they behave very similarly around that point). Then the plots of these curves are all over the place:

This is the core reason why the turning point and the maximums are so hard to predict from early data: we're not only trying to figure out the parameters of a logistic curve, but the functional form of the dampening function - a dampening function whose effect is insignificant in the early data.

In some cases, the dampening is impossible to distinguish in the initial data - for instance the no-initial-dampening versus the no dampening at all. If the initial dampening and the later dampening are due to different factors, then early data tells us nothing about the maximum and turning point of the curve.

For example, the technology adoption life cycle divides users into the categories of "innovators", "early adopters", "early majority", "late majority" and "laggards":

The turning point is the transition from early to late majority; the maximum value is the sum of all the categories (including laggards).

Early data, though, comes entirely from innovators and early adaptors. If these populations are actually distinct in how they behave and how innovations spread among them, then the early data will tell us nothing about the turning point or the maximum value.

Estimating the maximum value

Sometimes it is feasible to infer $L$, the maximum value. To do that, we don't need to figure out what $H$ is; we just need to figure out what would cause the dampening term $1-H\left(f\right(x\left)\right)$ to fall towards $0$.

A disease cannot infect more people than the total population, so we know that $L$ is a less than or equal to the total population $\Omega$. For diseases, $L$ can be inferred from the $R_0$, the number of people each infected person infects. In ideal situations, this would give 'herd immunity' once this many people are infected:

$$L=\Omega (1-\frac{1}{R_0}).$$

That formula assumes a homogeneous population that mixes randomly with each other, but you can get improved $L$ estimates by looking at similar diseases, estimating the network of spread, and so on.

So the maximum value of sigmoid can often be inferred based on 'meta' considerations, even if the turning point is extremely hard to predict.

If we replot all the above sigmoids with the same value for $L$ (in this instance, $L=1$), we get the following graph^[1] (again, setting $f(-5)=f^{\prime }(-5)=1/1000$ for all curves):

It seems that the best approach when $L$ is known is to first plot a pure exponential (in dotted red in that figure), then try and guess by how much the turning point will lag the pure exponential - maybe by making informed guesses as to the form of the dampening term $1-H\left(f\right(x\left)\right)$.

---

1. Respectively setting $H\left(f\right(x\left)\right)=f\left(x\right)$ (blue), $H\left(f\right(x\left)\right)=f(x{)}^2$ (purple), $H\left(f\right(x\left)\right)=\sqrt{f\left(x\right)}$ (green), and $H\left(f\right(x\left)\right)=max[0,2f(x)-1]$ (orange). ↩︎

7 comments

Comments sorted by top scores.