Completion Estimates

post by scarcegreengrass · 2024-03-09T22:56:50.802Z · LW · GW · 2 comments

Contents

  With % Complete Metric
  Without Any Helpful Metrics
None
2 comments

Recently i've been brainstorming algorithms for estimating the time remaining in a process. Progress bars, estimated time left until completion, etc. Sometimes, the thing you want just isn't here yet, & you want a greater understanding of when you will have it. Here are the most useful techniques i've found so far.

With % Complete Metric

Sometimes, you are waiting on a process to complete, but at least you have a decent estimate of the percent complete so far. For me, this is when i am trying to finish a long novel. I'm often partway thru a 600-page book which is rewarding but also requires some effort.

When a completion metric is available, my favorite technique i call Strict D=RT. I assume that we have seen a typical proportion between the total pages read so far & the total days since i started the book. If i read 300 pages in the first 3 days, then neglected to read anything further until Day 7, we can assume that 300 pages per 7 day is my typical rate. Then we would expect me to complete the 600-page book by Day 14. (Actually, i read slower in real life, but you get it.)

The core output of this algorithm is the date of expected completion. But you can also generate a pages/day line graph with intersecting lines, which is fun.

I like this because it does not assume that a delay will be the last delay. And indeed - sometimes i neglect to read for many days in a row, on a unpredictable basis.

Without Any Helpful Metrics

I'm also interested in estimating completion of murkier processes. I'm fascinated by the challenge of find a algorithm that could have accurately predicted the launch of NASA's SLS rocket in 2022. This rocket project began in 2010, aiming to launch in early 2017. It was delayed dozens of times, with a total delay of 5 years. In 2022 August, it almost launched, but postponed on the launch day. It ended up launching more than 2 months later. This is a project that would confuse even a robust algorithm! I've been testing various algorithms against the history of SLS delay announcements.

I haven't found a perfect solution yet. I've played around with a model i call Adaptive K. You maintain a number called k. This is your trust multiplier. If NASA's official launch day is N days from now, you estimate it is instead k * N days from now. Every day, you reduce k by 0.0025, with a floor of 1. No news is good news. But whenever a delay is announced, if it is even worse than your cynical k * N estimate, you 'panic' and double the value of k.

There are some more details to Adaptive K, but it's moot - this algorithm isnt as accurate as i want it to be. It panics 5 times with this dataset. I wish it would grow its cynicism more quickly after being disappointed twice. And then k stays high (around 8) in the last few weeks. It sometimes predicts the launch will be next year, when it's really a few weeks away. Tweaking k's decay rate hasn't helped - this just makes it panic more in the early years, forgetting all its cynicism too quickly. Part of the difficulty is that the early-years environment is rather different than the week-until-launch environment.

The best solution i've found so far is very simple. I call it Double Days Remaining. If the official launch day is N days from now, you estimate it to really be 2 * N days from now. Testing this against the SLS delay history, the moment of greatest overcynicism was in 2014, when DDR (Double Days Remaining) is 1 year too cynical. And 2017 has the moment of most overoptimism, which is more than 2 years too optimistic. But for the final 2.5 years of the project, the DDR estimate is within +- 6 months of the truth.

I'm risking overfitting here. 2 is close to the true ratio between estimated & true time remaining for this project. But DDR seems surprisingly robust considering that it's simple enough to estimate in your head.

I've read up a little on other ideas, like polynomial regression. I'm still intrigued by models that maintain a trust parameter, like k. I think there's demand for these algorithms for monitoring many technology or construction projects. In industries like space, timelines usually slip beyond the initial 'No Earlier Than' date. Yet saying 'It'll never happen' is also inaccurate.

I'm still thinking about this topic - let me know if you have any ideas!

2 comments

Comments sorted by top scores.