Calibrated estimation of workload

post by kerspoon · 2021-01-30T12:15:23.413Z · LW · GW · 2 comments

Contents

  Show me the data
  Data Gathering
  Creating the calibration chart
  Is it worth the effort (for me)?
  Is it worth the effort (for you)?
None
2 comments

I've been using a spreadsheet to track my work in various ways for about 10 years. There are some fun benefits to it that I want to share. It has really helped me improve my calibration and precision in an important area - planning my workload - and want to encourage others to try something similar for themselves.

This post follows on from how-quickly-can-you-get-this-done-estimating-workload [LW · GW] but you don't need to have read that for this to make sense.

Show me the data

Before I get to the details lets look at the data for this year so far.

The first chart answers the question: how well calibrated am I when someone asks me "what's the chances that you will be able to complete this by next week?". You can see in the chart that I complete 65% of tasks that I say have a 50% chance of being completed.

The second chart answers the question "when it say it will take X hours, how long will it actually take?". 

Data Gathering

At the start of each week I write down the most important tasks to complete that week. I mostly take these tasks from my current goals, which are set at the start of the year. I try to make some progress on my yearly goals every week; this helps me ensure that I keep my day-to-day tasks aligned with my long-term goals. The rest come from a longer "todo" list which mostly fills up with general admin/emails/etc.
Next to the tasks I also write down how long I think it will take and my probability of completing it that week. To make the maths easier my probabilities are to the nearest 10% (I've added in another bucket at 95% because I want to be able to distinguish between 90% and 95%). 
Here is how one week of tasks looks (I've hidden the names of the tasks):

For the duration I have learnt not to schedule tasks I expect to take more than 4 hours. My time estimates for these are always way off, and it's often unclear what "completed" even means for many of these. If I'm writing a draft of an article, for example, I will often realise part way through that I need to do more research. As you can see in the image I also highlight anything over 2 hours, just as a reminder to myself to see if I can easily break it down further. What I do with these large tasks is either split them into smaller tasks but I have to treat "deep work" tasks differently.

For some tasks, the quicker I tick them off the better e.g. clearing my email inbox. For others the more time I spend on them the better e.g. deep work such as researching. The former get time estimates, the latter get booked into my calendar as if they were meetings. e.g. I aim to schedule a 4h block every week for deep work on a particular project, I will still list "do at least 4h deep work on X project" in my tasks but it wont have a time estimate against it.

Creating the calibration chart

I've progressed through a few versions of the chart itself and since I'm not confident in the third method I thought I would list all of them.

• Version 1. I just drew a line for the total percentage of completed tasks (across all weeks) and another line for perfect calibration. It didn't work when I only had a small number of data points.
• Version 2. Some bins (e.g. 10%) had very few data points, so I wanted to construct a range that represented how much data I had in them. The easy way to do this was to draw two lines, one where I assumed I had completed one extra task and one where I assumed I had failed one extra task. If I have a lot of data this give a tight range but if there is very little data for a bin then it gives a wide range (in the extreme case it gives a range of 0% to 100% if I have no data)

• Version 3. [epistemic status: the first time I've done this so not sure of the maths]. Assume a prior of half a fail and half a successful task (Jeffreys prior). Add in my data on number of trials and number of successes to form a beta distribution giving Beta(0.5+successes, 0.5+failures). Work out the mean and the credible interval and plot those against a line of perfect calibration. I chose a 60% interval because the range on the chart felt about the right width to tell me how to improve my calibration. (incidentally if you know how to do something similar for my time estimation graph please can you point me in the right direction)

Is it worth the effort (for me)?

For me, yes, it's worth the effort. Not only do I benefit from having a clear (and short) list of what I should be working on each day but I enjoy playing with this spreadsheet, and I find the calibration aspect fun. Calibration training is also a useful skill to be able to accurately and precisely predict what I will be able to get done in a week.  

One of the real benefits in tracking what I want to complete each week is that I can look over which things I failed to do, and test out ways to fix whatever caused me to not complete them. The mindset of treating any task I have failed to complete as an opportunity to improve my work habits has been a useful one for me and the information that comes from having years of this data means I have a good feel of what things will work well for me and what wont.

One important note is that I do not force myself to stick to my own plan for the week. I will regularly have a burst of inspiration or motivation which I will gladly follow, ignoring what I had planned to do. Equally, I won't be too hard on myself If I don't manage to complete everything. But I will review whether I completed what I planned to, as part of my weekly review. "No, I didn't do what I planned to, I did something better" is an acceptable answer, but so is "no, and I should have completed it". For the cases where I should have completed something and didn't I will spend a bit of time thinking of TAPs ("if ___ then __ " plans) to overcome these problems if they occur again. 
By only sticking to my weekly plan if it results in a better commitment to my long term goals I complete vastly more work that is important to me. This is probably where most of the value comes for me and this could be done with a single sheet of paper and a pen. 

It takes about an hour a week for me to do my weekly review. This involves:

• Reviewing my progress towards long term goals, reviewing my completion of last weeks tasks, and writing a short diary of my motivation, productivity and happiness (MPH).
• Planning what I intend to work on that week including putting time estimates to each (I might have a meeting-heavy week where I can only do a couple of hours of useful work a day, or I might have a full 40h to do useful work), then I roughly work out which day I will do each task, finally adding a probability.

My weekly review is very clearly a net gain for me. I get a lot more done in the weeks that I write one than those that I don't. In fact, for me, the more planning I do the more I seem to get done, there must be a point of negative returns but I've been surprised how much planning you can do and still keep a marginal benefit. 

One final note. I don't know if this is something common to many people, something specific to me, or something caused my my processes but about 3 times a year, for between a week and a month I will 'break'. I fail to keep track of data, I stop doing deep work, my emails pile up. I know this happens when I believe I cannot fit my work into the deadlines I have been set. I also know that my efficiency drops to a very small fraction of my usual during this time meaning I really can't get my work finished in time. 

Is it worth the effort (for you)?

If you like playing with spreadsheets then I would strongly encourage you to try it for one month. I don't think it will be valuable for everyone but I do think it's worth trying. The cost is about 2 hours per week to potentially increase productivity and improve calibration/estimation. As a slight aside - the first time I tried tracking my time and tasks in detail, literally writing down what I did to a granularity of 10mins for a full week was really enlightening and change the way I conceptualised my time.

I wouldn't start going into all the detail that I have done, I actually track a lot more data than this about my days/weeks, but a short list of tasks for the week along with a probability of completion and an estimated duration is enough of a start to see if the process would work for you. 

Edit: A template is available https://docs.google.com/spreadsheets/d/1CrdJ3KwWWsDomgRqwSOn96Ix58VHUz88YQhAfiGQuBI/edit?usp=sharing. Add your tasks to the tasks sheet, make sure you set the date of each task you add to match the start of the work week (Monday) and use the dropdown for the likelihood.

2 comments

Comments sorted by top scores.