Half-Baked Products and Idea Kernels
post by Liron · 2020-06-24T01:00:20.466Z · LW · GW · 9 commentsContents
Diminishing Returns on Time Spent Efficient Course Correction None 9 comments
When I ask someone at work for a project proposal, I never want the person to go silent on me and put in 100 hours of solitary work, and then finally show me something and ask for my feedback. I always want to see a half-baked product.
You can half-bake something in an hour or two, or even in a few minutes.
The advantage of half-baking is that you get a quick feedback loop. The more you think there’s a possibility that I’ll say “no, that’s not what I wanted”, the more half-baked you should make your first effort before getting my feedback.
When brainstorming ideas, my term for a half-baked idea is a kernel. A kernel is usually a crappy idea on its own, but there’s “something to it” that could make it the seed of a better idea. I encourage people to toss out kernels.
There are two reasons why operating this way is efficient:
Diminishing Returns on Time Spent
Say you work on something for 100 hours. While each hour adds value, typically the highest-value hour is the first hour and the lowest-value hour is the last hour, and it follows a curve like this:
For example, if you're going to spend 100 hours writing a long report, spending one hour to brain-dump the key bullet points would give a reader a lot more than 1% of the final value of your report. Realistically, it’s probably more like 20%.
So the less time you spend working before getting feedback, the higher your productivity was in that time.
Efficient Course Correction
When you’re starting out on a new project that isn’t well-understood, you’re unlikely to go in the exact right direction. So you don’t want to go too far before getting a course correction, or you’ll waste time.
The top path shows how most people waste their time by investing too much effort between course corrections. The bottom path shows the efficient approach: you do a small chunk of work, then get feedback from your boss or your customers to correct your course, then do the next small chunk of work.
Once your course corrections become small, you can do larger chunks of work between course corrections. Until then, take small steps that produce half-baked products and idea kernels.
9 comments
Comments sorted by top scores.
comment by DirectedEvolution (AllAmericanBreakfast) · 2020-06-24T01:12:20.121Z · LW(p) · GW(p)
M̶u̶c̶h̶ ̶o̶f̶ ̶t̶h̶e̶ ̶a̶d̶v̶i̶c̶e̶ ̶o̶n̶ ̶w̶r̶i̶t̶i̶n̶g̶ ̶s̶o̶f̶t̶w̶a̶r̶e̶ ̶i̶s̶ ̶t̶h̶a̶t̶ ̶t̶h̶e̶ ̶s̶o̶o̶n̶e̶r̶ ̶y̶o̶u̶ ̶s̶t̶a̶r̶t̶ ̶c̶o̶d̶i̶n̶g̶,̶ ̶t̶h̶e̶ ̶l̶o̶n̶g̶e̶r̶ ̶t̶h̶e̶ ̶p̶r̶o̶j̶e̶c̶t̶ ̶i̶s̶ ̶g̶o̶i̶n̶g̶ ̶t̶o̶ ̶t̶a̶k̶e̶.̶ ̶T̶h̶e̶y̶ ̶a̶d̶v̶i̶c̶e̶ ̶a̶ ̶l̶e̶n̶g̶t̶h̶y̶ ̶p̶r̶o̶c̶e̶s̶s̶ ̶o̶f̶ ̶d̶e̶t̶e̶r̶m̶i̶n̶i̶n̶g̶ ̶t̶h̶e̶ ̶u̶s̶e̶r̶'̶s̶ ̶n̶e̶e̶d̶s̶,̶ ̶e̶s̶t̶a̶b̶l̶i̶s̶h̶i̶n̶g̶ ̶t̶h̶e̶ ̶f̶e̶a̶t̶u̶r̶e̶ ̶s̶e̶t̶,̶ ̶d̶e̶c̶i̶d̶i̶n̶g̶ ̶h̶o̶w̶ ̶y̶o̶u̶'̶l̶l̶ ̶s̶t̶r̶u̶c̶t̶u̶r̶e̶ ̶t̶h̶e̶ ̶c̶o̶d̶e̶,̶ ̶a̶n̶d̶ ̶o̶n̶l̶y̶ ̶s̶t̶a̶r̶t̶i̶n̶g̶ ̶w̶r̶i̶t̶i̶n̶g̶ ̶t̶h̶e̶ ̶t̶h̶i̶n̶g̶ ̶w̶h̶e̶n̶ ̶t̶h̶a̶t̶'̶s̶ ̶a̶l̶l̶ ̶p̶r̶e̶t̶t̶y̶ ̶c̶l̶e̶a̶r̶.̶ ̶I̶ ̶c̶o̶u̶l̶d̶ ̶s̶e̶e̶ ̶t̶h̶e̶ ̶h̶a̶l̶f̶-̶b̶a̶k̶e̶d̶ ̶p̶r̶o̶d̶u̶c̶t̶ ̶b̶e̶i̶n̶g̶ ̶p̶a̶r̶t̶ ̶o̶f̶ ̶t̶h̶e̶ ̶"̶d̶e̶t̶e̶r̶m̶i̶n̶i̶n̶g̶ ̶t̶h̶e̶ ̶u̶s̶e̶r̶'̶s̶ ̶n̶e̶e̶d̶s̶"̶ ̶s̶t̶e̶p̶.̶
↑ comment by [deleted] · 2020-06-24T01:32:44.131Z · LW(p) · GW(p)
This is not true? It seems to me that iterative agile approaches are much more popular right now, and advise explicitly against this kind of waterfall process.
Replies from: AllAmericanBreakfast, jmh↑ comment by DirectedEvolution (AllAmericanBreakfast) · 2020-06-24T01:44:37.683Z · LW(p) · GW(p)
That's great to know. I'm learning to code but am getting all my advice off the internet - never worked in the industry. Guess it's been some bad advice I've been reading!
Replies from: Viliam, T3t↑ comment by Viliam · 2020-06-24T19:52:52.690Z · LW(p) · GW(p)
At the beginning of the project, ask yourself this:
- do I know all relevant facts about the planned project?
- what is the chance that something is wrong, because the customer didn't think about their needs sufficiently, or forgot something, or there was a miscommunication with the customer?
- what is the chance that during the project either the customer will change his mind, or the external situation will change so that the customer will need something different than originally planned?
If you are confident that you know all you need to know, and the situation is unlikely to change, then I would agree: a week of planning can save a month of coding.
Problem is that this type of situation happens frequently at school, but rarely in real life. During 15 years of my career, it happened to me twice. Those were the best projects in my life. I did some research and planning first, then wrote the software according to the plan. Relaxed pace, low stress. Those were the good times.
Unfortunately, most situations are different. Sometimes there are good reasons. Most of the time, in my opinion, someone just didn't do their job properly, but it's your problem anyway, because as a software developer you are at the end of the chain. You have two options: start your own company, or develop proper learned helplessness and accept the lack of analysis and planning as a fact of life.
Yes, this opinion is controversial. What happened, in my opinion, is that at first, there were good insights like "mistakes happen, circumstances change, we should program in a way that allows us to flexibly adapt". However, when this became common knowledge, companies started using it as an excuse to not even try. Why bother doing analysis, if you can just randomly throw facts at programmers, and they are supposed to flexibly adapt? What was originally meant as a way to avoid disasters became the new normal.
Now people talk a lot about being "agile". But if you study some agile methodologies, and then look at what companies are actually doing, you will notice that they usually choose a subset that is most convenient for the management. The parts they choose are "no big analysis" and "daily meetings". The parts they ignore are "no deadlines" and "the team decides their own speed". (With automated testing they usually go halfway: it is important enough so that there is no need to hire specialized testers, but it is not important enough to allocate a part of budget to actually doing it. So you end up with 5% code coverage and no testing, and if something breaks in production, it's the developer's fault. Or no one's fault.) This is how you get the usual pseudo-agile, where no one makes proper analysis at the beginning, but they still specify the deadlines when the yet-unknown functionality must be completed. Now the team is free to choose which features they implement on which week, under the assumption that after six months they will implement all of them, including the ones even the management doesn't know about yet.
Yep, I am quite burned out. Seen too much bullshit to believe the usual excuses.
Anyway, even on shorter scale it makes sense to plan first and code later. If your "sprint" takes two weeks, it still makes sense to spend the first day thinking carefully about what you are going to do. But again, the management usually prefers to see your fingers moving. Thinking without typing may result in greater productivity, but if often creates a bad impression. And where productivity is hard to measure, the impressions are everything.
↑ comment by RobertM (T3t) · 2020-06-24T06:47:45.243Z · LW(p) · GW(p)
I wouldn't say it's bad advice; it depends heavily on the context of the work. In an environment where you have some combination of:
1) a tight feedback loop with the relevant stakeholder (ideally the individual(s) who are going to be using the end product),
2) the product itself is amenable to quick iteration (i.e. composed of many smaller features, ideally with a focus on the presentation),
3) the requirements aren't clear (for example, the client has a mostly intuitive sense of how certain features should work; perhaps there are many implicit business rules that aren't formally written down anywhere but will come up as obviously "oh, it's missing the ability to do [x]" as the product gains capabilities)
...then avoiding significant investment in upfront design and adopting an iterative approach will very often save you spending a bunch of time designing something that doesn't fit your stakeholder's needs.
On the other hand, if you're operating in an environment where those conditions don't exist, such as one where you're mostly working on features or products that aren't easily broken down into smaller components that can be individually released or demoed to a stakeholder, and you have fairly clear requirements upfront that don't often change (or you have access to a product manager who you work with to iterate on the requirements until they're sufficiently well-detailed), then doing upfront design can often save you a lot of headache in wandering down dark alleys of "oops, we totally didn't account for how we'd incorporate this niche but relatively predictable use-case, so we optimized our design in ways which makes it very difficult to add without redoing a lot of work".
Having some experience with both, I'll say that the second seems better, in the sense that there are fewer meetings and interruptions, and the work is both faster and more pleasant, since there's less context-switching, conditional on the planning and product design being competent enough to come up with requirements that won't change too often. The downsides when it goes wrong do seem larger (throwing away three months of work feels a lot worse than throwing away two weeks), but ultimately that degenerates into a question of mitigating tail risk vs optimizing for upside, and I have yet to lose three months of work (though I did manage to lose almost two consecutive months working at an agile shop prior to this, which was part of a broader pattern that motivated my departure). I would recommend side-stepping that by attempting to find a place that does the "planning" thing well; at that point whether the team you're on is shipping small features every week or two or working on larger projects that span months is more a question of domain rather than effective strategy.
↑ comment by jmh · 2020-06-25T16:07:53.563Z · LW(p) · GW(p)
That was the experience I had in the last corp I worked for. They traditionally were waterfall and trying to move towards agile. As T3t notes below, it's best not to think of things as either or (and I suspect you are not suggesting such).
A couple of related thought come to mind though. One clearly is the cost of the bug and effort to fix -- I don't think all environments support ease of update to code base or code modules. Additionally, in different settings having something go wrong for 30 minutes might be a minor inconvenience (I'll do something else this morning and come back this afternoon) while in other cases you might be talking about billions in damage/loses or even lives lost.
There is also something of a culture aspect here. Organizations and the staff who lived and breathed waterfall have a lot of business processes & procedures and human thought process in place that don't really support agile, and vise versa.
However, I think the approach in the OP is fully compatible with either waterfall or agile development. But it might actually be more valuable to the former. Might also generalize pretty well into things like, say, vacation planning????
Replies from: Liron↑ comment by [deleted] · 2020-06-24T01:33:46.849Z · LW(p) · GW(p)
Also I want to point out that, perhaps against better design judgment, in actual industry most of modern software engineering has embraced the "agile" methodology where the product is being iterated in small sprints. This means that the design team checks in with the users' needs, changes are made, tests are added, and the cycle begins again. (Simplifying things here, of course.)
It was more common in the past to spend much more time understanding the clients' needs, in what is termed the "waterfall" methodology, where code is shipped perhaps only a few times a year, rather than bi-weekly (or whatever your agile sprint duration is).
Replies from: Liron↑ comment by Liron · 2020-06-24T02:07:09.171Z · LW(p) · GW(p)
Related to this, here’s my site dedicated to explaining how Minimum Viable Products should be more half-baked and less bloated: http://bloatedmvp.com