The Power to Judge Startup Ideas

post by Liron · 2019-09-04T15:07:25.486Z · LW · GW · 28 comments

Contents

    This is Part III of the Specificity Sequence
  Startup Steves
  Every Startup's Demolishable Claim
  The Value Prop Story Test
  The Extra-Powerful Sanity Check
None
28 comments

This is Part III of the Specificity Sequence [LW · GW]

When Steve claims [LW · GW] that Acme exploits its workers, he's role-playing the surface behaviors of an opinionated intellectual, but doesn't bother to actually be an opinionated intellectual, which would require him to nail down a coherent opinion.

It turns out that a lot of startups are founded by people doing something analogous to Steve: they role-play the surface behaviors of running a company and building a product, but don't bother to nail down a coherent picture of what customers would ever come to their business for.

Startup Steves

Paul Graham, cofounder of Y Combinator, calls this failure mode one of The 18 Mistakes that Kill Startups:

Having No Specific User In Mind
A surprising number of founders seem willing to assume that someone, they’re not sure exactly who, will want what they’re building. Do the founders want it? No, they’re not the target market. Who is? Teenagers. People interested in local events (that one is a perennial tarpit). Or “business” users. What business users? Gas stations? Movie studios? Defense contractors?

I'm in the startup industry, and I watch a lot of startups committing suicide by not being specific enough about who their customer is. From my perspective, the failure to be specific isn’t just a top-18 mistake, it’s the #1 mistake that founders make.

If you watch Paul Graham Office Hours at Startup School 2011, you can see for yourself that most of the founders on stage don’t seem to have a specific idea of who they’re building their product for and what difference it makes in their lives. Eliezer observes [LW · GW]:

There was an exchange in Paul Graham [and Harj Taggar]’s office hours that went like this, while interviewing a startup that did metrics — analyzing pageviews, roughly — and the entrepreneur was having great trouble describing what they did that Mixpanel didn’t. It went on for a while. It was painful to watch.

Paul: I don’t get what the difference is. I still don’t get what the difference is. What’s the difference between you and Mixpanel?

Entrepreneur: The difference is — when you have to supplement — they’re a view company and we’re a platform. That’s what it comes down to. They’re like a view, a reporting company. If you need something they don’t have, a feature -

Harj: So what’s an example of somewhere you’d use your thing over Mixpanel? Can you give a use-case?

Entrepreneur: Yeah, I mean, we had revenue on day zero. There’s a good reason for um… it’s a start up, it’s a series A company in the daily deals space. One we’ve signed a social game company to -

Harj: And why do they prefer your thing?

Paul: That wasn’t what Harj was asking.

The problem (from the perspective of our present discussion) is that the Entrepreneur did not understand that Paul and Harj were repeatedly asking him to move downward on the ladder of abstraction. When the Entrepreneur said “We had revenue on day zero”, he was trying to offer confirmation of the abstract statement “We can do things Mixpanel can’t”, but Paul and Harj still had no idea what his startup actually did.

How many early-stage startups have no specific user in mind? I’d guess about 80% of them. And how bad is not having a specific user in mind? So bad that I don't think they should even be considered a real startup, in the same way that Steve's argument about Acme wasn't a real argument.

Every Startup's Demolishable Claim

Every startup founder makes the same claim to themselves and to the investors they pitch for funding: "We're going to make a lot of money." So what I do, naturally, is ask the founder to furnish a specific example of that claim: a hypothetical story about a single person who might be convinced to pay them a few bucks. And here's how the conversation usually goes:

Founder: We're going to make billions of dollars and have millions of users!

Liron: Ok, what's a hypothetical example of how you give one specific user some value?

Founder: [Nothing]

Maybe they don't literally say nothing, but they say something that doesn't count for one of these reasons:

• They answer in the abstract instead of giving the example I requested of how they might give value to a specific user
• They choose a specific example wherein their startup's product or service isn't any better for their hypothetical user than the user's available alternatives

At this point, I understand if you think I'm just knocking down a straw man, so here's a real example.

Golden is a 2-year-old startup with $5M in funding from Andreessen Horowitz, Founders Fund, and other notable investors. Their product is intended to be a superior alternative to Wikipedia.

Here's an excerpt from the conversation I had with Golden's founder, Jude Gomila, on Twitter:

Liron: What specific use case exists on Golden today which is better than could have been achieved if the same amount of writer-effort had been spent on a pre-existing platform?

Jude: Quick tldr on this, some points covered in the blog post, however, 1. 1000x the topic space as a mission 2. removal of notability req 3. Using AI to automate flows 4. Using AI to compile knowledge 5. Better fact validation/hi res cites 6. Better schema eg timeline 7. Features and functions eg faving, activity feed, parallel rabbit hole 8. query results like these as well plus many many more. Have you tested the editor: magic cells, citations product and AI suggestions?

As for the specific examples Jude provided in response to my question... well, I'll just give you the first two and you can judge them for yourself:

1. https://golden.com/wiki/Cryobacterium vs https://en.wikipedia.org/wiki/Cryobacterium

2. https://golden.com/wiki/Ginkgo_Bioworks vs https://en.wikipedia.org/wiki/Ginkgo_Bioworks

Golden has received more funding than startups normally get before having any market traction to show, and the company's high profile makes it a juicy example to illustrate my point here. But there are countless other companies I could have singled out instead. Remember, the majority of early-stage startups are operating in this same failure mode. There are enough examples of startups visibly failing this way that I've started a blog to collect them.

The Value Prop Story Test

When I chat with a founder about their new startup, or I look through the slide deck that they're using to pitch their idea to investors, the first thing I do is try to pull out what I call a Value Prop Story: one specific story wherein their startup gives somebody some value.

A well-formed Value Prop Story must fit into this template:

1. Describe a specific person with a specific problem
2. Describe their current best effort to solve their problem
3. Describe why it’s still a problem
4. Describe how their life gets better thanks to you

I've previously observed that telling a well-formed Value Prop Story doesn’t require you to show any market research or empirical evidence validating the quality of your idea. This is like how Steve didn't yet need to give us any empirical or theoretical justifications for his claim about Acme worker exploitation, he just needed to tell us a story about one hypothetical specific worker getting exploited in a specific way.

Who is a specific hypothetical person who will use your product, and in which specific scenario will they use it? That's it, that's the question most startups can’t answer.

Answering this question seems objectively easy to me, in the sense that a well-designed AI wouldn't stumble over it at all. What about for a brain [LW · GW] though, is it a tough mental operation?

Actually, I think you'll find that this is an easy mental operation if you actually have a good startup idea. Here's a Value Prop Story I wrote about my own startup without much trouble:

1. Describe a specific person with a specific problem
   23 year old male who can’t get a date
2. Describe their current best effort to solve their problem
   He gets a Tinder account and does his best to use it on his own
3. Describe why it’s still a problem
   His matches barely respond to his messages, and when they do, the conversation feels boring and forced. He uses it for 1 hour every day but only manages to get 1 date every 2 months.
4. Describe how their life gets better thanks to you
   Once Relationship Hero coaches guide him through writing his texts, he suddenly has much better conversations that result in a date each week

Since my startup actually has a broad range of use cases (clients come to us for help with a broad range of relationship issues), this Value Prop Story isn't particularly representative of what we do. Its job was merely to prove that there are more than zero plausible specific use cases for Relationship Hero, and it gets that job done.

Given how easy this exercise is - we're talking five minutes, tops - I find it mind-boggling that 80% of startups recklessly skip it and go straight to, um… whatever else they think startups are supposed to do. Paul Graham writes:

Another of the characteristic mistakes of young founders is to go through the motions of starting a startup. They make up some plausible-sounding idea, raise money at a good valuation, rent a cool office, hire a bunch of people. From the outside that seems like what startups do. But the next step after rent a cool office and hire a bunch of people is: gradually realize how completely fucked they are, because while imitating all the outward forms of a startup they have neglected the one thing that's actually essential: making something people want.

Why would you spend time and money building a product when you can’t yet tell a specific Value Prop Story? I think it's because designing and building a product is fun and gives you a false sense of control. You can lie to yourself the whole time about the likelihood that you’ll eventually get people to use what you’re building.

But people usually won’t use what you’re building. Whenever a new startup excitedly launches their product for the first time, the most likely outcome is that they get literally zero users.

The Secret famously claimed that wishing for something makes the universe give it to you, which is BS, but the converse is true: If you haven’t made a specific enough wish about what your initial market traction is supposed to look like, then the universe won’t give you any traction.

The Extra-Powerful Sanity Check

Is it healthy for us to be obsessed with judging startups and demolishing claims about their value propositions? When we say that a startup idea is bad on account of lacking a Value Prop Story, is it right and proper to feel pleased with ourselves, or are we being gratuitously adversarial?

Along these lines, Mixpanel cofounder Suhail Doshi has tweeted:

I get little satisfaction stomping on someone's startup idea. It's so easy to. Somewhere deep, hidden in their abstract description is a distinct yet narrow problem worth solving that's significant. It's more fun to attempt finding it, together.

I basically agree with this, and I basically agree with the commenters on my demolish bad arguments [LW · GW] post who emphasized that we should seek to shine a light on whatever kernels of truth our conversation partner may have brought to the table.

But...

Have you ever sanity checked something?

A sanity check is like when you punch 583x772 into your calculator, and you quickly multiply the two rightmost digits in your head, 3 x 2 = 6, and then confirm that the calculator's output ends in a 6. If you ever accidentally punch the wrong sequence of keys into the calculator, then you'll be pretty likely to see the calculator's answer end in something other than a 6. It's a good use of two seconds of your time to calculate 3 x 2; you get a substantial dose of Bayesian evidence for your trouble.

The Value Prop Story test is likewise a sanity check for startup ideas. In theory, of course a startup founder who is already hiring a team of engineers and building a software product should be able to describe how one specific user will get value from that product. In practice, they often can't. And it's easy for us to quickly check.

Here's what's crazy though: We usually expect sanity checks to have a low rate of detecting failures. You expect to successfully multiply numbers on your calculator most of the time, but you do the 3 x 2 sanity check anyway because it's quick. But with the Value Prop Story test, you'll see a high rate of failures!

A sanity check with a high failure rate is a rare treat; it's an extra-powerful sanity check. When you're lucky enough to have an extra-powerful sanity check in your toolbox, don't make it a final step in your process, make it the first step in the process.

So here's how you can use the Value Prop Story test to upend the traditional order of operations for building a startup: First, repeatedly sanity check yourself with the Value Prop Story test until you pass it. Second, do everything else. In all seriousness, I've recommended that early-stage startup founders follow this flow chart:

But how should we treat founders who are stuck in the flowchart's "Give Value to One Person" stage?

When someone is struggling to pass a sanity check, that doesn't mean we should write off their potential to succeed. It means we should focus our effort on helping them pass the sanity check.

Applying the Value Prop Story test is like placing a low bar in a founder's path. Yes, the bar will trip the ones who aren't seeing it. But for the ones who do see the bar, they can step up onto it and then be on their way. And the next step in their path, such as building a quality product, or building a sales funnel, is sure to be a steeper one than that little first one.

Next post: The Power to Make Scientific Breakthroughs [LW · GW]

Companion post: Examples of Examples [LW · GW]

28 comments

Comments sorted by top scores.