post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by Connor_Flexman · 2020-05-06T19:14:46.011Z · LW(p) · GW(p)

(grammatical edits for clarity)

Man, this post is special to me, because it is one of the most powerful rationality tools I know of and was the backbone of my own high-growth intro to rationality. So while I want to gesture at a patch for one problem I expect you will run into, I'm not that sure it's even worth paying attention to right now. Just running with the current skill seems fantastic. But, here's my concern for the future.

When you say "stop saying wrong things", most of your examples use logic to define "wrongness". Which makes sense. If you intend to learn Chinese and you don't check how long it will take you, but in fact you will never finish, logically you made a mistake. But if you intend on helming Etsy and don't run A/B tests to provide a feedback loop for whether you were correct about customer desire, you also logically made a mistake—despite the fact that I think this is a perfectly reasonable decision. I expect a fair number of these false positives to come up given the way you describe your current filter.

Before we get to what is going wrong, first we should examine a bit of what's going right. You're taking a situation where you have an urge to do your standard behavior, and noticing that you can pattern-match it to a case where people often do wrong things as defined by biases research, logic, and the people whom you read. These sources have a very good track record in many domains. When you then switch your behavior, you will often be right, as well as having explored some new behavior.

However, there are certain domains or classes of thing where logic does not do especially well, and likewise in which many people you read will probably give good-sounding advice which turns out to be wrong. I think startups is probably one of them. Zvi's post about Leaders of Men makes this point in a way which I like, using the example of baseball managers. There are definitely lots of "dumb" things that managers do that are easy to point out by a logician from the sidelines. These cost them some games. But the mistakes are driven by policies that are actually very powerful, more beneficial than the costs imposed on the games. I think the A/B testing example fits this. Yes, it helps, but it's not as helpful as running with other more important action policies, perhaps exploring design space or letting your designers' intuitions run or just focusing resources on management or who-knows-what (they probably do, though). A/B testing is optimizing, and you don't want to commit the sin of premature optimization. 5 years in sounds reasonably less premature to me.

So, to try to put a point on what goes wrong: logic has its weak points, as any straw-postmodernist will tell you, but they're obviously wrong if they say logic isn't patchable. But just because their pendulum has swung too far doesn't mean there aren't some classic mistakes with overapplying straw logic. I think premature optimization is a really good example. I think "ignoring complexity" is another good example. Garbage-in garbage-out modeling is another good example. A huge number of "biases" I think are actually the correct thing to do/think a significant fraction of the time. I think social skills, sports, dancing, music, politics, system design, etc are all sandboxes of complex domains in which logic doesn't work very well in practical usage, and we indeed see tons of mistakes in them by both straw-rationalists and real ones.

Maybe I sound like I'm preaching to the choir here. But I think there's a subtle-ish point I would still like to get across: if you override a behavior with logic, the original behavior was basically always in place for a very good reason. Your behaviors are built on each other. Immediately stopping a behavior will hurt behaviors on top of it or supported by it. For the general example, halting "saying wrong things" may cause you to stop putting models out there to be destroyed by reality, which could hamper the feedback and growth process. There are a plethora of more specific ones (e.g. halting "using little white lies" is great to explore but can often cause jarring but hard-to-identify ways people won't feel as comfortable with you).

I think the solution here is something like "while you're exploring what logic says to do instead, also explore heavily all the reasons the first action was being done, because your neural net is complex as shit and who knows what processes you may accidentally deprecate" (and sometimes, you'll find fascinating new subgoals and important dynamics you didn't know existed!). Running with the logical action is great because it's growthy and you can smash the model into reality, but *don't cling onto the naively logical model if reality if reality is hinting it's more complex than that*. That's the sinkhole to really avoid. Any other mistakes are fixable.

Excited to see where this takes you.

comment by TurnTrout · 2020-05-02T02:19:47.778Z · LW(p) · GW(p)

Meta: I like this post. I would appreciate if downvoters were willing to explain their disagreement / why they want to see fewer posts like this on the website (because that's what the downvote means de jure [? · GW]).

Replies from: Spiracular
comment by Spiracular · 2020-05-12T23:49:39.782Z · LW(p) · GW(p)

I didn't downvote it, but I did find the writing style mildly grating. (Relatedly: I cannot exceed your set-point of smug, it is over 9000 :) )

To be clear: I think the underlying point was pretty good, and I mostly had issues with the delivery. I still feel it was probably something worth writing, although I also think I'm not the target audience for this particular bit of advice.

Some of it was probably a tone thing, which I won't go into. But here are some things that seem tractable:

My experience of it was a bit better as soon as I switched out almost all the "You"s for "I"s. I have something of a distaste for the... puppety-feeling where someone seems to be trying to put words into my mouth, that don't fit with my actual experience. This set it off pretty badly. There are a lot of specifics, and it's clearly your personal account; own it.*

This got really stark for me at around...

First, you just end up being an asshole pretending to be honest

...which instantly broke my immersion. My experience of being painfully honest with myself, and then others, was radically different.**

It probably also could have used more short paragraphs, and some variety in presentation. Some of the goals you've acted on and then forgotten, such as Real Analysis or Mandarin, could have been better-presented as single bullet-points after going into only 1 of them in-depth. The Etsy section could have used a header, and been broken into more than 1 paragraph. That kind of thing.

*A lot of bad advice on persuasive essay writing encourages the formation of habits like this. One cannot list the number of times one has been told to make that unnatural substitution of "One," where "I" would have been better, and more honest. Teachers who do this are just... wrong. Technical writing is a real thing, but this way of teaching it is crap, and can ruin otherwise-decent writers.

** My experience was close to painful self-consciousness (for self-honesty), and weird social penalties (for honesty with others). Real honesty is often distinctly un-charming, but in my case... bluntness leaned closer to "overly-invested*** eccentric" than "asshole." If it had been framed as a self-account, this jarring wouldn't have been an issue.

*** Exhibit A: This overgrown write-up.

Replies from: TurnTrout
comment by TurnTrout · 2020-05-13T00:11:01.204Z · LW(p) · GW(p)

Strong upvoted, although I didn't have the same issues with the delivery.

Getting downvoted without discussion can be disheartening, especially for high-effort posts. For anyone wondering: when I wrote the original comment, this post was sitting at 3 karma and ~7 votes.

comment by Vaniver · 2020-05-02T17:44:06.892Z · LW(p) · GW(p)

See Paul Graham on Robert Morris. I also remember a blog post (discussed on LW) that I thought was called "stop making stupid mistakes", which wasn't this one, but instead was about someone who was okay at chess talking to a friend that was good at chess about how to get better, and getting the unpalatable lesson that it wasn't about learning cool new tricks, but slowly ironing out all of the mistakes that he's currently making. 

Replies from: Nisan
comment by Nisan · 2020-05-03T07:39:37.822Z · LW(p) · GW(p)

Also Dan Luu's essay 95%-ile isn't that good, where he claims that even 95th-percentile Overwatch players routinely make silly mistakes, suggesting that you can get to that level by not making mistakes.

Replies from: TurnTrout
comment by TurnTrout · 2020-05-03T16:57:16.915Z · LW(p) · GW(p)

Back in the day, I was a 50th percentile player. I spent a couple hours watching a 99th percentile streamer, and then just tried to play like them. I thought, what would MoonMoon do, and then i would do that. I won the next 19 games, ending up in the 97th percentile. I stayed there.

Even so, I'd review my gameplay videos and be stunned by how many stupid mistakes I still made, even at this "high level" of play.

comment by [deleted] · 2020-05-02T03:04:15.698Z · LW(p) · GW(p)

I think you've hit on one of the core problems and some of my own dissatisfactions with how instrumental rationality is being treated here. (Quick plug is that I've tried to do some theorycrafting on what I think are the relevant issues here).

Specifically for "Why doesn't LessWrong cover this stuff?", I think that CFAR goes over some of these things.

For me personally, I think a big part of solving parts of the problem you describe (e.g. noticing that your brain is making up excuses to not exercise, rearchitecturing an app, etc. etc.) are things that feel like "shoulds" to me. Ideally, what I'd like to do is to have my gut feel like these things are worth doing in the same way that I "already know". I think techniques to make this happen are where some of the CFAR content like Internal Double Crux can be relevant.

For all the other stuff (e.g. noticing when you should leave university if you don't have reasons to stay, learning stuff you when you don't have a solution to the forgetting curve, etc. etc.), I'm pattern-matching this to instances where paying more attention to your situation and asking the right questions about what you're doing / why you're there is important. To that end, I think things like regularly scheduling time to be attentive / update your plans is useful. Evidence-backed interventions you can do can be found in the psychological literature. (see here for planning and here for habits.)

I'm having some difficulty reconciling the above two types of issues with your thesis about not saying wrong things. I think you're pointing to something similar to what I think about when your brain mixes up generative vs recognizing and you end up tricking yourself into believing something wrong (but tempting).

Would definitely love to see extensions to this idea + some more details on the things you listed in your 2nd to last paragraph on what's worked for you!

comment by Max Hodges (max-hodges) · 2020-05-05T17:36:55.659Z · LW(p) · GW(p)
Most people just don't even try to believe correct things, and make major life decisions based on transparently bad logic.

That's quite an assertion. What's your source? Evidence? Or is this mere conjecture (here, in this sacred space!)? ;) You advise others to only say true things, but why are you sure this is true?

There is a lot of ambiguity around major life decisions. Many decisions aren't a matter of rationally weighing all the facts, but more a matter of analyzing a bunch of compromises and then rolling the dice. Many major decisions can be reversed later anyway.

I think most people, most of the time, make the best decisions for themselves that they know how. If someone's decision looks like transparently bad logic to you, why not ask them about it?

No one has "full knowledge and consciousness" (whatever that means to you). We're complicated beings with a multitude of urges and impulses.

No one needs to be a rationalist to understand these pathologies, and no one needs to be a rationalist to recognize them. Think of all the people who start and don't finish learning a new language, yet with clear intention at the beginning to make full use of time well spent. Do most of these people actually believe they will finish learning Chinese?

Failing to master a foreign language is a "pathology"? People experiment. They try things to see if they'll enjoy them enough to stick with them. I've tried to learn how to draw at various time in my life, but never made it a habit for very long. I guess it wasn't that important to me. I quit watching movies and Netflix serials before the end. I don't finish a lot of books that I start, and I don't view this as a failing. If I don't feel I'm getting much value out of that book, I'll pick up a different book.

There is only so much time in the day. It's not pathological to change your goals and priorities. There is no shame in quitting something. I proudly quit things all the time. I trained intensely as a freediver. I met interesting people, and learned more than I ever imagined I would. It was very enriching on many levels. I learned to hold my breath for five-and-a-half-minutes, entered and won a competition, and ranked in the top 10 here in Japan where I live. Then I quit, just like that. I felt that I learned about as much as I was going to learn and decided to free up my time. Same with photography. I got intensely into photography for a few years. Had a studio with +$30K in equipment. My photos landed on Page One of the NY Times, twice. Then I quit and sold off all my photo equipment to make time for other activities. So what? I started studying Chinese for 6-weeks while living in Beijing, then I stopped because I was traveling a lot, moved back to the US, and never continued with it. Am I pathological? Irrational? Did I do something wrong?

Etsy as an organization, did not care or want to be correct at predicting what its users would want,

People have different skill-sets. Maybe math is your thing, maybe networking or operations management, culture building, or something else was the Etsy founder's skill they got them started. You pick things up along the way. I'm been running a company for more than 10 years, and I'm only recently getting into doing a deeper analysis of our customer behaviors and fees. But they didn't stop me from the success I've had up until now. You don't have to be very intelligent to be successful in business. Look at Jack Ma.

Have you ever run a fast-growing startup? There are a metric-shit-ton of things you could be working on. Multiple fires are burning. No one has the time or the resources to do all the things they could be doing. You also have to to look at all the things Etsy did instead of A/B testing, like working on their platform, building their company culture, fund-raising, etc. I'm running multiple e-commerce companies with a team of 20. I've never done a proper A/B ever, yet I've served tens of thousands of customers in 100+ countries. I could get to work right away on those A/B tests, but everything comes with opportunity costs. Time I spend on the A/B test project is time I could spend doing R&D for our new data model, mentoring other managers, doing financial analysis, streamlining operations, or trying to give advice to some stranger on Less Wrong. What's the most important thing I should be doing this very moment? Maybe we should ask Jack Dorsey. He seems to have his priorities and daily schedule all sorted out, but then he has the personality of a wet towel, so maybe we shouldn't ask him ;) (And maybe being extremely-efficient all the time isn't the best way develop as a human.)

You could go to any company and tell them 10 things they should be doing, and they will probably tell you, "I have a list of hundreds of things I should be doing. But I only have so many resources and so many hours in the day. Having a strategy is about what you will not do as much as what you will do.

You know, there really is nothing in my brain that changes the dynamics of the choice I'm about to make. No matter what I say to myself about the impeding circumstances, if I don't somehow exercise today, I'm going to be very slightly physically weaker by tomorrow and have a very slightly shorter life expectancy.
I pretty much work out as often and as intensely as I figure is aligned with my goals now. I just do it, because I know working out is a good idea and I know it's usually still a good idea regardless of the weather that day. It's amazing the quality of life increases I have made by merely not trying my damnedest to believe incorrect things.

I don't follow this line-of-argument. Are you saying you didn't know that you'd be weaker if you stopped working out? Surely you knew that. Your story doesn't sound like an a matter of "having correct information," but an issue of resolve. Motivation is complex. We can't just reach into our brains and flip a switch--and that's a good thing. It's how nature prevents one part of the mind from shutting down all the other parts. Instead, when motivating oneself, we have to rely on some of the same strategies and tactics we use when motivating others.

The domain name of this website would lead me to believe most of you are attempting the same, but I don't see a lot of the discussion about this that I would expect. I don't see people discussing the same problems that I fight. The books and skills I pour hours and hours into, then forget; the university I study at because a friend I no longer hang out with decided to attend; the job I take without applying anywhere else, researching salary or internship expectations, or learning any negotation or marketing strategies. How distant and conceptual the prediction market is in the face of my absurd self-sabotage. How foreign timeless decision theory. Am I so unique? I refuse to believe it.

If you want some input on specific topics like this, perhaps they work better as individual posts.

I could share my process for getting the most value out of the books I read. Maybe post that question separately and send me a link.

But reading over that list, it sounds like the general issue is a matter of setting clear priorities. Not everything can be a priority. So perhaps decide your top priorities for the next six weeks, and don't beat yourself up about all things that you didn't make a priority. I recommend taking some quiet time. Take long walks to clear your head. Mindfulness meditation is also a great way to clear your head. Start journaling. Write for 15 minutes a day in a notebook with a pen (not on your iPad or mobile phone). It doesn't matter if you're handing writing is terrible--you never really go back and read the stuff. It more about cultivating a habit of making time to think deliberatively and self-reflect.

Some starter topics you could journal about (+15 minutes each)

  • What's your current context? Write a snapshot of your life at the moment.
  • What excites you about life? What brings you alive? When do you experience flow? What are you passionate about?
  • What do you love? What are the things you couldn't live without? What are your favorite things? What flavors/sights/sounds/textures do you love? What lights you up, turns you on? What delights you?
  • How do you celebrate life? Maybe you have rituals or traditions? What do you do to rest or play and honor yourself?
  • What's important to you? Why?
  • What do you need for self-care to keep you resourced / healthy / fit / resilient?
  • Describe who you are when you're in flow? What is life like? How do you treat others? How easy is it to accomplish things?
  • What would someone write about you in a letter of recommendation? What do other people appreciate about you?
  • What are you doing that you should stop doing?
  • What am you not doing that you should start doing?
  • Write a letter to your future self one-year from now.

Further reading:

What’s All This About Journaling?

One of the more effective acts of self-care is also, happily, one of the cheapest.

https://www.nytimes.com/2018/10/25/style/journaling-benefits.html

When setting your priorities, for each thing you decide to do, take a sheet of paper and write down all the reasons why you've chosen this thing as a priority. How does it fit in with your goals and values?

Replies from: None, lc
comment by [deleted] · 2020-05-05T18:08:58.608Z · LW(p) · GW(p)

First couple of sentences seem reasonable, something I was thinking but didn't comment. However, the rest of this seems needlessly aggressive. I'd almost recommend pairing down to the limited critique and fleshing that out in more detail.

Replies from: max-hodges, lc
comment by Max Hodges (max-hodges) · 2020-05-06T02:56:11.978Z · LW(p) · GW(p)

OK I've taken your advice. I toned it down and elaborated. Thanks

comment by lc · 2020-05-06T00:04:39.054Z · LW(p) · GW(p)

See commenting guidelines

comment by lc · 2020-05-05T23:58:45.348Z · LW(p) · GW(p)
That's quite an assertion. What's your source? Evidence? Or is this mere conjecture (here, in this sacred space!)?

Everything Robin Hanson/Eliezer Yudkowsky/Richard Thaler has written + https://i.kym-cdn.com/photos/images/facebook/001/820/208/0d5.jpg

Jesus! Failing to master a foreign language is a "pathology"? Lighten up, man. People try things to see if they'll enjoy them.

With regard to foreign languages, I disagree. Most people study foreign languages so they can use them, or at the very least most wouldn't try if they knew from the outset that there'd be no lasting utility to doing so. Language learning usually involves large time commitments and tedious amounts of memorization. If you study a foreign language because you think it will be useful or cool, and then don't end up speaking it to a degree that makes up for the time invested, then yes, that is a mistake in hindsight. That happens to be the experience of everyone I know, save a linguist friend of mine, that has studied a new language without moving to the country where they speak it. None of your statements are really analogous, because clearly you accomplished lots, and consider the journey worth the effort.

Have you ever run a fast-growing startup? There are a metric-shit-ton of things you could be working on. Multiple fires are burning. No one has the time or the resources to do all the things they could be doing. You also have to to look at all the things Etsy did instead of A/B testing, like working on their platform, building their company culture, fund-raising, etc. I'm running multiple e-commerce companies with a team of 20. I've never done a proper A/B, yet I've served tens of thousands of customers in 100+ countries. I could get to work right away on those A/B tests, but everything comes with opportunity costs. Times I spend on the A/B test project is time I could spend doing R&D for our new data model, mentoring other managers, doing financial analysis, or streamlining operations.

I think you mistook the A/B test thing as the only thing in that slideshow that was wrong with the way Etsy was prioritizing development for those first five years. A/B testing can absolutely be a wrong move depending on how many data points you could be collecting and what your programmers could be doing otherwise. I'm trying to build a startup right now and we don't do A/B testing because we don't have enough "interactions" on a regular basis to make it cost effective. But if you let your programmers start new, multi-month projects without asking quickly answered questions like "do we have enough customers who buy furniture for this feature to be worth maintaining or building", then that's clearly bad. It wasn't just that Etsy was too busy preventing the house from burning down to worry about building an A/B test infrastructure, they were advancing new, spurious features without thinking about them.

And clearly they were unimaginably successful anyways. Part of my point in the post is to show how poorly you can do things on the "make sure your decisions are correct" axis and still win hard.

I don't follow this line-of-argument. Are you saying you didn't know that you'd be weaker if you stopped working out? Surely you knew that. This doesn't sound like an a matter of "having correct information," but an issue of resolve. Motivation is complex. We can't just reach into our brains and flip a switch--and that's a good thing. It's how nature prevents one part of the mind from shutting down all the other parts. Instead, when motivating one's self, we have to rely on some of the same strategies and tactics we use when motivating others.

My point with the anecdote is that much of the time this is nonsense. I spent so much time twisting my brain into an M.C. Escher painting to try and push myself to take actions I knew were good for me, that I often forgot to try honestly bringing the costs and benefits of the thing to the forefront of my mind and seeing if that would be enough. It turns out, quite often enough it is, and even further sometimes I realize that I don't actually value the thing I'm trying to convince myself to do enough to do it.

Replies from: max-hodges, max-hodges
comment by Max Hodges (max-hodges) · 2020-05-06T03:38:00.735Z · LW(p) · GW(p)

Hi, I've updated my post, toned it down, and added some new content. Hope that helps.

comment by Max Hodges (max-hodges) · 2020-05-06T04:20:25.199Z · LW(p) · GW(p)

Regarding the way Etsy was prioritizing development, it sounds like even their late stage "idea > validate > prototype" cycle is wrong. How can you "validate" before you have a prototype to get feedback on? Where did the idea come from? I'd recommend starting with customer discovery talks. Read "The Mom Test" to learn how to talk to customers about their problems. Then you can take those ideas into a design sprint to mock-up and prototype a feature so that you have something you can get feedback on.

But just because Etsy lacked some better ideas about how to optimize their product doesn't mean they didn't get important shit done. Within one year of its initial release, Etsy had gained 10,000 artists (craft makers), pitching 100,000 items, and had a market of approximately 40,000 buyers.

Etsy as an organization, did not care or want to be correct at predicting what its users would want,

In a word, I'd say it's probably not that Etsy didn't care about how to do these things better. The person in charge of development probably just didn't know how to do these things better. I'm certainly doing things differently today than I did 5 years ago. We live, we learn. It's a process.

Regarding learning a foreign language, I'm not sure what I can say. I speak Japanese, and I run a company which makes some top selling Japanese language learning products. So I know something about this topic. You're right, learning a foreign language is a big commitment. So isn't it obvious that the longer the required commitment, the most likely it is that people will drop out? The same is true for college:

According to College Atlas, 70% of Americans will study at a four-year college, but less than two-thirds will graduate with a degree, and 30% of first-year students drop out after their first year of school.

In the case of learning a foreign language, maybe over time the quitter just decided that the effort was no longer worth it. Maybe that want to prioritize other hobbies and interests. Or maybe they just don't enjoy memorizing hanzi. Maybe the idea moving to China or the important of talking to people in Chinese has lost its luster. What's wrong with changing your mind about these things?

If you study a foreign language because you think it will be useful or cool, and then don't end up speaking it to a degree that makes up for the time invested, then yes, that is a mistake in hindsight.

That's one way to look at it. But could you been letting sunk costs fallacy get the best of you? Here's another way to look at it: what if our beginner or intermediate student of Chinese decides that speaking Chinese is no longer an important goals of theirs? From that point on, even moment they continue to spend learning Chinese is a waste of time and a mistake.

Also, maybe a lot of people don't realize how much of a commitment learning a language really is. The internet is full misleading information and false promises perpetuated by get-rich-quick schemes and douche bags like Tai Lopez and Jim Kwik who try to separate people from their money at an industrial-scale with claims like "how I learned 5 languages in a year" and promises that you too can "take your dream trip and actually be able to speak fluently" normally for 5 easy payments of $79.99, now for the low price of $39.95! ;)

My point with the anecdote is that much of the time this is nonsense. I spent so much time twisting my brain into an M.C. Escher painting to try and push myself to take actions I knew were good for me, that I often forgot to try honestly bringing the costs and benefits of the thing to the forefront of my mind and seeing if that would be enough. It turns out, quite often enough it is, and even further sometimes I realize that I don't actually value the thing I'm trying to convince myself to do enough to do it.

This sounds like valuable progress in your thinking! Probably just writing about it has helped you see things more clear. I hope you'll go back to my edited post. I shared some ideas about starting a journaling habit for precisely this reason!

Replies from: lc, lc, lc
comment by lc · 2020-05-07T10:15:09.250Z · LW(p) · GW(p)
Regarding learning a foreign language, I'm not sure what I can say. I speak Japanese, and I run a company which makes some top selling Japanese language learning products. So I know something about this topic. You're right, learning a foreign language is a big commitment. So isn't it obvious that the longer the required commitment, the most likely it is that people will drop out?

Yes. It's very obvious. Which is why it's that much more unusual that people make these commitments without more consideration than they tend to put forward. People will intuit that they should do some significant (meaning more than a couple hours) hard research into whether or not it's worth it to get their masters, and where to get it; they tend not to realize that successfully learning a language as remote as Mandarin requires similar time investment as getting the masters degree, even though a necessary part of any plan to do so would be to figure that out, and I don't know why. I suspect that many of these people are just going through the motions with no real objectives in mind, and that's a running theme of my OP.

In the case of learning a foreign language, maybe over time the quitter just decided that the effort was no longer worth it. Maybe that want to prioritize other hobbies and interests. Or maybe they just don't enjoy memorizing hanzi. Maybe the idea moving to China or the important of talking to people in Chinese has lost its luster. What's wrong with changing your mind about these things?

As you say, nothing. The problem, at least in hindsight, was the decision to start in the first place. You can say the decision was correct given the information they had, but I'm trying to make the point that much (most?) of the time this is clearly not the case, and people engage in studying habits that will not ensure they learn the language in the next forty years.

Which is not to say that people shouldn't use your product or service. I personally think language acquisition has some pretty significant positive externalities - languages are more useful the more people speak them, after all - but we don't even need to go there. It's just that the way most people set about learning them often implies they're not even trying, to a degree that's difficult to make sense of, at least for me.

Replies from: max-hodges
comment by lc · 2020-05-07T10:16:04.349Z · LW(p) · GW(p)
This sounds like valuable progress in your thinking! Probably just writing about it has helped you see things more clear. I hope you'll go back to my edited post. I shared some ideas about starting a journaling habit for precisely this reason!

Perhaps >.>

comment by lc · 2020-05-07T08:54:25.680Z · LW(p) · GW(p)
Regarding the way Etsy was prioritizing development, it sounds like even their late stage "idea > validate > prototype" cycle is wrong. How can you "validate" before you have a prototype to get feedback on?

You need to be a little creative. I'll give a trivial example that will probably be more enlightening if you went through the proofs C.S. bachelor students have to do of evaluating upper bounds of algorithms.

Lets say you hear about how matrix multiplication is a big problem in ML, and think you have a way to build an algorithm that should give you O(n*sqrt(n)) performance. Your idea happens to be an unusually complicated algorithm and thus a involves a complicated product; you think it will take you around 50 +-20 hours to prototype, and you don't expect to achieve O(n*sqrt(n)) your first try. You could start your project now and build a prototype as quickly as possible, so that you can get a concrete impression of your algorithms viability. Or, you could first sit and consider for a couple hours and see if there are any ways to test or debunk the feasibility of your algorithm. If you choose the first option, you might get twenty hours in and realize your mistake only then. If you choose the second option, you might realize, just by pondering constraints for an hour, that matrix multiplication requires at least O(n^2) reads, and that your algorithm can't in any universe work as fast as you expected it to, so no need to build a prototype without further considering where you're going wrong.

Some ideas aren't very easily validated. Art (I assume though I am not an artist) is an area where you pretty much just have a difficult to convey imagined finished product, and the best way to start validating is to build it and see if people like it. Lots of seed stage startups are founded to build things no one wants. The reason the team got funding anyways is often because the utility of the product they're trying to create is difficult to "debunk" from the outset, or fails in hard-to-anticipate ways that aren't caught by standard sanity checking.

However, outside of a few of those specific cases, most good ideas can be thought about. It might be impossible to completely rule out or in your hunch without building it first. But just by thinking about a proposed solution for the first twelve hours, you can usually establish some sort of probabilistic upper bound on the expected value and lower bound on the cost of implementation/maintenance. For a project that you think is going to take several months, even if you think there's only a small chance you will figure out a flaw in its viability without prototyping, it's worth it to sit down and do that sort of research.

None of this excludes trying to improve the source of your ideas, prototyping, talking to users, or fast iteration. All ideas are validated, though. The reason ideas come to your attention and stay there in the first place, among other things, is because they seem upon reflection plausible enough to be a good bet. The point of adding validation to that iteration process explicitly is to increase the thoroughness of your search for existing information that could encourage or discourage your efforts proportional to the costs associated with attempting.

But just because Etsy lacked some better ideas about how to optimize their product doesn't mean they didn't get important shit done. Within one year of its initial release, Etsy had gained 10,000 artists (craft makers), pitching 100,000 items, and had a market of approximately 40,000 buyers.

I agree, and that's why I clarified with

And clearly they were unimaginably successful anyways. Part of my point in the post is to show how poorly you can do things on the "make sure your decisions are correct" axis and still win hard.

I realize how hyperbolic my criticism of Etsy sounds now that I needed this long ass post to explain their judgement errors, but I really think going into this much detail is making the technique seem much more complex than it really is. If you're planning on spending two months improving the revenue created by feature X by 3%, do the napkin math to see if existing revenue coming from X justifies two months salary. If you're planning on making a widget that improves the experience of furniture purchasers on your site, make sure there are enough furniture purchasers (taking into consideration the new ones you will be enabling by said widget) to justify maintenance/implementation. According to the guy I linked to, those were the kinds of sanity checks that Etsy wasn't doing, which personally seems difficult to imagine for people that are otherwise obviously intensely competent.

Replies from: max-hodges
comment by Max Hodges (max-hodges) · 2020-05-07T10:32:39.359Z · LW(p) · GW(p)

> If you're planning on spending two months improving the revenue created by feature X by 3%, do the napkin math to see if existing revenue coming from X justifies two months salary.

How do you know they didn't? That deck is just a summary of years of work. Perhaps reach out to Dan McKinley for further discussion. I'm afraid we've both just making a lot of speculation at this point. Talk to Dan.

When your revenue is 7-8 digits, 3% can add up! Sometimes it's such a no-brainer that it's not worth opening Excel over. Last week I had my devs spend a few days implementing a way to create DHL labels using an API. We didn't do any cost-benefit analysis because it was so clearly something that needed to be done. Today we shipped 147 packages by DHL. That would have taken about 5m each to do the old manual way, so in one day it saved 9.5 hours of staff time. Over one-year that's an entire salary.

But I think the presentation tells you why they weren't doing that stuff:

And you know what, if the site’s growth is really insane, it looks like it’s working. You can release things and as long as they don’t completely destroy everything it will look like you’re a genius. All the graphs will go up and to the right.

You can read in the Farnum St. interview with Tobias Lutke, CEO at Shopify, how it held back Shopify's growth. I've met Tobias and he's a brilliant, level-headed engineer and CEO, but if you read that you'd probably conclude he's irrational, stupid, and not ambitious enough for not relentlessly maximizing growth for that period. ¯\_(ツ)_/¯

Regarding testing and validation, of course there are exceptions, but generally speaking, it still looks like they got it backwards to me. You're free to disagree. I'm running multiple companies and have produced many products and services--only one big failure. I've tried a lot of approaches and still mix and match many ideas, but can solidly endorse making the prototype in get the feedback.

Maybe you misunderstand how we think about prototyping. A realistic façade is all you need to test with customers. The prototype give you something concrete to put in front of customers for rich feedback and insights. But think lightweight (dirty) version of key aspects of a product or experience. The prototype only needs to be good enough to test out a hypothesis and nothing more. It's all about testing big ideas with minimal upfront investment.

I recommend the design thinking methodology presented in "Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days." Invented at Google by Jake Knapp, perfected with more than 150 startups at Google Ventuer (GV).

Prototype comes before Test.

https://www.thesprintbook.com/how

Cheers,

comment by Max Hodges (max-hodges) · 2020-05-06T05:04:05.769Z · LW(p) · GW(p)

Speaking to the larger issue you raise, yes, anyone who thinks we are purely rational creatures is deluding themselves:

. . . they failed to appreciate that the self illusion explains so many aspects of human behavior as well as our attitudes toward others. When we judge others, we consider them responsible for their actions. But was Mary Bale, the bank worker from Coventry who was caught on video dropping a cat into a garbage can, being true to her self? Or was Mel Gibson’s drunken anti-Semitic rant being himself or under the influence of someone else? What motivated Congressman Weiner to text naked pictures of himself to women he did not know? In the book, I consider some of the extremes of human behavior from mass murderers with brain tumors that may have made them kill, to rising politicians who self-destruct. By rejecting the notion of a core self and considering how we are a multitude of competing urges and impulses, I think it is easier to understand why we suddenly go off the rails. It explains why we act, often unconsciously, in a way that is inconsistent with our self image – or the image of our self as we believe others see us.

That said, the self illusion is probably an inescapable experience we need for interacting with others and the world, and indeed we cannot readily abandon or ignore its influence, but we should be skeptical that each of us is the coherent, integrated entity we assume we are.

ref:

https://samharris.org/the-illusion-of-the-self2/

comment by Decius · 2020-05-04T11:30:33.761Z · LW(p) · GW(p)

I think you've correctly identified that lots of people believe themselves when they say lots of wrong things, and did a fair job of explaining one part of the ways to reduce that error.

A different alternative worth considering is summarized by "Say enough wrong things that you know for sure that most of them are wrong." In practical terms that means developing three or more mutually contradictory plausible-sounding reasons for what and why. From there, instead of sharing them in a manner where politeness requires that they not be challenged, share them in a manner where politeness requires that they do be challenged, by insisting that most of them are wrong.

It's certainly more difficult, but I think it's better to say a thousand things, 1% of which are right (and know that 99% of them are wrong), then to say three things, two of which are right (even knowing that one of them is likely to be wrong). Say as many wrong things as you can afford to, because if you can't tell the wrong from the right without saying.

comment by norswap · 2020-05-06T13:38:47.553Z · LW(p) · GW(p)

I want to discuss the specific example you picked: Etsy & A/B (or generally, data-driven) testing.

I'll start by agreeing to your premise in the abstract: Etsy could probably have done better by using a better A/B testing methodology and doing so sooner.

But: while superior tools used effectively are superior, they also tend to be harder to use and to cause more damage when misused. I've seen a bunch of math-based data-driven analysis that wasn't worth a damn because they were misused. The more sophisticated these tools become, the easier they are to misuse.

A popular cautionary tale in that space is "Steve Jobs vs focus groups". Focus groups are a primitive but, I'd argue, *scientific* method to determine what your customers want. Yet what it comes up with is often lackluster to the point where a visionary with a intuition and taste can run circles around them. Sure, in this primitive case we can easily point to some design flaws in the process. For instance, bringing various people together to discuss a design is likely to produce a compromise design that truly satisfies no one. But is A/B testing free of any such psychological flaws? I think not, and now you also risk screwing up the statistical analysis in one hundred different ways.

Second, trade-offs. If Etsy chose to perfect it's A/B testing methodology, it has to forego doing other things, because it does not have illimited resources (even if it had, if an entity becomes too large to coordinate effectively that comes with a slew of issues — this is well documented (the mythical man month, etc)). Could it be that their unsophisticated/late application of A/B was an effective trade-off in terms of resource use (à la 20/80 principle)?

I think a part of instrumental rationality is the ability to decide which tools to deploy.

Generally, more sophisticated tools can yield better results, but come with an inherent cost. You seem to ignore this basic reality here. Cases where it's as easy to do the smart thing as to do the dumb thing exist, but they're not everywhere - especially in business context (and even more so in startups), where there is a strong evolutionary pressure to adopt these low-hanging fruits.

comment by Mary Chernyshenko (mary-chernyshenko) · 2020-05-02T14:00:48.179Z · LW(p) · GW(p)

I have not stopped saying wrong things, but I have found that turning noble tasks into people-related problems sometimes goes some of the way, motivation-wise. And so does knowing whether you can quietly work on your own / need someone else to need your work / just don't care about something.

Like, I wanted to study German, once. I had begun and then dropped it. Now I study it again, because my kid is going to be learning it later in school and we decided (I decided) to start together. Now I don't have to think that I want to know the language, I only have to think that we need to overcome some verbs by the end of the week. I also get to grouse about how he doesn't get such simple things.

But I can't work on my own, I already know it from life.

comment by Slider · 2020-05-02T13:40:11.566Z · LW(p) · GW(p)

Per the magic the gathering color scheme this reads to me as blue arguing why it's approach is superior to reds. (or I guess anti-red, white could argue that life would go merrily on if we could just lockup all the criminals). As lyrics go: "one step forward and two steps back, nobody gets far like that". If you can prevent moving back and do stuff you will inevitably go very forward.

However there is still the issue that in situations you typically need something. It could be worse to do absolutely nothing than a stupid thing (like it feels that breathing is a pretty safe bet over not breathing in a very wide range of circumstances even if you don't reflect on whether it is a good idea). Having an overtly suppressive mind control might mean that the general activity goes down (a kind of depression).

As with most good advice their negation is also good advice and a lot of the trick is to see where their area of applicability lies. There are solid points for red to score against blue too. Nothing happening means no experience and less food for generalization. One also needs to stay alive in the present moment to be able to live to the future. Very gradual steps might be too little too late for some purposes.

comment by Gordon Seidoh Worley (gworley) · 2020-05-08T01:12:05.634Z · LW(p) · GW(p)
Lots has been said within these servers on finding ways to say new exciting correct things. Or correcting things many think are right, but didn't know were actually wrong. This is a relentlessly optimistic perspective. I have gotten grand mileage by merely trying to hold back from saying or acting on information or ideas I know are wrong, as opposed to worrying about thinking better, or not making mistakes, or being more rational when I am concerned with the truth. This other part is the meaty part, the intense part - the struggle to stop mouthing the words, to stop myself from willingly burning more money or time.

I think this perspective is appealing and useful advice, but only up to a point. At some point you get diminishing returns from ceasing to punch yourself in the face, as it were, and you find you can't do the things that matter to you without sticking your neck out and risking being wrong by trying to say something new that might be correct.

I find nothing disagreeable in your post other than what I read as a sentiment that maybe just stop saying wrong things and you can improve. Sometimes you've already stopped doing that so much that you have to start saying some wrong things [LW · GW].

(For what it's worth, my comment basically comes down to saying don't forget about the law of equal and opposite advice.)

comment by ChristianKl · 2020-05-03T21:06:33.130Z · LW(p) · GW(p)

Akrasia is already a huge problem for a lot of rationalists. If you increase the filter for which actions can be taken you are likely get less actions taking in general and might reinforce already existing akrasia.

Replies from: lc, Aiyen, max-hodges
comment by lc · 2020-05-03T21:36:59.713Z · LW(p) · GW(p)

Most rationalists have lots of problems and believing that the biggest problem is consistently lack of willpower is the sort of blinding optimism I spoke about in the OP. I don't see why taking incorrect actions wouldn't make akrasia problems worse, and at best they're neutral because they're not advancing anything.

Replies from: ChristianKl
comment by ChristianKl · 2020-05-04T12:59:23.108Z · LW(p) · GW(p)

When system I and system II aren't aligned you get akrasia. When I read your post I imaging that you would classify cases where system II says one shouldn't engage in an action but system I wants to engages in an action as making an error.

If you allow system II to censor system I from engaging in such actions I expect that you will get less actions.

comment by Aiyen · 2020-05-03T21:49:40.817Z · LW(p) · GW(p)

This is a genuine concern, and this may be particularly high-variance advice. However, a focus on avoiding mistakes over trying new "superstrategies" might also help some people with akrasia. It's easier to do what you know than seek some special trick. Personally, at least, I find akrasia is worst when it comes from not knowing what to do next. And while taking fewer actions in general is usually a bad idea, trying to avoid mistakes could also be used for "the next time I'm about to sit around and do nothing, instead I'll clean/program/reach out to a friend." This doesn't sound like it has to be about necessarily doing less.

comment by Max Hodges (max-hodges) · 2020-05-06T05:10:35.003Z · LW(p) · GW(p)

and I think "akrasia" is better explained by "rejecting the notion of a core self and considering how we are a multitude of competing urges and impulses."

https://www.lesswrong.com/posts/cbgKfAHSLz99zLzuR/stop-saying-wrong-things?commentId=yFrF2YH6N5Z73P4ZM [LW(p) · GW(p)]

comment by Mary Chernyshenko (mary-chernyshenko) · 2020-05-03T18:20:40.795Z · LW(p) · GW(p)

Also, I am trying to not say wrong things, to track things I say for excitement or as a knee-jerk reaction, and to "hmm" in response to such things being said (by adults)... I don't see the grand mileage come out of it. For example, my kid just gets confused when he asks if I stay to live with his family and I say he's going to have a wife who has to have a say in such things. I also see some mileage getting eaten.

Suppose your friends are interesting people with strong opinions - truly good people saying things despite the threat of being fired or conceivably having a court case fabricated against them; such threats are frequent, and sometimes not empty, as was the case with one mathematician I know. (She was sued but not fired.) Some, in the face of even worse threats that... almost never get fulfilled, but are still very unpleasant. Your friends speak the truth loudly and untiringly, and at some point they lose patience with people who question it, and after a while start implying that questioning it must be motivated by more than curiosity.

You are the local ruminant who gets to hear stuff and chews at it. And then there comes the moment when nobody wants the ruminant, they were just being polite about it but it's really time to move on..?

I walk a fine line, being the ruminant; I still can give people gifts and listen to them and frame exciting things as questions, but I know I will pay for it. I just don't know, what.

comment by Max Hodges (max-hodges) · 2020-05-05T17:45:19.056Z · LW(p) · GW(p)

I hope this helps

The most important investment that people can make is not to learn a particular skill—”I'll learn how to code computers,” or “I will learn Chinese,” or something like that. No, the most important investment is really in building this more flexible mind or personality.

https://www.gq.com/story/yuval-noah-harari-tech-future-survival