What questions should we ask ourselves when trying to improve something?

post by weathersystems · 2021-05-06T19:03:06.638Z · LW · GW · No comments

This is a question post.

Contents

  What is the status quo?
      Here's the template so far:
  What are the goals we are aiming for?
  What are the goals we shouldn't aim for?
  What resources might help in making progress?
    What information might be relevant?
    Who may be able/willing to help?
  What are the possible improvements to the template?
  What outcomes are/should we measure in determining success?
  What are the possible difficulties in making improvements in this area?
  Are there any analogous systems that we might look to for inspiration?
None
  Answers
    2 karlkeefer
    2 nim
None
No comments

I'm trying to come up with a template for questions of the form "How should we improve X?"  I assume most such questions will involve answering many subquestions. Sometimes the answers to those subquestions will be implied in the context, or the questions may not all be relevant. But my guess is that the larger and more complicated the system is that would be improved, the more you'd want to explicitly answer these subquestions. 

There are many ways you may be able to help me. This question is a bit meta, in the sense that I'm trying to improve something (the template). Here are the questions from the template that seem relevant here. Feel free to contribute to any of these:

 

What is the status quo?

Here's the template so far:

 

What are the goals we are aiming for?

I'd like to have a template that could be used to start new question pages that have the form "How should we improve X?"

The template should be applicable across a wide range of progress type questions.

The template should feel useful to people asking these sorts of questions.

 

What are the goals we shouldn't aim for?

We shouldn't aim to cover every single possible question that could be relevant to the top level question. That would be too overwhelming.

 

What resources might help in making progress?

What information might be relevant?

Process Improvement Theory and Application (maybe this is relevant?)

Who may be able/willing to help?

reddit? what subreddits might be able to help?

 

What are the possible improvements to the template?

 

What outcomes are/should we measure in determining success?

I guess anyone who wants to test the thing can try to use the template and give feedback on if it was helpful or not.

 

What are the possible difficulties in making improvements in this area?

It will be a tough balance to strike to have a template that's sufficiently general such that it applies to many situations, but at the same time has enough detail to have some marginal value over just asking the question without the template.

 

Are there any analogous systems that we might look to for inspiration?

RFC's seem to have a similar structure to "how do we improve X" type questions.

Process Improvement theory may be close enough to be relevant.

[from nim]: "Science of decision-making may be interesting, as it tends to be predicated on the assumption that decisions need to pick or create the "best" option, aka an improvement."

Answers

answer by karlkeefer · 2021-05-06T22:43:11.865Z · LW(p) · GW(p)

There's a trade-off to be wary of here in trying to improve the list only by adding. Making it complete might also make it too long to actually get much use.

Can you optimize the list to keep the most valuable bits while also keeping it nice and short to increase the chance someone will go through the effort of answering the questions?

I noticed in your own use of the template, you only included a few of the sub-questions.

↑ comment by weathersystems · 2021-05-07T00:08:40.515Z · LW(p) · GW(p)

I was thinking that the template would be something where you could just keep the sections that seem relevant and delete the rest. 

But I guess even that would start to get annoying if the thing was super long. That's a good consideration to keep in mind.

↑ comment by weathersystems · 2021-05-07T00:25:14.409Z · LW(p) · GW(p)

I added in a few more of the questions from the template that seem relevant. Including the one about possible difficulties. I think what's there cover's your trade-off.

answer by nim · 2021-05-06T20:49:03.275Z · LW(p) · GW(p)

A suggested improvement to the template: When examining the status quo, also ask "for what related problems does the status quo have a built-in solution?".

You have spelled "stakeholders" as "steak-holders", which is charming but may reduce credibility in some circumstances.

The template might benefit from a section asking what preconceptions or stereotypes surround the topic. When I think back to times I've tried to improve things, I recall some being stymied by a group of people having a social memory of a time that something related was tried and went horribly wrong, or a time that some other group tried a similar thing and it backfired, or other expectations of negative outcomes.

The other question to consider adding, possibly adjacent to "what has been tried already", is along the lines of "why am I unwilling to tolerate this status quo when everyone else experiencing it does tolerate it?". I don't think this is the best phrasing, but comparing one's experience with something intolerable to the experiences of those who find it tolerable can often reveal techniques for making the experience tolerable, which shifts the value proposition of eliminating the experience.

What resources might help in improving this template?

Science of decision-making may be interesting, as it tends to be predicated on the assumption that decisions need to pick or create the "best" option, aka an improvement.

↑ comment by weathersystems · 2021-05-06T21:24:24.771Z · LW(p) · GW(p)

You have spelled "stakeholders" as "steak-holders", which is charming but may reduce credibility in some circumstances.

Heh. Funny mistake. Thanks.

A suggested improvement to the template: When examining the status quo, also ask "for what related problems does the status quo have a built-in solution?".

I want to make sure I understand your point here. Is the idea that sometimes we see that a system isn't solving some problem well enough, and so try to fix it. But we don't take into account the fact that the system isn't just trying to solve that problem, but other problems as well. And maybe our "fix" may be an improvement to the system in regards to the problem we're interested in, but hurt the system in regards to the other problems the system is trying to solve? (god that was long winded)

If that's the idea. I think I was trying to capture the same type of thing with "What are the strengths of the status quo (that we want to try and keep)?" But maybe I can improve the wording to make that more clear? Or do you still think you're making a separate point?

The template might benefit from a section asking what preconceptions or stereotypes surround the topic.

I like this. Not sure where to include it. From your description it seems like it should be either a top level question, under "What are the possible difficulties in making improvements in this area?", or under "What is the status quo?".

Replies from: karlkeefer, nim
↑ comment by karlkeefer · 2021-05-06T22:39:12.668Z · LW(p) · GW(p)

Not OP, but I read their comment about related problems as something more like this:

The system in question likely already has feedback or correction mechanisms that respond to other potential problems - asking about those mechanisms might reveal strengths of the system that can be easily adapted for your purposes. I'm not sure how easy it will be to find these, though, as the best-functioning ones might be invisible if they actually eliminate the other problems completely.

That might not be their intent, but I think it's also a useful consideration so even if my interpretation isn't matched I hope this comment is still useful :)

↑ comment by nim · 2021-05-07T00:29:09.621Z · LW(p) · GW(p)

I want to make sure I understand your point here.

Yikes, I see why -- I worded the concept quite poorly. The example I was trying to describe is in software engineering, where you have an ancient crufty mess that you're trying to rewrite in some snazzy new language. You think you can rewrite it and make it super simple, and so you write the new thing the simple way that "should work", but when you run the old code's tests against it (or when you put it to use in the real world...) you discover that the reason the old code was such a mess was partly that it had a bunch of logic to handle various edge cases that the application had hit in the past.

An alternative phrasing might be: "Where are the gaps between how I think the status quo 'should' work, and how it actually does?". Often, established systems are silently compensating for all kinds of problems that happen infrequently enough for any one person to forget that the problem exists when trying to replace the system.

↑ comment by weathersystems · 2021-05-06T21:27:50.400Z · LW(p) · GW(p)

I added "Given these problems, why are people still tolerating the status quo (if they are)?" to the template. Does that capture your idea well enough?

Replies from: nim
↑ comment by nim · 2021-05-07T00:31:02.108Z · LW(p) · GW(p)

I think that's a good snapshot of the concept I'm trying to get at. It asks what benefits the status quo may silently be providing, which a competitor would have to match or exceed to gain acceptance.

No comments

Comments sorted by top scores.