A Thorough Introduction to Abstraction

post by RohanS · 2023-01-13T00:30:25.964Z · LW · GW · 1 comments

Contents

  Aim
  Definitions
  Introductory Examples
      Circle
      Metaphor
      The Quadratic Formula
      Examples from John Wentworth's Public Static: What is Abstraction?:
  The Significance of Abstraction
    Why is abstraction a useful concept? What exactly does it allow us to accomplish?
      Abstract Answer
      Concrete Answer
      The Value of Being Consciously Aware of Abstraction
  Getting Excited about Abstraction
        If you are comfortable sharing, please put your responses to any or all of the exercises in the comments!
    Unique Example: Abstraction Itself
    Things with Precise Definitions
      Exercise: 
      My Examples:
        Computer Science Theory
        Abstract Algebra
        Game Theory
    Problem Solving Techniques
      Exercise: 
      My Examples:
    Storytelling Abstractions
      Exercise: 
      My Examples:
    Concluding Exercises:
  Miscellaneous
  Further resources on abstraction:
None
1 comment

Disambiguation: What I’m talking about here is essentially the same thing that John Wentworth works on and writes about, including in Public Static: What is Abstraction? [? · GW], and I stole some of the things he’s written (by quoting him directly) to save myself time. But I think this post has a different aim from anything I’ve read from him or anyone else (which isn’t to say no one has written a post with this aim; someone probably has). 

For a very quick TL;DR, I'd suggest reading the Definitions section [LW(p) · GW(p)] and the Abstract Answer subsection [LW · GW].

Aim

My aim is to get readers (with approximately any background?) to understand the concept of abstraction; see that the concept is ubiquitous in the world and even in their lives; appreciate the usefulness of the concept; hone the skill of recognizing, generating, and utilizing abstractions; internalize the concept deeply enough to install the habit of employing this skill in the wild; and, ideally, learn to love the concept.

Simply skimming this post will hopefully partially serve all of these purposes, but I think doing the exercises included below is probably also an important component.

I expect this to be less valuable if you are already familiar with the concept of abstraction, but perhaps skimming this post will still lead you to encounter some valuable new perspectives on it.

Definitions

Abstraction is a conceptual process in which general rules and concepts are derived from specific examples by identifying shared high-level features and ignoring low-level differences. ‘An abstraction’ is the outcome of this process: a general idea that unifies a collection of specific objects, especially in a way that provides non-trivial insight into the nature or relationships of those objects.[1]

John Wentworth said [LW(p) · GW(p)]:

Anyway, (my reading of) Jaynes' answer to the main question: abstraction is mainly about throwing away or ignoring information, in such a way that we can still make strong predictions about some aspects of the underlying concrete system.

These are closely related definitions of abstraction; having multiple framings available to you may be helpful.

Introductory Examples

This section contains a bunch of examples of abstractions - I urge you to read as many as you need to feel like you have a handle on the basic concept, then skip to the next section. That's because in this section, for the sake of simplicity and clarity, the abstractions are fairly mundane; I don't expect many people to find them very exciting. Later in this post, I'll try to provide more exciting examples, and provide exercises that allow you to generate exciting examples for yourself.

Circle

https://mathstat.slu.edu/escher/index.php/Circles

Metaphor

The Quadratic Formula

x = \frac{ -b \pm \sqrt{b^2 - 4ac}}{2a} when ax^2 + bx + c = 0

Examples from John Wentworth's Public Static: What is Abstraction? [? · GW]:

  • We have a gas consisting of some huge number of particles. We throw away information about the particles themselves, instead keeping just a few summary statistics: average energy, number of particles, etc. We can then make highly precise predictions about things like e.g. pressure just based on the reduced information we've kept, without having to think about each individual particle. That reduced information is the "abstract layer" - the gas and its properties.
  • We have a bunch of transistors and wires on a chip. We arrange them to perform some logical operation, like maybe a NAND gate. Then, we throw away information about the underlying details, and just treat it as an abstract logical NAND gate. Using just the abstract layer, we can make predictions about what outputs will result from what inputs. Note that there’s some fuzziness - 0.01 V and 0.02 V are both treated as logical zero, and in rare cases there will be enough noise in the wires to get an incorrect output.
  • I tell my friend that I'm going to play tennis. I have ignored a huge amount of information about the details of the activity - where, when, what racket, what ball, with whom, all the distributions of every microscopic particle involved - yet my friend can still make some reliable predictions based on the abstract information I've provided.
  • When we abstract formulas like "1+1=2*1" and "2+2=2*2" into "n+n=2*n", we're obviously throwing out information about the value of n, while still making whatever predictions we can given the information we kept. This is what abstraction is all about in math and programming: throw out as much information as you can, while still maintaining the core "prediction" - i.e. the theorem or algorithm.
  • I have a street map of New York City. The map throws out lots of info about the physical streets: street width, potholes, power lines and water mains, building facades, signs and stoplights, etc. But for many questions about distance or reachability on the physical city streets, I can translate the question into a query on the map. My query on the map will return reliable predictions about the physical streets, even though the map has thrown out lots of info.[4]

The Significance of Abstraction

Why is abstraction a useful concept? What exactly does it allow us to accomplish?

Abstract Answer

Given an abstraction, there are two main uses (that I know of): incorporation and generation of new examples into the framework defined by the abstraction.

Concrete Answer

Abstraction in action shapes the world, and will likely do so even more radically in the future. Multilayered abstractions are critically useful, and it intuitively seems to me that the extent to which we can use them is the main difference between humans and other animals that allows us to shape the world to our desires as much as we do.[5] Everyone relies on tools that other people provide without having the first clue how to produce them themselves (e.g. houses, cars, electronic devices, the internet, software packages, etc.), but because using these tools only requires knowledge of how to directly interface with them rather than low-level details of how to create them, we can build on each other’s work very effectively. Further, the creation of many of these tools also utilizes other deeper abstractions, which themselves are often built on abstractions, and so on for many levels. The state of human civilization today is the result of millennia of progress, much of which can be attributed to the development and application of new abstractions. This progress may have brought us to The Precipice, and the way we build on the abstractions we currently have in the near future may have a dramatic influence on the long-term future.

The Value of Being Consciously Aware of Abstraction

Frankly, abstraction may not be the most important idea to be consciously aware of, since people use it by default without paying too much direct attention to it. For example, here are a couple of abstraction exercises that I tried to do:

  1. Look for areas where abstractions are clearly useful to other people for things you care about, and familiarize yourself with those abstractions, or find ways to use them more effectively if you already are familiar with them.
  2. Figure out places where abstractions (or insights related to abstractions) would be useful for things you care about but haven’t yet been developed, and try to produce them.

If these exercises generate useful ideas for you, that's great! But they didn't for me. While at first I expected these to lead to useful insights, I realized after a few minutes that most of the things that came to mind were already familiar, and I just hadn't explicitly thought about them from the lens of abstraction before. They included things along the lines of "Read more about AI alignment (to figure out what relevant abstractions I'm not aware of yet)," "Get good at ML engineering," "Study more math," "Try to generate meaningful theoretical results for AI alignment", etc. The answers didn't really feel very abstraction-related, because that wasn't the most natural language in which to express them.

After that, my view on the value of being consciously aware of the concept of abstraction changed. I still do think that being deeply familiar with the concept of abstraction (in the ways described in the Aim section [LW(p) · GW(p)] above) can help a lot. However, this value doesn't come from just thinking about abstraction and getting a major shortcut to large important ideas. Instead, it makes you less likely to miss opportunities to learn general lessons. It makes you more likely to notice when cool things you come across share properties with other interesting things you've previously seen, and that kicks off a mental process that allows you to synthesize generalizable takeaways.

Getting Excited about Abstraction

By this point, I have on numerous occasions come across something, realized it is an example of an abstraction, and thought "Woah, that's super cool! I totally feel like I've seen this general idea in a bunch of interesting places." Then, I have often had one or more of the following thoughts:

I would then try to answer those questions, and a reasonable proportion of the time I had enough success to find it fruitful. Each time, this activity made a new abstraction more familiar to me, and I became more likely to incorporate new examples of it that I came across into an existing framework in my mind.

In the hopes that I can recreate this experience for you, this section will consist of:

  1.  Examples of abstractions that I find exciting, and 
  2.  Exercises for you to actively engage with abstraction.

If you are comfortable sharing, please put your responses to any or all of the exercises in the comments!

Unique Example: Abstraction Itself

The notion of abstraction is itself an extraordinarily deep and powerful abstraction. This whole post is dedicated to analyzing it through its examples and uses! It is the one that I most want you to appreciate, but I think you come to appreciate it by getting excited about several other examples of abstraction, and then registering that the thing that all of those have in common is that they are abstractions.

This may take some time, since the process is highly recursive: to appreciate the notion of abstraction, you first need to appreciate many abstractions, but to appreciate abstractions, you need to appreciate their specific examples, and sometimes those are also abstractions, and so on.[6] The good news is that I think this process gives rise to deeply intellectually satisfying and enlightening trains of thought, and is extremely fun. The next few subsections are all aimed at walking you through this process, so let's get into it!

One quick note before we do: Keep in mind that different prompts and examples will resonate for different people. It's fine if some of them don't work for you - hopefully, at least some of them will.

Things with Precise Definitions

Many things that have precise definitions are abstractions. The circle example above fits this category.

Exercise: 

Make a list of abstractions that have precise definitions. Write down some examples and some of the key insights that these abstractions provide or allow for (e.g. theorems related to them). Continue until you feel like you have a good grasp on the richness of this class of abstractions (or until you think it's unlikely that will be attainable in a reasonable amount of time).

Suggested starting point: Look at the high-level topics, core results, and major theorems covered in STEM subjects you have previously studied (ideally ones where you know the material particularly well), and try to identify abstractions within them. You may want to look at syllabi, review guides, cheat sheets, etc. which give concise overviews of important high-level topics. (I think this is probably easiest for math and theoretical computer science courses.)

My Examples:

This is essentially my (relatively brief) response to the exercise above. Looking at some of my examples may help you come up with ideas, but it may also use up some of the things you could have come up with on your own. I don't go into detailed descriptions of these abstractions or insights, and I only thought of examples in my head rather than writing them down, so this may not be very legible if you aren't already familiar with the subjects I draw from. My list may be unsatisfying to you, but your list should not be, since it should contain abstractions and insights that you are familiar with.

Indented bullets indicate nested abstractions; if an object is an example of the sub-bullet, that implies it is also an example of the parent bullet. [7]

Computer Science Theory

Abstractions

Insights

The layers of abstraction indicated above encode several key results themselves.

Abstract Algebra

Abstractions

Insights

There are a lot of other abstractions and theorems here - it's called abstract algebra for a reason!

Game Theory

Abstractions

Insights

Problem Solving Techniques

There are many problem solving techniques that are effective for large classes of problems. For example, doing a sanity check when you arrive at a solution to a problem often helps catch major flaws.

Exercise: 

Make a list of problem solving technique abstractions. Write down some examples and some of the key insights that these abstractions provide or allow for. Continue until you feel like you have a good grasp on the richness of this class of abstractions (or until you think it's unlikely that will be attainable in a reasonable amount of time).

Prompts: 

My Examples:

Storytelling Abstractions

There are countless themes, ideas, principles, strengths, and shortcomings that are shared across stories of all kinds. Numerous stories are variants of the Hero's Journey: "the common template of stories that involve a hero who goes on an adventure, is victorious in a decisive crisis, and comes home changed or transformed." Many stories suffer from deus ex machina, in which "a seemingly unsolvable problem in a story is suddenly and abruptly resolved by an unexpected and unlikely occurrence" (though sometimes this can be intentional and serve a purpose).

Exercise: 

Come up with as many storytelling abstractions as you can. Also, come up with as many examples of each storytelling abstraction as you can.

Prompts: 

My Examples:

Concluding Exercises:

  1. Tell me important things I've gotten wrong about abstraction, or important things I failed to mention, in the comments.
  2. Lots of abstractions belong to the classes of "things with precise definitions," "problem solving techniques," and "storytelling abstractions." Come up with three other large classes of abstractions.

Miscellaneous

  1. It seems to me that much of the field of mathematics is a large stack of abstractions. Abstract Algebra and Category Theory are two topics which I think may reflect this particularly well (though I haven't studied any category theory). The same goes for theoretical CS (which is really a branch of math).
  2. There aren’t unique ways to slice up some interesting concepts, so there are some overlapping or even coinciding abstractions.
    1. We use specific abstractions because they have allowed us to achieve lots of progress on incorporation and generation of useful objects (e.g. definitions in abstract algebra that allow us to prove interesting theorems get widely adopted), but I strongly suspect there are sometimes alternative formulations that look quite different and still accomplish similar levels of success. In fact, I’ve heard examples of competing explanations of things that turned out to be proven equivalent. From the Wikipedia entry on the Church-Turing Thesis: 
      1. “Church, Kleene, and Turing proved that these three formally defined classes of computable functions coincide: a function is λ-computable if and only if it is Turing computable, and if and only if it is general recursive.”
  3. Structure is a very important concept when it comes to abstraction. Every object that is an example of an abstraction must share the common features of the abstraction, and this necessity defines a basic structure of a generic example of the abstraction. Abstractions which involve more features, and more restrictive features, are more structured. Drawing on abstract algebra again: Groups are sets with additional structure, rings are groups with additional structure, and fields are rings with additional structure. Everything that is true in general with less structure remains true when additional structure is added, while the additional assumptions which generate more structure are often used to prove stronger statements to be true in general for objects with that structure. (The Wikipedia section on Basic Concepts in Abstract Algebra also describes this idea well.)
    1. So far, I've mostly explained how abstractions can be derived by looking at specific examples and extracting commonalities. This discussion of structure highlights an alternate way to derive abstractions: Start from an abstraction you already know, and add or remove structure. Of course, your choice of what structure to add or remove should be informed by the usefulness of the resulting abstraction.
    2. The aspects of specific examples that aren't pinned down by the abstract structure are the degrees of freedom that can vary between examples.
  4. Another way of framing the value of abstractions is that they compress information, and this compressed information is often easier to work with. 
  5. A pretty quote, paraphrased from my abstract algebra professor (Robert Friedman): "An abstraction is a way of calling many things by one name."

Further resources on abstraction:

  1. Two 3blue1brown videos:
    1. Group theory, abstraction, and the 196,883-dimensional monster
    2. 8:20-11:02 in this other 3b1b video, on structuring math explanations
  2. Wikipedia page on abstraction
  3. The LessWrong Abstraction tag [? · GW]
  4. All proof-based math and theoretical computer science
  5. The world: once you know to look for it, you see abstraction everywhere!
  1. ^

    This definition combines components from the Wikipedia page on abstraction, John Wentworth's writing on abstraction, and my own interpretation.

  2. ^

    That is, every quadratic equation with coefficients in the complex numbers has two solutions in the complex numbers.

  3. ^

    Here, a = 1, b = 0, and c = -1. Plugging these values into the quadratic formula yields x = 1 and x = -1.

  4. ^

    Some of John's examples show that you need not care about all of the specific objects that an abstraction unifies - in fact, sometimes you only care about one of the objects. For instance, in the map of NYC example, the degrees of freedom that arise from moving from the actual streets to their representations in the map are things like "street width, potholes, power lines and water mains, building facades, signs and stoplights, etc." So the abstraction unifies all the possible versions of NYC streets with different widths, potholes, power lines, etc. However, we don't care about all these different versions - we only care about the real version. In these cases, the value of the abstraction doesn't come from its unification of many specific objects, but rather from its compression of the description of the single object we care about into a form that is easier to work with.

  5. ^

    Of course, our use of abstractions to shape the world isn't always positive. Most people regularly use products that are sourced via factory farming and burning fossil fuels, and the fact that we can use these products without seeing their production processes makes it very easy to indirectly contribute harm to animal welfare and the climate. These are just two examples, and there are countless others.

  6. ^

    It isn't really a matter of needing to appreciate examples before starting to appreciate an abstraction, but I think in many cases, you slowly gain an appreciation for the abstraction as you see and come up with more and more examples. It may actually take quite a few examples before you stop seeing new interesting instantiations of the abstraction.

  7. ^

    In some cases, like the sets-groups-rings-fields example under abstract algebra, this is because the sub-bullet is a variant of the parent bullet abstraction with additional structure. See the discussion of structure in the Miscellaneous section.

  8. ^

    This is kind of just a list of some of my favorite stories, but thinking about the specific ways they instantiate this abstraction helps me remember many of the specific things their creators did to make the stories so great.

  9. ^

    If you've completed these stories and want to know what specifically I'm referring to, feel free to ask in a comment. I had a bit of trouble with spoiler formatting so I tried to avoid spoilers, but I know that may have made it hard to know what I was referring to in some cases.

  10. ^

    One of the best episodes of TV I've ever seen.

  11. ^

    Sanderson's First Law is closely related to the "Have answers to the question 'Why?' as often as possible" abstraction, and his Third Law is closely related to the "Make story components as interconnected as possible" abstraction. I read about Sanderson's Three Laws before crystallizing those abstractions in my head, so they probably helped me do that.

1 comments

Comments sorted by top scores.

comment by shminux · 2023-01-13T01:24:19.194Z · LW(p) · GW(p)

To me abstraction = lossy compression. I don't think there is anything else there. An example of abstraction is mapping a specific human face to the word "face". A really high compression ratio.