Forget Everything (Statistical Mechanics Part 1)

post by J Bostock (Jemist) · 2024-04-22T13:33:35.446Z · LW · GW · 5 comments

Contents

      Forget as much as possible, then find a way to forget some more.
  Particle(s) in a Box
    The Piston
    Maths of the Piston
  Conclusions
      You can only forget, never remember.
None
5 comments

EDIT: I somehow missed that John Wentworth and David Lorell are also in the middle of a sequence have written one post on this same topic here [LW · GW]. I will see where this goes from here! This sequence will continue!

Introduction to a sequence on the statistical thermodynamics of some things and maybe eventually everything. This will make more sense if you have a basic grasp on quantum mechanics, but if you're willing to accept "energy comes in discrete units" as a premise then you should be mostly fine.

The title of this post has a double meaning:

1. Forget the thermodynamics you've learnt before, because statistical mechanics starts from information theory.
2. The main principle of doing things with statistical mechanics is can be summed up as follows:

Forget as much as possible, then find a way to forget some more.

Particle(s) in a Box

All of practical thermodynamics (chemistry, engines, etc.) relies on the same procedure, although you will rarely see it written like this:

1. Take systems which we know something about
2. Allow them to interact in a controlled way
3. Forget as much as possible
4. If we have set our systems correctly, the information that is lost will allow us to learn some information somewhere else.

For example, consider a particle in a box.

What does it mean to "forget everything"? One way is forgetting where the particle is, so our knowledge of the particle's position could be represented by a uniform distribution over the interior of the box.

Now imagine we connect this box to another box:

If we forget everything about the particle now, we should also forget which box it is in!

If we instead have a lot of particles in our first box, we might describe it as a box full of gas. If we connect this to another box and forget where the particles are, we would expect to find half in the first box and half in the second box. This means we can explain why gases expand to fill space without reference to anything except information theory.

A new question might be, how much have we forgotten? Our knowledge gas particle has gone from the following distribution over boxes 1 and 2

$P\left(Box\right)=\{\begin{array}{cc}1& \text{Box 1}\\ 0& \text{Box 2}\end{array}$

To the distribution

$P\left(Box\right)=\{\begin{array}{cc}0.5& \text{Box 1}\\ 0.5& \text{Box 2}\end{array}$

Which is the loss of 1 bit of information per particle. Now lets put that information to work.

The Piston

Imagine a box with a movable partition. The partition restricts particles to one side of the box. If the partition moves to the right, then the particles can access a larger portion of the box:

In this case, to forget as much as possible about the particles means to assume they are in the largest possible space, which involves the partition being all the way over to the right. Of course there is the matter of forgetting where the partition is, but we can safely ignore this as long as the number of particles is large enough.

What if we have a small number of particles on the right side of the partition?

We might expect the partition to move some, but not all, of the way over, when we forget as much as possible. Since the region in which the pink particles can live has decreased, we have gained knowledge about their position. By coupling forgetting and learning, anything is possible. The question is, how much knowledge have we gained?

Maths of the Piston

Let the walls of the box be at coordinates 0 and 1, and let $x$ be the horizontal coordinate of the piston. The position of each green particle can be expressed as a uniform distribution over $(0,x)$, which has entropy ${\log }_2\left(x\right)$, and likewise each pink particle's position is uniform over $(x,1)$, giving entropy ${\log }_2(1-x)$.

If we have $n_g$ green particles and $n_p$ pink particles, the total entropy becomes $n_g{\log }_2\left(x\right)+n_p{\log }_2(1-x)$, which has a minimum at $x=\frac{n_g}{n_g+n_p}$. This means that the total volume occupied by each population of particles is proportional to the number of particles.

If we wanted to ditch this information-based way of thinking about things, we could invent some construct which is proportional to $\frac{n_g}{x}$ for the green particles and $\frac{n_p}{1-x}$ for the pink particles, and demand they be equal. Since the region with the higher value of this construct presses harder on the partition, and pushes it away, we might call this construct "pressure".

If we start with $x=1/3$ and $n_g=2\times n_p$, we will end up with $x=2/3$. We will have "forgotten" $n_g$ bits of information and learned $n_p$ bits of information. In total this is a net loss of $n_p$ bits of information, which are lost to the void.

The task of building a good engine is the task of minimizing the amount of information we lose.

Conclusions

We can, rather naturally and intuitively, reframe the behaviour of gases in a piston in terms of information first and pressure later. This will be a major theme of this sequence. Quantities like pressure and temperature naturally arise as a consequence of the ultimate rule of statistical mechanics:

You can only forget, never remember.

5 comments

Comments sorted by top scores.