Without models

This post is an example of how words can go wrong. Richard hasn't clearly specified what this 'model' or 'implicit model' stuff is, yet for the whole post he repeats again and again that it's not in control systems. What is the content of this assertion? If I accept it, or if I reject it, how is this belief going to pay its rent? What do I anticipate differently?

Can anything be 'model'? How do I know that there is a model somewhere?

The word itself is so loaded that without additionally specifying what you mean, it can be used only to weakly suggest, not strongly assert a property.

Any property you see in a system is actually in your interpretation of the system, in its semantics (you see a map, not the territory, this is not a pipe). Interpretation and the procedure of establishing it given a system are sometimes called a 'model' of the system, this is a general theme in what is usually meant by a model. Interpretation doesn't need to happen in anyone's head, it may exist in another system, for example in a computer program, or it can be purely mathematical, arising formally from the procedure that specifies how to build it.

In this sense, to call something a model is to interpret it as an interpretation of something else. Even a rock may be said to be a model of the universe, under the right interpretation, albeit a very abstract model, not useful at all. Of course, you can narrow down this general theme to assert that rocks can't model the universe, in particular because they can't simulate certain properties, or because your interpretation procedure breaks down when you present it with a rock. But you actually have to state the meaning of your terms in the cases like this, hopefully with a definition-independent goal to accomplish by finally getting the message through.

An employee is paid an hourly rate for however many hours he wishes to work. What will happen to the number of hours per week he works if the rate is increased?

It depends. The substitution effect says work more, the income effect says work less. The result depends on which one dominates.

If I use my rational model to give instructions to someone who blindly follows, lacking his own model, and he achieves his goal, I would not say he succeeded arationally. The fact that the agent did not contain the rationality but only received the conclusions does not mean the rationality was not important to the process.

If I extend the situation, so that I use my rational model to describe simple rules that produce success in a broad class of situations, and an agent lacking a model succeed by following my rules, this still is not arational. The agent is still using my rationality indirectly.

If I take the simple rules I derived from my rational model, and design a device to mechanically follow the rules, the device will work, not arationally, but indirectly rationally. This is your control system. It does not contain a model, it just blindly follows rules that were produced by rationality.

A collective reply to comments so far.

All the posted answers to the exercises so far are correct.

1. Warming the thermostat with a candle will depress the room temperature while leaving the thermostat temperature constant.

2. Pressing the brake when the cruise control does not disengage will leave the car speed constant while the accelerator pedal goes down -- until something breaks.

3. The effect of raising a piece-rate worker's hourly rate will depend on what the worker wants (and not on what the employer intended to happen).

4. The doctor's target will be met while patients will still have to wait just as long, they just won't be able to book more than four weeks ahead. (This is an actual example from the British National Health Service.)

Does no-one want to tackle 5 or 6? Anyone who knows the derivative of exp(a t) knows enough to do number 6.

Thank you, kpreid, for linking to the very article that I knew, even while writing the original post, I would be invoking in response to the comments. Anyone who has not come across it before, please read it, and then I will talk about the concept that (it turns out) we are all talking about, when we talk about models, except for the curious fit that comes over some of us when contemplating the simple thermostat.

i77: As you say, the Smith predictor contains a model, and the subsystem C does not. Likewise the MRAC. In the PID case, the engineer has a model. But don't slide from that to attributing a model to the PID system. There isn't one there.

Vladimir_Nesov, pretty much all the concepts listed in the first three sections of that article are special cases of what is here meant by the word. As for the rest, I think we can all agree that we are not talking about a professional clothes horse or a village in Gmina Pacyna. I don't believe I have committed any of these offences (another article I'd recommend to anyone who has only just now had the good fortune to encounter it), but let those call foul who see any.

So, what are we talking about, when we talk about models? What I am talking about -- I'll come to the "we" part -- I said in a comment of mine on my first post:

What is a model? A model is a piece of mathematics in which certain quantities correspond to certain properties of the thing modelled, and certain mathematical relationships between these correspond to certain physical relationships.

and more briefly in the current post:

signals ... that are designed to relate to each other in the same way as do corresponding properties of the world outside

This is exactly what is meant by the word in model-based control theory. I linked to one paper where models in precisely this sense appear, and I am sure Google Books or Amazon will show the first chapters of any number of books on the subject, all using the word in exactly the same way. There is a definite thing here, and that is the thing I am talking of when I talk of a model.

This is not merely a term of art from within some branch of engineering, in which no-one outside it need be interested. Overcoming Bias has an excellent feature, a Google search box specialised to OB. When I search for "model", I get 523 hits. The first five (as I write -- I daresay the ranking may change from time to time) all use it in the above sense, some with less mathematical content but still with the essential feature of one thing being similar in structure to another, especially for the purpose of predicting how that other thing will behave. Here they are:

"So rather than your model for cognitive bias being an alternative model to self-deception..." (The model here is an extended analogy of the brain to a political bureaucracy.)

"Data-based model checking is a powerful tool for overcoming bias" (The writer is talking about statistical models, i.e. "a set of mathematical equations which describe the behavior of an object of study in terms of random variables and their associated probability distributions.")

"the model predicts much lower turnout than actually occurs" (The model is "the Rational Choice Model of Voting Participation, which is that people will vote if p times B > C".)

"I don't think student reports are a very good model for this kind of cognitive bias." (I.e. a system that behaves enough like another system to provide insight about that other.)

The 5th is a duplicate of the 2nd.

Those are enough examples to quote, but I inspected the rest of the first ten and sampled a few other hits at random (nos. 314, 159, 265, and 358, in fact), and except for a mention of a "role model", which could be arguable but not in any useful way, found no other senses in use.

When I googlesearch LW, excluding my own articles and the comments on them, the first two hits are to this, and this. These are also using the word in the same sense. The models are not as mathematical as they would have to be for engineering use, but they are otherwise of the same form: stuff here (the model) which is similar in structure to stuff there (the thing modelled), such that the model can be used to predict properties of the modelled.

In other words, what I am talking about, when I talk about models, is exactly what we on OB and LW are all talking about, when we talk about models, every time we talk about models. There is a definite thing here that has an easily understood shape in thingspace, we all call it a model, and to a sufficiently good approximation we call nothing else a model.

Until, strangely, we contemplate some very simple devices that reliably produce certain results, yet contain nothing drawn from that region of thingspace. Suddenly, instead of saying, "well well, no models here, fancy that", the definition of "model" is immediately changed to mean nothing more than mere entanglement, most explicitly by SilasBarta:

"A controller has a model (explicit or implicit) of it's environment iff there is mutual information between the controller and the environment."

Or the model in the designer's head is pointed to, and some sort of contagion invoked to attribute it to the thing he designed. No, this is butter spread over too much bread. That is not what is called a model anywhere on OB or LW except in these comment threads; it is not what is called a model, period.

You can consider the curvature of a bimetallic strip a model of the temperature if you like. It's a trivial model with one variable and no other structure, but there it is. However, a thermometer and a thermostat both have that model of the temperature, but only the thermostat controls it. You can also consider the thermostat's reference input to be a model of the position of the control dial, and the signal to the relay a model of the state of the relay, and the relay state a model of the heater state, but none of these trivial models explain the thermostat's functioning. What does explain the thermostat's functioning is the relation "turn on if below T1, turn off if above T2". That relation is not a model of anything. It is what the thermostat does; it does not map to anything else.

Exercise 7. How can you discover someone's goals? Assume you either cannot ask them, or would not trust their answers.

It sounds like you think that, if someone thinks about control theory the way I do, they will make prediction mistakes or bad decisions. And you want to keep people from making prediction mistakes or bad decisions, but first you have to make them see that there's something wrong with their thinking, and you don't know a direct argument that will make them see that, so you have to use a lot of arguments from examples. Can you say more directly what kinds of prediction mistakes or bad decisions you think that people who think about control theory the way I do will make?

The cruise control does not sense the gradient of the road, nor the head wind. It senses the speed of the car. It may be tuned for some broad characteristics of the vehicle, but it does not itself know those characteristics, or sense when they change, such as when passengers get in and out.

I didn't expect that the cruise control would be able to do that (without another controller for its tuning). That's why one of my list of sufficient conditions for optimality of a PID controller was, "the system is a second-order linear system with constant coefficients". If the coefficients change, then the same PID controller may not be optimal. Did you expect that I would have expected the cruise control to be able to sense changes in the characteristics of the vehicle? Or are you trying to say that, if someone thought about control systems the way I did, they would have expected the cruise control to be able to know when the vehicle characteristics change, except if they were thinking carefully at the time the way I was? For example, are you trying to say that I might have had this mistaken expectation if I was only thinking about the cruise control as part of thinking about something else? And you want to make me see that there's something wrong with my thinking that makes me make prediction mistakes when I'm not thinking carefully?

An implicit model is one in which functional relationships are expressed not as explicit functions y=f(x), but as relations g(x,y)=k.

It sounds like you want to say that a controller doesn't have any implicit model unless it has a separate, identifiable physical part or software data structure that expresses a relationship and has no other function. If one controller is mathematically equivalent to another controller that does have a separate, identifiable part that expresses a relationship, but the controller itself doesn't have a separate, identifiable part that expresses a relationship, does it still not have an implicit model?

Linear controllers are optimal for many control problems that are natural limiting cases or approximations of real-world families of control problems. In a control problem where a linear controller is the Bayes-optimal controller, it is literally impossible for any controller with different outputs from the linear controller to have a lower average cost. Even if a controller was made of separate identifiable parts that implemented the separate identifiable parts of Bayesian sequential decision theory, and even if some of those parts expressed relations between past, present, or future perception signals, reference signals, control signals, or system states, the controller still couldn't do any better than the optimal linear controller. And all the information that a Bayesian optimal controller could use is already in either the state of the optimal linear controller or the state of the system which the controller has been controlling. If a Bayesian decision-theoretic controller had to take over from an optimal linear controller, it would have no use for any more information than the state of the controller and the perception and reference signals at the controller, which is also the only information that the linear controller was able to use. If the Bayesian controller was given more information about past reference signals or perception signals, that would not help its performance. At any time, the posterior belief distribution in a Bayesian controller can be set equal to a new posterior belief distribution defined using only the perception signal, the reference signal, and the state that the optimal linear controller would have had at that time, and the Bayesian controller will still have optimal performance. And the state in an optimal linear controller can be set equal to a new state defined using only the perception signal, the reference signal, and the posterior belief distribution that a Bayesian controller would have had at that time. This means that, whatever information processing a Bayesian optimal controller for a linear-quadratic-Gaussian control problem would be doing that would affect the control signal, the optimal linear controller (together with the system it is controlling) is already doing that information processing. They are mathematically equivalent.

A linear controller can be optimal for more than one control problem. To define a linear controller's implicit model of the system and disturbances, you need a model of the reference signal and the cost functional; to define a linear controller's implicit model of the reference signal, you need a model of the cost functional and the system and disturbances; and to define a linear controller's implicit model of the cost functional, you need a model of the reference signal and of the system and disturbances. Some of these implicit models are only defined up to a constant factor. And a linear controller that is optimal for some control problems can also perform well on other control problems that are near them.

(An optimal Bayesian controller for a linear-quadratic-Gaussian control problem isn't able to change its model when the system coefficients or the statistical properties of the disturbances or reference signal change. This is because the controller would have no prior belief that a change was possible. All of the controller's prior probability would be on the belief that the system and disturbances and reference signal would act like the problem the controller was designed to be optimal for. If the controller had any prior probability on any other belief, it would make decisions that wouldn't be optimal for the problem it was designed to be optimal for.)

Now, I am not explaining control systems merely to explain control systems. The relevance to rationality is that they funnel reality into a narrow path in configuration space by entirely arational means, and thus constitute a proof by example that this is possible.

It sounds like you are saying that the math for when a controller works doesn't leave any shadow at all in the math of what a good controller does. If you aren't saying that, then I disagree less with what you have said.

This must raise the question, how much of the neural functioning of a living organism, human or lesser, operates by similar means?

Agreed.

\5. What relates questions 3 and 4 to the subject of this article?

Are questions 3 and 4 situations in which people who think about control theory the way I do might make prediction mistakes when they aren't thinking carefully? Are they situations in which the employer has a mistaken implicit model of how to increase (reference signal) the employee's hours (perception signal) by changing his wages (control signal) and the medical bureaucrats have a mistaken implicit model of how to decrease (reference signal) the doctor's time per patient (perception signal) by controlling his target (control signal)? Are they situations in which the employer has a mistaken belief about the control system inside the employee and the medical bureaucrats have a mistaken belief about the control system inside the doctor?

\6. Controller: o = c×(r-p). Environment: dp/dt = k×o + d. o, r, and p as above; c and k are constants; d is an arbitrary function of time (the disturbance). How fast and how accurately does this controller reject the disturbance and track the reference?

Errors will decay exponentially with rate constant k×c, if k×c is positive.

If d is constant and r is constant, then p = r+d/(k×c).

If d is zero and r = m×t, then p = r-m/(k×c): p will lag by m/(k×c). The P controller implicitly predicts that the future changes of r will on average be equal to the integral of d, which is zero. Because the average future change of r is something other than zero, on average the P controller lags. A tuned PI controller could implicitly learn m and implicitly predict future changes in r and not lag on average.

If the controller had the control law o = -c×p, its implicit model would be that r(t) will on average be equal to the integral of e^(-k×c×s)×d(t-s) with respect to s for s from zero to infinity, and that there is no information about future values of r in the current value of r that's not also in the current value of that integral. If the controller had the implicit model that r was constant at zero, its control law would be o = -∞×p, because in our model of the environment that we are using to define the controller's implicit model, there are no measurement errors, no delayed effects, and no control costs.

Exercise 4. When targets were imposed on British GPs, the effect at my practice was that only a few appointments were available. I had to sit by the phone from 8.30am when the practice opened, phoning, getting the engaged tone, repeatedly pressing the redial button. Then I got the appointment time I wanted. Phone later in the day and there were no appointments available.

A GP may decide that his amour propre (signalling?) is more important than conforming to targets.

Experience on targets appears to indicate that people find ways of meeting the target, which may or may not be by achieving what the target-setter wished to achieve.

A control system doesn't model a system, to a large degree it is the designers' model of the system it controls.

"Now, I am not explaining control systems merely to explain control systems. The relevance to rationality is that they funnel reality into a narrow path in configuration space by entirely arational means, and thus constitute a proof by example that this is possible."

I don't think you needed control systems to show this. Gravity itself is as much of a 'control system' - it minimizes the potential energy of the system! Heck, if you're doing that, lots of laws of physics fit that definition - they narrow down the set of possible realities...

" This must raise the question, how much of the neural functioning of a living organism, human or lesser, operates by similar means? "

So, I'm still not sure what you mean by 'similar means'.

We know the broad overview of how brains work - sensory neurons get triggered, they trigger other neurons, and through some web of complex things motor neurons eventually get triggered to give outputs. The stuff in the middle is hard; some of it can be described as "memory" (patterns that somehow represent past inputs), some can be represented various other abstractions. Control systems are probably good ways of interpreting a lot of combinations of neurons, and some have been brought up here. It seems unlikely that they would capture all of them - but if you stretch the analogy enough, perhaps it can.

"And how much of the functioning of an artificial organism must be designed to use these means? "

Must? I'd guess absolutely none. The way you have described them, control systems are not basic - for 'future perception' to truly determine current actions, would break causality. So it's certainly possible to describe/build an artificial organism without using control systems, though it seems like it would be pretty inconvenient and pointlessly harder than an already impossible problem given how useful you're describing them to be.

OK. So let'ts take a controller with an explicit (I hope you agree) model, the Smith predictor. The controller as a whole has a model, but the subsystem C(z) (in the wiki example) has not (in your terms).

Or better yet, a Model Reference Adaptive Control. The system as a whole IS predictive, uses models, etc.. but the "core" controller subsystem does "not".

Then I'd argue that in the simple PID case, the engineer does the job of the Model/Adjusting Mechanism, and it's a fundamental part of the implementation process (you don't just buy a PID and install it without tuning it first!)

So, in every control systems there is a model. It's only that when the plant is "simple enough" and invariant in the long term the model/adjusting subsystem is implemented in wetware, and only used during install.

This is just arguing semantics, though.

All control systems DO have models of what they are controlling. However, the models are typically VERY simple.

A good principle for constructing control systems are: Given that I have a very simple model, how do I optimize it?

The models I learned about in cybernetics were all linear, implemented as matrices, resistors and capacitors, or discrete time step filters. The most important thing was to show that the models and reality together did not result in amplification of oscillations. Then one made sure that the system actually did some controlling, and then one could fine tune it to reality to make it faster, more stable, etc.

One big advantage of linear models is that they can be inverted, and eigenvectors found. Doing equivalent stuff for other kinds of models is often very difficult, requiring lots of computation, or is simply impossible.

As has someone has written before here: It is mathematically justified to consider linear control systems as having statistical models of reality, typically involving gaussian distributions.

Kim Øyhus

Exercise 1. Assuming the truth of the statement that the candle has the effect of raising the temperature of the thermostat like that, (I do not have the knowledge to state whether that is or is not the case) the room temperature will oscillate between 15 and 16 degrees as the temperature of the thermostat continues to oscillate between 20 and 21, until the candle burns out.

I agree with the thesis; the referenced paper is really interesting, but the article on LW is a bit long-winded in trying to convey the notion that "there is no internal model". Amusingly the paper's title is "Internal models in the cerebellum"!

Great post! It clears up many of my misunderstandings and misconceptions I had after reading your last one.

I certainly have to agree, my concept of a implicit model is very problematic. And control systems are creepy!

Exercise 2. The accelerator will go to maximum, and the driver would have to brake maximally, until something burned out or the driver put the car out of gear or turned off the cruise control.