Optimising under arbitrarily many constraint equations

post by dkl9 · 2024-09-12T14:59:28.475Z · LW · GW · 0 comments

This is a link post for https://dkl9.net/essays/general_lagrange.html

Contents

  Lagrange, anyway
  Determinants
  § Example
None
No comments

Say we have a multivariate function to optimise, like , under some constraints, like  and , both to equal zero.

The common method is that of Lagrange multipliers.

  1. Add a variable  for each constraint function — here, we'll use  and .
  2. Declare the set of equations .
  3. Bring in the equations  and  (etc, if there are more constraints).
  4. Solve for  and, more importantly, the inputs .

Lagrange multipliers annoy me, insofar as they introduce extra variables. There is another way — arguably more direct, if perhaps more tedious in calculation and less often taught. I found it alone, tho surely someone else did first — probably Euler.

Lagrange, anyway

For the sake of a standard answer to check against, let's use Lagrange multipliers.

The gradient of  is . Likewise, , and . So step 2 gives these equations:

It readily follows that  or .

If , then , and . By the second constraint, , find that . By the first constraint, , find that , which is a contradiction for real inputs.

If , then, by the first constraint, , and, by the second constraint, , so  and .

Determinants

With one constraint, the method of Lagrange multipliers reduces to  and  are vectors, which differ by a scalar factor iff they point in the same (or directly opposite) directions iff (for three dimensions) the cross product  iff (for two dimensions) the two-by-two determinant .

With two constraints, the method asks when . That would mean  is a linear combination of  and , which it is iff , and  are all coplanar iff (for three dimensions) the three-by-three determinant .

As it happens, the cross product is a wolf that can wear determinant's clothing. Just fill one column with basis vectors: .

Likewise, with zero constraints, the "method of Lagrange multipliers" — really, the first-derivative test — asks when . Fill a three-by-three matrix with two columns of basis vectors: . Suppose the basis vectors multiply like the cross product, as in geometric algebra. Then the determinant, rather than the usual 0 for a matrix with two equal columns, turns out to equal that ordinary column vector  (up to a scalar constant).

In every scenario so far — and I claim this holds for higher dimensions and more constraints — the core equations to optimise under constraints are the actual constraint equations, along with a single determinant. The matrix has its columns filled with the gradient of the function to optimise, each constraint gradient, and copies of the basis vectors, in order, to make it square.

§ Example

Fill a matrix with those gradients given above. We'll take its determinant.

The determinant, when simplified, is . The equations to consider are just

The first tells us that  or . If , so , so , and . If , then  and  is imaginary. These are the same results as above; the method works, using only the variables given in the problem.

0 comments

Comments sorted by top scores.