Quadratic Forms & Definiteness
§2.21 - When is a critical point a hilltop, a valley, or neither?
Prereq: §2.19 Maxima & Minima
Drag the slider. At what moment does the bowl break into a saddle? There's a precise number that controls this - and it has a beautiful name.
Zooming In - Everything Looks Quadratic
Here's the key idea. Near a critical point, any smooth function $f(x,y)$ behaves like a quadratic - the linear terms vanish (that's what "critical point" means), so the second-order terms dominate. Those second-order terms form what we call a quadratic form:
\[ Q(x,y) = ax^2 + 2bxy + cy^2 \]
We can pack the coefficients into a symmetric matrix $A = \begin{pmatrix}a&b\\b&c\end{pmatrix}$ so that $Q = \mathbf{x}^T A \mathbf{x}$. The shape of the surface $z = Q(x,y)$ is entirely determined by the eigenvalues of this matrix. Play with the sliders below to see for yourself:
Try making one eigenvalue positive and the other negative - what happens? The classification rule is simple and worth understanding intuitively:
\(\lambda_1, \lambda_2 > 0\) - Positive definite - bowl (curves up in every direction)
\(\lambda_1, \lambda_2 < 0\) - Negative definite - dome (curves down everywhere)
\(\lambda_1 \lambda_2 < 0\) - Indefinite - saddle (up in one direction, down in another)
The Eigenvectors Tell You Where to Look
So eigenvalues tell us the type of curvature. But the eigenvectors do something equally beautiful: they tell us the directions of steepest and gentlest curvature.
Think of it physically. Imagine you're standing at the bottom of a bowl that's not perfectly round - it's steeper in one direction than another (like an elliptical valley). The eigenvectors point along the axes of that ellipse, and the eigenvalues tell you the curvature along each axis.
In the visualization below, the red and green arrows show the two eigenvector directions. The colored curves are the cross-sections of the surface along those directions - each one is a parabola whose curvature is exactly the corresponding eigenvalue.
Try setting $b \neq 0$ - notice how the eigenvector axes rotate? The cross-term $2bxy$ tilts the principal axes away from the coordinate axes. The eigenvalues stay the same (curvature doesn't care about coordinates), but the directions change.
From Quadratic Forms to Real Functions - The Hessian
So far we've been playing with pure quadratic forms $ax^2 + 2bxy + cy^2$. But real functions are more complicated - they have cubic terms, exponentials, trig functions. How does this help?
Here's the connection: near a critical point, the Taylor expansion of any smooth $f(x,y)$ starts with a quadratic form. The matrix of that form is called the Hessian: \[H = \begin{pmatrix} f_{xx} & f_{xy}\\ f_{xy} & f_{yy}\end{pmatrix}\bigg|_{(x_0,y_0)}.\] Its eigenvalues classify the critical point exactly the same way - positive definite means local min, negative definite means local max, indefinite means saddle. Try a few examples:
Critical point: (0, 0)
Hessian at critical point:
The Second Derivative Test - A Recipe
Computing eigenvalues every time is overkill for a 2×2 matrix. It turns out there's a shortcut. The quantity $D = f_{xx} f_{yy} - (f_{xy})^2$ is the determinant of the Hessian, and it equals $\lambda_1 \lambda_2$. That's all you need:
If $D > 0$, both eigenvalues have the same sign (check $f_{xx}$ to see which). If $D \lt 0$, they have opposite signs - saddle. If $D = 0$, the test is inconclusive (we're right on the boundary between types). Play with the sliders and watch the flowchart highlight:
A Beautiful Connection - Extrema on the Unit Circle
We'll close with something genuinely elegant. Suppose we restrict a quadratic form $Q(\mathbf{x})$ to the unit circle $\|\mathbf{x}\| = 1$ and ask: where is $Q$ largest? Where is it smallest?
You might expect this to be a messy Lagrange multiplier calculation. But the answer is stunningly clean: the maximum of $Q$ on the unit circle is exactly $\lambda_{\max}$, and the minimum is $\lambda_{\min}$. The extrema occur precisely along the eigenvector directions.
Drag $\theta$ around the circle below. The purple dot traces out $Q$ values. Watch how the maximum always lands on the red eigenvector and the minimum on the green one.
This is the Rayleigh quotient in action. It's one of those results that connects linear algebra and optimization in a way that keeps showing up - in physics, statistics, and machine learning. The eigenvalues aren't just abstract numbers; they're the extremes of the quadratic form on the sphere.
Playground - Try Your Own Function
Type any $f(x,y)$ and let the computer find, classify, and visualize all the critical points.
Uses JS syntax: Math.sin, Math.exp, ** for powers.
Practice Problems - §2.21
From Kaplan, problems after §2.21
Find the critical points of \(z = 1 + x^2 + y^2\) and test for maxima and minima.
Step 1: Find critical points
\[f_x = 2x = 0,\quad f_y = 2y = 0.\] Critical point: \((0,0)\), with \(z = 1\).
Step 2: Compute second derivatives
\[f_{xx} = 2,\quad f_{yy} = 2,\quad f_{xy} = 0.\] So \(A = 2\), \(B = 0\), \(C = 2\).
Step 3: Classify
\[AC - B^2 = (2)(2) - 0^2 = 4 > 0,\quad A = 2 > 0.\]
Positive Definite ⇒ local minimum at \((0,0)\) with \(z = 1\). This is an upward-opening paraboloid.
Find the critical points of \(z = x^2 - 2xy + y^2\) and test for maxima and minima.
Step 1: Find critical points
\[z = (x-y)^2.\quad f_x = 2(x-y) = 0,\quad f_y = -2(x-y) = 0.\] Every point on the line \(x = y\) is a critical point.
Step 2: Compute second derivatives
\[f_{xx} = 2,\quad f_{xy} = -2,\quad f_{yy} = 2.\] \[AC - B^2 = (2)(2) - (-2)^2 = 4 - 4 = 0.\] The second derivative test is inconclusive.
Step 3: Classify by inspection
Since \(z = (x-y)^2 \ge 0\) everywhere, and \(z = 0\) on the line \(x = y\), every point on \(x = y\) is an absolute minimum (value 0). This is a "valley" along the line \(x = y\).
Find the critical points of \(z = x^3 - 3xy^2 + y^3\) and test for maxima and minima.
Step 1: Find critical points
\[f_x = 3x^2 - 3y^2 = 3(x^2 - y^2) = 0,\quad f_y = -6xy + 3y^2 = 3y(-2x + y) = 0.\] From the first equation: \(x = \pm y\). If \(x = y\): from the second, \(3y(-2y + y) = -3y^2 = 0\), so \(y = 0\). If \(x = -y\): \(3y(2y + y) = 9y^2 = 0\), so \(y = 0\). Only critical point: \((0,0)\).
Step 2: Compute second derivatives
\[f_{xx} = 6x,\quad f_{xy} = -6y,\quad f_{yy} = -6x + 6y.\] At \((0,0)\): all second derivatives are 0. The second derivative test is inconclusive.
Step 3: Classify by inspection
Along \(y = 0\), \(z = x^3\) changes sign near the origin. So \((0,0)\) is neither a max nor a min - this is a degenerate critical point (sometimes called a "monkey saddle" due to the \(x^3 - 3xy^2\) term).
Find the critical points of \(z = \dfrac{x}{x^2+y^2}\), classify them, and describe the level curves.
Step 1: Find critical points
Domain excludes the origin. Compute partial derivatives: \[f_x = \frac{(x^2+y^2) - x \cdot 2x}{(x^2+y^2)^2} = \frac{y^2 - x^2}{(x^2+y^2)^2} = 0 \quad\Rightarrow\quad y^2 = x^2.\] \[f_y = \frac{-2xy}{(x^2+y^2)^2} = 0 \quad\Rightarrow\quad xy = 0.\] Combined: \(y = 0\) and \(|y| = |x|\) gives no solution (except the origin, which is not in the domain). No critical points.
Step 2: Conclusion
The function has no maxima or minima on its domain \(\mathbb{R}^2 \setminus \{(0,0)\}\).
Step 3: Level curves
Setting \(\frac{x}{x^2+y^2} = c\) and rearranging: \[x^2 + y^2 - \frac{x}{c} = 0 \quad\Rightarrow\quad \left(x - \frac{1}{2c}\right)^2 + y^2 = \frac{1}{4c^2}.\] These are circles passing through the origin, centered at \(\left(\frac{1}{2c}, 0\right)\) with radius \(\frac{1}{2|c|}\).
Find the absolute minimum and maximum of \(w = x + y + z\) for \(x^2 + y^2 + z^2 \le 1\).
Step 1: Interior critical points
\[w_x = 1,\quad w_y = 1,\quad w_z = 1.\] These are never zero, so there are no interior critical points.
Step 2: Boundary via Lagrange multipliers
On \(x^2 + y^2 + z^2 = 1\), use Lagrange multipliers with \(g = x^2+y^2+z^2 - 1\): \[\nabla w = \lambda \nabla g \quad\Rightarrow\quad 1 = 2\lambda x,\; 1 = 2\lambda y,\; 1 = 2\lambda z.\] So \(x = y = z = \frac{1}{2\lambda}\). From the constraint: \(\frac{3}{4\lambda^2} = 1\), giving \(\lambda = \pm\frac{\sqrt{3}}{2}\).
Step 3: Evaluate and classify
\(\lambda = \frac{\sqrt{3}}{2}\): \(x = y = z = \frac{1}{\sqrt{3}}\), \(w = \sqrt{3}\) (absolute maximum).
\(\lambda = -\frac{\sqrt{3}}{2}\): \(x = y = z = -\frac{1}{\sqrt{3}}\), \(w = -\sqrt{3}\) (absolute minimum).
Find the absolute minimum and maximum, if they exist, of \(w = e^{-x^2 - y^2 - z^2}\) for all \((x,y,z)\).
Step 1: Find critical points
\[w_x = -2x\,e^{-x^2-y^2-z^2} = 0,\quad w_y = -2y\,e^{-x^2-y^2-z^2} = 0,\quad w_z = -2z\,e^{-x^2-y^2-z^2} = 0.\] Since the exponential is always positive, we need \(x = y = z = 0\). Critical point: \((0,0,0)\), \(w = 1\).
Step 2: Behavior at infinity
As \(|(x,y,z)| \to \infty\), \(w \to 0\) but never reaches 0. So 0 is an infimum but not a minimum.
Step 3: Classify
Absolute maximum \(w = 1\) at the origin. No absolute minimum (the infimum 0 is never attained).
Step 1: Write the associated matrix
The quadratic form \(3x^2 + 2xy + y^2\) corresponds to \(\mathbf{x}^T A \mathbf{x}\) where \[A = \begin{pmatrix}3 & 1\\1 & 1\end{pmatrix}.\] (The off-diagonal entry is half the coefficient of \(xy\): the coefficient of \(2bxy\) is 2, so \(b=1\).)
Step 2: Find eigenvalues
The characteristic equation: \[\det(A - \lambda I) = (3-\lambda)(1-\lambda) - 1 = \lambda^2 - 4\lambda + 2 = 0.\] \[\lambda = \frac{4 \pm \sqrt{16-8}}{2} = \frac{4 \pm 2\sqrt{2}}{2} = 2 \pm \sqrt{2}.\] So \(\lambda_1 = 2 + \sqrt{2} \approx 3.41\) and \(\lambda_2 = 2 - \sqrt{2} \approx 0.59\).
Step 3: Classify
Both eigenvalues are positive: \(\lambda_1 \approx 3.41 > 0\) and \(\lambda_2 \approx 0.59 > 0\).
Positive Definite - Yes, \(Q(x,y) > 0\) for all \((x,y) \neq (0,0)\).
Quick sanity check with the determinant test: \(D = \det(A) = 3 \cdot 1 - 1^2 = 2 > 0\) and \(a_{11} = 3 > 0\), confirming positive definiteness.
Step 1: Write the associated matrix
The coefficient of \(xy\) is \(-1\), so the off-diagonal entry is \(-1/2\): \[A = \begin{pmatrix}1 & -\tfrac{1}{2}\\[4pt]-\tfrac{1}{2} & -2\end{pmatrix}.\]
Step 2: Find eigenvalues
The characteristic equation: \[\det(A - \lambda I) = (1-\lambda)(-2-\lambda) - \tfrac{1}{4} = \lambda^2 + \lambda - \tfrac{9}{4} = 0.\] \[\lambda = \frac{-1 \pm \sqrt{1+9}}{2} = \frac{-1 \pm \sqrt{10}}{2}.\] So \(\lambda_1 = \frac{-1+\sqrt{10}}{2} \approx 1.08\) and \(\lambda_2 = \frac{-1-\sqrt{10}}{2} \approx -2.08\).
Step 3: Classify
One eigenvalue is positive (\(\lambda_1 \approx 1.08\)) and the other is negative (\(\lambda_2 \approx -2.08\)).
Indefinite - No, \(Q\) is not positive definite. It's indefinite, meaning it takes both positive and negative values.
Quick check: \(Q(1,0) = 1 > 0\) but \(Q(0,1) = -2 < 0\). The form changes sign, so it cannot be positive definite.
Want more worked examples? See §2.19 - Maxima & Minima for three detailed worked optimization problems: a bounded domain, an unbounded domain, and a Lagrange multiplier example - each with full second derivative classification.