The Pontryagin Maximum Principle

How to steer a system optimally when your actuators have limits — derived from scratch, for engineers. One running example (a braking problem), one classic (the brachistochrone), everything verified exactly.

Contents: 1 · The problem • 2 · Perturb and compare • 3 · Costates: prices for state • 4 · The theorem • 5 · Worked example • 6 · The brachistochrone • 7 · The wider landscape • 8 · Recipe card • Further reading

1The problem: optimal control with actuator limits

You control a dynamical system. The state $x(t) \in \mathbb{R}^n$ evolves by $\dot x = f(x, u)$, where $u(t)$ is your input. You pay a cost — fuel, time, tracking error — and you want the input that makes that cost as small as possible:

$$ J(u) \;=\; \varphi\bigl(x(T)\bigr) \;+\; \int_0^T L\bigl(x(t),\,u(t)\bigr)\,dt \qquad\longrightarrow\qquad \min_{u(\cdot)} $$

subject to the dynamics $\dot x = f(x,u)$, the initial condition $x(0)=x_0$, and — this is the part that matters — a constraint on the input: at every instant, $u(t) \in U$, a fixed set of admissible values (thrust you can produce, voltage your amplifier can supply, torque your motor can exert). Here $L(x,u)$ is the running cost, paid continuously, and $\varphi$ is a terminal cost, paid once at the final time $T$. Every symbol used in this page is collected in the table at the end of this section.

Why not just calculus of variations? If $u$ were unconstrained, the optimum would be an interior point: setting the first variation of $J$ to zero (the Euler–Lagrange machinery) would find it. But real actuators saturate, and when they do, the optimum almost never sits in the interior — it sits on the boundary of $U$, at full throttle. At a boundary point "set the derivative to zero" is simply the wrong condition: the derivative there is usually nonzero, and the optimum is optimal precisely because pushing further is forbidden. We need a first-order optimality condition that is valid on the boundary too. That is exactly what the maximum principle is: a replacement for "stationarity" that survives constraints.

A running example: stop the cart in least time

Everything in sections 1–5 is about one concrete problem. A cart at position $x$, moving with velocity $v$, driven by an acceleration $u$ you can command in $[-1, 1]$ (full brake to full thrust). Start at $x=1$, $v=0$; stop at the origin $x=0$, $v=0$; take as little time as possible. As a state-space system with $x_1 = x$, $x_2 = v$:

$$ \dot x_1 = x_2, \qquad \dot x_2 = u, \qquad |u| \le 1, \qquad (x_1, x_2)(0) = (1, 0), \qquad (x_1, x_2)(T) = (0,0). $$

This is $L \equiv 1$, $\varphi = 0$: the cost is the elapsed time, $J = T$. Figure 1 shows the answer we are after. The control is not a smooth function: it slams to $-1$ (full brake), holds, then flips to $+1$ (full thrust in the opposite sense, catching the cart exactly at the origin). Such "on/off" controls are called bang-bang, and the flip instant is a switch. The theorem will derive this shape, the switch time $t=1$, and the total time $T=2$ — nothing is guessed.

control u(t): -1 then +1, switching at t=1 — **Figure 1 — The optimal solution of the braking example.** Full brake $(u=-1)$ until $t=1$, then reversed full thrust $(u=+1)$ until the cart rests at the origin at $T=2$. Dashed line: the switch instant. Note the velocity is a sharp triangle with a corner at the switch — no smooth "optimal profile" exists here.

velocity v(t): triangle wave touching zero at t=2 — **Figure 1 — The optimal solution of the braking example.** Full brake $(u=-1)$ until $t=1$, then reversed full thrust $(u=+1)$ until the cart rests at the origin at $T=2$. Dashed line: the switch instant. Note the velocity is a sharp triangle with a corner at the switch — no smooth "optimal profile" exists here.

Symbol	Meaning	In the braking example
$x \in \mathbb{R}^n$	state (defined on first use in §1)	$x = (x_1, x_2) = (\text{position},\ \text{velocity})$
$u \in \mathbb{R}^m$	control (input you choose)	acceleration command
$U \subseteq \mathbb{R}^m$	admissible control set (fixed, compact)	$[-1, 1]$
$f(x,u)$	dynamics: $\dot x = f(x,u)$	$(x_2,\ u)$
$L(x,u)$	running cost rate	$1$ (time)
$\varphi(x)$	terminal cost	$0$
$J$	total cost to minimize	$T$
$p \in \mathbb{R}^n$	costate (adjoint) — §3	$p(t) = (-1,\ t-1)$
$p^0 \le 0$	multiplier on the running cost — §4	$-1$ (normal case)
$H$	Hamiltonian — §3	$p_1 x_2 + p_2 u - 1$
$\psi(t)$	switching function — §5	$p_2(t)$
$V(x)$	optimal cost-to-go — §7	time-to-origin from state $x$

Takeaway. Actuator limits move the optimum to the boundary of the input set, where "set the derivative to zero" fails; optimal control needs an optimality condition that works there — that condition is the maximum principle.

2The engine: perturb and compare

Every optimality condition in applied mathematics — Fermat's rule, Euler–Lagrange, KKT — is the same sentence: at an optimum, no allowed perturbation can improve the cost, even to first order. The whole game is choosing which perturbations to feed in. For optimal control there is a natural, almost surgical choice.

The needle variation

Take a candidate control $u^*(t)$ and modify it on one tiny interval $[s, s+\varepsilon]$ only, replacing it there by some other admissible value $v \in U$. Everything else is untouched. The modified control is called a needle variation of $u^*$: a sliver of width $\varepsilon$, full amplitude.

Why needles rather than small smooth wiggles? Two reasons, and both are load-bearing:

Bounded sets. If the optimum sits on the boundary of $U$ (it usually does), you cannot perturb $u^*$ by a small amplitude in an improving direction — the improving direction points outside $U$. But a perturbation that is small in duration while full in amplitude is always available. Smallness in $\varepsilon$ is all we need.
Locality. Because the modification lives on one short interval, its effect can be attributed to a single instant $s$. Feeding in needles at every admissible instant is what will turn one global statement ("no perturbation improves $J$") into one local condition per instant of time — the pointwise maximum condition.

control with a needle perturbation of width epsilon — **Figure 2 — A needle and the scar it leaves.** Top: baseline $u \equiv 0$ with one needle of width $\varepsilon$ at $t=1$ (width exaggerated for visibility). Bottom: the induced error in the state $x$ of the double integrator. During the needle the velocity picks up $\delta v = \varepsilon$; that velocity error then *persists forever*, so the position error grows linearly with slope $\varepsilon$. Total state displacement is $O(\varepsilon)$ — small, proportional to the needle's width, never to its amplitude.

resulting state error growing linearly — **Figure 2 — A needle and the scar it leaves.** Top: baseline $u \equiv 0$ with one needle of width $\varepsilon$ at $t=1$ (width exaggerated for visibility). Bottom: the induced error in the state $x$ of the double integrator. During the needle the velocity picks up $\delta v = \varepsilon$; that velocity error then *persists forever*, so the position error grows linearly with slope $\varepsilon$. Total state displacement is $O(\varepsilon)$ — small, proportional to the needle's width, never to its amplitude.

What the needle does to the state. Figure 2 carries the key quantitative fact. The perturbed trajectory $\tilde x(t) = x^*(t) + \delta x(t)$ stays within a tube of radius $O(\varepsilon)$ around $x^*$ for all $t$: the needle injects $O(\varepsilon)$ of "fluence" (here, $\varepsilon$ of velocity), and the dynamics transport it. Because the displacement is first-order small in $\varepsilon$, we may compute the cost change by linearization — second-order terms are $O(\varepsilon^2)$ and vanish as $\varepsilon \to 0$. That linearization step is the only regularity we will ever need on $f$ and $L$: first-order differentiability.

Takeaway. Optimality means: every needle (at every instant, toward every admissible $v$) leaves the cost unchanged or worse, to first order. One global statement, infinitely many local ones.

3Prices: the costate and the Hamiltonian

We now owe a computation: express the first-order cost change of a needle in terms of the needle alone. The obstacle is that a needle at time $s$ moves the state at every later time (the tube of Figure 2), and the cost samples the state along the whole trajectory. We need a bookkeeping device that folds all of that downstream influence into a single vector per instant. That device is the costate $p(t) \in \mathbb{R}^n$.

Step 1 — How state errors propagate

Linearize the dynamics along the candidate trajectory $x^*(t)$: a perturbation satisfies the variational equation

$$ \delta\dot x \;=\; A(t)\,\delta x \;+\; B(t)\,\delta u, \qquad A \;=\; \frac{\partial f}{\partial x}\bigg|_{*}, \quad B \;=\; \frac{\partial f}{\partial u}\bigg|_{*}, \qquad \delta x(0) = 0, $$

whose solution propagates an input blip into a state tube: $\delta x(t) = \int_0^t \Phi(t,\tau)\, B(\tau)\,\delta u(\tau)\, d\tau$, where $\Phi$ is the state-transition matrix of $A$ (the usual $\dot\Phi = A\Phi$, $\Phi(\tau,\tau)=I$). Engineers know this object as the sensitivity model of the system along its nominal trajectory.

Step 2 — Run the sensitivities backward: the adjoint

The first-order cost change of any perturbation is

$$ \delta J \;=\; \big\langle \nabla\varphi(x^*(T)),\, \delta x(T) \big\rangle \;+\; \int_0^T \Bigl( \langle \nabla_x L,\, \delta x \rangle + \langle \nabla_u L,\, \delta u \rangle \Bigr)\, dt. $$

The trouble is the $\delta x$ terms: they mix contributions from all earlier $\delta u$'s. The fix is a classic move (the same one as reverse-mode automatic differentiation, a.k.a. backpropagation): define a backward variable $p(t)$ by the adjoint ODE

$$ \dot p \;=\; -A(t)^{\!\top} p \;-\; \nabla_x L\bigl(x^*(t), u^*(t)\bigr), \qquad p(T) \;=\; -\,\nabla\varphi\bigl(x^*(T)\bigr), $$

and integrate the identity $\frac{d}{dt}\langle p, \delta x\rangle = \langle\dot p, \delta x\rangle + \langle p, \delta\dot x\rangle$ over $[0, T]$. Substituting $\dot p$ and $\delta\dot x = A\,\delta x + B\,\delta u$, the $A$-terms cancel because $\langle -A^{\!\top}p, \delta x\rangle + \langle p, A\,\delta x\rangle = 0$, and the boundary terms fold in $\nabla\varphi$ exactly. What survives is

$$ \delta J \;=\; -\int_0^T \big\langle p,\, B\,\delta u \big\rangle\, dt \;+\; \int_0^T \big\langle \nabla_u L,\, \delta u \big\rangle\, dt \;=\; -\int_0^T \Bigl\langle \underbrace{B(t)^{\!\top}p(t) \;-\; \nabla_u L}_{=\ \nabla_u H}\;,\; \delta u(t) \Bigr\rangle dt. $$

Every trace of $\delta x$ is gone: one backward sweep (one solve of the adjoint ODE) prices all instants simultaneously. The price of doing this forward — inverting the sensitivity map for each needle separately — would be one forward solve per perturbation; the adjoint gives every gradient in one pass. That is precisely why backpropagation trains neural nets in one backward sweep, and it is the same mathematics here.

Step 3 — The Hamiltonian

Collect the integrand. In the normal case (the multiplier on the running cost scaled to $-1$; §4 says when this may be done) define the Hamiltonian

$$ H(x, u, p) \;=\; \big\langle p,\, f(x, u) \big\rangle \;-\; L(x, u), \qquad\text{so that}\qquad \delta J \;=\; -\int_0^T \big\langle \nabla_u H,\, \delta u \big\rangle\, dt . $$

$\nabla_u H$ is the marginal cost of control: it converts any control perturbation directly into a cost change. Feed in a needle of amplitude $v$ at time $s$ (so $\delta u = v - u^*(s)$ on an interval of length $\varepsilon$):

$$ \delta J \;=\; \varepsilon\,\Bigl[\, H\bigl(x^*(s),\, u^*(s),\, p(s)\bigr) \;-\; H\bigl(x^*(s),\, v,\, p(s)\bigr) \Bigr] \;+\; o(\varepsilon) \;\ge\; 0 \qquad \text{for all } v \in U . $$

The inequality is the statement "no needle improves the cost." Divide by $\varepsilon$, let $\varepsilon \to 0$, and place the needle at any instant where $u^*$ is continuous (that is, almost everywhere): the bracket must be nonnegative pointwise, at almost every instant of time. That single line is the maximum condition — the heart of the theorem of §4.

What the costate means

The same adjoint computation, applied to a pure state kick (nudge the state by a vector $w$ at time $t$, leave the control alone) yields

$$ \delta J\bigl[\text{kick } w \text{ at time } t\bigr] \;=\; -\,\langle p(t),\, w \rangle . $$

So $p(t)$ is a price vector for state: $-p_i(t)$ is the marginal cost of one more unit of state $x_i$ at time $t$. This gives the whole construction a second reading: the Hamiltonian $H = \langle p, f\rangle - L$ is the rate at which the trajectory "earns" priced progress — the state velocity $f$, valued at state prices $p$, minus the burn rate $L$. The maximum condition then says: at each instant, among all inputs you are allowed to apply, choose the one with the greatest priced progress rate. Section 5 checks this price interpretation numerically, exactly, against the closed-form value function.

control $u(t)$

$u(t) \in U$ — bounded set

state $x(t)$

$\dot x = f(x, u)$

cost $J$

$\varphi(x(T)) + \int_0^T L\,dt$

Hamiltonian

$H(x,u,p) = \langle p, f(x,u)\rangle - L(x,u)$

max condition: $u^* \in \arg\max_{u \in U} H$

costate $p(t)$

$\dot p = -\partial H/\partial x$,
run backward from $p(T)$

$\dot x = f(x, u)$
forward pass

$L(x,u),\ \varphi(x(T))$

backward pass: $\dot p = -\dfrac{\partial H}{\partial x}$, $p(T) = -\nabla\varphi(x(T))$

$p(t)$

$x^*(t)$

$u^* = \arg\max_{u \in U} H$
pointwise in time

Figure 3 — The PMP machine. Forward pass (gray, top): the control drives the state, the state and control accumulate cost. Backward pass (gray, bottom): the adjoint ODE runs backward in time from the terminal condition, turning future cost into present prices. The Hamiltonian combines both into a scalar rate; the maximum condition (blue) closes the loop by choosing, at each instant, the control that maximizes it.

Takeaway. The costate $p(t)$ is the marginal price of state at time $t$, computed by one backward adjoint sweep; the Hamiltonian is the priced rate of progress, and its gradient in $u$ converts any perturbation into a cost change: $\delta J = -\int \langle\nabla_u H, \delta u\rangle dt$.

4The theorem

We have all the pieces: the needle argument (\S2) and the adjoint bookkeeping (\S3). Here is what they buy, stated for the problem of §1. Recall $H(x,u,p,p^0) = \langle p, f(x,u)\rangle + p^0 L(x,u)$ with a multiplier $p^0 \le 0$.

Theorem — Pontryagin Maximum Principle (engineering statement)

Let $u^*(t)$ be an optimal control with trajectory $x^*(t)$ for the problem of §1, with $U$ compact and $f, L, \varphi$ differentiable. Then there exist a number $p^0 \le 0$ and an absolutely continuous costate trajectory $p(t) \in \mathbb{R}^n$, not both zero, such that, writing $H$ for the Hamiltonian:

State equation. $\dot x^* = f(x^*, u^*)$, $x^*(0) = x_0$ — the trajectory is consistent with its own control (no new content, but it completes the system).
Costate equation. $\dot p = -\,\partial H/\partial x$ along $(x^*(t), u^*(t), p(t))$ — the adjoint ODE of §3.
Maximum condition. For almost every $t$: $u^*(t) \in \arg\max_{u \in U} H\bigl(x^*(t),\, u,\, p(t),\, p^0\bigr)$ — derived from needles in §3.
Transversality. (a) If the final state is free: $p(T) = -\nabla\varphi(x^*(T))$. (b) If $x(T)$ is confined to a set $S$: $p(T)$ must be normal to $S$ at $x^*(T)$. (c) If $T$ is free and the data are time-invariant: $H(T) = 0$, hence $H \equiv 0$ on $[0,T]$.
Nontriviality. $(p^0, p(\cdot)) \neq (0, 0)$ — the multipliers are not all zero (otherwise the theorem would say nothing).

Normal vs. abnormal. If $p^0 \neq 0$, nontriviality lets us rescale the multipliers to $p^0 = -1$: this is the normal case, the Hamiltonian of §3, $H = \langle p, f\rangle - L$, and it is what one meets in practice. The abnormal case $p^0 = 0$ means the optimality certificate does not reference the cost at all — the constraints alone (dynamics and endpoint conditions) are what force the solution. It is rare; when it happens, $H = \langle p, f \rangle$ and the theorem still holds, but "minimize $J$" has degenerated into a pure feasibility statement.

Why a maximum — and why that was a surprise. A naive first-derivative argument on the control suggests a stationarity condition, $\partial H/\partial u = 0$. Indeed, if $u^*(t)$ lies in the interior of $U$, the maximum condition reduces exactly to that. The content of the theorem is what happens on the boundary: there the best you can do is push against the limit, and the condition becomes "align with whatever direction increases $H$ fastest within $U$" — a maximization, not a stationarity. Pontryagin's school reportedly expected a minimum-type condition and the maximum came out of the analysis the other way; the sign convention $p^0 \le 0$ is exactly what makes it a maximum.

$H$ is conserved. If the problem data do not depend explicitly on $t$ (autonomous) and $T$ is fixed, then along the optimum $H$ is constant. Where $u^*$ is an interior maximizer the proof is two lines: $\frac{dH}{dt} = \frac{\partial H}{\partial x}\dot x + \frac{\partial H}{\partial p}\dot p = H_x\, H_p - H_p\, H_x = 0$, since $\dot x = \partial H/\partial p$ and $\dot p = -\partial H/\partial x$ — a Noether-style cancellation. On the boundary of $U$ the needle argument rescues the same conclusion. Intuition: $H$ is the priced progress rate, and an optimal schedule neither gains nor loses by re-timing itself.

Free final time forces $H \equiv 0$. If $T$ is itself a decision variable (our braking problem: how long is up to the optimizer), the schedule must be indifferent to stretching: the first-order optimality condition in $T$ is $H(T) = 0$, and by conservation $H$ vanishes everywhere. For a minimum-time problem this reads $\langle p, f\rangle = 1$: the trajectory's priced speed exactly equals the rate at which time is being paid for.

Hamiltonian as a function of u: two lines, maximized at the boundary of U — **Figure 4 — The maximum condition in action.** For the braking example $H(u) = p_1 v + p_2 u - 1$ is *affine in $u$*: at a fixed instant, with $(x, p)$ fixed, it is a straight line over the admissible set $U = [-1, 1]$. A line never has an interior maximizer (its slope $p_2$ is generically nonzero — stationarity is unavailable!), so the maximum is always at an endpoint: $u^* = +1$ if $p_2 \gt 0$, $u^* = -1$ if $p_2 \lt 0$. Bang-bang is not an assumption; it is the geometry of a linear Hamiltonian on a bounded interval.

Fine print (for honesty, skim on first read). The statement above is the classical one under standard hypotheses: $U$ compact, $f, L$ continuously differentiable, $u^*$ piecewise continuous, endpoint conditions as stated. The maximum condition holds almost everywhere (the needle argument needs continuity points of $u^*$, which is all of them except finitely many switches). The general theorem also requires a nontriviality argument and, for existence of an optimizer in the first place, a compactness/continuity result (Filippov's theorem, §7). Full rigor: Pontryagin et al. (1961) or Liberzon (2012).

Takeaway. An optimal control must maximize the Hamiltonian pointwise over the admissible set, along a trajectory generated by the coupled state–costate (Hamiltonian) system, with boundary conditions supplied by transversality. Interior optima reduce to stationarity; boundary optima — the interesting ones — become bang-bang.

5Worked example: the braking problem, start to finish

Now run the machine. The example is small enough to solve by hand, completely, and every number below has been verified exactly (symbolically, not just numerically).

Step 1 — Write the Hamiltonian and maximize it

Normal case ($p^0 = -1$, $L = 1$): $H = p_1 x_2 + p_2 u - 1$. The maximum condition over $U = [-1,1]$ gives, whenever $p_2 \neq 0$,

$$ u^*(t) \;=\; \operatorname{sign}\bigl(p_2(t)\bigr), \qquad \psi(t) := p_2(t) \;\; =: \;\; \text{switching function}. $$

The sign of the second costate alone decides the control. Nothing here assumed bang-bang — it fell out of maximizing an affine function (Figure 4).

Step 2 — The costate equations restrict the switching structure

$$ \dot p_1 \;=\; -\frac{\partial H}{\partial x_1} \;=\; 0, \qquad \dot p_2 \;=\; -\frac{\partial H}{\partial x_2} \;=\; -p_1 \qquad\Longrightarrow\qquad \psi(t) = p_2(t) \;\text{is affine in } t . $$

An affine function crosses zero at most once: the optimal control switches at most once. This is the kind of structural conclusion PMP is loved for — the shape of the solution came from the costate equation, before any algebra.

Step 3 — Transversality pins the multipliers

The terminal state is fixed, so $p(T)$ is free; but $T$ is free and the system is autonomous, so $H \equiv 0$. Evaluate it where the control and velocity are known:

$$ t=0:\;\; v=0,\; u=-1:\;\; H = p_2(0)(-1) - 1 = 0 \;\Rightarrow\; p_2(0) = -1; \qquad t=T:\;\; v=0,\; u=+1:\;\; H = p_2(T)(+1) - 1 = 0 \;\Rightarrow\; p_2(T) = +1 . $$

With $\psi$ affine from $-1$ to $+1$: $\psi(t) = -1 + 2t/T$, which crosses zero at $t = T/2$, and $u^* = \operatorname{sign}\psi$ is $-1$ first, $+1$ second — matching Figure 1. (Which endpoint gets which sign is forced by consistency: starting at $x=1 \gt 0$ with zero velocity, braking means $u = -1$ first.) One more unknown, $p_1$: on the first arc, $H = p_1(-t) + \psi(t)(-1) - 1 = -\bigl(p_1 + 2/T\bigr)\,t \equiv 0$, so $p_1 = -2/T$. Hence

$$ p(t) \;=\; \Bigl(-\tfrac{2}{T},\; -1 + \tfrac{2t}{T}\Bigr). $$

Step 4 — Integrate the state and close the boundary conditions

On the first arc ($u = -1$, $0 \le t \le \tau$, with $\tau$ the switch time): $v = -t$, $x = 1 - t^2/2$. On the second arc ($u = +1$, $\tau \lt t \le T$): starting from $(x_\tau, v_\tau) = (1 - \tau^2/2,\, -\tau)$, integrating gives $v(T) = -\tau + (T - \tau)$ and $x(T) = 1 - \tau^2/2 - \tau(T-\tau) + (T-\tau)^2/2$. The terminal conditions $v(T) = 0$ and $x(T) = 0$ give $T = 2\tau$ and $\tau = 1$:

$$ \tau = 1, \qquad T = 2, \qquad (x_\tau, v_\tau) = \bigl(\tfrac12,\, -1\bigr), \qquad p(t) = (-1,\ t-1). $$

A consistency check that the whole certificate is honest: on the first arc, $H = (-1)(-t) + (t-1)(-1) - 1 = 0$ ✓; on the second, $H = (-1)(t-2) + (t-1)(+1) - 1 = 0$ ✓ — the free-time condition $H \equiv 0$ holds identically, exactly.

switching function psi(t) = t - 1 crossing zero at t = 1 — **Figure 5 — The switching function.** Top: the costate component $p_2(t) = t-1$ is affine and crosses zero exactly once, at $t=1$. Bottom: the control is its sign. The entire "when to switch" question collapsed into the zero of a linear function — that is the costate machinery paying off.

optimal control u = sign psi: bang-bang — **Figure 5 — The switching function.** Top: the costate component $p_2(t) = t-1$ is affine and crosses zero exactly once, at $t=1$. Bottom: the control is its sign. The entire "when to switch" question collapsed into the zero of a linear function — that is the costate machinery paying off.

Step 5 — The phase-plane picture: a feedback law for free

Repurpose the computation: what is the optimal control from any state, not just $(1,0)$? The final arc always rides one of two parabolas through the origin: integrating $\frac{d(v^2)}{dx} = 2v\,\frac{dv/dt}{dx/dt} = 2\frac{u}{1}$ shows that along a $u = +1$ arc ending at rest at the origin, $x = v^2/2$ (with $v \lt 0$); along a $u = -1$ arrival, $x = -v^2/2$ (with $v \gt 0$). Together they form the switching curve

$$ \gamma:\quad x \;=\; -\,\tfrac12\, v\,|v| , $$

and the optimal feedback is: apply $u = -1$ from anywhere above/right of $\gamma$, $u = +1$ from below/left, and switch upon hitting $\gamma$ — equivalently $u^*(x, v) = -\operatorname{sign}\!\bigl(x + \tfrac12 v|v|\bigr)$. Every optimal trajectory in Figure 6 is one parabolic arc, one switch, and a ride along $\gamma$ into the origin. The costate story and the state-space story are the same story: $\psi(t) = p_2(t)$ hitting zero is the trajectory hitting $\gamma$.

phase plane with switching curve and three optimal trajectories — **Figure 6 — The engineer's picture of PMP.** State plane $(x, v)$; dark curve: switching curve $x = -v|v|/2$. Three optimal trajectories from different starts (dots), each a full-brake arc (red) or full-thrust arc (blue) followed by the opposite arc riding the switching curve into the origin (rings: switch points). E.g. from $(0, 1.5)$ the cart overshoots the origin region braking the whole way, switches at $(0.5625, -1.06)$, and comes back along $\gamma$. This figure is the feedback controller.

Step 6 — Check the price interpretation

Minimum-time-to-origin from $(1, 0)$ along the $u=-1$-first branch has the closed form $V(x, v) = v + 2\sqrt{x + v^2/2}$ (on that region). Then $V(1,0) = 2 = T$ ✓, and $\nabla V(1,0) = (1,\,1)$, while the theorem's costate at $t=0$ is $p(0) = (-1, -1)$:

$$ p(t) \;=\; -\,\nabla V\bigl(x^*(t)\bigr) \quad \text{at } t = 0 \text{ — verified exactly.} $$

The costate is the negative gradient of the optimal cost-to-go, exactly as the price interpretation of §3 predicted. (This identity is proved in general wherever $V$ is differentiable — it is the bridge to §7.)

Why there is no third possibility here. Could the maximum condition be silent — $p_2 \equiv 0$ on some interval, leaving $u$ undetermined (a "singular arc")? On such an interval $\dot p_2 = -p_1 = 0$ forces $p \equiv 0$, hence $H = -1$, contradicting the free-time condition $H \equiv 0$. The degenerate case is excluded — for this problem, bang-bang is not just sufficient structure, it is the whole space of candidates.

Takeaway. PMP derived everything: the control structure (bang-bang, from a linear Hamiltonian), the switching structure (at most one switch, from an affine costate), the switch time ($t = T/2$, from $H \equiv 0$), the total time ($T = 2$, from the boundary conditions), and a phase-plane feedback law — with an exact price check at the end.

6PMP recovers the classics: the brachistochrone

The maximum principle contains classical calculus of variations as the special case $\dot x = u$, $U = \mathbb{R}^n$ (unconstrained): the maximum condition becomes $\partial H/\partial u = 0$, i.e. $p = \nabla_{\!u} L$, and the costate equation $\dot p = \nabla_x L$ is exactly Euler–Lagrange, $\frac{d}{dt}\frac{\partial L}{\partial \dot x} = \frac{\partial L}{\partial x}$. To see the machinery digest a genuine classic, take the problem that launched the calculus of variations — and solve it with PMP.

The problem

A bead starts from rest at the origin and slides without friction under gravity $g$. Which wire shape $y(x)$ (with $y$ measured downward) gets it to a target $(a, b)$ in least time? Speed follows from energy conservation, $w = \sqrt{2 g y}$, so the bead trades depth for speed: going deep early buys velocity that pays off later.

The PMP setup

State: $(x, y, w)$ with $w$ the speed; control: the wire's tangent angle $\theta$ below horizontal. Dynamics: $\dot x = w\cos\theta$, $\dot y = w\sin\theta$, $\dot w = g\sin\theta$ (gravity's component along the wire accelerates the bead). Minimum time: $L = 1$, $\varphi = 0$, $T$ free, endpoint fixed. The Hamiltonian:

$$ H \;=\; p_x\, w \cos\theta \;+\; p_y\, w \sin\theta \;+\; p_w\, g \sin\theta \;-\; 1 . $$

The control $\theta$ is unconstrained (a full circle), so the maximum condition is stationarity: $\partial H/\partial\theta = 0$. Writing $\sigma := p_y w + g\, p_w$, the two conditions $\partial H/\partial\theta = 0$ and $H \equiv 0$ (free time) are a 2×2 linear system in $(p_x w,\ \sigma)$:

$$ p_x w \sin\theta - \sigma \cos\theta = 0, \qquad p_x w \cos\theta + \sigma \sin\theta = 1, \qquad\Longrightarrow\qquad \cos\theta = p_x\, w, \quad \sigma = \sin\theta . $$

The transversality conditions $\dot p_x = 0$, $\dot p_y = 0$ say $p_x$ is a constant. Now three facts click together: $\cos\theta = 1/\sqrt{1 + y'^2}$ (geometry: $\tan\theta = dy/dx = y'$), $w = \sqrt{2gy}$ (energy), and $\cos\theta = p_x \sqrt{2gy}$ (just derived, with $p_x$ constant). Combining,

$$ \frac{1}{\sqrt{1 + y'^2}} \;=\; p_x \sqrt{2 g y} \qquad\Longrightarrow\qquad y\,\bigl(1 + y'^2\bigr) \;=\; \frac{1}{2 g\, p_x^2} \;\equiv\; 2r \;=\; \text{const}. $$

This first integral — the famous equation of the cycloid — is the same Beltrami identity that Euler–Lagrange produces, but it fell out of the maximum condition and two constant costates. Its solution, parametrized by the rolling angle $\phi$, is the curve traced by a point on a wheel of radius $r$ rolling under the start point:

$$ x = r\,(\phi - \sin\phi), \qquad y = r\,(1 - \cos\phi), \qquad 0 \le \phi \le \phi_T . $$

Check: $y' = \frac{\sin\phi}{1-\cos\phi} = \cot\frac{\phi}{2}$, so $1 + y'^2 = \csc^2\frac{\phi}{2}$, and since $y = 2r\sin^2\frac{\phi}{2}$,

$$ y\bigl(1 + y'^2\bigr) \;=\; 2r\,\sin^2\!\tfrac{\phi}{2}\cdot\csc^2\!\tfrac{\phi}{2} \;=\; 2r \quad\checkmark\ \text{constant, as the first integral demands.} $$

The descent time, exactly

Along the cycloid, $dx = r(1-\cos\phi)\,d\phi$, $dy = r\sin\phi\,d\phi$, so the arc element is $ds = r\sqrt{(1-\cos\phi)^2 + \sin^2\phi}\, d\phi = 2r \sin\frac{\phi}{2}\, d\phi$, while the speed is $w = \sqrt{2 g\, r (1 - \cos\phi)} = 2\sqrt{gr}\,\sin\frac{\phi}{2}$. The ratio is miraculously clean:

$$ dt \;=\; \frac{ds}{w} \;=\; \sqrt{\frac{r}{g}}\; d\phi \qquad\Longrightarrow\qquad T \;=\; \phi_T \sqrt{\frac{r}{g}} . $$

For our figure's endpoint $(a, b) = (2\pi, 4)$: $r = 2$, $\phi_T = \pi$, giving $T = \pi\sqrt{2/g} = \frac{\sqrt{10}\,\pi}{7} \approx 1.42$ s — verified symbolically. Figure 7 compares against two rivals. The straight line takes $\approx 1.68$ s ($+19\%$): it never builds speed early. Worse, any candidate that leaves the start horizontally — e.g. the parabola $y = x^2/\pi^2$ — takes infinite time: near the start $y \sim x^2$, so $dt = ds/\sqrt{2gy} \sim dx/x$ and the time integral diverges. The cusp at the start of the cycloid is not a defect; it is the optimizer buying velocity as early as the laws of physics allow. (Newton famously identified this curve in one day, anonymously, in 1697.)

cycloid vs straight line vs parabola descent comparison — **Figure 7 — Three wire shapes from $(0,0)$ to $(2\pi, 4)$, descent times computed exactly for $g = 9.8\ \text{m/s}^2$.** Cycloid (blue): 1.42 s. Straight line (dashed): 1.68 s. Parabola (red): leaves the start horizontally, and its descent-time integral diverges — a flat start is infinitely bad. The optimal wire dives first, then flattens: the PMP maximum condition, balancing depth-gained against time-spent, at every instant.

Takeaway. With unconstrained controls the maximum condition collapses to Euler–Lagrange; with the brachistochrone, two constant costates and $H \equiv 0$ re-derive the cycloid and its exact descent time — PMP is the constrained generalization of the classical theory, not a separate one.

7The wider landscape

PMP and dynamic programming: two maps of the same territory

Bellman's dynamic programming attacks the same problem through the value function $V(x, t)$ — the best achievable cost from state $x$ at time $t$ — which satisfies the Hamilton–Jacobi–Bellmann (HJB) partial differential equation $-\partial V/\partial t = \min_{u \in U}\bigl[L(x,u) + \langle \nabla_{\!x} V, f(x,u)\rangle\bigr]$. PMP and HJB are not competitors; they are the local and global views of one optimality notion, and the bridge is the price interpretation of §3:

$$ p(t) \;=\; -\,\nabla_{\!x}\, V\bigl(x^*(t),\, t\bigr) \qquad \text{(where } V \text{ is differentiable)}, $$

as we verified exactly in §5. The maximum condition in the HJB's minimum over $u$ is the same pointwise optimization; the costate equation is the HJB differentiated along the optimal path. What remains is §3's caveat: $V$ is typically not differentiable everywhere (in §5's own example, $V$ has a kink exactly on the switching curve $\gamma$), which is why HJB needs viscosity solutions while PMP, requiring no differentiability of anything but the data, stays in the clean world of ODEs.

	Pontryagin Maximum Principle	Hamilton–Jacobi–Bellman
Output	one optimal trajectory $x^, u^$ + multipliers $p$	the value function $V(x,t)$ — optimal cost from every state
Type of result	necessary condition (candidates)	sufficient condition (the optimum, globally)
Object you solve	2n-dimensional two-point boundary-value ODE system	nonlinear PDE in $n$ spatial dimensions
Control constraints	handled elegantly — pointwise $\arg\max$ over $U$	handled, but with nonsmooth (viscosity) machinery
Scaling	fine in high $n$; the hard part is the BVP	curse of dimensionality beyond $n \approx 4$
Typical use	offline trajectory design: launch guidance, orbit transfers, time-optimal motion	feedback synthesis, games, reinforcement learning's $V$-functions

Relatives and descendants

KKT conditions. PMP is the continuum limit of Karush–Kuhn–Tucker: discretize time and the constraint $\dot x = f(x,u)$ becomes finitely many equality constraints whose Lagrange multipliers, at each time step, are exactly the costate $p(t)$. This is why direct optimal-control solvers (collocation/transcription methods — the world of acados, SNOPT, GPOPS) report "costate estimates": they compute the KKT multipliers of the discretized problem, which converge to $p$. If you have used an NLP solver on a trajectory problem, you have used a discretized maximum principle.
Singular arcs. When $H$ is affine in $u$ and the switching function vanishes on an interval, the maximum condition goes silent there (\S5's remark) and extra higher-order conditions (Legendre–Cohn) are needed. Singularity is the exception, but in min-fuel problems it is the rule — the "coast at exactly the right throttle" arcs.
Existence. PMP is necessary only. Before trusting a candidate you want to know an optimizer exists (Filippov's theorem: compact convex $U$ + continuity in $(x, u)$ suffice) — without it, sequences of controls can chase an unattained infimum, typically by chattering with ever-faster switching (Zeno behavior, the bane of naive sliding-mode and minimum-fuel designs).
Where you meet it. Launch and landing guidance (the Saturn V's powered explicit guidance is PMP-derived), minimum-fuel orbital transfers, time-optimal race/robot trajectories, energy-efficient train driving, and the adjoint gradients inside trajectory optimization and machine-learning backprop — the same backward sweep as §3.

History, briefly. Lev Pontryagin — a blind topologist — led a seminar with Boltyanskii, Gamkrelidze and Mishchenko that attacked time-optimal control problems in the early 1950s. The necessary condition they found in 1956 surprised them (a maximum over the control set, where a minimum was expected) and reorganized the field; the monograph The Mathematical Theory of Optimal Processes (1961) is the citation of record. Bellman's dynamic programming, the sibling viewpoint, was published the same decade. The Soviet–American race to apply it to rockets tracked the race to space, rocket by rocket.

Takeaway. PMP and HJB are the same optimality seen locally (a trajectory with prices $p$) and globally (a field of costs $V$), tied by $p = -\nabla V$; PMP's descendants run inside every serious trajectory optimizer you will touch.

8Recipe card: using PMP in practice

Write the Hamiltonian

$H = \langle p, f(x,u)\rangle - L(x,u)$ (normal case). Confirm the problem is sane: compact $U$, differentiable data, an endpoint condition for every free quantity.

Maximize pointwise over $U$

Solve $u^*(x, p) \in \arg\max_{u \in U} H(x, u, p)$ as algebra, not calculus of variations. Affine-in-$u$ problems give bang-bang or linear feedback; strictly convex $L$ in $u$ gives interior laws, i.e. clipped rational functions of the pair $(x, p)$.

Assemble the canonical system + transversality

$\dot x = \partial H/\partial p$, $\dot p = -\partial H/\partial x$. Pin the $2n$ boundary conditions: $x(0) = x_0$; then $p(T) = -\nabla\varphi$ if $x(T)$ free, $p(T) \perp S$ if constrained, nothing if $x(T)$ fixed. Free $T$ (autonomous): add $H \equiv 0$.

Solve the boundary-value problem

By hand when the structure is kind (affine costates ⇒ few switches; first integrals ⇒ classical curves), else numerically by shooting on the unknown $p(0)$ or by collocation. Unknown switch times are extra unknowns with extra conditions $\psi(t_i) = 0$.

Certify the candidate

PMP is necessary, not sufficient: check existence (Filippov), compare candidates, and — when possible — verify sufficiency via convexity of $H$ in $u$ / concavity of the value function. Cross-check prices: $p = -\nabla V$ where $V$ is known.

Symbol	Meaning	In the braking example
\(x \in \mathbb{R}^n\)	state (defined on first use in §1)	\(x = (x_1, x_2) = (\text{position},\ \text{velocity})\)
\(u \in \mathbb{R}^m\)	control (input you choose)	acceleration command
\(U \subseteq \mathbb{R}^m\)	admissible control set (fixed, compact)	\([-1, 1]\)
\(f(x,u)\)	dynamics: \(\dot x = f(x,u)\)	\((x_2,\ u)\)
\(L(x,u)\)	running cost rate	\(1\) (time)
\(\varphi(x)\)	terminal cost	\(0\)
\(J\)	total cost to minimize	\(T\)
\(p \in \mathbb{R}^n\)	costate (adjoint) — §3	\(p(t) = (-1,\ t-1)\)
\(p^0 \le 0\)	multiplier on the running cost — §4	\(-1\) (normal case)
\(H\)	Hamiltonian — §3	\(p_1 x_2 + p_2 u - 1\)
\(\psi(t)\)	switching function — §5	\(p_2(t)\)
\(V(x)\)	optimal cost-to-go — §7	time-to-origin from state \(x\)

	Pontryagin Maximum Principle	Hamilton–Jacobi–Bellman
Output	one optimal trajectory \(x^, u^\) + multipliers \(p\)	the value function \(V(x,t)\) — optimal cost from every state
Type of result	necessary condition (candidates)	sufficient condition (the optimum, globally)
Object you solve	2n-dimensional two-point boundary-value ODE system	nonlinear PDE in \(n\) spatial dimensions
Control constraints	handled elegantly — pointwise \(\arg\max\) over \(U\)	handled, but with nonsmooth (viscosity) machinery
Scaling	fine in high \(n\); the hard part is the BVP	curse of dimensionality beyond \(n \approx 4\)
Typical use	offline trajectory design: launch guidance, orbit transfers, time-optimal motion	feedback synthesis, games, reinforcement learning's \(V\)-functions