System identification is the art and science of building a dynamical model of a plant from measured input–output data. This page teaches the methods (what the problem is, why each idea exists) and the algorithms (least squares, prediction-error, recursive, and subspace methods) that turn raw samples into a model you can hand to a controller. Everything is one self-contained file: the math is typeset live by an embedded MathJax, all fonts are embedded, and it works offline.
Every model-based control design — an LQR regulator, a Kalman filter, a model-predictive controller, a state observer — starts from the same premise: we have a mathematical description of the plant, a map from what we do (inputs $u$) to what we observe ($y$) over time. Where does that description come from? Two routes exist:
System identification is the discipline that makes the second route rigorous. It answers three questions in order: what structure should the model have?, how do we compute its parameters from data?, and how do we know it is good enough? This page is organized exactly along those questions. The same toolbox also powers soft sensors (virtual measurements), fault detection, digital twins, and adaptive control — anywhere a model must be (re)built from data.
Let's pin down the object we work with for most of this page: a discrete-time, linear time-invariant (LTI) system, sampled at a fixed period. We write it in transfer-function form using the backward shift operator $q^{-1}$, defined by $q^{-1}u(t) = u(t-1)$:
where $u(t)$ is the scalar input at sample $t$, $y(t)$ the scalar output, $e(t)$ is white noise — a sequence of independent, zero-mean random variables with variance $\sigma^2$ (the randomness we cannot predict) — and $G(q)$, $H(q)$ are rational functions of $q^{-1}$. Everything we cannot explain by $u$ is lumped into the disturbance $v(t)$, which itself is shaped by the "noise filter" $H(q)$. If $H(q)=1$, the disturbance is white; if $H(q)$ has dynamics, the disturbance is colored (correlated across time) — a distinction that will drive the whole story of Section 5.
We collect $N$ samples of input and output, and denote this data set
A model set is a family of candidate models parameterized by a vector $\theta \in \mathcal{D}_\mathcal{M} \subseteq \mathbb{R}^{p}$ (here $p$ is the number of parameters — later we also use $p$ for the AIC penalty, same thing):
For example, $M(\theta)$ might be "all second-order transfer functions $G(q; \theta)$ with $\theta$ the four coefficients". The identification problem is then beautifully compact:
Two remarks before we proceed, because they shape every algorithm below:
The whole discipline fits into one loop (Figure 1): design an experiment, gather data, choose a structure, estimate parameters, validate; if validation fails, loop back and change the experiment or the structure. This is why real identification projects are iterative, and why the "algorithm" is only one box in the loop.
Where do candidate structures come from? Think of a spectrum of prior knowledge (Figure 2): at one end you trust physics and only estimate a few constants (a white-box model); at the other end you admit ignorance and let a generic model family absorb everything (black-box). In between lies the enormously practical grey-box regime: the structure comes from insight, the numbers from data.
What makes one candidate model "explain the data better" than another? The answer that organizes almost all of identification is: a model is useful if it predicts. Whatever we will do with the model — simulate, design a controller, run a Kalman filter — we will ask it to forecast the future output from the past. So define the one-step-ahead predictor: the best guess of $y(t)$ using only data up to time $t-1$ and the model $M(\theta)$,
and the prediction error — the part of the measured output we did not foresee:
If the model were perfect and the disturbance were absent, $\varepsilon$ would be zero. With noise, the prediction error is exactly the unpredictable part — which is why, for a well-chosen model, the prediction error is statistically indistinguishable from white noise (we will exploit that test in Section 9).
The prediction-error method (PEM) chooses the parameter vector that minimizes the average squared prediction error:
Why squared error? Three reasons stack up. (i) Geometry and calculus: the criterion is smooth and, for models linear in $\theta$, quadratic — which gives us closed-form solutions (next section). (ii) Statistics: for Gaussian white noise, minimizing $\sum \varepsilon^2$ is exactly maximum likelihood — the estimate inherits the attractive properties of MLEs: consistency, asymptotic normality, and (asymptotically) minimum variance. (iii) Interpretation: minimizing mean squared prediction error is minimizing forecast variance, which is what a controller or observer ultimately cares about.
Notice what the predictor does that a plain simulation does not (Figure 3): a simulator feeds its own past outputs back into the model, so its error accumulates and even diverges if the model is unstable; the predictor, by contrast, feeds the measured past output back in. Prediction is therefore always well-behaved — an unstable plant can still have a perfectly stable one-step predictor — and it uses strictly more information than simulation does.
Why is the predictor itself a model output rather than a free design choice? For a given model $M(\theta) = (G(q,\theta), H(q,\theta))$ the statistically optimal one-step predictor is fixed — it is the conditional expectation of $y(t)$ given the past, which for LTI systems with rational $H$ has a neat closed form. We will meet its concrete instances in Section 5 (the model structures), but the shape of the machinery — pick a model set, derive its predictor, minimize squared prediction error — is the skeleton of PEM, and least squares (next section) is simply the special case where the predictor is linear in the parameters.
Start with the simplest model structure that captures dynamics, the ARX model (Auto-Regressive with eXogenous input). It is a difference equation with a white-noise disturbance entering directly:
with orders $n_a, n_b$ chosen by the user. Expanding and rearranging, the output regresses on its own past and on the past input:
Collect the regressors and the parameters into vectors — this is the crucial trick, because $y(t)$ is now linear in the unknown vector $\theta$:
Note the minus signs: we put the $y$-terms on the left as $-a_i y(t-i)$, so they appear in the regression vector as $-y(t-i)$. Why bother? Because now least squares becomes a linear algebra problem. Stack all $N$ equations, one per time sample, into one matrix equation:
The least-squares (LS) estimate minimizes the sum of squared residuals over all $\theta$:
assuming the $p\times p$ matrix $\Phi^{\mathsf{T}}\Phi$ is invertible (the identifiability condition — Section 8 says exactly what input makes this true). Why this formula? It is the solution of the normal equations $\Phi^{\mathsf{T}}\Phi\,\theta = \Phi^{\mathsf{T}}Y$, obtained by setting the gradient of the quadratic $\|Y-\Phi\theta\|^2 = Y^{\mathsf{T}}Y - 2\theta^{\mathsf{T}}\Phi^{\mathsf{T}}Y + \theta^{\mathsf{T}}\Phi^{\mathsf{T}}\Phi\theta$ to zero. Because the criterion is a convex parabola in $\theta$ (the matrix $\Phi^{\mathsf{T}}\Phi$ is positive semidefinite by construction), that stationary point is the global minimum — no iteration, no local minima, one shot.
The geometry explains the formula. Think of $Y$ as a vector in $N$-dimensional space (one dimension per sample). The matrix $\Phi$ has $p \le N$ columns, so $\{\Phi\theta : \theta \in \mathbb{R}^p\}$ is a $p$-dimensional subspace — the set of all perfectly explainable outputs. We want the point of that subspace closest to $Y$; "closest" in Euclidean distance is the orthogonal projection (Figure 4). The residual $\hat{E} = Y - \Phi\hat\theta_N$ is therefore perpendicular to every column of $\Phi$: $\Phi^{\mathsf{T}}\hat E = 0$, which is the normal equation. The formula $\hat\theta_N = (\Phi^{\mathsf{T}}\Phi)^{-1}\Phi^{\mathsf{T}}Y$ is just the projection expressed in coordinates.
Why trust this estimate? Three classical statistical results pin down when LS is good, and each has a one-line "why":
Let's make this concrete with four noise-free samples of a first-order plant $y(t) = 0.5\,y(t-1) + 1.0\,u(t-1)$, i.e. $n_a = n_b = 1$, true $\theta^\star = (0.5, 1.0)$. Choose input $u(0)=1,\ u(1)=-1,\ u(2)=2,\ u(3)=-1$ and initial condition $y(0)=1$, then the recurrence gives $y(1)=1.5,\ y(2)=-0.25,\ y(3)=1.875,\ y(4)=-0.0625$. Stacking rows $t=1,\ldots,4$ with regression vector $\varphi(t) = [y(t-1),\ u(t-1)]^{\mathsf{T}}$:
| $t$ | $u(t-1)$ | $y(t-1)$ | $y(t)$ | $\varphi(t)^{\mathsf{T}}$ |
|---|---|---|---|---|
| 1 | 1 | 1 | 1.5 | $(1,\ 1)$ |
| 2 | −1 | 1.5 | −0.25 | $(1.5,\ -1)$ |
| 3 | 2 | −0.25 | 1.875 | $(-0.25,\ 2)$ |
| 4 | −1 | 1.875 | −0.0625 | $(1.875,\ -1)$ |
Then the normal-equation ingredients — each entry is a sum over the four rows — and their solution are:
exactly the true parameters — as promised for the noise-free case. With noise, $\hat\theta$ would scatter around $(0.5, 1.0)$ with covariance $\sigma^2(\Phi^{\mathsf{T}}\Phi)^{-1}$; more data shrinks the scatter like $1/\sqrt{N}$. (Keep the numbers in sight when you play with the live demo in Section 11: it is this same computation, scaled up.)
Section 4's elegance rests on one assumption: the disturbance entering the ARX equation is white. Real plants violate this constantly. Sensor noise, unmeasured disturbances, and load changes arrive after the dynamics, so the true structure is closer to
with $v(t)$ a colored (time-correlated) noise, e.g. $v(t) = \tfrac{1}{1-0.7q^{-1}}\,e(t)$. The regression vector $\varphi(t)$ contains past outputs $y(t-1),\ldots$ — but each past output contains its own past noise, and colored noise persists across time. Hence $\varphi(t)$ and $v(t)$ are correlated: $\mathbb{E}[\varphi(t)\,v(t)] \neq 0$. Substitute $Y = \Phi\theta + V$ into the LS formula and take expectations (as $N\to\infty$):
There is the bias: a systematic error proportional to the noise–regressor correlation $\mathbb{E}[\varphi v]$. Least squares is not estimating the plant; it is estimating "plant plus whatever leaks through the correlation". Figure 5 shows why: the criterion surface that LS actually minimizes is a shifted version of the ideal one — its bottom sits at the biased $\hat\theta$ rather than at the truth $\theta^\star$.
Two families of remedies exist, and both show up throughout the field. Instrumental-variable (IV) methods swap the regressors for proxies; general PEM models the noise itself. Let's take IV first because it is one line, then PEM because it is the deep answer.
The bias term is caused by $\mathbb{E}[\varphi v] \neq 0$. If we could multiply the equation by a vector $\zeta(t)$ that is correlated with the regressors but uncorrelated with the noise, the noise term would average out. That is the instrument. The IV estimate is
with $\zeta(t)$ chosen so that $\mathbb{E}[\zeta(t)\,v(t)] = 0$ (noise-blindness) while $\tfrac{1}{N}\Psi^{\mathsf{T}}\Phi$ stays nonsingular (still enough information). In the old normal equation the noise term was $\Phi^{\mathsf{T}}V$; now it is $\Psi^{\mathsf{T}}V$, and its expectation vanishes. Why would LS work at all then? Because when the noise is white, choosing $\zeta = \varphi$ makes IV exactly LS — LS is the special case where the regressors are their own valid instruments.
The deep fix is to parameterize both the plant and the noise filter and estimate them together. Then the predictor (Section 3) filters out the correlation by construction. The standard families, ordered by how the disturbance enters (Figure 6):
Each structure induces its own one-step predictor (from Section 3's recipe, $\hat y = H^{-1} G u + (1-H^{-1}) y$):
The predictors for ARMAX, OE and BJ are nonlinear in $\theta$ (parameters appear in denominators), so we cannot solve the PEM criterion in closed form. Instead we iterate — the standard algorithm is Gauss–Newton: linearize the predictor about the current guess and take a damped step,
where $\psi(t,\theta) = \tfrac{\partial \hat y}{\partial \theta}$ is the gradient of the predictor, and the Hessian is approximated by $\tfrac{2}{N}\sum \psi \psi^{\mathsf{T}} \succeq 0$ (a Gauss–Newton, not full Newton, choice: it stays positive semi-definite and ignores curvature of $\varepsilon$ itself). The beautiful part: those gradients are themselves filtered regression vectors. For ARMAX, for instance, $\psi(t) = \tfrac{1}{C(q)}\,\big[-y(t-1),\ldots,\ u(t-1),\ldots,\ -\varepsilon(t-1),\ldots\big]^{\mathsf{T}}$: the very same regressors as in §4, but passed through $1/C(q)$ and with lagged prediction errors in place of the noise. So PEM is "least squares on filtered regressors, iterated" — Section 4 reappears inside the loop. One iteration from a good start already removes most of the bias.
Batch least squares needs all $N$ samples at once. Three real-world pressures push us to a recursive version that updates the estimate sample by sample: (i) the plant drifts (aging actuators, temperature swings) and we want to track it; (ii) storage and latency — sensors stream, controllers must act now; (iii) adaptive control — identify and redesign the controller every sample, the "self-tuning regulator" idea.
RLS is not an approximation of LS; it computes exactly the same answer at every time $t$ as a batch fit over all data seen so far, but with a constant per-step cost. The derivation is one algebraic identity: write the batch solution at time $t$ in terms of the solution at $t-1$ and the new sample $(u(t), y(t))$, then apply the matrix inversion lemma to avoid inverting $\Phi^{\mathsf{T}}\Phi$ afresh. Let $\varepsilon(t)$ be the prediction error of the previous estimate against the new sample, and $P(t) = (\Phi_t^{\mathsf{T}}\Phi_t)^{-1}$ the "information inverse" (later interpretable as a covariance). The update is:
Read the structure aloud and it makes complete sense: new estimate = old estimate + gain × how wrong the old estimate just was. The gain $K(t)$ is a vector in parameter space that says "the new sample mostly corrects these directions, not those" — it is large along directions where the information matrix $P^{-1}$ is weak. The scalar denominator $1 + \varphi^{\mathsf{T}}P\varphi$ normalizes by the uncertainty of the prediction itself. This is precisely the Kalman filter update, specialized to a static parameter ($\theta(t+1) = \theta(t)$): RLS is the Kalman filter for the "constant state" $\theta$, with $P(t) = \operatorname{cov}(\hat\theta(t))$.
Tracking a drifting plant: the forgetting factor. The covariance update as written keeps accumulating all past information, so after long runs $P(t) \to 0$ and the estimate freezes, no longer able to follow changes. The fix is to discount old samples exponentially by a forgetting factor $0 \ll \lambda \le 1$:
which is equivalent to minimizing $\sum_{k=1}^{t} \lambda^{t-k}\varepsilon(k)^2$ — recent samples weigh $\lambda^{-j}$ more. The price of tracking is variance: with $\lambda < 1$ the effective data window is only $1/(1-\lambda)$ samples, so the estimate jitters. Do not forget (pun intended): $\lambda$ is the online counterpart of "how much data do you believe" that we will meet again as model-order selection in Section 9.
RLS is the algorithmic seed of a whole family: RELS (recursive extended least squares) and the recursive prediction-error methods (RPEM) generalize it to ARMAX by including lagged prediction errors in the regression vector — exactly the filtered-regressor idea of Section 5, now online. Every adaptive controller since the 1970s self-tuning regulator ([Åström & Wittenmark], §10) owes its existence to this loop.
PEM from Section 5 is statistically beautiful but computationally and structurally demanding: you choose a model family, orders, and a noise model; then you run a nonlinear search that can stall in local minima — and everything scales badly to multi-input multi-output (MIMO) plants, where the parameterization choices multiply. Subspace methods answer with a radically different strategy: go straight from the raw data to a state-space model using linear algebra only — no user-chosen noise model, no iteration, no parameterization. The price is some statistical efficiency (they are consistent but not minimum-variance in general), which practice happily pays for robustness.
First, recall what a state-space model is — at every sample $t$ the plant has a compact summary, the state $x(t) \in \mathbb{R}^{n}$, from which the future unfolds linearly:
with system matrices $A$ ($n\times n$), $B$, $C$, $D$. If we knew $A,B,C,D$, everything downstream (Kalman filter, MPC, LQG) is standard. Subspace identification's task: recover $A,B,C,D$ (and, roughly, the order $n$) directly from the data blocks.
The key structural fact. Stack the future outputs into a block-Hankel matrix -- rows are successive time shifts of the measurements, columns are different starting times:
i.e. the future-output matrix $Y_f$ lives in the row space of the inputs $U_f$ plus the column space of the observability matrix $O_i$, driven by the (unknown) state matrix $X_f$. If we could remove the input contribution, the remaining part would live exactly in $\operatorname{range}(O_i)$ — and $O_i$ contains $A$ and $C$ by construction. That removal is a projection (an oblique one, along the row space of $U_f$ onto the past-data space $W_p = [U_p;\ Y_p]$):
Then one singular value decomposition (SVD) reveals both the order and the matrices (Figure 8): the number of "large" singular values of $\widehat{\mathcal{O}}$ is the state dimension $n$ (the rest are noise-floor), the left singular vectors give $O_i$ up to a similarity transform, and $A, C$ fall out of $O_i$'s shift-invariance: dropping the first/last block rows of $O_i$ and solving a least-squares problem yields $A$; $C$ is just the first block row. Finally $B, D$ are estimated by one more least-squares step on the state equation with the reconstructed states. Two famous algorithms instantiate this skeleton — N4SID (van Overschee & De Moor 1994, combined deterministic–stochastic by using both past input and output as instruments) and MOESP (Verhaegen, based on an LQ factorization that decouples the input before the SVD); they share the projection-then-SVD core and differ in how the projections are computed and combined.
Why the SVD is the right tool here. The SVD of a matrix is the canonical way to separate a low-rank "signal" from a "noise" part: the rank of the noiseless $\widehat{\mathcal{O}}$ is exactly $n$ (its columns span $\operatorname{range} O_i$), and noise inflates the smaller singular values. Looking at the scree plot and choosing $n$ where the values plateau is the subspace analogue of choosing the order $n_a, n_b$ in Section 4 — we will formalize both choices in Section 9. This is also the modern, data-driven heir of the classical Ho–Kalman realization theorem (1966), which recovered a minimal state-space realization from the impulse response: subspace methods recover it from input–output data in one shot (the impulse response is conceptually the special case of a suitable input).
What you give up. Subspace estimates are consistent under mild conditions (persistently exciting inputs, mild noise assumptions — roughly Section 8's conditions) but they are not maximum likelihood: the projection throws away the fine noise structure that PEM exploits, so for small samples and very precise noise models PEM wins. In return you get a numerically robust, MIMO-native, iteration-free black-box route to exactly the $(A,B,C,D)$ that Kalman filtering and MPC want. The modern data-driven methods (e.g. data-enabled predictive control) push this "model-free from data" philosophy one step further by applying subspace-style projections directly in the controller — Section 10's outlook.
Every algorithm so far assumed the numbers can be extracted — that the normal matrix $\Phi^{\mathsf{T}}\Phi$ is invertible, that the excitation "reaches" the dynamics. That is not free: it is a contract between the model structure and the input signal, and when the contract is violated no estimator can help. Two distinct notions are usually conflated under "identifiability" (Ljung makes the split explicit):
Persistent excitation in one line: a scalar signal is persistently exciting of order $n$ when, in the limit, the empirical autocorrelation matrix built from $n$ consecutive samples is nonsingular — roughly, its spectrum is nonzero at at least $n$ frequencies. Consequences for inputs (Figure 9):
Practical rules of thumb follow directly. For an ARX model, the regressors span $p = n_a + n_b$ directions, so $\mathrm{PE}(n_a + n_b)$ is the ticket to a nonsingular normal matrix; OE-type structures need roughly order $2n$ (effectively you must excite enough to separate plant poles from noise-model poles); and in closed-loop identification the feedback correlation makes things subtler — the input spectrum is shaped by the controller, and standard open-loop conditions fail (one needs direct/indirect or joint input–output variants). Sampling must be faster than the fastest relevant dynamics (anti-aliasing first!), and long enough that the averages in the PE condition actually converge — the estimate's covariance shrinks only like $\sigma^2/N$, so there is no shortcut around collecting data.
Estimation is only half the loop (Figure 1). The other half asks the uncomfortable question: would this model fool us on data it has never seen? Three complementary guards exist, and skipping any of them is how confident but wrong models get shipped.
If the model captured the plant and the noise model captured the disturbance, the one-step prediction errors $\hat\varepsilon(t) = y(t) - \hat y(t \mid t-1; \hat\theta_N)$ should be unpredictable — that is, white. Test this statistically with the sample autocorrelation of the residuals and their cross-correlation with the input (valuable beyond $\tau=0$: residual energy at lag $\tau$ vs $u$ means a missed input path):
intuitively: if any lag sticks out of the band $\pm 1.96/\sqrt{N}$, there is still a predictable pattern in the residuals — a systematic dynamics or noise-model flaw — and the model is incomplete (Figure 11, right). Exactly the residual shapes we can fix: colored residual with structure $\Rightarrow$ add dynamics or a noise model; residual correlated with $u$ $\Rightarrow$ missing input path.
Residual analysis uses the same data the model was fitted on, so it can be fooled by overfitting (Section 10's bias–variance picture). The honest benchmark is to split the data: fit on the training part, then evaluate prediction error on the validation part (or use an entirely fresh experiment). If the model predicts unseen data about as well as it predicts the training data, it has learned dynamics — not memorized samples. This is the same held-out instinct modern machine learning uses; identification papers call it "cross-validation" or "fresh-data validation". For parametric fits the leave-one-out and $k$-fold variants are the common incarnations.
More parameters always reduce the training error (the criterion $V_N$ can only decrease as the model family grows) but add variance: the estimate becomes sensitive to the specific noise realization, and generalization degrades. Figure 10 shows the canonical picture: training error falls monotonically with complexity $p$; validation error is U-shaped, initially falling as we remove bias (undershooting), later rising as variance takes over (overshooting).
When you cannot afford hold-out data (small $N$), information criteria approximate validation from the fit itself. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) both penalize the minimum criterion value by the number of parameters $p$ (which equals $n_a + n_b + \ldots$ for PEM structures):
and you pick the model minimizing the criterion. Why these formulas? AIC stems from the asymptotic likelihood (Section 3's MLE view): $N\log V_N$ is essentially minus twice the log-likelihood, and $2p$ is the expected overfitting penalty under asymptotics; BIC adds the stiffer $p\log N$ penalty, trading some fit for guaranteed consistency as $N$ grows. Neither replaces validation — they are cheap proxies that encode the same U-curve of Figure 10 into one number. For subspace identification, the same selection question is answered by the SVD gap (Figure 8): order $n$ = number of singular values above the noise floor.
One more layer completes the story: uncertainty. A point estimate $\hat\theta_N$ without a covariance is a promise without a warranty — the PEM covariance $\operatorname{cov}(\hat\theta_N) \approx \tfrac{\sigma^2}{N} \left[\tfrac{1}{N}\sum \psi \psi^{\mathsf{T}}\right]^{-1}$ (with $\psi$ of Section 5) propagates directly into controller-design margins, robust stability certificates, and Kalman-filter tuning.
Step back and you see that "identification" is not a separate little corner of control theory — it is the same statistical machinery that appears everywhere in estimation, wearing different hats:
Three horizons are worth knowing about before you leave: nonlinear identification (grey-box with physics, NARX/NARMAX models, neural state-space models, operator-Koopman lifting — same PEM criterion, harder optimization), closed-loop identification (identify under feedback, e.g. while an existing controller runs — the correlation calculus of Section 5 reappears and requires specialized methods), and robust/uncertainty-aware design (propagate $\operatorname{cov}(\hat\theta_N)$ into control design, robust controllers against the identified model class). The toolbox below is the load-bearing foundation of all of them: predict → fit → validate, then decide what to measure next.
This widget simulates a second-order plant and fits an ARX(2,2) model to its input–output data with the batch least-squares formula of Section 4 — exactly the computation of the worked example, scaled up. For readability the model is written in the direct difference-equation convention $y(t)=a_1 y(t-1)+a_2 y(t-2)+b_1 u(t-1)+b_2 u(t-2)+e(t)$ (the same ARX as $A(q)y(t)=B(q)u(t)+e(t)$ with $A(q)=1-a_1q^{-1}-a_2q^{-2}$). Two switches make the story visible:
| Symbol | Meaning |
|---|---|
| $t$, $N$ | discrete time index; number of data samples |
| $q^{-1}$ | backward shift operator, $q^{-1}u(t)=u(t-1)$ |
| $u(t)$, $y(t)$, $e(t)$ | input, output, white noise (zero-mean, variance $\sigma^2$) |
| $v(t)$ | disturbance; $v = H(q)e(t)$, colored if $H$ has dynamics |
| $G(q)$, $H(q)$ | plant and noise transfer functions |
| $A(q), B(q), C(q), D(q), F(q)$ | polynomials of the ARX/ARMAX/OE/BJ families |
| $n_a, n_b, n_c, n_d, n_f$ | orders of those polynomials |
| $\theta$, $p$ | parameter vector; number of parameters ($p = \dim\theta$) |
| $\varphi(t)$, $\Phi$ | regression vector; $N\times p$ regression matrix (rows $\varphi(t)^{\mathsf{T}}$) |
| $Y$, $E$, $V$ | stacked output, noise, disturbance vectors |
| $\hat y(t\mid t-1;\theta)$, $\varepsilon(t,\theta)$ | one-step predictor; prediction error |
| $V_N(\theta)$, $\hat\theta_N$ | PEM criterion $\frac1N\sum\varepsilon^2$; its minimizer |
| $\psi(t,\theta)$ | gradient of the predictor w.r.t. $\theta$ (filtered regressors) |
| $\zeta(t)$, $\Psi$ | instrument vector; instrument matrix (IV methods) |
| $K(t)$, $P(t)$, $\lambda$ | RLS gain, covariance ("information inverse"), forgetting factor |
| $A,B,C,D$, $x(t)$, $n$ | state-space matrices, state, state dimension |
| $O_i$, $H_i$, $X_f$ | extended observability, Toeplitz input matrix, future-state block (subspace) |
| $U_p,Y_p,U_f,Y_f$, $W_p$ | past/future input-output block-Hankel matrices; $W_p=[U_p;Y_p]$ |
| $\widehat{\mathcal{O}}$, $U\Sigma V^{\mathsf{T}}$ | projected future-output matrix; its SVD |
| $\hat\rho(\tau)$ | sample autocorrelation of residuals at lag $\tau$ |
| AIC, BIC | Akaike / Bayesian information criteria |
| PE$(n)$ | persistently exciting of order $n$ (input richness) |
ARX auto-regressive with exogenous input · ARMAX + moving-average noise · OE output-error · BJ Box–Jenkins · PEM prediction-error method · LS least squares · IV instrumental variables · RLS recursive LS · PE persistent excitation · PRBS pseudo-random binary sequence · SVD singular value decomposition · MIMO multi-input multi-output · MPC model-predictive control · MLE maximum-likelihood estimate.
Identification turns input–output data into a model by (1) choosing a model set that encodes how much physics you trust, (2) minimizing squared prediction error — closed-form least squares for ARX models, iterated Gauss–Newton for ARMAX/OE/BJ, one-shot linear algebra (projection + SVD) for subspace/state-space models, all with a recursive variant for online use — (3) supplying an input rich enough (persistent excitation) that the parameters are identifiable, and (4) validating that nothing predictable is left in the residuals, on data the model has never seen.
Self-contained single file · math typeset by an embedded MathJax · all fonts embedded · works offline · built with the teach-an-engineer skill.