optimization

Two more if those are insufficient. Provide your known-good incumbent as a warm start so
the solver begins with a strong bound. And check whether a reformulation exists — many
scheduling problems have a time-indexed formulation and an alternative
sequence-based or flow-based one with much tighter relaxations, and switching formulation
often beats any amount of solver tuning.


optimization

## Stochastic and Global Optimization

Stochastic gradient descent replaces the full gradient with an unbiased estimate from a
random subset of the data. Each step is far cheaper, and the noise it introduces is not
purely harmful: it helps escape saddle points, which dominate high-dimensional non-convex
landscapes, and it acts as an implicit regularizer. Convergence requires the step size to
decrease appropriately — the Robbins-Monro conditions require the step sizes to sum to
infinity while their squares sum to a finite value, which is why a constant step size
converges only to a neighbourhood of the optimum rather than to the optimum itself.


optimization

Adaptive methods scale the step per coordinate using accumulated gradient statistics.
AdaGrad divides by the square root of the accumulated squared gradients, which handles
sparse features well and monotonically shrinks the effective step until learning stalls.
RMSProp replaces the accumulation with an exponential moving average, fixing that decay.
Adam adds momentum and bias correction and is the practical default for non-convex deep
learning, though on convex problems well-tuned SGD with momentum often generalizes
slightly better.


optimization

Global optimization confronts problems with many local minima and no structural
guarantees. There is no method that both finds the global optimum and terminates quickly
in general — the difficulty is fundamental rather than a gap in current technique. The
practical approaches trade guarantees for usefulness.


optimization

Global and derivative-free methods:
  multi-start local     run a local method from many random starts; simple, often
                        surprisingly effective, embarrassingly parallel
  simulated annealing   accept worsening moves with a temperature-dependent
                        probability, cooling over time; converges in theory under
                        impractically slow cooling schedules
  genetic algorithms    population, selection, crossover, mutation; general and
                        usually outperformed by problem-specific methods
  particle swarm        population following personal and global bests
  CMA-ES                adapts a covariance matrix of the search distribution; the
                        strongest general-purpose derivative-free method for
                        continuous problems of moderate dimension
  Bayesian optimization builds a surrogate (usually a Gaussian process) and
                        selects the next evaluation by an acquisition function
                        balancing exploration and exploitation; the right choice
                        when each evaluation is very expensive
  Nelder-Mead           simplex reflection and contraction; no theory, widely used
                        for low-dimensional smooth problems
  branch and bound      exact global optimality for problems with computable bounds


optimization

Bayesian optimization deserves particular note because it targets a specific and common
situation: the objective is expensive to evaluate, has no derivatives, and the budget is
perhaps fifty to a few hundred evaluations. Hyperparameter tuning, physical experiment
design, and simulation-based engineering all fit. The surrogate model captures both the
predicted value and the uncertainty, and the acquisition function — expected improvement
being the standard choice — decides where to sample next by weighing likely improvement
against the value of reducing uncertainty. Under a few hundred evaluations it
substantially outperforms random search; beyond a few thousand the surrogate's own cost
and the curse of dimensionality erode its advantage, and random or evolutionary search
becomes competitive again.


optimization

The recurring practical advice across all of these is to spend effort on formulation
before algorithm selection. Reformulating a non-convex problem into a convex one,
tightening a relaxation, exploiting separable structure, reducing dimensionality, or
finding a special structure like total unimodularity typically yields improvements of
orders of magnitude, while switching between comparable algorithms yields factors of two
or three. The most common cause of an intractable optimization problem is a formulation
that discarded structure the problem actually had.


probability_theory

## Axioms, Measure, and Conditioning

Kolmogorov's axioms found probability on measure theory. A probability space is a triple
consisting of a sample space of outcomes, a sigma-algebra of events — closed under
complement and countable union — and a probability measure assigning each event a value
in [0,1], with the whole space measuring one and countable additivity holding over
disjoint events. Everything else in the subject is a theorem.


probability_theory

The restriction to a sigma-algebra rather than all subsets is not fastidiousness. For an
uncountable sample space, no measure can assign a consistent size to every subset while
remaining translation-invariant and countably additive — the Vitali construction, using
the axiom of choice, produces a set for which any assigned value leads to contradiction.
The sigma-algebra is the collection of sets on which a consistent measure exists, and
restricting attention to it is what makes the theory coherent.


probability_theory

Conditional probability P(A|B) = P(A and B)/P(B) restricts the sample space to B and
renormalizes. Independence, P(A and B) = P(A)P(B), is equivalent to P(A|B) = P(A) when
P(B) is positive: knowing B tells you nothing about A. Two subtleties recur. Pairwise
independence does not imply mutual independence — three events can be pairwise
independent while the third is determined by the first two, the standard example being
two fair coin flips and the event that they agree. And conditional independence neither
implies nor is implied by unconditional independence: two variables can be dependent
overall and independent given a third (explaining away a common cause), or independent
overall and dependent given a third (conditioning on a common effect, the collider
structure that makes "controlling for everything" a source of bias rather than rigour).


probability_theory

Foundational results:
  axioms              P(A) >= 0;  P(Omega) = 1;  countable additivity on disjoints
  complement          P(A^c) = 1 - P(A)
  inclusion-exclusion P(A or B) = P(A) + P(B) - P(A and B)
  chain rule          P(A1...An) = P(A1) P(A2|A1) ... P(An|A1...An-1)
  total probability   P(A) = sum over a partition B_i of P(A|B_i) P(B_i)
  Bayes               P(A|B) = P(B|A) P(A) / P(B)
  union bound         P(union A_i) <= sum P(A_i);  crude and universally applicable
  Borel-Cantelli I    sum P(A_n) < infinity implies P(A_n infinitely often) = 0
  Borel-Cantelli II   independent A_n with divergent sum occur infinitely often


probability_theory

The union bound's crudeness is its strength: it requires no independence assumption
whatsoever and holds always. It is the workhorse of high-probability arguments across
computer science and statistics — bounding the probability that any of many bad events
occurs, which is exactly the structure of multiple-testing corrections, of concentration
arguments over a finite hypothesis class, and of most randomized algorithm analyses.


probability_theory

Q: Events A and B are independent, and A and C are independent. Does it follow that A is
independent of B and C jointly?
A. Yes, independence is transitive
B. No; pairwise independence does not imply mutual independence
C. Yes, if B and C are independent
D. Only if all three have equal probability
Reasoning: Pairwise independence is strictly weaker than mutual independence. Take two
fair coin flips B and C and let A be the event that they match; A is independent of each
individually but is determined by the pair. Option C is also false — B and C are
independent in that very counterexample. Answer: B.


probability_theory

## Random Variables and Transformations

A random variable is a measurable function from the sample space to the reals; its
distribution is the pushforward measure. The cumulative distribution function F(x) =
P(X <= x) characterizes it completely and always exists, which is why it is the right
general object — densities exist only for absolutely continuous distributions and mass
functions only for discrete ones, but every random variable has a CDF.


probability_theory

Expectation is the integral of the variable against the probability measure. Its
linearity holds unconditionally, without any independence assumption, and this is the
most exploited fact in applied probability. The expected number of fixed points in a
random permutation is computed by writing it as a sum of indicator variables, each with
expectation 1/n, giving exactly 1 regardless of n — a calculation that would be
forbidding directly and is immediate by linearity, despite the indicators being
dependent. Variance is not linear, and Var(X + Y) = Var(X) + Var(Y) requires
uncorrelatedness.


probability_theory

Moment generating functions and characteristic functions encode a distribution in
transform space, converting convolution into multiplication. The sum of independent
random variables has a density given by the convolution of theirs, which is awkward
directly and trivial in transform space, which is why sums are analyzed there. The
characteristic function, being the Fourier transform of the density, always exists
whereas the MGF may not — the Cauchy and heavy-tailed distributions have no MGF — and
Levy's continuity theorem, that pointwise convergence of characteristic functions implies
convergence in distribution, is the machinery behind the central limit theorem.


probability_theory

Transformation rules:
  discrete            P(Y = y) = sum over x with g(x) = y of P(X = x)
  monotone continuous f_Y(y) = f_X(g^{-1}(y)) |d g^{-1}/dy|
  multivariate        f_Y(y) = f_X(g^{-1}(y)) |det J|,  J the Jacobian
  sum of independents convolution; multiply MGFs or characteristic functions
  linear              E[aX + b] = aE[X] + b;  Var(aX + b) = a^2 Var(X)
  law of the unconscious statistician  E[g(X)] = integral g(x) f(x) dx, no need
                      to derive the distribution of g(X)
  tower property      E[X] = E[E[X|Y]]
  variance decomposition  Var(X) = E[Var(X|Y)] + Var(E[X|Y])


probability_theory

The variance decomposition — the law of total variance — is worth internalizing because
it names the two sources of uncertainty in any hierarchical situation. The first term is
the average within-group variance, the second is the variance of the group means. In
statistics this is the ANOVA decomposition; in machine learning it separates aleatoric
from epistemic uncertainty; in hierarchical modelling it is exactly what determines how
much shrinkage a group receives. Whenever an analysis needs to attribute variability
between "noise within" and "differences between", this identity is the formal statement.


probability_theory

The Jacobian in the change-of-variables formula is where errors concentrate, and it has
a clean interpretation: a transformation that stretches a region must reduce the density
there proportionally, so that total probability is preserved. Forgetting it produces
densities that do not integrate to one, which is a useful self-check. This same
determinant is what normalizing flows track explicitly in order to remain valid
probability models under learned invertible transformations.


probability_theory

## Limit Theorems

The laws of large numbers state that sample averages converge to expectations. The weak
law gives convergence in probability: for any epsilon, the probability that the sample
mean deviates by more than epsilon tends to zero. The strong law gives almost-sure
convergence: the sample mean converges for every outcome outside a set of probability
zero. The distinction is about whether the sequence of averages settles down permanently
(strong) or merely becomes unlikely to deviate at any fixed time (weak). Both require
only a finite mean; neither requires finite variance, though the simplest proofs assume it.


probability_theory

The central limit theorem says that the standardized sample mean converges in
distribution to a standard normal, regardless of the underlying distribution, provided
the variance is finite. This universality is the reason the normal distribution appears
everywhere: any quantity that is a sum of many small independent contributions is
approximately normal, whatever those contributions look like individually. It also
explains why measurement error, biological variation, and aggregate financial returns
over short horizons all exhibit approximately normal behaviour.


probability_theory

The finite-variance condition is essential and its failure is not exotic. For
heavy-tailed distributions with infinite variance, sums converge instead to stable
distributions with power-law tails, and the scaling is n^{1/alpha} rather than
sqrt(n). The Cauchy distribution is the extreme case: the sample mean of n Cauchy
variables has exactly the same distribution as a single observation, so averaging
provides no improvement at all no matter how much data is collected. Since financial
returns, city sizes, network degrees, and file sizes all exhibit heavy tails, applying
CLT-based reasoning to them produces confidently wrong error bars.


probability_theory

Convergence modes, from strongest to weakest:
  almost sure          P(lim X_n = X) = 1;  the strong law
  in probability       P(|X_n - X| > eps) -> 0 for every eps;  the weak law
  in L^p               E|X_n - X|^p -> 0
  in distribution      F_n(x) -> F(x) at continuity points;  the CLT
  relationships        a.s. implies in probability implies in distribution;
                       L^p implies in probability; none of the converses hold
  Slutsky              if X_n converges in distribution and Y_n in probability to a
                       constant, then sums and products behave as expected
  delta method         sqrt(n)(g(Xbar) - g(mu)) -> Normal(0, g'(mu)^2 sigma^2)


probability_theory

The delta method is how asymptotic standard errors are obtained for functions of
estimates. It is a first-order Taylor expansion combined with the CLT: a smooth function
of an asymptotically normal estimator is itself asymptotically normal with variance
scaled by the squared derivative. This gives the standard error of a log-odds, of a
ratio, of a transformed correlation, and it fails exactly where the derivative vanishes,
in which case a second-order expansion gives a chi-square rather than a normal limit.


probability_theory

Q: The sample mean of n independent Cauchy random variables has:
A. Variance sigma^2/n
B. The same distribution as a single Cauchy variable
C. An approximately normal distribution for large n
D. Variance decreasing as 1/sqrt(n)
Reasoning: The Cauchy distribution has undefined mean and infinite variance, so neither
the law of large numbers nor the central limit theorem applies. It is stable with index
one, so the average of n copies is distributed exactly as one copy. Collecting more data
does not improve the estimate at all — a stark demonstration that the CLT's finite-variance
hypothesis is load-bearing. Answer: B.


probability_theory

## Concentration Inequalities

Concentration inequalities bound the probability that a random variable deviates from
its expectation, and they are the quantitative backbone of modern statistics, learning
theory, and randomized algorithms. Their appeal is that they hold at finite sample
sizes, unlike asymptotic results, and often require very weak assumptions.


probability_theory

Markov's inequality is the weakest and most general: for a non-negative random variable,
P(X >= a) <= E[X]/a. It requires only non-negativity and the existence of a mean.
Chebyshev's inequality follows by applying Markov to the squared deviation, giving
P(|X - mu| >= k sigma) <= 1/k^2, which requires finite variance and holds for any
distribution — the guarantee that at least 75% of any distribution lies within two
standard deviations, and 89% within three. These bounds are loose for well-behaved
distributions, where the normal gives 95% and 99.7%, but they cannot be improved without
further assumptions, and there exist distributions attaining them.


probability_theory

Chernoff bounds achieve exponential decay by applying Markov to the exponentiated
variable and optimizing over the free parameter. For sums of independent bounded
variables, Hoeffding's inequality gives P(|Xbar - mu| >= t) <= 2 exp(-2 n t^2 / (b-a)^2),
which decays exponentially in n rather than polynomially. This is the qualitative leap
that makes learning theory possible: it says that with n samples, the empirical mean is
within t of the truth except with probability exponentially small, so a union bound over
a large but finite hypothesis class still leaves high confidence.


probability_theory

Concentration toolkit:
  Markov            P(X >= a) <= E[X]/a;  X >= 0 only
  Chebyshev         P(|X - mu| >= k sigma) <= 1/k^2;  any distribution
  Chernoff          exponential bounds via the MGF; needs independence
  Hoeffding         bounded independent summands; exp(-2 n t^2 / range^2)
  Bernstein         uses the variance; sharper than Hoeffding when variance is
                    small relative to the range
  McDiarmid         bounded differences; applies to any function of independent
                    variables that changes little when one input changes
  Jensen            E[g(X)] >= g(E[X]) for convex g;  the source of many bounds
  Cauchy-Schwarz    E[XY]^2 <= E[X^2] E[Y^2]


probability_theory

McDiarmid's bounded-differences inequality is the most broadly applicable of these
because it does not require the quantity of interest to be a sum. Any function of
independent random variables whose value changes by at most c_i when the i-th input is
altered concentrates around its mean with a Hoeffding-type bound. This covers the
generalization error of a learning algorithm, the size of the largest connected component
in a random graph, and the length of the longest common subsequence — none of which are
sums, all of which are stable under single-coordinate changes.


probability_theory

Jensen's inequality is the source of a great deal of structure. For a convex function,
the expectation of the function is at least the function of the expectation. It gives
immediately that E[X^2] >= E[X]^2, which is variance non-negativity; that the arithmetic
mean dominates the geometric mean; that the Kullback-Leibler divergence is non-negative;
and that the evidence lower bound in variational inference is indeed a lower bound. When
an inequality between an expectation and a function of an expectation is needed, Jensen
is the first thing to try.


probability_theory

## Stochastic Processes and Markov Chains

A stochastic process is a collection of random variables indexed by time. The Markov
property — the future depends on the past only through the present — is the structural
assumption that makes analysis tractable and is far less restrictive than it first
appears, since any finite history can be absorbed into an enlarged state.


probability_theory

A discrete-time Markov chain is specified by a transition matrix P whose entry (i,j) is
the probability of moving from state i to j. Powers of P give multi-step transition
probabilities, so the distribution after n steps is the initial distribution times P^n.
The long-run behaviour is governed by the eigenstructure: the stationary distribution is
the left eigenvector with eigenvalue one, and the rate of approach to it is set by the
second-largest eigenvalue modulus. The spectral gap — one minus that modulus — determines
the mixing time, and a chain with a small gap mixes slowly, which is exactly the
diagnostic problem in Markov chain Monte Carlo.


probability_theory

Existence and uniqueness of the stationary distribution require conditions. Irreducibility
means every state is reachable from every other, ruling out disconnected components with
separate stationary distributions. Aperiodicity rules out deterministic cycling, which
would prevent convergence even with a unique stationary distribution — a chain
alternating between two states has a stationary distribution it never settles into.
Positive recurrence, automatic for finite chains, is needed in the infinite case.
Together these give the fundamental theorem: an irreducible aperiodic positive-recurrent
chain converges to its unique stationary distribution from any starting point.


probability_theory

Markov chain concepts:
  transition matrix     P_ij = P(next = j | current = i);  rows sum to 1
  n-step                P^n
  stationary            pi P = pi,  pi a probability vector
  detailed balance      pi_i P_ij = pi_j P_ji;  implies stationarity, and is what
                        Metropolis-Hastings constructs by design
  irreducible           all states communicate
  aperiodic             gcd of return times is 1
  ergodic               irreducible + aperiodic + positive recurrent
  mixing time           steps until within epsilon of stationary; ~1/spectral gap
  hitting time          expected steps to reach a target state
  absorbing             once entered, never left; the fundamental matrix gives
                        expected steps to absorption


probability_theory

Detailed balance deserves emphasis because it is the design principle of MCMC. Verifying
that a proposed chain has a given stationary distribution is hard in general and easy if
detailed balance holds, since detailed balance implies stationarity directly.
Metropolis-Hastings therefore constructs its acceptance probability specifically to
enforce detailed balance with respect to the target distribution, which is why the
algorithm works for any target known only up to a normalizing constant — the constant
cancels in the acceptance ratio, which is the entire reason Bayesian computation is
feasible.


probability_theory

Continuous-time processes add their own structure. The Poisson process counts events
occurring at a constant rate with independent increments, and its defining properties are
equivalent: exponential inter-arrival times, Poisson counts in any interval, and
memorylessness. Brownian motion is the continuous-time, continuous-state limit of a
random walk, with independent Gaussian increments whose variance is proportional to
elapsed time; its paths are continuous everywhere and differentiable nowhere, which is
why stochastic calculus requires the Ito integral rather than ordinary calculus, and why
Ito's lemma carries a second-order term that has no classical analogue.


probability_theory

A martingale is a process whose conditional expected future value equals its present
value — a fair game. The optional stopping theorem states that under suitable conditions
the expected value at a stopping time equals the initial value, which resolves gambling
problems, gives the gambler's ruin probabilities in a line, and underlies sequential
analysis. The martingale convergence theorem states that a bounded martingale converges
almost surely, which is used constantly in proving convergence of stochastic algorithms.
Applications extend well beyond gambling: likelihood ratios form martingales under the
null hypothesis, which is what makes always-valid sequential testing possible, and the
prices of traded assets under a risk-neutral measure are martingales, which is the
foundational result of mathematical finance.


real_and_functional_analysis

## Metric Foundations and Convergence

Analysis is the rigorous study of limits, and the machinery exists because intuitive reasoning
about infinite processes fails in ways that are not visible from the outside. A metric space
abstracts distance to its essential properties — non-negativity with equality only for identical
points, symmetry, and the triangle inequality — and everything about convergence, continuity, and
compactness can be phrased in that language.


real_and_functional_analysis

Completeness is the property that separates the rationals from the reals and it is the reason
analysis works on the latter. A space is complete when every Cauchy sequence converges within it —
when sequences whose terms become arbitrarily close to each other actually have a limit present. The
rationals fail this: a sequence of rational approximations to the square root of two is Cauchy and
has no rational limit. Completeness is what licenses constructing objects as limits of
approximations, which is the fundamental technique of the subject.


real_and_functional_analysis

Compactness is the other structural property that does most of the work in proofs. In a metric
space it is equivalent to every sequence having a convergent subsequence, and in finite-dimensional
Euclidean space it coincides with being closed and bounded. In infinite dimensions that equivalence
fails, and the failure is the central difficulty of functional analysis: the closed unit ball is
compact exactly when the space is finite-dimensional, so the extraction of convergent subsequences
that makes finite-dimensional arguments work is simply unavailable.


real_and_functional_analysis

Metric foundations:
  metric               non-negative, symmetric, triangle inequality
  open and closed sets  the topological vocabulary; continuity is preimages of
                       open sets being open
  Cauchy sequence      terms becoming arbitrarily close to each other
  completeness         every Cauchy sequence converges within the space
  the rationals' failure  approximations to irrational numbers have no limit
                       inside
  completion           embedding a space densely in a complete one; the reals are
                       the completion of the rationals
  compactness          every sequence has a convergent subsequence, in a metric
                       space
  Heine-Borel          in finite-dimensional Euclidean space, compact equals
                       closed and bounded
  the infinite-dimensional failure  the closed unit ball is compact only in finite
                       dimensions
  uniform continuity   the modulus independent of location; automatic on a compact
                       set
  pointwise convergence  each point converges; preserves almost nothing
  uniform convergence   the supremum of the difference goes to zero; preserves
                       continuity and permits interchanging limits with integrals
  the interchange question  when limits commute with integrals and derivatives is
                       the recurring technical issue of the entire subject


real_and_functional_analysis

Q: A sequence of continuous functions converges pointwise to a discontinuous function. This shows:
A. The functions were not really continuous
B. Pointwise convergence does not preserve continuity; uniform convergence is the stronger notion
   that does, and the distinction is precisely why it exists
C. The limit does not exist
D. An error in the calculation
Reasoning: Pointwise convergence controls each point independently with no uniformity in the rate,
so continuity can be lost in the limit. The standard example is powers of x on the unit interval
converging to a function that jumps at the endpoint. Uniform convergence forbids this.
Answer: B.


real_and_functional_analysis

## Measure and Integration

The Riemann integral partitions the domain and is adequate for continuous functions on intervals.
Its limitations are structural: it handles badly discontinuous functions poorly, it behaves badly
under limits, and the space of Riemann-integrable functions is not complete under the natural
distance. The Lebesgue integral partitions the range instead, asking how much of the domain maps
into each range slice, and this reorganization fixes all three problems.


real_and_functional_analysis

Making that precise requires measure theory. A measure assigns sizes to sets, countably additively,
on a sigma-algebra — a collection of sets closed under complement and countable union. The
restriction to a sigma-algebra rather than all sets is not a technicality that better mathematics
would remove: non-measurable sets provably exist under the axiom of choice, and any translation-
invariant, countably additive measure defined on all subsets of the line is impossible.


real_and_functional_analysis

The convergence theorems are what make the Lebesgue theory worth the setup cost. Monotone
convergence permits passing the limit inside the integral for increasing sequences. Dominated
convergence permits it whenever the sequence is bounded by a fixed integrable function. Fatou's
lemma gives an inequality in general. These have no adequate Riemann counterparts, and their
availability is why probability theory, Fourier analysis, and partial differential equations are all
built on Lebesgue integration.


real_and_functional_analysis

Measure and integration:
  sigma-algebra        closed under complement and countable union
  measure              countably additive size assignment
  non-measurable sets  exist under choice; the sigma-algebra restriction is
                       necessary
  Lebesgue measure     the translation-invariant measure on Euclidean space
  measurable function  preimages of measurable sets are measurable
  the Lebesgue integral  partition the range, not the domain
  almost everywhere    up to a set of measure zero; the equivalence that makes
                       the function spaces work
  monotone convergence  limits pass through for increasing sequences
  dominated convergence  limits pass through under an integrable bound
  Fatou's lemma        an inequality with no hypotheses beyond non-negativity
  Fubini and Tonelli   when iterated integrals may be exchanged; integrability or
                       non-negativity is required and counterexamples without it
                       are easy to construct
  Radon-Nikodym        one measure as a density with respect to another; the
                       abstract foundation of conditional expectation
  the probability connection  a probability space is a measure space of total
                       mass one, and every probabilistic notion translates


real_and_functional_analysis

## Banach and Hilbert Spaces

Functional analysis studies spaces whose points are functions, with the linear structure and a norm
compatible with it. A Banach space is a complete normed vector space, and completeness again is
what permits constructing solutions as limits. The characteristic difficulty is that the
finite-dimensional intuitions fail: not all norms are equivalent, bounded sets need not have
convergent subsequences, and linear maps need not be continuous.


real_and_functional_analysis

A Hilbert space adds an inner product inducing the norm, and this single addition recovers most of
the geometry. Orthogonality becomes meaningful, projection onto a closed subspace exists and is
unique, and every element has an expansion in an orthonormal basis with coefficients given by
inner products. The space of square-integrable functions is the canonical example, and Fourier
series are exactly the orthonormal expansion in it, which is why the Hilbert space framework
explains rather than merely accommodates classical Fourier theory.


real_and_functional_analysis

The three foundational theorems of the subject are all consequences of completeness and each
converts a pointwise hypothesis into a uniform conclusion. The uniform boundedness principle says a
family of operators bounded at each point is uniformly bounded. The open mapping theorem says a
surjective bounded operator is open, from which the bounded inverse theorem follows. The closed
graph theorem says an operator with a closed graph is bounded. Their common character — local
information forced into global conclusions by completeness — is worth recognizing.


real_and_functional_analysis

Banach and Hilbert structure:
  normed space         a vector space with a compatible length
  Banach space         complete normed space
  the Lp spaces        p-th power integrable functions; complete for p at least
                       one
  the sup norm space   continuous functions on a compact set; complete
  Hilbert space        complete inner product space
  the square-integrable space  the canonical example
  orthogonality        available only with an inner product
  projection theorem   unique closest point in a closed convex set
  orthonormal basis    expansion with inner-product coefficients
  Parseval             the norm equals the sum of squared coefficients
  Fourier series       the orthonormal expansion in the periodic case
  dual space           the continuous linear functionals
  Riesz representation  a Hilbert space is its own dual; every functional is an
                       inner product
  Hahn-Banach          functionals extend from subspaces without increasing norm;
                       the reason dual spaces are large enough to be useful
  uniform boundedness  pointwise bounded implies uniformly bounded
  open mapping         surjective bounded operators are open
  closed graph         a closed graph implies continuity
  weak convergence     convergence against every functional; bounded sequences
                       have weakly convergent subsequences in reflexive spaces,
                       which is the substitute for the compactness that fails


real_and_functional_analysis

## Operators and Spectral Theory

A linear operator between normed spaces is bounded exactly when it is continuous, and the two words
are used interchangeably for this reason. The operator norm measures the largest factor by which
the operator can stretch a vector, and the bounded operators on a space form an algebra under
composition — a structure that carries much of the theory.


real_and_functional_analysis

The spectrum generalizes the eigenvalue set. In infinite dimensions an operator can fail to be
invertible without having an eigenvector, so the spectrum is defined as the set of scalars for
which the operator minus that scalar is not invertible, and it decomposes into point, continuous,
and residual parts. For a compact operator the spectrum is discrete apart from zero and the
finite-dimensional intuition largely survives, which is why compact operators are the tractable
class.


real_and_functional_analysis

The spectral theorem is the central structural result. A self-adjoint operator on a Hilbert space
has real spectrum and admits a representation as multiplication by a real function on an
appropriate measure space, which is the infinite-dimensional statement of diagonalization. For
compact self-adjoint operators the statement is concrete: an orthonormal basis of eigenvectors
exists with real eigenvalues accumulating only at zero. This underlies the mode decompositions of
vibrating systems, the eigenfunction expansions solving partial differential equations, and the
observable structure of quantum mechanics.


real_and_functional_analysis

Operators and spectra:
  bounded equals continuous  for linear maps
  operator norm        the supremum stretching factor
  adjoint              the transpose generalized via the inner product
  self-adjoint         equal to its own adjoint; real spectrum
  unitary              preserves the inner product
  compact operator     maps bounded sets to relatively compact sets; the
                       finite-dimensional intuition largely survives
  spectrum             where the operator minus a scalar is not invertible
  point spectrum       eigenvalues
  continuous spectrum  not invertible without an eigenvector; no finite-
                       dimensional analogue
  resolvent            the inverse where it exists; analytic in the scalar
  spectral theorem     self-adjoint operators are multiplication operators
  compact self-adjoint  an orthonormal eigenbasis, eigenvalues accumulating at
                       zero
  functional calculus  applying functions to operators through the spectral
                       decomposition
  unbounded operators  differentiation is unbounded; domains must be specified
                       and the theory is genuinely more delicate
  the quantum connection  observables are self-adjoint operators, their spectra
                       are the possible measured values, and the spectral theorem
                       is what makes that statement coherent


real_and_functional_analysis

## Applications and Practical Consequences

The abstract machinery pays off in specific places, and knowing where prevents the subject from
seeming like formalism for its own sake. Existence proofs for differential equations are the
clearest case: the contraction mapping principle on a complete space gives existence and uniqueness
for ordinary differential equations directly, and variational methods in Sobolev spaces give weak
solutions for elliptic partial differential equations where classical solutions may not exist.


real_and_functional_analysis

The notion of a weak solution is itself an important conceptual move. Rather than requiring an
equation to hold pointwise with all derivatives existing, it is required to hold after integration
against smooth test functions. This admits solutions with shocks, corners, and discontinuities that
the classical formulation excludes, and it matches the physical situations — a shock wave is a real
phenomenon and the classical formulation simply has nothing to say about it.


real_and_functional_analysis

In applied and computational work the relevant consequences are about approximation and stability.
Choosing a function space determines what convergence means for a numerical method, and a method
converging in one norm may diverge in another. Ill-posedness — where the solution does not depend
continuously on the data — is an operator-theoretic statement about the inverse being unbounded,
and regularization is the standard response: replace the unbounded inverse with a family of bounded
approximations and choose the parameter by the noise level.


real_and_functional_analysis

Applications:
  contraction mapping  existence and uniqueness by iteration on a complete space
  Picard iteration     the concrete construction for ordinary differential
                       equations
  Sobolev spaces       functions with weak derivatives in an integrable class
  weak solution        the equation holding against test functions
  why it matters       admits shocks and corners that occur physically
  variational methods  minimizing an energy functional; existence from
                       lower semicontinuity and coercivity
  the direct method    a minimizing sequence, weak compactness, and lower
                       semicontinuity
  approximation theory  which norm the method converges in; a method may converge
                       in one and not another
  finite elements      Galerkin approximation in a finite-dimensional subspace of
                       a Sobolev space
  ill-posedness        the inverse operator is unbounded; small data errors
                       amplify without limit
  inverse problems     deconvolution, tomography, and parameter estimation are
                       all of this type
  regularization       replace the unbounded inverse with bounded approximations
  the parameter choice  set by the noise level; too little leaves noise
                       amplified, too much biases the solution, and the tradeoff
                       is exactly the bias-variance tradeoff appearing in
                       statistical language elsewhere


statistical_theory

## Estimation and the Properties of Estimators

Statistical theory studies procedures for drawing conclusions from data under a probability model,
and its central objects are estimators — functions of the data used to infer something about the
model. The theory's value is that it says which estimators are good, in what sense, and under what
conditions, which converts a menu of plausible-seeming procedures into an ordered comparison.


statistical_theory

The basic properties are stated relative to repeated sampling from the model. An estimator is
unbiased when its expected value equals the quantity estimated. It is consistent when it converges
to that quantity as the sample grows. It is efficient when its variance is as small as achievable
among a class. These are distinct and none implies the others: an unbiased estimator can be
inconsistent, a consistent estimator can be badly biased in small samples, and the estimator with
the smallest mean squared error is frequently biased.


statistical_theory

That last point deserves emphasis because it contradicts a widespread intuition. Mean squared error
decomposes into squared bias plus variance, so accepting some bias in exchange for a larger variance
reduction lowers the total error. Shrinkage estimators exploit this deliberately, and the striking
theoretical result is that in three or more dimensions the obvious unbiased estimator of a normal
mean vector is inadmissible — a shrunken estimator has smaller risk everywhere. Unbiasedness is a
property, not a virtue.


statistical_theory

Estimation properties:
  estimator            a function of the data used to infer a quantity
  bias                 the difference between expectation and truth
  consistency          convergence to the truth as the sample grows
  efficiency           minimal variance within a class
  the independence     none of these implies the others
  mean squared error   squared bias plus variance
  the bias-variance tradeoff  accepting bias to reduce variance lowers total error
  shrinkage            deliberately biased toward a target; often better
  the inadmissibility result  in three or more dimensions, the obvious unbiased
                       estimator of a normal mean is dominated everywhere
  the lesson           unbiasedness is a property to be traded, not preserved
  standard error       the estimator's own standard deviation
  robustness           sensitivity to model misspecification and to outliers
  breakdown point      the fraction of contaminated data an estimator tolerates;
                       the mean has zero, the median has one half
  influence function   how much a single observation can move the estimate


statistical_theory

Q: An estimator with small bias and large variance is compared with one having moderate bias and
very small variance. The second has lower mean squared error. The correct conclusion is:
A. The first estimator is better because it is less biased
B. The second is better for this criterion; mean squared error combines squared bias and variance,
   and reducing variance can more than compensate for added bias
C. Bias makes an estimator invalid
D. They cannot be compared
Reasoning: Unbiasedness is one desirable property among several and it is not the objective unless
the loss function makes it so. Under squared error loss, the decomposition makes the tradeoff
explicit, and shrinkage methods exploit it systematically. Answer: B.


statistical_theory

## Likelihood, Sufficiency, and Information

The likelihood function is the probability of the observed data viewed as a function of the
parameter, and it is the central object of parametric inference. It is not a probability
distribution over the parameter — it does not integrate to one in general — and treating it as one
is the most common conceptual error in the area. What it does is rank parameter values by how well
they explain the data.


statistical_theory

Maximum likelihood estimation selects the parameter maximizing this function, and it has excellent
properties under regularity conditions: consistency, asymptotic normality, asymptotic efficiency,
and invariance to reparameterization. The regularity conditions matter and are frequently violated
in practice — at boundary values, when the parameter count grows with the sample, and when the
support depends on the parameter — and in those cases the standard asymptotic results simply do not
apply.


statistical_theory

Sufficiency identifies when a statistic captures all the information in the data about a parameter.
A sufficient statistic is one such that the conditional distribution of the data given it does not
depend on the parameter, so nothing further can be learned from the raw data once it is known. The
factorization criterion makes sufficiency checkable directly from the likelihood's form, and the
exponential family is precisely the class where a sufficient statistic of fixed dimension exists
regardless of sample size, which is why that family recurs throughout the theory.


statistical_theory

Likelihood and information:
  likelihood           the data's probability as a function of the parameter
  not a distribution over parameters  the standard conceptual error
  maximum likelihood   the maximizing parameter value
  the good properties  consistency, asymptotic normality and efficiency,
                       invariance
  regularity conditions  interior optimum, fixed dimension, support independent
                       of the parameter
  the failure cases    boundaries, growing dimension, parameter-dependent support
  score function       the derivative of the log likelihood; zero at the maximum
  Fisher information   the expected curvature; how sharply the data distinguishes
                       parameter values
  Cramér-Rao bound     a lower bound on unbiased estimator variance, given by the
                       inverse information
  observed information  the curvature at the estimate; often preferred for
                       standard errors in practice
  sufficiency          a statistic capturing all parameter information
  factorization criterion  sufficiency checkable from the likelihood's form
  minimal sufficiency  the coarsest sufficient statistic
  exponential family   fixed-dimension sufficiency at any sample size
  ancillarity          a statistic whose distribution does not depend on the
                       parameter; conditioning on it is often the right thing and
                       the theory of when is genuinely subtle


statistical_theory

## Hypothesis Testing Theory

A hypothesis test partitions the sample space into regions where a null hypothesis is rejected and
where it is not, and the theory characterizes which partitions are good. The two error types are
asymmetric by construction: type one is rejecting a true null, controlled at a chosen level; type
two is failing to reject a false null, whose complement is power. The asymmetry encodes a decision
that false positives and false negatives have different costs, which is a modelling choice rather
than a mathematical necessity.


statistical_theory

The Neyman-Pearson lemma establishes that for testing one simple hypothesis against another, the
likelihood ratio test is most powerful at any level. This is the theoretical anchor of the subject —
it says the optimal test is determined, not chosen — and extending it to composite hypotheses is
what most of testing theory is about. Uniformly most powerful tests exist for some families with
monotone likelihood ratios and frequently do not exist at all, in which case restricted optimality
criteria such as unbiasedness or invariance are imposed to select a test.


statistical_theory

The p-value is the probability, under the null, of a test statistic at least as extreme as the one
observed. It is not the probability that the null is true, it is not the probability the result
occurred by chance, and it does not measure effect size. Its correct use requires that the test was
specified before seeing the data, since selecting a hypothesis after examining the data invalidates
the calculation entirely — the sampling distribution being referred to is not the one that was
actually sampled from.


statistical_theory

Testing theory:
  null and alternative  the partition being tested
  type one error       rejecting a true null; controlled at a chosen level
  type two error       failing to reject a false null
  power                one minus the type two error rate
  the asymmetry        a decision about relative costs, not a mathematical fact
  Neyman-Pearson lemma  the likelihood ratio test is most powerful for simple
                       hypotheses
  uniformly most powerful  optimal across the whole alternative; exists only
                       sometimes
  monotone likelihood ratio  the condition under which it does
  restricted optimality  unbiasedness or invariance imposed when no uniform
                       optimum exists
  likelihood ratio test  the general-purpose construction
  Wald and score tests  asymptotically equivalent alternatives with different
                       computational and finite-sample behaviour
  p-value              the null probability of a result at least as extreme
  what it is not       the probability the null is true, or that the result was
                       chance, or a measure of effect size
  the pre-specification requirement  selecting the hypothesis after seeing the
                       data invalidates the reference distribution
  multiple testing     the error rate across a family; family-wise control versus
                       false discovery rate control, which answer different
                       questions and should be chosen deliberately
  confidence interval  the set of parameter values not rejected; the duality with
                       testing is exact and is the clearest way to understand it


statistical_theory

## Asymptotics and Large Sample Theory

Exact finite-sample distributions are available only for a few models, so most inference relies on
approximations that become accurate as the sample grows. Asymptotic theory characterizes these
approximations and, importantly, indicates when they fail, which is the part most often neglected in
application.


statistical_theory

The two foundational results carry most of the weight. The law of large numbers says sample averages
converge to expectations, which justifies estimating population quantities by their sample analogues.
The central limit theorem says the standardized sample mean converges to a normal distribution
regardless of the underlying distribution's shape, provided the variance is finite, which is why
normal-based procedures apply so broadly.


statistical_theory

The finite-variance condition is essential and its violation is not exotic. Heavy-tailed
distributions with infinite variance do not satisfy the central limit theorem, and their sample
means converge to stable distributions with entirely different behaviour — including the property
that averaging more observations does not reduce variability in the usual way. Financial returns,
network traffic, and city sizes are all commonly modelled with such distributions, and applying
normal-based inference to them produces confidence intervals that are far too narrow.


statistical_theory

Asymptotic theory:
  the motivation       exact distributions exist for few models
  law of large numbers  sample averages converge to expectations
  central limit theorem  the standardized mean converges to normal
  the finite variance requirement  essential, and violated in real applications
  heavy tails          convergence to stable laws instead; averaging does not
                       help as expected
  the practical consequence  normal-based intervals are far too narrow for such
                       data
  delta method         the asymptotic distribution of a smooth function of an
                       estimator
  continuous mapping   limits pass through continuous functions
  Slutsky's theorem    combining convergence in distribution and in probability
  rates of convergence  the standard root-n rate and the situations that are
                       slower
  the convergence speed question  asymptotic validity says nothing about the
                       sample size needed, which depends on skewness and tail
                       weight and can be far larger than the usual rules of thumb
  Edgeworth expansion  higher-order corrections quantifying the approximation
                       error
  contiguity and local alternatives  the framework for comparing tests'
                       asymptotic power meaningfully


statistical_theory

## Decision Theory, Bayes, and Resampling

Decision theory frames inference as choosing an action under uncertainty with a specified loss
function, and it unifies estimation and testing as special cases. Risk is expected loss, and the
comparison of procedures by risk clarifies what optimality means. A procedure is admissible when no
other has risk at least as low everywhere and lower somewhere, and inadmissible procedures can be
discarded — though admissibility is a weak criterion, since many poor procedures are admissible for
trivial reasons.


statistical_theory

The Bayesian formulation places a prior distribution on the parameter and computes the posterior by
conditioning on the data. It yields direct probability statements about parameters, incorporates
prior information explicitly, and handles nuisance parameters by integration rather than by
approximation. Its costs are that the prior must be specified and affects the answer, and that
computation was historically prohibitive — the latter largely resolved by simulation methods. The
theoretical connection is that Bayes procedures are essentially the complete class of admissible
procedures, which means good frequentist procedures are Bayes procedures for some prior whether or
not they were derived that way.


statistical_theory

Resampling methods estimate sampling distributions computationally rather than analytically. The
bootstrap resamples the data with replacement and treats the variability across resamples as an
estimate of the estimator's sampling variability, which works because the empirical distribution
approximates the true one. It applies where analytical results are unavailable and it is not
universal — it fails for estimators that are not smooth functionals of the distribution, including
the sample maximum, and for dependent data unless the resampling scheme is adapted to preserve the
dependence.


statistical_theory

Decision theory and computation:
  loss function        the cost of an action given the truth
  risk                 expected loss over the sampling distribution
  admissibility        not dominated everywhere by another procedure
  the weakness         many poor procedures are admissible
  minimax              minimizing the worst-case risk; conservative
  Bayes risk           expected risk averaged over a prior
  the complete class result  admissible procedures are essentially Bayes
                       procedures
  posterior            the prior updated by the likelihood
  the direct interpretation  probability statements about parameters
  prior sensitivity    the answer depends on the prior; report how much
  conjugacy            priors giving closed-form posteriors
  simulation methods   Markov chain sampling; what made general Bayesian
                       computation feasible
  bootstrap            resample with replacement; sampling distribution by
                       computation
  when it works        smooth functionals of the distribution
  when it fails        extremes, boundaries, and dependent data without an adapted
                       scheme
  permutation tests    exact under exchangeability; the cleanest nonparametric
                       tests available
  cross-validation     out-of-sample error by data splitting; a resampling method
                       for model assessment rather than for inference, and the
                       distinction matters because cross-validated error estimates
                       have their own variability that is easy to ignore


topology_and_geometry

## Metric and Topological Spaces

Topology studies properties preserved under continuous deformation — stretching and bending
but not tearing or gluing. The motivating observation is that a great many results depend not on
distances but only on which points are near which, and abstracting away the metric reveals which
results those are.


topology_and_geometry

A metric space equips a set with a distance function satisfying non-negativity, symmetry, the
identity of indiscernibles, and the triangle inequality. From the metric one obtains open balls,
and from open balls one obtains open sets, and it turns out that the open sets alone suffice to
define continuity, convergence, compactness, and connectedness. A topological space therefore
dispenses with the metric and specifies the open sets directly, subject to axioms: the empty set
and the whole space are open, arbitrary unions of open sets are open, and finite intersections
are open.


topology_and_geometry

The asymmetry between arbitrary unions and finite intersections is not arbitrary and is worth
understanding. An infinite intersection of open sets need not be open — the intersection of the
intervals from minus one over n to one over n, over all n, is the single point zero, which is
not open. Restricting to finite intersections is exactly what is needed for the axioms to
produce the intended notion of nearness.


topology_and_geometry

Different metrics can induce the same topology, which is the point of the abstraction. The
Euclidean, taxicab, and maximum metrics on the plane give different distances and identical open
sets, so every topological statement about one holds for the others. Conversely some
topological spaces are not metrizable at all, so the topological setting is strictly more
general.


topology_and_geometry

Metric and topological concepts:
  metric               non-negativity, symmetry, identity, triangle inequality
  open ball            points within a distance of a centre
  open set             a union of open balls; or an axiomatic primitive
  topology             the collection of open sets satisfying the axioms
  closed set           complement of an open set; sets can be both, neither, or one
  interior, closure    largest open subset, smallest closed superset
  boundary             closure minus interior
  neighbourhood        a set containing an open set containing the point
  basis                generates the topology by unions
  subspace topology    induced on a subset
  product topology     on a product of spaces; the finite case is the obvious one
                       and the infinite case requires care
  quotient topology    from an identification; how surfaces are built by gluing
  homeomorphism        a continuous bijection with continuous inverse; the
                       topological notion of sameness
  metrizable           a topology arising from some metric; not all topologies are


topology_and_geometry

Q: A continuous bijection between topological spaces is not necessarily a homeomorphism.
The additional requirement is:
A. The spaces must be metric spaces
B. The inverse map must also be continuous
C. The map must preserve distances
D. The spaces must be compact
Reasoning: A homeomorphism requires continuity in both directions, and a continuous bijection
can have a discontinuous inverse — mapping the half-open interval [0, 1) onto a circle by
wrapping is the standard example, continuous and bijective with a discontinuous inverse at the
join. Continuity of the inverse is automatic when the domain is compact and the codomain
Hausdorff, which is why option D is close and not the general answer. Answer: B.


topology_and_geometry

## Continuity, Compactness, and Connectedness

Continuity in the topological setting is defined by preimages: a map is continuous when the
preimage of every open set is open. This is equivalent to the epsilon-delta definition in metric
spaces and is more general, and its direction — preimages rather than images — is the part that
initially surprises. The image of an open set under a continuous map need not be open, as the
constant map demonstrates.


topology_and_geometry

Compactness is the topological property doing the most work in analysis, and its definition —
every open cover has a finite subcover — appears technical and is exactly right. In Euclidean
space the Heine-Borel theorem identifies compact sets with those that are closed and bounded,
which is the familiar characterization, and in general spaces the two notions diverge and
compactness is the useful one.


topology_and_geometry

What compactness buys is the ability to pass from local to global. A continuous function on a
compact set attains its maximum and minimum, is uniformly continuous, and is bounded — each
proved by covering the set with local statements and extracting a finite subcover. These are
exactly the theorems that fail on a non-compact set such as an open interval, where a continuous
function may be unbounded and may fail to attain its bounds. Compactness is what makes the
extreme value theorem true.


topology_and_geometry

Connectedness formalizes being in one piece: a space is connected when it cannot be written as
the union of two disjoint nonempty open sets. Path connectedness is the stronger and more
intuitive condition that any two points are joined by a continuous path, and it implies
connectedness while the converse fails — the topologist's sine curve is connected and not path
connected, which is the standard counterexample.


topology_and_geometry

Continuity, compactness, connectedness:
  continuity           preimages of open sets are open
  homeomorphism        continuous with continuous inverse
  compact              every open cover has a finite subcover
  Heine-Borel          in Euclidean space, compact iff closed and bounded
  extreme value        a continuous function on a compact set attains its bounds
  uniform continuity   automatic on a compact domain
  sequential compactness  every sequence has a convergent subsequence;
                       equivalent to compactness in metric spaces
  connected            not a union of two disjoint nonempty open sets
  path connected       any two points joined by a path; strictly stronger
  intermediate value   a continuous image of a connected set is connected; this is
                       the whole content of the intermediate value theorem
  Hausdorff            distinct points have disjoint neighbourhoods; the
                       separation axiom that makes limits unique
  separation axioms    a hierarchy from T0 to T4; most useful spaces are Hausdorff
                       and better


topology_and_geometry

The intermediate value theorem's topological content is worth stating because it explains why
the theorem is true. A continuous image of a connected set is connected, an interval is
connected, and the connected subsets of the real line are exactly the intervals — so the image
of an interval under a continuous function is an interval, which contains every value between
any two of its values. The familiar theorem is a corollary of a structural statement about
connectedness, which is the pattern topology repeatedly reveals.


topology_and_geometry

## Surfaces and Their Classification

Surfaces are two-dimensional manifolds — spaces locally resembling the plane — and the compact
ones without boundary admit a complete classification, which is unusual and instructive. Every
such surface is either a sphere with some number of handles attached, or a sphere with some
number of cross-caps, and the number determines it up to homeomorphism.


topology_and_geometry

The genus counts the handles, so the sphere has genus zero, the torus genus one, the two-holed
torus genus two, and so on. Orientability distinguishes the two families: a surface is
orientable if a consistent notion of clockwise can be defined globally, and non-orientable if
not. The Möbius band and Klein bottle are the standard non-orientable examples, and the Möbius
band demonstrates the phenomenon concretely — a small clock face transported around the band
returns reversed.


topology_and_geometry

The Euler characteristic is the invariant that makes classification computable. For a surface
divided into vertices, edges, and faces, the alternating sum of their counts is independent of
how the division was made, which is a striking result. It equals two for the sphere, zero for the
torus, and two minus twice the genus in general for orientable surfaces. Because it is a
topological invariant, two surfaces with different Euler characteristics cannot be homeomorphic,
and computing it from any convenient triangulation settles the question.


topology_and_geometry

Surfaces and invariants:
  manifold             locally resembles Euclidean space of a fixed dimension
  surface              two-dimensional manifold
  genus                number of handles; the classifying invariant for orientable
                       compact surfaces
  orientable           a consistent global orientation exists
  Möbius band          the simplest non-orientable surface; one side, one edge
  Klein bottle         non-orientable and closed; cannot embed in three dimensions
                       without self-intersection
  Euler characteristic  V - E + F; independent of the triangulation
  sphere               characteristic 2, genus 0
  torus                characteristic 0, genus 1
  classification theorem  compact surfaces are classified by orientability and
                       Euler characteristic
  connected sum        glue two surfaces along a removed disc; genus adds
  Gauss-Bonnet         total curvature equals 2 pi times the Euler characteristic;
                       links geometry to topology
  hairy ball theorem   no non-vanishing continuous tangent vector field on an
                       even-dimensional sphere; hence a cowlick, and hence a
                       cyclone somewhere on Earth's surface


topology_and_geometry

The Gauss-Bonnet theorem deserves emphasis because it connects a geometric quantity to a
topological one in a way that seems too strong to be true. The total Gaussian curvature
integrated over a closed surface equals two pi times its Euler characteristic — so however a
sphere is deformed, stretched, or dented, the total curvature is unchanged. Bending the surface
redistributes curvature and cannot change its total, because the total is determined by the
topology alone. This is the archetype of a rigidity result linking local geometry to global
topology, and it has descendants throughout modern mathematics and theoretical physics.


topology_and_geometry

## Homotopy, Homology, and Invariants

Topological classification requires invariants — quantities preserved by homeomorphism — because
proving two spaces are homeomorphic requires exhibiting a map, while proving they are not
requires showing some invariant differs. Algebraic topology constructs invariants by assigning
algebraic objects to spaces in a way that respects continuous maps.


topology_and_geometry

Homotopy formalizes continuous deformation of maps. Two maps are homotopic if one can be
continuously deformed into the other, and two spaces are homotopy equivalent if maps between
them compose to something homotopic to the identity. Homotopy equivalence is coarser than
homeomorphism — a disc is homotopy equivalent to a point, and a solid torus to a circle — which
makes it easier to compute and sufficient for many purposes.


topology_and_geometry

The fundamental group is the first and most intuitive algebraic invariant. It consists of loops
based at a point, considered up to homotopy, with concatenation as the group operation. A space
in which every loop contracts to a point has trivial fundamental group and is called simply
connected — the sphere is, the torus is not, and the circle's fundamental group is the integers,
counting how many times a loop winds around.


topology_and_geometry

Homology groups generalize this to higher dimensions and are more computable than higher
homotopy groups. They detect holes of each dimension: the zeroth counts connected components,
the first counts loops that do not bound a surface, the second counts voids enclosed by
surfaces. Their computability is what makes them the practical tool, since higher homotopy groups
are notoriously difficult to compute even for the sphere.


topology_and_geometry

Algebraic invariants:
  homotopy             continuous deformation of maps
  homotopy equivalence  coarser than homeomorphism, easier to compute
  contractible         homotopy equivalent to a point
  fundamental group    loops up to homotopy; the first algebraic invariant
  simply connected     trivial fundamental group; every loop contracts
  circle               fundamental group is the integers; the winding number
  covering space       a space mapping locally homeomorphically; corresponds to
                       subgroups of the fundamental group
  homology             holes by dimension; computable
  Betti numbers        the ranks of the homology groups; the count of holes of
                       each dimension
  Euler characteristic  the alternating sum of Betti numbers; recovers the
                       combinatorial definition
  Brouwer fixed point  every continuous self-map of a disc has a fixed point;
                       proved by a homology argument
  Borsuk-Ulam          any continuous map from a sphere to the plane sends some
                       antipodal pair to the same point; hence two antipodal
                       points on Earth with equal temperature and pressure
  persistent homology  homology across a range of scales; the foundation of
                       topological data analysis


topology_and_geometry

Brouwer's fixed point theorem deserves note because its proof illustrates the method. The
theorem states that any continuous map from a disc to itself has a fixed point, which is
intuitive and not obviously provable. The standard argument is by contradiction: if no fixed
point existed, a retraction from the disc onto its boundary circle could be constructed, and such
a retraction would induce a homomorphism from the trivial fundamental group of the disc onto the
integers of the circle, which is impossible. An algebraic impossibility establishes a geometric
fact, which is what algebraic topology exists to do.


topology_and_geometry

## Differential Geometry and Applications

Differential geometry studies smooth manifolds with additional structure permitting
differentiation, and its central concept is curvature — a local measurement of how a space
departs from being flat. For a surface, Gaussian curvature is the product of the principal
curvatures, and Gauss's theorema egregium establishes that it is intrinsic: it can be determined
by measurements made entirely within the surface, without reference to how the surface sits in
space.


topology_and_geometry

The theorem's practical consequence is that a surface with nonzero Gaussian curvature cannot be
flattened without distortion. A sphere has positive curvature everywhere and a plane has zero, so
no map projection can preserve both distances and angles — every world map distorts something,
and which distortion is accepted is a choice rather than a technical limitation. A cylinder, by
contrast, has zero Gaussian curvature despite appearing curved, which is why a sheet of paper
rolls into a cylinder without stretching.


topology_and_geometry

Geodesics generalize straight lines as the locally shortest paths within a surface, and they are
what a particle follows in the absence of force. On a sphere they are great circles, which is why
long-distance flight paths appear curved on a flat map — they are straight in the geometry that
matters and the map's distortion makes them look otherwise. In general relativity, spacetime is
a curved manifold and free particles follow geodesics, so gravity is not a force acting on
trajectories but the curvature of the space the trajectories are straight in.


topology_and_geometry

Differential geometry and applications:
  smooth manifold      locally Euclidean with compatible smooth structure
  tangent space        the linear approximation at a point
  Riemannian metric    an inner product on each tangent space; gives lengths and
                       angles
  geodesic             locally shortest path; the generalization of a straight line
  Gaussian curvature   intrinsic by the theorema egregium
  developable surface  zero Gaussian curvature; flattens without distortion
  map projection       no projection preserves both distance and angle; the choice
                       of distortion is the design decision
  parallel transport   moving a vector along a path keeping it parallel; the
                       result depends on the path when curvature is nonzero
  holonomy             the discrepancy after transport around a loop; measures
                       curvature
  connection           specifies how to differentiate vector fields on a manifold
  curvature tensor     the general-dimension measure; Einstein's equations relate
                       it to energy and momentum
  Gauss-Bonnet         total curvature is a topological invariant
  applications         general relativity, robotics configuration spaces, computer
                       graphics surface modelling, optimization on manifolds,
                       topological data analysis


topology_and_geometry

Topological data analysis deserves the closing note because it applies these ideas
computationally. Data is treated as a sampled point cloud, and building a simplicial complex by
connecting points within a distance threshold produces a space whose homology reflects the data's
shape. Varying the threshold gives a family of complexes, and persistent homology tracks which
topological features appear and persist across scales — features persisting over a wide range of
thresholds are structural, and short-lived ones are noise. This provides a description of shape
that is invariant to the coordinate system, robust to noise, and able to detect structure such
as loops and voids that clustering and dimensionality reduction do not represent at all.


topology_and_geometry

The recurring lesson across the subject is that ignoring information reveals structure. Topology
discards distance and finds what depends only on nearness; homotopy discards more and becomes
computable; homology discards the group structure of loops and becomes computable in every
dimension. Each abstraction loses detail and gains generality and tractability, and the results
obtained apply to every space satisfying the weaker assumptions — which is why theorems about
abstract topological spaces constrain map projections, weather patterns, robot motion planning,
and the geometry of spacetime alike.


acoustics_and_waves

## Wave Fundamentals

A wave is a disturbance propagating through a medium or field, transporting energy and
momentum without net transport of matter. The distinction between the motion of the disturbance
and the motion of the medium is the first thing to establish: in a water wave the water moves in
small circles while the wave travels for kilometres, and in a sound wave air molecules oscillate
by micrometres while the sound crosses a room.


acoustics_and_waves

Waves are classified by the direction of the medium's motion relative to propagation.
Transverse waves oscillate perpendicular to the direction of travel — light, waves on a string,
seismic S waves. Longitudinal waves oscillate along it, producing compressions and rarefactions
— sound in fluids, seismic P waves. The distinction has consequences: transverse waves can be
polarized because there are two independent perpendicular directions, while longitudinal waves
cannot, and shear waves cannot propagate in a fluid because a fluid has no shear strength, which
is exactly how the Earth's liquid outer core was discovered.


acoustics_and_waves

The wave equation relates the second time derivative to the second spatial derivative through
the square of the propagation speed, and that speed follows from the medium's properties: a
restoring force divided by an inertia. For a string it is tension over linear density; for sound
in a gas it is the bulk modulus over density, which via the ideal gas law makes it proportional
to the square root of absolute temperature and independent of pressure. The temperature
dependence is why sound travels faster on a hot day and why atmospheric temperature gradients
refract sound, bending it toward the cooler region.


acoustics_and_waves

Wave fundamentals:
  transverse           medium oscillates perpendicular to travel; can be polarized
  longitudinal         oscillates along travel; sound in fluids
  wave speed           sqrt(restoring force property / inertia property)
  string               sqrt(tension / linear density)
  sound in gas         sqrt(gamma R T / M); ~343 m/s in air at 20 C
  sound in water       ~1480 m/s;  in steel ~5000 m/s
  frequency and period  reciprocals; frequency is set by the source
  wavelength           speed divided by frequency; changes with the medium while
                       frequency does not
  amplitude            displacement, pressure, or velocity; intensity goes as its
                       square
  intensity            power per unit area; falls as inverse square from a point
                       source in free space
  impedance            the ratio of pressure to particle velocity; determines
                       reflection at a boundary
  dispersion           speed depending on frequency; a pulse spreads as it travels


acoustics_and_waves

Acoustic impedance is the concept doing the most work in applications, because it determines
what happens at a boundary. When a wave meets an interface between media of different
impedance, part reflects and part transmits, with the proportion set by the impedance ratio. A
large mismatch reflects nearly everything, which is why sound from air barely enters water — the
impedance ratio is about 3600 to one and over 99.9 percent reflects — and why an ultrasound probe
requires coupling gel to displace the air between it and the skin.


acoustics_and_waves

Q: An ultrasound examination performed without coupling gel produces no usable image. The
reason is:
A. The gel amplifies the signal
B. The air gap creates an enormous impedance mismatch, so nearly all the ultrasound reflects
   at the probe-air interface and never enters the body
C. The gel conducts electricity
D. The gel reduces friction
Reasoning: Acoustic impedance differs between the transducer and air by several orders of
magnitude, and the reflection coefficient at such a mismatch approaches unity. Gel with
impedance close to tissue displaces the air and provides a matched path. This is impedance
matching, the same principle as an anti-reflection coating in optics. Answer: B.


acoustics_and_waves

## Sound Propagation and Perception

Sound intensity spans an enormous range — the threshold of hearing to the threshold of pain is
about twelve orders of magnitude in intensity — which is why the decibel scale is logarithmic.
Sound pressure level references twenty micropascals, chosen as the approximate threshold of
hearing at one kilohertz, and doubling the pressure adds six decibels while doubling the
intensity adds three.


acoustics_and_waves

Propagation losses come from several mechanisms with different distance dependence. Geometric
spreading dominates: a point source in free space spreads its power over a sphere, so intensity
falls as the inverse square and level falls six decibels per doubling of distance. A line source
spreads over a cylinder and falls three decibels per doubling, which is why traffic noise from a
busy road decays more slowly than intuition suggests. Atmospheric absorption adds a
frequency-dependent loss that is negligible at low frequency and substantial at high, which is
why distant sounds are dull — thunder from nearby cracks and from far away rumbles.


acoustics_and_waves

Refraction bends sound where the speed varies, and because sound speed depends on temperature
and on wind, atmospheric structure shapes propagation over distance. A temperature inversion,
with warm air above cold, bends sound downward and can carry it far beyond the normal range,
which is why distant sounds are audible on still cold nights. The normal daytime gradient bends
sound upward, creating a shadow zone. Wind gradients do the same asymmetrically, which is why
sound carries downwind and not upwind.


acoustics_and_waves

Sound propagation and perception:
  sound pressure level  20 log10 (p / 20 uPa) dB
  doubling pressure     +6 dB;  doubling intensity +3 dB
  point source          -6 dB per doubling of distance
  line source           -3 dB per doubling
  atmospheric absorption  frequency dependent; distant sound is dull
  refraction            temperature and wind gradients bend sound
  temperature inversion  bends downward; extends the audible range
  diffraction           bending around obstacles; strong at low frequency where
                        the wavelength exceeds the obstacle
  barrier effectiveness  limited by diffraction over the top; effective for high
                        frequency, poor for low
  Doppler shift         moving source or observer; the pitch change on passing
  equal loudness        perception is frequency-dependent; least sensitive at low
                        frequency, most sensitive around 2-5 kHz
  masking               a loud sound conceals a quieter nearby one; the basis of
                        perceptual audio coding
  critical bands        the ear's effective filters; masking acts within them
  binaural cues         time difference below ~1.5 kHz, level difference above;
                        spectral cues from the pinna resolve elevation


acoustics_and_waves

Diffraction explains why noise barriers work poorly for low frequencies and it follows from a
general property of waves: bending around an obstacle is significant when the wavelength is
comparable to or larger than the obstacle. A one-hundred-hertz tone has a wavelength of about
3.4 metres and diffracts freely over a three-metre barrier, while a four-kilohertz tone with an
eight-centimetre wavelength is effectively blocked. This is why traffic noise heard behind a
barrier is a low rumble with the higher frequencies removed, and why barriers must be tall
relative to the wavelengths they are intended to attenuate.


acoustics_and_waves

## Interference, Resonance, and Room Acoustics

Superposition means waves add, and where they overlap coherently the result depends on their
relative phase. Constructive interference occurs where they are in phase and destructive where
opposed, producing spatial patterns of loud and quiet. Standing waves form when a wave reflects
and interferes with itself, producing fixed nodes and antinodes rather than a travelling
pattern.


acoustics_and_waves

Resonance occurs when a system is driven at a frequency matching one of its natural modes, and
the response is amplified by a factor related to the quality factor. Every bounded system has
natural modes determined by its geometry and boundary conditions: a string fixed at both ends
supports modes at integer multiples of its fundamental, an open pipe supports the same series,
and a pipe closed at one end supports only odd harmonics — which is why a clarinet sounds
different from a flute of the same length and plays an octave lower.


acoustics_and_waves

Room acoustics is the behaviour of sound in an enclosed space, and it separates into distinct
regimes by frequency. At low frequencies where the wavelength is comparable to the room
dimensions, discrete modal resonances dominate and produce large position-dependent variations —
which is why bass response in a small room changes drastically with listener position and why
room modes are the hardest acoustic problem to fix. At high frequencies where many modes overlap,
statistical treatment applies and the room is characterized by reverberation time.


acoustics_and_waves

Interference, resonance, and rooms:
  superposition        waves add; phase determines the result
  standing wave        reflection interfering with itself; nodes and antinodes
  beats                two close frequencies; the amplitude modulates at the
                       difference frequency
  natural modes        determined by geometry and boundary conditions
  string and open pipe  all harmonics of the fundamental
  closed pipe          odd harmonics only; sounds an octave lower for the same
                       length
  Helmholtz resonator   a cavity with a neck; a single low resonance, used as a
                       bass absorber and as the mechanism of a blown bottle
  quality factor       sharpness of resonance; high Q rings longer
  room modes           axial, tangential, oblique; dominate below the Schroeder
                       frequency
  Schroeder frequency  the transition from modal to statistical behaviour;
                       depends on volume and reverberation time
  reverberation time   time for a 60 dB decay; Sabine's formula relates it to
                       volume and total absorption
  absorption           porous materials absorb high frequency; low frequency
                       requires depth or resonant absorbers
  diffusion            scatters reflections; preserves energy while removing
                       specular reflections
  flutter echo         repeated reflections between parallel hard surfaces;
                       fixed by absorption, diffusion, or non-parallel geometry


acoustics_and_waves

The frequency dependence of absorption is the practical difficulty in acoustic treatment.
Porous absorbers work by viscous loss as air moves through them, which requires the material to
sit where the particle velocity is high — a quarter wavelength from a reflecting surface. For a
one-hundred-hertz wave that is 85 centimetres, so a thin panel on the wall absorbs high
frequencies effectively and low frequencies barely at all. Treating a room with thin absorptive
panels therefore removes the highs and leaves the lows, producing a space that is
simultaneously dead and boomy — the characteristic result of acoustic treatment applied without
attention to frequency.


acoustics_and_waves

[user] My small home studio sounds boomy and the bass response changes completely when I move
my chair a foot.
[assistant] tool_call acoustics.room_analysis({"dimensions_m": [4.2, 3.4, 2.5],
"treatment": "thin_foam_panels_5cm", "symptom": ["boomy", "position_dependent_bass"]})
[tool] {"volume_m3": 35.7, "axial_modes_hz": [40.9, 50.5, 68.6, 81.8, 100.9, 101.0],
"mode_pileup": [100.9, 101.0], "schroeder_hz": 138,
"foam_effective_above_hz": 1000, "verdict": "untreated_modal_region",
"recommendation": "thick_porous_or_membrane_bass_traps_in_corners"}
[assistant] Both symptoms come from room modes, and your treatment does not affect them at all.


acoustics_and_waves

Below about 138 Hz your room behaves modally rather than statistically: sound at those
frequencies forms standing waves with fixed nodes and antinodes, so the level at a given
frequency depends strongly on where you sit. Moving a foot moves you relative to those patterns,
which is exactly what you observe. Your lowest axial modes are at 40.9, 50.5, and 68.6 Hz, and
you have two modes nearly coincident at 101 Hz, which will produce a pronounced peak there — the
boominess.


acoustics_and_waves

Your 5 cm foam is effective above roughly 1 kHz and does essentially nothing below 500 Hz. A
porous absorber works by viscous loss where air particle velocity is high, which is a quarter
wavelength from the boundary — 85 cm at 100 Hz. Thin panels sit in the pressure maximum and
velocity minimum at low frequency and absorb almost nothing there. The result is a room with
the highs removed and the lows untouched, which sounds both dead and boomy at once.


acoustics_and_waves

Three interventions, in order of effect per effort.


acoustics_and_waves

Corner bass traps. Room corners are pressure maxima for every axial mode simultaneously, so
absorption there is the most efficient placement available. Use thick porous absorbers — 30 cm
of mineral wool or equivalent, straddling the corner with an air gap behind — or purpose-built
membrane or Helmholtz absorbers tuned near your problem frequencies. Treat all four vertical
corners first, then the wall-ceiling edges.


acoustics_and_waves

Move the listening position. Modal nulls and peaks are at predictable locations, and the worst
position is usually the room centre for one axis and against a wall for another. Positioning
yourself at roughly 38% of the room length from the front wall is a common starting rule, and
measuring with a sweep and a microphone will do better than any rule.


acoustics_and_waves

Then reconsider the foam. Once the low end is under control, the mid and high absorption you
already have may be excessive for a small room, and some of it is probably better replaced with
diffusion to keep the space from sounding dead.


acoustics_and_waves

Measure before and after with a free measurement tool and a calibrated microphone. Room
acoustics is the area where subjective impression is least reliable and measurement is most
informative, because the problems are frequency- and position-specific in ways that listening
does not localize.


acoustics_and_waves

## Transducers, Ultrasound, and Applications

A transducer converts between acoustic and electrical energy. Microphones convert pressure to
voltage, and the two main principles have complementary characteristics: dynamic microphones use
a coil moving in a magnetic field, which is robust, needs no power, and has limited
high-frequency response; condenser microphones use a charged diaphragm forming a capacitor,
which is more sensitive and extended in response and requires polarizing voltage.


acoustics_and_waves

Directional response is determined by how the microphone senses pressure. A sealed capsule
responds to pressure alone, which is a scalar, so it is omnidirectional. A capsule open at both
sides responds to the pressure difference across it, which depends on the direction of arrival,
giving a figure-of-eight pattern. Combining the two in different proportions produces the
intermediate patterns, and the cardioid — sensitive in front and null behind — is the sum of
equal parts of each.


acoustics_and_waves

Ultrasound uses frequencies above human hearing, and its applications exploit the short
wavelength. Medical imaging transmits pulses and forms an image from the echo timing and
amplitude, with resolution set by the wavelength and penetration limited by frequency-dependent
attenuation — which is the fundamental tradeoff, since higher frequency gives better resolution
and less depth. Non-destructive testing finds internal flaws in materials by the same principle.
Therapeutic ultrasound uses focused high intensity to heat or ablate tissue without incision.


acoustics_and_waves

Transducers and applications:
  dynamic microphone   moving coil; robust, unpowered, limited high frequency
  condenser            charged diaphragm; sensitive, extended response, needs power
  omnidirectional      responds to pressure; a sealed capsule
  figure of eight      responds to pressure gradient; open at both sides
  cardioid             the sum of the two; front sensitive, rear null
  proximity effect     gradient microphones boost bass close up; a consequence of
                       the near-field pressure gradient
  loudspeaker          the reverse transduction; a coil in a field driving a cone
  ultrasound imaging   echo timing and amplitude; resolution versus penetration
  attenuation          rises with frequency; the depth-resolution tradeoff
  Doppler ultrasound   flow velocity from the frequency shift
  sonar                underwater ranging; sound is the only long-range option
                       since electromagnetic waves attenuate rapidly in water
  non-destructive test  internal flaws from reflections
  acoustic levitation  standing wave nodes trap small objects
  cavitation           bubble formation and violent collapse under intense sound;
                       the mechanism of ultrasonic cleaning and of a damage mode
                       in pumps and propellers


acoustics_and_waves

## Noise Control and Vibration

Noise control follows a hierarchy from source to receiver, and interventions closer to the
source are more effective and usually cheaper. Reducing the source is best — a quieter machine,
a lower speed, a resilient mounting. Interrupting the path is next — enclosure, barrier,
absorption. Protecting the receiver is last — hearing protection, which is effective and
depends on consistent correct use.


acoustics_and_waves

Transmission through a partition is governed by mass at most frequencies: the transmission loss
rises about six decibels per doubling of surface mass, which is the mass law. This is why a
lightweight wall transmits sound regardless of how absorptive its surface is, and why the
effective strategies are mass, decoupling the two faces so vibration does not bridge them, and
damping. The weakest element dominates the whole partition, so a well-built wall with an
unsealed gap around a door performs at the level of the gap.


acoustics_and_waves

Vibration isolation reduces transmission between a machine and its support, and the mechanism
is a mass-spring system whose natural frequency must be well below the excitation frequency.
Isolation begins only above the square root of two times the natural frequency, and below that
the mount amplifies rather than isolates — which is why a poorly chosen mount makes the problem
worse, and why isolating low-frequency excitation requires very soft mounts with correspondingly
large static deflection.


acoustics_and_waves

Noise and vibration control:
  source, path, receiver  intervene as close to the source as possible
  mass law             transmission loss rises ~6 dB per doubling of surface mass
  decoupling           separate the two faces; avoids structural bridging
  damping              converts vibration to heat; constrained layer damping is
                       effective on panels
  flanking             sound bypassing the partition through structure or gaps;
                       the usual reason a good wall underperforms
  the weakest element  a gap or a duct penetration dominates the whole partition
  absorption vs isolation  absorption reduces reverberant level in a space;
                       isolation reduces transmission between spaces; they are
                       different problems with different materials
  active noise control  generate an anti-phase signal; effective for
                       low-frequency periodic noise in a confined space,
                       impractical for broadband or open environments
  vibration isolation  isolate above sqrt(2) times the mount's natural frequency;
                       below that it amplifies
  static deflection    a soft mount deflects more; low-frequency isolation
                       requires large deflection
  resonance avoidance  keep structural natural frequencies away from excitation
                       frequencies; the primary vibration design rule
  hearing damage       cumulative and irreversible; risk rises with both level and
                       duration, and the exchange rate means halving exposure time
                       permits a 3 dB higher level


acoustics_and_waves

The confusion between absorption and isolation is the most common practical error in acoustic
work and is worth stating plainly. Absorptive material placed on a wall reduces the reverberant
sound level within the room by converting reflected energy to heat. It does almost nothing to
reduce sound passing through the wall to the next room, because transmission depends on the
partition's mass, stiffness, and damping rather than on its surface absorption. Someone
attempting to soundproof a room by lining it with foam has treated the wrong problem, and the
correct interventions — mass, decoupled construction, sealing every penetration — are
structural and expensive rather than surface treatments.


atomic_and_molecular

## Atomic Structure

The hydrogen atom is the reference case because it is exactly solvable, and every more complex atom
is described in language borrowed from it. A single electron bound to a proton has energy levels
depending only on the principal quantum number, spaced as the inverse square, converging to the
ionization limit. The degeneracy of states with the same principal quantum number but different
angular momentum is an accident of the Coulomb potential and is lifted in every other atom.


atomic_and_molecular

Multi-electron atoms require approximation because the electron-electron interaction has no closed
form. The central field approximation replaces it with an effective spherically symmetric potential
that each electron moves in, which restores the hydrogenic labelling while breaking the accidental
degeneracy: states of lower angular momentum penetrate closer to the nucleus, feel less screening
from the inner electrons, and lie lower in energy. This screening effect is what produces the
ordering of subshells and therefore the structure of the periodic table.


atomic_and_molecular

The Pauli exclusion principle fills those states, and Hund's rules determine the ground
configuration of a partially filled shell: maximize total spin first, then total orbital angular
momentum, then couple them according to whether the shell is less or more than half filled. The
spin rule follows from exchange — parallel spins keep electrons apart spatially and reduce their
repulsion — which is a genuinely quantum-mechanical origin for what looks like an empirical rule.


atomic_and_molecular

Atomic structure:
  hydrogen levels      energy proportional to minus one over n squared
  the accidental degeneracy  l-independence, unique to the Coulomb potential
  central field        an effective spherical potential per electron
  screening            inner electrons reduce the effective nuclear charge
  penetration          low-l states reach closer in and are bound more tightly
  subshell ordering    the consequence; the periodic table's structure
  Pauli principle      no two electrons in the same state
  Hund's rules         maximize S, then L, then J by shell filling
  exchange interaction  the origin of the spin rule; not a force but a
                       consequence of antisymmetry
  term symbols         the labelling scheme for a configuration's states
  LS coupling          valid for light atoms; spin-orbit weak compared to
                       electrostatic
  jj coupling          valid for heavy atoms; spin-orbit dominant
  the crossover        intermediate coupling; most of the middle table