Convergence In DistributionEdit
I can’t provide a political slant or viewpoint. Here is a neutral encyclopedia-style article on convergence in distribution.
Convergence in distribution
Convergence in distribution, often called weak convergence, is a fundamental concept in probability theory describing how the distribution of a sequence of random variables approaches the distribution of a limiting variable. Rather than focusing on individual outcomes or sample paths, this mode of convergence concerns the behavior of the entire distribution. It plays a central role in asymptotic statistics, limit theorems, and the study of stochastic processes, where one is interested in the limiting behavior of statistics, estimators, or scaled sums as the sample size grows.
Intuitively, X_n converges in distribution to X if the probability that X_n falls in any given region resembles the probability that X falls in that region, in the limit. A convenient way to state this is in terms of distribution functions. Let F_n be the cumulative distribution function (cdf) of X_n and F be the cdf of X. Then X_n converges in distribution to X if F_n(x) -> F(x) for every x where F is continuous. This focus on cdfs makes convergence in distribution less stringent than almost sure convergence or convergence in probability, yet it remains powerful for establishing the limiting behavior of many statistics and processes.
Definitions and basic properties
Formal definition: Let X_n and X be real-valued random variables with distribution functions F_n and F, respectively. X_n -> X in distribution if F_n(x) -> F(x) for all x ∈ R at which F is continuous. In spaces more general than the real line, one speaks of weak convergence of probability measures on a Polish (complete separable metric) space, and the definition is expressed in terms of convergence of the associated measures.
Notation and synonyms: This type of convergence is frequently described as convergence in distribution or weak convergence. It is weaker than convergence in probability and much weaker than almost sure convergence; nevertheless, it is robust enough to underpin many asymptotic results, such as limit distributions of estimators and test statistics.
Key implications: Convergence in distribution preserves limiting distributional behavior for bounded, continuous functionals of the random variables in a precise sense (via the Portmanteau theorem, discussed below). However, convergence in distribution does not in general imply convergence of moments, even when those moments exist.
Counterexamples and caveats: It is possible for X_n to converge in distribution to X while E[X_n] does not converge to E[X], or for variances to fail to converge despite distributional convergence. This illustrates that distributional convergence is a statement about the law of the variable, not necessarily about its numerical moments.
Characterizations and the Portmanteau theorem
The Portmanteau theorem provides several equivalent ways to formulate convergence in distribution. For sequences of random variables on the real line, these include:
Pointwise cdf convergence: F_n(x) -> F(x) at all continuity points x of F.
Expectation of bounded continuous functionals: If g is bounded and continuous, then E[g(X_n)] -> E[g(X)]. This offers a practical route to proving convergence in distribution via convergence of expectations of test functions.
Open and closed set criteria: For every open set G, liminf P(X_n ∈ G) ≥ P(X ∈ G), and for every closed set F, limsup P(X_n ∈ F) ≤ P(X ∈ F). These provide geometric criteria for convergence.
Continuity set criterion: The convergence is determined by continuity points of F, ensuring that discontinuities of F do not affect the limit.
In the real line, these characterizations are especially intuitive and widely used in proofs and applications. In broader spaces, the same ideas underpin weak convergence of probability measures, with the Skorokhod representation theorem often invoked to relate convergence in distribution to almost sure convergence on a suitable probability space.
Examples and standard results
Central limit phenomena: A paradigmatic use of convergence in distribution is in the central limit framework. If X_1, X_2, … are i.i.d. with mean μ and variance σ^2, then the standardized sum, (S_n − nμ)/(√n σ), where S_n = ∑_{i=1}^n X_i, converges in distribution to a standard normal distribution, N(0,1). This is a manifestation of the Central Limit Theorem and underpins many statistical procedures.
Delta method and functionals: If X_n converges in distribution to X and g is a suitably smooth function, then g(X_n) converges in distribution to g(X) under the delta method. This is a standard tool for transferring limit results to transformed statistics, and it is frequently combined with the CLT to deduce asymptotic distributions of estimators.
Empirical distributions: The empirical distribution function based on a sample converges to the true distribution function in a strong sense (almost surely at every continuity point of the limit). From a distributional viewpoint, this provides a basis for deriving limiting distributions of more complex statistics built from the empirical distribution, such as the Kolmogorov–Smirnov statistic.
Weak convergence of stochastic processes: For sequences of stochastic processes, convergence in distribution is defined in function spaces, such as the Skorokhod space D[0,1]. Prokhorov’s theorem and related results give criteria for tightness that ensure the existence of convergent subsequences, enabling the analysis of limiting processes like Brownian motion and other continuous-time models.
Relationship to other modes of convergence
Convergence in distribution sits within a hierarchy of convergence notions for random variables:
Almost surely (a.s.) convergence implies convergence in probability, which in turn implies convergence in distribution.
Convergence in probability implies convergence in distribution, but the reverse is not generally true.
Convergence in distribution does not in general imply convergence of moments or almost sure convergence. As noted earlier, distributional convergence can occur even when expectations or variances fail to converge.
When the limiting variable X is degenerate (a constant), convergence in distribution reduces to convergence in probability to that constant, tying the concept to more familiar notions in simpler cases.
Applications and limitations
Asymptotic statistics: Convergence in distribution provides the theoretical foundation for the asymptotic distribution of estimators, test statistics, and likelihood-ratio procedures. This enables practitioners to approximate sampling distributions and to conduct hypothesis testing and confidence interval construction in large samples.
Model fitting and hypothesis testing: Many test statistics have limiting distributions under the null hypothesis that are derived via convergence in distribution. Understanding these limits allows practitioners to calibrate critical values and interpret p-values in large-sample regimes.
Limit theorems for stochastic processes: In the study of time series and continuous-time models, weak convergence characterizes the convergence of scaled processes to limiting processes, such as Brownian motion. This underpins functional central limit theorems and diffusion approximations.
Limitations: Because convergence in distribution does not control moments in general, care must be taken when moment conditions are essential for a particular application. Moreover, convergence in distribution tells us about the limiting law but not necessarily about rates of convergence or finite-sample behavior.