Least Trimmed SquaresEdit

Least Trimmed Squares, abbreviated LTS, is a robust regression technique designed to yield reliable fits in the presence of outliers. By focusing on a subset of the data with the smallest residuals and trimming away the largest ones, LTS protects the estimated relationship between predictors and response from being distorted by aberrant observations. The method sits squarely in the tradition of robust statistics, a field that emphasizes reliability under model deviations rather than chasing perfectly Gaussian behavior. LTS was introduced in the 1980s by Peter J. Rousseeuw as part of a broader program to build regression tools with high resistance to data contamination and influential points. Its core idea—minimizing the sum of squared residuals over a chosen subset—remains straightforward and appealing for practitioners who want transparent, interpretable results in the presence of data quality problems.

In practice, LTS is best understood as a balance between fidelity to the bulk of the data and resistance to a minority of troublesome observations. With a data set of size n and p predictors (including the intercept), the method selects a subset of h observations (h > p) and computes the regression fit that minimizes the sum of squared residuals on that subset. By construction, if one can trim enough points, the estimator becomes largely immune to outliers that would otherwise pull the fit toward spurious directions. This leads to a high breakdown point, meaning the estimator can tolerate a substantial fraction of contaminated data without collapsing. In modern terms, LTS is affine equivariant and scale-invariant, so it behaves sensibly under linear changes of coordinates and units, and it remains interpretable as a regression relation between the predictors and the response. For readers exploring the mathematical underpinnings, see discussions of robust regression and breakdown point.

History

The Least Trimmed Squares approach emerged during the development of robust regression techniques in the late 20th century. It was motivated by the need to handle datasets in which a minority of observations—such as measurement errors, data-entry mistakes, or rare but influential market shocks—could distort ordinary least squares estimates. The early work connected LTS to the broader family of estimators that aim to retain the essential signal from the center of the data while downweighting or excluding outliers. For historical context and connections to related methods, see Least Median of Squares and the lineage of S-estimator and MM-estimator approaches. Contemporary surveys also discuss how LTS fits within the wider framework of robust regression and compared alternatives such as M-estimator, RANSAC, and other robust schemes.

Definition and properties

Least Trimmed Squares solves, for a chosen subset size h, the optimization problem that minimizes the sum of squared residuals over the h observations with the smallest residuals. Equivalently, it selects a subset H of h indices and finds the β that minimizes sum_{i in H} (y_i − x_i^T β)^2. The resulting estimator inherits several desirable features:

High breakdown point: LTS can tolerate a substantial fraction of contaminated observations without breaking down, depending on h.
Affine equivariance: If the data are transformed by an affine change of coordinates, the estimator transforms in the same way as the underlying model.
Robustness to outliers in both the response and the predictors, making it attractive for real-world data where clean, well-behaved assumptions rarely hold.
Interpretability: Like OLS, the estimator offers a linear relation between predictors and response, but with far less distortion from anomalous points.

In practice, the choice of h controls the efficiency–robustness trade-off. A larger h yields behavior closer to ordinary least squares on typical data but reduces robustness; a smaller h increases robustness but can sacrifice efficiency when the bulk of the data is well-behaved. In most applications, practitioners select h to balance practical reliability with reasonable efficiency, often guided by empirical studies and diagnostic checks. For a broader view of these trade-offs, see entries on MM-estimator and S-estimator.

Computation and algorithms

Computing LTS exactly is combinatorially challenging because it requires searching over many possible subsets of size h. In response, a family of fast algorithms was developed to make LTS practical for moderate to large datasets. The key idea is to sample a large number of subsets, compute regression fits on each subset, evaluate the objective on the corresponding h-small residuals, and then refine the best candidates with iterative procedures. The resulting method, commonly referred to as FAST-LTS, delivers reliable results much more quickly than a brute-force search. Modern implementations may incorporate additional refinements such as reweighting steps or MM-estimation techniques to improve efficiency and robustness. For readers comparing approaches, see robust regression and RANSAC as alternative strategies for handling outliers.

Relationships to other methods

LTS sits alongside a spectrum of robust regression techniques, each with its own strengths and weaknesses:

M-estimators minimize a sum of a chosen loss function of the residuals, downweighting outliers but often with a different sensitivity profile than LTS.
S-estimators focus on achieving a high breakdown point through scale-based objectives, sometimes at the cost of efficiency for clean data; LTS and S-estimators can be used in tandem or as starting points.
MM-estimators combine high breakdown and high efficiency by starting with a robust estimator and refining it to improve efficiency while maintaining robustness.
RANSAC is another robust fitting approach that iteratively fits models to random subsets and selects the best consensus, often used in computer vision and related fields. For context, see M-estimator, S-estimator, MM-estimator, and RANSAC.

Applications and practical considerations

LTS and related robust regression methods are widely used across disciplines where data quality varies and outliers are a real possibility. In economics and finance, LTS helps protect model estimates from anomalous observations that can arise from data entry errors, regime changes, or extreme market events. In engineering and environmental science, sensor glitches or irregular measurements can unduly influence regression fits, making robustness desirable. In statistics education and practice, LTS provides a clear, interpretable alternative to OLS that remains aligned with the instinct that a model should describe the central tendency of the majority of the data rather than be swayed by a handful of aberrant points. See also robust statistics for the broader methodological context.

Controversies and debates surrounding robust methods

As with many methodological choices, there is debate about when and how to use LTS. A frequent talking point in favor of robust regression is its resilience to outliers and data contamination, which aligns with a pragmatic, real-world view of data analysis. Critics—often emphasizing efficiency under ideal, well-specified models—argue that robust methods can be unnecessarily conservative and suffer from lower efficiency when the data genuinely follow the assumed model without contamination. In practice, the choice of method should reflect the data-generating process and the analyst’s tolerance for risk of distortion.

From a practical, market-minded perspective, the main defense of LTS rests on reliability and predictability. In decision contexts where a handful of outliers can drive materially different conclusions—such as risk assessment, pricing, or policy analysis—robust methods reduce the chance of cataclysmic inference errors. Critics who portray robust statistics as merely ideological or fashion-driven miss the point: LTS is about maintaining credible inference when data do not conform to idealized assumptions. Those who advocate for broader methodological flexibility argue that a layered approach—using LTS as a robust starting point or as part of an MM-estimation pipeline—often yields the best balance of resilience and efficiency.

See also discussions of how robust regression compares to standard econometric practices, and how contemporary robust methods fit within the wider toolbox of data analysis.