This article is about estimation and inference methods for high dimensional sparse (HDS) regression models in econometrics. High dimensional sparse models arise in situations where many regressors (or series terms) are available and the regression function is well-approximated by a parsimonious, yet unknown set of regressors. The latter condition makes it possible to estimate the entire regression function effectively by searching for approximately the right set of regressors. We discuss methods for identifying this set of regressors and estimating their coefficients based on l1 -penalization and describe key theoretical results. In order to capture realistic practical situations, we expressly allow for imperfect selection of regressors and study the impact of this imperfect selection on estimation and inference results. We focus the main part of the article on the use of HDS models and methods in the instrumental variables model and the partially linear model. We present a set of novel inference results for these models and illustrate their use with applications to returns to schooling and growth regression. -- inference under imperfect model selection ; structural effects ; high-dimensional econometrics ; instrumental regression ; partially linear regression ; returns-to-schooling ; growth regression
In this chapter we discuss conceptually high dimensional sparse econometric models as well as estimation of these models using l1-penalization and post-l1-penalization methods. Focusing on linear and nonparametric regression frameworks, we discuss various econometric examples, present basic theoretical results, and illustrate the concepts and methods with Monte Carlo simulations and an empirical application. In the application, we examine and confirm the empirical validity of the Solow-Swan model for international economic growth.
The “Stats in the Château” summer school was held at the CRC château on the campus of HEC Paris, Jouy-en-Josas, France, from August 31 to September 4, 2009. This event was organized jointly by faculty members of three French academic institutions ─ ENSAE ParisTech, the Ecole Polytechnique ParisTech, and HEC Paris ─ which cooperate through a scientific foundation devoted to the decision sciences. The scientific content of the summer school was conveyed in two courses, one by Laurent Cavalier (Université Aix-Marseille I) on "Ill-posed Inverse Problems", and one by Victor Chernozhukov (Massachusetts Institute of Technology) on "High-dimensional Estimation with Applications to Economics". Ten invited researchers also presented either reviews of the state of the art in the field or of applications, or original research contributions. This volume contains the lecture notes of the two courses. Original research articles and a survey complement these lecture notes. Applications to economics are discussed in various contributions.
Econometric models based on observational data are often endogenous due to measurement error, autocorrelated errors, simultaneity and omitted variables, non-random sampling, self-selection, etc. Parameter estimates of these models without corrective measures may be inconsistent. The potential high-dimensional feature of these models (where the dimension of the parameters of interests is comparable to or even larger than the sample size) further complicates the statistical estimation and inference. My dissertation studies two different types of high-dimensional endogenous econometrics problems in depth and develops statistical tools together with their theoretical guarantees. The first essay in this dissertation explores the validity of the two-stage regularized least squares estimation procedure for sparse linear models in high-dimensional settings with possibly many endogenous regressors. The second essay is focused on the semiparametric sample selection model in high-dimensional settings under a weak nonparametric restriction on the form of the selection correction, for which a multi-stage projection-based regularized procedure is proposed. The number of regressors in the main equation, p, and the number of regressors in the first-stage equation, d, can grow with and exceed the sample size n in the respective models. The analysis considers the sparsity case where the number of non-zero components in the vectors of coefficients is bounded above by some integer which is allowed to grow with n but slowly compared to n, or the vectors of coefficients can be approximated by exactly sparse vectors. Simulations are conducted to gain insight on the small-sample performance of these high-dimensional multi-stage estimators. The proposed estimators in the second essay are also applied to study the pricing decisions of the gasoline retailers in the Greater Saint Louis area. The main theoretical results of both essays are finite-sample bounds from which sufficient scaling conditions on the sample size for estimation consistency and variable selection consistency (i.e., the multi-stage high-dimensional estimation procedures correctly select the non-zero coefficients in the main equation with high probability) are established. A technical issue regarding the so-called "restricted eigenvalue (RE) condition" for estimation consistency and the "mutual incoherence (MI) condition" for selection consistency arises in these multi-stage estimation procedures from allowing the number of regressors in the main equation to exceed n and this paper provides analysis to verify these RE and MI conditions. In particular, for the semiparametric sample selection model, these verifications also provide a finite-sample guarantee of the population identification condition required by the semiparametric sample selection models. In the second essay, statistical efficiency of the proposed estimators is studied via lower bounds on minimax risks and the result shows that, for a family of models with exactly sparse structure on the coefficient vector in the main equation, one of the proposed estimators attains the smallest estimation error up to the (n, d, p)-scaling among a class of procedures in worst-case scenarios. Inference procedures for the coefficients of the main equation, one based on a pivotal Dantzig selector to construct non-asymptotic confidence sets and one based on a post-selection strategy (when perfect or near-perfect selection of the high-dimensional coefficients is achieved), are discussed. Other theoretical contributions of this essay include establishing the non-asymptotic counterpart of the familiar asymptotic "oracle" type of results from previous literature: the estimator of the coefficients in the main equation behaves as if the unknown nonparametric component were known, provided the nonparametric component is sufficiently smooth.
This paper reviews the literature on sparse high dimensional models and discusses some applications in economics and finance. Recent developments of theory, methods, and implementations in penalized least squares and penalized likelihood methods are highlighted. These variable selection methods are proved to be effective in high dimensional sparse modeling. The limits of dimensionality that regularization methods can handle, the role of penalty functions, and their statistical properties are detailed. Some recent advances in ultra-high dimensional sparse modeling are also briefly discussed.
This is the third of three volumes containing edited versions of papers and commentaries presented at invited symposium sessions of the Tenth World Congress of the Econometric Society, held in Shanghai in August 2010. The papers summarize and interpret key developments in economics and econometrics, and they discuss future directions for a wide variety of topics, covering both theory and application. Written by the leading specialists in their fields, these volumes provide a unique, accessible survey of progress on the discipline. The first volume primarily addresses economic theory, with specific focuses on nonstandard markets, contracts, decision theory, communication and organizations, epistemics and calibration, and patents.
Quantile regression constitutes an ensemble of statistical techniques intended to estimate and draw inferences about conditional quantile functions. Median regression, as introduced in the 18th century by Boscovich and Laplace, is a special case. In contrast to conventional mean regression that minimizes sums of squared residuals, median regression minimizes sums of absolute residuals; quantile regression simply replaces symmetric absolute loss by asymmetric linear loss. Since its introduction in the 1970's by Koenker and Bassett, quantile regression has been gradually extended to a wide variety of data analytic settings including time series, survival analysis, and longitudinal data. By focusing attention on local slices of the conditional distribution of response variables it is capable of providing a more complete, more nuanced view of heterogeneous covariate effects. Applications of quantile regression can now be found throughout the sciences, including astrophysics, chemistry, ecology, economics, finance, genomics, medicine, and meteorology. Software for quantile regression is now widely available in all the major statistical computing environments. The objective of this volume is to provide a comprehensive review of recent developments of quantile regression methodology illustrating its applicability in a wide range of scientific settings. The intended audience of the volume is researchers and graduate students across a diverse set of disciplines.
In this paper I assess the ability of econometric and machine learning techniques to predict fiscal crises out of sample. I show that the econometric approaches used in many policy applications cannot outperform a simple heuristic rule of thumb. Machine learning techniques (elastic net, random forest, gradient boosted trees) deliver significant improvements in accuracy. Performance of machine learning techniques improves further, particularly for developing countries, when I expand the set of potential predictors and make use of algorithmic selection techniques instead of relying on a small set of variables deemed important by the literature. There is considerable agreement across learning algorithms in the set of selected predictors: Results confirm the importance of external sector stock and flow variables found in the literature but also point to demographics and the quality of governance as important predictors of fiscal crises. Fiscal variables appear to have less predictive value, and public debt matters only to the extent that it is owed to external creditors.
