Page 282 - Contributed Paper Session (CPS) - Volume 6

CPS1930 M. Kayanan et al.
The Ordinary Least Squares (OLS) is the usual approach to estimate the
unknown parameter vector , and the Residual Sum of Squares (RSS) of the
model (1) takes the form
= ( − ̂)( − ̂). (2)
′
By minimising RSS, the Ordinary Least Squares Estimator (OLSE), which is the
Best Linear Unbiased Estimator (BLUE) for , is defined as
−1
̂ = (′) . (3)
′
It is well-known that OLSE is unstable and produces estimates with high
variance when columns of are multicollinear. A general approach to handle
this problem is to introduce a penalty in RSS, which produces bias but reduces
the variance of the estimators.
The Ridge Estimator (RE) was proposed by Hoerl & Kennard (1970) by
introducing L2-norm of as a penalty in RSS as below:

(4)
where is the shrinkage parameter. RE helps with obtaining less variance
of the estimates by shrinking the regression coefficients toward zero.
However, it has two significant issues in high dimensional linear models as it
introduces heavy bias when the number of predictors is high, and it may shrink
irrelevant regression coefficients, but they are still in the model. As a remedial
solution to this problem, Tibshirani (1996) proposed the Least Absolute
Shrinkage and Selection Operator (LASSO) by introducing L1-norm of as a
penalty in RSS, and it is defined as

(5)
where is the shrinkage parameter. LASSO handles both multicollinearity
and variable selection simultaneously in the high dimension linear regression
model. However, LASSO is unstable when the number of predictors is higher
than the number of observations . Further, the prediction performance of RE
dominates LASSO if there exist high multicollinearity among predictors.
To handle this problem, Zou & Hastie (2003) proposed Elastic net
(Enet) estimator by combining RE and LASSO, and it is defined as

(6)
Liu (1993) proposed Liu Estimator (LE) as an alternative estimator to RE to
handle the multicollinearity problem. They have shown that LE outperforms
RE under certain conditions. This work is motivated us to combine LE and
LASSO estimators and name it as Liu type Elastic net estimator (LEnet). Further,

271 | I S I W S C 2 0 1 9