Page 281 - Contributed Paper Session (CPS) - Volume 6

CPS1930 M. Kayanan et al.

Combining LASSO and Liu type Estimator in the
Linear Regression Model
M. Kayanan , P. Wijekoon
3
1, 2
1 Postgraduate Institute of Science, University of Peradeniya, Peradeniya, Sri Lanka
2 Deparment of Physical Science, Vavuniya Campus of the University of Jaffna, Vavuniya, Sri
Lanka
3 Department of Statistics and Computer Science, University of Peradeniya, Peradeniya, Sri
Lanka

Abstract
The Ordinary Least Square Estimator (OLSE) has been widely used to estimate
unknown parameters in the linear regression model. Since OLSE produces
high variance on the estimates when multicollinearity exists among the
predictor variables, the Ridge Estimator (RE) is introduced as an alternative
estimator. However, RE yields heavy bias in the high dimensional linear
regression models, and it also produces irrelevant predictors to the estimated
model. Hence, the Least Absolute Shrinkage and Selection Operator (LASSO)
has been used to ensure the variable selection as well as to handle the
multicollinearity problem simultaneously. It is noted that LASSO failed to
outperform RE when high multicollinearity exists among the predictor
variables. Further, the LASSO estimator is unstable when the number of
predictors is higher than the number of observations. Hence, the Elastic net
(Enet) estimator is introduced to address this problem by combining LASSO
and RE. Since Liu Estimator (LE) is an alternative estimator for RE to address
multicollinearity problem, the objective of this study was to propose Liu type
Elastic net estimator by combining LASSO and LE. Then, we compared the
prediction performance of the Liu type Elastic net (LEnet) estimator with the
Elastic net and LASSO estimators in Root Mean Square Error (RMSE) sense
using the real-world examples. The results showed that LEnet outperforms the
other two estimators in RMSE sense.

Keywords
Multicollinearity; Variable selection; Liu estimator; LASSO; Elastic net

1. Introduction
Consider the linear regression model
= + (1)
where is the × 1 vector of observations on the predictor variable, is the
× matrix of observations on non stochastic regressor variables, is a
× 1 vectors of unknown parameters, is the × 1 vector of disturbances, ∼
(0, σ ).
2

270 | I S I W S C 2 0 1 9