Quantcast
Channel: Question and Answer » regularization
Viewing all articles
Browse latest Browse all 33

Range of lambda in elastic net regression

$
0
0

$defl{|!|}$
Given the elastic net regression

$$min_b frac{1}{2}l y – Xb l^2 + alphalambda l bl_2^2 + (1 – alpha) lambda l bl_1$$

how can an appropriate range of $lambda$ be chosen for cross-validation?

In the $alpha=1$ case (ridge regression) the formula

$$textrm{dof} = sum_j frac{s_j^2}{s_j^2+lambda}$$

can be used to give an equivalent degrees of freedom for each lambda (where $s_j$ are the singular values of $X$), and degrees of freedom can be chosen in a sensible range.

In the $alpha=0$ case (lasso) we know that

$$lambda > lambda_{textrm{max}} = max_j|sum_t y_t X_{tj}|$$

will result in all $b_j$ being zero, and $lambda$ can be chosen in some range $(0, lambda_textrm{max})$.

But how to handle the mixed case?


Viewing all articles
Browse latest Browse all 33

Trending Articles