Lasso with constraint on some coefficients (not all)
I would like to run a lasso regression (L1 penalisation) with a twist: there are different constraints on my problem. The coefficients for my features (predictors) are $beta_i$. I want to find the...
View ArticleWhy is it that xgb.cv performs well but xgb.train does not
I am trying to control overfitting using xgboost in R using eta but when I compare the overfitting of my xgb.cv readout to the xgb.train readout, I don’t know why xgb.cv doesn’t seem to overfit and...
View ArticleClik here to view.
GMM via mclust in R builds models with only some cluster numbers,...
I am using the mclust package in R for Gaussian mixture modelling. On some data I encountered that the some model types were used only up to some cluster numbers. Here is a reproducible example:...
View ArticleDifference between dropout and neurons with 0 weights
In the dropout method of regularization, we randomly delete half of the hidden neurons, leaving the input and output layers the same. In a theoretical sense, wouldn’t the same effect occur if we just...
View ArticleClik here to view.
Dependence on UV cut off of some $phi^4$ diagrams
Consider the one loop corrections to the propagator and the vertex in $phi^4$-theory: The former gives an integral representation proportional to $int d^4...
View ArticleClik here to view.
Bayesian regularisation for ANNs — How to modify the Jacobian?
Introduction I have implemented the Levenberg-Marquard algorithm (from Hagan’s “Artifical Neural Network Design” — 2014) for a two layer network with 20 neurons in the hidden layer. This network can...
View ArticleSelecting a loss-function for k-fold cross-validation over shrinkage parameter
I am doing a penalized regression with categorical (ordinal) outcomes. I would like to select the shrinkage parameter $lambda$ on the basis of cross-validation (CV). In this case, I have 50k...
View ArticleTikhonov regularization in the context of deconvolution
I came across “Tikhonov regularization” and I have bare knowledge on it. It seems that it is a type of regularization that is important for deconvolution. Are there any good resources and examples?...
View ArticleCan L1 linear regression perform worse than vanilla linear regression on...
I have a data set with 2 features and I’m trying to predict one real-valued variable. I use linear regression and I measure the error using 10-fold CV and absolute mean error as a metric. I noticed...
View ArticleRidge/Lasso Lambda greater than 1
I ran Ridge and Lasso regressions using an algorithm to automatically find the optimum lambda. However, the algorithm couldn’t find an optimum lambda between 0 and 1. In some cases I could find optimum...
View Article