|
Post by Karel on Jun 5, 2016 15:01:31 GMT
So on p 214 the book says:
"Calculate SE of the estimated test MSE for each model size."
How is this done? Bootstrap it?
|
|
ShittySpaghettiPlot
Guest
|
Post by ShittySpaghettiPlot on Jun 5, 2016 20:50:40 GMT
Here is how I understand it to work (for k fold CV): within_one_se.R (the script pulls the Credit dataset from the website of ISLR). Basically, for best subset and CV k fold = 10, you estimate the test MSE for each k-fold 1 <= k <= 10, for each of the best predictor subsets. Each set of MSEs allows you to calculate a mean MSE and standard error of the mean MSE for each subset of predictors. I calculated the standard error by taking the standard deviation of the MSE for each predictor set and dividing it with sqrt(k), where k is the highest k fold. Lemme know if you all think this solution is right! ~ Alex
|
|
|
Post by Lodovico on Jun 5, 2016 21:44:12 GMT
I agree with the solution!
|
|
|
Post by Unicorn on Jun 6, 2016 12:39:15 GMT
I did something similar (played with the Hitters set instead). But instead of using Ariel's lines, I drew horizontal dashed red lines representing the oneSE area (the lower one falling in is the selected), which may be clearer ! abline(h=mean.cv.errors[min]+sd.cv.errors[min]/sqrt(k), col="red", lty=2) abline(h=mean.cv.errors[min]-sd.cv.errors[min]/sqrt(k), col="red", lty=2)
|
|
|
Post by NullModel on Jun 6, 2016 17:20:04 GMT
I did something similar (played with the Hitters set instead). But instead of using Ariel's lines, I drew horizontal dashed red lines representing the oneSE area (the lower one falling in is the selected), which may be clearer ! abline(h=mean.cv.errors[min]+sd.cv.errors[min]/sqrt(k), col="red", lty=2) abline(h=mean.cv.errors[min]-sd.cv.errors[min]/sqrt(k), col="red", lty=2) That's pretty nice! I added your plot to my R script. Also, I fixed a mistake that caused the error bars for the first predictor column to be re-used across all the datapoints. So, if anyone downloaded the original version for use during the exam, then please grab the new one.
|
|