1. The longley data set. > library(MASS) > data(longley) > head(longley) GNP.deflator

GNP Unemployed Armed.Forces Population Year Employed

1947

83.0 234.289

235.6

159.0

107.608 1947 60.323

1948

88.5 259.426

232.5

145.6

108.632 1948 61.122

1949

88.2 258.054

368.2

161.6

109.773 1949 60.171

1950

89.5 284.599

335.1

165.0

110.929 1950 61.187

#Generation of the test and training sets > nn<-nrow(longley) > index<-1:nrow(longley) > testindex<-sample(index,trunc(length(index)/3)) > testindex [1] 5 10 1 15 4 > testset<-longley[testindex,] > trainset<-longley[-testindex,] #Exclude the dependent var >trainset1<- trainset[,-7]

1(a). Ridge regression using the lm.ridge() function # Find the best lambda parameter > select(lm.ridge(trainset$Employed~.,data=trainset1,lambda=seq(0,10,0.001)) +) modified HKB estimator is 4.074423e-05 modified L-W estimator is 0.0004975858 smallest value of GCV at 0

> ridgereg2<-lm.ridge(trainset$Employed ~ ., data = trainset1, lambda = 0) #Print the coefficients of the regression so that we can see the offset (the first coefficient). #We will then offset our predicted values with it. > lm.ridge(trainset$Employed ~ ., data = trainset1, lambda = 0) GNP.deflator

GNP

Unemployed Armed.Forces

Population

Year

-4.402906e+03 -8.868277e-03 -5.470213e-02 -2.260566e-02 -1.167793e-02 -4.800817e-02 2.305525e+00 > pred.value<-scale(testset[,1:6],center = F, scale = ridgereg2$scales)%*% ridgereg2$coef > sumh=0 > tt<-nrow(testset) #Let’s have a look at our predicted values. > head(pred.value) 1951 4465.487 1956 4470.334 1947 4462.957 1961 4471.758 1950 4464.585 #Below we add the offset to our predicted values. sumh=0 > for(i in 1:tt){ sumh=sumh+(pred.value[i]-4402.90-testset[i,7])^2 } > sumg<-sumh/tt > sumg [1] 0.2245632

1. (b) Ridge Regression using the linearRidge() function.

ridge1_reg<-linearRidge(formula=trainset$Employed~., data=trainset1,lambda="automatic") > nn<-predict(ridge1_reg,testset[,1:6]) > sumh=0 > for(i in 1:tt) { sumh=sumh+(nn[i]-testset[i,7])^2 } sumg<-sumh/tt >sumg 0.3804 NOTE: The test error for the case of linear regression is 0.227 , and so the lm.ridge() produces slightly better results.

2. The prostate data set. > library(ElemStatLearn) > data(prostate) > head(prostate) lcavol

lweight age

lbph svi

lcp gleason pgg45

lpsa train

1 -0.5798185 2.769459

50 -1.386294

0 -1.386294

6

0 -0.4307829

TRUE

2 -0.9942523 3.319626

58 -1.386294

0 -1.386294

6

0 -0.1625189

TRUE

3 -0.5108256 2.691243

74 -1.386294

0 -1.386294

7

20 -0.1625189

TRUE

4 -1.2039728 3.282789

58 -1.386294

0 -1.386294

6

0 -0.1625189

TRUE

> trainset<-prostate[prostate$train==TRUE,] > testset<-prostate[prostate$train==FALSE,] #Exclude the logic column > trainset1<-trainset[,-10] #Exclude the dependent var > trainset2<-trainset1[,-9] #The trainset contains 512 cases, and the testset contains 256 cases. The dependent variable is lpsa.

2(a). Ridge regression using the lm.ridge() function > select(lm.ridge(trainset$lpsa~.,data=trainset2,lambda=seq(0,10,0.001)) +) modified HKB estimator is 3.355691 modified L-W estimator is 3.050708 smallest value of GCV

at 4.922

> ridgereg2<-lm.ridge(trainset$lpsa ~ ., data = trainset2, lambda = 4.922) > lm.ridge(trainset$lpsa ~ ., data = trainset2, lambda = 4.922) lcavol

lweight

age

lbph

svi

lcp 0.095859807 -0.116496939

0.492493357

gleason

pgg45

0.017243225

0.007074211

0.601029107 -0.014805753

0.137989392

0.679417525

> pred.value<-scale(testset[,1:8],center = F, scale = ridgereg2$scales)%*% ridgereg2$coef > sumh=0 > tt<-nrow(testset)

sumh=0 #Note how we add the offset (first coefficient above) to the predicted values. > for(i in 1:tt) { + sumh=sumh+(pred.value[i]+0.0958-testset[i,9])^2 +} > sumg<-sumh/tt > sumg [1] 0.4943575

2 (b) Ridge Regression using the linearRidge() function.

>library(ridge) > ridge1_reg<-linearRidge(formula=trainset$lpsa~., data=trainset2,lambda="automatic") > nn<-predict(ridge1_reg,testset[,1:8]) > sumh=0 > for(i in 1:tt) { + sumh=sumh+(nn[i]-testset[i,9])^2 +} > sumg<-sumh/tt > sumg 7 0.4895868 NOTE: The linear regression test error is 0.5212 and so ridge regression does better in terms of test error.

ridge regression.pdf

Find the best lambda parameter. > select(lm.ridge(trainset$Employed~.,data=trainset1,lambda=seq(0,10,0.001)). + ). modified HKB estimator is 4.074423e-05.

588KB Sizes 8 Downloads 278 Views

Recommend Documents

Eagle Ridge -
The pool is for the exclusive use of the property owners and their guests. Property owners will be responsible for all actions and conduct of their guests, and must accompany their guests when they use the pool. Residents may have ten guests accompan

Cache Creek Ridge
Stalley, a recent graduate of Wellesley College; and her sister Alyssa Stalley, a recent ... Chuck Stalley is the former California state champion in Ride & Tie.

Cache Creek Ridge
For the BLM website, go to: www.blm.gov/ca/st/en/fo/ukiah/cachecreek.html ... a lush bed of red clover that is evidently more tasty than the bales of grass-hay back at ... At right, Sharon Wimberg is delighted with her 5th place finish and Top-Ten ..

glenwood ridge apartments -
SMITH CHARLES A JR ETALS CO TRS. 3811 GLENWOOD AVE. ZONING: R-63. PARCEL ID: E0001116037. ACREAGE: 0.271 AC. SMITH CHARLES A JR ...

Blue Ridge Elementary.pdf
There was a problem previewing this document. Retrying... Download. Connect more apps... Try one of the apps below to open or edit this item. Blue Ridge ...

blue ridge mountains
By Fleet Foxes. Fro the album 'Fleet Foxes'. CAPO IV. Em. E A D G B E. F#m. E A D G B E. G. E A D G B E. 0 2 2 0 3 0. 2 4 4 2 2 2. 3 5 5 4 3 3. Em*. E A D G B E.

River Ridge Crossing.pdf
Sign in. Loading… Whoops! There was a problem loading more pages. Retrying... Whoops! There was a problem previewing this document. Retrying.

Rock Ridge Updates.pdf
Sign in. Loading… Whoops! There was a problem loading more pages. Retrying... Whoops! There was a problem previewing this document. Retrying.

Ridge 360 final.pdf
Page 1 of 13. Exercices de révision- Oxydo-réduction et Piles électrochimiques, SMPC2, fsr. 2015. Prs . A. Eyahyaoui, A. Zrineh at M. Alaoui Elbelghiti. Université Mohammed V. Faculté des Sciences. Département de Chimie. Rabat. Année Universit

basking ridge docs.pdf
unequal treatment before the Planning Board, including in relation to purported fire safety issues. More evidence of discrimination emerges from .... others) without the Court's intervention, in addition to phone calls. Defendants have refused to. en

oak ridge boys greatest.pdf
Loading… Page 1. Whoops! There was a problem loading more pages. oak ridge boys greatest.pdf. oak ridge boys greatest.pdf. Open. Extract. Open with.

Edy Ridge Fall Order Form
$18.00 each. YOUTH. Make check payable to Edy Ridge PAC. Grand Total. Name: Phone#:. Teacher: Room#:. SCREENPRINTED. EMBROIDERED. SM (6-8) MED (10-12) LRG (14-16) XL (18-20). SM (6-8) MED (10-12) LRG (14-16) XL (18-20). Long Sleeve T-Shirt Black. Lon

Rock Ridge Sale Order.pdf
There was a problem previewing this document. Retrying... Download. Connect more apps... Try one of the apps below to open or edit this item. Rock Ridge ...

Oak Ridge School Information.pdf
Loading… Whoops! There was a problem loading more pages. Whoops! There was a problem previewing this document. Retrying... Download. Connect more apps... Oak Ridge Sc ... ormation.pdf. Oak Ridge Sch ... formation.pdf. Open. Extract. Open with. Sign

The Ridge SFA 2014.pdf
Page 1 of 26. Page 1 of 26. Page 2 of 26. Page 2 of 26. Page 3 of 26. Page 3 of 26. The Ridge SFA 2014.pdf. The Ridge SFA 2014.pdf. Open. Extract. Open with. Sign In. Details. Comments. General Info. Type. Dimensions. Size. Duration. Location. Modifi

(503) 925-8282 laurel ridge laurel ridge (503) 925 ... -
DOMINO'S IS PROUD TO SUPPORT OUR LOCAL. SCHOOLS BY HELPING TO RAISE MONEY FOR THEIR. MUCH NEEDED PROGRAMS AND SUPPLIES.

Vimy Ridge Canadian Heritage Tour - Parent & Student ...
Vimy Ridge Canadian Heritage Tour - Parent & Student Communication - February 2016.pdf. Vimy Ridge Canadian Heritage Tour - Parent & Student ...

Laurel Ridge Skating Club
International: AUS, CON, GW, MB, RW, SAM, TR, YP. $45.00 each. Free Dance (Circle Appropriate Test). Standard, Adult, Masters. Juvenile. $35.00 each. Intermediate. $40.00 each. Novice. $50.00 each. Junior. $60.00 each. Senior. $65.00 each. Make your

PROMONTORY AT KIOWA RIDGE OWNERS ...
PROMONTORY AT KIOWA RIDGE OWNERS ASSOCIATION, INC. Annual Meeting Agenda. October 3, 2015 at 1:00 p.m.. Rocky Mountain Church, Conference ...

Dakota Ridge Learner Profile.pdf
Whoops! There was a problem loading more pages. Dakota Ridge Learner Profile.pdf. Dakota Ridge Learner Profile.pdf. Open. Extract. Open with. Sign In.

Cordilleras Ledge and Ridge Circuit.pdf
Retrying... Whoops! There was a problem loading this page. Retrying... Cordilleras Ledge and Ridge Circuit.pdf. Cordilleras Ledge and Ridge Circuit.pdf. Open.