Evaluating Linear Regression Model in R

R has many different commands to quickly analyze a linear model. Below is a list of such commands and some tips on how to interpret there output.

1. Summary


>model <- lm(Income~(Price+Temp+Consumption)^2,data=ice) >summary(model)

Sample Output


Parameter Formula Explanation
Residual (R) =y-h_theta(x) In general, residuals should have a normal distribution with mean close to zero and 1Q and 3Q having about the same absolute value.
Degree of Freedom (df ) Number of Training Records – Number of Coefficients
Residual Standard Error (sigma ) sqrt{frac{sum{R^2}}{df}} Residual standard error should be as small as possible (Note, the whole objective of machine learning was minimize error. Also think of overfitting.)
Standard Error Of Variable (sigma_i) $latex frac{sigma}{sqrt{n}} Its the standard deviation of the sample divided by square root of the sample size. This is same as estimating population variation from sample variation.
t-value frac{theta_i}{hat{sigma_i}} t value make sense only if we have P value, which is listed in the next column
P Value From table It gives the probability of achieving a value as large as t so that null hypothesis is true. Here, null hypothesis is that coefficient estimate is zero. Assuming we are using 5% confidence interval for two-tailed t-test, then P value should be less than 0.05 in order for us to believe in the coefficient estimate.
F Statistics ?? ??

2. Plot





Plot General behavior and Interpretation
Residual Vs Fitted Points should be randomly scattered around the center line (dotted). Any pattern (such as all points on one side) indicates either violation of linearlity or homoscedasticity. The plot also help identify the boundary of the model. For instance, the red line coincides with the center line initially but moves upwards as fitted value increases. This in general indicates that the model is good only lower fitted values.
Scale Location ??
Normal Q-Q Plot This plot helps analyze whether the distribution of residual error is normal or not. In general, the points should be along the diagonal line. If you notice any particular pattern (such as bump), it usually indicate the the residual error is not uniformly distributed. Sometimes, points will be in a line but either parallel to the diagonal or in some other direction. This usually indicate ?
Residual Vs Leverage Plot ??


About Ritesh Agrawal

I am a applied researcher who enjoys anything related to statistics, large data analysis, data mining, machine learning and data visualization.
This entry was posted in General, Machine Learning. Bookmark the permalink.

10 Responses to Evaluating Linear Regression Model in R

  1. Hello, i read your blog from time to time and i own a similar one and i was just wondering if you get a lot of spam remarks? If so how do you reduce it, any plugin or anything you can recommend? I get so much lately it’s driving me crazy so any help is very much appreciated.

    • Ritesh Agrawal says:

      Hi. I recently moved from wordpress.com to my own server and definitely since then seeing many more comments. At wordpress.com they had Akismet which helped stop spam. But that service is not free. I haven’t search for anything but I am sure there will be some free plugins to stop spam

  2. I really like your blog.. very nice colors & theme. Did you create this website yourself or did you hire someone to do it for you? Plz reply as I’m looking to design my own blog and would like to know where u got this from. thanks

  3. Adel says:

    Fantastic goods from you, man. I’ve understand your stuff previous to and you’re just too great. I really like what you’ve acquired here, really like what you’re saying and the way in which you say it. You make it entertaining and you still take care of to keep it smart. I can’t wait to read much more from you. This is really a wonderful site.

  4. Its really like you read my thoughts! You could know a great deal concerning this, like you wrote the book in it or something. I do believe that you could do with some pics to drive what it’s all about home somewhat, but aside from that, it will be superb blog..

  5. I don’t ordinarily comment but I gotta say regards for that post on this one.

  6. Hello! I just wish to make a huge thumbs up for the terrific advice you have here on this article. I will be returning to your site for more shortly.

  7. get a loan says:

    hey buddy, this is a very interesting article

  8. Hello! This is my first comment here so I just wanted to give a quick shout out and tell you I really enjoy reading your posts. Can you suggest any other blogs/websites/forums that go over the same topics? Appreciate it!

  9. I’m extremely impressed with your writing skills as well as with the layout on your weblog. Is this a paid theme or did you customize it yourself? Either way keep up the nice quality writing, it’s rare to see a great blog like this one these days..

Comments are closed.