Which predictors should I keep?
You have eight columns and no idea which belong in the model. Two classical tools answer that — and both come with a warning.
Best subsets fits every combination and ranks them. With 8 predictors that is 255 models; the panel shows the best few at each size so you can see the trade-off directly: does the 5-variable model really explain more than the 3-variable one?
Stepwise adds the most useful predictor, then re-checks whether anything already in has become redundant, and repeats. It scales to more variables, and the panel lists every decision it made so the result is not a black box.
Which number to read. Plain R-squared never falls when you add a variable — it cannot tell you when to stop. Adjusted R-squared penalises each extra predictor, so it can fall; that is the number to compare. Mallows Cp near the parameter count means nothing important is missing; a much larger Cp says a predictor is.
The warning that matters. Automatic selection produces a hypothesis, not proof. We measured it: with pure random data and a 5% threshold, forward stepwise still found at least one "significant" predictor 45% of the time. Worse, the p-values in the model it hands you are no longer trustworthy, because the same data was used to choose the model and to test it. Treat the shortlist as a suggestion to check against process knowledge — or against fresh data.
Best subsets fits every combination and ranks them. With 8 predictors that is 255 models; the panel shows the best few at each size so you can see the trade-off directly: does the 5-variable model really explain more than the 3-variable one?
Stepwise adds the most useful predictor, then re-checks whether anything already in has become redundant, and repeats. It scales to more variables, and the panel lists every decision it made so the result is not a black box.
Which number to read. Plain R-squared never falls when you add a variable — it cannot tell you when to stop. Adjusted R-squared penalises each extra predictor, so it can fall; that is the number to compare. Mallows Cp near the parameter count means nothing important is missing; a much larger Cp says a predictor is.
The warning that matters. Automatic selection produces a hypothesis, not proof. We measured it: with pure random data and a 5% threshold, forward stepwise still found at least one "significant" predictor 45% of the time. Worse, the p-values in the model it hands you are no longer trustworthy, because the same data was used to choose the model and to test it. Treat the shortlist as a suggestion to check against process knowledge — or against fresh data.
Try it in the app
Try: load "Steel - Extended", pick a Y, press Suggest with Best subsets, and compare the 2-variable and 4-variable rows. Then switch to Stepwise and read the decision list.