Tree Models — Thresholds and Interactions
Regression answers one question: how much does the response move per unit of an input? Two very common shop-floor patterns escape it — an effect that only starts above a threshold (fine up to 700 °C, ruinous above it) and an effect that only appears when two inputs are high together.
A tree looks for split points instead, so it finds both. A single tree is readable but memorises its training data; a random forest averages hundreds of trees and generalises far better, at the cost of not being drawable.
Importance here is measured by permutation on rows the model never saw: shuffle one variable, see how far the score falls. That avoids the bias of impurity-based importance, which flatters columns with many distinct values. Importance is still not causation — two copies of the same measurement split the credit and can both look unimportant.
A tree looks for split points instead, so it finds both. A single tree is readable but memorises its training data; a random forest averages hundreds of trees and generalises far better, at the cost of not being drawable.
Importance here is measured by permutation on rows the model never saw: shuffle one variable, see how far the score falls. That avoids the bias of impurity-based importance, which flatters columns with many distinct values. Importance is still not causation — two copies of the same measurement split the credit and can both look unimportant.
Try it in the app
Try: Tree Models tab → Load example → Fit model. Compare the held-out score with the training score, then switch the model to "single tree" and read the splits.