Decision Tree vs Logistic Regression
THE VERDICT
Two interpretable baselines with opposite shapes: logistic regression captures smooth additive effects and extrapolates linearly; a tree captures thresholds and interactions but predicts in flat steps and cannot extrapolate at all. Continuous risk score → logistic. Rule-like domain ("if income < X and tenure < Y") → tree. Both fit in a code review either way.
[ 01 ] Side by side
| Dimension | Decision Tree | Logistic Regression |
|---|---|---|
| Family | Classification/Regression | Classification |
| Interpretability | High | High |
| Training speed | Fast | Fast |
| Data needed | Small | Small |
| Complexity | Low | Low |
| Training cost | O(n·d·log n) | O(n·d) per iteration |
| Inference cost | O(depth) ≈ O(log n) | O(d) |
[ 02 ] When to choose each
Choose Decision Tree when…
- Interpretable models
- Mixed data types
- Quick exploration
…but not when
- You need the best accuracy - a single tree is almost always beaten by its ensembled versions
- Smooth linear relationships dominate (a tree approximates a line with clumsy stair-steps)
- Small data with noisy labels - deep trees will memorise the noise
How it works: Play twenty-questions with your data. At every node the tree asks the single yes/no question that best un-mixes the classes (or reduces variance for regression), splits the data, and recurses. Predictions follow the questions down to a leaf. The result is a flowchart a domain expert can audit line by line.
Full Decision Tree dossier →Choose Logistic Regression when…
- Binary classification
- When probability scores needed
- Interpretable predictions
…but not when
- Decision boundary is strongly non-linear and feature crosses cannot fix it
- Classes are perfectly separable - weights diverge without regularisation
- You have millions of sparse one-hot features but need interactions - trees handle those natively
How it works: Linear regression squeezed through a sigmoid: compute a weighted score, then map it to a probability between 0 and 1. The decision boundary is still a straight line - what changes is that the output is a calibrated "how sure am I", and the weights are trained to make observed labels as likely as possible.
Full Logistic Regression dossier →[ 03 ] Quick answers
Q.01When should I use Decision Tree instead of Logistic Regression?
Decision Tree is the better choice for: Interpretable models; Mixed data types; Quick exploration. Avoid it when: You need the best accuracy - a single tree is almost always beaten by its ensembled versions
Q.02When should I use Logistic Regression instead of Decision Tree?
Logistic Regression is the better choice for: Binary classification; When probability scores needed; Interpretable predictions. Avoid it when: Decision boundary is strongly non-linear and feature crosses cannot fix it
Q.03Is Decision Tree or Logistic Regression easier to interpret?
Decision Tree: high interpretability. Logistic Regression: high interpretability. Two interpretable baselines with opposite shapes: logistic regression captures smooth additive effects and extrapolates linearly; a tree captures thresholds and interactions but predicts in flat steps and cannot extrapolate at all.