Gradient Boosting (XGBoost/LightGBM) vs ARIMA
THE VERDICT
For one clean series, ARIMA’s statistical machinery is hard to beat. The moment you have many related series plus covariates - promotions, weather, prices - gradient boosting on lag/window features usually wins, because it pools strength across series and eats exogenous drivers natively. One store → ARIMA. A thousand stores → boosted trees (with time-aware validation, always).
[ 01 ] Side by side
| Dimension | Gradient Boosting (XGBoost/LightGBM) | ARIMA |
|---|---|---|
| Family | Classification/Regression | Time Series |
| Interpretability | Medium | High |
| Training speed | Medium | Fast |
| Data needed | Medium | Medium |
| Complexity | High | Medium |
| Training cost | O(M·n·d) with histogram tricks; sequential across rounds | O(n·iterations) - seconds for typical series |
| Inference cost | O(M·depth) | O(horizon) |
[ 02 ] When to choose each
Choose Gradient Boosting (XGBoost/LightGBM) when…
- Kaggle competitions
- Structured/tabular data
- When accuracy is priority
…but not when
- Tiny noisy datasets - boosting will happily fit the noise
- You need heavy uncertainty quantification out of the box
- Unstructured data (images, audio, raw text) - deep learning owns those
How it works: Build the model one small tree at a time, where each new tree is trained on the errors the ensemble is still making. Every round nudges predictions in the direction that most reduces the loss - literally gradient descent, but the "step" is a tree. Modern implementations (XGBoost, LightGBM, CatBoost) are the default winner on tabular data.
Full Gradient Boosting (XGBoost/LightGBM) dossier →Choose ARIMA when…
- Economic forecasting
- Demand planning
- Short-term prediction
…but not when
- Many related series with shared patterns (use pooled/global models: boosting, DeepAR-style)
- Strong exogenous drivers dominate (promotions, weather) - use ARIMAX or feature-based ML
- Multiple overlapping seasonalities and holiday effects (Prophet or ML handles these more gracefully)
How it works: Explain a series by its own past: tomorrow ≈ weighted recent values (AR), plus weighted recent forecast errors (MA), after differencing away the trend (I). Small, transparent, statistically principled - and with confidence intervals that mean something. For one well-behaved series, it is still a formidable baseline.
Full ARIMA dossier →[ 03 ] Quick answers
Q.01When should I use Gradient Boosting (XGBoost/LightGBM) instead of ARIMA?
Gradient Boosting (XGBoost/LightGBM) is the better choice for: Kaggle competitions; Structured/tabular data; When accuracy is priority. Avoid it when: Tiny noisy datasets - boosting will happily fit the noise
Q.02When should I use ARIMA instead of Gradient Boosting (XGBoost/LightGBM)?
ARIMA is the better choice for: Economic forecasting; Demand planning; Short-term prediction. Avoid it when: Many related series with shared patterns (use pooled/global models: boosting, DeepAR-style)
Q.03Is Gradient Boosting (XGBoost/LightGBM) or ARIMA easier to interpret?
Gradient Boosting (XGBoost/LightGBM): medium interpretability. ARIMA: high interpretability. For one clean series, ARIMA’s statistical machinery is hard to beat.