I say “hair-splitting,” but I think the philosophical difference between these fields is actually pretty profound.


  • Statistics
    • Attempts to answer questions about truth; epistemology. Rather idealistic.
    • Proposes a set of possible models, and posits the existence of a “true” model within that set. This “true” model explains the observations.
      • This description applies to frequentist and bayesian statistics.
      • Both camps take their models seriously as descriptions of reality. They only differ in their description of epistemic state, and in their methods for generating that description.
    • Attempts to quantify our epistemic state.
      • Frequentists: confidence intervals, \(p\)-values
      • Bayesians: posterior distributions; Bayes factors


  • Machine Learning
    • More pragmatic: what is useful?
    • Seeks a model for accomplishing some task: e.g., prediction or decision making.
    • Selects a model based on its performance at that task—fitness for purpose. (A splash of Darwinism.)
    • Makes no pretense at determining truth—is fundamentally an engineering discipline.
      • We may choose different models based on trade-offs between performance, available memory, available processing power, available data, or other details of the operating environment.
    • Machine learning takes algorithmic concerns into account.
      • Is there an efficient procedure for determining the best model?
      • This is the result of its origins within computer science.


My personal inclinations can probably be guessed from these descriptions.

\( \blacksquare\)