Regression and Machine Learning

Abstract

Machine learning (ML) techniques have become one of the most successful scientific tools and changed the everyday life of people around the globe (e.g., search engines). A vast amount of digital data sources on human behaviour has emerged due to the rise of the internet and opened the door for computer scientists to apply ML on social phenomena. In the social sciences, however, the adoption of ML has been less enthusiastic. To investigate the relation of traditional statistics and ML, this paper shows how ML might be used as regression analysis. For that purpose, we illustrate what a typical social science approach might look like and how using ML techniques could contribute additional insights when it comes to estimators (non-linearity) or the assessment of model fit (predictive power). In particular, we reveal how epistemological differences shape the potential usage of ML in the social sciences and discuss the methodological trade-off of applying ML compared to traditional statistics.