Decision-making in the face of uncertainty is a challenge in machine learning, and the multi-armed bandit model is a common framework to address it. This comprehensive introduction is an excellent reference for established researchers and a resource for graduate students interested in exploring stochastic, adversarial and Bayesian frameworks.