Introduction.- Stochastic Multi-armed Bandit.- Markov Multi-armed Bandit.- Adversarial Multi-armed Bandit.- Spectrum Sensing and Access in Cognitive Radio Networks.- Sniffer Channel Assignment in Multi-channel Wireless Networks.- Online Routing in Multi-hop Wireless Networks.- Channel Selection and User Association in WiFi Networks.