explore/exploit tradeoff: model vs. reality
To navigate life we create mental models of the world out there, and then we confuse the models for reality. What is the explore/exploit tradeoff? The gambler needs to learn new knowledge about the machines and simultaneously use what they have already learned to optimize their decisions. In the literature, these two activities are referred to as exploring and exploiting. You can’t do both things at the same time. When you explore, you are pulling new arms on the bandit trying to figure out their expected payout....