Entropy is one of the most important concepts in physics. In information theory it is quantitatively associated with information. Information is associated with a probability distribution.
