Calibration

noun

say it: “kal-uh-BRAY-shun” /ˌkæləˈbreɪʃən/

How closely the probabilities a model gives match how often things really happen.

What is calibration for a model's probabilities?

Calibration is how well a model's probabilities match what really happens: of all the cases it gives 0.7, about 7 in 10 should turn out positive. A well-calibrated model's 0.7 means the same in every group, a fairness goal that, when base rates differ, can't be met together with equalized odds.

In a sentence

  • “The audit checked calibration separately for each district.”
  • “Good calibration meant the model's 0.7 really did come true about 70% of the time.”

Level: Freelancer · More AI Basics terms at this level

Updated