Equalized Odds

noun

say it: “EE-kwuh-lyzd ODZ” /ˈikwəˌlaɪzd ˈɑdz/

A fairness rule that a model should correctly accept qualified people, and wrongly accept unqualified people, at the same rates in every group.

What is equalized odds?

Equalized odds is a fairness test that asks for the same true positive rate (recall) and the same false positive rate in every group: qualified people picked equally often, and unqualified ones wrongly picked equally often. Fairlearn reports the larger of the two gaps with equalized_odds_difference().

from fairlearn.metrics import equalized_odds_difference
qualified = [1, 0, 1, 0, 1, 0, 1, 0]
invited = [1, 1, 1, 0, 1, 0, 0, 0]
district = ["N", "N", "N", "N", "S", "S", "S", "S"]
print(equalized_odds_difference(qualified, invited, sensitive_features=district))   # 0.5

In a sentence

  • “Equalized odds failed: qualified applicants from South were picked half as often.”
  • “Unlike demographic parity, equalized odds looks at who really qualified.”

Level: Freelancer · More AI Basics terms at this level

Updated