Sam's Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== Statistical Fairness ======
The idea is to check that a classifier is fair by testing if membership in the class is //independent// from membership in a protected group (such as race, sex, etc.).

More formally, let the class attribute be $y = \{+, -\}$, and the protected attribute be $S = \{s, \overline{s}\}$, where $s$ denotes a sample //inside// the protected class, and $\overline{s}$ denotes a sample //outside// the protected class.(([[https://arxiv.org/abs/2110.00530|Quy et al, 2021]]))

There are a few ways to concretize the notion of fairness, the simplest of which is maybe statistical parity(([[https://arxiv.org/abs/1104.3913|Dwork et al., 2011]])), which says that if a class prediction is fair, it should satisfy:
$$
P(\hat{y}|S=s) - P(\hat{y}|S=\overline{s}) \leq \epsilon
$$
Which can be intuitively understood to mean that for any outcome $\hat{y}$, the probability of the outcome //conditioned on// it being in the protected class ought to be about the same as (upto some tolerance $\epsilon$) the probability if it's //not// in the protected class.