Algorithmic bias

An algorithmic bias is an (undesirable) bias of an algorithm. In machine learning, this can typically occur if the training dataset contains biased data, e.g. data with historical gender or racial biaises.

Impossibility theorem

Group and individual fairness are incompatible Stucchio18.

Word embedding

The case of word embedding is particularly important, as algorithms rely more and more on natural language processing trained with historical texts. Such texts usually contain a lot of implicit biases which are essentially impossible to clean.

BCZSK16 GSJZ18 showed that the word embedding of occupations correlated with gender. They found out that "computer programmer - man + woman ≈ homemaker", among other disturbing results.

Note however that NNG19 show that the highly publicized "doctor-man+woman=nurse" is actually an artefact due to forbidding the use of "doctor" as a reply.