# Sexism spotted with Maths!

I did a talk in May this year called ‘Restore the balance of data’ at the Data Innovation Summit.  It was about sexism and other biases that are implicit in our existing electronic traces (actual and historical data) and my concern because we are using that data as baseline information to create the new prediction algorithms.

I’ve discussed this many times at home when preparing the talk.  We had vivid discussions with my husband and lovely sons over our family Sunday lunches. That’s how it didn’t surprise me that my eldest son, Alex, thought of me when reading  this article of the MIT Technology Review about sexism in our language.

The article is about a dataset of texts that researchers are using to “better understand everything from machine translation to intelligent Web searching.”  They are transforming words in the text into vectors, and then applying mathematical properties to derive meaning:

It turned out that words with similar meanings occupied similar parts of this vector space. And the relationships between words could be captured by simple vector algebra. For example, “man is to king as woman is to queen” or, using the common notation, “man : king :: woman : queen.” Other relationships quickly emerged too such as  “sister : woman :: brother : man,” and so on. These relationships are known as word embeddings.

The article is about the problem that researchers have identified on this data set, they say “: it is blatantly sexist.”  Here are some examples they provide:

But ask the database “father : doctor :: mother : x” and it will say x = nurse. And the query “man : computer programmer :: woman : x” gives x = homemaker.

Thinking about it, isn’t it obvious that if we have biases on our behavior, the writings about our world would be biased too?  And anything derived from our biased writing traces will reflect our views with all our biases too.

So we learned to extrapolate from our old behavior to predict our future behaviour… just to discover that we don’t like what we are getting out of it!  Our old behavior, amplified by the algorithm, doesn’t seem so good isn’t it? It’s clearer than ever that we don’t want to continue behaving like that in the future… Well, that’s a positive point, it’s good that this uncovers our blind spots, isn’t it?

Now the good news: it can be fixed!

The Boston team has a solution. Since a vector space is a mathematical object, it can be manipulated with standard mathematical tools.

The solution is obvious. Sexism can be thought of as a kind of warping of this vector space. Indeed, the gender bias itself is a property that the team can search for in the vector space. So fixing it is just a question of applying the opposite warp in a way that preserves the overall structure of the space.

Oh, seems so easy…for mathematicians anyway 😉  But no, even for mathematicians it is difficult to find and to measure the distortions:

That’s the theory. In practice, the tricky part is measuring the nature of this warping. The team does this by searching the vector space for word pairs that produce a similar vector to “she: he.” This reveals a huge list of gender analogies. For example, she;he::midwife:doctor; sewing:carpentry; registered_nurse:physician; whore:coward; hairdresser:barber; nude:shirtless; boobs:ass; giggling:grinning; nanny:chauffeur, and so on.

Having compiled a comprehensive list of gender biased pairs, the team used this data to work out how it is reflected in the shape of the vector space and how the space can be transformed to remove this warping. They call this process  “hard de-biasing.”

Finally, they use the transformed vector space to produce a new list of gender analogies[…]

Read the full article if you are interested on their process to de-biased.  Their conclusion, with which I completely agree is:

“One perspective on bias in word embeddings is that it merely reflects bias in society, and therefore one should attempt to debias society rather than word embeddings,” say Bolukbasi and co. “However, by reducing the bias in today’s computer systems (or at least not amplifying the bias), which is increasingly reliant on word embeddings, in a small way debiased word embeddings can hopefully contribute to reducing gender bias in society.”

That seems a worthy goal. As the Boston team concludes: “At the very least, machine learning should not be used to inadvertently amplify these biases.”