Can An Algoritm be “Racist”?

Library of Congress Classification - Reading Room

David Auerbach has written this article pointing out that some classification algorithms may be racists :

Can a computer program be racist? Imagine this scenario: A program that screens rental applicants is primed with examples of personal history, debt, and the like. The program makes its decision based on lots of signals: rental history, credit record, job, salary. Engineers “train” the program on sample data. People use the program without incident until one day, someone thinks to put through two applicants of seemingly equal merit, the only difference being race. The program rejects the black applicant and accepts the white one. The engineers are horrified, yet say the program only reflected the data it was trained on. So is their algorithm racially biased?

Yes and a classification algorithm could not only be racist but, as humans write them, or more accurately with the learning algorithms, as they are built upon human examples and counter-examples, the algorithms may have any human bias that we have.  With the abundance of data, we are training programs with examples from the real world; the resulting programming will be an image of how we act and not a reflection on how we would like to be.  Exactly as the saying on educating kids: they do as they see and not as they are told :- )

To make things worse, when dealing with learning algorithms, not even the programmer can predict the resulting classification. So knowing that there may be errors,  who is there to ensure their correctness?

What about the everyday profiling that goes on without anyone noticing? [… ]
Their goal is chiefly “microtargeting,” knowing enough about users so that ads can be customized for tiny segments like “soccer moms with two kids who like Kim Kardashian” or “aging, cynical ex-computer programmers.”

Some of these categories are dicey enough that you wouldn’t want to be a part of them. Pasquale writes that some third-party data-broker microtargeting lists include “probably bipolar,” “daughter killed in car crash,” “rape victim,” and “gullible elderly.” […]

There is no clear process for fixing these errors, making the process of “cyberhygiene” extraordinarily difficult.[…]

For example, just because someone has access to the source code of an algorithm does not always mean he or she can explain how a program works. It depends on the kind of algorithm. If you ask an engineer, “Why did your program classify Person X as a potential terrorist?” the answer could be as simple as “X had used ‘sarin’ in an email,” or it could be as complicated and nonexplanatory as, “The sum total of signals tilted X out of the ‘non-terrorist’ bucket into the ‘terrorist’ bucket, but no one signal was decisive.” It’s the latter case that is becoming more common, as machine learning and the “training” of data create classification algorithms that do not behave in wholly predictable manners.

Further on, the author mentions the dangers or this kind of programming that is not fully predictable.

Philosophy professor Samir Chopra has discussed the dangers of such opaque programs in his book A Legal Theory for Autonomous Artificial Agents, stressing that their autonomy from even their own programmers may require them to be regulated as autonomous entities.

Chopra sees these algorithms as autonomous entities.  They may be unpredictable, but till now there is no will or conscious choice to go one path instead of another.  Programs are being told to maximize a particular benefit, and how to measure that benefit is a calculated by a  human written function.  Now as time goes by, and technological advances go their way, I can easily see that the benefit function could include certain feedback the program gets from ‘real world’ that could make the behavior of the algorithm still more unpredictable than now.  At that point we can think of algorithms that can evaluate or ‘choose’ to be on the regulated side.. or not? Will it reaches the point of them having a kind of survival instinct?   Where it may lead that…we’ll know it soon enough.

photo by:

Changing schools with gaming techniques

Could you imagine a world where children will ask you to bring them to school?  Well, that world doesn’t seem so far away… at least I know my son would be happy to  go to the school Ian Livingstone is planning to open in 2016 in Hammersmith, London.  Read what technology reporter Dave Lee wrote on his article in the BBC News:

By bringing gaming elements into the learning process, Mr Livingstone argued, students would learn how to problem-solve rather than just how to pass exams.


Mr Livingstone said he wanted to bring the principles of his interactive books to the classroom

[…] Mr Livingstone is best known for being the man behind huge franchises such as Tomb Raider and tabletop game Warhammer.

In the 80s, his Fighting Fantasy books brought an interactive element to reading that proved extremely popular.

Speaking to the BBC about the plans, Mr Livingstone said he wanted to bring those interactive principles to schooling, but stressed the school would provide learning across all core subjects.

There is more behind his idea than just making children wanting to go to school.  It fosters a ‘hands-on’ approach that allows students not only to know, but to know how to use the learned knowledge.  Plus the added benefit of allowing diverse paths to reach the goal:

By bringing gaming elements into the learning process, Mr Livingstone argued, students would learn how to problem-solve rather than just how to pass exams.

[…] “There needs to be a shift in the pedagogy of learning in classrooms because there’s still an awful lot of testing and conformity instead of diversity.

“I’m not saying knowledge is bad – I’m just trying to get a bit more know-how into the curriculum.”

He said he considers the trial-and-error nature of creating games as a key model for learning.

“For my mind, failure is just success work-in-progress. Look at any game studio and the way they iterate. Angry Birds was Rovio’s 51st game.

“You’re allowed to fail. Games-based learning allows you to fail in a safe environment.”

Let’s wish him a great success!

About Internet of Things and Privacy


Innovation is creating new materials, new sensors each time smaller, cheaper, more flexible, more powerful and at the same time less power-consuming. It allows to put them everywhere: we are surrounded with devices crowded with those sensors as our phones with cameras, gyroscopes and gps. And all those measurements captured by the sensors are being used by applications, many of which are connected to the cloud and to Internet.

Internet of Things (as this technology is called) is becoming ubiquitous, leaving us each time more exposed on our daily life.  How many of us have our whereabouts known by the GPS company, the Phone provider and even the car manufacturer?  Also our personal biometrical information is being left all over our running paths not to mention the new gym-centers.

On the other hand, Nicole Dewandre reminds us on this recorded presentation of two basic human needs: our human need of privacy and the fact that we construct ourselves through the public eye.

We need privacy to express our internal thoughts without public judgement, we need to be in a safe place to test and confront to others our lines of reasoning.  On our hyper-connected world, the spaces where we can profit from this privacy are vanishing.

As for our second need, the image the others have of us is very important. The information we leave behind influences this public image and it has a great effect not only on what others think of us, but also on our own perception of ourselves, on our self-esteem and finally it ends reflecting on our happiness.

Living on this hyper-connected world in which we are immersed is a real challenge!

Our 2 ways of thinking: Fast and Slow

From Jim Holt review  in The New York Times. Illustration by David Plunkert.

From Jim Holt review in The New York Times. Illustration by David Plunkert.

I just came back from holidays, and I want to share with you my last reading: “Thinking, Fast and Slow” by David Kahneman.  He describes our mind as having 2 different ways of functioning: a fast one, based on our ‘intuition’ and a slower one, where we have to do the effort of reasoning.

  • The fast one is the intuitive way, used on everyday tasks, and is also called by the psychologists our ‘unconscious mind’.  It is based on the inputs of our senses (hearing, sight, smell..). They trigger a search in our memory and bring through associations a representation of our situation and an immediate response to it.
  • The slower functioning way is when we focus our attention on the inputs at hand, and we follow a line of reasoning based on our knowledge to come to a conclusion.  This method requires more energy, we must direct our attention to each piece of information, and as we evaluate things sequentially (one thing after the other) it is slower.

As our body is lazy by nature, this second ‘slow’ way of reasoning is only used if needed, that is if the situation requires our ‘special attention’.  It is a great thing that our faster and energy-saving functioning way is our ‘default’…except for the fact that David Kahneman points out very interesting experiences that show the pitfalls of our intuition!

One great example he presents is the ambiguity resolution that goes behind our knowledge: when a sentence or image could be interpreted in different ways, our ‘fast mind’ resolves the ambiguity with the most recent context, which is good in many situations.  The problem is that it doesn’t even let us know that there was another interpretation at all!  We are not aware that our mind took only one of the possible alternatives. And moreover, it takes the easiest available memory to give sense to the world as we sense it.  So recent events that are more vivid on our memory have a greater impact on our interpretation of the world.  This is called the ‘availability bias’.

 Not only our memories play us games, but our whole body is linked to our intuitive way of functioning.  He mentions an experiment they performed in the United States where they asked the participants to look at photos and words related with elderly, then they asked them to move to another room, and that was the aim of the experience: they measured the time it took them to walk from their actual location to the other one.  They realized that the participants that have been shown pictures related to elderly were slower than the others, like if our body was related to what we have been thinking.  This is called the ‘priming’ effect.


And what may seem more surprising, this body-mind link works also the other way around: people requested to hold a pencil on their mouth had their mood adapted to the grimace they have been forced into.  Here is the details of the experiment: some participants were requested to hold the pencil by the middle of it, so having on one side of the mouth the point and on the other the eraser, some others were requested to hold the pencil putting their lips around the eraser end.  Then the 2 groups have been presented with the same cartoon images, and the first group found it on average  more funnier that the second group.  The first group seemed on a happier mood as if they have been smiling.  The second group were less positive after they have been forced on frowning before looking at it.

The conclusion is that we have to be really careful with our  mind’s evaluation of a situation if we have left it to our unconscious or intuitive mind.  It is biased by design!  The more aware we are about those biases, the better we are to counter them.

Games for breakthrough thinking

Using games for brainstorming is really great.  Instead of doing a standard meeting, the idea is to set a series of rules, and then play that game.  There is a clear beginning, once the rules have been explained and everybody agrees to play by the rules.  Then when the game is being played, the participants are free to explore the ‘game space’ that are all the possible situations that we can reach by applying the predefined rules.  And there is an end when the declared goal is reached.

image from book Gamestorming by Dave Gray, Sunny Brown and James Macanufo

image from book Gamestorming by Dave Gray, Sunny Brown and James Macanufo

Some goals are clearly defined like the ones limited by time: for example to come up in 3 minutes with as many ideas or words around a subject as possible.  Others have no time constraints; the end is to reach a desired end situation as in the 4-in-line or chess games.

But typically, in real situations where there is need of brainstorming, the goal is not so clear.  For problems that need creativity, new ideas or innovation usually the goal cannot be fully defined; it’s more like a general purpose.  We may have a general direction on where we want to go and we count on measures to see if we have succeeded.

But why playing a game for brainstorming?  Because we just love playing games 🙂 but more important because when we are on a game we feel free to explore all the alternatives and go beyond conventions.  And that facilitates innovative ideas to come up.  We just free ourselves from standard agreed conventions to cover all the possible alternatives that the rules of the game offer us.

As an example, I can mention the story of Timothy Ferriss, author of ‘The 4-hour workweek’ that won the gold medal at the Chinese Kickboxing National Championships.  He did not use to practice kickboxing, but he read the rules of that sport, and he explore the ‘game space’ of the Championship.  He then took advantage of 2 loopholes to participate with only 4-weeks of preparation!  One of the rules said that if the combatant fell off the platform 3 times in the row, his opponent won by default.   Another one allowed him to play in classes of lower weight than what he should have played in.  Those 2 rules combined made him the World Champion on Kickboxing.  He was not really playing; he was just pushing his opponents and won with that technique.  For sure you can argue it is not a fair way of winning, but it’s an interesting way of thinking in order to reach the goal of the game.

Another example comes from my son who had an assignment last year at the university: to program a robot so that it will follow a circuit, then it has to throw a piece of wood as far as it could and finishes by going back to its parking place.  There were points for each action: to reach the start line, to follow the path without going out of the route, to throw the piece of wood in a predetermined place and also to go back to the garage.  The path was unknown, only revealed at the time of the exam.  When the fatidic day came, the path that was presented to them was quite complicated and most of the robots failed.  But in one of the teams they had a ‘plan B’ that was a different set of programming instructions: they only programmed the robot  to do the tasks that gave points with the minimum risk: go to the starting point, go to the predetermined place and throw the piece of wood and then return to the garage. The robot didn’t even try to do the circuit, but with that strategy they were one of the 5 finalists!  Again, this is the same situation as with Timothy Ferriss: it doesn’t feel fair even if it played by the rules,  but worked for the assignement.

Now if your survival is at stake, let’s imagine a planetary catastrophe, wouldn’t it be good to have a ‘B’ strategy on your sleeve? 

MOOCs: the new learning style

Last week I presented MOOCs (Massive Open Online Courses) at the Professional Women International association in Brussels, Belgium.

I had the pleasure of talking to the participants afterwards.  They told me they were so pleased to learn they had such an easy way of taking good quality courses that they were going to check that same night for their preferred subjects 🙂

Happy to have contributed to spread the word about the availability of the MOOCs, putting all their encapsulated knowledge encapsulated at any user’s fingertips!

On the last slide, I just dropped words  with the main implications of this trend;  I encourage you to put a comment if any of the subjects I mention resonates with you:

As New Services Track Habits, the E-Books Are Reading You


In this article of The New York Times David Sreitfeld is discussing a new service ScribD is beginning to offer. Scribd is a subscription-based library, where you can read books through their interface.  They are now collecting information from their readers, like how long they stay on a page, the pace on specific chapters, do they reach the end of the book?.. The idea is to offer this insight to the authors, for them to improve their future deliveries.

Last week, Smashwords made a deal to put 225,000 books on Scribd, a digital library here that unveiled a reading subscription service in October. Many of Smashwords’ books are already on Oyster, a New York-based subscription start-up that also began in the fall.

The move to exploit reading data is one aspect of how consumer analytics is making its way into every corner of the culture. Amazon and Barnes & Noble already collect vast amounts of information from their e-readers but keep it proprietary. Now the start-ups — which also include Entitle, a North Carolina-based company — are hoping to profit by telling all.

“We’re going to be pretty open about sharing this data so people can use it to publish better books,” said Trip Adler, Scribd’s chief executive.

Quinn Loftis, a writer of young adult paranormal romances who lives in western Arkansas, interacts extensively with her fans on Facebook, Pinterest, Twitter, Goodreads, YouTube, Flickr and her own website. These efforts at community, most of which did not exist a decade ago, have already given the 33-year-old a six-figure annual income. But having actual data about how her books are being read would take her market research to the ultimate level.

Here are some results they could extract from their data:

Scribd is just beginning to analyze the data from its subscribers. Some general insights: The longer a mystery novel is, the more likely readers are to jump to the end to see who done it. People are more likely to finish biographies than business titles, but a chapter of a yoga book is all they need. They speed through romances faster than religious titles, and erotica fastest of all.

They are “reading us” while we read 🙂 but let’s not be paranoid, we will be getting more attractive books… Let’s hope it doesn’t limit our choices in the future.

Massive Open Online Courses

Massive Open Online Courses (MOOC) are very recent, but are quickly gaining popularity.  Coursera is one of the big platforms that offer those free courses, along with edX and Khanacademy just to mention a few.  Last year I took a fantastic course offered by Coursera  called ‘Model Thinking’ given  by Prof. Scott E. Page, who’s the Director of the Center for the Study of Complex Systems at the University of Michigan ( I posted already about it here : – ).

In March this year, I was glad to receive a mail from Scott Page, giving us some feedback from his experience doing this course, and sending us also a link to a presentation he did about the making of the course.

To give you an idea of the popularity of this course, there were 60.000 students enrolled on the first run of Model Thinking, beginning of 2012.  It grew to 100.000 for the fall run (by the way, if you are interested there will be a new run this fall 2013, and it may be the last one, says Prof. Page).

I would like to share with you Scott’s insights on his experience on making this online course contrasting it with the making of his online course ‘The hidden Factor’.  This last one was professionally done in a studio and he called ‘Model Thinking’: my garage band online course : – )

In fact, it was really recorded in one unused room of his house, because he said that the starting and stopping of the heating system in the rest of the house was picked up by his mike, so sensible it was even though it was just a $100 one.

To prepare the course, he thought of making it more modular.  So he cut it in small chunks, so that each video was independent, and treated a subject in no more than 15 minutes.  But as he said, that was the easiest part because what took him much more time was the recording of each lecture.  One big issue he had was that he was alone in this room to do the recordings, and trying to be smiling, engaging and enthusiastic is difficult without an audience.  Not only that, but he had unforeseen events from time to time, like his dog wandering around, and he laughed and found himself doing funny movement to chase him.

The editing took a lot of time, each video had to be reviewed, and in case of errors, it was difficult to fix it.  So at the end, some mistakes remained.   On the other hand in the professional approach, they took care of each error, but they had better tools and a battery of technicians to look into them and find different alternatives to correct them.  Sometimes he had to repeat one word they detected he had staggered with, and they told them even the intonation he had to use to repeat it; sometimes they just put a picture about the subject he was talking about, and he could rephrase one sentence.

In conclusion, here’s his comparison regarding costs to do the 2 videos:


So it is much more costly for a professional quality. Time-wise, it was surprisingly more or less equivalent:


The studio made video was undisputable better, being much easier to correct any mistakes:



But in the end, is the improvement in quality worth the cost?  Not really he says; the best quality is not needed, a good enough approach is better, even more if the cost prohibits its making.  So the best solution stands between those 2 options.

I found also very important his comment on how presenting this course changed his everyday work life.  He has now 1 hour per day reading his mail, answering to diverse requests on his subject of expertise.  He receives inquiries from technical advisors, deans, diverse influencial people that he cannot really discard.  On the one hand it’s not strictly his job, for what he is paid for, but on the other hand, can these requests be ignored? Is it responsible if you know your intervention can have such an impact as to do better policies, to improve many people’s life?

Snowden showed us the dangers of Big Data with PRISM, are we up to the challenge to steer its use?

A television screen shows former U.S. spy agency contractor Edward Snowden during a news bulletin at a cafe at Moscow’s Sheremetyevo airport June 26, 2013. Credit: Reuters/Sergei Karpukhin


As we already discussed on my Big Data presentations,   being able to analyse the amount of data that traces all our actions and movements is a great opportunity to improve our lives, as much as to do business, but it can also be exploited for the worst.  Now Edward Snowden has put a clear case under the spotlights, will this make us move? Will this lead to change?

It’s time to consider what ethical codes and regulations can be issued, so that this excellent opportunity that technology is putting in our hands, that is sharing, measuring and extracting knowledge from all aspects of our lives, is not misused.

Nine Key Questions To Evaluate A New Technology

Technical and scientific knowledge is growing at a fast speed.  This allows for a lot of experimentation and innovation but in order to do it, you need first to acquire very specific knowledge, to educate yourself in many technical disciplines.

This is not at everybody’s reach.  In particular, it is hardly at reach of non-technical decision-makers, as business managers or politicians. How can we expect politicians and the general public to be able to take an ethical or societal position, to legislate, on issues they don’t have the background knowledge nor the time (or will) to learn?

Nevertheless, some of these new inventions need legislation, and definitely you need to be aware of it as a business decision maker.  Here’s a practical approach to tackle this issue: use critical thinking, using contextual knowledge instead of technical one.

I’m extrapolating here from Miguel Aznar’s 9 questions to tackle the nanotechnology issue, in order to get a contextual comprehension of any technical issue:

  1. What is this new technique?
  2. Why do we use it?
  3. Where does it come from?
  4. How does it work? (just roughly, you don’t have to get too technical, exploit analogies )
  5. How is it evolving?
  6. How is this technology changing us (as an individual, as a society)?
  7. How could we change/adapt it?
  8. What are the pros and cons?
  9. How to evaluate/measure it?

This is a good set of questions to remember before taking action, don’t you agree?

photo by: x-ray delta one