Big Data workshop at the First European Celebration of Women in Computing

ECWCThis last Tuesday, I lead the ‘Discover Big Data’ workshop at the First European Celebration of Women in Computing.  There were many parallel sessions that morning and I received some questions about my presentation from the participants that couldn’t divide themselves to attend this workshop 😉

Welcome to the Big Data workshop, we need women in Big Data!

This workshop is called ‘Discover Big Data’ because Big Data is a hyped word. It is being used for anything where data is involved, but it still remains confusing as what it means.

  • You are also in Big Data  if you are dealing with data that has to be processed at great velocity, as is the case for GPS or for mobile phones.
  • You are in Big Data if you cross information that come on a variety of formats, like your customer’s transactions and your customer’s emails, or if you go to the social networks, like Facebook or  Twitter.  You can discover what are the topics being discussed, what is being said about your company or  who is talking about your product.
  • You are in Big Data  if you’re exploiting one of the many big available datasets like weather information, official administration records like property records or  financial information, economic indicators…

What can be done with Big Data?

It is mainly used for customer intimacy, discovering your customer profiles and target them on a one to one base. Finding their preferences and the hidden patterns to predict customer churn.

It can be used for optimisation, finding patterns of systematic problems hidden in your historical data. It can help for organising your maintenance, or to improve the supply-chain, finding better logistic solutions, optimise processes.

It is also used for innovation: It can help you create your new product. Looking at your competitors and finding the white-spaces or uncovering market trends.

And more generally, with all the available data you can create models forecasting future events and behaviors. Through what-if analysis to predict the outcomes of potential changes, you can direct your business strategy. It helps anticipating previously unforeseen opportunities, as well as avoiding costly situations, finding new revenue opportunities or identifying more effective business models.

As you see, there are great business opportunities!

How can we do all that?

There are many techniques like statistical analysis, data mining, text analysis, sentiment analysis, graph analysis, machine learning, predictive analysis, neural networks, conceptual clustering…

You may have heard already some of those words that sound promising but that also sound very complicated. And even so, the Big Data field is growing exponentially as men are running for it.  There are only 10% of women, don’t you want to be part of it? Companies that took this wave are thriving, well ahead of classical business. They are proposing you the right product at the right time, with the features you are looking for, for the price you are willing to pay. They are  increasing their profits while shaping our future with the products and business strategies they are creating.

I hear you saying: This is great but I don’t know a thing about this and it sounds so complicated. I’m here to tell you that not all of it is that difficult.

YOU could be in Big Data.

If you are in computing you have a leg up. And if you like mathematics you’ll enjoy being a data scientist. But you could be in Big Data even if you are not a techy person.  If you are in HR, in marketing, if you are a manager or a decision-maker with the right mindset open to data, you can exploit the Big Data wave.

Even if you see the potential, women tend to think ‘it’s not for me, I don’t have the competencies’.

Let me use some feminine stereotypes to illustrate we have the basic skills:

  • We have a tradition of getting together and talking too much.  And we have a tendency to be matchmakers.  We can put those skills of information gathering and making connections to good use finding relationships between data.
  • Who recognises herself in this? We are control freaks and plan everything, even the time of our loved ones.  Don’t you have a TODO list for your partner on Saturdays?  I do: Love, since you are driving Alex to the scouts, could you please pass by and drop the trousers at the dry cleaner?  What if you knew what your GPS knows already, that a road is blocked?  You could have asked him to bring some bread back as he’s going to pass near the bakery.  Don’t you feel satisfaction when doing things efficiently, optimising the Saturday time? So imagine tapping into all the available information and using it to improve the processes, it’s a rewarding job.
  • And if you have artistic skills, visualisation is your field. This is a new branch of data science, they are creating new techniques very interesting to show more than 3 dimensions of data, so you can see easily relationships graphically.
  • Generally speaking, I think we women have a natural talent to be data analysts: the ‘What if’ comes natural to us, we always investigate all possibilities before deciding for one, isn’t it?

Summarising, we saw there is business in here, and that we have the basic skills to be in the Data business.

Moreover, it is important that more women move into this field, not only because of the many business opportunities, but also because there are ethical issues involved in Big Data. We can mention data privacy and price gauging as some of these issues, but there are other business models that can be controversial.

The rules of what can be done with the data and what is off-limits, are being defined right now.  Let’s not miss the opportunity to give our view on this.

As an example, there is a great initiative from the Data2X program of the UN, who’s doing a research on women’s freedom of movements through satellite images and their phone geolocation.  Are they limited in their movements in some countries, do they have access to education, to health care? Great initiative, but what about the same at a private level: is following the movement of your partner with her/his phone geolocation ethical? What about tracking the movement of your children, as it’s done already in some countries?

It’s important to have our saying in the ethical uses of all those lakes of data and be represented in the decisions that will define our future society. We, women, have a natural tendency of looking after our loved ones, taking their needs in consideration. That’s what Big Data is needing, people that set the rules for using the incredible amounts of data, taking into account the different perspectives and with a long term view in mind. It’s the moment to use our feminine voice to shape a better society for all of us, participating also in the creation of the new business models.

In this workshop you will hear success stories to show you the opportunities to be included in this field. I hope you’ll join the Big Data movement.

The rise of the Self-Tuning Enterprise

Alibaba

As you may know, I am a fan of Machine Learning, a subfield of Artificial Intelligence (AI) that englobes computer programs that exhibit some kind of intelligent behavior. The first researchers on AI began analyzing how we (humans) did intelligent tasks in order to create programs that reproduced our behavior. So look at the irony of this HBR article”The self-Tuning Enterprise” where the authors Martin Reeves, Ming Zeng and Amin Venjara use the analogy of how machine learning programs do to transpose the behavior to enterprise strategy tuning:

[…] These enterprises [he’s talking about internet companies like Google, Netflix, Amazon, and Alibaba] have become extraordinarily good at automatically retooling their offerings for millions of individual customers, leveraging real-time data on their behavior. Those constant updates are, in fact, driven by algorithms, but the processes and technologies underlying the algorithms aren’t magic: It’s possible to pull them apart, see how they operate, and use that know-how in other settings. And that’s just what some of those same companies have started to do.

In this article we’ll look first at how self-tuning algorithms are able to learn and adjust so effectively in complex, dynamic environments. Then we’ll examine how some organizations are applying self-tuning across their enterprises, using the Chinese e-commerce giant Alibaba as a case example.”

You may have notice those new programs at work to recommend you books or other products each time you buy something on Internet (and in fact, even if you are just looking and didn’t buy anything ;-). Those programs are based on Machine Learning algorithms, and they improve over time with the new information of success (if you bought the proposed article) or failure (if you didn’t).

How do they work?

There is a ‘learning’ part that finds similarities between customers in order to propose you products that another customer similar to you bought. But it’s not so simple, these programs are coupled with other learning modules like the one that does some ‘experimentation’ not to get stuck with always the same kind of products. This module will propose you something different from time to time. Even if you like polar books, after the tenth one, you would like to read something else, isn’t it? So the trick is to find equilibrium between showing you books you have great chances to like and novelties to make you discover new horizons. You have to have the feeling that they know what they are doing when they propose you a book (so they fine-tune to be good at similarities) but you may like to change from time to time not to get bored, and also they are very interested in making you discover another bounty/category of literature, let’s say poems. If you don’t like it, you won’t accept so easily next recommendation, so here comes the next ‘tuning’ on how often to do it.

And that’s where self-tuning comes in. Self-tuning is related to the concepts of agility (rapid adjustment), adaptation (learning through trial and error), and ambidexterity (balancing exploration and exploitation). Self-tuning algorithms incorporate elements of all three—but in a self-directed fashion.

The ‘self-tuning’ process they are talking about adjusts the tool to the new information available to him without the need of reprogramming. The analogy the authors are doing is to do in organizations this same kind of automatics tunings that Machine Learning systems are doing: to ‘self-tune’ the companies without any top-down directive, to have agility, adaptation through trial and error and ambidexterity balancing exploration and exploitation.

To understand how this works, think of the enterprise as a nested set of strategic processes. At the highest level, the vision articulates the direction and ambition of the firm as a whole. As a means to achieving the vision, a company deploys business models and strategies that bring together capabilities and assets to create advantageous positions. And it uses organizational structure, information systems, and culture to facilitate the effective operation of those business models and strategies.

In the vast majority of organizations, the vision and the business model are fixed axes around which the entire enterprise revolves. They are often worked out by company founders and, once proven successful, rarely altered. Consequently, the structure, systems, processes, and culture that support them also remain static for long periods. Experimentation and innovation focus mostly on product or service offerings within the existing model, as the company leans on its established recipe for success in other areas.

The self-tuning enterprise, in contrast, takes an evolutionary approach at all levels. The vision, business model, and supporting components are regularly calibrated to the changing environment by applying the three learning loops. The organization is no longer viewed as a fixed means of transmitting intentions from above but, rather, as a network that shifts and develops in response to external feedback. To see what this means in practice, let’s look at Alibaba.[…]

Keep resetting the vision.

When Alibaba began operations, internet penetration in China was less than 1%. While most expected that figure to grow, it was difficult to predict the nature and shape of that growth. So Alibaba took an experimental approach: At any given time, its vision would be the best working assumption about the future. As the market evolved, the company’s leaders reevaluated the vision, checking their hypotheses against reality and revising them as appropriate.

In the early years, Alibaba’s goal was to be “an e-commerce company serving China’s small exporting companies.” This led to an initial focus on Alibaba.com, which created a platform for international sales. However, when the market changed, so did the vision. As Chinese domestic consumption exploded, Alibaba saw an opportunity to expand its offering to consumers. Accordingly, it launched the online marketplace Taobao in 2003. Soon Alibaba realized that Chinese consumers needed more than just a site for buying and selling goods. They needed greater confidence in internet business—for example, to be sure that online payments were safe. So in 2004, Alibaba created Alipay, an online payment service. […] Ultimately, this led Alibaba to change its vision again, in 2008, to fostering “the development of an e-commerce ecosystem in China.” It started to offer more infrastructure services, such as a cloud computing platform, microfinancing, and a smart logistics platform. More recently, Alibaba recalibrated that vision in response to the rapid convergence between digital and physical channels. Deliberately dropping the “e” from e-commerce, its current vision statement reads simply, “We aim to build the future infrastructure of commerce.”

Experiment with business models.

Alibaba could not have built a portfolio of companies that spanned virtually the entire digital spectrum without making a commitment to business model experimentation from very early on.

[…]At each juncture in its evolution, Alibaba continued to generate new business model options, letting them run as separate units. After testing them, it would scale up the most promising ones and close down or reabsorb those that were less promising.[…]

Again there was heated debate within the company about which direction to take and which model to build. Instead of relying on a top-down decision, Alibaba chose to place multiple bets and let the market pick the winners.[…]

Increasing experimentation at the height of success runs contrary to established managerial wisdom, but for Alibaba it was necessary to avoid rigidity and create options. Recalibrating how and how much to experiment was fundamental to its ability to capitalize on nascent market trends.

Focus on seizing and shaping strategic opportunities, not on executing plans.

In volatile environments, plans can quickly become out-of-date. In Alibaba’s case, rapid advances in technology, shifting consumer expectations in China and beyond, and regulatory uncertainty made it difficult to predict the future. […]

Alibaba does have a regular planning cycle, in which business unit leaders and the executive management team iterate on plans in the fourth quarter of each year. However, it’s understood that this is only a starting point. Whenever a unit leader sees a significant market change or a new opportunity, he or she can initiate a “co-creation” process, in which employees, including senior business leaders and lead implementers, develop new directions for the business directly with customers.

At Alibaba co-creation involves four steps. The first is establishing common ground: identifying signals of change (based on data from the market and insights from customers or staff) and ensuring that the right people are present and set up to work together. This typically happens at a full-day working session. The second step is getting to know the customer. Now participants explore directly with customers their evolving needs or pain points and brainstorm potential solutions. The third step entails developing an action plan based on the outcome of customer discussions. An action plan must identify a leader who can champion the opportunity, the supporting team (or teams) that will put the ideas into motion, and the mechanisms that will enable the work to get done. The final step is gathering regular customer feedback as the plan is implemented, which can, in turn, trigger further iterations.

So now you know how Alibaba does it, how is it in your company?  What ideas from them would you adopt?

Testing tool for Big Ideas

For all of you who have an idea but are not sure it will work, or for the ones that are juggling with many ideas and never concretize any 🙂 here is a tool that could help you decide if to go further or not: the Pimento Map

 

Pimentochart

[..] the Pimento Map methodology is a fast, easy and accurate way to evaluate the chances of success of your business model. It gives the opportunity to entrepreneurs, business angels or venture capital firms to build an objective opinion on a new business idea.  It also points out in detail where the model can be improved.

The tool was presented on the Tech Startup Day last week, it’s easy of use, the system asks you to answer some questions around a factor, and then shows that slice of pie in Green, Yellow, Orange or Red.  Guess what color you should wish to have?  Yes, green or yellow are ok, if you get the other ones, you found the weak factor of your idea.

If you are afraid of letting Pimento know about it (yes, the question has been clearly asked), as they said: you are the one filling the description 🙂

So take action, test you Big Idea … and fine tune it if it needs it. I wish you a big success!

MOOCs: the new learning style

Last week I presented MOOCs (Massive Open Online Courses) at the Professional Women International association in Brussels, Belgium.

I had the pleasure of talking to the participants afterwards.  They told me they were so pleased to learn they had such an easy way of taking good quality courses that they were going to check that same night for their preferred subjects 🙂

Happy to have contributed to spread the word about the availability of the MOOCs, putting all their encapsulated knowledge encapsulated at any user’s fingertips!

On the last slide, I just dropped words  with the main implications of this trend;  I encourage you to put a comment if any of the subjects I mention resonates with you:

Alex Pentland’s article on Data-Driven Society

I recently got the new issue from Scientific American (October 2013), and in the front page was announced the article ‘The Data-Driven Society’ by Alex Pentland.  I just had to read it 🙂

He co-leads the World Economic Forum on Big Data and Personal Data initiatives.  He was talking about all the digital bread crumbs we leave behind on our daily life (like gps and gsm info, or electronic payments) and what can be done with it.

With his students of the MIT Human Dynamics Laboratory, he is discovering mathematical patterns through data analytics that can predict human behaviour. ‘Bread crumbs record our behaviors as it really happens’ he says, it is more accurate than the information from social media, where we choose what we want to disclose from ourselves.  Alex and his team are in particular interested in the patterns of idea flows.

Among the most surprising findings that my students and I have discovered is that patterns of idea flow (measured by purchasing behavior, physical mobility or communications) are directly related to productivity growth and creative output.

Analysing those flows, he uncovered 2 factors that have a positive pattern of healthy idea flow:

  • engagement: connecting to others, usually in the same team or organisation, and
  • exploration: going abroad to exchange ideas.

Both are needed for creativity and innovation to flourish.  To find those factors, he based his research on graphs of different types of interactions, like person-to-person, emails, sms..

We may not have the tools he used (like an electronic badges for tracking person-to-person interactions) but intuitively this is something we know, a good communication is essential for the success of a team, but talking to an external person may provide a new insight.  It’s always good to be proved right, isn’t it?

Check my next post, I’ll continue with his article, there are a lot of great concepts he is presenting as the ‘new deal on data’ for personal data protection.

 

Massive Open Online Courses

Massive Open Online Courses (MOOC) are very recent, but are quickly gaining popularity.  Coursera is one of the big platforms that offer those free courses, along with edX and Khanacademy just to mention a few.  Last year I took a fantastic course offered by Coursera  called ‘Model Thinking’ given  by Prof. Scott E. Page, who’s the Director of the Center for the Study of Complex Systems at the University of Michigan ( I posted already about it here : – ).

In March this year, I was glad to receive a mail from Scott Page, giving us some feedback from his experience doing this course, and sending us also a link to a presentation he did about the making of the course.

To give you an idea of the popularity of this course, there were 60.000 students enrolled on the first run of Model Thinking, beginning of 2012.  It grew to 100.000 for the fall run (by the way, if you are interested there will be a new run this fall 2013, and it may be the last one, says Prof. Page).

I would like to share with you Scott’s insights on his experience on making this online course contrasting it with the making of his online course ‘The hidden Factor’.  This last one was professionally done in a studio and he called ‘Model Thinking’: my garage band online course : – )

In fact, it was really recorded in one unused room of his house, because he said that the starting and stopping of the heating system in the rest of the house was picked up by his mike, so sensible it was even though it was just a $100 one.

To prepare the course, he thought of making it more modular.  So he cut it in small chunks, so that each video was independent, and treated a subject in no more than 15 minutes.  But as he said, that was the easiest part because what took him much more time was the recording of each lecture.  One big issue he had was that he was alone in this room to do the recordings, and trying to be smiling, engaging and enthusiastic is difficult without an audience.  Not only that, but he had unforeseen events from time to time, like his dog wandering around, and he laughed and found himself doing funny movement to chase him.

The editing took a lot of time, each video had to be reviewed, and in case of errors, it was difficult to fix it.  So at the end, some mistakes remained.   On the other hand in the professional approach, they took care of each error, but they had better tools and a battery of technicians to look into them and find different alternatives to correct them.  Sometimes he had to repeat one word they detected he had staggered with, and they told them even the intonation he had to use to repeat it; sometimes they just put a picture about the subject he was talking about, and he could rephrase one sentence.

In conclusion, here’s his comparison regarding costs to do the 2 videos:

 

So it is much more costly for a professional quality. Time-wise, it was surprisingly more or less equivalent:

 

The studio made video was undisputable better, being much easier to correct any mistakes:

 

 

But in the end, is the improvement in quality worth the cost?  Not really he says; the best quality is not needed, a good enough approach is better, even more if the cost prohibits its making.  So the best solution stands between those 2 options.

I found also very important his comment on how presenting this course changed his everyday work life.  He has now 1 hour per day reading his mail, answering to diverse requests on his subject of expertise.  He receives inquiries from technical advisors, deans, diverse influencial people that he cannot really discard.  On the one hand it’s not strictly his job, for what he is paid for, but on the other hand, can these requests be ignored? Is it responsible if you know your intervention can have such an impact as to do better policies, to improve many people’s life?

Snowden showed us the dangers of Big Data with PRISM, are we up to the challenge to steer its use?

A television screen shows former U.S. spy agency contractor Edward Snowden during a news bulletin at a cafe at Moscow’s Sheremetyevo airport June 26, 2013. Credit: Reuters/Sergei Karpukhin

 

As we already discussed on my Big Data presentations,   being able to analyse the amount of data that traces all our actions and movements is a great opportunity to improve our lives, as much as to do business, but it can also be exploited for the worst.  Now Edward Snowden has put a clear case under the spotlights, will this make us move? Will this lead to change?

It’s time to consider what ethical codes and regulations can be issued, so that this excellent opportunity that technology is putting in our hands, that is sharing, measuring and extracting knowledge from all aspects of our lives, is not misused.

Small talk on Big Data

Last week I presented this topic to professional women at PWI here in Brussels. It’s called ‘small talk’ because it is not a technical presentation but one for a broader audience, to create awareness on this Big Data trend.   The main concept I wanted them to take away is the change in the business arena and in our society due to Big Data. If you are interested on this subject, just drop a line and let me know!

Prices of discs and storage devices have dropped a lot, so now basically any digital data is being stored.  Cost is so low, that it is worth to save it ‘just in case, and we’ll see in the future what we can do with this data’.  Technology has made also  huge advances with massive parallel processing, and we can manage to jungle through thousands of servers to analyse a bunch of diverse data and extract information from it in a usable time-frame.

This allows business strategists to make smarter decisions based on facts, better than how it was done before, based on experience or intuition.  So the message for all decision-makers is: go and check your data, you’ll find there valuable information to decide any business matter.  Also, be aware that your competition is going into it too, it can out-smart you!

At the society level, there are many ethical issues to deal with, like privacy or equality and fairness.  What to you think, is it fair to have a subsidy that is ‘personalised’, that may give more to someone than to others because of a particular factor, or allow access to a health treatment to someone and not to another based on his life expectancy for example?  What about basing the decision on his ‘ROI’  like the capability of paying back for the given  treatment?  Or is it more fair to have instead equality on subsidies, same amount for everyone? Even for the ones that could pay it by themselves? Either we discuss them before-hand, or we will be at the mercy of any politician or entrepreneur taking a step deeper in an unethical direction.

And as a last twist, I would like to point out that the basic value of knowledge is challenged.  We are already experiencing a change of values, knowledge is less and less valued as an asset anymore, but value remains in knowing how to get to the knowledge,where to find it and what to extract from data.