According to over 380 enterprise leaders, a third of enterprises are aggressively pushing for increased adoption of AI technologies in their business operations. However, it is critical to design a method for developing algorithms in real-time to mitigate bias, in order to achieve the desired goals of the AI strategy.

Exhibit 1: 31% of enterprises are placing significant investment in AI

How much investment or focus is your organization making in the following in the next year to help you achieve operational cost saving goals? (top five with significant or some investment focus)

Source: HFS Research in conjunction with KPMG, State of Operations and Outsourcing 2018. Sample: Global 2000 Enterprise Buyers = 381

It’s not hard to see how bias in insurance, healthcare, and justice system AI solutions could cause detrimental effects to society. So, enterprise leaders looking to implement AI, must ensure they recognize the potential consequences. Also, they should adopt these technologies, which they may not fully understand, at a pace that factors in any additional required safeguards.

From the mainstream media to dystopian sci-fi films—we already know bias in technology is damaging. But how do we fix it?

The crux of the problem is that if bias goes uncorrected, AI tools will present a false caricature of the world, rather than an accurate and useful representation. For example, researchers from the University of Washington demonstrated how unintended correlations could creep into training data and cause errors in results. In this case, an algorithm designed to differentiate huskies from wolves classified a series of huskies as wolves simply because there was snow in the background. Bias had crept into the data set because all training pictures of wolves were set with snowy backgrounds; the algorithm used this characteristic of the data instead of others to make its determination (see Exhibit 1). This unconscious bias reveals just how quickly correlation can pull at the wrong strings. It’s not difficult to imagine scenarios where the ramifications could be more extensive.

Exhibit 2: Explaining the prediction of a classifier of wolves

Source: Ribeiro, Singh and Guestrin 2016. ACM SIGKDD international conference on knowledge discovery and data mining 

These biases, defined as systematic errors by IBM Researchers, can be difficult to detect simply by looking at the results and data. In the husky vs wolf example, less than half of the machine learning students guessed that snow was a potential determiner when they were shown picture (a). Robotics researcher Peter Haas highlighted the need for explainable AI, (see our recent article on XAI), as even developers may be unaware of the reason behind an algorithm’s decision. Simply put, business leaders must wake up to the looming threat of bias; already, consumers, regulatory bodies, and journalists are taking note and are ready to hold companies to account.

Investigative journalists, for example, are already holding companies accountable for prejudice in their algorithms. ProPublica, a nonprofit newsroom that aims to produce investigative journalism in the public interest, recently reported on a software suite called COMPAS developed by Northpointe (now Equivant), a software firm focused on the justice industry. This tool is commonly used in the US justice system to determine sentences and set bail. The software labeled African-American defendants as future criminals at almost twice the rate as white defendants. Even though race was not one of the 137 questions used in the assessment, ProPublica found that some of the questions had an ethnic affinity—data such as the ZIP Code of the individual’s address. Outdated court rulings are also biasing the judgement. For example, ‘Was one of your parents ever sent to jail or prison?’.  Many of these historic decisions were made during times of greater racially disparate treatment, but were nevertheless fed into the technology to help make future decisions. Data that already holds biases fueled by human beliefs and prejudices, are influencing the system’s decisions.

Data hygiene isn’t a “nice to have”—it must be mandatory for every company taking a serious look at AI

An important step to tackling this problem is to clean and regularly update training data. Algorithms are getting better at training on less data, but also synthetic data created by these smaller samples. Platforms such as IBM Watson now require  smaller samples to fuel the decision-making engine. Enterprise leaders now have less justification to use outdated historical data, which may hold greater bias and prejudice. IBM Watson reported that it is “the quality of your data, not the quantity, (that) makes the difference.” But, in cases where there is little or no data that doesn't include some form of prejudice, several companies are rethinking the problem of bias. For example, Microsoft Research teamed with Boston University to remove gender bias in word embeddings (e.g., which words are closer to “he” than to “she”). They worked on the neural network, Word2vec, and trained the tool on words from Google News articles, using their “hard de-biasing algorithm” to adjust for bias. Crucially, they claimed they were able to do this while maintaining the overall structure.

Adjusting for bias while holding on to the insight the data is revealing is an important step; equalizing or neutralizing associations in data can reduce the value organizations can derive from it considerably. In this instance, it could unhelpfully remove important distinctions and gender definition words, such as “sister.” But evidence suggests that the hard debiasing algorithm was successful in neutralizing bias while retaining the integrity of the data. For example, the system reduced gender stereotypes, such as “he” to “doctor” and “she” to “nurse,” while preserving gender-appropriate pairings such as “she” to “ovarian cancer” and “he” to “prostate cancer.”

Several companies are leading the way with AI solutions to combat bias

Several companies that recognize the major consequences and potential economic opportunities of AI bias are leading the way with tools to mitigate the prejudice. In many cases, these solutions are spawned from the companies’ internal desire to correct for bias before it negatively impacts them.

• On May 2, 2018, Facebook announced that its ethics team was testing a tool called Fairness Flow as an internal project. Supposedly the tool can automatically warn if an algorithm is making an unfair judgment about someone based on their gender, race, or age. The tool is still in its early stages of development. Shortly after, Microsoft announced that its Fairness Accountability Transparency and Ethics group was developing a tool that could help businesses make use of AI while avoiding discriminating against certain groups of people.
  
  
• In June 2018, Accenture led the way in developing an AI fairness tool for organizations. This tool uses statistical methods to investigate and control for the correlations between sensitive variables, such as age and gender. Rumman Chowdhury, Accenture’s responsible AI lead, admitted that there could be an accuracy cost to correcting bias in the AI model (see Exhibit 2). She recommended that in cases where you see a wide shift in model accuracy, you may need to go back and get new data.
  
  

Exhibit 3: Accenture’s AI Fairness Tool

Source: Accenture

• In September 2018, IBM released AI Fairness 360, an open-source Python package that includes tools and examples to help identify and mitigate bias. These tools include algorithms for detecting and reducing unwanted bias in both training data and models (see Exhibit 3). It is then possible to compare the original results with those that have had the bias mitigated.
  
  

Exhibit 4: IBM’s AI Fairness 360 (open-source toolkit for mitigating bias through the AI lifecycle)

Source: IBM

Bottom line: With almost a third of enterprises are investing in AI; they must ensure bias-mitigation forms a central part of their decision making

Enterprises will need to ensure they have safeguards in place to detect bias and systematically correct to reduce the risk of contamination. They must also ensure they are working with partners that take the ethics of AI seriously, given the financial and reputational exposure this could bring. This may be particularly pertinent for companies who handle personally identifiable information (PII). The risk of leaving bias in your AI system is not just bad publicity; it has a deeper, long-term impact on customer trust. At the extreme, you may even risk a lawsuit, especially if the AI algorithms are being used to make life-changing decisions for people and particularly if you fall foul of the increasing amount of privacy legislation.

To overcome the impact of bias in AI systems, the modern enterprise must:

• Perform extensive data cleansing activities: In many cases, the root of bias in AI systems sits in the data that feeds it. Enterprises must make sure their data is clean and fit for purpose.
  
  
• Train algorithms with varied data sets: Fueling algorithms with restricted data sets will focus the model too much on particular characteristics and will inevitably build a bias that misrepresents the environment. Enterprise leaders must use varied data sets to ensure AI tools paint a credible picture of the world.
  
  
• Investigate de-biasing tools: The range of tools and expertise is growing rapidly; enterprise leaders must bring in the brains and brawn they need to mitigate bias in their platforms using the raft of solutions in development and already available in the market.