04 January 2017

Artificial intelligence has taken the public imagination by storm in the last few years. Consumer-facing technology has seen a wide range of new products--from in-home assistants to self-driving car technology--that show us a glimpse of the potential for AI technology. The adoption of AI in the enterprise, however, is an entirely different story.

Although AI can unleash revenue opportunities or cost savings within an enterprise, its benefits are both domain-specific and company-specific. In the near term, it is up to each enterprise how it adopts AI within its businesses. To progress towards the full potential of AI, however, enterprises must have a few things in place. I will outline what these ingredients are in order.

Crawl: Set Up a Cloud Computing Infrastructure

The first step towards achieving the full potential of AI within the enterprise is a robust cloud computing infrastructure. Specifically, enterprises must achieve the capability to spin up dynamic workloads on demand, as well as the capability to track economics of compute and storage at a fine-grained level. Although these capabilities are available as turnkey features in public clouds today, enterprises should still think about how to adapt these workloads to their businesses. In the longer-term, these enterprises will need to transform their businesses to be cloud-native rather than just part-time users of public clouds. Such a transformation will enable enterprises to spin up agile analytic and compute workloads wherever data resides.

Walk: Build An Enterprise-Wide Data Lake

The next step towards realizing the full potential of AI within the enterprise is a data lake. Enterprises believe in the value of data-driven decisions and have begun to construct data repositories that contain a wide variety of data. For enterprise AI, however, data infrastructure needs to really go toe-to-toe with agile compute infrastructure in the form of an enterprise-wide data lake. Due to requirements of processing speed and data residency, an enterprise-wide data lake is usually a logical construct rather than a physical repository. So as not to turn their data lakes into data swamps, enterprises must design effective retention policies. In addition, the contents of data lakes must be introspectable enough that curation and data prep capabilities can be executed by human analysts or algorithms.

data exhaust can be the means to convert the data lake from a cost center into a strategic asset.
Share this

Enterprises should also pay attention to "data exhaust", or byproducts of data ordinarily produced by business processes, e.g. log files. Usually overlooked by enterprises at first blush, data exhaust can end up providing insights, e.g. those related to customer experience, that enterprises may have otherwise derived only from purchased datasets. In this way, data exhaust can be the means to convert the data lake from a cost center into a strategic asset.

Run: Use Algorithms Well

Machine learning algorithms are the workhorses that are essential to realizing any vision of AI within the enterprise. However, even the best algorithms are setup to fail if they don't operate on good data. Supervised machine learning techniques need a corpus of training data that is high-quality and free of bias. Training datasets should be large enough to minimize risks of overfitting and refreshed periodically with new data from real business processes. Once deployed into production, machine learning models often need to be recalibrated periodically to stay in sync with reality. Otherwise, the predictions made by algorithms can end up diverging from actual business conditions.

Most enterprises start out by building and deploying machine learning models within specific domains, e.g. fraud detection. If multiple machine learning algorithms are relevant to a domain, emerging techniques such as ensembling, reinforcement learning or transfer learning may enable algorithms to build on one another’s strengths. Reinforcement learning sets up a well-defined reward framework for multiple algorithms to maximize learning within a domain. Transfer learning occurs when training and test data for an algorithm are from different domains. Sometimes, constraints of a domain make it impracticable to collect high quality training data; in such cases, transfer learning from a closely related domain can help bootstrap algorithms that can subsequently be tuned to the target domain.

AI in the economy will far surpass that brought about by similar shifts in the past, such as the Industrial Revolution.
Share this

Fly: Set Up The Right Processes

Machine learning requires a fundamentally new model for thinking about a problem than past flavors of programming. In previous models of programming, algorithms were handcrafted to "work forward" and convert known inputs into as-yet-unknown outputs. Machine learning is a paradigm where inputs and outputs are known in advance, and the algorithm "works backward" from outputs to learn the mapping between them. This paradigm is a poor fit for conventional "feed-forward" processes from the past, such as the software development and data analysis lifecycles.

If enterprises don't set up the right governance structures for this new paradigm, they can end up creating black boxes--machine learning algorithms whose behavior cannot be explained from first principles. In industries such as financial services and healthcare, this lack of explainability can even be a regulatory issue. Although there isn't an easy general solution to this problem, enterprises should think hard about the governance and ethics issues that arise from black box algorithms. Enterprises should also learn to look beyond the hype and learn how to pick the right machine learning algorithm for the job. For instance, although deep learning is making startling breakthroughs, older techniques such as random forests have a much better performance record in real-world data competitions.

Finally, as artificial intelligence spreads through their businesses, enterprises should think about where algorithms can augment human-mediated business processes and where they may be able to fully automate them. While there is considerable anxiety that artificial intelligence may automate millions of jobs away, very few business processes can be fully automated away today. For every instance where AI-driven automation may reduce the role of a human in the loop, enterprises may well discover that those workers can be put to work on higher-value processes that haven't been discovered yet.


Artificial intelligence is not just the new kid on the technology block; rather it represents a fundamentally new platform for the future of business. At scale, the transformation brought about by AI in the economy will far surpass that brought about by similar shifts in the past, such as the Industrial Revolution. Despite short-term disruptions, humankind has dealt successfully with such shifts in the past. While not all enterprises will adopt AI technologies to the same degree or in the same manner, examining what they need to do to thrive in this new economic world order simply makes business sense.

Tap to read full article