Artificial Intelligence (AI) vs. Machine Learning vs. Deep Learning

Artificial intelligence (AI), machine learning and deep learning are three terms often used interchangeably to describe software that behaves intelligently. However, it is useful to understand the key distinctions among them.

You can think of deep learning, machine learning and artificial intelligence as a set of Russian dolls nested within each other, beginning with the smallest and working out. Deep learning is a subset of machine learning, and machine learning is a subset of AI, which is an umbrella term for any computer program that does something smart. In other words, all machine learning is AI, but not all AI is machine learning, and so forth.

Learn How to Apply AI to Simulations »

Artificial Intelligence, Symbolic AI and GOFAI

John McCarthy, widely recognized as one of the godfathers of AI, defined it as “the science and engineering of making intelligent machines.”

Here are a few other definitions of artificial intelligence:

• A branch of computer science dealing with the simulation of intelligent behavior in computers.
• The capability of a machine to imitate intelligent human behavior.
• A computer system able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

There are a lot of ways to simulate human intelligence, and some methods are more intelligent than others.

AI can be a pile of if-then statements, or a complex statistical model mapping raw sensory data to symbolic categories. The if-then statements are simply rules explicitly programmed by a human hand. Taken together, these if-then statements are sometimes called rules engines, expert systems, knowledge graphs or symbolic AI. Collectively, these are known as Good, Old-Fashioned AI (GOFAI).

The intelligence that rules engines mimic could be that of an accountant with knowledge of the tax code, who takes information you feed it, runs the information through a set of static rules, and gives your the amount of taxes you owe as a result. In the US, we call that TurboTax.

Usually, when a computer program designed by AI researchers actually succeeds at something – like winning at chess – many people say it’s “not really intelligent”, because the algorithm’s internals are well understood. The critics think intelligence must be something intangible, and exclusively human. A wag would say that true AI is whatever computers can’t do yet.

Machine Learning: Programs That Alter Themselves

Machine learning is a subset of AI. That is, all machine learning counts as AI, but not all AI counts as machine learning. For example, symbolic logic – rules engines, expert systems and knowledge graphs – could all be described as AI, and none of them are machine learning.

One aspect that separates machine learning from the knowledge graphs and expert systems is its ability to modify itself when exposed to more data; i.e. machine learning is dynamic and does not require human intervention to make certain changes. That makes it less brittle, and less reliant on human experts.

In 1959, Arthur Samuel, one of the pioneers of machine learning, defined machine learning as a “field of study that gives computers the ability to learn without being explicitly programmed.” That is, machine-learning programs have not been explicitly entered into a computer, like the if-then statements above. Machine-learning programs, in a sense, adjust themselves in response to the data they’re exposed to (like a child that is born knowing nothing adjusts its understanding of the world in response to experience).

Samuel taught a computer program to play checkers. His goal was to teach it to play checkers better than himself, which is obviously not something he could program explicitly. He succeeded, and in 1962 his program beat the checkers champion of the state of Connecticut.

The “learning” part of machine learning means that ML algorithms attempt to optimize along a certain dimension; i.e. they usually try to minimize error or maximize the likelihood of their predictions being true. This has three names: an error function, a loss function, or an objective function, because the algorithm has an objective… When someone says they are working with a machine-learning algorithm, you can get to the gist of its value by asking: What’s the objective function?

How does one minimize error? Well, one way is to build a framework that multiplies inputs in order to make guesses as to the inputs’ nature. Different outputs/guesses are the product of the inputs and the algorithm. Usually, the initial guesses are quite wrong, and if you are lucky enough to have ground-truth labels pertaining to the input, you can measure how wrong your guesses are by contrasting them with the truth, and then use that error to modify your algorithm. That’s what neural networks do. They keep on measuring the error and modifying their parameters until they can’t achieve any less error.

They are, in short, an optimization algorithm. If you tune them right, they minimize their error by guessing and guessing and guessing again.

Deep Learning: More Accuracy, More Math & More Compute

Deep learning is a subset of machine learning. Usually, when people use the term deep learning, they are referring to deep artificial neural networks.

Deep artificial neural networks are a set of algorithms that have set new records in accuracy for many important problems, such as image recognition, sound recognition, recommender systems, natural language processing etc. For example, deep learning is part of DeepMind’s well-known AlphaGo algorithm, which beat the former world champion Lee Sedol at Go in early 2016, and the current world champion Ke Jie in early 2017. A more complete explanation of neural works is here.

Deep is a technical term. It refers to the number of layers in a neural network. A shallow network has one so-called hidden layer, and a deep network has more than one. Multiple hidden layers allow deep neural networks to learn features of the data in a so-called feature hierarchy, because simple features (e.g. two pixels) recombine from one layer to the next, to form more complex features (e.g. a line). Nets with many layers pass input data (features) through more mathematical operations than nets with few layers, and are therefore more computationally intensive to train. Computational intensivity is one of the hallmarks of deep learning, and it is one reason why a new kind of chip call GPUs are in demand to train deep-learning models.

So you could apply the same definition to deep learning that Arthur Samuel did to machine learning – a “field of study that gives computers the ability to learn without being explicitly programmed” – while adding that it tends to result in higher accuracy, require more hardware or training time, and perform exceptionally well on machine perception tasks that involved unstructured data such as blobs of pixels or text.

Reinforcement Learning: A Framework in which DL is Embedded

Reinforcement learning is goal-oriented learning. That is, rather than trying to classify or cluster data, you define what you want to achieve, which metrics you want to maximize or minimize, and RL agents learn how to do that. It is not mutually exclusive with deep learning, but rather a framework in which neural networks can be used to learn the relationship between actions and their rewards. Combined, this is called deep reinforcement learning, which DeepMind trained successfully on the game of Go, numerous video games, and harder problems in real life.

What’s Next for AI?

The advances made by researchers at DeepMind, Google Brain, OpenAI and various universities are accelerating. AI is capable of solving harder and harder problems better than humans can.

This means that AI is changing faster than its history can be written, so predictions about its future quickly become obsolete as well. Are we chasing a breakthrough like nuclear fission (possible), or are attempts to wring intelligence from silicon more like trying to turn lead into gold?¹

There are four main schools of thought, or churches of belief if you will, that group together how people talk about AI.

Those who believe that AI progress will continue apace tend to think a lot about strong AI, and whether or not it is good for humanity. Among those who forecast continued progress, one camp emphasizes the benefits of more intelligent software, which may save humanity from its current stupidities; the other camp worries about the existential risk of a superintelligence.

Given that the power of AI progresses hand in hand with the power of computational hardware, advances in computational capacity, such as better chips or quantum computing, will set the stage for advances in AI. On a purely algorithmic level, most of the astonishing results produced by labs such as DeepMind come from combining different approaches to AI, much as AlphaGo combines deep learning and reinforcement learning. Combining deep learning with symbolic reasoning, analogical reasoning, Bayesian and evolutionary methods all show promise.

Those who do not believe that AI is making that much progress relative to human intelligence are forecasting another AI winter, during which funding will dry up due to generally disappointing results, as has happened in the past. Many of those people have a pet algorithm or approach that competes with deep learning.

Finally, there are the pragmatists, plugging along at the math, struggling with messy data, scarce AI talent and user acceptance. 

Machine learning’s rise, applications, and challenges

Image Credit: peterhowell / Getty Images

Where does your enterprise stand on the AI adoption curve? Take our AI survey to find out.

The terms “artificial intelligence” and “machine learning” are often used interchangeably, but there’s an important difference between the two. AI is an umbrella term for a range of techniques that allow computers to learn and act like humans. Put another way, AI is the computer being smart. Machine learning, however, accounts for how the computer becomes smart.

But there’s a reason the two are often conflated: The vast majority of AI today is based on machine learning. Enterprises across sectors are prioritizing it for various use cases across their organizations, and the subfield tops AI funding globally by a significant margin. In the first quarter of 2019 alone, a whopping $28.5 billion was allocated to machine learning research. Overall, the machine learning market is expected to grow from around $1 billion in 2016 to $8.81 billion by 2022. When VentureBeat collected thoughts from the top minds across the field, they had a variety of predictions to share. But one takeaway was that machine learning is continuing to shape business and society at large.

Rise of machine learning

While AI is ubiquitous today, there were times when the whole field was thought to be a dud. After initial advancements and a lot of hype in the mid-late 1950s and 1960s, breakthroughs stalled and expectations went unmet. There wasn’t enough computing power to bring the potential to life, and running such systems cost exorbitant amounts of money. This caused both interest and funding to dry up in what was dubbed the “AI winter.”

The pursuit later picked up again in the 1980s, thanks to a boost in research funds and expansion of the algorithmic toolkit. But it didn’t last, and there was yet another decade-long AI winter.

Then two major changes occurred that directly enabled AI as we know it today. Artificial intelligence efforts shifted from rule-based systems to machine learning techniques that could use data to learn without being externally programmed. And at the same time, the World Wide Web became ubiquitous in the homes (and then hands) of millions (and eventually billions) of people around the world. This created the explosion of data and data sharing on which machine learning relies.

How machine learning works

Machine learning enables a computer to “think” without being externally programmed. Instead of programming it by hand to accomplish specific tasks, as is the case with traditional computers, machine learning allows you to instead provide data and describe what you want the program to do.

The computer trains itself with that data, and then uses algorithms to carry out your desired task. It also collects more data as it goes, getting “smarter” over time. A key part of how this all works is the data labeling. If you want a program to sort photos of ice cream and pepperoni pizza, for example, you first need to first label some of the photos to give the algorithm an idea of what ice cream and pepperoni pizza each look like.

This labeling is also a key difference between machine learning and a popular subset within the field, called deep learning. Deep learning doesn’t require any labeling, instead relying on neural networks, which are inspired by the human brain both in structure and name. To sort the photos of ice cream and pepperoni pizza using this technique, you instead have to provide a significantly larger set of photos. The computer then puts the photos through several layers of processing — which make up the neural network — to distinguish the ice cream from the pepperoni pizza one step at a time. Earlier layers look at basic properties like lines or edges between light and dark parts of the images, while subsequent layers identify more complex features like shapes or even faces.

Applications

Machine learning and its subsets are useful for a wide range of problems, tasks, and applications. There’s computer vision, which allows computers to “see” and make sense of images and videos. Additionally, natural language processing (NLP) is a rising part of machine learning, which allows computers to extract the meaning of unstructured text. There’s also voice and speech recognition, which powers services like Amazon’s Alexa and Apple’s Siri and introduced many consumers to AI for the first time.

Across industries, enterprises are using machine learning in their products as well as internally within their organizations. Machine learning can simplify, streamline, and enhance supply chain operations, for example. It’s also widely used for business analytics, security, sales, and marketing. Machine learning has even been used to help fight COVID-19. Facebook leans on machine learning to take down harmful content. Google uses it to improve search. And American Express recently tapped NLP for its customer service chatbots and to run a predictive search capability inside its app. The list goes on and on.

Limitations and challenges

While machine learning holds promise and is already benefiting enterprises around the globe, there are challenges and issues associated with the field. For example, machine learning is useful for recognizing patterns, but it doesn’t perform well when it comes to generalizing knowledge. For users, there’s also the issue of “algorithm fatigue.”

Some of the issues related to machine learning have significant consequences that are already playing out today. The lack of explainability and interpretability — known as the “black box problem” — is one. Machine learning models create their own behaviors and decisions in ways that even their creators can’t understand. This makes it difficult to fix errors and ensure the information a model puts out is accurate and fair. When people noticed Apple’s algorithm for credit cards was offering women significantly smaller lines of credit than men, for example, the company couldn’t explain why and didn’t know how to fix the issue.

This is related to the most significant issue plaguing the field: data and algorithmic bias. Since the technology’s inception, machine learning models have been routinely and primarily built on data that was collected and labeled in biased ways, sometimes for specifically biased purposes. It’s been found that algorithms are often biased against women, Black people, and other ethnic groups. Researchers at Google’s DeepMind, one of the world’s top AI labs, warned the technology poses a threat to individuals who identify as queer.

This issue is widespread and widely known, but there is resistance to taking the significant action many in the field are urging is necessary. Google itself fired the co-leads of its ethical AI team, Timnit Gebru and Margaret Mitchell, in what thousands of the company’s employees called a “retaliatory firing,” after Gebru refused to rescind research about the risks of deploying large language models. And in a survey of researchers, policy leaders, and activists, the majority said they worry the evolution of AI by 2030 will continue to be primarily focused on optimizing profits and social control, at the expense of ethics. Legislation regarding AI — especially immediately and obviously harmful uses, like facial recognition for policing — is being debated and adopted across the country. These deliberations will likely continue. And the changing data privacy laws will soon affect data collection, and thus machine learning, as well.