“How does IBM Watson work? It talks, answers my questions and makes recommendations on what and where I should eat…”
Named after IBM’s first CEO, Thomas J. Watson, IBM Watson is a supercomputer-artificial intelligence (AI) hybrid functioning as a question and answer system. Unlike most of today’s technology and computers, IBM Watson gets better with age and leverages natural processing and machine learning technologies to uncover insights from the internet’s ‘unstructured data.’
Its question and answer computing system mimics the human brain and learns from its mistakes like a human. It evaluates, retrieves data and makes decisions faster than humans as well, having access to 90 servers and over 200-million pages online, including all of Wikipedia, all your digital documents, journals, statistics, emails, social media posts and more — to answer questions fast.
Anyone with access to IBM Watson, whether it be sales associates or customers, will gain access to expert-level information on demand, anytime, faster than ever before — the world and the workforce is about to change.
Below we highlight an entertaining animation explanation video on how IBM Watson works, take a gander and feel free to read the full transcript along the way.
Transcript for IBM Watson: how it works
IBM Watson is at the forefront of a new era in computing: cognitive computing.
It’s a radically new kind of computing, very different from the programmable systems that preceded it.
As different as those systems were from the tabulating machines of a century ago.
Conventional computing solutions, based on mathematical principles that emanate from the 1940s are programmed based on rules and logic intended to derive mathematically precise answers, often following rigid decision tree approach but with today’s wealth of big data and the need for more complex evidence-based decisions such a rigid approach often breaks.
Or fails to keep up with available information.
Cognitive computing enables people to create a profoundly new kind of value finding answers and insights locked away in volumes of data.
Whether we consider a doctor diagnosing a patient, a wealth manager advising a client on their retirement portfolio, or even a chef creating a new recipe, they need new approaches to put into context the volume of information they deal with on a daily basis in order to derive value from it.
This process services to enhance human expertise.
Watson and its cognitive capabilities veer some of the key cognitive elements of human expertise, systems that reason about problems like a human does.
When we as humans seek to understand something and to make a decision, we go through 4 key steps:
First, we observe visible phenomena and bodies of evidence;
Second, we draw on what we know to interpret what we’re seeing to generate hypothesis about what it means;
Third, we evaluate which hypothesis are right or wrong;
Finally, we decide, choosing the option which seems best and acting accordingly.
Just as humans become experts by going through the process of observation, evaluation and decision making, cognitive systems like Watson, use similar processes to reason about the information they read.
Watson can also do this at massive speed and scale.
So how does Watson do it? (How Watson Works?)
Unlike conventional processes to computing which can only handle neatly structured data, such as what is stored in a database, Watson can understand unstructured data, which is 80% of data today.
All of the information that is produced primarily by humans for other humans to consume, this includes everything from literature, articles, research reports to blogs, posts and tweets.
While structured data is governed by well-defined fields that contain well-specified information, Watson relies on natural language which is governed by rules of grammar, context and culture.
It’s implicit, ambiguous, complex and a challenge to process.
While all human language is difficult to parse, certain idioms can be particularly challenging.
In English for instance, we can feel blue because it’s raining cats and dogs, while we’re filling in a form someone asked us to fill out.
When it comes to text, Watson doesn’t look for keyword matches or synonymous like a search engine.
It actually reads and interprets text like a person. It does this by breaking down a sentence grammatically, relationally and structurally.
Discerning meaning from semantics of the written material.
Watson understands context. This is very different from very simple speech recognition, which is how a computer translates human speech into a set of words.
Watson tries to understand the real intent of the user’s language and uses that understanding to possibly extract logical responses and draw inferences to potential answers through a broader array of linguistic models and algorithms.
How Watson learns?
When Watson goes to work in a particular field it learns the language, the jargon and the mode of thought of that domain.
Take the term ‘cancer’ for instance, there are many different types of cancer and each type has different symptoms and treatments.
However, those symptoms can also be associated with diseases other than cancer.
Treatments can have side effects and affect people differently depending on many factors.
Watson evaluates standard-of-care practices and thousands of pages of literature that capture the best science in the field.
And from all of that, Watson identifies the therapies that offer the best choices for the doctor to consider in their treatment of the patient.
With the guidance of human experts, Watson collects the knowledge required to have literacy in a particular domain, what’s called a corpus of knowledge.
Collecting a corpus starts off with loading a relevant body of literature into Watson, building the corpus also requires some human intervention to cull through the information and discard anything that is out of date, poorly regarded or immaterial to the problem domain.
We refer to this as curating the content.
Next the data is pre-processed by Watson, building indices and other meta data that make working with that content more efficient.
This is known as ingestion. At this time Watson may also create a knowledge graph to assist and answer more precise questions.
Now that Watson has ingested the corpus, it needs to be trained by a human expert to learn how to interpret the information, to learn the best possible responses and acquire the ability to find patterns, Watson partners with experts, who train it in using an approach called machine learning.
An expert will upload training data into Watson in the form of question-answer pairs that serve as ground truth.
This doesn’t give explicit answers for every question it receives, but rather teaches it the linguistic patterns of meaning in the domain.
Once Watson has been trained on QA pairs it continues to learn through ongoing interaction, interactions between users and Watson are periodically reviewed by experts and fed back into the system to help Watson better interpret information, likewise, as new information is published Watson is updated so that it’s constantly adapting to shifts in knowledge and linguistic interpretation in any given field.
Watson is now ready to respond to questions about highly complex situations and quickly provide a range of potential responses and recommendations that are backed by evidence.
It’s also prepared to identify new insights or patterns locked away in information.
How Watson builds and enhances expertise?
From metallurgists looking for new alloys to researchers looking to develop more effective drugs, human experts are using Watson to uncover new possibilities in data and make better evidence-based decisions.
Across all of these different applications there is a common approach that Watson follows, after identifying parts of speech in a question or inquiry, it generates hypothesis, Watson then looks for evidence to support or refute the hypothesis, it scores each passage based on statistical modelling for each piece of evidence, known as weighted-evidence scores.
Watson estimates its confidence based on how high the response is rated during evidence scoring and ranking.
In essence, Watson is able to run analytics against a body of data to glean insights which Watson can turn into inspirations allowing human experts make better and more informed decisions.
Across an organization, Watson scales and democratizes expertise by surfacing accurate responses and answers to an inquiry or question.
Watson also accelerates expertise by surfacing a set of possibilities from a large body of data, saving valuable time.
Today Watson is revolutionizing the way we make decisions, become experts and share expertise in fields diverse as law, medicine and even cooking.
Further Watson is discovering and offering answers in patterns we hadn’t known existed.
Faster than any known person or group of people ever could, in ways that make a material difference every day.
Most important of all, Watson learns, adapts and keeps getting smarter.
It actually gains value with age by learning from its interactions with us and from its own successes and failures just like we do.
So now that you know how it works, how do these ideas inspire how you work?
How can Watson make you a better expert? What will you do with Watson?
Source: IBM Watson YouTube