Study: IBM Watson agrees with cancer docs on treatment options 90% of the time

Print 12 December 2016
Amirah Al Idrus / Fierce Medical Device

Since it launched its Watson Health unit last spring, IBM has rolled out cognitive computing tools for a number of applications, including genomics and oncology. But while the Jeopardy-winning supercomputer has generated a lot of buzz, critics are still wondering: is Watson really useful in these areas?

IBM is trying to prove that.

Its Watson for Oncology platform is a cognitive computing system that helps oncologists make treatment decisions for individual patients. It analyzes numerous attributes from each patient and runs through a wealth of medical literature “in a matter of seconds” to return a recommendation to the physician on whether the standard treatment should be followed, considered, or abandoned.

At the San Antonio Breast Cancer Symposium, Dr. S. P. Somashekhar, chairman of the Manipal Comprehensive Cancer Center in Bengaluru, India, presented positive data from a concordance study pitting the supercomputer against the cancer center’s molecular tumor board, a group of specialists that meets weekly to evaluate cancer cases.

The double-blind study, which involved 638 breast cancer cases, found that in 90% of cases, Watson for Oncology made the same treatment decision as the molecular tumor board. When broken down into different subgroups of breast cancer, the numbers vary, with an 80% concordance rate in nonmetastatic disease, a 45% concordance rate in metastatic cancers and a 35% rate in HER2/neu-negative cases.

The study was not designed to prove that either the machine or the doctors were right or wrong, said Dr. Andrew Norden, Watson Health deputy chief health officer. It was designed to see how often Watson would agree with the experts. The 90% figure validates the system as useful, he said.

As for the lower rates of agreement in more complex cases, both Somashekhar and Norden said these were not unexpected. “Including HER2/neu cases opens up many more treatments and variables for consideration,” Somashekhar said. “This increases the demands on human thinking capacity. More complicated cases lead to more divergent opinions on the recommended treatment.”

A decision support tool such as Watson takes far less time to comb through the literature than a human oncologist would. This frees the physician to spend more time with his or her patients and “understand their values and goals in a way that I don’t see cognitive computing trying to do,” Norden said. “Which is what I think is the real benefit. Cognitive systems can take the intellectual burden off the doctors to help them have more time to interact with patients.”

IBM has entered into several partnerships around its Watson systems; the latest involves using Watson for Drug Discovery to accelerate Pfizer’s efforts in the immuno-oncology space. And while this one concordance study may not sway skeptical critics, IBM does have other concordance studies under way, evaluating its Watson for Genomics and Watson for Clinical Trial Matching, data from which will surface over the coming year.

Source

Return