When I first show the PBS documentary on IBM’s Watson, then watched it in the live Jeopardy, the first thing that came to mind is how this could be used in medicine (given my model of medicine from personal experience, and, of course, watching House). Not literally as a doctor, but as an aid to a real human doctor. Now it appears that IBM is headed in this direction with it as well.
There is a lot to say about this but I’ll really focus on just one part, attributed to:
Others, including physician Mark Graber, a former chief of the Veterans Administration hospital in Northport, New York, are less enthused. “Doctors have enough knowledge,” said Graber, who now heads the Society to Improve Diagnosis in Medicine. “In medicine, that’s not the problem we face.”
Now Dr. Graber seems to have the credentials for his statements to be quite credible and he doesn’t appear merely to be a Luddite, but in this claim:
Several years ago, Graber, who has written extensively on diagnostic error, led an Archives of Internal Medicine analysis of 100 cases of misdiagnoses. Only in a few cases did doctors err because they lacked necessary information. Most misdiagnoses arose from cognitive problems, such as overconfidence or inattention, or systemic problems of miscommunication, inefficiency and poor teamwork. Doctors had many problems, but being unable to find the latest journal article wasn’t one of them.
I think he demonstrates he doesn’t really understand how Watson (or any deductive pseudo-AI software works). Watson isn’t just finding and spouting information from journals; Watson is, in fact, making deep connections even though it doesn’t “understand” those connections (no more than it “understood” puns in Jeopardy). But as early disagreements over AI (dominated, unfortunately, by Minksy), Watson is not doing this in a way that is comfortable to amateur computer observers. Originally a dogmatic POV ruled AI and basically asserted that nothing could be accomplished by brute force and everything had to be based on logic and math. That path prevailed, suppressed other approaches, and turned out to be completely sterile. In fact, now brute force (heavily advanced and promoted by IBM) has turned out to the useful, but also the underlying statistical/heuristic approaches have proven to be more robust than logical approaches (hopefully putting an end to Lisp as the solution of all problems).
Tell me, Dr. Graber, how if “overconfidence” and “inattention” and “teamwork” are significant human flaws in medicine Watson isn’t going to add something there. Watson has no ego to protect, doesn’t get tired, and actually has few preconceived notions or biases. It isn’t just looking up journal articles, it’s finding connections. Now if Dr. Graber wishes to say that real medicine is totally unlike the fictional Gregory House’s differential diagnosis, I’ll totally accept that, BUT, if the real process even remotely resembles what happens in every House episode (connecting obscure information to ambiguous symptoms), frankly I’ll put my bet (and possibly life) on Watson.
I nearly lost my vision to a misdiagnosis and I learned several things from this. First, my input to my doctor biased his judgment, so next time I’ll stop suggesting things (I had been working with some harsh chemicals that I thought I might have gotten into my eye and thus caused the irritation and his quick diagnosis confirmed this, incorrectly as it turned out). Second, doctors are quite logical, but have a basic flaw that they tend to pick the statistically most likely thing, as the problem, and also perhaps the easiest and safest to treat (perhaps both to save costs as well as reduce side-effects). So my eye looked like it was inflamed due to the chemicals and my doctor prescribed steroids, a totally logical approach. BUT, it turns out a more unlikely cause could exist and given that cause steroids would be exactly the wrong thing to do. My doctor was not an ophthalmologist so he drew a logical but incorrerct conclusion. How could any doctor be an expert in everything. And had I not suggested the “cause” by talking about working with the chemicals I might not have limited my doctor’s thinking to the obvious conclusion.
OTOH, when my condition got worse and I did go to an ophthalmologist he saw the problem differently, quickly halted the steroids and starting treating the viral infection which had actually grown worse due to the immune system inhibiting effect of the steroids.
I don’t fault my doctor at all in this (although had I lost my sight I would have been very angry). He looked at the data he had with the knowledge he had and reached a sound, but wrong conclusion. This is where Watson, purely as an aid (I don’t want a pile of silicon actually treating me) might have been useful. Watson, I assume, would have potentially raised the viral infection possibility the ophthalmologist eventually correctly diagnosed and informed my doctor, plus presumably indicated how potentially dangerous the steroid treatment was if the problem was infection. In that case I would simply have expected my doctor to pause, possibly consult Watson and learn he didn’t have the necessary tools for to decisively test, and thus referred me immediately to an ophthalmologist who could do the stains and tests to determine what is wrong. Or my doctor could have said Watson was completely wrong and gone ahead with his treatment, but then perhaps set some future appointment to confirm the results. My doctor, who I still completely trust, and a machine, working as partners, would be my ideal.
Decades ago a friend of mine, using the old school (Minsky) AI, was part of a project to bring computers into the diagnosis process. The technology was way more crude simply because the hardware didn’t have the horsepower that Watson can apply. Only a few doctors would even cooperate with the project since most just automatically rejected the idea that computers could possibly do what they do, as they saw their diagnostic process as way to sophisticated, and, importantly, “human” and “intuitive” than was possible to do in software.
Back then, they were probably right. Today they’re probably wrong. So I hope concerned and capable doctors, like Dr. Graber are more open-minded.