CS 522: Andrew Ng, Pranav Rajpurkar, Anand Avati

Andrew Ng

• Overview of AI and why it is affecting healthcare

• AI is the new electricity

• Electrification of country transformed every major industry

• Reached surprisingly clear path for AI to transform industries just like electricity 100 years ago

• Almost all the value that is created through AI today (economic, etc.) comes from one type - supervised learning

• Supervised learning

• Learn the following mapping: Input A -> Output B

• Ex: facial recognition to unlock cell phones

• A = image, B = is it you?

• Ex: targeted advertising

• A = (ad, user), B = click

• Ex: speech recognition

• A = audio, B = text transcript

• Lots of users, such as when searching -- but only recently has this been enabled

• Ex: automated diagnosis

• A = radiology image, B = diagnosis

• Turns out a lot of the value creation the best opportunities is finding the creative ways to implement supervised learning

• Supervised learning has been around for a long time; why is it only now taking off?

• We have much more data now

• X-axis: amount of data, y-axis: performance

• With traditional learning algorithms, performance takes a logarithmic curve

• Hence, these algorithms don't take into too much account the amount of data we have

• With neural networks, and especially deeper ones, we can take greater advantage of the data

• However, you need a lot of data to train these neural networks, and we only recently have this in healthcare

• The rise of EHR records being publicly available partially helps fuel this

• Over the last 5 years we went from having a very little data to a lot of data

• Finally, the other breakthrough was the rise of very large neural networks

The rise of larger neural networks has led to higher performance with greater utilization of data

Pranav Rajpurkar

• Arrhythmia Detection

• Arrhythmia = irregular beating of the heart

• Costs a lot to treat

• Affects 100s of millions of people

• Diagnosed with ECG test, which shows heart's electrical activity over time

• The problem is many times if we see a small sample of the ECG, some issues may not be noticed

• Ideally we want to monitor for longer periods of time

• We now have the devices to do this

• If we collect data for a 14-day period, this is way too much data for doctors to analyze

• Would want to automate analysis of this data

• Even doctors have a hard time diagnose this

• There are many components that are part of a ECG waveform, and the doctor looks at these features

• There are many rules but programming into a computer is futile since there's so much variation

• If we look at the performance of doctors on Atrial Fibrillation, we see about 70% accuracy

Various components of a ECG waveform

• In the past, machine learning relied on shallow models and little data

• In this work, we use data from 30,000 unique patients

• We use a Convolutional Neural Network (CNN): go from A (ECG data) to B (annotated Arrhythmia)

• In a span of 5 months, we hit human performance

• And in the 6th month, we beat cardiologists and reached 81% accuracy

• Automation has a lot of impact

• Saves patients cost of misdiagnosis

• Saves doctors time

• Low cost

• Allows for diagnosis of arrhythmia in low-resource settings, where people may not have access to cardiologists

• Could outperform board-certified physicians at Stanford

• https://stanfordmlgroup.github.io/projects/ecg/

• https://arxiv.org/pdf/1707.01836.pdf

Anand Avati

• Palliative Care

• Providing relief from pain, rather than on curative treatments

• Ex: hospice care

• EHR (electronic health record) + electronic version of patient's health history

• Billing, insurance claims, demographic data, clinical order data

• Noisy, prone to human error (as humans input the data), missing data

• Project: Deep Learning to predict whether patient will die in next 3-12 months using EHR data

• Why?

• 80% of Americans prefer to spend final days at home if possible but only 20% do

• 60% of deaths occur in hospital while receiving curative treatment

• Why the mismatch?

• Doctors tend to be over-optimistic, work under time pressure

• Palliative care teams are understaffed; expensive to proactively chart review all admitted patients for palliative care needs

• A big reason is the economic incentives in the healthcare system

• Goal:

• Improve access to palliative care for patients

• Automatically screen all patients admitted to hospital

• Use deep learning model to make automated prediction and share with palliative care team

• Trained a model with 18 years of data and 30,000 patients

• Impact

• Offline validation: AUC = 0.94

• Validation in real-world:

• Suggested 50 patients to palliative care; all 50 were reviewed by palliative care team and verified as legitimate

• Currently undergoing pilots in Stanford hospital

• Hospital board of directors are excited

• Increases the quality care of patients

• Morally right thing to do

Andrew Ng

• Many industries are undergoing the IT revolution

• Over the last 10 years, what was once stored in a piece of paper are now stored digitally

• Ex: Coursera: keeps a record of educational lectures

• Ex: garment industry: all purchases are stored in a database rather than filing cabinet

• Results in rise of data

• IT revolution -> Data -> AI revolution

• The IT revolution is still going on generally

• Healthcare is still undergoing IT revolution

• Alison Darcy: Woebot

• Epidemic of mental illness: $200B total cost to US, 58% do not receive care

• Student population is acutely underserved

• Huge demand for mental health support

• 50% of college students experience debilitating anxiety and depression

• Problems are growing in prevalence and severity

• Ex: Stanford CAPS: has ~2 week wait time, which is too long

• Up to 75% of students who need help do not receive it

• Cost, stigma, time

• Created a Facebook Messenger bot, woebot

• Randomized controlled trial results: control vs. woebot

• High engagement: users interacted with wombat 12.12 days out of 14 day period

• High satisfaction: 4.3/5

• Very high acceptability: users report emotional stability

• Those who talked to the chatbot relieved significant symptoms of depression

• Chatbots make good therapists

• Talk to bot, you may open up more than if you talk to human; slower to open up to a human; bots don't judge you

• Non-human, integrated in everyday life, customized therapy

• You're not bothering anyone in the middle of the night (i.e. feel less guilty to talk to a bot at 2am rather than call a clinician)

• Therapy is super social important and unmet need.

• Application of healthcare that impacts so many people

• AI is at the stage -- specifically NLP (natural language processing) -- where bots are able to have rudimentary understanding

• Healthcare is influencing the way we build software

• Silicon Valley: have standard ways to develop a mobile app

• PM has a wireframe, which the engineer builds

• Woebot: not as important what the UI/UX looks like per se; more important what the actual content of the chatbot is

• In conclusion, seen IT and AI transform several different industries

• I think this is a good time to get into AI as well

• A lot of this stuff is new; just implemented in the last 2 years

Q&A:

• How do we train a supervised learning model when gold standard is subjective (e.g. two clinicians may not agree on a given diagnosis)

• Definitely the case, so often get multiple labels for data

• Pranav's project: Computer agreed with a human more often than human with human, helping indicate the model is doing something substantial

• Averaging the intuition of humans

• Healthcare is notorious for being difficult for clinical/regulatory approval and sustainable business model. Is this a challenge for adoptions in AI? What are ways to mitigate this?

• Many healthcare entrepreneurs spend 50% innovating/50% worrying about regulation

• Among entrepreneurial communities debate:

• 1) pursue consumerization route (Fitbit)

• 2) pursue regulatory approval

• AI is blackbox

• We are comfortable taking drugs whose biological function we don't really understand, but clinical trial shows it works

• AI can be treated the same

• Ethical issues in healthcare. People may not want to know with certainty particular facts (e.g. date of death with high probability)

• Can do a lot of good and a lot of harm just like electricity

• Challenge of labeling datasets when the process if costly (e.g. requires clinicians)

• Limits availability of labeled data

• We will evolve better and better processes to labeling data efficiently; crowdsourcing