WEBVTT

00:00:00.080 --> 00:00:02.430
The following content is
provided under a Creative

00:00:02.430 --> 00:00:03.810
Commons license.

00:00:03.810 --> 00:00:06.060
Your support will help
MIT OpenCourseWare

00:00:06.060 --> 00:00:10.150
continue to offer high quality
educational resources for free.

00:00:10.150 --> 00:00:12.690
To make a donation, or to
view additional materials

00:00:12.690 --> 00:00:16.600
from hundreds of MIT courses,
visit MIT OpenCourseWare

00:00:16.600 --> 00:00:17.260
at ocw.mit.edu.

00:00:31.680 --> 00:00:36.210
PROFESSOR: So welcome
everybody, and I actually

00:00:36.210 --> 00:00:38.830
used to be at MIT in the
'90s, so it's good to be back.

00:00:38.830 --> 00:00:41.540
And so we're going to talk
today about a different kind

00:00:41.540 --> 00:00:42.170
of security.

00:00:42.170 --> 00:00:44.550
It's going to be less on the
technical mechanism side,

00:00:44.550 --> 00:00:46.200
and more on the,
well, what happens

00:00:46.200 --> 00:00:48.492
when all this technology gets
put in place in something

00:00:48.492 --> 00:00:49.824
where there's high consequences?

00:00:49.824 --> 00:00:52.060
Not quite so high-consequence
as, say, an airplane in

00:00:52.060 --> 00:00:55.290
the sky, but getting
pretty close.

00:00:55.290 --> 00:00:57.470
Just to let you know
where I'm coming from.

00:00:57.470 --> 00:00:59.680
So I used to be part of the
midnight coffeehouse club

00:00:59.680 --> 00:01:02.299
myself, but this is
Michigan, actually.

00:01:02.299 --> 00:01:04.090
We're not quite as big
as your campus here.

00:01:04.090 --> 00:01:06.895
But a short while
ago somebody decided

00:01:06.895 --> 00:01:09.700
to put a hot tub on our
computer science building,

00:01:09.700 --> 00:01:11.364
so they're doing
research inside there.

00:01:11.364 --> 00:01:13.030
But what we're going
to talk about today

00:01:13.030 --> 00:01:16.016
is some of the research
that bubbled out of that.

00:01:16.016 --> 00:01:17.640
So we're going to
talk about everything

00:01:17.640 --> 00:01:23.010
from exploding defibrillators
to other issues of privacy

00:01:23.010 --> 00:01:24.667
in medical devices.

00:01:24.667 --> 00:01:26.250
And this mainly is
going to be related

00:01:26.250 --> 00:01:29.460
to just one thread of research
from one of my former graduate

00:01:29.460 --> 00:01:32.880
students here, who is
actually at this point

00:01:32.880 --> 00:01:36.020
sanitizing explanted pacemakers.

00:01:36.020 --> 00:01:38.540
But we're going to mostly
talk about the security

00:01:38.540 --> 00:01:41.210
of medical devices today.

00:01:41.210 --> 00:01:43.670
Got a bunch of acknowledgements.

00:01:43.670 --> 00:01:44.510
There it is on tape.

00:01:47.030 --> 00:01:49.370
This work is by tons
of people, and I'm

00:01:49.370 --> 00:01:51.480
going to try to
summarize for you some

00:01:51.480 --> 00:01:54.400
of the modern bits about
medical device security

00:01:54.400 --> 00:01:56.530
through all sorts of places.

00:01:56.530 --> 00:01:58.620
I'm also required to
put up this boilerplate

00:01:58.620 --> 00:02:01.580
slide of my potential
conflict of interest,

00:02:01.580 --> 00:02:03.860
so now you can know about
any potential biases

00:02:03.860 --> 00:02:05.080
on my thinking.

00:02:05.080 --> 00:02:06.740
But I'd like to
think that I am less

00:02:06.740 --> 00:02:08.820
biased than the average person.

00:02:08.820 --> 00:02:11.360
OK.

00:02:11.360 --> 00:02:12.580
So moving on.

00:02:12.580 --> 00:02:15.256
So an interesting thing
happened about a year ago,

00:02:15.256 --> 00:02:17.130
when FDA-- the Food and
Drug Administration--

00:02:17.130 --> 00:02:19.540
released a draft document
saying they are now

00:02:19.540 --> 00:02:22.580
going to be expecting
manufacturers to consider cyber

00:02:22.580 --> 00:02:24.814
security-- or as we call
it, security and privacy--

00:02:24.814 --> 00:02:27.480
not only in their implementation
of the medical device software,

00:02:27.480 --> 00:02:29.560
but in their design
of their software.

00:02:29.560 --> 00:02:31.970
Before a single line of
code has been written.

00:02:31.970 --> 00:02:33.744
And so we're going
to talk about how

00:02:33.744 --> 00:02:35.910
this has affected the
thinking in the medical device

00:02:35.910 --> 00:02:37.340
manufacturing community.

00:02:37.340 --> 00:02:40.040
Their final guidance came
out just a couple weeks ago,

00:02:40.040 --> 00:02:42.330
and we just held
a conference call.

00:02:42.330 --> 00:02:44.730
FDA held a conference
call, and over 650 people

00:02:44.730 --> 00:02:46.922
decided to join
the teleconference.

00:02:46.922 --> 00:02:49.380
So there's a lot of interest
in the manufacturing community

00:02:49.380 --> 00:02:51.838
about how to take some of the
concepts you're learning here

00:02:51.838 --> 00:02:54.760
in your class and actually apply
it to the medical community.

00:02:54.760 --> 00:02:55.640
But it's really hard.

00:02:55.640 --> 00:02:58.864
And I noticed one of the
questions up on the website

00:02:58.864 --> 00:03:01.530
was about how to get the culture
change in the medical community

00:03:01.530 --> 00:03:03.080
to understand security.

00:03:03.080 --> 00:03:05.260
And this slide illustrates that.

00:03:05.260 --> 00:03:08.010
So, who washed their
hands this morning?

00:03:08.010 --> 00:03:08.635
OK.

00:03:08.635 --> 00:03:11.250
Oh, this is not MIT, everybody.

00:03:11.250 --> 00:03:15.520
So actually about
164, 165 years ago,

00:03:15.520 --> 00:03:18.670
there was a famous
physician, Ignaz Semmelweis,

00:03:18.670 --> 00:03:21.130
who was looking into something
called childbed fever.

00:03:21.130 --> 00:03:23.650
And he discovered that
his medical students

00:03:23.650 --> 00:03:26.150
who were working in the morgue
in the morning who later went

00:03:26.150 --> 00:03:27.941
to work with patients,
well, those patients

00:03:27.941 --> 00:03:29.320
tended to die more often.

00:03:29.320 --> 00:03:32.260
And he discovered if you washed
your hands, then statistically

00:03:32.260 --> 00:03:35.740
you were less likely to pass
on some kind of probability

00:03:35.740 --> 00:03:38.220
of not living longer.

00:03:38.220 --> 00:03:41.435
So he recommended that
physicians wash their hands.

00:03:41.435 --> 00:03:43.310
And the reaction from
the physician community

00:03:43.310 --> 00:03:45.290
was, doctors are
gentleman, and therefore

00:03:45.290 --> 00:03:48.190
their hands are always clean.

00:03:48.190 --> 00:03:50.487
And to some extent
we're seeing some

00:03:50.487 --> 00:03:52.570
of those kinds of attitudes
toward security today,

00:03:52.570 --> 00:03:53.780
so it's not too surprising.

00:03:53.780 --> 00:03:55.490
But I'll try to
draw some parallels

00:03:55.490 --> 00:03:56.960
with that throughout the talk.

00:03:59.452 --> 00:04:00.910
I've got a lot of
material to cover

00:04:00.910 --> 00:04:02.618
so I'm going to whip
through some things.

00:04:02.618 --> 00:04:04.930
But first thing I'm going
to do-- anyone a physician?

00:04:04.930 --> 00:04:05.430
No?

00:04:05.430 --> 00:04:07.040
OK, well you're all
going to be able to have

00:04:07.040 --> 00:04:08.740
some good material
for cocktail parties

00:04:08.740 --> 00:04:10.320
with your doctor friends.

00:04:10.320 --> 00:04:11.210
We're going to talk
a little bit about

00:04:11.210 --> 00:04:12.500
implantable medical devices.

00:04:12.500 --> 00:04:14.221
Actually I'll pass
this guy around.

00:04:14.221 --> 00:04:15.470
You can feel free to touch it.

00:04:15.470 --> 00:04:16.386
It's been de-dangered.

00:04:16.386 --> 00:04:17.779
Just don't lick it.

00:04:17.779 --> 00:04:21.790
This is a defibrillator
from a former patient.

00:04:21.790 --> 00:04:25.241
And actually this is a device
here-- about 50 years ago,

00:04:25.241 --> 00:04:27.740
some of the first pacemakers
started to appear on the scene.

00:04:27.740 --> 00:04:28.531
They were external.

00:04:28.531 --> 00:04:31.420
You had to have a burly
nurse to cart it around.

00:04:31.420 --> 00:04:34.440
And then as the
decades wore on, they

00:04:34.440 --> 00:04:36.160
became small enough
to be implanted,

00:04:36.160 --> 00:04:37.790
completely implanted
in the body.

00:04:37.790 --> 00:04:39.290
And here you see a
picture of what's

00:04:39.290 --> 00:04:42.060
called a wand that's
using inductive coupling.

00:04:42.060 --> 00:04:43.300
It's technically wireless.

00:04:43.300 --> 00:04:44.091
There are no wires.

00:04:44.091 --> 00:04:47.634
To wirelessly program the device
to be 60 beats per minute.

00:04:47.634 --> 00:04:49.550
But interesting to me
as a security researcher

00:04:49.550 --> 00:04:51.720
was that in around
2003 or so, we

00:04:51.720 --> 00:04:53.870
began to see defibrillators,
such as the one I'm

00:04:53.870 --> 00:04:58.890
passing around, that started to
embrace wireless technologies

00:04:58.890 --> 00:05:00.950
and networking that
you'd be more used to

00:05:00.950 --> 00:05:03.440
as sort of general computation.

00:05:03.440 --> 00:05:06.470
And I was thinking what
could possibly go wrong?

00:05:06.470 --> 00:05:08.650
Luckily there are a lot
of engineers also thinking

00:05:08.650 --> 00:05:12.480
that same question in
companies, but security, it

00:05:12.480 --> 00:05:13.564
takes a different mindset.

00:05:13.564 --> 00:05:15.563
And I'm going to tell you
a little bit about how

00:05:15.563 --> 00:05:16.774
that mindset is changing.

00:05:16.774 --> 00:05:18.940
So if you were to open up
one of those devices, what

00:05:18.940 --> 00:05:21.716
you find inside are vast
resource constraints.

00:05:21.716 --> 00:05:23.340
If you want a hard
engineering problem,

00:05:23.340 --> 00:05:25.630
pop open one of these devices.

00:05:25.630 --> 00:05:28.070
So about half of the
device is just a battery.

00:05:28.070 --> 00:05:29.500
A very high quality battery.

00:05:29.500 --> 00:05:33.330
These cost about $40,000
a pop on the market.

00:05:33.330 --> 00:05:34.530
Silver vanadium oxide.

00:05:34.530 --> 00:05:36.613
And you've got little
microcontrollers at the top.

00:05:36.613 --> 00:05:38.270
Typically you have
some antennas where

00:05:38.270 --> 00:05:40.769
you can do your communication
for your control of the device

00:05:40.769 --> 00:05:41.760
as well.

00:05:41.760 --> 00:05:44.917
This is all hermetically
sealed, implanted in your body.

00:05:44.917 --> 00:05:47.250
We're talking one of the
harshest environments possible.

00:05:47.250 --> 00:05:49.541
You want to recharge a battery
in your body, good luck.

00:05:49.541 --> 00:05:52.230
Did you know that batteries
give off heat and gas?

00:05:52.230 --> 00:05:54.745
So there are very
challenging constraints

00:05:54.745 --> 00:05:56.330
to engineering the device.

00:05:56.330 --> 00:06:00.000
When you want to add security,
it gets just a little bit hard.

00:06:00.000 --> 00:06:02.260
So there is, however,
a very good reason

00:06:02.260 --> 00:06:06.000
for having a wirelessly
controlled medical device.

00:06:06.000 --> 00:06:08.390
There are good reasons, but
there are these subtle risks.

00:06:08.390 --> 00:06:10.600
So to illustrate
that, I want you

00:06:10.600 --> 00:06:13.440
to see what pacemakers
used to look like.

00:06:13.440 --> 00:06:15.630
So this is a pacemaker
from the Medtronic Museum

00:06:15.630 --> 00:06:17.000
up in Minneapolis.

00:06:17.000 --> 00:06:19.590
And can anyone guess what
that little piece of metal

00:06:19.590 --> 00:06:21.760
is on the right hand side?

00:06:21.760 --> 00:06:24.630
What its function is?

00:06:24.630 --> 00:06:25.300
Antenna?

00:06:25.300 --> 00:06:26.870
Control?

00:06:26.870 --> 00:06:27.936
Control is very close.

00:06:27.936 --> 00:06:28.685
Any other guesses?

00:06:31.650 --> 00:06:34.730
So this is a device before
there was wireless communication

00:06:34.730 --> 00:06:35.772
to control a pacemaker.

00:06:35.772 --> 00:06:37.980
In the old days, when you
want to change the settings

00:06:37.980 --> 00:06:40.320
on your device, the
physician says, "Patient,

00:06:40.320 --> 00:06:41.470
please lift up your arm.

00:06:41.470 --> 00:06:43.720
I'm going to put a needle
through your armpit to twist

00:06:43.720 --> 00:06:45.890
the dial to change
your heart rate. "

00:06:45.890 --> 00:06:48.300
So one of the great
reasons for wireless

00:06:48.300 --> 00:06:50.260
is that it actually
reduces infection rates,

00:06:50.260 --> 00:06:53.350
because the more you put
foreign objects in your body,

00:06:53.350 --> 00:06:56.040
the more likely you are
to contract an infection.

00:06:56.040 --> 00:06:57.350
It is a serious risk.

00:06:57.350 --> 00:07:01.840
Actually, 1% of implantations
have major complications,

00:07:01.840 --> 00:07:03.870
and of those,
about 1% are fatal.

00:07:03.870 --> 00:07:06.370
So controlling infection is one
of the most important things

00:07:06.370 --> 00:07:10.440
you can do in the implantation
and changing of the device.

00:07:10.440 --> 00:07:13.140
Of course, if you go the
other extreme and just say,

00:07:13.140 --> 00:07:14.740
I want to put
wireless everywhere,

00:07:14.740 --> 00:07:16.601
you'll get different
kinds of risks.

00:07:16.601 --> 00:07:18.975
So I've sort of dubbed this
the bacon theory of wireless.

00:07:21.725 --> 00:07:23.100
Now my mother's
from the Midwest,

00:07:23.100 --> 00:07:25.920
so she used to say bacon
makes everything better.

00:07:25.920 --> 00:07:28.390
And I've noticed there are
some device manufacturers who

00:07:28.390 --> 00:07:30.460
seem to be putting wireless
everywhere without necessarily

00:07:30.460 --> 00:07:31.751
thinking through all the risks.

00:07:31.751 --> 00:07:33.970
It does have its
benefits, but you

00:07:33.970 --> 00:07:36.290
need to very strategically
think before you add this

00:07:36.290 --> 00:07:37.732
to a safety critical device.

00:07:37.732 --> 00:07:39.606
What are the security
risks for instance that

00:07:39.606 --> 00:07:41.580
are going to be opening up?

00:07:41.580 --> 00:07:43.820
Oops, I had one
misplaced slide, but I

00:07:43.820 --> 00:07:45.410
guess I'll just say it anyway.

00:07:45.410 --> 00:07:47.534
I'm not going to talk a
whole lot about networking,

00:07:47.534 --> 00:07:50.740
but I thought this quote was
just too good not to mention.

00:07:50.740 --> 00:07:53.230
Does anyone remember the
ship off the coast of Italy?

00:07:53.230 --> 00:07:55.020
The captain says,
"These days, everything

00:07:55.020 --> 00:07:58.980
is much safer, thanks to modern
instruments and the internet."

00:07:58.980 --> 00:08:02.830
And there's his ship
that turned over there.

00:08:02.830 --> 00:08:04.887
So you add internet
connectivity and wireless

00:08:04.887 --> 00:08:07.220
to your medical device, there
are going to be new risks.

00:08:07.220 --> 00:08:08.720
And you don't need
to be afraid of them,

00:08:08.720 --> 00:08:10.770
but you just need to have
appropriate mitigating

00:08:10.770 --> 00:08:12.796
controls.

00:08:12.796 --> 00:08:13.920
So I'm flying through this.

00:08:13.920 --> 00:08:15.730
But what I want to
give you is paint

00:08:15.730 --> 00:08:19.920
a picture of what's a typical
day in a medical device,

00:08:19.920 --> 00:08:22.140
and how it's used
in clinical care,

00:08:22.140 --> 00:08:23.740
and how that might
change your mindset

00:08:23.740 --> 00:08:25.420
if you come from a
security background,

00:08:25.420 --> 00:08:27.360
and how you think about risk.

00:08:27.360 --> 00:08:29.620
So first going to
talk about the world

00:08:29.620 --> 00:08:34.000
where there aren't real threats,
just unsafe practices and some

00:08:34.000 --> 00:08:35.490
carelessness.

00:08:35.490 --> 00:08:39.470
So the FDA maintains a database
of near misses, malfunctions,

00:08:39.470 --> 00:08:40.539
injuries, and deaths.

00:08:40.539 --> 00:08:41.330
This is all public.

00:08:41.330 --> 00:08:42.705
You can go look
this up yourself.

00:08:42.705 --> 00:08:43.820
It's called MAUDE.

00:08:43.820 --> 00:08:47.630
And one of the devices was
called this volumetric infusion

00:08:47.630 --> 00:08:48.130
pump.

00:08:48.130 --> 00:08:51.660
This is a device that
infuses drugs into your body

00:08:51.660 --> 00:08:53.970
through an IV mechanically.

00:08:53.970 --> 00:08:55.900
And this patient died.

00:08:55.900 --> 00:08:59.030
And if you look carefully,
it says one of the causes

00:08:59.030 --> 00:09:01.356
was a buffer overflow.

00:09:01.356 --> 00:09:03.105
I think you learned
about buffer overflows

00:09:03.105 --> 00:09:04.490
in your first lecture.

00:09:04.490 --> 00:09:07.950
So they are very real and they
happen and in every sector.

00:09:07.950 --> 00:09:11.270
So in this particular case when
the buffer overflow occurred,

00:09:11.270 --> 00:09:14.880
it was actually caught in their
error checking in the software,

00:09:14.880 --> 00:09:17.450
but the action it took
was to shut the pump down.

00:09:17.450 --> 00:09:19.630
To bring it down to a safe mode.

00:09:19.630 --> 00:09:22.080
What they didn't realize
was that for some patients,

00:09:22.080 --> 00:09:24.440
shutting down the pump is
basically a death sentence.

00:09:24.440 --> 00:09:28.030
So this patient died
after the increase

00:09:28.030 --> 00:09:32.740
in intracranial pressure,
followed by brain death because

00:09:32.740 --> 00:09:35.750
of the buffer overflow.

00:09:35.750 --> 00:09:38.730
So there's nothing really
complicated here, right?

00:09:38.730 --> 00:09:40.595
You all know you don't
want to have buffer

00:09:40.595 --> 00:09:41.720
overflows in your software.

00:09:41.720 --> 00:09:43.670
There's no adversary
at this point.

00:09:43.670 --> 00:09:47.600
So this kind of illustrates
the state of software, at least

00:09:47.600 --> 00:09:49.195
for this particular device.

00:09:49.195 --> 00:09:51.700
It's very challenging.

00:09:51.700 --> 00:09:53.940
The other challenging
part that doesn't come up

00:09:53.940 --> 00:09:56.520
a whole lot in a security
course is the human side.

00:09:56.520 --> 00:09:59.150
So there are few
universities that

00:09:59.150 --> 00:10:02.600
focus on the human element, but
I think there ought to be more.

00:10:02.600 --> 00:10:05.240
So I set out on some life
experience of my own.

00:10:05.240 --> 00:10:07.290
My wife asked to
remain anonymous,

00:10:07.290 --> 00:10:10.460
so she said as long as
I don't reveal her name.

00:10:10.460 --> 00:10:13.530
So that's me, that's our
infusion pump in the back,

00:10:13.530 --> 00:10:15.600
and that's our baby in there.

00:10:15.600 --> 00:10:18.920
And for us luckily the
pump worked just fine.

00:10:18.920 --> 00:10:23.080
But pumps are great for
delivering medical care,

00:10:23.080 --> 00:10:28.250
but they have resulted in over
500 deaths due to various forms

00:10:28.250 --> 00:10:29.930
of malfunctions.

00:10:29.930 --> 00:10:34.010
So I'm going to tell you
about one more malfunction.

00:10:34.010 --> 00:10:35.990
There's also an
implantable kind of pump.

00:10:35.990 --> 00:10:37.770
They're not just
bedside pumps, the kind

00:10:37.770 --> 00:10:41.090
you see on daytime
hospital dramas.

00:10:41.090 --> 00:10:43.330
But here's an implantable
pump, and it's

00:10:43.330 --> 00:10:45.080
got this semipermeable
membrane where

00:10:45.080 --> 00:10:46.890
you can replenish the drugs.

00:10:46.890 --> 00:10:49.610
And this is a user interface
that the nurse or the clinician

00:10:49.610 --> 00:10:52.070
uses to change the dosage rate.

00:10:52.070 --> 00:10:55.670
So does anyone see where you
type in the quantity of drug?

00:10:58.660 --> 00:11:00.246
You've got to kind
of squint, right?

00:11:00.246 --> 00:11:02.310
So you squint really closely.

00:11:02.310 --> 00:11:04.600
And one thing you'll notice
is here by number six

00:11:04.600 --> 00:11:07.060
it says we're going to
dose this bolus-- bolus

00:11:07.060 --> 00:11:11.061
is a quantum of drug-- over
20 minutes and 12 seconds.

00:11:11.061 --> 00:11:12.810
We're going to dose
this into the patient.

00:11:12.810 --> 00:11:14.643
And this is implanted,
so you don't feel it.

00:11:14.643 --> 00:11:16.450
There's no nerve.

00:11:16.450 --> 00:11:18.860
And this user
interface is actually

00:11:18.860 --> 00:11:21.830
after an FDA recall
went into effect

00:11:21.830 --> 00:11:23.720
for the software quality.

00:11:23.720 --> 00:11:28.960
So what was missing before the
recall were eight key elements.

00:11:28.960 --> 00:11:34.442
In particular HH:MM:SS.

00:11:34.442 --> 00:11:38.650
So what do you think happens,
or what you think could happen,

00:11:38.650 --> 00:11:41.690
if that label were missing?

00:11:41.690 --> 00:11:46.120
It's really easy to
get the units wrong.

00:11:46.120 --> 00:11:48.410
Make an order of
magnitude error.

00:11:48.410 --> 00:11:52.390
So unfortunately for this
patient, who later expired,

00:11:52.390 --> 00:11:56.654
he or she had his or
her pump reprogrammed,

00:11:56.654 --> 00:11:58.320
and the health care
professional noticed

00:11:58.320 --> 00:12:01.370
that the bolus was given over
20 minutes instead of 20 hours

00:12:01.370 --> 00:12:02.670
after the fact.

00:12:02.670 --> 00:12:05.710
Unfortunately the patient
left the facility,

00:12:05.710 --> 00:12:07.650
got into a motor
vehicle accident,

00:12:07.650 --> 00:12:10.870
and then later died after the
family removed life support.

00:12:10.870 --> 00:12:14.920
But if you look at this from
a technical perspective,

00:12:14.920 --> 00:12:16.810
the problem is
pretty simple, right?

00:12:16.810 --> 00:12:19.050
In terms of you didn't
have the label there.

00:12:19.050 --> 00:12:21.670
But human factors is
very easy to overlook.

00:12:21.670 --> 00:12:23.590
It's not always right
there, front and center,

00:12:23.590 --> 00:12:24.395
in the engineering process.

00:12:24.395 --> 00:12:26.580
Do you have a human factors
part in this lecture?

00:12:26.580 --> 00:12:27.480
See what I mean?

00:12:27.480 --> 00:12:29.730
Blame Nickolai.

00:12:29.730 --> 00:12:30.940
No, Nickolai is great.

00:12:30.940 --> 00:12:33.620
But it's a very
important element

00:12:33.620 --> 00:12:36.350
of improving the
trustworthiness of devices

00:12:36.350 --> 00:12:38.330
that rely on software.

00:12:38.330 --> 00:12:41.170
So I encourage you to think
about better human elements

00:12:41.170 --> 00:12:43.440
and human factors
for your software,

00:12:43.440 --> 00:12:46.760
even if it's on
something non-critical.

00:12:46.760 --> 00:12:49.260
So that should begin
to paint a picture

00:12:49.260 --> 00:12:53.710
of the typical problems
in medical device failures

00:12:53.710 --> 00:12:55.710
post [INAUDIBLE] 25.

00:12:55.710 --> 00:12:57.640
And the other thing
I want to talk about

00:12:57.640 --> 00:12:59.840
is the exciting
world of management.

00:12:59.840 --> 00:13:02.320
Management, exciting.

00:13:02.320 --> 00:13:05.634
I used to collect all these
little dialogue boxes whenever

00:13:05.634 --> 00:13:07.300
my computer would get
a software update,

00:13:07.300 --> 00:13:09.520
but this all happens
in the background now.

00:13:09.520 --> 00:13:11.390
Like my iPhone's
constantly getting updates

00:13:11.390 --> 00:13:13.530
and drawing more power.

00:13:13.530 --> 00:13:15.990
But now it just sort of happens.

00:13:15.990 --> 00:13:20.160
But medical devices also
take software updates.

00:13:20.160 --> 00:13:23.540
They're not really
fundamentally different from

00:13:23.540 --> 00:13:25.430
traditional computing devices.

00:13:25.430 --> 00:13:28.869
They just happen to control
vital functions of your body.

00:13:28.869 --> 00:13:30.160
So there's an interesting case.

00:13:30.160 --> 00:13:31.920
It's now been about four years.

00:13:31.920 --> 00:13:35.010
So McAfee-- there are a
number of antivirus companies

00:13:35.010 --> 00:13:37.210
that produce products
that hospitals use--

00:13:37.210 --> 00:13:40.040
and in this particular case,
McAfee had this software

00:13:40.040 --> 00:13:42.350
update that actually
misclassified

00:13:42.350 --> 00:13:44.266
a critical Windows
DL as malicious,

00:13:44.266 --> 00:13:46.015
and then decided to
quarantine the system.

00:13:49.280 --> 00:13:52.947
So when it
quarantined, let's see.

00:13:56.617 --> 00:14:00.370
[COMPUTER SOUND]

00:14:00.370 --> 00:14:01.980
That always happens, right?

00:14:01.980 --> 00:14:02.480
OK.

00:14:02.480 --> 00:14:05.650
So, ha ha ha.

00:14:05.650 --> 00:14:08.030
In this particular
case with McAfee,

00:14:08.030 --> 00:14:10.770
when they quarantined
this critical Windows DL

00:14:10.770 --> 00:14:14.690
as malicious, the machine
just started rebooting.

00:14:14.690 --> 00:14:16.430
Blue Screen of
Death and cycling.

00:14:16.430 --> 00:14:17.969
And in Rhode Island,
they basically

00:14:17.969 --> 00:14:19.760
stopped admitting
patients at one hospital,

00:14:19.760 --> 00:14:22.200
except for severe cases
like gunshot wounds,

00:14:22.200 --> 00:14:24.800
because their admission systems
weren't working properly.

00:14:24.800 --> 00:14:30.270
So clinical care depends heavily
on the function of software,

00:14:30.270 --> 00:14:34.650
and we sometimes forget
about the role of security.

00:14:34.650 --> 00:14:38.990
On the topic of depending
on other people's software,

00:14:38.990 --> 00:14:42.275
Microsoft has one of the largest
footprint of operating systems.

00:14:42.275 --> 00:14:44.650
And believe it or not, there
are a lot of medical devices

00:14:44.650 --> 00:14:46.730
that run on Windows XP.

00:14:46.730 --> 00:14:48.230
Windows XP, in case
you didn't hear,

00:14:48.230 --> 00:14:51.694
went out of service
half a year ago.

00:14:51.694 --> 00:14:53.610
So you should not be
using this, because there

00:14:53.610 --> 00:14:56.680
are no more updates, security
updates, function updates.

00:14:56.680 --> 00:14:58.340
It's antiquated software.

00:14:58.340 --> 00:15:00.660
But there are still medical
devices today being shipped

00:15:00.660 --> 00:15:03.140
brand new with Windows XP.

00:15:03.140 --> 00:15:06.424
The software life cycles
are a little bit misaligned.

00:15:06.424 --> 00:15:08.840
If you're used to downloading
updates for your open source

00:15:08.840 --> 00:15:12.100
software on a daily basis, well,
think about medical devices.

00:15:12.100 --> 00:15:14.760
You might not be able to
get to it, say, for a year.

00:15:14.760 --> 00:15:16.860
It might be in the
field for 20 years.

00:15:16.860 --> 00:15:18.970
So it's very difficult
to locate software

00:15:18.970 --> 00:15:21.420
that's appropriate for
a 20-year life cycle.

00:15:21.420 --> 00:15:24.280
It's basically flying in space.

00:15:24.280 --> 00:15:26.210
So the Food and
Drug Administration

00:15:26.210 --> 00:15:29.190
has now released some
guidance-- actually,

00:15:29.190 --> 00:15:33.120
this was just
exactly a month ago--

00:15:33.120 --> 00:15:35.666
on what they expect to
see from manufacturers.

00:15:39.060 --> 00:15:40.590
Think of it as a design project.

00:15:40.590 --> 00:15:43.100
As you're writing down
all the requirements

00:15:43.100 --> 00:15:45.640
of your medical device,
they're asking manufacturers

00:15:45.640 --> 00:15:48.082
how have they thought through
the security problems.

00:15:48.082 --> 00:15:50.290
How have they thought through
all the security risks?

00:15:50.290 --> 00:15:51.760
How are they mitigating it?

00:15:51.760 --> 00:15:53.850
What risks are they
accepting as what

00:15:53.850 --> 00:15:56.630
they call residual risk,
things that they don't solve?

00:15:56.630 --> 00:15:59.590
But they expect them to least
be aware of all the risks

00:15:59.590 --> 00:16:02.080
and ideally mitigate them.

00:16:02.080 --> 00:16:04.230
So with the management
of software,

00:16:04.230 --> 00:16:06.430
when no one person
is accountable,

00:16:06.430 --> 00:16:08.400
all sorts of crazy
things happen.

00:16:08.400 --> 00:16:11.280
But there is some
guidance now that's

00:16:11.280 --> 00:16:14.570
beginning to emerge to help
the manufacturing community

00:16:14.570 --> 00:16:18.740
to better integrate security
into their products.

00:16:18.740 --> 00:16:21.480
So I think we're making
some pretty good time.

00:16:21.480 --> 00:16:22.170
All right.

00:16:22.170 --> 00:16:25.050
So now we're going to be able
to go into the security side.

00:16:25.050 --> 00:16:26.840
I wanted to get the
non-security stuff out

00:16:26.840 --> 00:16:29.040
of the way for the context.

00:16:29.040 --> 00:16:33.770
So let's put on our gray
hats and black hats.

00:16:33.770 --> 00:16:36.200
Before I begin this,
though, I guess

00:16:36.200 --> 00:16:39.280
what I want to say is this
is a very challenging area

00:16:39.280 --> 00:16:42.240
to do research, because
there are patients.

00:16:42.240 --> 00:16:48.140
And if I were given a medical
device, for instance, today,

00:16:48.140 --> 00:16:50.480
I'd still take it even if
the security problems weren't

00:16:50.480 --> 00:16:52.560
all worked out, because
I know I'm much better

00:16:52.560 --> 00:16:54.690
off with that medical device.

00:16:54.690 --> 00:16:56.500
But that said, of
course, I'd prefer

00:16:56.500 --> 00:16:58.920
to have medical devices
that are more secure.

00:16:58.920 --> 00:17:02.610
So there is the emergence of
more and more secure devices,

00:17:02.610 --> 00:17:05.750
but today, if you have to choose
between a device and no device,

00:17:05.750 --> 00:17:07.500
I'd strongly
recommend taking it,

00:17:07.500 --> 00:17:10.069
because you're going to be
in a much better position.

00:17:10.069 --> 00:17:12.460
But that said, let's
take a look now.

00:17:12.460 --> 00:17:15.220
If we consider the adversary,
and if the adversary

00:17:15.220 --> 00:17:17.790
wants to cause problems
to a medical device.

00:17:17.790 --> 00:17:21.578
So who's got the
defibrillator at the moment?

00:17:21.578 --> 00:17:22.619
Oh, it's right over here.

00:17:22.619 --> 00:17:24.413
Good.

00:17:24.413 --> 00:17:26.329
So I'd like to tell you
a little bit about how

00:17:26.329 --> 00:17:28.380
these defibrillators
are implanted.

00:17:28.380 --> 00:17:31.130
This is a very special device
because, well, number one,

00:17:31.130 --> 00:17:33.930
it's implanted, therefore
it's very high risk.

00:17:33.930 --> 00:17:36.310
It's life sustaining.

00:17:36.310 --> 00:17:38.660
If it's pacing your heart,
for instance, and it fails,

00:17:38.660 --> 00:17:40.110
the results can be catastrophic.

00:17:40.110 --> 00:17:42.485
So it's very interesting from
an engineering perspective.

00:17:42.485 --> 00:17:45.680
It needs to work
24/7 for many years.

00:17:45.680 --> 00:17:48.635
So this is a programmer.

00:17:48.635 --> 00:17:49.780
Not a person, but a device.

00:17:49.780 --> 00:17:52.230
It's basically a ruggedized
computer, and attached to it

00:17:52.230 --> 00:17:53.320
is a little wand.

00:17:53.320 --> 00:17:54.340
That's not a mouse.

00:17:54.340 --> 00:18:00.040
That's a transmitter/receiver
speaking a proprietary wireless

00:18:00.040 --> 00:18:02.210
signal over a licensed spectrum.

00:18:02.210 --> 00:18:04.220
We're not talking
802.11, we're talking

00:18:04.220 --> 00:18:06.640
specially-licensed
spectrum there.

00:18:06.640 --> 00:18:09.630
And what happens is it
takes about 90 minutes.

00:18:09.630 --> 00:18:11.510
The patient is
awake, just slightly

00:18:11.510 --> 00:18:14.540
sedated to remain calm, and
there's a local anesthetic.

00:18:14.540 --> 00:18:17.640
A small incision is made
beneath the clavicle.

00:18:17.640 --> 00:18:19.560
And then the team--
typically it's

00:18:19.560 --> 00:18:24.930
a team of about six people--
will weave electrodes

00:18:24.930 --> 00:18:27.420
through a sacrificed
blood vessel that then

00:18:27.420 --> 00:18:28.760
terminates inside the heart.

00:18:28.760 --> 00:18:31.150
And actually I have
one of them right here.

00:18:31.150 --> 00:18:34.550
This was not previously used.

00:18:34.550 --> 00:18:35.750
You can pass this around.

00:18:35.750 --> 00:18:37.960
You see the little
tines on the end.

00:18:37.960 --> 00:18:41.650
And on some of the devices
there's both a sensor,

00:18:41.650 --> 00:18:43.215
so it can sense
your cardiac rhythm,

00:18:43.215 --> 00:18:44.340
and there's also actuation.

00:18:44.340 --> 00:18:47.410
You can send shocks,
both small and large,

00:18:47.410 --> 00:18:51.430
to either pace the heart or
to basically reboot the heart

00:18:51.430 --> 00:18:53.100
if there's a chaotic rhythm.

00:18:53.100 --> 00:18:54.740
It's a very highly
advanced device.

00:18:54.740 --> 00:18:57.100
It's a steroid-tipped
piece of metal on the end,

00:18:57.100 --> 00:18:59.200
so it doesn't bind
to the tissue.

00:18:59.200 --> 00:19:02.390
You can pass that around.

00:19:02.390 --> 00:19:05.870
It's basically a
USB cable, right?

00:19:05.870 --> 00:19:10.090
So after that's
implanted into the body,

00:19:10.090 --> 00:19:11.660
the patient is sewn up.

00:19:11.660 --> 00:19:12.617
They do some testing.

00:19:12.617 --> 00:19:14.200
And typically the
patient will receive

00:19:14.200 --> 00:19:16.710
what looks like a
little base station.

00:19:16.710 --> 00:19:18.380
Like a little access point.

00:19:18.380 --> 00:19:20.410
It's very proprietary.

00:19:20.410 --> 00:19:23.725
Typically they speak a
proprietary RF to the implant

00:19:23.725 --> 00:19:25.700
so it can gather
all the telemetry,

00:19:25.700 --> 00:19:29.320
so that it can send it back up
through the cloud-- typically

00:19:29.320 --> 00:19:31.825
through a private cloud,
for whatever private means--

00:19:31.825 --> 00:19:34.182
so that the health care
professionals can keep tabs

00:19:34.182 --> 00:19:34.890
on their patient.

00:19:34.890 --> 00:19:36.830
So for instance, if
you notice that there's

00:19:36.830 --> 00:19:40.060
some odd measurement
coming from patient Mary,

00:19:40.060 --> 00:19:41.620
you might call up
Mary and say, "You

00:19:41.620 --> 00:19:42.540
should really make
an appointment

00:19:42.540 --> 00:19:44.706
and come in, because I'd
like to see what's going on

00:19:44.706 --> 00:19:45.770
with your defibrillator."

00:19:45.770 --> 00:19:47.603
So one of the nice
things about the wireless

00:19:47.603 --> 00:19:49.670
is they're able to have
more continuous care

00:19:49.670 --> 00:19:51.580
rather than come back in a year.

00:19:54.280 --> 00:19:56.440
We had a team of students
at several universities

00:19:56.440 --> 00:19:59.490
get together, and I gave them
one of these defibrillators

00:19:59.490 --> 00:20:01.080
and an oscilloscope,
and they went off

00:20:01.080 --> 00:20:02.895
into a cave for
about nine months.

00:20:02.895 --> 00:20:06.390
And they came back and
said, "Look what we found!"

00:20:06.390 --> 00:20:11.640
So this is a screenshot of the
communication between a device

00:20:11.640 --> 00:20:12.860
and the programmer.

00:20:12.860 --> 00:20:15.890
And what you can see is first
of all, it's in the clear.

00:20:15.890 --> 00:20:18.500
There's no cryptography, at
least none that we could find.

00:20:18.500 --> 00:20:21.190
You'll find inside here the name
of the implanting physician,

00:20:21.190 --> 00:20:23.210
the diagnosis, the hospital.

00:20:23.210 --> 00:20:25.380
Basically a complete
electronic health record.

00:20:25.380 --> 00:20:28.585
This is an older device,
from about 10 years ago.

00:20:28.585 --> 00:20:31.080
But that was the state of
the art about 10 years ago.

00:20:31.080 --> 00:20:33.170
There didn't appear to
be any use of encryption,

00:20:33.170 --> 00:20:36.550
at least for the privacy
of the health information.

00:20:36.550 --> 00:20:38.430
So when we noticed
this, we thought,

00:20:38.430 --> 00:20:41.110
well then, we definitely need
to look at the security side

00:20:41.110 --> 00:20:42.670
about how the device
is controlled.

00:20:42.670 --> 00:20:44.980
How do they ensure the
authenticity of the control?

00:20:44.980 --> 00:20:46.780
The integrity?

00:20:46.780 --> 00:20:50.180
And that's when we decided to
do the following experiment.

00:20:50.180 --> 00:20:53.060
So we started learning how to
use something called a software

00:20:53.060 --> 00:20:53.610
radio.

00:20:53.610 --> 00:20:55.735
Probably some of you have
played around with these.

00:20:55.735 --> 00:20:57.130
There are a bunch of them now.

00:20:57.130 --> 00:20:59.330
About 10 years ago,
the most popular one

00:20:59.330 --> 00:21:02.740
was the USRP and
GNU radio software.

00:21:02.740 --> 00:21:08.050
So we took an antenna from a
pacemaker that we didn't need,

00:21:08.050 --> 00:21:12.150
created a little antenna, and
we recorded the RF communication

00:21:12.150 --> 00:21:14.700
of inducing a
fatal heart rhythm.

00:21:14.700 --> 00:21:17.640
And then we replayed
that communication back.

00:21:17.640 --> 00:21:19.810
And then the device
happily emitted

00:21:19.810 --> 00:21:23.160
a large-- something on the
order of a 500-volt shock.

00:21:23.160 --> 00:21:27.270
On the order of about 32 joules
in one millisecond, which

00:21:27.270 --> 00:21:30.062
I'm told if you were
to get that on you,

00:21:30.062 --> 00:21:32.020
it's like being kicked
in the chest by a horse.

00:21:32.020 --> 00:21:36.090
So it's a rather powerful shock.

00:21:36.090 --> 00:21:38.840
And the interesting thing
was how we discovered this.

00:21:38.840 --> 00:21:41.500
So I was in the operating
room, and recall back,

00:21:41.500 --> 00:21:43.880
I said that when
you're a patient

00:21:43.880 --> 00:21:47.240
and the procedure is
ending, the health care

00:21:47.240 --> 00:21:51.030
team tests if the defibrillator
is working properly.

00:21:51.030 --> 00:21:55.830
So how do you end-to-end
test if a defibrillator's

00:21:55.830 --> 00:21:59.860
working properly if the
heart is beating normally?

00:21:59.860 --> 00:22:00.590
Right?

00:22:00.590 --> 00:22:03.580
So what's built into
the defibrillator

00:22:03.580 --> 00:22:06.160
is a command to induce
the very fatal heart

00:22:06.160 --> 00:22:10.200
rhythm that the defibrillator
is designed to restore you from.

00:22:10.200 --> 00:22:12.890
It's called a command shock.

00:22:12.890 --> 00:22:15.130
So when I asked the
physicians about that,

00:22:15.130 --> 00:22:18.047
they didn't seem to understand
the concept of authentication.

00:22:18.047 --> 00:22:19.630
And that's when we
decided we'd really

00:22:19.630 --> 00:22:22.672
need to look more deeply into
how to solve these problems.

00:22:22.672 --> 00:22:24.130
So in this particular
case, we were

00:22:24.130 --> 00:22:25.980
able to send the
command to the device,

00:22:25.980 --> 00:22:32.130
and we weren't authenticated,
and we could induce that shock.

00:22:32.130 --> 00:22:35.810
The good news is
these devices have

00:22:35.810 --> 00:22:38.450
been able to solve these
problems through some software

00:22:38.450 --> 00:22:39.010
updates.

00:22:39.010 --> 00:22:40.968
And they've been aware
of it for quite a while,

00:22:40.968 --> 00:22:42.660
so they're able to
spin out devices

00:22:42.660 --> 00:22:44.600
that now take into
account some of these more

00:22:44.600 --> 00:22:47.010
adversarial conditions.

00:22:47.010 --> 00:22:48.510
Where are those
tines going around?

00:22:48.510 --> 00:22:49.300
Over there?

00:22:49.300 --> 00:22:52.000
OK, great.

00:22:52.000 --> 00:22:53.357
So that's the implant side.

00:22:53.357 --> 00:22:55.190
There's a huge amount
of innovation going on

00:22:55.190 --> 00:22:56.680
with implants.

00:22:56.680 --> 00:22:59.070
It's not really science
fiction anymore,

00:22:59.070 --> 00:23:01.660
but there are real people
and patients behind it.

00:23:01.660 --> 00:23:05.340
And most people care deeply
about delivering quality health

00:23:05.340 --> 00:23:07.080
care.

00:23:07.080 --> 00:23:11.030
But sometimes they just don't
realize how to fit security

00:23:11.030 --> 00:23:13.060
into their designing process.

00:23:13.060 --> 00:23:16.760
So it's a challenge culturally.

00:23:16.760 --> 00:23:18.540
Another stakeholder
are the people

00:23:18.540 --> 00:23:20.380
who provide health care
in the first place.

00:23:20.380 --> 00:23:22.560
Hospitals, primarily,
or small clinics.

00:23:22.560 --> 00:23:24.620
If you want to find
malware, go to a hospital.

00:23:24.620 --> 00:23:27.090
You're going to find
some interesting malware.

00:23:27.090 --> 00:23:28.350
And here's why.

00:23:28.350 --> 00:23:32.550
So here's a screenshot
from a colleague who

00:23:32.550 --> 00:23:34.850
used to work at Beth Israel
Deaconess Medical Center

00:23:34.850 --> 00:23:36.040
here in Boston.

00:23:36.040 --> 00:23:38.440
And he gave a map of his
network architecture.

00:23:38.440 --> 00:23:40.610
There's nothing particularly
earth-shattering

00:23:40.610 --> 00:23:41.770
about the architecture.

00:23:41.770 --> 00:23:43.490
What was interesting,
though, was

00:23:43.490 --> 00:23:46.030
he listed the
number of operating

00:23:46.030 --> 00:23:48.530
systems in his
hospital on what were

00:23:48.530 --> 00:23:50.710
considered medical devices.

00:23:50.710 --> 00:23:53.490
And I looked at him-- I like
to add up numbers and insanity

00:23:53.490 --> 00:23:55.880
check things-- and I
said, "Well, you've

00:23:55.880 --> 00:23:58.720
got Service Pack one, two,
three of Windows XP, zero

00:23:58.720 --> 00:24:00.310
15 plus one.

00:24:00.310 --> 00:24:01.460
That equals 16.

00:24:01.460 --> 00:24:02.630
That doesn't equal 600.

00:24:02.630 --> 00:24:04.580
Your addition's wrong."

00:24:04.580 --> 00:24:06.910
And he looked at me and he
said, "No, Kevin, that's

00:24:06.910 --> 00:24:11.020
600 Service Pack zero
machines in the hospital."

00:24:11.020 --> 00:24:12.580
So these are medical
devices where

00:24:12.580 --> 00:24:14.880
they've been unable to
get the manufacturer

00:24:14.880 --> 00:24:17.910
to provide patches and update
it to the modern software.

00:24:17.910 --> 00:24:20.202
Which means it's
that old software,

00:24:20.202 --> 00:24:22.410
vulnerable to all the old
malware that's been hitting

00:24:22.410 --> 00:24:26.140
Windows XP for 15 years.

00:24:26.140 --> 00:24:29.260
So it's very difficult
in the clinical setting

00:24:29.260 --> 00:24:32.197
to keep yourself protected,
because the product life cycles

00:24:32.197 --> 00:24:33.530
are just completely out of sync.

00:24:33.530 --> 00:24:36.350
They think in terms of
decades in health care,

00:24:36.350 --> 00:24:39.350
but in the fast hockey stick
world of Silicon Valley,

00:24:39.350 --> 00:24:43.110
we think about days, weeks, or
months for software updates.

00:24:43.110 --> 00:24:45.700
You can see down here in
their clinical systems,

00:24:45.700 --> 00:24:47.830
average time to infection
is about 12 days

00:24:47.830 --> 00:24:51.020
when they don't have any kind
of protection against malware.

00:24:51.020 --> 00:24:52.762
And they can get
almost up to a year

00:24:52.762 --> 00:24:54.970
if they're able to get an
antivirus product on there.

00:24:54.970 --> 00:24:56.510
But even that's not perfect.

00:24:59.422 --> 00:25:01.380
And feel free to ask
questions too, by the way,

00:25:01.380 --> 00:25:02.421
if you want to know more.

00:25:02.421 --> 00:25:04.180
Go deeper dive on any
of these incidents.

00:25:04.180 --> 00:25:06.900
But one of the
interesting things I found

00:25:06.900 --> 00:25:11.030
was that one relatively
common source of infection

00:25:11.030 --> 00:25:12.209
is the vendor themselves.

00:25:12.209 --> 00:25:13.750
Sometimes they don't
even realize it.

00:25:13.750 --> 00:25:15.208
So I'm going to go
over a few cases

00:25:15.208 --> 00:25:18.240
where the vendor has
sort of accidentally

00:25:18.240 --> 00:25:20.145
been the carrier of the malware.

00:25:20.145 --> 00:25:23.420
I was talking with the
chief field security

00:25:23.420 --> 00:25:26.330
officer for the Veterans
Administration, the VA.

00:25:26.330 --> 00:25:29.790
They have about 153 clinics
in the United States.

00:25:29.790 --> 00:25:32.750
And one day there was
a vendor showing up

00:25:32.750 --> 00:25:35.540
to do software updates on
some of their clinical medical

00:25:35.540 --> 00:25:36.660
devices.

00:25:36.660 --> 00:25:38.390
And her intrusion
detection software

00:25:38.390 --> 00:25:39.870
was just chirping
away everywhere--

00:25:39.870 --> 00:25:41.411
I think his name
was Bob-- everywhere

00:25:41.411 --> 00:25:43.790
Bob was walking and
plugging in his USB drive

00:25:43.790 --> 00:25:45.050
to update the software.

00:25:45.050 --> 00:25:47.410
He was infecting the machines
with malware by accident,

00:25:47.410 --> 00:25:50.380
because somehow malware
got onto his USB drive.

00:25:50.380 --> 00:25:52.660
So there's a
perception out there

00:25:52.660 --> 00:25:54.619
that if you're not
networked, you're safe.

00:25:54.619 --> 00:25:56.160
But if you think
about it for moment,

00:25:56.160 --> 00:25:58.220
very few people used the
internet 20 years ago

00:25:58.220 --> 00:25:59.940
and there were still
computer viruses.

00:25:59.940 --> 00:26:02.520
So in a hospital, a
common infection vector

00:26:02.520 --> 00:26:04.290
is the USB drive.

00:26:04.290 --> 00:26:06.410
I'm even aware of
two manufacturers--

00:26:06.410 --> 00:26:11.360
I can't tell you their
names-- but they almost

00:26:11.360 --> 00:26:13.480
shipped malware-infected
medical devices.

00:26:13.480 --> 00:26:15.900
And they caught it
by chance, by luck,

00:26:15.900 --> 00:26:18.440
before it went out
into the product line.

00:26:21.440 --> 00:26:23.730
Who's done any work
on the programming

00:26:23.730 --> 00:26:26.980
with the cloud or
software distribution?

00:26:26.980 --> 00:26:28.180
A few of you.

00:26:28.180 --> 00:26:33.050
So the medical community is
also embracing the cloud.

00:26:33.050 --> 00:26:35.420
It gives them more
distributive control.

00:26:35.420 --> 00:26:37.020
But it also comes
with risks that

00:26:37.020 --> 00:26:40.300
are qualitatively different
from your typical software.

00:26:40.300 --> 00:26:44.100
If you want to get the
newest word processor,

00:26:44.100 --> 00:26:45.450
that's one thing.

00:26:45.450 --> 00:26:47.680
But if you want to get an
update for your ventilator,

00:26:47.680 --> 00:26:49.070
completely different.

00:26:49.070 --> 00:26:51.560
So I noticed there was
a recall on the firmware

00:26:51.560 --> 00:26:52.600
for a ventilator.

00:26:52.600 --> 00:26:56.410
And the manufacture sent out
a handy dandy website where

00:26:56.410 --> 00:26:58.729
you could download an update.

00:26:58.729 --> 00:27:00.770
Now I was going to go
check their PGP signatures.

00:27:00.770 --> 00:27:02.670
Couldn't find those,
but what I did find

00:27:02.670 --> 00:27:04.000
was a little link down here.

00:27:04.000 --> 00:27:06.180
It says, "Click here for
your software update."

00:27:06.180 --> 00:27:09.590
I thought, oh, goody,
let's go do that.

00:27:09.590 --> 00:27:13.050
So I did that and up
popped this dialogue box.

00:27:13.050 --> 00:27:14.960
It says, "Warning--
Visiting this site may

00:27:14.960 --> 00:27:16.130
harm your computer.

00:27:16.130 --> 00:27:20.710
This website you are visiting
appears to contain malware."

00:27:20.710 --> 00:27:24.290
Has anyone seen this before?

00:27:24.290 --> 00:27:26.390
Do you know what
it was what it is?

00:27:26.390 --> 00:27:27.478
What's going on?

00:27:27.478 --> 00:27:30.894
AUDIENCE: So that's probably
your antivirus software,

00:27:30.894 --> 00:27:31.870
correct?

00:27:31.870 --> 00:27:32.980
PROFESSOR: Close.

00:27:32.980 --> 00:27:35.140
It's not my antivirus
software, but it's

00:27:35.140 --> 00:27:36.755
sort of a similar concept.

00:27:36.755 --> 00:27:37.630
In the back, I heard.

00:27:37.630 --> 00:27:39.270
AUDIENCE: I would
bet this is Chrome.

00:27:39.270 --> 00:27:41.020
PROFESSOR: Chrome.

00:27:41.020 --> 00:27:43.500
Yeah, so in this case I
believe I was using Chrome.

00:27:43.500 --> 00:27:45.690
But effectively
what's going on is

00:27:45.690 --> 00:27:50.640
Google has something they call
the Safe Web Browsing service.

00:27:50.640 --> 00:27:53.440
So actually, the guy who did
this is Neil [INAUDIBLE].

00:27:53.440 --> 00:27:56.100
He's one of the lead programmers
for, I believe, OpenSSH.

00:27:56.100 --> 00:27:58.060
He's actually from Michigan.

00:27:58.060 --> 00:27:59.820
But he created this
service at Google

00:27:59.820 --> 00:28:01.380
that goes around
the internet just

00:28:01.380 --> 00:28:05.400
downloading random executables
and then running them.

00:28:05.400 --> 00:28:07.550
And what's interesting is
they create a whole bunch

00:28:07.550 --> 00:28:08.540
of virtual machines.

00:28:08.540 --> 00:28:09.609
This is my understanding.

00:28:09.609 --> 00:28:11.650
I may be misrepresenting
it, but my understanding

00:28:11.650 --> 00:28:13.691
is they create a whole
bunch of virtual machines,

00:28:13.691 --> 00:28:15.540
download those executables,
and just run it

00:28:15.540 --> 00:28:17.680
and then see if the virtual
machine gets infected.

00:28:17.680 --> 00:28:19.470
And if the virtual
machine gets infected,

00:28:19.470 --> 00:28:22.509
you flag that website
as distributing malware.

00:28:22.509 --> 00:28:24.300
They don't know the
intentions necessarily,

00:28:24.300 --> 00:28:26.535
but it's a participant in
the malware distribution.

00:28:29.552 --> 00:28:31.510
This is what you might
call drive-by downloads.

00:28:31.510 --> 00:28:33.860
It's a very common
way of getting malware

00:28:33.860 --> 00:28:37.190
to you on the internet,
especially with the spammers,

00:28:37.190 --> 00:28:40.164
and some of the organized crime.

00:28:40.164 --> 00:28:42.705
But in this case their website
appears have been infiltrated,

00:28:42.705 --> 00:28:45.195
and instead of sending me the
ventilator software update,

00:28:45.195 --> 00:28:46.850
they were giving me malware.

00:28:46.850 --> 00:28:50.270
And at least according
to the Google website,

00:28:50.270 --> 00:28:54.310
it says that over
the past 90 days,

00:28:54.310 --> 00:28:56.570
that's what the website
was resulting in.

00:28:56.570 --> 00:28:58.610
So all I could think
was, all right,

00:28:58.610 --> 00:29:00.365
so if there's an FDA
recall, and you're

00:29:00.365 --> 00:29:02.920
a biomedical engineer
working for a hospital,

00:29:02.920 --> 00:29:04.820
and your job is to
keep your hospital

00:29:04.820 --> 00:29:06.760
medical devices
safe and effective.

00:29:06.760 --> 00:29:08.510
You're going to go
download that software.

00:29:08.510 --> 00:29:11.770
So which box do you
think they clicked?

00:29:11.770 --> 00:29:17.680
Do you think they
clicked close or ignore?

00:29:17.680 --> 00:29:18.180
Right?

00:29:18.180 --> 00:29:22.770
I am sure, I would bet you
dollars to donuts, 99% of them

00:29:22.770 --> 00:29:23.820
clicked ignore.

00:29:23.820 --> 00:29:24.320
Right?

00:29:24.320 --> 00:29:26.200
And so all I'm
imagining now is we've

00:29:26.200 --> 00:29:28.074
got thousands of
clinical engineers

00:29:28.074 --> 00:29:30.240
and biomedical engineers
walking around with malware

00:29:30.240 --> 00:29:32.710
on their laptops in hospitals.

00:29:32.710 --> 00:29:35.320
Hopefully not on the
ventilator, but most likely

00:29:35.320 --> 00:29:36.490
on their local computer.

00:29:39.490 --> 00:29:42.310
So other fun things
you can do is

00:29:42.310 --> 00:29:45.330
you can go search the
MAUDE database for keywords

00:29:45.330 --> 00:29:48.010
like computer virus and
see what's in there.

00:29:48.010 --> 00:29:49.990
And these are all
narratives submitted

00:29:49.990 --> 00:29:51.350
by hospitals and manufacturers.

00:29:51.350 --> 00:29:53.440
One of the more
interesting ones is

00:29:53.440 --> 00:29:55.670
something called a compounder.

00:29:55.670 --> 00:29:57.210
So I have one of
these in my lab.

00:29:57.210 --> 00:29:58.920
It's kind of hard to get.

00:29:58.920 --> 00:30:00.550
But it makes liquid drugs.

00:30:00.550 --> 00:30:04.560
So it has I think on the
order of 16 ports on the top,

00:30:04.560 --> 00:30:06.450
where you can have
the little serums,

00:30:06.450 --> 00:30:09.689
and then it deposits
it into a saline bag.

00:30:09.689 --> 00:30:11.980
And then you can use IV
delivery to deliver it directly

00:30:11.980 --> 00:30:13.210
to your veins.

00:30:13.210 --> 00:30:16.750
So many hospitals will
have these for custom, just

00:30:16.750 --> 00:30:19.290
in time drug delivery,
special cocktails of drugs

00:30:19.290 --> 00:30:20.660
for patients.

00:30:20.660 --> 00:30:22.200
And what was
interesting is here,

00:30:22.200 --> 00:30:23.783
there was a report
that the compounder

00:30:23.783 --> 00:30:25.440
was infected with a virus.

00:30:25.440 --> 00:30:26.140
OK?

00:30:26.140 --> 00:30:28.650
So we bought that
compounder, and we found

00:30:28.650 --> 00:30:30.640
it runs Windows XP embedded.

00:30:30.640 --> 00:30:31.880
Surprise.

00:30:31.880 --> 00:30:34.282
And so it was
vulnerable to malware,

00:30:34.282 --> 00:30:35.990
all the malware that
any other Windows XP

00:30:35.990 --> 00:30:37.700
box would be vulnerable to.

00:30:37.700 --> 00:30:39.450
But what was a little
bit surprising to me

00:30:39.450 --> 00:30:41.522
was manufacturer
response at the time.

00:30:41.522 --> 00:30:43.480
I hope they changed their
tune, but at the time

00:30:43.480 --> 00:30:45.810
they said, "Well,
we do not regularly

00:30:45.810 --> 00:30:49.100
install operating system
updates or patches."

00:30:49.100 --> 00:30:51.320
This struck me as whoa, what?

00:30:51.320 --> 00:30:52.720
What do you mean?

00:30:52.720 --> 00:30:55.260
I said maybe they
had a bit flip.

00:30:55.260 --> 00:30:58.850
But there's a huge
misunderstanding

00:30:58.850 --> 00:31:01.474
about expectations
of software updates.

00:31:01.474 --> 00:31:02.140
Let me be clear.

00:31:02.140 --> 00:31:07.240
FDA expects manufacturers to
keep the software up to date.

00:31:07.240 --> 00:31:09.990
But many manufacturers
will claim

00:31:09.990 --> 00:31:13.090
that they are not able to do
updates because of some FDA

00:31:13.090 --> 00:31:14.680
nonexistent rules.

00:31:14.680 --> 00:31:16.930
So if you ever run into a
medical device manufacturer,

00:31:16.930 --> 00:31:19.260
and they claim that the
FDA rules prevent them

00:31:19.260 --> 00:31:20.860
from doing software
updates, just

00:31:20.860 --> 00:31:23.180
tell them, no,
actually that's untrue.

00:31:23.180 --> 00:31:26.360
And Professor Freeman
created a poster for this.

00:31:26.360 --> 00:31:27.370
So here we go.

00:31:27.370 --> 00:31:29.560
"Homework prevents me
from passing class,

00:31:29.560 --> 00:31:31.490
eHarmony prevents me
from getting dates,

00:31:31.490 --> 00:31:33.760
and yes, FDA rules
prevent software updates.

00:31:33.760 --> 00:31:34.550
Yeah, right.

00:31:34.550 --> 00:31:36.260
Bull."

00:31:36.260 --> 00:31:40.310
So it is true that issuing a
software update takes effort.

00:31:40.310 --> 00:31:41.670
It takes engineering time.

00:31:41.670 --> 00:31:43.090
It's not a simple process.

00:31:43.090 --> 00:31:45.790
It's not like-- I don't know
what course it's called these

00:31:45.790 --> 00:31:49.300
days, 6.170, what it's
become-- but it's not as simple

00:31:49.300 --> 00:31:52.540
as typing "make" and then
submit to the auto-grader.

00:31:52.540 --> 00:31:54.940
There's a huge amount of
verification and validation

00:31:54.940 --> 00:31:55.890
that goes on.

00:31:55.890 --> 00:31:57.660
But that's what
you're expected to do

00:31:57.660 --> 00:31:59.800
if you're in the medical
device manufacturing game.

00:31:59.800 --> 00:32:01.925
If you're in that industry,
that's the expectation.

00:32:05.240 --> 00:32:07.890
So a question that
often comes up

00:32:07.890 --> 00:32:10.160
is, do we need to
worry about this?

00:32:10.160 --> 00:32:12.750
And are there any intentional
malicious malfunctions?

00:32:12.750 --> 00:32:14.720
How significant are these?

00:32:14.720 --> 00:32:18.020
And the good news is, I'm not
aware of any specific instance

00:32:18.020 --> 00:32:19.970
where there's been
a targeted attack,

00:32:19.970 --> 00:32:21.940
and I hope none ever happens.

00:32:21.940 --> 00:32:23.500
But I think it'd be
foolish to assume

00:32:23.500 --> 00:32:26.040
that bad people don't exist.

00:32:26.040 --> 00:32:29.070
So if you look back in
history, in 1982, actually,

00:32:29.070 --> 00:32:31.730
there was an incident in
Chicago where somebody

00:32:31.730 --> 00:32:34.020
deliberately tampered with
extra-strength Tylenol

00:32:34.020 --> 00:32:38.020
on the shelves of pharmacies
and inserted cyanide.

00:32:38.020 --> 00:32:41.340
A number of people
ingested it and died.

00:32:41.340 --> 00:32:42.902
A short time later,
at the funeral,

00:32:42.902 --> 00:32:44.985
additional members of
family used the same bottle.

00:32:44.985 --> 00:32:46.560
They also died.

00:32:46.560 --> 00:32:50.710
Within days, the US
had pulled Tylenol

00:32:50.710 --> 00:32:52.460
from all the shelves
in the United States.

00:32:52.460 --> 00:32:55.650
You could not find Tylenol
in the United States.

00:32:55.650 --> 00:32:58.270
And within one year, Congress
had passed new legislation

00:32:58.270 --> 00:33:01.295
requiring tamper-evident
packaging and physical security

00:33:01.295 --> 00:33:03.020
of over-the-counter drugs.

00:33:03.020 --> 00:33:05.530
This incident is the reason
when you open up your medicine,

00:33:05.530 --> 00:33:07.610
you see a little metal foil.

00:33:07.610 --> 00:33:10.790
So we know bad people exist.

00:33:10.790 --> 00:33:14.620
The cases that we are aware
of are more about tomfoolery,

00:33:14.620 --> 00:33:15.950
but still dangerous.

00:33:15.950 --> 00:33:18.590
So this woman said she had
one of the worst seizure she's

00:33:18.590 --> 00:33:20.210
ever experienced
when somebody decided

00:33:20.210 --> 00:33:22.900
to post flashing animations
on an epilepsy support group

00:33:22.900 --> 00:33:24.090
website.

00:33:24.090 --> 00:33:25.497
So quite malicious.

00:33:25.497 --> 00:33:27.955
It was probably someone who
didn't realize the ramification

00:33:27.955 --> 00:33:30.410
of their actions,
because you can actually

00:33:30.410 --> 00:33:34.140
severely harm a patient who's
sensitive to those kinds

00:33:34.140 --> 00:33:35.180
of things.

00:33:35.180 --> 00:33:37.900
But again, bad people do exist.

00:33:37.900 --> 00:33:41.150
So one of the problems
with the culture gap

00:33:41.150 --> 00:33:43.600
is that much of medical
device manufacturing

00:33:43.600 --> 00:33:46.150
thinks statistically,
and they think

00:33:46.150 --> 00:33:48.960
about past performance
of a device predicting

00:33:48.960 --> 00:33:50.430
future performance.

00:33:50.430 --> 00:33:52.610
So in the security world,
we know that actually,

00:33:52.610 --> 00:33:55.110
if you see no security problems,
that might be because there

00:33:55.110 --> 00:33:56.780
are a bunch more to come soon.

00:33:56.780 --> 00:33:59.030
So if you take a look at the
Mac, for instance, right?

00:33:59.030 --> 00:34:03.030
Before two years ago, basically
no malware was on the Mac.

00:34:03.030 --> 00:34:05.400
But then one night over
half a million Macs

00:34:05.400 --> 00:34:07.840
got infected by Flashback.

00:34:07.840 --> 00:34:10.840
So one of the problems is
bridging that culture gap.

00:34:10.840 --> 00:34:12.299
To move from,
well, there haven't

00:34:12.299 --> 00:34:13.840
been any reported
problems yet, so we

00:34:13.840 --> 00:34:17.010
don't need to worry about it,
to explaining more about how

00:34:17.010 --> 00:34:19.170
to fit security into
the risk management

00:34:19.170 --> 00:34:22.380
thinking of medical
device manufacturing.

00:34:22.380 --> 00:34:24.520
So hopefully we can
avoid this, and keep

00:34:24.520 --> 00:34:29.980
that to be on the Weekly World
News, but it could happen.

00:34:29.980 --> 00:34:32.620
So trying to bring
that analogy home now.

00:34:32.620 --> 00:34:35.909
Before we get into a little
bit more on the solutions here,

00:34:35.909 --> 00:34:39.060
is that way back when,
there was a lot of denial

00:34:39.060 --> 00:34:40.690
that hand washing was a problem.

00:34:40.690 --> 00:34:42.840
But there was a real
reason for that.

00:34:42.840 --> 00:34:45.699
In the 1800s, running
water was not exactly

00:34:45.699 --> 00:34:47.710
common in hospitals.

00:34:47.710 --> 00:34:49.449
Latex gloves did not exist yet.

00:34:49.449 --> 00:34:53.110
So to ask someone to merely wash
their hands for each procedure

00:34:53.110 --> 00:34:55.389
was actually a
pretty tall order.

00:34:55.389 --> 00:34:57.990
And the same thing can be
said of security today,

00:34:57.990 --> 00:34:59.290
in almost any context.

00:34:59.290 --> 00:35:01.970
There's no magic pixie
dust you can sprinkle.

00:35:01.970 --> 00:35:05.150
There are no magic latex
gloves you can put to somehow

00:35:05.150 --> 00:35:06.870
magically add security.

00:35:06.870 --> 00:35:09.710
So when you ask a
manufacturer or clinician

00:35:09.710 --> 00:35:11.580
to, say, keep your
device secure,

00:35:11.580 --> 00:35:12.700
it's a pretty tall order.

00:35:12.700 --> 00:35:14.534
So it's going to take
some time, I think.

00:35:14.534 --> 00:35:15.950
But if they were
alive today, they

00:35:15.950 --> 00:35:17.840
might be saying medical
devices should be secure,

00:35:17.840 --> 00:35:20.131
and doctors are gentleman
and therefore their computers

00:35:20.131 --> 00:35:21.550
are secure.

00:35:21.550 --> 00:35:24.810
But I'm optimistic we're
going to get there,

00:35:24.810 --> 00:35:27.430
because most manufacturers
I talk to now realize it's

00:35:27.430 --> 00:35:28.772
a real problem.

00:35:28.772 --> 00:35:30.980
They're just not necessarily
sure on what to do next.

00:35:30.980 --> 00:35:33.730
So maybe they'll be hiring
you people for the future,

00:35:33.730 --> 00:35:36.250
to help them solve
these security problems.

00:35:36.250 --> 00:35:38.340
But what it all
boils down to is it's

00:35:38.340 --> 00:35:40.460
very difficult to add
security on after the fact.

00:35:40.460 --> 00:35:42.330
Bolting it on is
very challenging.

00:35:42.330 --> 00:35:45.070
It's possible in some
cases, but it's really hard,

00:35:45.070 --> 00:35:46.620
and often very expensive.

00:35:46.620 --> 00:35:48.953
And you've really got to
design it in from the beginning

00:35:48.953 --> 00:35:49.760
to get it right.

00:35:49.760 --> 00:35:51.774
So FDA is expecting
manufacturers

00:35:51.774 --> 00:35:53.190
to get it right
when they're still

00:35:53.190 --> 00:35:55.160
working with pen and
paper, on whiteboards,

00:35:55.160 --> 00:35:56.660
before they've
actually manufactured

00:35:56.660 --> 00:35:59.950
the medical device.

00:35:59.950 --> 00:36:04.570
So how are we doing on time?

00:36:04.570 --> 00:36:06.400
Oh, quite a bit?

00:36:06.400 --> 00:36:07.690
40 minutes, awesome.

00:36:07.690 --> 00:36:09.187
OK.

00:36:09.187 --> 00:36:10.520
I'm going faster than I thought.

00:36:10.520 --> 00:36:12.871
Sorry if you're taking notes.

00:36:12.871 --> 00:36:16.211
I'll talk slower now.

00:36:16.211 --> 00:36:18.730
I want to talk a little
bit about technology

00:36:18.730 --> 00:36:22.010
to make a medical devices
actually more trustworthy.

00:36:22.010 --> 00:36:25.380
So I'm going to try to
blow your mind, all right?

00:36:25.380 --> 00:36:32.450
So why do you trust the sensor
on, let's say, your smartphone?

00:36:32.450 --> 00:36:33.782
You've got a smartphone there.

00:36:33.782 --> 00:36:35.781
Do you know what sensors
are on that smartphone?

00:36:38.547 --> 00:36:39.340
AUDIENCE: GPS.

00:36:39.340 --> 00:36:40.298
PROFESSOR: There's GPS?

00:36:42.510 --> 00:36:44.339
Accelerometer, I heard.

00:36:44.339 --> 00:36:45.130
Any other thoughts?

00:36:45.130 --> 00:36:47.380
What else would we
find on a phone?

00:36:47.380 --> 00:36:48.350
AUDIENCE: Compass.

00:36:48.350 --> 00:36:49.280
PROFESSOR: Compass?

00:36:49.280 --> 00:36:50.111
Light?

00:36:50.111 --> 00:36:51.955
AUDIENCE: [INAUDIBLE].

00:36:51.955 --> 00:36:54.737
PROFESSOR:
Electromagnetic field?

00:36:54.737 --> 00:36:56.195
Everything's
temperature-sensitive.

00:36:59.000 --> 00:37:01.197
Camera's technically
got a CCD sensor.

00:37:01.197 --> 00:37:02.780
So there's sensors
all over the place.

00:37:02.780 --> 00:37:04.380
Medical devices
have sensors, too.

00:37:04.380 --> 00:37:06.170
Now, why do you trust
what the sensor's

00:37:06.170 --> 00:37:07.140
telling your processor?

00:37:07.140 --> 00:37:10.180
If you write software
and your sensor

00:37:10.180 --> 00:37:15.210
tells you it's 77 degrees
today, or 25 Celsius,

00:37:15.210 --> 00:37:17.960
why do you believe that?

00:37:17.960 --> 00:37:20.650
So at least in my lab, we
do a lot of work on sensors.

00:37:20.650 --> 00:37:22.580
So I try to pass
this one around.

00:37:22.580 --> 00:37:25.980
This is a batteryless sensor.

00:37:25.980 --> 00:37:29.482
It's got an MSP430
microcontroller.

00:37:29.482 --> 00:37:30.440
But there's no battery.

00:37:30.440 --> 00:37:32.700
It actually runs off a
10 microfarad capacitor,

00:37:32.700 --> 00:37:35.876
and it harvests RF energy to
power up that microprocessor.

00:37:35.876 --> 00:37:39.300
I'll pass it up
this side, I guess.

00:37:39.300 --> 00:37:41.030
And it's got all the
fun little things

00:37:41.030 --> 00:37:43.870
like a 3D accelerometer,
temperature sensors, light,

00:37:43.870 --> 00:37:45.470
all that fun stuff.

00:37:45.470 --> 00:37:48.029
But it's really
hard to power up.

00:37:48.029 --> 00:37:49.820
But again, how do you
trust what's actually

00:37:49.820 --> 00:37:51.110
coming into that sensor?

00:37:51.110 --> 00:37:54.530
Something's translating it from
all these physical phenomena

00:37:54.530 --> 00:37:56.390
to little electrical pulses.

00:37:56.390 --> 00:38:00.534
So one thing I want
to highlight is

00:38:00.534 --> 00:38:02.117
why you might not
want to trust what's

00:38:02.117 --> 00:38:03.460
coming out of that sensor.

00:38:03.460 --> 00:38:05.890
So this is work from one of
my post-docs, Denis Foo Kune

00:38:05.890 --> 00:38:08.369
here, who's kiteboarding
on Lake Michigan.

00:38:08.369 --> 00:38:09.910
But in his other
spare time, he likes

00:38:09.910 --> 00:38:11.700
to interfere with sensors.

00:38:11.700 --> 00:38:14.370
So let me tell you about--
forget security for a moment,

00:38:14.370 --> 00:38:17.295
to safety-- there was a
gentleman in 2009 who reported

00:38:17.295 --> 00:38:19.420
that every time his cell
phone rang in his kitchen,

00:38:19.420 --> 00:38:21.214
his oven turned on.

00:38:21.214 --> 00:38:23.130
So you can go find this
in the New York Times.

00:38:23.130 --> 00:38:26.460
It just happened to be that
that resonant frequency was just

00:38:26.460 --> 00:38:30.100
perfect to get that ignition
to go off in the over.

00:38:30.100 --> 00:38:32.360
So there's interference
all over the place.

00:38:32.360 --> 00:38:35.950
It's a constant battle, because
we have different devices

00:38:35.950 --> 00:38:37.720
speaking in the same spectrum.

00:38:37.720 --> 00:38:41.090
But there are technologies
to reduce that interference.

00:38:41.090 --> 00:38:44.260
The problem is, what happens
when the interference is

00:38:44.260 --> 00:38:45.740
in the baseband?

00:38:45.740 --> 00:38:47.990
I'm going to go a little bit
analog on you for moment.

00:38:47.990 --> 00:38:51.110
So does 6.003 still exist?

00:38:51.110 --> 00:38:51.610
It does?

00:38:51.610 --> 00:38:52.370
OK, good.

00:38:52.370 --> 00:38:54.411
So I encourage you all to
take it if you haven't.

00:38:54.411 --> 00:38:56.850
It's one of the most awesome
courses for a CS person,

00:38:56.850 --> 00:39:00.250
because you don't have to
go too deep into the circus.

00:39:00.250 --> 00:39:02.112
So what was
interesting to me was,

00:39:02.112 --> 00:39:04.070
I was trying to understand
why I should believe

00:39:04.070 --> 00:39:05.510
what a sensor's telling me.

00:39:05.510 --> 00:39:07.740
And so I started to look
at the block diagram.

00:39:07.740 --> 00:39:11.154
And so for instance, if you've
got a Bluetooth headset,

00:39:11.154 --> 00:39:13.570
what you're going to find
inside that Bluetooth headset is

00:39:13.570 --> 00:39:18.260
a microphone, piece of wire,
an amplifier-- right, 003--

00:39:18.260 --> 00:39:21.335
some more wire, or
some traces on a PCB.

00:39:21.335 --> 00:39:22.960
It goes to an
analog/digital converter.

00:39:22.960 --> 00:39:24.750
There might be some filtering.

00:39:24.750 --> 00:39:27.000
And then it goes to
your microprocessor.

00:39:27.000 --> 00:39:29.590
But there's all this other
stuff that gets in the way

00:39:29.590 --> 00:39:31.320
before it gets to your software.

00:39:31.320 --> 00:39:33.160
And for some reason,
your software

00:39:33.160 --> 00:39:36.820
just believes anything
this wire says.

00:39:36.820 --> 00:39:39.247
So what was interesting to
me was, well, you know what?

00:39:39.247 --> 00:39:41.580
That piece of wire from the
microphone to the amplifier,

00:39:41.580 --> 00:39:42.350
it has a length.

00:39:42.350 --> 00:39:44.750
It also has a
resonant frequency.

00:39:44.750 --> 00:39:48.650
So what would happen if somebody
generates custom electromagnet

00:39:48.650 --> 00:39:51.580
interference that's
optimized to latch

00:39:51.580 --> 00:39:53.820
onto that resonant frequency
of that piece of wire?

00:39:53.820 --> 00:39:55.319
Well, it would go
into the amplifier

00:39:55.319 --> 00:39:57.140
and it would get amplified.

00:39:57.140 --> 00:39:59.670
And then it would go into
that analog/digital converter,

00:39:59.670 --> 00:40:02.140
and you'd pass onto
the microprocessor.

00:40:02.140 --> 00:40:05.280
One of the questions we had
was, was this possible at all?

00:40:05.280 --> 00:40:07.560
And if so, how hard would it be?

00:40:07.560 --> 00:40:10.050
What kind of power
would you need to do it?

00:40:10.050 --> 00:40:12.840
And what would be the quality
of the signal that actually

00:40:12.840 --> 00:40:15.020
reaches the microprocessor?

00:40:15.020 --> 00:40:18.460
So the fundamental reason
why this is even possible

00:40:18.460 --> 00:40:21.780
is because we're talking
about intentional, as opposed

00:40:21.780 --> 00:40:23.520
to accidental
interference, we're

00:40:23.520 --> 00:40:24.770
throwing it into the baseband.

00:40:24.770 --> 00:40:27.320
So here's an example.

00:40:27.320 --> 00:40:29.040
Imagine that your
medical device is

00:40:29.040 --> 00:40:32.140
designed to accept physiologic
signals in the low hertz.

00:40:32.140 --> 00:40:34.330
Like your heart
doesn't beat that fast.

00:40:34.330 --> 00:40:37.180
We're talking a
few hertz or less.

00:40:37.180 --> 00:40:39.712
So if your electrodes were to
pick up some high frequency

00:40:39.712 --> 00:40:41.670
signals, you'd just put
in some analog filters.

00:40:41.670 --> 00:40:43.566
You'd say, that
cannot be real, right?

00:40:43.566 --> 00:40:44.982
If your heart's
beating that fast,

00:40:44.982 --> 00:40:48.370
you're probably just
picking up something

00:40:48.370 --> 00:40:52.630
like an electric mixer while
you're making your lunch.

00:40:52.630 --> 00:40:55.630
So similarly you can filter out
pulses in the high frequency.

00:40:55.630 --> 00:40:58.450
But if you send interference
that's in the baseband,

00:40:58.450 --> 00:41:00.520
those filters are going
to be meaningless.

00:41:00.520 --> 00:41:02.730
Because those analog
filters cannot get rid

00:41:02.730 --> 00:41:08.750
of if it's in the same frequency
area as what you're expecting.

00:41:08.750 --> 00:41:11.970
So it's hard to
filter in the analog.

00:41:11.970 --> 00:41:13.890
So I'm going to go
through a couple examples.

00:41:13.890 --> 00:41:15.806
We're going to start
with a Bluetooth headset,

00:41:15.806 --> 00:41:18.440
and then work our way
up to a medical device.

00:41:18.440 --> 00:41:21.750
So Denis, he built a bunch
of homebrew dipole antennas

00:41:21.750 --> 00:41:23.770
and transmitters and amplifiers.

00:41:23.770 --> 00:41:28.282
Now what he's got up here is
you can see he's got a webcam.

00:41:28.282 --> 00:41:30.490
I guess not too many of us
need to buy these anymore,

00:41:30.490 --> 00:41:31.820
because they're built in.

00:41:31.820 --> 00:41:36.100
But that webcam
has a microphone,

00:41:36.100 --> 00:41:38.060
and then it's got
a little USB cable

00:41:38.060 --> 00:41:40.445
to deliver the audio
to the computer.

00:41:40.445 --> 00:41:42.320
So what he's done is
he's set up the computer

00:41:42.320 --> 00:41:45.790
to record the video and
audio and then play it back.

00:41:45.790 --> 00:41:51.230
So what's interesting
is-- you'll see this now.

00:41:51.230 --> 00:41:52.750
He was in a completely
silent room.

00:41:52.750 --> 00:41:53.958
It sort of sounded like this.

00:41:53.958 --> 00:41:57.260
All you could hear was
the ventilation system.

00:41:57.260 --> 00:41:58.230
He's got the camera.

00:41:58.230 --> 00:42:00.021
He removed the housing,
just so it's easier

00:42:00.021 --> 00:42:03.630
to tap in and measure
the interference.

00:42:03.630 --> 00:42:06.045
And then he's got a
software radio about a meter

00:42:06.045 --> 00:42:09.110
away, generating custom
electromagnetic interference.

00:42:09.110 --> 00:42:13.950
He writes it in Python, and
then sends over his signals.

00:42:13.950 --> 00:42:17.210
So here's what the
computer on the left

00:42:17.210 --> 00:42:20.259
thought it heard, even
in this silent room.

00:42:20.259 --> 00:42:21.257
[AUDIO PLAYBACK]

00:42:21.257 --> 00:42:22.756
[MUSIC WEEZER, "ISLAND IN THE
 SUN"]

00:42:30.310 --> 00:42:31.230
[END PLAYBACK]

00:42:31.230 --> 00:42:32.370
PROFESSOR: So yeah.

00:42:32.370 --> 00:42:34.286
The last time I did that,
somebody in the back

00:42:34.286 --> 00:42:37.000
actually started dancing.

00:42:37.000 --> 00:42:39.765
So it's actually
relatively high fidelity.

00:42:39.765 --> 00:42:43.020
And it actually turns out that
in the manufacturing community,

00:42:43.020 --> 00:42:43.990
they're so cheap.

00:42:43.990 --> 00:42:46.880
They use really
cheap microphones

00:42:46.880 --> 00:42:48.240
with poor frequency responses.

00:42:48.240 --> 00:42:50.010
So we actually got
higher quality audio

00:42:50.010 --> 00:42:53.520
through interference than
going to the microphone.

00:42:53.520 --> 00:42:55.520
So if you ever don't like
your Bluetooth headset

00:42:55.520 --> 00:42:57.150
and you want to play
classical music,

00:42:57.150 --> 00:42:58.500
just do it with interference.

00:42:58.500 --> 00:43:02.010
But don't tell the FCC
I told you to do that,

00:43:02.010 --> 00:43:03.370
because you're not supposed to.

00:43:03.370 --> 00:43:06.820
But the point is
if you're talking

00:43:06.820 --> 00:43:08.950
intentional magnetic
interference,

00:43:08.950 --> 00:43:11.330
it's kind of outside
the security model.

00:43:11.330 --> 00:43:13.840
And so your processor
just trusts it.

00:43:13.840 --> 00:43:16.160
So some interesting
things you can do.

00:43:16.160 --> 00:43:18.510
Let's say your office
mate decides to call up

00:43:18.510 --> 00:43:21.135
his bank to make some deposits.

00:43:21.135 --> 00:43:23.170
Well, you can insert DTMF tones.

00:43:23.170 --> 00:43:24.900
That's kind of fun.

00:43:24.900 --> 00:43:26.849
So we were just playing around.

00:43:26.849 --> 00:43:28.640
You can change the
language as the person's

00:43:28.640 --> 00:43:31.274
trying to make deposits
from account to account.

00:43:31.274 --> 00:43:32.690
But there's all
just interference.

00:43:32.690 --> 00:43:35.310
And actually the person
on the Bluetooth headset

00:43:35.310 --> 00:43:36.220
didn't hear it.

00:43:36.220 --> 00:43:42.124
Because remember it's
coming from the person,

00:43:42.124 --> 00:43:44.290
so that it doesn't actually
get echoed back to them.

00:43:44.290 --> 00:43:48.250
But the bank heard it and
made all the transactions.

00:43:48.250 --> 00:43:51.020
So there are ways to do this.

00:43:51.020 --> 00:43:54.070
It doesn't take a whole
bunch of analog skills.

00:43:54.070 --> 00:43:55.800
We're mostly
computer scientists.

00:43:55.800 --> 00:43:59.140
But you do need
to somehow convert

00:43:59.140 --> 00:44:01.505
the signal you want to have
appear at the microprocessor

00:44:01.505 --> 00:44:04.980
into something else
that's easier to transmit.

00:44:04.980 --> 00:44:08.240
So the first thing
you can do is think

00:44:08.240 --> 00:44:09.727
about just
overwhelming the thing

00:44:09.727 --> 00:44:10.810
with a very strong signal.

00:44:10.810 --> 00:44:12.150
That's the brute force approach.

00:44:12.150 --> 00:44:14.922
It doesn't work so well,
but it works a little bit.

00:44:14.922 --> 00:44:16.630
So if you send something
out that matches

00:44:16.630 --> 00:44:18.950
the resonant frequency of
that little piece of wire,

00:44:18.950 --> 00:44:22.110
yeah, that'll get the
job done to some extent.

00:44:22.110 --> 00:44:26.010
The problem is a lot of these
signals are low frequency,

00:44:26.010 --> 00:44:28.880
and it's more
difficult to transmit.

00:44:28.880 --> 00:44:30.172
It's got less power, basically.

00:44:30.172 --> 00:44:32.088
So it's going to be
harder to send the signal.

00:44:32.088 --> 00:44:35.170
So what you really want to do is
send a higher frequency signal,

00:44:35.170 --> 00:44:37.620
and it's going to be easier
to deliver the power.

00:44:37.620 --> 00:44:40.280
But if you send a really
high frequency signal, that's

00:44:40.280 --> 00:44:42.065
going to be outside
the baseband,

00:44:42.065 --> 00:44:44.270
so all the filters
are going to go at it.

00:44:44.270 --> 00:44:47.130
So here's what you do instead.

00:44:47.130 --> 00:44:51.840
You treat this circuit as an
unintentional demodulator.

00:44:51.840 --> 00:44:54.554
So what you do is, we had
that original sine wave

00:44:54.554 --> 00:44:55.470
we wanted to transmit.

00:44:55.470 --> 00:44:58.995
Instead we modulate it onto
a higher frequency sine wave.

00:44:58.995 --> 00:45:02.160
And we send it in
to the amplifier,

00:45:02.160 --> 00:45:04.520
and eventually it's going
to work its way in because

00:45:04.520 --> 00:45:05.530
of sampling theory.

00:45:05.530 --> 00:45:07.960
You can think about
Nyquist and all that.

00:45:07.960 --> 00:45:12.370
So up on the top is
the interfering signal

00:45:12.370 --> 00:45:14.580
we're actually sending,
and then on the bottom

00:45:14.580 --> 00:45:16.240
is what the microprocessor sees.

00:45:16.240 --> 00:45:18.370
Because remember the
analog-to-digital converter

00:45:18.370 --> 00:45:20.390
is not continuously sampling.

00:45:20.390 --> 00:45:22.020
There's an interrupt
on the processor.

00:45:22.020 --> 00:45:24.560
Wake up, take a reading,
wake up, take a reading.

00:45:24.560 --> 00:45:25.935
So it's actually
going to sample,

00:45:25.935 --> 00:45:27.812
and then try to
infer the signal.

00:45:27.812 --> 00:45:29.770
So as we're sending out
our really fast signal,

00:45:29.770 --> 00:45:32.170
it takes a sample,
it takes a sample,

00:45:32.170 --> 00:45:35.180
it takes a sample,
et cetera, et cetera.

00:45:35.180 --> 00:45:37.750
Your microprocessor thinks it
got this nice low frequency

00:45:37.750 --> 00:45:40.180
sine wave, but we actually
used a high frequency one,

00:45:40.180 --> 00:45:44.151
because that allowed us
to transmit more easily.

00:45:44.151 --> 00:45:46.400
So I'm not going to go through
all the nitty-gritties,

00:45:46.400 --> 00:45:49.250
but one another kind
of cool way to do

00:45:49.250 --> 00:45:54.320
this is to muck around with
the non-linear components

00:45:54.320 --> 00:45:55.590
of the circuit.

00:45:55.590 --> 00:45:58.160
But this is all about violating
security models, right?

00:45:58.160 --> 00:46:00.860
So we're completely violating
what the circuit designer

00:46:00.860 --> 00:46:02.760
had intended.

00:46:02.760 --> 00:46:06.650
It turns out that
if you send in, say,

00:46:06.650 --> 00:46:09.440
in this case you're
sending in 826 megahertz

00:46:09.440 --> 00:46:12.540
is the resonant
frequency of our wire.

00:46:12.540 --> 00:46:15.780
But I can't speak that fast.

00:46:15.780 --> 00:46:21.220
So what we do is we modulate
our voice on an 826 megahertz

00:46:21.220 --> 00:46:22.130
carrier.

00:46:22.130 --> 00:46:24.690
Problem is it's going to
get, for instance, all

00:46:24.690 --> 00:46:27.294
this replication of the signal.

00:46:27.294 --> 00:46:28.710
You're going to
see the frequency.

00:46:28.710 --> 00:46:30.334
Here we're looking
at frequency domain.

00:46:30.334 --> 00:46:31.590
It gets repeated.

00:46:31.590 --> 00:46:33.490
But it turns out because
of the filters built

00:46:33.490 --> 00:46:35.240
into most of these
devices, it's actually

00:46:35.240 --> 00:46:37.531
going to chop off
the repeated copies.

00:46:37.531 --> 00:46:39.655
So the end of the day, what
the microprocessor sees

00:46:39.655 --> 00:46:41.900
is our original 1
kilohertz signal

00:46:41.900 --> 00:46:43.070
we were trying to send in.

00:46:43.070 --> 00:46:47.800
It's been unintentionally
demodulated.

00:46:47.800 --> 00:46:49.530
So that's the easiest
example that I've

00:46:49.530 --> 00:46:51.030
been able to come
up with to explain

00:46:51.030 --> 00:46:54.370
the idea of this
intentional interference.

00:46:54.370 --> 00:46:56.950
And now we're going to try
to apply it to defibrillators

00:46:56.950 --> 00:46:58.680
and medical devices.

00:46:58.680 --> 00:47:02.090
So again, the defibrillator's
implanted into the clavicle.

00:47:02.090 --> 00:47:05.520
And it has these electrodes--
you can kind of see them here--

00:47:05.520 --> 00:47:07.640
that go into the
chambers of the heart,

00:47:07.640 --> 00:47:11.190
and it's used for both
sensing and actuation.

00:47:11.190 --> 00:47:13.190
So it's just a signal.

00:47:13.190 --> 00:47:18.310
So this is the time domain, and
this is the Fourier transform,

00:47:18.310 --> 00:47:19.750
effectively.

00:47:19.750 --> 00:47:23.780
So this is a single
heartbeat, and the heartbeat

00:47:23.780 --> 00:47:26.110
is actually quite intricate.

00:47:26.110 --> 00:47:27.860
The physicians have
actually labeled

00:47:27.860 --> 00:47:29.660
the different components
of the heart rate.

00:47:29.660 --> 00:47:31.560
You've got the
QRS complex, which

00:47:31.560 --> 00:47:33.990
is typically what you would
think of as the heartbeat.

00:47:33.990 --> 00:47:36.970
The actual beat is
this giant R here.

00:47:36.970 --> 00:47:38.890
That's the one you'll feel.

00:47:38.890 --> 00:47:41.370
But there are also these
other smaller waves,

00:47:41.370 --> 00:47:46.110
as your tissue is
energizing and relaxing.

00:47:46.110 --> 00:47:49.030
So if do a Fourier transform
on your cardiac rhythm,

00:47:49.030 --> 00:47:51.060
you're going to end up
with most of the signal

00:47:51.060 --> 00:47:53.890
in the tens of hertz.

00:47:53.890 --> 00:47:55.680
You're not going to
see things a whole lot

00:47:55.680 --> 00:47:59.120
beyond 100 hertz in a
typical cardiac signal.

00:47:59.120 --> 00:48:01.531
So most of these
devices are designed

00:48:01.531 --> 00:48:04.030
to filter out things that are
really low frequency or really

00:48:04.030 --> 00:48:05.340
high frequency.

00:48:05.340 --> 00:48:07.865
But if you choose to insert
intentional electromagnetic

00:48:07.865 --> 00:48:09.940
interference on the
baseband, then it

00:48:09.940 --> 00:48:12.840
gets through all the
analog circuit filters.

00:48:12.840 --> 00:48:14.580
And now the only
approach to [INAUDIBLE]

00:48:14.580 --> 00:48:18.010
that would be things more on
the computer science side.

00:48:18.010 --> 00:48:20.270
So this is where
my students began

00:48:20.270 --> 00:48:22.890
to have a little bit of fun.

00:48:22.890 --> 00:48:27.075
So we wanted to test this
in as realistic a situation

00:48:27.075 --> 00:48:27.790
as we could.

00:48:27.790 --> 00:48:30.290
We couldn't get volunteers.

00:48:30.290 --> 00:48:34.030
So instead we discovered there's
actually a national standard.

00:48:34.030 --> 00:48:34.850
This is a body.

00:48:34.850 --> 00:48:35.835
This is you.

00:48:35.835 --> 00:48:38.279
It turns out that we're all
just bags of saline solution.

00:48:38.279 --> 00:48:40.570
And so if you have a highly
calibrated saline solution,

00:48:40.570 --> 00:48:44.740
that's the best way to
simulate human tissue.

00:48:44.740 --> 00:48:48.360
The other thing we've done is
we used the synthetic cadaver.

00:48:48.360 --> 00:48:50.900
She's actually
anatomically correct.

00:48:50.900 --> 00:48:52.400
She's got all the
same vital organs

00:48:52.400 --> 00:48:53.930
as anyone else
would have inside,

00:48:53.930 --> 00:48:55.630
and a working
circulatory system.

00:48:55.630 --> 00:48:57.880
So it has all the surface
properties of the RF.

00:48:57.880 --> 00:49:01.730
So here we're doing
radiation fluoroscopy

00:49:01.730 --> 00:49:04.890
to do 3D imaging-- 4D.

00:49:04.890 --> 00:49:07.716
We see light imaging as we're
implanting the electrodes

00:49:07.716 --> 00:49:08.840
into our synthetic cadaver.

00:49:12.062 --> 00:49:13.770
So what we're going
to do now is generate

00:49:13.770 --> 00:49:15.145
some electromagnetic
interference

00:49:15.145 --> 00:49:17.520
and then try to see what
the device is perceiving

00:49:17.520 --> 00:49:19.610
as a trustworthy signal.

00:49:19.610 --> 00:49:21.010
So a couple ways we did this.

00:49:21.010 --> 00:49:25.710
In the saline solution, we used
just a spool of magnet wire.

00:49:25.710 --> 00:49:28.815
Here we have the wand that's
reading out the telemetry

00:49:28.815 --> 00:49:30.565
to see what the device
thinks it's seeing,

00:49:30.565 --> 00:49:32.240
and then another
experimental case.

00:49:32.240 --> 00:49:34.660
So I had some leftover
pipes from plumbing,

00:49:34.660 --> 00:49:36.760
so we created a dipole antenna.

00:49:36.760 --> 00:49:39.080
And on the back there
on that poster board,

00:49:39.080 --> 00:49:41.190
we created a 2D
version of a patient.

00:49:41.190 --> 00:49:43.890
You can see that's the curvature
of the electrode, that's

00:49:43.890 --> 00:49:46.000
the electrode, and
then the pacemaker

00:49:46.000 --> 00:49:48.300
is right underneath the tape.

00:49:48.300 --> 00:49:50.700
And we're transferring to it.

00:49:50.700 --> 00:49:52.457
So here's what the
device thought it saw,

00:49:52.457 --> 00:49:53.790
even though it wasn't happening.

00:49:53.790 --> 00:49:56.460
So keep in mind this should have
been a flat line, because there

00:49:56.460 --> 00:49:57.100
is no patient.

00:49:57.100 --> 00:49:58.600
There is no heart beating.

00:49:58.600 --> 00:50:01.970
So we tried a couple different
signals of interest are we

00:50:01.970 --> 00:50:03.680
pulsed a sinusoid.

00:50:03.680 --> 00:50:06.020
So that's really a
sine wave, but it's

00:50:06.020 --> 00:50:07.295
so fast you can't quite tell.

00:50:07.295 --> 00:50:09.670
But we pulsed it
like a heart beat.

00:50:09.670 --> 00:50:12.470
So every one second
we sent out a pulse.

00:50:12.470 --> 00:50:15.380
And then we also did
one that's modulated.

00:50:15.380 --> 00:50:17.430
That's a little bit noisier.

00:50:17.430 --> 00:50:21.020
So this is a screenshot of
the pacemaker programmer

00:50:21.020 --> 00:50:24.100
which tells us live what
telemetry is going out.

00:50:24.100 --> 00:50:27.220
And it's hard to read, but
the little green up there,

00:50:27.220 --> 00:50:30.870
VP, says that the device sent
out the ventricular pace.

00:50:30.870 --> 00:50:33.520
This is the pacemaker sending
an artificial heartbeat,

00:50:33.520 --> 00:50:36.449
basically, to make
the tissue contract.

00:50:36.449 --> 00:50:37.990
What's interesting
is when we started

00:50:37.990 --> 00:50:40.320
sending our interference,
it got what's

00:50:40.320 --> 00:50:42.330
called a VS, a
ventricular sense.

00:50:42.330 --> 00:50:44.520
The little purple VS,
there's three of them.

00:50:44.520 --> 00:50:46.400
So the pacemaker thought
that the heart was

00:50:46.400 --> 00:50:48.710
beating on its own, so it
chose to inhibit the pacing

00:50:48.710 --> 00:50:50.446
to save power.

00:50:50.446 --> 00:50:52.320
And then when we turned
off the interference,

00:50:52.320 --> 00:50:54.240
the pacing began again.

00:50:54.240 --> 00:50:57.840
Similarly over here you see
where the interference starts,

00:50:57.840 --> 00:51:00.130
and it's sensing
ventricular sense.

00:51:00.130 --> 00:51:03.140
It says, oh, the body's
pacing itself naturally.

00:51:03.140 --> 00:51:05.840
I don't need to waste my
energy pacing the heart.

00:51:05.840 --> 00:51:08.070
So we're able to induce
that interference,

00:51:08.070 --> 00:51:09.910
and then trick
the microprocessor

00:51:09.910 --> 00:51:13.560
into believing the long state.

00:51:13.560 --> 00:51:15.030
There is a silver
lining, though.

00:51:15.030 --> 00:51:18.900
The good news is, that
only works in vitro.

00:51:18.900 --> 00:51:21.800
Whenever we would do
this in saline solution

00:51:21.800 --> 00:51:23.920
or in anything that
approximated the body,

00:51:23.920 --> 00:51:25.400
it basically didn't work.

00:51:25.400 --> 00:51:26.900
And that's because
your body absorbs

00:51:26.900 --> 00:51:30.800
a lot of that RF energy,
and it doesn't actually

00:51:30.800 --> 00:51:31.750
get to the sensor.

00:51:31.750 --> 00:51:34.010
So the closest we were
able to get this to work

00:51:34.010 --> 00:51:39.180
was, with the saline,
like three centimeters.

00:51:39.180 --> 00:51:41.540
So that basically
means there's no worry

00:51:41.540 --> 00:51:44.370
for this particular kind of
interference from an implant.

00:51:44.370 --> 00:51:47.610
However, an externally
worn device, we don't know.

00:51:47.610 --> 00:51:50.600
We hadn't done any tests
on insulin pumps yet.

00:51:50.600 --> 00:51:52.100
There are plenty
of different kinds.

00:51:52.100 --> 00:51:53.683
There's glucose
sensors, for instance,

00:51:53.683 --> 00:51:57.970
that are percutaneous.

00:51:57.970 --> 00:52:00.010
I wouldn't be surprised
if someone here has one.

00:52:00.010 --> 00:52:01.610
They're pretty common.

00:52:01.610 --> 00:52:04.540
But we just don't know yet.

00:52:04.540 --> 00:52:10.210
But one of the approaches
we're taking to solve this

00:52:10.210 --> 00:52:12.630
follows the end-to-end
principle to some extent.

00:52:12.630 --> 00:52:14.607
A lot of these, I
just don't think

00:52:14.607 --> 00:52:16.190
the analog is able
to distinguish good

00:52:16.190 --> 00:52:17.520
from bad signal.

00:52:17.520 --> 00:52:20.340
And so you have to do it closer
to the application layer.

00:52:20.340 --> 00:52:27.440
So one of the defenses that we
tried out was the following.

00:52:27.440 --> 00:52:30.694
It has its own limitations,
but here's the basic idea.

00:52:30.694 --> 00:52:32.860
So imagine you're a pacemaker,
and you want to know,

00:52:32.860 --> 00:52:35.000
are you getting a
trustworthy signal?

00:52:35.000 --> 00:52:37.370
So what you do is
you selectively

00:52:37.370 --> 00:52:41.930
choose to send test pulses every
now and then, to basically keep

00:52:41.930 --> 00:52:43.100
the adversary in check.

00:52:43.100 --> 00:52:45.890
So here's the interesting thing
we discovered when we worked

00:52:45.890 --> 00:52:47.440
with electrophysiologists.

00:52:47.440 --> 00:52:49.340
We learned that if you
send a pacing pulse

00:52:49.340 --> 00:52:52.830
to a heart that recently was
beating, within about 200

00:52:52.830 --> 00:52:55.750
milliseconds, that cardiac
tissue is physically

00:52:55.750 --> 00:52:57.360
incapable of beating again.

00:52:57.360 --> 00:53:00.110
It's also physically
incapable of sending out

00:53:00.110 --> 00:53:01.800
an electrical
response, just because

00:53:01.800 --> 00:53:05.300
of the polarization-- the way
that cardiac tissue works.

00:53:05.300 --> 00:53:08.360
So we said, so what would happen
if we send an extra pacing

00:53:08.360 --> 00:53:11.230
pulse right after a
ventricular sense?

00:53:11.230 --> 00:53:14.700
He said, oh, well,
if the heart actually

00:53:14.700 --> 00:53:17.310
had beat, as your
sensor told you,

00:53:17.310 --> 00:53:18.950
then you should get no response.

00:53:18.950 --> 00:53:21.400
Because it would be incapable
of sending a response.

00:53:21.400 --> 00:53:23.860
So therefore, if we
saw the heart send us

00:53:23.860 --> 00:53:26.160
an electrical signal
back, we knew-- then

00:53:26.160 --> 00:53:28.190
that proves to us
that we were fooled

00:53:28.190 --> 00:53:29.720
on the previous heartbeat.

00:53:29.720 --> 00:53:32.910
And there we raise
our warning signs

00:53:32.910 --> 00:53:34.450
that it appears to
be we're getting

00:53:34.450 --> 00:53:36.550
intentional electromagnetic
interference.

00:53:36.550 --> 00:53:38.760
So the basic idea is,
again, we probe it,

00:53:38.760 --> 00:53:40.450
and we make use
of some of what we

00:53:40.450 --> 00:53:42.740
know about the
physiology of the body

00:53:42.740 --> 00:53:45.020
to have better trustworthiness.

00:53:45.020 --> 00:53:48.170
Another approach we didn't
look into too deeply

00:53:48.170 --> 00:53:50.223
was looking at
propagation delay.

00:53:50.223 --> 00:53:52.440
Because if you have
electromagnetic interference

00:53:52.440 --> 00:53:54.620
coming at you, it's
basically light, right?

00:53:54.620 --> 00:53:55.810
Speed of light.

00:53:55.810 --> 00:53:59.370
And if it's hitting you all at
once, if you have two sensors,

00:53:59.370 --> 00:54:02.240
and you simultaneously see
the same cardiac signal

00:54:02.240 --> 00:54:04.890
at the same time,
something's wrong.

00:54:04.890 --> 00:54:07.480
Because there's an
electrochemical propagation

00:54:07.480 --> 00:54:10.115
delay from your vagus nerve
as the electrical signal

00:54:10.115 --> 00:54:12.390
is traveling down
through your heart.

00:54:12.390 --> 00:54:14.430
So there are other ways
to try to tease out

00:54:14.430 --> 00:54:18.050
whether the physiologic
signal is trustworthy,

00:54:18.050 --> 00:54:19.960
but this is new ground.

00:54:19.960 --> 00:54:22.610
There's not a lot going
on in this space yet.

00:54:22.610 --> 00:54:25.191
A lot of fun projects for
graduate and undergraduate

00:54:25.191 --> 00:54:25.690
research.

00:54:29.300 --> 00:54:33.070
We end at-- oh, 25?

00:54:33.070 --> 00:54:33.830
Oh.

00:54:33.830 --> 00:54:38.095
So I want to tell you
about another project,

00:54:38.095 --> 00:54:41.350
and that is detecting
malware at power outlets.

00:54:41.350 --> 00:54:47.030
So a few years ago, one of
my students, Shane, he said,

00:54:47.030 --> 00:54:50.160
hey, I built this power
outlet, and I can tell

00:54:50.160 --> 00:54:52.630
what website you're browsing.

00:54:52.630 --> 00:54:57.530
So he put a little
sense resistor in here,

00:54:57.530 --> 00:54:59.810
and he measures what's
called the phase

00:54:59.810 --> 00:55:01.850
shift in the reactive power.

00:55:01.850 --> 00:55:05.980
It's basically a proxy for
the load on your computer.

00:55:05.980 --> 00:55:08.980
And he can basically tell
how your computer-- how

00:55:08.980 --> 00:55:11.670
your processor is
changing the load as it's

00:55:11.670 --> 00:55:14.570
going out onto the
AC power system.

00:55:14.570 --> 00:55:15.560
This is not new.

00:55:15.560 --> 00:55:17.539
Has anyone heard of Tempest?

00:55:17.539 --> 00:55:18.330
Tempest protection?

00:55:18.330 --> 00:55:19.130
A few of you.

00:55:19.130 --> 00:55:22.230
So Tempest has been
around for years.

00:55:22.230 --> 00:55:25.620
Basically signals leak
all over the place,

00:55:25.620 --> 00:55:27.750
and so there's a whole fine
art to stopping signals

00:55:27.750 --> 00:55:29.730
from leaking.

00:55:29.730 --> 00:55:31.310
What was interesting
to me was, I

00:55:31.310 --> 00:55:33.143
like to keep all my old
computers-- actually

00:55:33.143 --> 00:55:35.670
I have an exokernel machine.

00:55:35.670 --> 00:55:39.307
And it's an old-- I
think it's a Pentium 4.

00:55:39.307 --> 00:55:41.640
And this was before there was
advanced power management.

00:55:41.640 --> 00:55:44.139
So if you measured the power
coming out of this old Pentium,

00:55:44.139 --> 00:55:44.942
it was constant.

00:55:44.942 --> 00:55:46.900
Just doesn't matter if
you were doing anything.

00:55:46.900 --> 00:55:49.950
If you have a spin while
loop, doesn't matter.

00:55:49.950 --> 00:55:53.600
It's the same thing as
actually doing processing.

00:55:53.600 --> 00:55:56.440
But if you buy a modern
computer, whether it be desktop

00:55:56.440 --> 00:56:00.150
or phone, your workload is
being revealed over the power

00:56:00.150 --> 00:56:02.240
line in subtle ways.

00:56:02.240 --> 00:56:08.789
And so what he discovered was
that what's going on here.

00:56:08.789 --> 00:56:10.330
If you have an
embedded system that's

00:56:10.330 --> 00:56:13.740
very difficult to change, and
you want to retrofit security

00:56:13.740 --> 00:56:19.560
onto it, what you can do is
put in basically a power strip.

00:56:19.560 --> 00:56:21.400
An intelligent power strip.

00:56:21.400 --> 00:56:25.581
And it uses machine learning
classification of the frequency

00:56:25.581 --> 00:56:26.080
domain.

00:56:26.080 --> 00:56:27.650
Actually looking at the
frequency components

00:56:27.650 --> 00:56:28.733
of your power consumption.

00:56:28.733 --> 00:56:31.010
It's not looking at how
much power you consume.

00:56:31.010 --> 00:56:34.530
Instead it's looking at how
often do you consume it.

00:56:34.530 --> 00:56:36.960
So let me give you
some intuition here.

00:56:36.960 --> 00:56:38.870
So imagine you have
a medical device that

00:56:38.870 --> 00:56:41.020
gets infected by malware.

00:56:41.020 --> 00:56:44.960
Let's say this malware is going
to wake up every few minutes

00:56:44.960 --> 00:56:46.060
to send out spam.

00:56:46.060 --> 00:56:49.058
How might that change
the power consumption?

00:56:54.220 --> 00:56:55.108
Yeah.

00:56:55.108 --> 00:56:56.024
AUDIENCE: [INAUDIBLE].

00:56:58.535 --> 00:57:00.660
PROFESSOR: Yeah, every few
minutes that interrupt's

00:57:00.660 --> 00:57:02.910
going to go off, and the
processor's going to wake up.

00:57:02.910 --> 00:57:05.850
It's probably going to
power up its memory.

00:57:05.850 --> 00:57:09.400
It's going to do some
cycling, or it might actually

00:57:09.400 --> 00:57:11.490
insert a few extra
cycles in what

00:57:11.490 --> 00:57:14.540
used to be a very constant
set of instructions.

00:57:14.540 --> 00:57:17.527
Medical devices generally
do a small set of things,

00:57:17.527 --> 00:57:19.235
as opposed to a general
purpose computer.

00:57:19.235 --> 00:57:21.290
So it's a very regular pattern.

00:57:21.290 --> 00:57:23.610
So when you suddenly
have malware getting in,

00:57:23.610 --> 00:57:27.510
it just changes the behavior of
its power consumption patterns.

00:57:27.510 --> 00:57:28.510
So you can pick that up.

00:57:28.510 --> 00:57:29.635
You do a Fourier transform.

00:57:29.635 --> 00:57:32.640
You do some other magic
involving machine learning.

00:57:32.640 --> 00:57:34.080
The devil's in the details.

00:57:34.080 --> 00:57:36.820
But you can basically
use the machine learning

00:57:36.820 --> 00:57:40.670
to identify with very high
precision, very high accuracy,

00:57:40.670 --> 00:57:42.850
low false positive,
low false negative,

00:57:42.850 --> 00:57:45.840
the presence of malware
and other anomalies.

00:57:45.840 --> 00:57:47.880
And so that's a project
he had been working

00:57:47.880 --> 00:57:49.870
on for a number of years.

00:57:49.870 --> 00:57:52.130
He initially created
this project, though,

00:57:52.130 --> 00:57:54.819
to identify what website
you were browsing.

00:57:54.819 --> 00:57:57.360
And unfortunately, he submitted
it to a bunch of conferences,

00:57:57.360 --> 00:58:01.860
and they all said, well, why
would you ever want to do that?

00:58:01.860 --> 00:58:04.410
But it's kind of interesting,
because he picked the top 50

00:58:04.410 --> 00:58:05.870
Alexa websites.

00:58:05.870 --> 00:58:07.890
And then he profiled
his computer,

00:58:07.890 --> 00:58:10.700
used that as a training set
for the machine learning,

00:58:10.700 --> 00:58:13.520
and then again, with very high
accuracy, very high precision,

00:58:13.520 --> 00:58:16.350
was able to identify which
website you were going to.

00:58:16.350 --> 00:58:19.060
And we were really confused
why it worked at all.

00:58:19.060 --> 00:58:21.370
And we still don't
know exactly why,

00:58:21.370 --> 00:58:22.703
but we have some strong hunches.

00:58:22.703 --> 00:58:26.540
And [INAUDIBLE] Drupal.

00:58:26.540 --> 00:58:29.020
So there's been a movement
over the last 10 years

00:58:29.020 --> 00:58:33.396
on websites to move from-- who
still writes in Emacs HTML?

00:58:33.396 --> 00:58:35.045
All right, me too.

00:58:35.045 --> 00:58:37.170
That's why I have all these
mistakes on my website.

00:58:37.170 --> 00:58:40.190
But there has been a
large movement, especially

00:58:40.190 --> 00:58:43.025
in institutions, to have
code automatically generate

00:58:43.025 --> 00:58:47.340
a web content file that follows
a very regular structure.

00:58:47.340 --> 00:58:49.525
So can you imagine
if you go to CNN.com,

00:58:49.525 --> 00:58:52.290
and they always have an ad in
the upper right-hand corner,

00:58:52.290 --> 00:58:55.860
with flash animation that
lasts exactly 22 seconds?

00:58:55.860 --> 00:58:59.500
So your GPU might kick in
at a very regular rate.

00:58:59.500 --> 00:59:01.740
So some very interesting
things bleed through

00:59:01.740 --> 00:59:04.790
into the power consumption
patterns from the web browser,

00:59:04.790 --> 00:59:06.820
and from other things in
your operating system,

00:59:06.820 --> 00:59:09.670
as a result of your activity.

00:59:09.670 --> 00:59:12.726
The only website we couldn't
classify too well was GoDaddy.

00:59:12.726 --> 00:59:16.230
We still don't know
why, but we don't care.

00:59:21.406 --> 00:59:23.530
So this is going to branch
a little bit further out

00:59:23.530 --> 00:59:26.156
from the security side.

00:59:26.156 --> 00:59:27.530
But one of the
things you find is

00:59:27.530 --> 00:59:31.500
that when you're helping your
colleagues in the hospital

00:59:31.500 --> 00:59:34.002
system, they often ask
back for some of your help.

00:59:34.002 --> 00:59:35.460
And one of the
interesting projects

00:59:35.460 --> 00:59:37.376
that we got involved
with, purely by accident,

00:59:37.376 --> 00:59:40.640
from some of the
pacemaker security work

00:59:40.640 --> 00:59:43.800
was in some humanitarian
aid in developing countries,

00:59:43.800 --> 00:59:47.010
especially Ghana, to
give patients new life.

00:59:47.010 --> 00:59:48.955
Literally new life,
because it turns out

00:59:48.955 --> 00:59:50.580
if you don't have a
health care system,

00:59:50.580 --> 00:59:52.730
it's very difficult
to, say, get a $40,000

00:59:52.730 --> 00:59:54.910
pacemaker plus
the surgical team.

00:59:54.910 --> 00:59:56.499
Very challenging.

00:59:56.499 --> 00:59:58.040
So what they've been
doing is they've

00:59:58.040 --> 01:00:01.690
been recovering discarded
pacemakers and defibrillators,

01:00:01.690 --> 01:00:02.835
and then sterilizing them.

01:00:02.835 --> 01:00:04.210
It's actually
pretty interesting.

01:00:04.210 --> 01:00:06.001
You have to use-- well,
you don't have to--

01:00:06.001 --> 01:00:10.260
but what's typically
used is ethylene oxide.

01:00:10.260 --> 01:00:12.685
It's a gas chamber to
sterilize and remove

01:00:12.685 --> 01:00:15.680
all the pyrogens,
things that cause fever.

01:00:15.680 --> 01:00:17.130
But these devices
are sterilized,

01:00:17.130 --> 01:00:18.650
and then reimplanted
in patients.

01:00:18.650 --> 01:00:19.810
So here's a gentleman.

01:00:19.810 --> 01:00:22.010
I believe he was suffering
from a slow heart

01:00:22.010 --> 01:00:25.050
rate, which was basically
a death sentence for him.

01:00:25.050 --> 01:00:27.200
But because he was able
to get a pacemaker,

01:00:27.200 --> 01:00:29.670
it gave him extra years of life.

01:00:29.670 --> 01:00:32.470
So the problem they
came to us with was,

01:00:32.470 --> 01:00:35.100
how do they know if the
devices are still safe?

01:00:35.100 --> 01:00:36.990
They weren't even used.

01:00:36.990 --> 01:00:39.370
So obviously you can
look at the battery life.

01:00:39.370 --> 01:00:40.620
So that's one thing you do.

01:00:40.620 --> 01:00:42.430
And if the battery
is too low, you

01:00:42.430 --> 01:00:46.276
would not reimplant it, because
that wouldn't last too long.

01:00:46.276 --> 01:00:48.150
But then what about some
of the other things?

01:00:48.150 --> 01:00:49.887
Has some of the metal corroded?

01:00:49.887 --> 01:00:52.220
How do we do an end-to-end
check to see if you can still

01:00:52.220 --> 01:00:54.200
detect arrhythmias properly?

01:00:54.200 --> 01:00:58.360
So the students in my lab
created a special tester

01:00:58.360 --> 01:01:02.770
that sends out
what you would see

01:01:02.770 --> 01:01:08.860
from the electrical components
of cardiac arrhythmias.

01:01:08.860 --> 01:01:10.220
Things other than sinusoid.

01:01:10.220 --> 01:01:12.470
Cardiac rhythms that you
wouldn't want to have, right?

01:01:12.470 --> 01:01:13.640
So, anomalies.

01:01:13.640 --> 01:01:16.000
And it replays these
against the pacemaker leads.

01:01:16.000 --> 01:01:18.590
The pacemaker thinks it's
connected to the patient,

01:01:18.590 --> 01:01:19.530
and then it responds.

01:01:19.530 --> 01:01:21.030
And so we check
that response to see

01:01:21.030 --> 01:01:23.960
if it's actually diagnosing
the cardiac arrhythmias,

01:01:23.960 --> 01:01:26.350
and whether it's actually
sending out the lifesaving

01:01:26.350 --> 01:01:28.080
shocks properly.

01:01:28.080 --> 01:01:33.280
So they're now starting to
test this through the whole FDA

01:01:33.280 --> 01:01:35.760
process to get their blessing.

01:01:35.760 --> 01:01:37.220
And that's a work in progress.

01:01:37.220 --> 01:01:39.660
But it's called the My
Heart Your Heart program.

01:01:39.660 --> 01:01:42.110
You can go look it up if
you're curious about it.

01:01:42.110 --> 01:01:45.000
And then we also
interact quite a bit

01:01:45.000 --> 01:01:47.060
with the medical device
manufacturing community.

01:01:47.060 --> 01:01:50.720
We bring them in each
summer out to Ann Arbor,

01:01:50.720 --> 01:01:54.930
and we have the
manufacturers sit down,

01:01:54.930 --> 01:01:58.230
while some of the persons
who are in charge of running

01:01:58.230 --> 01:01:59.980
hospitals sit down
next to them, and they

01:01:59.980 --> 01:02:05.740
start sharing their gripes and
problems with medical devices.

01:02:05.740 --> 01:02:09.270
We had one company come in and
just reveal all the problems

01:02:09.270 --> 01:02:11.507
that none of the
people would respond to

01:02:11.507 --> 01:02:12.965
at the medical
device manufacturer.

01:02:12.965 --> 01:02:16.550
And one guy in the corner
was like, that's my team.

01:02:16.550 --> 01:02:18.899
And so they decided to go
out for lunch, have a beer,

01:02:18.899 --> 01:02:20.190
and just work out the problems.

01:02:20.190 --> 01:02:21.950
So a lot of it is cultural.

01:02:21.950 --> 01:02:26.070
So I don't know if anyone here
has done any security analysis

01:02:26.070 --> 01:02:27.890
work, or reverse engineering.

01:02:27.890 --> 01:02:29.618
Anyone here?

01:02:29.618 --> 01:02:30.710
Couple people.

01:02:30.710 --> 01:02:32.610
So it's really delicate.

01:02:32.610 --> 01:02:34.710
It's almost an
art, because you're

01:02:34.710 --> 01:02:37.430
dealing with the social elements
of the manufacturing side.

01:02:37.430 --> 01:02:39.680
And it's even more so in
medical device manufacturing,

01:02:39.680 --> 01:02:41.940
because lives are at stake.

01:02:41.940 --> 01:02:43.900
And so it can be
very, very tricky

01:02:43.900 --> 01:02:46.740
to share these kinds of
problems with the people who

01:02:46.740 --> 01:02:48.630
are most able to fix it.

01:02:48.630 --> 01:02:51.450
So it often results
in in-person meetings

01:02:51.450 --> 01:02:55.810
and actually going
to their facilities.

01:02:55.810 --> 01:02:58.010
So I want to save
some more time here.

01:02:58.010 --> 01:02:59.510
Hopefully we'll
have some questions,

01:02:59.510 --> 01:03:02.930
because I think we have
five or 10 minutes.

01:03:02.930 --> 01:03:07.020
But I want to dispel
a couple of myths.

01:03:07.020 --> 01:03:10.630
You'll hear a lot of newspaper
headlines and TV shows

01:03:10.630 --> 01:03:14.290
talking about hackers
breaking into medical devices.

01:03:14.290 --> 01:03:18.037
Let me say it is a problem,
but it's not the problem.

01:03:18.037 --> 01:03:19.870
It's not the only
problem, and it's probably

01:03:19.870 --> 01:03:22.124
not the most
significant problem.

01:03:22.124 --> 01:03:24.040
And it's hard to say
that, especially when you

01:03:24.040 --> 01:03:25.952
enjoy doing security analysis.

01:03:25.952 --> 01:03:27.910
It's hard to say that,
because there's actually

01:03:27.910 --> 01:03:29.900
two problems that I
think are more important.

01:03:29.900 --> 01:03:34.200
One is preventing wide-scale
unavailability of patient care.

01:03:34.200 --> 01:03:36.080
Because forget adversaries--
what if you just

01:03:36.080 --> 01:03:37.990
have malware that
accidentally breaks

01:03:37.990 --> 01:03:40.254
into a medical device
in a monoculture

01:03:40.254 --> 01:03:42.420
where they're all running
the same operating system?

01:03:42.420 --> 01:03:44.860
What happens when you
lose 50,000 infusion

01:03:44.860 --> 01:03:46.470
pumps all at once?

01:03:46.470 --> 01:03:49.970
It's very difficult to
deliver patient care.

01:03:49.970 --> 01:03:55.120
One of my colleagues wrote to me
saying that his cath labs were

01:03:55.120 --> 01:03:55.780
shut down.

01:03:55.780 --> 01:03:58.949
Catheterization lab is a
relatively new specialization.

01:03:58.949 --> 01:04:00.490
It's a special kind
of operating room

01:04:00.490 --> 01:04:02.600
for minimally invasive surgery.

01:04:02.600 --> 01:04:05.620
And at his hospital,
they had to shut down

01:04:05.620 --> 01:04:08.060
the cath lab, because
turned out a nurse

01:04:08.060 --> 01:04:10.480
had accidentally
brought in a USB stick.

01:04:10.480 --> 01:04:12.960
Something about transferring
family photos up to Yahoo.

01:04:12.960 --> 01:04:14.770
And somehow malware
had gotten in

01:04:14.770 --> 01:04:17.300
and infected their cath labs.

01:04:17.300 --> 01:04:20.360
So they had to shut
the thing down.

01:04:20.360 --> 01:04:22.190
So if you're waiting
to get an angioplasty,

01:04:22.190 --> 01:04:24.106
that particular center's
not available to you.

01:04:24.106 --> 01:04:26.520
You'll have to use one
of the backup centers.

01:04:26.520 --> 01:04:29.940
So availability is, I think, one
of the key things that is often

01:04:29.940 --> 01:04:32.640
forgotten about in security.

01:04:32.640 --> 01:04:35.820
Second one is the
integrity of the sensor.

01:04:35.820 --> 01:04:38.340
So if your medical
device gets infected

01:04:38.340 --> 01:04:43.240
by malware, or any kind
of malicious software,

01:04:43.240 --> 01:04:44.490
things are going to change.

01:04:44.490 --> 01:04:45.948
Things are going
to change in a way

01:04:45.948 --> 01:04:47.980
that the designers
didn't anticipate.

01:04:47.980 --> 01:04:50.240
So a very simple example.

01:04:50.240 --> 01:04:53.670
Let's say some malware
gets in, and adds a timer

01:04:53.670 --> 01:04:58.440
to every now and then wake
up, send some network packets,

01:04:58.440 --> 01:05:00.630
and send out some spam.

01:05:00.630 --> 01:05:01.980
This took some time.

01:05:01.980 --> 01:05:04.697
Well, what happens if your
medical device assumed

01:05:04.697 --> 01:05:07.030
that it had complete control
over the interrupt handler,

01:05:07.030 --> 01:05:09.700
and suddenly now it's
missing interrupts?

01:05:09.700 --> 01:05:12.720
Maybe the sensor has
some data to supply

01:05:12.720 --> 01:05:14.940
to the medical device, but
because of the malware,

01:05:14.940 --> 01:05:16.990
it missed the interrupt.

01:05:16.990 --> 01:05:19.420
You may actually start
misdiagnosing patients now,

01:05:19.420 --> 01:05:22.977
because that device is going
to be getting bad data.

01:05:22.977 --> 01:05:24.685
So I'm very concerned
about the integrity

01:05:24.685 --> 01:05:25.780
of the medical sensors.

01:05:25.780 --> 01:05:29.210
There was actually a reported
case of a high-risk pregnancy

01:05:29.210 --> 01:05:30.900
monitor getting
infected with malware

01:05:30.900 --> 01:05:33.400
and giving out
incorrect readings.

01:05:33.400 --> 01:05:36.770
The good news is a
highly-trained clinician

01:05:36.770 --> 01:05:40.800
can look at the device and
say, that makes no sense.

01:05:40.800 --> 01:05:44.300
That's not a sane number coming
out of my medical device.

01:05:44.300 --> 01:05:47.990
But we're basically cutting
down the safety margins

01:05:47.990 --> 01:05:52.859
when we can't have the integrity
of our medical sensors.

01:05:52.859 --> 01:05:54.775
As I mentioned, very
difficult to bolt on this

01:05:54.775 --> 01:05:57.570
stuff after the fact.

01:05:57.570 --> 01:06:00.100
You think changing software's
hard on an internet scale?

01:06:00.100 --> 01:06:01.690
Try it on a medical device.

01:06:01.690 --> 01:06:06.070
So I met a guy from one
hospital where his MRI is still

01:06:06.070 --> 01:06:08.840
running on Windows 95.

01:06:08.840 --> 01:06:14.040
I have a pacemaker
programmer that runs on OS/2.

01:06:14.040 --> 01:06:17.440
And they recently
upgraded to Windows XP.

01:06:17.440 --> 01:06:21.100
So they have some really old
stuff out there, so changing

01:06:21.100 --> 01:06:25.410
things for security after the
fact is going to be difficult.

01:06:25.410 --> 01:06:28.700
Not impossible, but difficult.
And the other reason

01:06:28.700 --> 01:06:31.380
is the interruption
of clinical workflow.

01:06:31.380 --> 01:06:32.880
If you ever go off
and want to start

01:06:32.880 --> 01:06:35.585
implementing medical
devices or doing something

01:06:35.585 --> 01:06:36.960
security related
for health care,

01:06:36.960 --> 01:06:39.050
I encourage you to go off
and call up some people,

01:06:39.050 --> 01:06:41.050
and say, hey, can I go
into your operating room?

01:06:41.050 --> 01:06:43.630
That's what we did.

01:06:43.630 --> 01:06:45.850
Because you'll see some
weird things happen.

01:06:45.850 --> 01:06:49.360
I took all my students to live
surgery, pediatric surgery.

01:06:49.360 --> 01:06:52.040
And as they were
watching the surgery,

01:06:52.040 --> 01:06:55.626
they were watching one
clinician checking Gmail

01:06:55.626 --> 01:06:57.385
on one of the medical devices.

01:06:57.385 --> 01:07:01.780
And they're like, oh OK, so,
drive-by downloads, check.

01:07:01.780 --> 01:07:04.940
At the same time, they
wanted to calm the patient,

01:07:04.940 --> 01:07:10.290
so they logged into
Pandora to play music.

01:07:10.290 --> 01:07:12.290
Actually I was just at
my dentist the other day,

01:07:12.290 --> 01:07:13.900
and she was playing Pandora.

01:07:13.900 --> 01:07:18.290
And these ads for various beers
started coming up on the screen

01:07:18.290 --> 01:07:19.915
as she was looking
at my dental x-rays.

01:07:19.915 --> 01:07:22.825
And I was trying to figure
why Dos Equis was on my-- I

01:07:22.825 --> 01:07:26.191
was like, did I drink that much?

01:07:26.191 --> 01:07:27.940
She's like, no, we
just play Pandora here.

01:07:27.940 --> 01:07:31.080
It's the same web browser,
and just click here.

01:07:31.080 --> 01:07:34.440
So there's a lot
of mixing going on.

01:07:34.440 --> 01:07:37.730
Maybe it's not malicious,
but it's opening cracks.

01:07:37.730 --> 01:07:39.450
It's out of sight, out of mind.

01:07:39.450 --> 01:07:41.870
The hand washing
sterile technique

01:07:41.870 --> 01:07:44.630
is driven into the mindset
of anyone who's a clinician.

01:07:44.630 --> 01:07:45.560
Wash your hands.

01:07:45.560 --> 01:07:47.470
Don't touch the gloves
after you put them on.

01:07:47.470 --> 01:07:49.170
But when it comes
to security hygiene,

01:07:49.170 --> 01:07:51.000
it's really out of
sight, out of mind.

01:07:51.000 --> 01:07:52.602
It's not part of
the culture yet.

01:07:52.602 --> 01:07:54.060
They don't even
realize they should

01:07:54.060 --> 01:07:55.260
be asking these questions.

01:07:55.260 --> 01:07:57.680
Should I be running Pandora
on the same device that's

01:07:57.680 --> 01:08:00.657
controlling my x-rays?

01:08:00.657 --> 01:08:02.740
So but the important thing
is on the designer side

01:08:02.740 --> 01:08:04.480
is not to interrupt
the clinical workflow.

01:08:04.480 --> 01:08:05.920
Because that's when
mistakes happen.

01:08:05.920 --> 01:08:07.540
You want to keep the
clinical workflow

01:08:07.540 --> 01:08:11.960
regular, predictable, easy
for them to make decisions.

01:08:11.960 --> 01:08:16.202
And if you add a new dialogue
box to enter a password, what

01:08:16.202 --> 01:08:18.410
do you think a problem could
be in the operating room

01:08:18.410 --> 01:08:20.620
if you ask the clinician
to enter a password, say,

01:08:20.620 --> 01:08:21.640
every ten minutes?

01:08:28.540 --> 01:08:29.689
Distractions?

01:08:29.689 --> 01:08:31.640
You're sitting there,
doing this, right?

01:08:31.640 --> 01:08:32.700
Scalpel.

01:08:32.700 --> 01:08:34.695
Oh, yeah, let me walk
over here and type in my

01:08:34.695 --> 01:08:36.250
pass-- oh, no, I
got my gloves on.

01:08:36.250 --> 01:08:37.580
Let me take those off.

01:08:37.580 --> 01:08:39.390
Oh, I've got to resterilize now.

01:08:39.390 --> 01:08:40.630
Nurse!

01:08:40.630 --> 01:08:43.050
So if you're a
security engineer,

01:08:43.050 --> 01:08:45.020
you have to take into
account all the rather

01:08:45.020 --> 01:08:48.380
special conditions of the
clinical setting with infection

01:08:48.380 --> 01:08:49.170
control.

01:08:49.170 --> 01:08:51.361
Which, surprisingly, not
everybody knows about.

01:08:51.361 --> 01:08:53.569
There are definitely some
very talented engineers who

01:08:53.569 --> 01:08:57.826
know about it, but not enough.

01:08:57.826 --> 01:08:59.670
The other big problem
is I've noticed

01:08:59.670 --> 01:09:06.689
that security people tend to
specialize in the mechanisms

01:09:06.689 --> 01:09:08.130
to control security.

01:09:08.130 --> 01:09:09.330
You can wield crypto.

01:09:09.330 --> 01:09:12.790
I know CBC mode this, and I
know public key crypto this.

01:09:12.790 --> 01:09:14.370
That's great.

01:09:14.370 --> 01:09:16.330
And you know how to
prevent the problems.

01:09:16.330 --> 01:09:18.609
You know how to
detect the problems.

01:09:18.609 --> 01:09:20.240
The issue is from
the medical world.

01:09:20.240 --> 01:09:21.740
Most people in the
medical world are

01:09:21.740 --> 01:09:23.510
coming from a very
different mindset, one

01:09:23.510 --> 01:09:25.720
that's called risk management.

01:09:25.720 --> 01:09:27.580
Let me try to explain it.

01:09:27.580 --> 01:09:29.960
In risk management, you
look at risks and benefits,

01:09:29.960 --> 01:09:31.910
and you ask yourself,
do they balance?

01:09:31.910 --> 01:09:35.609
If I take an action, does that
improve my risk management

01:09:35.609 --> 01:09:36.318
outlook?

01:09:36.318 --> 01:09:38.109
So if you're going to
decide, for instance,

01:09:38.109 --> 01:09:41.920
am I going to deploy
a password system

01:09:41.920 --> 01:09:43.765
on all my medical devices.

01:09:43.765 --> 01:09:46.259
A security person
might say, duh,

01:09:46.259 --> 01:09:48.050
of course you're going
to deploy passwords,

01:09:48.050 --> 01:09:49.819
because you need
to authenticate.

01:09:49.819 --> 01:09:52.500
The safety person might
say, well, wait a minute.

01:09:52.500 --> 01:09:56.310
If I required we have
passwords on every system,

01:09:56.310 --> 01:09:58.910
we're going to worry
about sterilization.

01:09:58.910 --> 01:10:00.900
How do we know how
often to time out?

01:10:00.900 --> 01:10:02.710
And what about emergency access?

01:10:02.710 --> 01:10:04.300
What if we forget the password?

01:10:04.300 --> 01:10:07.459
We want to make sure we can get
a response time in 30 seconds.

01:10:07.459 --> 01:10:09.500
So they might actually
make a different decision.

01:10:09.500 --> 01:10:12.280
They might actually decide
not to have passwords at all.

01:10:12.280 --> 01:10:14.930
Actually many hospitals
don't have passwords.

01:10:14.930 --> 01:10:18.190
Excuse me, many hospitals
don't have access control

01:10:18.190 --> 01:10:19.030
on medical records.

01:10:19.030 --> 01:10:21.610
Instead they have what's called
audit-based access control.

01:10:21.610 --> 01:10:23.860
After the fact, if you look
at something you shouldn't

01:10:23.860 --> 01:10:25.860
look at, they come get you.

01:10:25.860 --> 01:10:28.540
Because they know that it's
very difficult to predict what

01:10:28.540 --> 01:10:30.040
you're going to
need in your routine

01:10:30.040 --> 01:10:32.390
of your clinical workflow.

01:10:32.390 --> 01:10:35.560
So the risk
management kind of way

01:10:35.560 --> 01:10:39.120
will depend upon deploying
the security controls

01:10:39.120 --> 01:10:40.822
and all the technology
learn about.

01:10:40.822 --> 01:10:42.280
But in the risk
management picture,

01:10:42.280 --> 01:10:44.530
you might actually decide
not to deploy something,

01:10:44.530 --> 01:10:48.120
because it could cause
harm somewhere else.

01:10:48.120 --> 01:10:49.770
But trustworthy
medical device software

01:10:49.770 --> 01:10:52.880
is going to require both.

01:10:52.880 --> 01:10:54.857
So I'll just finish up here.

01:10:54.857 --> 01:10:56.940
I think there's a lot of
interesting things to do.

01:10:56.940 --> 01:10:59.104
So you're taking this
cool security course.

01:10:59.104 --> 01:11:01.020
I encourage you to go
out and use those tools.

01:11:01.020 --> 01:11:03.280
But as you're thinking about
where to go afterwards,

01:11:03.280 --> 01:11:05.060
whether it's industry
or graduate school,

01:11:05.060 --> 01:11:07.435
think about medical devices,
because they need your help.

01:11:07.435 --> 01:11:09.430
They need a lot of
smart people there.

01:11:09.430 --> 01:11:12.080
And so there's just one
thing missing-- you are.

01:11:12.080 --> 01:11:14.410
And I think there's a
lot of interesting stuff

01:11:14.410 --> 01:11:15.590
still to be done.

01:11:15.590 --> 01:11:17.932
So I think we have five
or 10 minutes or so.

01:11:17.932 --> 01:11:19.390
I'd be glad to take
some questions.

01:11:19.390 --> 01:11:20.640
Or I could go more into depth.

01:11:20.640 --> 01:11:22.730
I got some fun
videos I could show.

01:11:22.730 --> 01:11:25.810
But I think I'll at least
take a break for a moment

01:11:25.810 --> 01:11:28.628
to see if you have
any questions.

01:11:28.628 --> 01:11:29.128
Yes?

01:11:29.128 --> 01:11:31.096
AUDIENCE: So that
pacemaker or whatever

01:11:31.096 --> 01:11:35.086
it was that you were passing
around, does that [INAUDIBLE].

01:11:35.086 --> 01:11:36.460
PROFESSOR: Oh,
the defibrillator.

01:11:36.460 --> 01:11:37.085
AUDIENCE: Yeah.

01:11:37.085 --> 01:11:41.448
How does that
interact with the fact

01:11:41.448 --> 01:11:44.370
that they are [INAUDIBLE]
these kind of things.

01:11:44.370 --> 01:11:46.132
PROFESSOR: OK so
a couple things.

01:11:46.132 --> 01:11:48.340
So there are defibrillators
and there are pacemakers.

01:11:48.340 --> 01:11:50.650
They're very related.

01:11:50.650 --> 01:11:51.850
This is a defibrillator.

01:11:51.850 --> 01:11:53.320
It sends out large shocks.

01:11:53.320 --> 01:11:55.570
Pacemakers send
out small shocks.

01:11:55.570 --> 01:11:59.020
But in the US, it's
illegal to reimplant these.

01:11:59.020 --> 01:12:01.110
So it doesn't matter if you can.

01:12:01.110 --> 01:12:02.670
It's just illegal.

01:12:02.670 --> 01:12:05.720
But in many developing
countries it's not illegal.

01:12:05.720 --> 01:12:08.150
And if you look from--
let me back up a slide.

01:12:10.730 --> 01:12:14.110
If you look from not the
control mechanism, but the risk

01:12:14.110 --> 01:12:16.840
management side of
equation, it might actually

01:12:16.840 --> 01:12:19.750
lead to better public
health outcomes

01:12:19.750 --> 01:12:22.750
to allow reimplantation and
reuse in developing countries

01:12:22.750 --> 01:12:25.646
where they have no other choice.

01:12:25.646 --> 01:12:26.890
And this is not my project.

01:12:26.890 --> 01:12:29.600
This is just a project
we're assisting on.

01:12:29.600 --> 01:12:32.280
But in that particular case, the
patients really have no choice.

01:12:32.280 --> 01:12:34.670
It's basically a death sentence.

01:12:34.670 --> 01:12:36.804
To sterilize it
is pretty tricky.

01:12:36.804 --> 01:12:38.720
There's a whole lot of
science and engineering

01:12:38.720 --> 01:12:40.970
that goes into how
to properly sterilize

01:12:40.970 --> 01:12:44.020
to get rid of the pathogens.

01:12:44.020 --> 01:12:48.560
Because these were in blood,
so first abrasive cleaner,

01:12:48.560 --> 01:12:50.710
but the ethylene
oxide is one way

01:12:50.710 --> 01:12:52.974
to destroy most, if not
all, of the pathogens.

01:12:52.974 --> 01:12:54.390
There's a whole
testing procedure.

01:12:54.390 --> 01:12:58.155
You actually put
special little-- I

01:12:58.155 --> 01:12:59.280
forgot what they're called.

01:12:59.280 --> 01:13:02.610
They're little wafers-- with
known quantities of pathogens.

01:13:02.610 --> 01:13:05.120
And you put it in alongside
some of the devices

01:13:05.120 --> 01:13:08.200
as it's going into the chamber,
and when you pull it out,

01:13:08.200 --> 01:13:12.710
you test to see if all those
organisms have been killed.

01:13:12.710 --> 01:13:14.220
Did that answer
all your questions?

01:13:14.220 --> 01:13:16.610
You had a follow up.

01:13:16.610 --> 01:13:17.180
OK.

01:13:17.180 --> 01:13:17.680
Yes?

01:13:17.680 --> 01:13:20.224
AUDIENCE: So what you're
saying, integrity of sensors

01:13:20.224 --> 01:13:22.190
is a bigger risk
for hacker attack,

01:13:22.190 --> 01:13:26.776
because most of the examples of
sensory interference you showed

01:13:26.776 --> 01:13:28.240
are intentional interference.

01:13:28.240 --> 01:13:29.916
So it's kind of [INAUDIBLE].

01:13:29.916 --> 01:13:31.290
PROFESSOR: Oh, so
the question is

01:13:31.290 --> 01:13:36.070
why focus on integrity of
sensors rather than hacking,

01:13:36.070 --> 01:13:38.220
because everything I
showed was about hacking?

01:13:38.220 --> 01:13:40.340
That's selection bias.

01:13:40.340 --> 01:13:42.460
I selected those cases,
but that doesn't mean

01:13:42.460 --> 01:13:45.300
that's statistically relevant.

01:13:45.300 --> 01:13:48.850
I divided up into two
cases-- maybe three.

01:13:48.850 --> 01:13:50.890
The past, the present,
and the future.

01:13:50.890 --> 01:13:54.805
So at the present,
most of the problems

01:13:54.805 --> 01:13:58.145
we're seeing from malware in our
very rudimentary surveillance

01:13:58.145 --> 01:14:01.090
of medical devices has to do
with malware that accidentally

01:14:01.090 --> 01:14:05.280
gets in, and then causes
near misses and malfunctions.

01:14:05.280 --> 01:14:06.470
But we're no dummies.

01:14:06.470 --> 01:14:08.830
We know that there could
be an intentional adversary

01:14:08.830 --> 01:14:09.800
in the future.

01:14:09.800 --> 01:14:11.570
They just haven't
materialized yet.

01:14:11.570 --> 01:14:15.480
The closest example
would be-- this

01:14:15.480 --> 01:14:16.750
is just from the news reports.

01:14:16.750 --> 01:14:18.125
I don't know if
it's true, but it

01:14:18.125 --> 01:14:20.720
was from I believe
the New York Times--

01:14:20.720 --> 01:14:23.150
that there was a hospital.

01:14:23.150 --> 01:14:25.586
CHS, I think, was
the hospital, Where

01:14:25.586 --> 01:14:27.460
they brought in a security
company, Mandiant.

01:14:27.460 --> 01:14:32.000
And they believe that a nation
state had actually come in

01:14:32.000 --> 01:14:33.540
to steal the medical records.

01:14:33.540 --> 01:14:36.710
They don't know exactly
why, but nation states.

01:14:36.710 --> 01:14:38.900
And nation states are
powerful adversaries, right?

01:14:38.900 --> 01:14:40.441
If you run up against
a nation state,

01:14:40.441 --> 01:14:43.995
you might as well just give up,
because none of these controls

01:14:43.995 --> 01:14:45.490
are going to help you.

01:14:45.490 --> 01:14:47.024
But here's my concern.

01:14:47.024 --> 01:14:48.440
If the nation
state, for instance,

01:14:48.440 --> 01:14:51.350
is very dedicated on getting
one piece of information,

01:14:51.350 --> 01:14:53.897
what if they make--
they're human too, right?

01:14:53.897 --> 01:14:55.605
What if they make a
mistake along the way

01:14:55.605 --> 01:14:57.146
and accidentally
hit a medical device

01:14:57.146 --> 01:15:00.340
as they're trying to extract
whatever kind of information

01:15:00.340 --> 01:15:02.210
they're trying to get at?

01:15:02.210 --> 01:15:05.890
And that could
affect the integrity.

01:15:05.890 --> 01:15:09.440
In the future, there could be
instances of custom malware,

01:15:09.440 --> 01:15:16.050
but I think it takes that
one more step of someone

01:15:16.050 --> 01:15:19.952
really wanting to cause harm.

01:15:19.952 --> 01:15:21.660
And I'm hoping that
there aren't too many

01:15:21.660 --> 01:15:23.030
of those kinds of people.

01:15:23.030 --> 01:15:24.960
But there are people
who write malware

01:15:24.960 --> 01:15:26.600
who don't realize
that malware gets

01:15:26.600 --> 01:15:28.216
into medical devices
in hospitals,

01:15:28.216 --> 01:15:31.020
and it's still causing problems.