WEBVTT 00:00:15.120 --> 00:00:19.440 PETER SZOLOVITS: Fortunately, I have a guest today, Dr. Adam 00:00:19.440 --> 00:00:24.570 Wright, who will be doing an interview-style session 00:00:24.570 --> 00:00:27.360 and will answer questions for you. 00:00:27.360 --> 00:00:30.720 This is Adam's bread and butter, exactly how 00:00:30.720 --> 00:00:35.440 to translate this kind of technology into the clinic. 00:00:35.440 --> 00:00:40.260 He's currently in the partner system at the Brigham, I guess. 00:00:40.260 --> 00:00:45.840 But he's about to become a traitor and leave us in Boston 00:00:45.840 --> 00:00:51.330 and occupy a position at Vanderbilt University, 00:00:51.330 --> 00:00:52.810 for which we wish him luck. 00:00:52.810 --> 00:00:57.810 But I'm glad that we caught him before he leaves this summer. 00:00:57.810 --> 00:01:02.340 OK, so quite frankly, I wish that I could tell you 00:01:02.340 --> 00:01:04.860 a much happier story than the one 00:01:04.860 --> 00:01:08.730 that you're going to hear from me during the prepared part 00:01:08.730 --> 00:01:10.380 of my talk. 00:01:10.380 --> 00:01:15.780 And maybe Adam will cheer us up and make us more optimistic, 00:01:15.780 --> 00:01:19.680 based on his experience. 00:01:19.680 --> 00:01:23.700 So you may have noticed that AI is hot. 00:01:23.700 --> 00:01:27.990 So HIMSS, for example, is the Health Information Management 00:01:27.990 --> 00:01:29.670 Systems Society. 00:01:29.670 --> 00:01:33.520 It's a big-- they hold annual meetings 00:01:33.520 --> 00:01:37.800 and consist of a lot of vendors and a lot of academics. 00:01:37.800 --> 00:01:41.670 And it's one of these huge trade show kinds of things, 00:01:41.670 --> 00:01:48.930 with balloons hanging over booths and big open spaces. 00:01:48.930 --> 00:01:51.240 So for example, they're now talking 00:01:51.240 --> 00:01:54.120 about a AI-powered health care. 00:01:54.120 --> 00:01:56.820 On the other hand, it's important to remember 00:01:56.820 --> 00:01:58.260 this graph. 00:01:58.260 --> 00:02:03.210 So this is the sort of technology adoption graph. 00:02:03.210 --> 00:02:06.300 And it's called the hype cycle. 00:02:06.300 --> 00:02:10.500 And what you see here is that R&D-- 00:02:10.500 --> 00:02:15.600 that's us-- produces some wonderful, interesting idea. 00:02:15.600 --> 00:02:19.380 And then all of a sudden, people get excited about it. 00:02:19.380 --> 00:02:22.205 So who are the people that get most excited about it? 00:02:22.205 --> 00:02:23.830 It's the people who think they're going 00:02:23.830 --> 00:02:26.080 to make a fortune from it. 00:02:26.080 --> 00:02:29.250 And these are the so-called vulture capitalists-- 00:02:29.250 --> 00:02:32.620 venture capitalists. 00:02:32.620 --> 00:02:35.010 And so the venture capitalists come in 00:02:35.010 --> 00:02:37.740 and they encourage people like us to go out 00:02:37.740 --> 00:02:43.290 and found companies-- or if not us, then our students to go 00:02:43.290 --> 00:02:44.370 found companies. 00:02:44.370 --> 00:02:48.900 And figure out how to turn this nascent idea 00:02:48.900 --> 00:02:53.620 into some important moneymaking enterprise. 00:02:53.620 --> 00:02:56.710 Now the secret of venture capital 00:02:56.710 --> 00:03:02.560 is that they know that about 90% of the companies that they fund 00:03:02.560 --> 00:03:04.180 are going to tank. 00:03:04.180 --> 00:03:06.290 They're going to do very badly. 00:03:06.290 --> 00:03:11.290 And so as a result, what they hope for and what they expect-- 00:03:11.290 --> 00:03:13.600 and what the good ones actually get-- 00:03:13.600 --> 00:03:18.310 is that one in 10 that becomes successful 00:03:18.310 --> 00:03:20.410 makes so much money that it makes up 00:03:20.410 --> 00:03:24.010 for all of the investment that they poured into the nine out 00:03:24.010 --> 00:03:26.390 of 10 that do badly. 00:03:26.390 --> 00:03:31.040 So I actually remember in the 1990s, 00:03:31.040 --> 00:03:35.170 I was helping a group pitch a company 00:03:35.170 --> 00:03:39.070 to Kleiner Perkins, which is the big venture-- 00:03:39.070 --> 00:03:43.240 one of the big venture capital funds in Silicon Valley. 00:03:43.240 --> 00:03:45.130 And we walked into their boardroom 00:03:45.130 --> 00:03:48.790 and they had a copy of the San Jose Mercury News, 00:03:48.790 --> 00:03:55.270 which is the local newspaper for Silicon Valley, on their table. 00:03:55.270 --> 00:03:57.250 And they were just beaming, because there 00:03:57.250 --> 00:04:02.920 was an article that said that in the past year, 00:04:02.920 --> 00:04:07.600 the two best and the two worst investments in Silicon Valley 00:04:07.600 --> 00:04:11.240 had been by their company. 00:04:11.240 --> 00:04:13.190 But that's pretty good, right? 00:04:13.190 --> 00:04:17.870 If you get two winners and two really bad losers, 00:04:17.870 --> 00:04:20.790 you're making tons and tons of money. 00:04:20.790 --> 00:04:23.250 So they were in a good mood and they funded us. 00:04:23.250 --> 00:04:24.500 We didn't make them any money. 00:04:28.970 --> 00:04:31.370 So what you see on this curve is that there 00:04:31.370 --> 00:04:36.220 is a kind of set of rising expectations that 00:04:36.220 --> 00:04:40.660 comes from the development of these technologies. 00:04:40.660 --> 00:04:42.820 And you have some early adopters. 00:04:42.820 --> 00:04:45.430 And then you have the newspapers writing 00:04:45.430 --> 00:04:48.520 about how this is the revolution and everything will 00:04:48.520 --> 00:04:51.250 be different from here on out. 00:04:51.250 --> 00:04:54.490 Then you have some additional activity 00:04:54.490 --> 00:04:56.350 beyond the early adopters. 00:04:56.350 --> 00:04:58.660 And then people start looking at this and going, 00:04:58.660 --> 00:05:03.940 well, it really isn't as good as it's cracked up to be. 00:05:03.940 --> 00:05:08.470 Then you have the steep decline where 00:05:08.470 --> 00:05:11.660 there's some consolidation and some failures. 00:05:11.660 --> 00:05:13.930 And people have to go back to venture capital 00:05:13.930 --> 00:05:16.480 to try to get more money in order 00:05:16.480 --> 00:05:18.760 to keep their companies going. 00:05:18.760 --> 00:05:20.500 And then there's a kind of trough, 00:05:20.500 --> 00:05:25.630 where people go, oh well, this was another of these failed 00:05:25.630 --> 00:05:29.050 technological innovations. 00:05:29.050 --> 00:05:35.980 Then gradually, you start reaching what this author calls 00:05:35.980 --> 00:05:40.420 the slope of enlightenment, where people realize that, 00:05:40.420 --> 00:05:44.530 OK, it's not really as bad as we thought it was when it didn't 00:05:44.530 --> 00:05:47.080 meet our lofty expectations. 00:05:47.080 --> 00:05:49.790 And then gradually, if it's successful, 00:05:49.790 --> 00:05:52.390 then you get multiple generations of the product 00:05:52.390 --> 00:05:54.610 and it does achieve adoption. 00:05:54.610 --> 00:05:58.870 The adoption almost never reaches the peak 00:05:58.870 --> 00:06:03.160 that it was expected to reach at the time of the top of the hype 00:06:03.160 --> 00:06:04.330 cycle. 00:06:04.330 --> 00:06:05.380 But it becomes useful. 00:06:05.380 --> 00:06:06.820 It becomes profitable. 00:06:06.820 --> 00:06:09.280 It becomes productive. 00:06:09.280 --> 00:06:13.420 Now I've been around long enough to see a number of these cycles 00:06:13.420 --> 00:06:14.560 go by. 00:06:14.560 --> 00:06:20.440 So in the 1980s, for example, at a time that was now jokingly 00:06:20.440 --> 00:06:22.690 referred to as AI summer-- 00:06:22.690 --> 00:06:26.650 where people were building expert systems and these expert 00:06:26.650 --> 00:06:31.600 systems were going to just revolutionize everything-- 00:06:31.600 --> 00:06:36.070 I remember going to a conference where the Campbell Soup 00:06:36.070 --> 00:06:40.900 Company had built an expert system that 00:06:40.900 --> 00:06:43.870 was based on the expertise of some old timers who 00:06:43.870 --> 00:06:45.520 were retiring. 00:06:45.520 --> 00:06:48.610 And what this expert system did is it told you 00:06:48.610 --> 00:06:51.280 how to clean the vats of soup-- 00:06:51.280 --> 00:06:54.880 y know, these giant million-gallon things where 00:06:54.880 --> 00:06:56.650 they make soup-- 00:06:56.650 --> 00:06:59.290 when you're switching from making one kind of soup 00:06:59.290 --> 00:07:00.920 to another. 00:07:00.920 --> 00:07:04.600 So you know, if you're making beef consomme 00:07:04.600 --> 00:07:07.720 and you switch to making beef barley soup, 00:07:07.720 --> 00:07:10.400 you don't need to clean the vat at all. 00:07:10.400 --> 00:07:16.780 Whereas if you're switching from something like clam chowder 00:07:16.780 --> 00:07:20.740 to a consomme, then you need to clean it really well. 00:07:20.740 --> 00:07:24.010 So this was exactly the kind of thing that they were doing. 00:07:24.010 --> 00:07:26.890 And there were literally thousands 00:07:26.890 --> 00:07:30.070 of these applications being built. 00:07:30.070 --> 00:07:33.220 At the top of the hype cycle, all kinds 00:07:33.220 --> 00:07:36.010 of companies, like Campbell's Soup and the airlines 00:07:36.010 --> 00:07:40.880 and everybody was investing huge amounts of money into this. 00:07:40.880 --> 00:07:44.450 And then there was a kind of failure of expectations. 00:07:44.450 --> 00:07:47.110 These didn't turn out to be as good as people thought 00:07:47.110 --> 00:07:50.230 they were going to be, or as valuable as people 00:07:50.230 --> 00:07:52.060 thought they were going to be. 00:07:52.060 --> 00:07:55.330 And then all of a sudden came AI winter. 00:07:55.330 --> 00:07:57.640 So AI winter followed AI summer. 00:07:57.640 --> 00:08:02.170 There was no AI fall, except in a different sense 00:08:02.170 --> 00:08:04.130 of the word fall. 00:08:04.130 --> 00:08:07.300 And all of a sudden, funding dried up 00:08:07.300 --> 00:08:10.270 and the whole thing was declared a failure. 00:08:10.270 --> 00:08:14.580 But in fact today, if you go out there and you look at-- 00:08:14.580 --> 00:08:20.860 Microsoft Excel has an expert system-based help system 00:08:20.860 --> 00:08:22.540 bundled inside it. 00:08:22.540 --> 00:08:25.270 And there are tons of such applications. 00:08:25.270 --> 00:08:27.010 It's just that now they're no longer 00:08:27.010 --> 00:08:29.530 considered cutting-edge applications 00:08:29.530 --> 00:08:30.940 of artificial intelligence. 00:08:30.940 --> 00:08:34.240 They're simply considered routine practice. 00:08:34.240 --> 00:08:38.289 So they've become incorporated, without the hype, 00:08:38.289 --> 00:08:40.510 into all kinds of existing products. 00:08:40.510 --> 00:08:43.120 And they're serving a very useful role. 00:08:43.120 --> 00:08:45.310 But they didn't make those venture capital 00:08:45.310 --> 00:08:49.270 firms the tons of money that they had hoped to make. 00:08:49.270 --> 00:08:51.460 There was a similar boom and bust 00:08:51.460 --> 00:08:56.830 cycle in the 2000s around the creation of the worldwide web 00:08:56.830 --> 00:08:58.530 and e-commerce. 00:08:58.530 --> 00:09:00.690 OK, so e-commerce. 00:09:00.690 --> 00:09:04.150 Again, there was this unbelievably inflated set 00:09:04.150 --> 00:09:06.460 of expectations. 00:09:06.460 --> 00:09:09.920 Then around the year 2000, there was a big crash, 00:09:09.920 --> 00:09:13.210 where all of a sudden people realized 00:09:13.210 --> 00:09:15.940 that the value in these applications 00:09:15.940 --> 00:09:19.510 was not as high as what they expected it to be. 00:09:19.510 --> 00:09:23.590 Nevertheless, you know Amazon is doing just fine. 00:09:23.590 --> 00:09:26.710 And there are plenty of online e-commerce sites 00:09:26.710 --> 00:09:31.690 that are in perfectly good operating order today. 00:09:31.690 --> 00:09:36.290 But it's no longer the same hype about this technology. 00:09:36.290 --> 00:09:38.860 It's just become an accepted part of the way 00:09:38.860 --> 00:09:41.380 that you do business in almost everything. 00:09:41.380 --> 00:09:41.880 Yeah. 00:09:41.880 --> 00:09:43.672 AUDIENCE: When you speak of expert systems, 00:09:43.672 --> 00:09:45.388 does that mean rule-based systems? 00:09:45.388 --> 00:09:47.680 PETER SZOLOVITS: They were either rule-based or pattern 00:09:47.680 --> 00:09:48.730 matching systems. 00:09:48.730 --> 00:09:50.980 There were two basic kinds. 00:09:50.980 --> 00:09:53.930 I think a week from today, I'm going to talk about some 00:09:53.930 --> 00:09:57.500 of that and how it relates to modern machine learning. 00:09:57.500 --> 00:10:01.150 So we'll see some examples. 00:10:01.150 --> 00:10:09.670 OK, well, a cautionary tale is IBM's Watson Health. 00:10:09.670 --> 00:10:14.590 So I assume most of you remember when 00:10:14.590 --> 00:10:19.300 Watson hit the big time by beating the Jeopardy champions. 00:10:19.300 --> 00:10:22.510 This was back in the early 2010s or something. 00:10:22.510 --> 00:10:25.300 I don't remember exactly which year. 00:10:25.300 --> 00:10:28.540 And they had, in fact, built a really impressive set 00:10:28.540 --> 00:10:32.200 of technologies that went out and read 00:10:32.200 --> 00:10:36.070 all kinds of online sources and distilled them 00:10:36.070 --> 00:10:39.340 into a kind of representation that they could very 00:10:39.340 --> 00:10:42.670 quickly look up things when they were challenged with a Jeopardy 00:10:42.670 --> 00:10:44.230 question. 00:10:44.230 --> 00:10:46.900 And then it had a sophisticated set 00:10:46.900 --> 00:10:50.830 of algorithms that would try to find the best 00:10:50.830 --> 00:10:53.320 answer for some question. 00:10:53.320 --> 00:10:57.340 And they even had all kinds of bizarre special-purpose things. 00:10:57.340 --> 00:11:00.310 I remember there was a probabilistic model that 00:11:00.310 --> 00:11:03.340 figured out where the Daily Double 00:11:03.340 --> 00:11:07.660 squares were most likely to be on the Jeopardy board. 00:11:07.660 --> 00:11:11.770 And then they did a utility theoretic calculation 00:11:11.770 --> 00:11:14.770 to figure out if they did hit the Daily Double, 00:11:14.770 --> 00:11:17.950 what was the optimum amount of money to bet, 00:11:17.950 --> 00:11:20.620 based on the machine's performance, 00:11:20.620 --> 00:11:22.360 in order to optimize. 00:11:22.360 --> 00:11:24.550 They decided that humans typically 00:11:24.550 --> 00:11:29.350 don't bet enough when they have a chance on the Daily Double. 00:11:29.350 --> 00:11:31.870 So there was a lot of very special-purpose stuff 00:11:31.870 --> 00:11:32.980 done for this. 00:11:32.980 --> 00:11:37.030 So this was a huge publicity bonanza. 00:11:37.030 --> 00:11:42.550 And IBM decided that next they were going to tackle medicine. 00:11:42.550 --> 00:11:45.070 So they were going to take this technology 00:11:45.070 --> 00:11:46.940 and apply it to medicine. 00:11:46.940 --> 00:11:49.750 They were going to read all of the medical journals 00:11:49.750 --> 00:11:55.150 and all of the electronic medical records 00:11:55.150 --> 00:11:57.310 that they could get their hands on. 00:11:57.310 --> 00:11:59.650 And somehow this technology would again 00:11:59.650 --> 00:12:03.130 distill the right information, so that they 00:12:03.130 --> 00:12:06.250 could answer questions like a Jeopardy question, 00:12:06.250 --> 00:12:09.790 except not stated in its funny backward way. 00:12:09.790 --> 00:12:12.760 Where you might say, OK, for this patient, 00:12:12.760 --> 00:12:14.800 what is the optimum therapy? 00:12:14.800 --> 00:12:17.470 And it would go out and use the same technology 00:12:17.470 --> 00:12:20.020 to figure that out. 00:12:20.020 --> 00:12:24.370 Now that was a perfectly reasonable thing to try. 00:12:24.370 --> 00:12:27.970 The problem they ran into was this hype cycle, 00:12:27.970 --> 00:12:31.810 that the people who made this publicly-known 00:12:31.810 --> 00:12:35.680 were their marketing people and not their technical people. 00:12:35.680 --> 00:12:39.460 And the marketing people overpromised like crazy. 00:12:39.460 --> 00:12:41.950 They said surely this is just going 00:12:41.950 --> 00:12:43.360 to solve all these problems. 00:12:43.360 --> 00:12:46.460 And we won't need anymore research in this area, 00:12:46.460 --> 00:12:49.450 because man, we got it. 00:12:49.450 --> 00:12:53.650 I'm overstating it, even from the marketing point of view. 00:12:53.650 --> 00:12:59.710 And so Watson for Oncology used this cloud-based supercomputer 00:12:59.710 --> 00:13:01.960 to digest massive amounts of data. 00:13:04.960 --> 00:13:08.750 That data included all kinds of different things. 00:13:08.750 --> 00:13:10.750 So I'm going to go into a little bit of detail 00:13:10.750 --> 00:13:13.000 about what some of their problems were. 00:13:13.000 --> 00:13:18.910 This is from an article in this journal, Statnews, 00:13:18.910 --> 00:13:24.010 which did an investigative piece on what happened with Watson. 00:13:24.010 --> 00:13:26.260 So you know, they say what I just said. 00:13:26.260 --> 00:13:29.050 Breathlessly promoting its signature brand, 00:13:29.050 --> 00:13:31.810 IBM sought to capture the world's imagination 00:13:31.810 --> 00:13:35.030 and quickly zeroed in on a high-profile target, 00:13:35.030 --> 00:13:36.470 which was cancer. 00:13:36.470 --> 00:13:40.210 So this was going to solve the problem of some patient 00:13:40.210 --> 00:13:42.730 shows up, is diagnosed with cancer, 00:13:42.730 --> 00:13:44.890 and you want to know how to treat this person. 00:13:44.890 --> 00:13:47.530 So this would use all of the literature 00:13:47.530 --> 00:13:49.510 and all of everything that it had 00:13:49.510 --> 00:13:51.850 gathered from previous treatments 00:13:51.850 --> 00:13:53.470 of previous patients. 00:13:53.470 --> 00:13:56.920 And it would give you the optimal solution. 00:13:56.920 --> 00:13:59.240 Now it has not been a success. 00:13:59.240 --> 00:14:02.870 There are a few dozen hospitals that have adopted the system. 00:14:02.870 --> 00:14:07.240 Very few of them in the United States, more of them abroad. 00:14:07.240 --> 00:14:11.590 And the foreigners complain that its advice 00:14:11.590 --> 00:14:14.170 is biased toward American patients 00:14:14.170 --> 00:14:16.390 and American approaches. 00:14:16.390 --> 00:14:19.120 To me, the biggest problem is that they haven't actually 00:14:19.120 --> 00:14:22.480 published anything that validates, 00:14:22.480 --> 00:14:25.690 in a scientific sense, that this is a good idea. 00:14:25.690 --> 00:14:28.330 That it's getting the right answers. 00:14:28.330 --> 00:14:30.310 My guess is the reason for this is 00:14:30.310 --> 00:14:32.230 because it's not getting the right answers, 00:14:32.230 --> 00:14:33.880 a lot of the time. 00:14:33.880 --> 00:14:39.790 But that doesn't prevent marketing from selling it. 00:14:39.790 --> 00:14:43.810 The other problem is that they made a deal with Memorial Sloan 00:14:43.810 --> 00:14:46.600 Kettering-- which is one of the leading cancer hospitals 00:14:46.600 --> 00:14:47.800 in the country-- 00:14:47.800 --> 00:14:51.490 to say, we're going to work with you guys and your oncologists 00:14:51.490 --> 00:14:55.870 in order to figure out what really is the right answer. 00:14:55.870 --> 00:14:59.140 So I think they tried to do what their marketing says 00:14:59.140 --> 00:15:02.090 that they're doing, which is to really derive 00:15:02.090 --> 00:15:05.320 the right answer from reading all of the literature 00:15:05.320 --> 00:15:07.420 and looking at past cases. 00:15:07.420 --> 00:15:09.680 But I don't think that worked well enough. 00:15:09.680 --> 00:15:12.190 And so what they wound up doing is turning 00:15:12.190 --> 00:15:14.350 to real oncologists, saying, what would you 00:15:14.350 --> 00:15:16.690 do under these circumstances? 00:15:16.690 --> 00:15:18.460 And so what they wound up building 00:15:18.460 --> 00:15:21.880 is something like a rule-based system that says, 00:15:21.880 --> 00:15:23.880 if you see the following symptoms 00:15:23.880 --> 00:15:26.470 and you have the following genetic defects, 00:15:26.470 --> 00:15:30.130 then this is the right treatment. 00:15:30.130 --> 00:15:32.800 So the promise that this was going 00:15:32.800 --> 00:15:37.330 to be a machine learning system that revolutionized cancer 00:15:37.330 --> 00:15:40.690 care by finding the optimal treatment really 00:15:40.690 --> 00:15:43.480 is not what they provided. 00:15:43.480 --> 00:15:45.670 And as the article says, the system 00:15:45.670 --> 00:15:48.280 doesn't really create new knowledge. 00:15:48.280 --> 00:15:53.170 So it's AI only in the sense of providing a search 00:15:53.170 --> 00:15:57.370 engine that, when it makes a recommendation, 00:15:57.370 --> 00:16:01.600 can point you to articles that are a reasonable reflection 00:16:01.600 --> 00:16:04.900 of what it's recommending. 00:16:04.900 --> 00:16:07.630 Well, I'm going to stop going through this litany. 00:16:07.630 --> 00:16:13.160 But you'll see it in the slides, which we'll post. 00:16:13.160 --> 00:16:15.490 They had a big contract with M.D. Anderson, 00:16:15.490 --> 00:16:19.180 which is another leading cancer center in the United States. 00:16:19.180 --> 00:16:24.040 M.D. Anderson spent about $60 million on this contract, 00:16:24.040 --> 00:16:25.210 implementing it. 00:16:25.210 --> 00:16:27.940 And they pulled the plug on it, because they decided that it 00:16:27.940 --> 00:16:32.110 just wasn't doing the job. 00:16:32.110 --> 00:16:39.700 Now by contrast, there was a much more successful attempt 00:16:39.700 --> 00:16:45.880 years ago, which was less driven by marketing and more driven 00:16:45.880 --> 00:16:47.800 by medical need. 00:16:47.800 --> 00:16:51.430 And the idea here was CPOE, stands for Computerized 00:16:51.430 --> 00:16:53.410 Physician Order Entry. 00:16:53.410 --> 00:16:57.640 The idea behind CPOE was that if you 00:16:57.640 --> 00:17:00.610 want to affect the behavior of clinicians 00:17:00.610 --> 00:17:06.510 in ordering tests or drugs or procedures, what you want to do 00:17:06.510 --> 00:17:10.050 is to make sure that they are interacting with the computer. 00:17:10.050 --> 00:17:12.240 So that when they order, for example, 00:17:12.240 --> 00:17:17.400 some insanely expensive drug, the system can come back 00:17:17.400 --> 00:17:19.410 and say, hey, do you realize that there's 00:17:19.410 --> 00:17:23.430 a drug that costs 1/100 as much, which according 00:17:23.430 --> 00:17:26.400 to the clinical trials that we have on record 00:17:26.400 --> 00:17:29.760 is just as effective as the one that you've ordered? 00:17:29.760 --> 00:17:34.560 And so for example, here at the Beth Israel many years ago, 00:17:34.560 --> 00:17:36.640 they implemented a system like that. 00:17:36.640 --> 00:17:38.850 And in the first year, they showed 00:17:38.850 --> 00:17:43.320 that they saved something like $16 million in the pharmacy, 00:17:43.320 --> 00:17:46.700 just by ordering cheaper variants of drugs that 00:17:46.700 --> 00:17:48.630 could have been very expensive. 00:17:48.630 --> 00:17:50.400 And they also found that the doctors who 00:17:50.400 --> 00:17:54.750 were doing the ordering were perfectly satisfied with that, 00:17:54.750 --> 00:17:58.860 because they just didn't know how expensive these drugs were. 00:17:58.860 --> 00:18:02.610 That's not one of the things that they pay attention to. 00:18:02.610 --> 00:18:05.460 So there are many applications like that 00:18:05.460 --> 00:18:08.100 that are driven by this. 00:18:08.100 --> 00:18:10.320 And again, here are some statistics. 00:18:10.320 --> 00:18:12.630 You can reduce error rates by half. 00:18:12.630 --> 00:18:20.880 You can reduce severe medication errors by 88%. 00:18:20.880 --> 00:18:26.310 You can have a 70% reduction in antibiotic-related adverse drug 00:18:26.310 --> 00:18:27.600 events. 00:18:27.600 --> 00:18:31.200 You can reduce length of stay, which is another big goal 00:18:31.200 --> 00:18:33.720 that people go after. 00:18:33.720 --> 00:18:38.370 And at least if you're an optimist, 00:18:38.370 --> 00:18:41.100 you can believe these extrapolations 00:18:41.100 --> 00:18:44.100 that say, well, we could prevent 3 million adverse drug 00:18:44.100 --> 00:18:49.350 events at big city hospitals in the United States 00:18:49.350 --> 00:18:53.520 if everybody used systems like this. 00:18:53.520 --> 00:18:57.330 So the benefits are that it prompts 00:18:57.330 --> 00:19:00.930 with warnings against possible drug interactions, allergies, 00:19:00.930 --> 00:19:03.570 or overdoses. 00:19:03.570 --> 00:19:07.710 It can be kept up to date by some sort of mechanism where 00:19:07.710 --> 00:19:09.960 people read the literature and keep 00:19:09.960 --> 00:19:13.740 updating the databases this is driven from. 00:19:13.740 --> 00:19:19.890 And it can do mechanical things like eliminate confusion 00:19:19.890 --> 00:19:22.740 about drug names that sound similar. 00:19:22.740 --> 00:19:24.510 Stuff like that. 00:19:24.510 --> 00:19:27.890 So the Leapfrog Group, which does a lot of meta analyses 00:19:27.890 --> 00:19:30.330 and studies of what's effective, really 00:19:30.330 --> 00:19:35.100 is behind this and pushing it very strongly. 00:19:35.100 --> 00:19:37.890 Potential future benefits, of course, 00:19:37.890 --> 00:19:40.680 are that if the kinds of machine learning techniques 00:19:40.680 --> 00:19:43.920 that we talk about become widely used, 00:19:43.920 --> 00:19:46.950 then these systems can be updated automatically rather 00:19:46.950 --> 00:19:48.930 than by manual review. 00:19:48.930 --> 00:19:52.980 And you can gain the advantages of immediate feedback 00:19:52.980 --> 00:19:57.020 as new information becomes available. 00:19:57.020 --> 00:20:04.610 Now the adoption of CPOE was recommended by the National 00:20:04.610 --> 00:20:06.380 Academy of Medicine. 00:20:06.380 --> 00:20:11.360 They wanted every hospital to use this by 1999. 00:20:11.360 --> 00:20:13.010 And of course, it hasn't happened. 00:20:13.010 --> 00:20:18.140 So I couldn't find current data, but 2014 data 00:20:18.140 --> 00:20:21.980 shows that CPOE, for example, for medication orders, 00:20:21.980 --> 00:20:25.880 is only being used in about 25% of the hospitals. 00:20:25.880 --> 00:20:30.540 And at that time, people were extrapolating and saying, well, 00:20:30.540 --> 00:20:34.040 it's not going to reach 80% penetration until the year 00:20:34.040 --> 00:20:35.720 2029. 00:20:35.720 --> 00:20:39.260 So it's a very slow adoption cycle. 00:20:39.260 --> 00:20:42.150 Maybe it's gotten better. 00:20:42.150 --> 00:20:45.920 The other problem-- and one of the reasons for resistance-- 00:20:45.920 --> 00:20:50.300 is that it puts additional stresses on people. 00:20:50.300 --> 00:20:54.680 So for example, this is a study of how 00:20:54.680 --> 00:20:57.110 pharmacists spend their time. 00:20:57.110 --> 00:21:01.770 So clinical time is useful. 00:21:01.770 --> 00:21:04.310 That's when they're consulting with doctors, 00:21:04.310 --> 00:21:06.500 helping them figure out appropriate dosage 00:21:06.500 --> 00:21:07.790 for patients. 00:21:07.790 --> 00:21:10.790 Or they're talking to patients, explaining to them 00:21:10.790 --> 00:21:14.330 how to take their medications, what side effects to watch out 00:21:14.330 --> 00:21:15.770 for, et cetera. 00:21:15.770 --> 00:21:19.820 These distributive tasks-- it's a funny term-- 00:21:19.820 --> 00:21:24.590 mean the non-clinical part of what they're doing. 00:21:24.590 --> 00:21:27.380 And what you see is that hospitals that 00:21:27.380 --> 00:21:33.020 have adopted CPOE, they wind up spending a little bit more time 00:21:33.020 --> 00:21:36.590 on the distributive tasks and a little bit less time 00:21:36.590 --> 00:21:37.850 on the clinical tasks. 00:21:37.850 --> 00:21:40.820 Which is probably not in the right direction, 00:21:40.820 --> 00:21:43.520 in terms of what pharmacists were hoping 00:21:43.520 --> 00:21:47.190 for out of systems like this. 00:21:47.190 --> 00:21:49.740 Now people have studied the diffusion 00:21:49.740 --> 00:21:52.440 of new medical technologies. 00:21:52.440 --> 00:21:55.920 And I think I'll just show you the graph. 00:21:55.920 --> 00:22:01.950 So this is in England, but this is the adoption for statins. 00:22:01.950 --> 00:22:04.140 So from the time they were introduced-- 00:22:04.140 --> 00:22:08.670 statins is the drug that keeps your cholesterol low. 00:22:08.670 --> 00:22:10.350 From the time they were introduced 00:22:10.350 --> 00:22:12.930 until they were being used, essentially, 00:22:12.930 --> 00:22:18.390 at 100% of places was about five and a half, six years. 00:22:18.390 --> 00:22:20.820 So reasonably fast. 00:22:20.820 --> 00:22:25.110 If you look at the adoption of magnetic resonance imaging 00:22:25.110 --> 00:22:29.070 technology, it took five years for it 00:22:29.070 --> 00:22:31.470 to have any adoption whatsoever. 00:22:31.470 --> 00:22:34.470 And that's because it was insanely expensive. 00:22:34.470 --> 00:22:37.170 So there were all kinds of limitations. 00:22:37.170 --> 00:22:39.540 You know, even in Massachusetts, you 00:22:39.540 --> 00:22:43.110 have to get permission from some state committee 00:22:43.110 --> 00:22:45.420 to buy a new MRI machine. 00:22:45.420 --> 00:22:49.260 And if another hospital in your town already had one, 00:22:49.260 --> 00:22:51.480 then they would say, well, you shouldn't buy one 00:22:51.480 --> 00:22:54.900 because you should be able to use this other hospital's MRI 00:22:54.900 --> 00:22:56.310 machine. 00:22:56.310 --> 00:22:58.530 Same thing happened with CT. 00:22:58.530 --> 00:23:02.190 But as soon as those limitations were lifted, boom. 00:23:05.040 --> 00:23:08.580 It went up and then continues to go up. 00:23:08.580 --> 00:23:11.880 Whereas stents, I actually don't know why 00:23:11.880 --> 00:23:14.590 they were delayed by that long. 00:23:14.590 --> 00:23:18.180 But this is for people with blockages in coronary arteries 00:23:18.180 --> 00:23:19.560 or other arteries. 00:23:19.560 --> 00:23:22.710 You can put in a little mesh tube that 00:23:22.710 --> 00:23:24.870 just keeps that artery open. 00:23:24.870 --> 00:23:28.770 And that adoption was incredibly quick. 00:23:28.770 --> 00:23:31.470 So different things get adopted at different rates. 00:23:39.410 --> 00:23:42.390 Now the last topic I want to talk about before-- 00:23:42.390 --> 00:23:42.890 yeah. 00:23:42.890 --> 00:23:45.240 AUDIENCE: So what happens in those years 00:23:45.240 --> 00:23:46.650 where you just have spikes? 00:23:46.650 --> 00:23:49.010 What's doing it? 00:23:49.010 --> 00:23:51.800 PETER SZOLOVITS: So according to those authors, 00:23:51.800 --> 00:23:55.190 in the case of stents, there were some champions 00:23:55.190 --> 00:23:58.070 of the idea of stenting who went around 00:23:58.070 --> 00:24:00.680 and convinced their colleagues that this was 00:24:00.680 --> 00:24:03.590 the right technology to use. 00:24:03.590 --> 00:24:08.150 So there was just an explosive growth in it. 00:24:08.150 --> 00:24:12.230 In the other technologies, in the MRI case, 00:24:12.230 --> 00:24:15.200 money mattered a lot because they're so expensive. 00:24:15.200 --> 00:24:17.690 Stents are relatively cheap. 00:24:17.690 --> 00:24:22.070 And in the case of statins, those 00:24:22.070 --> 00:24:24.530 are also relatively cheap. 00:24:24.530 --> 00:24:27.350 Or they've become cheap since they went off patent. 00:24:27.350 --> 00:24:30.252 Originally, they were much more expensive. 00:24:32.910 --> 00:24:35.590 But there are still adoption problems. 00:24:35.590 --> 00:24:39.220 So for example, there was a recommendation-- 00:24:39.220 --> 00:24:42.960 I think about 15, maybe even 20 years ago-- 00:24:42.960 --> 00:24:45.330 that said that anybody who has had a heart 00:24:45.330 --> 00:24:49.140 attack or coronary artery disease 00:24:49.140 --> 00:24:52.170 should be taking beta blockers. 00:24:52.170 --> 00:24:55.470 And I don't remember what the adoption rate is today, 00:24:55.470 --> 00:24:59.040 but it's only on the order of a half. 00:24:59.040 --> 00:25:02.730 And so why? 00:25:02.730 --> 00:25:06.000 This is a dirt cheap drug. 00:25:06.000 --> 00:25:08.790 For reasons not quite understood, 00:25:08.790 --> 00:25:11.820 it reduces the probability of having a second heart 00:25:11.820 --> 00:25:15.340 attack by about 35%. 00:25:15.340 --> 00:25:17.340 So it's a really cheap protective way 00:25:17.340 --> 00:25:19.560 of keeping people healthier. 00:25:19.560 --> 00:25:23.990 And yet it just hasn't suffused practice as much 00:25:23.990 --> 00:25:25.620 as people think it should have. 00:25:29.428 --> 00:25:31.280 All right. 00:25:31.280 --> 00:25:34.820 So how do we assure the quality of these technologies 00:25:34.820 --> 00:25:38.390 before we foist them on the world? 00:25:38.390 --> 00:25:40.410 This is tricky. 00:25:40.410 --> 00:25:44.960 So John Ioannidis, a Stanford professor, 00:25:44.960 --> 00:25:48.230 has made an extremely successful career 00:25:48.230 --> 00:25:51.800 out of pointing out that most biomedical research is crap. 00:25:54.330 --> 00:25:56.830 It can't be reproduced. 00:25:56.830 --> 00:25:59.620 And there are some famous publications 00:25:59.620 --> 00:26:04.720 that show that people have taken some area of biomedicine, 00:26:04.720 --> 00:26:08.260 and they've looked at a bunch of well-respected published 00:26:08.260 --> 00:26:09.245 studies. 00:26:09.245 --> 00:26:10.870 And they've gone to the lab and they've 00:26:10.870 --> 00:26:13.630 tried to replicate those studies. 00:26:13.630 --> 00:26:16.090 Half the time or three-quarters of the time, 00:26:16.090 --> 00:26:18.180 they fail to do so. 00:26:18.180 --> 00:26:21.960 You go, oh my god, this is horrible. 00:26:21.960 --> 00:26:22.760 It is horrible. 00:26:22.760 --> 00:26:23.455 Yeah. 00:26:23.455 --> 00:26:25.956 AUDIENCE: You mean like they failed to do so, 00:26:25.956 --> 00:26:29.374 so they won't reproduce the exact same results? 00:26:29.374 --> 00:26:31.710 Or what exactly-- 00:26:31.710 --> 00:26:33.250 PETER SZOLOVITS: Worse than that. 00:26:33.250 --> 00:26:35.730 So it's not that there are slight differences. 00:26:35.730 --> 00:26:37.770 It's that, for example, a result that 00:26:37.770 --> 00:26:42.210 was shown to be statistically significant in one study, when 00:26:42.210 --> 00:26:44.280 they repeat the study, is no longer 00:26:44.280 --> 00:26:46.920 statistically significant. 00:26:46.920 --> 00:26:52.710 That's bad, if you base policy on that kind of decision. 00:26:52.710 --> 00:26:56.460 So Ioannidis has a suggestion, which 00:26:56.460 --> 00:26:58.560 would probably help a lot. 00:26:58.560 --> 00:27:01.110 And that is, basically, make known 00:27:01.110 --> 00:27:04.960 to everybody all the studies that have failed. 00:27:04.960 --> 00:27:08.760 So the problem is that if you give me a big data set 00:27:08.760 --> 00:27:11.580 and I start mining this data set, 00:27:11.580 --> 00:27:14.880 I'm going to find tons and tons of interesting correlations 00:27:14.880 --> 00:27:17.160 in this data. 00:27:17.160 --> 00:27:21.180 And as soon as I get one that has a good p value, 00:27:21.180 --> 00:27:24.780 my students and I go, fantastic. 00:27:24.780 --> 00:27:25.500 Time to publish. 00:27:28.620 --> 00:27:30.930 Now consider the fact that I'm not 00:27:30.930 --> 00:27:33.030 the only person in this role. 00:27:33.030 --> 00:27:35.770 So you know, David's group is doing the same thing. 00:27:35.770 --> 00:27:39.600 And John Guttag's and Regina Barzilay's and all 00:27:39.600 --> 00:27:42.670 of our colleagues at every other major university 00:27:42.670 --> 00:27:45.600 and hospital in the United States. 00:27:45.600 --> 00:27:48.480 So there may be hundreds of people 00:27:48.480 --> 00:27:50.700 who are mining this data. 00:27:50.700 --> 00:27:53.710 And each of us has slightly different ways of doing it. 00:27:53.710 --> 00:27:55.770 We select our cases differently. 00:27:55.770 --> 00:27:58.380 We preprocess the data differently. 00:27:58.380 --> 00:28:02.310 We apply different learning algorithms to them. 00:28:02.310 --> 00:28:05.610 But just by random chance, some of us 00:28:05.610 --> 00:28:09.240 are going to find interesting results, interesting patterns. 00:28:09.240 --> 00:28:12.960 And of course, those are the ones that get published. 00:28:12.960 --> 00:28:16.150 Because if you don't find an interesting result, 00:28:16.150 --> 00:28:19.080 you're not going to submit it to a journal and say, 00:28:19.080 --> 00:28:22.170 you know I looked for the following fact phenomenon 00:28:22.170 --> 00:28:24.450 and I was unable to find it. 00:28:24.450 --> 00:28:26.850 Because the journal says, well, that's 00:28:26.850 --> 00:28:30.910 not interesting to anybody. 00:28:30.910 --> 00:28:35.820 So Ioannidis is recommending that, basically, 00:28:35.820 --> 00:28:38.370 every study that anybody undertakes should 00:28:38.370 --> 00:28:40.170 be registered. 00:28:40.170 --> 00:28:42.660 And if you don't get a significant result, 00:28:42.660 --> 00:28:44.250 that should be known. 00:28:44.250 --> 00:28:46.230 And this would allow us to make at least 00:28:46.230 --> 00:28:50.130 some reasonable estimate of whether the significant results 00:28:50.130 --> 00:28:53.940 that were gotten are just the statistical outliers that 00:28:53.940 --> 00:28:58.830 happened to reach p equal 0.05 or whatever your threshold is, 00:28:58.830 --> 00:29:03.253 or whether it's a real effect because not that many people 00:29:03.253 --> 00:29:04.170 have been trying this. 00:29:04.170 --> 00:29:04.670 Yeah. 00:29:04.670 --> 00:29:07.120 AUDIENCE: [INAUDIBLE] why do you think this is? 00:29:07.120 --> 00:29:09.495 Is it because of the size of some core patients? 00:29:09.495 --> 00:29:10.460 Or bias in the assay? 00:29:10.460 --> 00:29:14.000 Or just purely randomness in the study? 00:29:14.000 --> 00:29:15.750 PETER SZOLOVITS: It could be any of those. 00:29:15.750 --> 00:29:20.310 It could be that your hospital has some biased data 00:29:20.310 --> 00:29:21.520 collection. 00:29:21.520 --> 00:29:22.950 And so you find an effect. 00:29:22.950 --> 00:29:26.610 My hospital doesn't, and so I don't find it. 00:29:26.610 --> 00:29:30.020 It could be that we just randomly sub-sampled 00:29:30.020 --> 00:29:31.920 a different sample of the population. 00:29:38.110 --> 00:29:39.190 So it's very interesting. 00:29:39.190 --> 00:29:40.980 Last year I was invited to a meeting 00:29:40.980 --> 00:29:45.330 by Jeff Drazen, who's the executive editor of the New 00:29:45.330 --> 00:29:46.470 England Journal. 00:29:46.470 --> 00:29:48.270 And he's thinking about-- 00:29:48.270 --> 00:29:50.160 has not decided-- but he's thinking 00:29:50.160 --> 00:29:53.040 about a policy for the New England Journal, which 00:29:53.040 --> 00:29:57.060 is like the top medical journal, that says that he will not 00:29:57.060 --> 00:30:00.570 publish any result unless it's been replicated 00:30:00.570 --> 00:30:04.530 on two independent data sets. 00:30:04.530 --> 00:30:05.580 So that's interesting. 00:30:05.580 --> 00:30:09.030 And that's an attempt to fight back against this problem. 00:30:09.030 --> 00:30:15.060 It's a different solution than what Ioannidis is recommending. 00:30:15.060 --> 00:30:20.070 So this was a study by Enrico Carrara. 00:30:20.070 --> 00:30:23.910 And he's talking about what it means to replicate. 00:30:23.910 --> 00:30:26.190 And again, I'm not going to go through all this. 00:30:26.190 --> 00:30:29.190 But there's the notion that replication might 00:30:29.190 --> 00:30:31.820 mean exact replication, i.e. 00:30:31.820 --> 00:30:35.940 You do exactly the same thing on exactly the same kind of data, 00:30:35.940 --> 00:30:38.450 but in a different data set. 00:30:38.450 --> 00:30:42.510 And then partial replication, conceptual replication, 00:30:42.510 --> 00:30:45.480 which says, you follow the same procedures 00:30:45.480 --> 00:30:47.850 but in a different environment. 00:30:47.850 --> 00:30:49.860 And then quasi replication-- 00:30:49.860 --> 00:30:52.020 either partial or conceptual. 00:30:52.020 --> 00:30:54.090 And these have various characteristics 00:30:54.090 --> 00:30:55.540 that you can look at. 00:30:55.540 --> 00:30:56.850 It's an interesting framework. 00:31:01.590 --> 00:31:04.980 So this is not a new idea. 00:31:04.980 --> 00:31:09.000 The first edition of this book, Evaluation Methods 00:31:09.000 --> 00:31:12.750 in Biomedical Informatics, was called Evaluation Methods 00:31:12.750 --> 00:31:16.110 in Medical Informatics by the same authors 00:31:16.110 --> 00:31:20.502 and was published a long time ago. 00:31:20.502 --> 00:31:21.210 I can't remember. 00:31:21.210 --> 00:31:24.270 This one is relatively recent. 00:31:24.270 --> 00:31:30.540 And so they do a multi-hundred page, very detailed evaluation 00:31:30.540 --> 00:31:33.540 of exactly how one should evaluate 00:31:33.540 --> 00:31:35.730 clinical systems like this. 00:31:35.730 --> 00:31:38.280 And it's very careful and very cautious, 00:31:38.280 --> 00:31:41.070 but it's also very conservative. 00:31:41.070 --> 00:31:44.250 So for example, one of the things that they recommend 00:31:44.250 --> 00:31:46.530 is that the people doing the evaluation 00:31:46.530 --> 00:31:50.300 should not be the people who developed the technique, 00:31:50.300 --> 00:31:53.540 because there's innately bias. 00:31:53.540 --> 00:31:56.850 You know, I want my technique to succeed. 00:31:56.850 --> 00:31:58.970 And so they say, hand it off to somebody 00:31:58.970 --> 00:32:01.920 else who doesn't have that same vested interest. 00:32:01.920 --> 00:32:06.290 And then you're going to get a more careful evaluation. 00:32:06.290 --> 00:32:09.740 So Steve Pauker and I wrote a response 00:32:09.740 --> 00:32:12.590 to one of their early papers recommending 00:32:12.590 --> 00:32:16.910 this that said, well, that's so conservative that it 00:32:16.910 --> 00:32:19.880 sort of throws the baby out with the bathwater. 00:32:19.880 --> 00:32:23.810 Because if you make it so difficult to do an evaluation, 00:32:23.810 --> 00:32:26.630 you'll never get anything past it. 00:32:26.630 --> 00:32:31.070 So we proposed instead a kind of staged evaluation 00:32:31.070 --> 00:32:35.030 that says, first of all, you should do regression testing 00:32:35.030 --> 00:32:39.320 so that every time you use these agile development methods, 00:32:39.320 --> 00:32:42.710 you should have the set of cases that your program has 00:32:42.710 --> 00:32:44.300 worked on before. 00:32:44.300 --> 00:32:46.340 You should automatically rerun them 00:32:46.340 --> 00:32:48.770 and see which ones you've made better 00:32:48.770 --> 00:32:50.630 and which ones you've made worse. 00:32:50.630 --> 00:32:52.100 And that will give you some insight 00:32:52.100 --> 00:32:55.340 into whether what you're doing is reasonable. 00:32:55.340 --> 00:32:57.950 Then you might also build tools that 00:32:57.950 --> 00:33:02.180 look at automating ways of looking for inconsistencies 00:33:02.180 --> 00:33:05.180 in the models that you're building. 00:33:05.180 --> 00:33:08.660 Then you have retrospective review, judged by clinicians. 00:33:08.660 --> 00:33:11.570 So you run a program that you like 00:33:11.570 --> 00:33:14.630 over a whole bunch of existing data, 00:33:14.630 --> 00:33:18.830 like what you're doing with Mimic or with Market Scan. 00:33:18.830 --> 00:33:20.840 And then you do it prospectively, 00:33:20.840 --> 00:33:24.470 but without actually affecting patients. 00:33:24.470 --> 00:33:28.670 So you do it in real time as the data is coming in, 00:33:28.670 --> 00:33:32.930 but you don't tell anybody what the program results in. 00:33:32.930 --> 00:33:35.990 You just ask them to evaluate in retrospect 00:33:35.990 --> 00:33:38.510 to see whether it was right. 00:33:38.510 --> 00:33:40.550 And you might say, well, what's the difference 00:33:40.550 --> 00:33:42.770 between collecting the data in real time 00:33:42.770 --> 00:33:46.190 and collecting the data retrospectively? 00:33:46.190 --> 00:33:49.170 Historically, the answer is there is a difference. 00:33:49.170 --> 00:33:51.950 So circumstances differ. 00:33:51.950 --> 00:33:55.740 The mechanisms that you have for collecting the data differ. 00:33:55.740 --> 00:34:00.140 So this turns out to be an important issue. 00:34:00.140 --> 00:34:03.860 And then you can run a prospective controlled trial 00:34:03.860 --> 00:34:07.430 where you're interested in evaluating both the answer 00:34:07.430 --> 00:34:10.489 that you get from the program, and ultimately 00:34:10.489 --> 00:34:13.679 the effect on health outcomes. 00:34:13.679 --> 00:34:16.460 So if I have a decision support system, 00:34:16.460 --> 00:34:18.560 the ultimate proof of the pudding 00:34:18.560 --> 00:34:21.949 is if I run that decision support system. 00:34:21.949 --> 00:34:25.370 I give advice to clinicians, the clinicians 00:34:25.370 --> 00:34:29.239 change their behavior sometimes, and the patients 00:34:29.239 --> 00:34:30.739 get a better outcome. 00:34:30.739 --> 00:34:33.960 Then I'm convinced that this is really useful. 00:34:33.960 --> 00:34:35.719 But you have to get there slowly, 00:34:35.719 --> 00:34:38.659 because you don't want to give them worse outcomes. 00:34:38.659 --> 00:34:41.914 That's unethical and probably illegal. 00:34:47.260 --> 00:34:49.690 And you want to compare this to the performance 00:34:49.690 --> 00:34:52.130 of unaided doctors. 00:34:52.130 --> 00:34:54.460 So the Food and Drug Administration 00:34:54.460 --> 00:34:57.370 has been dealing with this issue for many, many years. 00:34:57.370 --> 00:35:01.540 I remember talking to them in about 1976, when 00:35:01.540 --> 00:35:05.140 they were reading about the very first expert system 00:35:05.140 --> 00:35:09.340 programs for diagnosis and therapy selection. 00:35:09.340 --> 00:35:12.100 And they said, well, how should we regulate these? 00:35:12.100 --> 00:35:15.580 And my response at the time was, God help us. 00:35:15.580 --> 00:35:17.830 Keep your hands off. 00:35:17.830 --> 00:35:21.760 Because if you regulate it, then you're 00:35:21.760 --> 00:35:23.830 going to slow down progress. 00:35:23.830 --> 00:35:27.040 And in any case, none of these programs are being used. 00:35:27.040 --> 00:35:29.320 These programs are being developed 00:35:29.320 --> 00:35:33.190 as experimental programs in experimental settings. 00:35:33.190 --> 00:35:35.320 They're not coming anywhere close to being 00:35:35.320 --> 00:35:37.700 used on real patients. 00:35:37.700 --> 00:35:40.730 And so there is not a regulatory issue. 00:35:40.730 --> 00:35:46.000 And about every five years, FDA has revisited that question. 00:35:46.000 --> 00:35:50.590 And they have continued to make essentially the same decision, 00:35:50.590 --> 00:35:53.230 based on the rationale that, for example, they 00:35:53.230 --> 00:35:55.270 don't regulate books. 00:35:55.270 --> 00:35:57.760 If I write a textbook that explains something 00:35:57.760 --> 00:36:01.930 about medicine, the FDA is not going 00:36:01.930 --> 00:36:04.310 to see whether it's correct or not. 00:36:04.310 --> 00:36:06.460 And the reason is because the expectation 00:36:06.460 --> 00:36:10.000 is that the textbook is making recommendations, 00:36:10.000 --> 00:36:14.500 so to speak, to clinical practitioners who are 00:36:14.500 --> 00:36:17.290 responsible experts themselves. 00:36:17.290 --> 00:36:20.140 So the ultimate responsibility for how they behave 00:36:20.140 --> 00:36:23.390 rests with them and not with the textbook. 00:36:23.390 --> 00:36:26.050 And they said, we're going to treat these computer programs 00:36:26.050 --> 00:36:28.960 as if they were dynamic textbooks, rather 00:36:28.960 --> 00:36:33.460 than colleagues who are acting independently and giving 00:36:33.460 --> 00:36:34.540 advice. 00:36:34.540 --> 00:36:37.630 Now as soon as you try to give that advice, not 00:36:37.630 --> 00:36:41.200 to a professional, but to a patient, then 00:36:41.200 --> 00:36:45.520 you are immediately under the regulatory auspices of FDA. 00:36:45.520 --> 00:36:48.970 Because now there is no professional intermediate 00:36:48.970 --> 00:36:52.820 that can evaluate the quality of that advice. 00:36:52.820 --> 00:36:58.840 So what FDA has done, just in the past year, 00:36:58.840 --> 00:37:03.130 is they've said that we're going to treat 00:37:03.130 --> 00:37:08.830 these AI-based quote-unquote devices as medical devices. 00:37:08.830 --> 00:37:12.790 And we're going to apply the same regulatory requirements 00:37:12.790 --> 00:37:16.660 that we have for these devices, except we don't really 00:37:16.660 --> 00:37:18.310 know how to do this. 00:37:18.310 --> 00:37:21.790 So there's a kind of experiment going on right now 00:37:21.790 --> 00:37:26.320 where they're saying, OK, submit applications for review 00:37:26.320 --> 00:37:28.570 of these devices to us. 00:37:28.570 --> 00:37:30.610 We will review them. 00:37:30.610 --> 00:37:36.640 And we will use these criteria-- 00:37:36.640 --> 00:37:40.510 product quality, patient safety, clinical responsibility, 00:37:40.510 --> 00:37:44.590 cybersecurity responsibility, and a so-called proactive 00:37:44.590 --> 00:37:48.040 culture in the organization that's developing them-- 00:37:48.040 --> 00:37:50.860 in order to make a judgment of whether or not to let you 00:37:50.860 --> 00:37:55.640 proceed with marketing one of these things. 00:37:55.640 --> 00:38:00.610 So if you look, there are in fact about 10 devices, 00:38:00.610 --> 00:38:03.070 quote-unquote-- these are all software-- 00:38:03.070 --> 00:38:06.160 that have been approved so far by FDA. 00:38:06.160 --> 00:38:10.150 And almost all of them are imaging devices. 00:38:10.150 --> 00:38:14.170 They're things that do convolutional networks 00:38:14.170 --> 00:38:16.520 over one thing or another. 00:38:16.520 --> 00:38:19.540 And so here are just a few examples. 00:38:19.540 --> 00:38:25.000 Imagen has OsteoDetect, which analyzes two-dimensional X-ray 00:38:25.000 --> 00:38:28.150 images for signs of distal radius fracture. 00:38:28.150 --> 00:38:32.530 So if you break your wrist, then this system 00:38:32.530 --> 00:38:36.100 will look at the X-ray and decide whether or not 00:38:36.100 --> 00:38:38.200 you've done that. 00:38:38.200 --> 00:38:44.450 Here's one from IDx, which looks at the photographs 00:38:44.450 --> 00:38:47.410 of your retina and decides whether you 00:38:47.410 --> 00:38:49.540 have diabetic retinopathy. 00:38:49.540 --> 00:38:51.130 And actually, they've published a lot 00:38:51.130 --> 00:38:55.030 of papers that show that they can also identify heart disease 00:38:55.030 --> 00:38:57.760 and stroke risk and various other things 00:38:57.760 --> 00:38:59.740 from those same photographs. 00:38:59.740 --> 00:39:04.970 So FDA has granted them approval to market this thing. 00:39:04.970 --> 00:39:10.660 Another one is Viz, which automatically analyzes 00:39:10.660 --> 00:39:15.910 CT scans for ER patients and is looking for blockages 00:39:15.910 --> 00:39:18.550 and major brain blood vessels. 00:39:18.550 --> 00:39:21.190 So this can obviously lead to a stroke. 00:39:21.190 --> 00:39:25.090 And this is an automated technique that does that. 00:39:25.090 --> 00:39:27.390 Here's another one. 00:39:27.390 --> 00:39:32.860 Arterys measures and tracks tumors or potential cancers 00:39:32.860 --> 00:39:34.105 in radiology images. 00:39:36.700 --> 00:39:41.700 So these are the ones that have been approved. 00:39:41.700 --> 00:39:43.560 And then I just wanted to remind you 00:39:43.560 --> 00:39:46.860 that there's actually plenty of literature 00:39:46.860 --> 00:39:48.810 about this kind of stuff. 00:39:48.810 --> 00:39:52.200 So the book on the left actually comes out next week. 00:39:54.810 --> 00:39:58.860 I got to read a pre-print of it, by Eric Topol, 00:39:58.860 --> 00:40:00.720 who's one of these doctors who writes 00:40:00.720 --> 00:40:03.750 a lot about the future of medicine. 00:40:03.750 --> 00:40:06.630 And he actually goes through tons and tons 00:40:06.630 --> 00:40:09.450 of examples of not only the systems 00:40:09.450 --> 00:40:12.030 that have been approved by FDA, but also 00:40:12.030 --> 00:40:13.980 things that are in the works that he's 00:40:13.980 --> 00:40:18.000 very optimistic that these will again revolutionize 00:40:18.000 --> 00:40:20.220 the practice of medicine. 00:40:20.220 --> 00:40:23.700 Bob Wachter, who wrote the book on the left a couple of years 00:40:23.700 --> 00:40:26.700 ago, is a little bit more cautious 00:40:26.700 --> 00:40:30.900 because he's chief of medicine at UC San Francisco. 00:40:30.900 --> 00:40:33.090 And he wrote this book in response 00:40:33.090 --> 00:40:36.870 to them almost killing a kid by giving him 00:40:36.870 --> 00:40:42.280 a 39x overdose of a medication. 00:40:42.280 --> 00:40:45.260 They didn't quite succeed in killing the kid. 00:40:45.260 --> 00:40:47.290 So it turned out OK. 00:40:47.290 --> 00:40:49.960 But he was really concerned about how 00:40:49.960 --> 00:40:54.040 this wonderful technology led to such a disastrous outcome. 00:40:54.040 --> 00:40:56.500 And so he spent a year studying how 00:40:56.500 --> 00:40:58.870 these systems were being used, and writes a more 00:40:58.870 --> 00:41:01.610 cautionary tale. 00:41:01.610 --> 00:41:06.230 So let me turn to Adam, who as I said, 00:41:06.230 --> 00:41:11.160 is a professor at the Brigham and Harvard Medical School. 00:41:11.160 --> 00:41:16.625 Please come and join me, and we can have a conversation. 00:41:16.625 --> 00:41:18.250 ADAM WRIGHT: So my name is Adam Wright. 00:41:18.250 --> 00:41:20.590 I'm an associate professor of medicine at Harvard Medical 00:41:20.590 --> 00:41:21.090 School. 00:41:21.090 --> 00:41:23.630 In that role, I lead a research program 00:41:23.630 --> 00:41:26.603 and I teach the introduction to biomedical informatics courses 00:41:26.603 --> 00:41:27.520 at the medical school. 00:41:27.520 --> 00:41:29.103 So if you're interested in the topics 00:41:29.103 --> 00:41:30.520 that Pete was talking about today, 00:41:30.520 --> 00:41:32.560 you should definitely consider cross-registering 00:41:32.560 --> 00:41:34.810 in VMI 701 or 702. 00:41:34.810 --> 00:41:37.120 The medical school certainly always 00:41:37.120 --> 00:41:39.970 could use a few more enthusiastic and 00:41:39.970 --> 00:41:44.270 technically-minded machine learning experts in our course. 00:41:44.270 --> 00:41:48.070 And then I have a operational job at Partners. 00:41:48.070 --> 00:41:49.870 Partners is the health system that 00:41:49.870 --> 00:41:52.540 includes Mass General Hospital and the Brigham 00:41:52.540 --> 00:41:54.160 and some community hospitals. 00:41:54.160 --> 00:41:57.370 And I work on Partners eCare, which 00:41:57.370 --> 00:42:00.460 is our kind of cool brand name for Epic. 00:42:00.460 --> 00:42:03.100 So Epic is the EHR that we use at Partners. 00:42:03.100 --> 00:42:06.382 And I help oversee the clinical decision support there. 00:42:06.382 --> 00:42:07.840 So we have a decision support team. 00:42:07.840 --> 00:42:10.388 I'm the clinical lead for monitoring and evaluation. 00:42:10.388 --> 00:42:12.430 And so I help make sure that our decision support 00:42:12.430 --> 00:42:15.920 systems of the type that Pete's talking about work correctly. 00:42:15.920 --> 00:42:18.338 So that's my job at the Brigham and at Partners. 00:42:18.338 --> 00:42:19.255 PETER SZOLOVITS: Cool. 00:42:19.255 --> 00:42:21.167 And I appreciate it very much. 00:42:21.167 --> 00:42:22.000 ADAM WRIGHT: Thanks. 00:42:22.000 --> 00:42:23.167 I appreciate the invitation. 00:42:23.167 --> 00:42:24.350 It's fun to be here. 00:42:24.350 --> 00:42:27.310 PETER SZOLOVITS: So Adam, the first obvious question 00:42:27.310 --> 00:42:30.340 is what kind of decision support systems 00:42:30.340 --> 00:42:32.420 have you guys actually put in place? 00:42:32.420 --> 00:42:33.420 ADAM WRIGHT: Absolutely. 00:42:33.420 --> 00:42:36.640 So we've had a long history at the Brigham and Partners 00:42:36.640 --> 00:42:38.680 of using decision support. 00:42:38.680 --> 00:42:41.330 Historically, we developed our own electronic health record, 00:42:41.330 --> 00:42:43.480 which was a little bit unusual. 00:42:43.480 --> 00:42:45.040 About three years ago, we switched 00:42:45.040 --> 00:42:47.507 from our self-developed system to Epic, 00:42:47.507 --> 00:42:49.840 which is a very widely-used commercial electronic health 00:42:49.840 --> 00:42:50.340 record. 00:42:50.340 --> 00:42:52.618 And to the point that you gave, we really 00:42:52.618 --> 00:42:54.660 started with a lot of medication-related decision 00:42:54.660 --> 00:42:55.160 support. 00:42:55.160 --> 00:42:57.655 So that's things like drug interaction, alerting. 00:42:57.655 --> 00:43:00.280 So you prescribe two drugs that might interact with each other. 00:43:00.280 --> 00:43:02.120 And we use a table-- 00:43:02.120 --> 00:43:04.660 no machine learning or anything too complicated-- that says, 00:43:04.660 --> 00:43:06.190 we think this drug might interact with this. 00:43:06.190 --> 00:43:08.315 We raise an alert to the doctor, to the pharmacist. 00:43:08.315 --> 00:43:10.512 And they make a decision, using their expertise 00:43:10.512 --> 00:43:12.220 as the learned intermediary, that they're 00:43:12.220 --> 00:43:13.928 going to continue with that prescription. 00:43:13.928 --> 00:43:16.240 Let's have some dosing support, allergy checking, 00:43:16.240 --> 00:43:17.510 and things like that. 00:43:17.510 --> 00:43:19.510 So our first set of decision support 00:43:19.510 --> 00:43:21.160 really was around medications. 00:43:21.160 --> 00:43:23.680 And then we turned to a broader set of things 00:43:23.680 --> 00:43:25.150 like preventative care reminders, 00:43:25.150 --> 00:43:28.240 so identifying patients that are overdue for a mammogram 00:43:28.240 --> 00:43:30.760 or a pap smear or that might benefit from a statin 00:43:30.760 --> 00:43:31.870 or something like that. 00:43:31.870 --> 00:43:35.440 Or a beta blocker, in the case of acute myocardial infarction. 00:43:35.440 --> 00:43:37.630 And we make suggestions to the doctor 00:43:37.630 --> 00:43:40.060 or to other members of the care team to do those things. 00:43:40.060 --> 00:43:42.820 Again, those historically have largely been rule-based. 00:43:42.820 --> 00:43:46.540 So some experts sat down and wrote Boolean if-then rules, 00:43:46.540 --> 00:43:48.910 using variables that are in a patient's chart. 00:43:48.910 --> 00:43:50.882 We have increasingly, though, started 00:43:50.882 --> 00:43:52.840 trying to use some predictive models for things 00:43:52.840 --> 00:43:55.990 like readmission or whether a patient is at risk of falling 00:43:55.990 --> 00:43:57.798 down in the hospital. 00:43:57.798 --> 00:43:59.590 A big problem that patients often encounter 00:43:59.590 --> 00:44:02.630 is they're in the hospital, they're kind of delirious. 00:44:02.630 --> 00:44:03.880 The hospital is a weird place. 00:44:03.880 --> 00:44:04.360 It's dark. 00:44:04.360 --> 00:44:05.780 They get up to go to the bathroom. 00:44:05.780 --> 00:44:08.830 They trip on their IV tubing, and then they 00:44:08.830 --> 00:44:09.820 fall and are injured. 00:44:09.820 --> 00:44:11.820 So we would like to prevent that from happening. 00:44:11.820 --> 00:44:13.760 Because that's obviously kind of a bad thing 00:44:13.760 --> 00:44:15.160 to happen to you once you're in the hospital. 00:44:15.160 --> 00:44:17.230 So we have some machine learning-based tools 00:44:17.230 --> 00:44:19.520 for predicting patients that are at risk for falls. 00:44:19.520 --> 00:44:21.250 And then there is a set of interventions 00:44:21.250 --> 00:44:24.190 like putting the bed rails up or putting an alarm that buzzes 00:44:24.190 --> 00:44:25.870 when if they get out of bed. 00:44:25.870 --> 00:44:28.625 Or in more extreme cases, having a sitter, like a person 00:44:28.625 --> 00:44:30.250 who actually sits in the room with them 00:44:30.250 --> 00:44:32.127 and tries to keep them from getting up 00:44:32.127 --> 00:44:33.460 or assists them to the bathroom. 00:44:33.460 --> 00:44:35.668 Or calls someone who can assist them to the bathroom. 00:44:35.668 --> 00:44:37.420 So we have increasingly started using 00:44:37.420 --> 00:44:39.610 those machine learning tools. 00:44:39.610 --> 00:44:41.470 Some of which we get from third parties, 00:44:41.470 --> 00:44:42.910 like from our electronic health record vendor, 00:44:42.910 --> 00:44:45.220 and some of which we sort of train ourselves 00:44:45.220 --> 00:44:46.750 on our own data. 00:44:46.750 --> 00:44:49.728 That's a newer pursuit for us, is this machine learning. 00:44:49.728 --> 00:44:51.520 PETER SZOLOVITS: So when you have something 00:44:51.520 --> 00:44:53.410 like a risk model, how do you decide 00:44:53.410 --> 00:44:55.390 where to set the threshold? 00:44:55.390 --> 00:45:00.650 You know, if I'm at 53% risk of falling, 00:45:00.650 --> 00:45:02.868 should you get a sitter to sit by my bedside? 00:45:02.868 --> 00:45:04.410 ADAM WRIGHT: It's complicated, right? 00:45:04.410 --> 00:45:06.202 I mean, I would like to say that what we do 00:45:06.202 --> 00:45:08.110 is a full kind of utility analysis, 00:45:08.110 --> 00:45:10.600 where we say, we pay a sitter this much per hour. 00:45:10.600 --> 00:45:12.190 And the risk of falling is this much. 00:45:12.190 --> 00:45:13.780 And the cost of a fall-- 00:45:13.780 --> 00:45:15.340 most patients who fall aren't hurt. 00:45:15.340 --> 00:45:16.270 But some are. 00:45:16.270 --> 00:45:20.350 And so you would calculate the cost-benefit 00:45:20.350 --> 00:45:22.510 of each of those things and figure out 00:45:22.510 --> 00:45:25.060 where on the ROC curve you want to place yourself. 00:45:25.060 --> 00:45:29.230 In practice, I think we often just play it by ear, 00:45:29.230 --> 00:45:31.300 in part because a lot of our things 00:45:31.300 --> 00:45:33.270 are intended to be suggestions. 00:45:33.270 --> 00:45:35.582 So our threshold for saying to the doctor, 00:45:35.582 --> 00:45:37.540 hey, this patient is at elevated risk for fall, 00:45:37.540 --> 00:45:40.270 consider doing something, is pretty low. 00:45:40.270 --> 00:45:42.840 If the system were, say, automatically ordering 00:45:42.840 --> 00:45:45.030 a sitter, we might set it higher. 00:45:45.030 --> 00:45:48.300 I would say that's an area of research. 00:45:48.300 --> 00:45:50.730 I would also say that one challenge we have is we 00:45:50.730 --> 00:45:53.750 often set and forget these kinds of systems. 00:45:53.750 --> 00:45:56.220 And so there is kind of feature drift and patients 00:45:56.220 --> 00:45:57.048 change over time. 00:45:57.048 --> 00:45:59.340 We probably should do a better job of then looking back 00:45:59.340 --> 00:46:01.007 to see how well they're actually working 00:46:01.007 --> 00:46:02.635 and making tweaks to the thresholds. 00:46:02.635 --> 00:46:03.510 Really good question. 00:46:03.510 --> 00:46:05.260 PETER SZOLOVITS: But these are, of course, 00:46:05.260 --> 00:46:07.260 very complicated decisions. 00:46:07.260 --> 00:46:13.230 I remember 50 years ago talking to some people in the Air Force 00:46:13.230 --> 00:46:18.120 about how much should they invest in safety measures. 00:46:18.120 --> 00:46:22.060 And they had a utility theoretic model 00:46:22.060 --> 00:46:26.340 that said, OK, how much does it cost to replace 00:46:26.340 --> 00:46:28.600 a pilot if you kill them? 00:46:28.600 --> 00:46:29.460 ADAM WRIGHT: Yikes. 00:46:29.460 --> 00:46:30.700 Yeah. 00:46:30.700 --> 00:46:34.118 PETER SZOLOVITS: And this was not publicized a lot. 00:46:34.118 --> 00:46:35.910 ADAM WRIGHT: I mean, we do calculate things 00:46:35.910 --> 00:46:37.410 like quality-adjusted life-years and 00:46:37.410 --> 00:46:38.760 disability-adjusted life-years. 00:46:38.760 --> 00:46:43.040 So there is-- in all of medicine as people deploy resources, 00:46:43.040 --> 00:46:43.800 this calculus. 00:46:43.800 --> 00:46:46.030 And I think we tend to assign a really high weight 00:46:46.030 --> 00:46:48.660 to patient harm, because patient harm is-- 00:46:48.660 --> 00:46:51.510 if you think about the oath the doctors swear, 00:46:51.510 --> 00:46:52.450 first do no harm. 00:46:52.450 --> 00:46:55.050 The worst thing we can do is harm you in the hospital. 00:46:55.050 --> 00:46:58.410 So I think we have a pretty strong aversion to do that. 00:46:58.410 --> 00:47:00.240 But it's very hard to weigh these things. 00:47:00.240 --> 00:47:02.430 I think one of the challenges we often run into 00:47:02.430 --> 00:47:05.292 is that different doctors would make different decisions. 00:47:05.292 --> 00:47:07.500 So if you put the same patient in front of 10 doctors 00:47:07.500 --> 00:47:09.592 and said, does this patient need a sitter? 00:47:09.592 --> 00:47:11.550 Maybe half would say yes and half would say no. 00:47:11.550 --> 00:47:13.050 So it's especially hard to know what 00:47:13.050 --> 00:47:15.030 to do with a decision support system 00:47:15.030 --> 00:47:17.070 if the humans can't agree on what you 00:47:17.070 --> 00:47:18.870 should do in that situation. 00:47:18.870 --> 00:47:20.995 PETER SZOLOVITS: So the other thing we talked about 00:47:20.995 --> 00:47:24.010 on the phone yesterday is I was concerned-- a few years ago, 00:47:24.010 --> 00:47:30.210 I was visiting one of these august Boston-area hospitals 00:47:30.210 --> 00:47:35.940 and asked to see an example of somebody interacting with this 00:47:35.940 --> 00:47:38.630 Computerized Physician Order Entry system. 00:47:38.630 --> 00:47:44.700 And the senior resident who was taking me around went up 00:47:44.700 --> 00:47:46.170 to the computer and said, well, I 00:47:46.170 --> 00:47:49.470 think I remember how to use this. 00:47:49.470 --> 00:47:52.650 And I said, wait a minute. 00:47:52.650 --> 00:47:56.340 This is something you're expected to use daily. 00:47:56.340 --> 00:47:59.730 But in reality, what happens is that it's not 00:47:59.730 --> 00:48:03.960 the senior doctors or even the medium senior doctors. 00:48:03.960 --> 00:48:06.690 It's the interns and the junior residents 00:48:06.690 --> 00:48:08.007 who actually use the systems. 00:48:08.007 --> 00:48:09.090 ADAM WRIGHT: This is true. 00:48:09.090 --> 00:48:11.640 PETER SZOLOVITS: And the concern I had 00:48:11.640 --> 00:48:17.820 was that it takes a junior resident with a lot of guts 00:48:17.820 --> 00:48:21.810 to go up to the chief of your service 00:48:21.810 --> 00:48:26.610 and say, doctor x, even though you asked me to order 00:48:26.610 --> 00:48:29.500 this drug for this patient, the computer 00:48:29.500 --> 00:48:33.000 is arguing back that you should use this other one instead. 00:48:33.000 --> 00:48:34.450 ADAM WRIGHT: Yeah, it does. 00:48:34.450 --> 00:48:36.658 And in fact, I actually thought of this a little more 00:48:36.658 --> 00:48:37.860 after we chatted about it. 00:48:37.860 --> 00:48:39.443 We've heard from residents that people 00:48:39.443 --> 00:48:41.400 have said to them, if you dare page me 00:48:41.400 --> 00:48:43.770 with an Epic suggestion in the middle of the night, 00:48:43.770 --> 00:48:45.150 I'll never talk to you again. 00:48:45.150 --> 00:48:48.150 So just override all of those alerts. 00:48:48.150 --> 00:48:51.990 I think that one of the challenges is-- 00:48:51.990 --> 00:48:55.080 and some culpability on our part-- 00:48:55.080 --> 00:48:57.840 is that a lot of these alerts we give 00:48:57.840 --> 00:49:00.930 have a PPV of like, 10 or 20%. 00:49:00.930 --> 00:49:02.727 They are usually wrong. 00:49:02.727 --> 00:49:04.560 We think it's really important, so we really 00:49:04.560 --> 00:49:06.060 raise these alerts a lot. 00:49:06.060 --> 00:49:09.240 But people experience this kind of alert fatigue, or what 00:49:09.240 --> 00:49:10.440 people call alarm fatigue. 00:49:10.440 --> 00:49:11.923 You see this in cockpits, too. 00:49:11.923 --> 00:49:13.590 But people get too many alerts, and they 00:49:13.590 --> 00:49:15.120 start ignoring the alerts. 00:49:15.120 --> 00:49:16.435 They assume that they're wrong. 00:49:16.435 --> 00:49:18.060 They tell the resident not to page them 00:49:18.060 --> 00:49:20.602 in the middle of the night, no matter what the computer says. 00:49:20.602 --> 00:49:23.340 So I do think that we have some responsibility to improve 00:49:23.340 --> 00:49:25.110 the accuracy of these alerts. 00:49:25.110 --> 00:49:26.940 I do think machine learning could help us. 00:49:26.940 --> 00:49:28.440 We're actually just having a meeting 00:49:28.440 --> 00:49:30.740 about a pneumococcal vaccination alert. 00:49:30.740 --> 00:49:32.850 This is something that helps people remember 00:49:32.850 --> 00:49:36.030 to prescribe this vaccination to help you not get pneumonia. 00:49:36.030 --> 00:49:39.537 And it takes four or five variables into account. 00:49:39.537 --> 00:49:41.370 We started looking at the cases where people 00:49:41.370 --> 00:49:42.630 would override the alert. 00:49:42.630 --> 00:49:44.280 And they were mostly appropriate. 00:49:44.280 --> 00:49:47.568 So the patient is in a really extreme state right now. 00:49:47.568 --> 00:49:49.860 Or conversely, the patient is close to the end of life. 00:49:49.860 --> 00:49:52.050 And they're not going to benefit from this vaccination. 00:49:52.050 --> 00:49:53.675 If the patient has a phobia of needles, 00:49:53.675 --> 00:49:55.390 if the patient has an insurance problem. 00:49:55.390 --> 00:49:58.110 And we think there's probably more like 30 or 40 variables 00:49:58.110 --> 00:50:00.540 that you would need to take into account to make 00:50:00.540 --> 00:50:01.800 that really accurate. 00:50:01.800 --> 00:50:04.170 So the question is, when you have that many variables, 00:50:04.170 --> 00:50:08.400 can a human develop and maintain that logic? 00:50:08.400 --> 00:50:11.550 Or would we be better off trying to use a machine learning 00:50:11.550 --> 00:50:12.660 system to do that? 00:50:12.660 --> 00:50:15.353 And would that really work or not? 00:50:15.353 --> 00:50:16.770 PETER SZOLOVITS: So how far are we 00:50:16.770 --> 00:50:20.320 from being able to use a machine learning system to do that? 00:50:20.320 --> 00:50:23.460 ADAM WRIGHT: I think that the biggest challenge, honestly, 00:50:23.460 --> 00:50:25.230 relates to the availability and accuracy 00:50:25.230 --> 00:50:27.280 of the data in our systems. 00:50:27.280 --> 00:50:29.940 So Epic, which is the EHR that we're using-- 00:50:29.940 --> 00:50:32.610 and Cerner and Allscripts and most of the major systems-- 00:50:32.610 --> 00:50:36.480 have various ways to run even sophisticated machine 00:50:36.480 --> 00:50:38.970 learning models, either inside of the system 00:50:38.970 --> 00:50:42.460 or bolted onto the system and then feeding model inferences 00:50:42.460 --> 00:50:44.337 back into the system. 00:50:44.337 --> 00:50:46.420 When I was giving that example of the pneumococcal 00:50:46.420 --> 00:50:48.003 vaccination, one of the major problems 00:50:48.003 --> 00:50:51.100 is that there's not always a really good structured way 00:50:51.100 --> 00:50:53.350 in the system that we indicate that a patient is 00:50:53.350 --> 00:50:55.990 at the end of life and receiving comfort measures only. 00:50:55.990 --> 00:50:58.300 Or that the patient is in a really extreme state, 00:50:58.300 --> 00:51:00.760 that we're in the middle of a code blue 00:51:00.760 --> 00:51:02.470 and that we need to pause for a second 00:51:02.470 --> 00:51:04.720 and stop giving these kind of friendly preventive care 00:51:04.720 --> 00:51:05.350 suggestions. 00:51:05.350 --> 00:51:08.140 So I would actually say that the biggest barrier 00:51:08.140 --> 00:51:10.540 to really good machine-learning-based decision 00:51:10.540 --> 00:51:14.110 support is just the lack of good, reliably documented, 00:51:14.110 --> 00:51:16.720 coded usable features. 00:51:16.720 --> 00:51:19.120 I think that the second challenge, obviously, 00:51:19.120 --> 00:51:20.080 is workflow. 00:51:20.080 --> 00:51:23.530 You said-- it's sometimes hard to know in the hospital who 00:51:23.530 --> 00:51:24.720 a patient's doctor is. 00:51:24.720 --> 00:51:25.720 The patient is admitted. 00:51:25.720 --> 00:51:28.660 And on the care team is an intern, a junior resident, 00:51:28.660 --> 00:51:31.540 and a fellow, an attending, several specialists, 00:51:31.540 --> 00:51:32.347 a couple of nurses. 00:51:32.347 --> 00:51:34.680 Who should get that message or who should get that page? 00:51:34.680 --> 00:51:37.703 I think workflow is second. 00:51:37.703 --> 00:51:39.370 This is where I think you may have said, 00:51:39.370 --> 00:51:40.245 I have some optimism. 00:51:40.245 --> 00:51:42.910 I actually think that the technical ability of our EHR 00:51:42.910 --> 00:51:45.430 software to run these models is better 00:51:45.430 --> 00:51:47.213 than it was three or five years ago. 00:51:47.213 --> 00:51:49.630 And it's, actually, usually not the barrier in the studies 00:51:49.630 --> 00:51:51.130 that we've done. 00:51:51.130 --> 00:51:54.160 PETER SZOLOVITS: So there were attempts-- 00:51:54.160 --> 00:51:56.350 again, 20 years ago-- 00:51:56.350 --> 00:51:59.380 to create formal rules about who gets 00:51:59.380 --> 00:52:01.930 notified under what circumstances. 00:52:01.930 --> 00:52:05.500 I remember one of the doctors I worked with at Tufts Medical 00:52:05.500 --> 00:52:11.020 Center was going crazy, because when they implemented a new lab 00:52:11.020 --> 00:52:16.932 information system, it would alert on every abnormal lab. 00:52:16.932 --> 00:52:19.300 And this was crazy. 00:52:19.300 --> 00:52:22.840 But there were other hospitals that said, well, 00:52:22.840 --> 00:52:25.180 let's be a little more sophisticated about when 00:52:25.180 --> 00:52:27.010 it's necessary to alert. 00:52:27.010 --> 00:52:30.550 And then if somebody doesn't respond to an alert 00:52:30.550 --> 00:52:33.320 within a very short period of time, 00:52:33.320 --> 00:52:36.520 then we escalate it to somebody higher up or somebody else 00:52:36.520 --> 00:52:37.840 on the care team. 00:52:37.840 --> 00:52:40.060 And that seemed like a reasonable idea to me. 00:52:40.060 --> 00:52:42.252 But are there things like that in place now? 00:52:42.252 --> 00:52:43.210 ADAM WRIGHT: There are. 00:52:43.210 --> 00:52:45.700 It works very differently in the inpatient and the outpatient 00:52:45.700 --> 00:52:46.200 setting. 00:52:46.200 --> 00:52:48.850 At the inpatient setting, we're writing very acute care 00:52:48.850 --> 00:52:49.660 to a patient. 00:52:49.660 --> 00:52:52.750 And so we have processes where people sign in and out 00:52:52.750 --> 00:52:55.090 of the care team. 00:52:55.090 --> 00:52:57.640 In fact, these prevalence of these automated messages 00:52:57.640 --> 00:53:00.320 is an incentive to do that well. 00:53:00.320 --> 00:53:03.033 If I go home, I better sign myself out of that patient, 00:53:03.033 --> 00:53:05.200 otherwise I'm going to get all these pages all night 00:53:05.200 --> 00:53:05.810 about them. 00:53:05.810 --> 00:53:07.870 And the system will always make sure 00:53:07.870 --> 00:53:10.840 that somebody is the responding provider. 00:53:10.840 --> 00:53:13.210 It becomes a little thornier in the outpatient setting, 00:53:13.210 --> 00:53:15.820 because a lot of the academic doctors at the Brigham 00:53:15.820 --> 00:53:18.070 only have clinic half a day a week. 00:53:18.070 --> 00:53:20.860 And so the question is, if an abnormal result comes back, 00:53:20.860 --> 00:53:22.780 should I send it to that doctor? 00:53:22.780 --> 00:53:25.870 Should I send it to the person that's on call in that clinic? 00:53:25.870 --> 00:53:27.910 Should I send it to the head of the clinic? 00:53:27.910 --> 00:53:32.050 There are also these edge cases that mess us up a lot. 00:53:32.050 --> 00:53:34.810 So a classic one is a patient is in the hospital. 00:53:34.810 --> 00:53:36.340 I've ordered some lab tests. 00:53:36.340 --> 00:53:38.920 They're looking well, so I discharge the patient. 00:53:38.920 --> 00:53:41.890 The test is still pending at the time the patient is discharged. 00:53:41.890 --> 00:53:43.210 And now, who does that go to? 00:53:43.210 --> 00:53:45.610 Should it go to the patient's primary care doctor? 00:53:45.610 --> 00:53:47.140 Do they have a primary care doctor? 00:53:47.140 --> 00:53:49.182 Should it go to the person that ordered the test? 00:53:49.182 --> 00:53:51.310 That person may be on vacation now, 00:53:51.310 --> 00:53:53.560 if it's a test that takes a few weeks to come back. 00:53:53.560 --> 00:53:55.602 So we still struggle with-- we call those TPADs-- 00:53:55.602 --> 00:53:56.727 tests pending at discharge. 00:53:56.727 --> 00:53:58.730 We still struggle with some of those edge cases. 00:53:58.730 --> 00:54:02.572 But I think in the core, we're pretty good at it. 00:54:02.572 --> 00:54:04.780 PETER SZOLOVITS: So one of the things we talked about 00:54:04.780 --> 00:54:10.360 is an experience I've had and you've probably had that-- 00:54:10.360 --> 00:54:12.160 for example, a few years ago I was 00:54:12.160 --> 00:54:16.120 working with the people who run the clinical labs at Mass 00:54:16.120 --> 00:54:17.860 General. 00:54:17.860 --> 00:54:22.480 And they run some ancient laboratory information systems 00:54:22.480 --> 00:54:25.700 that, as you said, can add and subtract but not multiply 00:54:25.700 --> 00:54:26.200 or divide. 00:54:26.200 --> 00:54:27.680 ADAM WRIGHT: They can add and multiply, but not subtract 00:54:27.680 --> 00:54:28.285 or divide. 00:54:28.285 --> 00:54:28.785 Yes. 00:54:28.785 --> 00:54:30.600 And it doesn't support negative numbers. 00:54:30.600 --> 00:54:33.820 Only unsigned integers. 00:54:33.820 --> 00:54:37.270 PETER SZOLOVITS: So there are these wonderful legacy systems 00:54:37.270 --> 00:54:41.500 around that really create horrendous problems, 00:54:41.500 --> 00:54:44.050 because if you try to build anything-- 00:54:44.050 --> 00:54:47.710 I mean, even a risk prediction calculator-- 00:54:47.710 --> 00:54:54.040 it really helps to be able to divide as well as multiply. 00:54:54.040 --> 00:54:57.820 So we've struggled in that project. 00:54:57.820 --> 00:55:01.930 And I'm sure you've had similar experiences with how do we 00:55:01.930 --> 00:55:07.030 incorporate a decision support system into some 00:55:07.030 --> 00:55:11.890 of this squeaky old technology that just doesn't support it? 00:55:11.890 --> 00:55:13.555 So what's the right approach to that? 00:55:13.555 --> 00:55:15.430 ADAM WRIGHT: There are a lot of architectures 00:55:15.430 --> 00:55:17.240 and they all have pros and cons. 00:55:17.240 --> 00:55:19.490 I'm not sure if any one of them is the right approach. 00:55:19.490 --> 00:55:24.100 I think we often do favor using these creaky old technology 00:55:24.100 --> 00:55:25.590 or the new technology. 00:55:25.590 --> 00:55:28.660 So Epic has a built in rule engine. 00:55:28.660 --> 00:55:32.110 That laboratory you talked about has a basic calculation engine 00:55:32.110 --> 00:55:35.260 with some significant limitations to it. 00:55:35.260 --> 00:55:37.030 So where we can, we often will try 00:55:37.030 --> 00:55:39.460 to build rules internally using these systems. 00:55:39.460 --> 00:55:42.620 Those tend to have real-time availability of data, the best 00:55:42.620 --> 00:55:45.220 ability to sort of push alerts to the person 00:55:45.220 --> 00:55:48.335 right in their workflow and make though those alerts actionable. 00:55:48.335 --> 00:55:50.460 In cases where we can't do that-- like for example, 00:55:50.460 --> 00:55:53.430 a model that's too complex to execute in the system-- 00:55:53.430 --> 00:55:57.170 one thing that we've often done is run that model 00:55:57.170 --> 00:55:58.470 against our data warehouse. 00:55:58.470 --> 00:56:00.178 So we have a data warehouse that extracts 00:56:00.178 --> 00:56:01.928 the data from the electronic health record 00:56:01.928 --> 00:56:02.930 every night at midnight. 00:56:02.930 --> 00:56:07.020 So if we don't need real-time data, it's possible to run-- 00:56:07.020 --> 00:56:09.270 extract the data, run a model, and then actually write 00:56:09.270 --> 00:56:12.890 a risk score or a flag back into the patient's record 00:56:12.890 --> 00:56:14.930 that can then be shown to the clinician, 00:56:14.930 --> 00:56:17.060 or used to drive an alert or something like that. 00:56:17.060 --> 00:56:21.110 That works really well, except that a lot of things that 00:56:21.110 --> 00:56:23.040 happen-- particularly in an inpatient setting, 00:56:23.040 --> 00:56:24.230 like predicting sepsis-- 00:56:24.230 --> 00:56:25.842 depend on real-time data. 00:56:25.842 --> 00:56:27.050 Data that we need right away. 00:56:27.050 --> 00:56:30.500 And so we run into the challenge where that particular approach 00:56:30.500 --> 00:56:36.170 only works on a 24-hour kind of retrospective basis. 00:56:36.170 --> 00:56:40.580 We have also developed systems that depend on messages. 00:56:40.580 --> 00:56:42.830 So there's this-- HL7 is a standard format 00:56:42.830 --> 00:56:45.260 for exchanging data with an electronic health record. 00:56:45.260 --> 00:56:49.160 There's various versions and profiles of HL7. 00:56:49.160 --> 00:56:50.660 But you can set up an infrastructure 00:56:50.660 --> 00:56:53.780 that sits outside of the EHR and gets messages in real time 00:56:53.780 --> 00:56:54.560 from the EHR. 00:56:54.560 --> 00:56:59.000 It makes inferences and sends messages back into the EHR. 00:56:59.000 --> 00:57:01.970 Increasingly, EHRs also do support kind 00:57:01.970 --> 00:57:03.500 of web service approaches. 00:57:03.500 --> 00:57:05.570 So that you can register a hook and say, 00:57:05.570 --> 00:57:07.280 call my hook whenever this thing happens. 00:57:07.280 --> 00:57:09.980 Or you can pull the EHR to get data out and use another web 00:57:09.980 --> 00:57:11.720 service to write data back in. 00:57:11.720 --> 00:57:14.850 That's worked really well for us. 00:57:14.850 --> 00:57:20.030 You can also ask the EHR to embed an app that you develop. 00:57:20.030 --> 00:57:21.800 So people here may have heard-- or should 00:57:21.800 --> 00:57:23.540 hear at some point-- about SMART on FHIR, 00:57:23.540 --> 00:57:27.560 which is a open kind of API that allows you to develop 00:57:27.560 --> 00:57:29.840 an application and embed that application 00:57:29.840 --> 00:57:31.580 into an electronic health record. 00:57:31.580 --> 00:57:35.220 We've increasingly been building some of those applications. 00:57:35.220 --> 00:57:37.110 The downside right now of the smart apps 00:57:37.110 --> 00:57:39.110 is that they're really good for reading data out 00:57:39.110 --> 00:57:41.900 of the record and sort of visualizing or displaying it. 00:57:41.900 --> 00:57:44.150 But they don't always have a lot of capability 00:57:44.150 --> 00:57:47.600 to write data back into the record or take actions. 00:57:47.600 --> 00:57:51.590 Most of the EHR vendors also have a proprietary approach, 00:57:51.590 --> 00:57:52.520 like an app store. 00:57:52.520 --> 00:57:54.270 So Epic calls theirs the App Orchard. 00:57:54.270 --> 00:57:56.490 And most of the EHRs have something similar, 00:57:56.490 --> 00:57:58.670 where you can join a developer program 00:57:58.670 --> 00:58:01.460 and build an application. 00:58:01.460 --> 00:58:04.910 And those are often more full-featured. 00:58:04.910 --> 00:58:06.210 They tend to be proprietary. 00:58:06.210 --> 00:58:08.020 So if you build one Epic app, you 00:58:08.020 --> 00:58:10.228 have to then build a Cerner app and an Allscripts app 00:58:10.228 --> 00:58:12.410 and an eClinicalWorks app separately. 00:58:12.410 --> 00:58:16.760 There are often heavy fees for joining those programs, 00:58:16.760 --> 00:58:18.470 although the EHR vendors-- 00:58:18.470 --> 00:58:21.613 Epic in particular-- have lowered their prices a lot. 00:58:21.613 --> 00:58:23.030 The federal government, the Office 00:58:23.030 --> 00:58:24.767 of the National Coordinator of Health IT, 00:58:24.767 --> 00:58:27.350 just about a week and a half ago released some new regulations 00:58:27.350 --> 00:58:32.600 which really limit the rate at which vendors can charge 00:58:32.600 --> 00:58:36.050 application developers for API access 00:58:36.050 --> 00:58:38.450 basically to almost nothing, except for 00:58:38.450 --> 00:58:42.020 incremental computation costs or special support. 00:58:42.020 --> 00:58:43.893 So I think that may change everything now 00:58:43.893 --> 00:58:45.560 that that regulation's been promulgated. 00:58:45.560 --> 00:58:47.000 So we'll see. 00:58:47.000 --> 00:58:51.020 PETER SZOLOVITS: So contrary to my pessimistic beginning, 00:58:51.020 --> 00:58:54.830 this actually is the thing that makes me most optimistic. 00:58:54.830 --> 00:58:57.320 That even five years ago, if you looked 00:58:57.320 --> 00:59:02.360 at many of these systems, they essentially locked you out. 00:59:02.360 --> 00:59:06.350 I remember in the early 2000s, I was 00:59:06.350 --> 00:59:09.200 at the University of Pittsburgh, where 00:59:09.200 --> 00:59:13.490 they had one of the first centers that was 00:59:13.490 --> 00:59:17.210 doing heart-lung transplants. 00:59:17.210 --> 00:59:22.370 So their people had built a special application 00:59:22.370 --> 00:59:26.420 for supporting heart-lung transplant patients, 00:59:26.420 --> 00:59:30.440 in their own homemade electronic medical records system. 00:59:30.440 --> 00:59:35.580 And then UPMC went to Cerner at the time. 00:59:35.580 --> 00:59:38.110 And I remember I was at some meeting 00:59:38.110 --> 00:59:41.660 where the doctors who ran this heart-lung transplant unit 00:59:41.660 --> 00:59:45.260 were talking to the Cerner people and saying, 00:59:45.260 --> 00:59:49.310 how could we get something to support our special needs 00:59:49.310 --> 00:59:50.960 for our patients? 00:59:50.960 --> 00:59:53.840 And Cerner's answer was, well, commercially it doesn't 00:59:53.840 --> 00:59:55.340 make sense for us to do this. 00:59:55.340 --> 00:59:58.610 Because at the time there were like four hospitals 00:59:58.610 --> 01:00:00.820 in the country that did this. 01:00:00.820 --> 01:00:04.010 And so it's not a big money maker. 01:00:04.010 --> 01:00:07.280 So their offer was, well, you pay us an extra $3 million 01:00:07.280 --> 01:00:12.290 and within three years we will develop 01:00:12.290 --> 01:00:15.040 the appropriate software for you. 01:00:15.040 --> 01:00:17.000 So that's just crazy, right? 01:00:17.000 --> 01:00:20.000 I mean, that's a totally untenable way 01:00:20.000 --> 01:00:21.590 of going about things. 01:00:21.590 --> 01:00:25.040 And now that there are systematic ways for you 01:00:25.040 --> 01:00:29.430 either to embed your own code into one of these systems, 01:00:29.430 --> 01:00:33.200 or at least to have a well-documented, reasonable way 01:00:33.200 --> 01:00:36.440 of feeding data out and then feeding results back 01:00:36.440 --> 01:00:41.130 into the system, that makes it possible to do 01:00:41.130 --> 01:00:44.790 special-purpose applications like this. 01:00:44.790 --> 01:00:48.270 Or experimental applications or all kinds of novel things. 01:00:48.270 --> 01:00:49.420 So that's great. 01:00:49.420 --> 01:00:51.420 ADAM WRIGHT: That's what we're optimistic about. 01:00:51.420 --> 01:00:54.867 And I think it's worth adding that there's two barriers you 01:00:54.867 --> 01:00:55.950 have to get through right. 01:00:55.950 --> 01:00:57.540 One is Epic has to sort of let you 01:00:57.540 --> 01:01:00.163 into their App Orchard, which is the barrier that 01:01:00.163 --> 01:01:01.080 is increasingly lower. 01:01:01.080 --> 01:01:03.288 And then you need to find a hospital or a health care 01:01:03.288 --> 01:01:05.400 provider that wants to use your app, right. 01:01:05.400 --> 01:01:08.190 So you have to clear both of those, 01:01:08.190 --> 01:01:10.320 but I think it's increasingly possible. 01:01:10.320 --> 01:01:11.820 You've got smart people here at MIT, 01:01:11.820 --> 01:01:17.047 or at the hospitals that we have in Boston always wanting 01:01:17.047 --> 01:01:17.880 to build these apps. 01:01:17.880 --> 01:01:21.300 And I would say five years ago we would've told people, sorry, 01:01:21.300 --> 01:01:22.150 it's not possible. 01:01:22.150 --> 01:01:24.025 And today we're able, usually, to tell people 01:01:24.025 --> 01:01:26.190 that if there's clinical interest, 01:01:26.190 --> 01:01:28.180 the technical part will fall into place. 01:01:28.180 --> 01:01:30.165 So that's exciting for us. 01:01:30.165 --> 01:01:31.170 PETER SZOLOVITS: Yeah 01:01:31.170 --> 01:01:31.878 ADAM WRIGHT: Yeah 01:01:31.878 --> 01:01:33.240 AUDIENCE: Question about that. 01:01:33.240 --> 01:01:33.570 ADAM WRIGHT: Absolutely 01:01:33.570 --> 01:01:35.362 AUDIENCE: Some of the applications that you 01:01:35.362 --> 01:01:37.650 guys develop in house, do you also 01:01:37.650 --> 01:01:40.150 put those on the Epic Orchard, or do you 01:01:40.150 --> 01:01:42.827 just sort of implement it one time within your own system? 01:01:42.827 --> 01:01:44.910 ADAM WRIGHT: Yeah, there's a lot of different ways 01:01:44.910 --> 01:01:46.850 that we share these applications, right. 01:01:46.850 --> 01:01:48.270 So a lot of us are researchers. 01:01:48.270 --> 01:01:50.370 So we will release an open source 01:01:50.370 --> 01:01:54.240 version of the application or write a paper and say, 01:01:54.240 --> 01:01:54.990 this is available. 01:01:54.990 --> 01:01:56.340 And we'll share it with you. 01:01:56.340 --> 01:01:59.550 The App Orchard is particularly focused on applications 01:01:59.550 --> 01:02:00.945 that you want to sell. 01:02:00.945 --> 01:02:02.820 So our hospital hasn't decided that we wanted 01:02:02.820 --> 01:02:03.900 to sell any applications. 01:02:03.900 --> 01:02:05.642 We've given a lot of applications away. 01:02:05.642 --> 01:02:08.100 Epic also has something called the Community Library, which 01:02:08.100 --> 01:02:11.465 is like the AppOrchard, but it's free instead of costing money. 01:02:11.465 --> 01:02:12.840 And so we released a ton of stuff 01:02:12.840 --> 01:02:15.440 through the Community Library. 01:02:15.440 --> 01:02:18.420 To the point that I was poking out before, 01:02:18.420 --> 01:02:21.780 one of the challenges is that if we build a Smart on FHIR app, 01:02:21.780 --> 01:02:23.750 we're able to sort of share that publicly. 01:02:23.750 --> 01:02:25.917 And we can post that on the web or put it on GitHub. 01:02:25.917 --> 01:02:27.480 And anybody can use it. 01:02:27.480 --> 01:02:34.710 Epic has a position that their APIs are proprietary. 01:02:34.710 --> 01:02:37.230 And they represent Epic's valuable intellectual property 01:02:37.230 --> 01:02:38.280 or trade secrets. 01:02:38.280 --> 01:02:41.640 And so we're only allowed to share those apps 01:02:41.640 --> 01:02:44.200 through the Epic ecosystem. 01:02:44.200 --> 01:02:46.300 And so, we often now, when we get a grant-- 01:02:46.300 --> 01:02:47.850 most of my work is through grants-- 01:02:47.850 --> 01:02:49.230 we'll have an Epic site. 01:02:49.230 --> 01:02:50.790 And we'll share that through the Community Library. 01:02:50.790 --> 01:02:51.998 And we'll have a Cerner site. 01:02:51.998 --> 01:02:54.030 And we'll share it through Cerner's equivalent. 01:02:54.030 --> 01:02:57.830 But I think until the capability of the open APIs, 01:02:57.830 --> 01:03:00.030 like Smart on FHIR, reaches the same level 01:03:00.030 --> 01:03:02.365 as the proprietary APIs, we're still somewhat locked 01:03:02.365 --> 01:03:03.990 into having to build different versions 01:03:03.990 --> 01:03:06.660 and distribute three-- each EHR under separate channels. 01:03:06.660 --> 01:03:08.645 Really, really good question. 01:03:08.645 --> 01:03:10.800 PETER SZOLOVITS: And so what's lacking 01:03:10.800 --> 01:03:13.080 in things like Smart on FHIR-- 01:03:13.080 --> 01:03:14.055 ADAM WRIGHT: Yeah. 01:03:14.055 --> 01:03:16.513 PETER SZOLOVITS: --that you get from the native interfaces? 01:03:16.513 --> 01:03:19.290 ADAM WRIGHT: So it's very situational, right. 01:03:19.290 --> 01:03:22.680 So, for example, in some EHR implementations, 01:03:22.680 --> 01:03:25.063 the Smart on FHIR will give you a list of the patient's 01:03:25.063 --> 01:03:26.730 current medications but may not give you 01:03:26.730 --> 01:03:28.020 historical medications. 01:03:28.020 --> 01:03:30.240 Or it will tell you that the medicine is ordered, 01:03:30.240 --> 01:03:32.620 but it won't tell you whether it's been administered. 01:03:32.620 --> 01:03:36.880 So one half of the battle is less complete data. 01:03:36.880 --> 01:03:40.920 The other one is that most EHRs are not 01:03:40.920 --> 01:03:42.990 implementing, at this point, the sort 01:03:42.990 --> 01:03:47.220 of write back capabilities, or the actionable capabilities, 01:03:47.220 --> 01:03:49.050 that Smart on FHIR is sort of working on. 01:03:49.050 --> 01:03:50.633 And it's really some standards for us. 01:03:50.633 --> 01:03:52.530 So if we want to build an application that 01:03:52.530 --> 01:03:55.325 shows how a patient fits on a growth curve, that's fine. 01:03:55.325 --> 01:03:56.950 If we went to build an application that 01:03:56.950 --> 01:04:00.040 suggests ordering medicines, that can be really challenging. 01:04:00.040 --> 01:04:03.180 Whereas the internal APIs that the vendors provide typically 01:04:03.180 --> 01:04:04.940 have both read and write capabilities. 01:04:04.940 --> 01:04:05.960 So that's the other challenge. 01:04:05.960 --> 01:04:07.418 PETER SZOLOVITS: And do the vendors 01:04:07.418 --> 01:04:11.117 worry about, I guess two related things, 01:04:11.117 --> 01:04:13.595 one is sort of cognitive overload. 01:04:13.595 --> 01:04:16.770 Because if you build 1,000 Smart on FHIR apps, 01:04:16.770 --> 01:04:20.040 and they all start firing for these inpatients, 01:04:20.040 --> 01:04:22.260 you're going to be back in the same situation 01:04:22.260 --> 01:04:24.270 of over-alerting. 01:04:24.270 --> 01:04:27.520 And the other question is, are they worried about liability? 01:04:27.520 --> 01:04:30.270 Since if you were using their system 01:04:30.270 --> 01:04:35.040 to display recommendations, and those recommendations turn out 01:04:35.040 --> 01:04:37.350 to be wrong and harm some patient, 01:04:37.350 --> 01:04:39.620 then somebody will reach out to them 01:04:39.620 --> 01:04:41.440 legally because they have a lot of money. 01:04:41.440 --> 01:04:42.440 ADAM WRIGHT: Absolutely. 01:04:42.440 --> 01:04:44.065 They're worried about both of those. 01:04:44.065 --> 01:04:45.690 Related particularly to the second one, 01:04:45.690 --> 01:04:48.450 they're also worried about just sort of corruption or integrity 01:04:48.450 --> 01:04:49.500 of the data, right. 01:04:49.500 --> 01:04:52.140 So somehow if I can write a medication order directly 01:04:52.140 --> 01:04:55.920 to the database, and it may bypass certain checks 01:04:55.920 --> 01:04:57.390 that would be done normally. 01:04:57.390 --> 01:05:02.500 And I could potentially enter a wrong or dangerous order. 01:05:02.500 --> 01:05:04.560 The other thing that we're increasingly hearing 01:05:04.560 --> 01:05:08.280 is concerns about protection of data, sort of Cambridge 01:05:08.280 --> 01:05:10.470 Analytica style worries, right. 01:05:10.470 --> 01:05:15.390 So if I, as an Epic patient, authorize the Words 01:05:15.390 --> 01:05:17.910 With Friends app to see my medical record, 01:05:17.910 --> 01:05:20.190 and then they post that on the web, 01:05:20.190 --> 01:05:23.370 or monetize it in some sort of a tricky way, 01:05:23.370 --> 01:05:25.320 what liability, if any, does my health care 01:05:25.320 --> 01:05:27.826 provider organization, or my-- 01:05:27.826 --> 01:05:30.130 the EHR vendor, have for that? 01:05:30.130 --> 01:05:32.340 And the new regulations are extremely strict, right. 01:05:32.340 --> 01:05:36.120 They say that if a patient asks you to, and authorizes an app 01:05:36.120 --> 01:05:40.560 to access their record, you may not block that access, 01:05:40.560 --> 01:05:42.770 even if you consider that app to be a bad actor. 01:05:42.770 --> 01:05:46.760 So that's I think an area of liability that is just 01:05:46.760 --> 01:05:47.930 beginning to be sorted out. 01:05:47.930 --> 01:05:50.857 And it is, I think, some cause for concern. 01:05:50.857 --> 01:05:52.940 But at the same time, you could imagine a universe 01:05:52.940 --> 01:05:55.370 where, I think, there are conservative health 01:05:55.370 --> 01:05:57.980 organizations that would choose to never authorize 01:05:57.980 --> 01:06:01.230 any application to avoid risk. 01:06:01.230 --> 01:06:03.980 So how you balance that is not yet solved. 01:06:03.980 --> 01:06:05.855 PETER SZOLOVITS: Well-- and to avoid leakage. 01:06:05.855 --> 01:06:06.855 ADAM WRIGHT: Absolutely. 01:06:06.855 --> 01:06:09.050 PETER SZOLOVITS: So I remember years ago there 01:06:09.050 --> 01:06:13.160 was a lot of reluctance, even among Boston area hospitals, 01:06:13.160 --> 01:06:15.140 to share data, because they were worried 01:06:15.140 --> 01:06:17.480 that another hospital could cherry 01:06:17.480 --> 01:06:21.950 pick their most lucrative patients by figuring out 01:06:21.950 --> 01:06:23.000 something about them. 01:06:23.000 --> 01:06:26.300 So I'm sure that that hasn't gone away as a concern. 01:06:26.300 --> 01:06:27.550 ADAM WRIGHT: Absolutely, yeah. 01:06:27.550 --> 01:06:30.710 PETER SZOLOVITS: OK, we're going to try to remember to repeat 01:06:30.710 --> 01:06:31.960 the questions you're asking-- 01:06:31.960 --> 01:06:33.290 ADAM WRIGHT: Oh great, OK. 01:06:33.290 --> 01:06:34.680 PETER SZOLOVITS: --because of the recording. 01:06:34.680 --> 01:06:35.597 ADAM WRIGHT: Happy to. 01:06:35.597 --> 01:06:36.575 PETER SZOLOVITS: Yeah. 01:06:36.575 --> 01:06:39.230 AUDIENCE: So how does a third party vendor 01:06:39.230 --> 01:06:42.530 deploy a machine learning model on your system? 01:06:42.530 --> 01:06:44.530 So is that done through Epic? 01:06:44.530 --> 01:06:46.700 Obviously, there's the App Orchard kind of thing, 01:06:46.700 --> 01:06:49.160 but is there ways to go around that and go directly 01:06:49.160 --> 01:06:50.650 into partners and whatnot? 01:06:50.650 --> 01:06:51.150 And how does that work? 01:06:51.150 --> 01:06:51.530 ADAM WRIGHT: Yeah. 01:06:51.530 --> 01:06:53.630 So the question is how does a third party vendor 01:06:53.630 --> 01:06:55.640 deploy an application or a machine 01:06:55.640 --> 01:06:57.290 learning model or something like that? 01:06:57.290 --> 01:07:01.550 And so with Epic, there's always a relationship 01:07:01.550 --> 01:07:05.420 between the vendor of the application and the health care 01:07:05.420 --> 01:07:06.630 provider organization. 01:07:06.630 --> 01:07:09.510 And so we could work together directly. 01:07:09.510 --> 01:07:12.140 So if you had an app that the Brigham wanted to use, 01:07:12.140 --> 01:07:16.160 you could share that app with us in a number of ways. 01:07:16.160 --> 01:07:19.550 So Epic supports this thing called Predictive Modeling 01:07:19.550 --> 01:07:21.230 Markup Language, or PMML. 01:07:21.230 --> 01:07:24.160 So if you train a model, you can export a PMML model. 01:07:24.160 --> 01:07:26.990 And I can import it into Epic and run it natively. 01:07:26.990 --> 01:07:31.320 Or you can produce a web service that I call out to and gives me 01:07:31.320 --> 01:07:31.820 an answer. 01:07:31.820 --> 01:07:34.320 We could work together directly. 01:07:34.320 --> 01:07:36.620 However, there are some limitations 01:07:36.620 --> 01:07:39.770 in what I'm allowed to tell you or share with you about Epic's 01:07:39.770 --> 01:07:41.840 data model and what Epic perceives to be 01:07:41.840 --> 01:07:43.530 their intellectual property. 01:07:43.530 --> 01:07:48.300 And it is facilitated by you joining this program. 01:07:48.300 --> 01:07:49.700 Because if you join this program, 01:07:49.700 --> 01:07:52.200 you get access to documentation that you would otherwise not 01:07:52.200 --> 01:07:53.223 have access to. 01:07:53.223 --> 01:07:55.640 You may get access to a test harness or a test system that 01:07:55.640 --> 01:07:57.930 lets you sort of validate your work. 01:07:57.930 --> 01:07:59.720 However, people who join the program often 01:07:59.720 --> 01:08:01.430 think that means that I can then just 01:08:01.430 --> 01:08:03.050 run my app at every customer, right. 01:08:03.050 --> 01:08:04.820 But with Epic, in particular, you 01:08:04.820 --> 01:08:08.240 have to then make a deal with me to use it at the Brigham 01:08:08.240 --> 01:08:11.750 and make a deal with my colleague to use at Stanford. 01:08:11.750 --> 01:08:14.120 Other EHR vendors have developed a more sort 01:08:14.120 --> 01:08:16.069 of centralized model where you can actually 01:08:16.069 --> 01:08:17.450 release it and sell it, and I can 01:08:17.450 --> 01:08:21.290 pay for it directly through the app store and integrate it. 01:08:21.290 --> 01:08:23.180 I think that last mile piece hasn't really 01:08:23.180 --> 01:08:24.729 been standardized yet. 01:08:24.729 --> 01:08:26.271 AUDIENCE: I guess one of my questions 01:08:26.271 --> 01:08:28.040 there is, what happens in the case 01:08:28.040 --> 01:08:30.020 that I don't want to talk to Epic at all? 01:08:30.020 --> 01:08:32.600 And just I looked at your data and just 01:08:32.600 --> 01:08:34.217 like Brigham and Women's stuff. 01:08:34.217 --> 01:08:35.550 And I build a really good model. 01:08:35.550 --> 01:08:38.060 You saw how it works, and we just want to deploy it. 01:08:38.060 --> 01:08:40.410 ADAM WRIGHT: Epic would not stop us from doing that. 01:08:40.410 --> 01:08:42.260 The only real restriction is that Epic 01:08:42.260 --> 01:08:45.710 would limit my ability to tell you stuff about Epic's guts. 01:08:45.710 --> 01:08:48.490 And so you would need a relatively sophisticated health 01:08:48.490 --> 01:08:51.240 care provider organization who could 01:08:51.240 --> 01:08:54.229 map between some kind of platonic data, clinical data, 01:08:54.229 --> 01:08:56.609 model and Epic's internal data model. 01:08:56.609 --> 01:08:58.560 But if you had that, you could. 01:08:58.560 --> 01:09:01.670 And at the Brigham, we have this iHub Innovation Program. 01:09:01.670 --> 01:09:04.880 And we're probably working with 50 to 100 startups 01:09:04.880 --> 01:09:06.710 doing work like that, some of whom 01:09:06.710 --> 01:09:08.600 are members of the Epic App Orchard 01:09:08.600 --> 01:09:10.905 and some who choose not to be members of the Epic App 01:09:10.905 --> 01:09:11.210 Orchard. 01:09:11.210 --> 01:09:12.793 It's worth saying that joining the App 01:09:12.793 --> 01:09:14.870 Orchard or these programs entails revenue sharing 01:09:14.870 --> 01:09:16.609 with Epic and some complexity. 01:09:16.609 --> 01:09:18.745 That may go way down with these new regulations. 01:09:18.745 --> 01:09:20.120 But right now, some organizations 01:09:20.120 --> 01:09:22.037 have chosen not to partner with the vendors 01:09:22.037 --> 01:09:23.620 and work directly with the health care 01:09:23.620 --> 01:09:24.578 provider organizations. 01:09:24.578 --> 01:09:27.470 PETER SZOLOVITS: So on the quality side of that question, 01:09:27.470 --> 01:09:31.970 if you do develop an application and field it at the Brigham, 01:09:31.970 --> 01:09:35.330 will Stanford be interested in taking it? 01:09:35.330 --> 01:09:37.580 Or are they going to be concerned about the fact 01:09:37.580 --> 01:09:40.910 that somehow you've fit it to the patient population 01:09:40.910 --> 01:09:43.575 in Boston, and it won't be appropriate to their data? 01:09:43.575 --> 01:09:45.950 ADAM WRIGHT: Yeah, I think that's a fundamental question, 01:09:45.950 --> 01:09:48.859 right, is to what extent do these models generalize, right? 01:09:48.859 --> 01:09:50.990 Can you train a model at one place 01:09:50.990 --> 01:09:52.700 and transfer it to another place? 01:09:52.700 --> 01:09:54.950 We've generally seen that many of them 01:09:54.950 --> 01:09:56.240 transfer pretty well, right. 01:09:56.240 --> 01:09:58.370 So if they really have more to do 01:09:58.370 --> 01:10:00.080 with kind of core human physiology, 01:10:00.080 --> 01:10:02.420 that can be pretty similar between organizations. 01:10:02.420 --> 01:10:04.700 If they're really bound up in a particular workflow, 01:10:04.700 --> 01:10:07.550 right, they assume that you're doing this task, this task, 01:10:07.550 --> 01:10:10.360 this task in this order, they tend to transfer really, really 01:10:10.360 --> 01:10:10.950 poorly. 01:10:10.950 --> 01:10:13.250 So I would say that our general approach has 01:10:13.250 --> 01:10:15.230 been to take a model that somebody has, 01:10:15.230 --> 01:10:17.158 run it retrospectively on our data warehouse, 01:10:17.158 --> 01:10:18.200 and see if it's accurate. 01:10:18.200 --> 01:10:19.950 And if it is, we might go forward with it. 01:10:19.950 --> 01:10:22.340 If it's not, we would try to retrain it on our data, 01:10:22.340 --> 01:10:23.840 and then see how much improvement we 01:10:23.840 --> 01:10:24.963 get by retraining it. 01:10:24.963 --> 01:10:26.630 PETER SZOLOVITS: And so have you in fact 01:10:26.630 --> 01:10:28.745 imported such models from other places? 01:10:28.745 --> 01:10:29.870 ADAM WRIGHT: We have, yeah. 01:10:29.870 --> 01:10:32.140 Epic provides five or six models. 01:10:32.140 --> 01:10:35.300 And we've just started using some of them at the Brigham 01:10:35.300 --> 01:10:37.800 or just kind of signed the license to begin using them. 01:10:37.800 --> 01:10:40.920 And I think Epic's guidance and our experience 01:10:40.920 --> 01:10:44.097 is that they work pretty well out of the box. 01:10:44.097 --> 01:10:45.590 PETER SZOLOVITS: Great. 01:10:45.590 --> 01:10:47.423 AUDIENCE: So could you say a little bit more 01:10:47.423 --> 01:10:49.660 about these rescores that are being deployed, 01:10:49.660 --> 01:10:50.390 maybe they work. 01:10:50.390 --> 01:10:51.345 Maybe they don't. 01:10:51.345 --> 01:10:54.400 How can you really tell whether they're working, even 01:10:54.400 --> 01:10:57.170 just beyond patient shift over time, 01:10:57.170 --> 01:10:58.902 just like how people react to the scores. 01:10:58.902 --> 01:11:00.860 Like I know a lot of the bias in fairness works 01:11:00.860 --> 01:11:03.625 is like people, if a score agrees with their intuition, 01:11:03.625 --> 01:11:04.590 they'll trust it. 01:11:04.590 --> 01:11:06.560 And if it doesn't, they ignore the score. 01:11:06.560 --> 01:11:08.540 So like how-- what does the process look 01:11:08.540 --> 01:11:11.022 like before you deploy the score thing 01:11:11.022 --> 01:11:12.730 and then see whether it's working or not? 01:11:12.730 --> 01:11:13.260 ADAM WRIGHT: Yeah, absolutely. 01:11:13.260 --> 01:11:14.927 So the question is, we get a risk score, 01:11:14.927 --> 01:11:16.660 or we deploy a new risk score that says, 01:11:16.660 --> 01:11:18.368 patient has a risk of falling, or patient 01:11:18.368 --> 01:11:20.830 has a risk of having sepsis or something like that. 01:11:20.830 --> 01:11:22.960 We tend to do several levels of evaluation, right. 01:11:22.960 --> 01:11:25.085 So the first level is, when we show the score, what 01:11:25.085 --> 01:11:25.990 do people do, right? 01:11:25.990 --> 01:11:27.940 If we-- typically we don't just show a score, 01:11:27.940 --> 01:11:28.982 we make a recommendation. 01:11:28.982 --> 01:11:30.540 We say, based on the score we think 01:11:30.540 --> 01:11:32.665 you should order a lactate to see if the patient is 01:11:32.665 --> 01:11:33.772 at risk of having sepsis. 01:11:33.772 --> 01:11:35.980 First we look to see if people do what we say, right. 01:11:35.980 --> 01:11:38.890 So we think it's a good sign if people follow the suggestions. 01:11:38.890 --> 01:11:40.678 But ultimately, we view ourselves 01:11:40.678 --> 01:11:42.220 as sort of clinical trialists, right. 01:11:42.220 --> 01:11:45.340 So we deploy this model with an intent to move something, 01:11:45.340 --> 01:11:48.440 to reduce the rate of sepsis, or to reduce the rate of mortality 01:11:48.440 --> 01:11:48.940 in sepsis. 01:11:48.940 --> 01:11:51.652 And so we would try to sort of measure, if nothing else, 01:11:51.652 --> 01:11:53.110 do a before and after study, right, 01:11:53.110 --> 01:11:55.510 measure the rates before, implement this intervention, 01:11:55.510 --> 01:11:57.580 and measure the rates after. 01:11:57.580 --> 01:11:59.823 In cases where we're less sure, or where we really 01:11:59.823 --> 01:12:01.240 care about the results, we'll even 01:12:01.240 --> 01:12:02.448 do a randomized trial, right. 01:12:02.448 --> 01:12:04.912 So we'll give half of the units will get the alert, 01:12:04.912 --> 01:12:06.370 half the units won't get the alert. 01:12:06.370 --> 01:12:08.890 And we'll compare the effect on a clinical outcome 01:12:08.890 --> 01:12:10.310 and see what the difference is. 01:12:10.310 --> 01:12:12.910 In our opinion, unless we can show an effect 01:12:12.910 --> 01:12:16.180 on these clinical measures, we shouldn't 01:12:16.180 --> 01:12:17.360 be bothering people, right. 01:12:17.360 --> 01:12:19.600 Pete made this point that what's the purpose 01:12:19.600 --> 01:12:21.152 of having-- if we have 1,000 alerts, 01:12:21.152 --> 01:12:22.360 everyone will be overwhelmed. 01:12:22.360 --> 01:12:23.700 So we should only keep alerts on if we 01:12:23.700 --> 01:12:25.090 can show that they're making a real clinical difference. 01:12:25.090 --> 01:12:27.520 AUDIENCE: And are those sort of like just internal checks, 01:12:27.520 --> 01:12:30.160 are there papers of some of these deployments? 01:12:30.160 --> 01:12:31.180 ADAM WRIGHT: It's our-- 01:12:31.180 --> 01:12:32.860 it's our intent to publish everything, right. 01:12:32.860 --> 01:12:34.200 I mean, I think we're behind. 01:12:34.200 --> 01:12:35.915 But I'd say, we publish everything. 01:12:35.915 --> 01:12:37.540 We have some things that we've finished 01:12:37.540 --> 01:12:38.790 that we haven't published yet. 01:12:38.790 --> 01:12:40.950 They're sort of the next thing to sort of come out. 01:12:40.950 --> 01:12:43.290 Yeah. 01:12:43.290 --> 01:12:45.470 AUDIENCE: I guess so earlier we were talking 01:12:45.470 --> 01:12:50.040 about how the models are just used to give recommendations 01:12:50.040 --> 01:12:51.620 to doctors. 01:12:51.620 --> 01:12:55.340 Do you have any metric, in terms of how often the model 01:12:55.340 --> 01:12:58.770 recommendation matches with the doctor's decision? 01:12:58.770 --> 01:13:00.020 ADAM WRIGHT: Yeah, absolutely. 01:13:00.020 --> 01:13:00.530 AUDIENCE: Can you repeat the question? 01:13:00.530 --> 01:13:01.150 ADAM WRIGHT: Oh yeah. 01:13:01.150 --> 01:13:01.730 Thanks, David. 01:13:01.730 --> 01:13:03.050 So the question is, do we ever check 01:13:03.050 --> 01:13:05.092 to see how often the model recommendation matches 01:13:05.092 --> 01:13:06.448 what the doctor does? 01:13:06.448 --> 01:13:08.240 And so there's sort of two ways we do that. 01:13:08.240 --> 01:13:11.240 We'll often retrospectively test the model back. 01:13:11.240 --> 01:13:12.920 I think Pete shared a paper from Cerner 01:13:12.920 --> 01:13:14.840 where they looked at these sort of suggestions 01:13:14.840 --> 01:13:16.310 that they made to order lactates or to do 01:13:16.310 --> 01:13:17.393 other sort of sepsis work. 01:13:17.393 --> 01:13:20.120 And they looked to see whether the recommendations that they 01:13:20.120 --> 01:13:22.790 made matched what the doctors had actually done. 01:13:22.790 --> 01:13:24.717 And they showed that they, in many cases, did. 01:13:24.717 --> 01:13:26.550 So that'll be the first thing that we do is, 01:13:26.550 --> 01:13:29.120 before we even turn the model on, we'll run it in silent mode 01:13:29.120 --> 01:13:31.115 and see if the doctor does what we suggest. 01:13:31.115 --> 01:13:33.240 Now the doctor is not a perfect supervision, right, 01:13:33.240 --> 01:13:35.368 because the doctor may neglect to do something 01:13:35.368 --> 01:13:36.410 that would be good to do. 01:13:36.410 --> 01:13:38.120 So then when we turn it on, we actually 01:13:38.120 --> 01:13:39.620 look to see whether the doctor takes 01:13:39.620 --> 01:13:41.883 the action that we suggested. 01:13:41.883 --> 01:13:43.800 And if we're doing it in this randomized mode, 01:13:43.800 --> 01:13:45.950 we would then look to see whether the doctor takes 01:13:45.950 --> 01:13:49.280 the action we suggested more often in the case where we show 01:13:49.280 --> 01:13:52.280 the alert, than where we generate the alert but just 01:13:52.280 --> 01:13:54.634 logged it and don't-- don't show it. 01:13:54.634 --> 01:13:55.134 Yeah. 01:13:58.960 --> 01:13:59.750 Yes, sir? 01:13:59.750 --> 01:14:05.020 AUDIENCE: So you'd mentioned how there's 01:14:05.020 --> 01:14:06.505 kind of related to fatigue-- 01:14:06.505 --> 01:14:06.800 ADAM WRIGHT: Yeah. 01:14:06.800 --> 01:14:08.130 AUDIENCE: --if it's a code blue, these alarms will-- 01:14:08.130 --> 01:14:09.080 ADAM WRIGHT: Right. 01:14:09.080 --> 01:14:10.930 AUDIENCE: And you said that cockpits have-- 01:14:10.930 --> 01:14:11.180 pilots now-- 01:14:11.180 --> 01:14:11.610 ADAM WRIGHT: Yeah. 01:14:11.610 --> 01:14:13.235 AUDIENCE: --that have similar problems. 01:14:13.235 --> 01:14:15.690 My very limited understanding of aviation 01:14:15.690 --> 01:14:18.120 is that if you're flying, say, below 10,000 feet, 01:14:18.120 --> 01:14:19.120 then almost all of the-- 01:14:19.120 --> 01:14:19.430 ADAM WRIGHT: Yeah. 01:14:19.430 --> 01:14:20.890 AUDIENCE: --alarms get turned off, and-- 01:14:20.890 --> 01:14:21.190 ADAM WRIGHT: Yeah. 01:14:21.190 --> 01:14:22.630 AUDIENCE: --I don't know if there seems to be an airlock 01:14:22.630 --> 01:14:23.430 for that, for-- 01:14:23.430 --> 01:14:23.740 ADAM WRIGHT: Yeah. 01:14:23.740 --> 01:14:24.040 AUDIENCE: --hospitals yet. 01:14:24.040 --> 01:14:26.340 And is that just because the technology workflow 01:14:26.340 --> 01:14:28.395 is not mature enough yet, only 10 years old? 01:14:28.395 --> 01:14:29.145 ADAM WRIGHT: Yeah. 01:14:29.145 --> 01:14:31.145 AUDIENCE: Or is that kind of the team's question 01:14:31.145 --> 01:14:34.960 about the incentives between if you build the tool 01:14:34.960 --> 01:14:36.610 and it doesn't flag this thing-- 01:14:36.610 --> 01:14:37.000 ADAM WRIGHT: Yeah. 01:14:37.000 --> 01:14:38.440 AUDIENCE: --the patient dies, then they could sued. 01:14:38.440 --> 01:14:39.523 And so they're just very-- 01:14:39.523 --> 01:14:41.290 ADAM WRIGHT: Yeah, no, we try, right? 01:14:41.290 --> 01:14:43.660 So since we often don't know about the situations 01:14:43.660 --> 01:14:45.460 in a structured way at the EHR. 01:14:45.460 --> 01:14:48.440 And so most of our alerts are suppressed in the operating 01:14:48.440 --> 01:14:48.940 room, right? 01:14:48.940 --> 01:14:52.270 So during an-- when a patient is on anesthesia, 01:14:52.270 --> 01:14:54.280 their physiology is being sort of manually 01:14:54.280 --> 01:14:56.040 controlled by a doctor. 01:14:56.040 --> 01:14:59.327 And so we often suppress the alerts in those situations. 01:14:59.327 --> 01:15:01.660 I guess I didn't say the question, but the question was, 01:15:01.660 --> 01:15:03.610 do we try to take situations into account 01:15:03.610 --> 01:15:05.010 or how much can we? 01:15:05.010 --> 01:15:07.218 We didn't used to know that a code blue was going on, 01:15:07.218 --> 01:15:09.593 because we used to do most of our code blue documentation 01:15:09.593 --> 01:15:10.210 on paper. 01:15:10.210 --> 01:15:11.810 We now use this code narrator, right? 01:15:11.810 --> 01:15:13.150 So we can tell when a code blue starts 01:15:13.150 --> 01:15:14.260 and when a code blue ends. 01:15:14.260 --> 01:15:17.280 A code blue is a cardiac arrest and resuscitation of a patient. 01:15:17.280 --> 01:15:18.970 And so we actually do increasingly 01:15:18.970 --> 01:15:22.030 turn a lot of alerting off during a code blue. 01:15:22.030 --> 01:15:24.310 I get an email or a page whenever 01:15:24.310 --> 01:15:27.520 a doctor overrides an alert and writes a cranky message. 01:15:27.520 --> 01:15:30.610 And they'll often say something like, this patient is dying 01:15:30.610 --> 01:15:32.668 of a myocardial infarction right now, 01:15:32.668 --> 01:15:34.960 and your bothering me about this influenza vaccination. 01:15:34.960 --> 01:15:36.585 And then what I'll do is I'll go back-- 01:15:36.585 --> 01:15:38.260 no, seriously, I had that yesterday. 01:15:38.260 --> 01:15:40.677 And so what I'll do is I'll go back and look in the record 01:15:40.677 --> 01:15:42.940 and say, what signs did I have this patient sort 01:15:42.940 --> 01:15:43.780 of in extremis? 01:15:43.780 --> 01:15:45.970 And in that particular case, it was 01:15:45.970 --> 01:15:48.820 a patient who came into the ED and very little documentation 01:15:48.820 --> 01:15:50.445 had been started, and so there actually 01:15:50.445 --> 01:15:53.020 were very few signs that the patient was in the acute state. 01:15:53.020 --> 01:15:55.380 I think this, someday, could be sorted 01:15:55.380 --> 01:15:58.630 by integrating monitor data and device data to figure that out. 01:15:58.630 --> 01:16:01.053 But at that point, we didn't have a good, structured data 01:16:01.053 --> 01:16:02.470 at that moment, in the chart, that 01:16:02.470 --> 01:16:05.050 said this patient is so ill that it's 01:16:05.050 --> 01:16:07.300 offensive to suggest an influenza vaccination right 01:16:07.300 --> 01:16:08.382 now. 01:16:08.382 --> 01:16:10.090 PETER SZOLOVITS: Now, there are hospitals 01:16:10.090 --> 01:16:13.930 that have started experimenting with things like acquiring data 01:16:13.930 --> 01:16:16.930 from the ambulance as the patient is coming in 01:16:16.930 --> 01:16:20.230 so that the ED is already primed with preliminary data. 01:16:20.230 --> 01:16:20.980 ADAM WRIGHT: Yeah. 01:16:20.980 --> 01:16:23.530 PETER SZOLOVITS: And in that circumstance, you could tell. 01:16:23.530 --> 01:16:25.330 ADAM WRIGHT: So this is the interoperability challenge, 01:16:25.330 --> 01:16:25.830 right? 01:16:25.830 --> 01:16:27.985 So we actually get the run sheet, all 01:16:27.985 --> 01:16:29.860 of the ambulance data, to us. 01:16:29.860 --> 01:16:34.630 It comes in as a PDF that's transmitted from the ambulance 01:16:34.630 --> 01:16:37.510 emergency management system to our EHR. 01:16:37.510 --> 01:16:40.780 And so it's not coming in in a way that we can read it well. 01:16:40.780 --> 01:16:43.750 But to your point, exactly, if we were better 01:16:43.750 --> 01:16:44.840 at interoperability-- 01:16:44.840 --> 01:16:47.350 I've also talked to hospitals who use things 01:16:47.350 --> 01:16:50.440 like video cameras and people's badges, 01:16:50.440 --> 01:16:52.630 and if there's 50 people hovering around a patient, 01:16:52.630 --> 01:16:54.547 that's a sign that something bad is happening. 01:16:54.547 --> 01:16:58.240 And so we might be able to use something like that. 01:16:58.240 --> 01:17:02.000 But yeah, we'd like to be better at that. 01:17:02.000 --> 01:17:05.340 PETER SZOLOVITS: So why did HL7 version 3 not solve 01:17:05.340 --> 01:17:06.795 all of these problems? 01:17:06.795 --> 01:17:08.920 ADAM WRIGHT: This is a good philosophical question. 01:17:08.920 --> 01:17:13.240 Come to BMI 701 and 702 and we'll talk about the standards. 01:17:13.240 --> 01:17:15.740 HL7 version-- to his question-- version 2 01:17:15.740 --> 01:17:16.990 was a very practical standard. 01:17:16.990 --> 01:17:19.490 Version 3 was a very deeply philosophical standard-- 01:17:19.490 --> 01:17:20.740 PETER SZOLOVITS: Aspirational. 01:17:20.740 --> 01:17:24.310 ADAM WRIGHT: --aspirational, that never quite caught on. 01:17:24.310 --> 01:17:25.340 And it did in pieces. 01:17:25.340 --> 01:17:29.030 I mean, FHIR is a simplification of that. 01:17:29.030 --> 01:17:29.947 PETER SZOLOVITS: Yeah. 01:17:29.947 --> 01:17:30.863 ADAM WRIGHT: Yes, sir? 01:17:30.863 --> 01:17:33.430 AUDIENCE: So I think usually, the machine learning models 01:17:33.430 --> 01:17:35.050 evaluates the difficult [INAUDIBLE].. 01:17:35.050 --> 01:17:36.170 ADAM WRIGHT: Yes, sir. 01:17:36.170 --> 01:17:38.170 AUDIENCE: When it comes to a particular patient, 01:17:38.170 --> 01:17:40.930 is there a way to know how reliable the model is? 01:17:40.930 --> 01:17:43.220 ADAM WRIGHT: Yeah, I mean, there's calibration, right? 01:17:43.220 --> 01:17:44.690 So we can say this model works particularly 01:17:44.690 --> 01:17:47.150 well in these patients, or not as well in these patients. 01:17:47.150 --> 01:17:48.712 There are some very simple equations 01:17:48.712 --> 01:17:50.420 or models that we use, for example, where 01:17:50.420 --> 01:17:53.480 we use a different model in African-American patients 01:17:53.480 --> 01:17:55.617 versus non-African-American patients, 01:17:55.617 --> 01:17:57.950 because there's some data that says this model is better 01:17:57.950 --> 01:18:01.230 calibrated in this subgroup of patients versus another. 01:18:01.230 --> 01:18:02.900 I do think, though, to your point, 01:18:02.900 --> 01:18:06.020 that there's a suggestion, an inference 01:18:06.020 --> 01:18:08.570 from a model-- this patient is at risk of a fall. 01:18:08.570 --> 01:18:14.550 And then there's this whole set of value judgments and beliefs 01:18:14.550 --> 01:18:18.310 and knowledge and understanding of a patient's circumstances 01:18:18.310 --> 01:18:19.460 that are very human. 01:18:19.460 --> 01:18:21.290 And I think that that's largely why 01:18:21.290 --> 01:18:24.380 we deliver these suggestions to a doctor or to a nurse. 01:18:24.380 --> 01:18:28.010 And then that human uses that information 01:18:28.010 --> 01:18:30.740 plus their expertise and their relationship 01:18:30.740 --> 01:18:35.300 and their experience to make a suggestion, rather than just 01:18:35.300 --> 01:18:38.978 having the computer adjust the knob on the ventilator itself. 01:18:38.978 --> 01:18:40.520 A question that people always ask me, 01:18:40.520 --> 01:18:42.603 and that you should ask me, is, will we eventually 01:18:42.603 --> 01:18:43.780 not need that human? 01:18:43.780 --> 01:18:47.000 And I think I'm more optimistic than some people 01:18:47.000 --> 01:18:49.550 that there are cases where the computer is good enough, 01:18:49.550 --> 01:18:51.380 or the human is poor enough, that it 01:18:51.380 --> 01:18:55.260 would be safe to have a close to closed loop. 01:18:55.260 --> 01:18:57.712 However, I think those cases are not the norm. 01:18:57.712 --> 01:19:00.170 I think that there'll be more cases where human doctors are 01:19:00.170 --> 01:19:02.290 still very much needed. 01:19:02.290 --> 01:19:04.040 PETER SZOLOVITS: So just to add that there 01:19:04.040 --> 01:19:09.740 are tasks where patients are fungible, in the words 01:19:09.740 --> 01:19:12.980 that I used a few lectures ago. 01:19:12.980 --> 01:19:14.990 So for example, a lot of hospitals 01:19:14.990 --> 01:19:19.850 are developing models that predict whether a patient will 01:19:19.850 --> 01:19:24.770 show up for their optional surgery, because then they 01:19:24.770 --> 01:19:27.680 can do a better job of over-scheduling the operating 01:19:27.680 --> 01:19:32.600 room in the same way that the airlines over over-sell seats. 01:19:32.600 --> 01:19:36.260 Because, statistically, you could win doing that. 01:19:36.260 --> 01:19:39.380 Those are very safe predictions, because the worst thing that 01:19:39.380 --> 01:19:41.480 happens is you get delayed. 01:19:41.480 --> 01:19:43.340 But it's not going to have a harmful outcome 01:19:43.340 --> 01:19:45.406 on an individual patient. 01:19:45.406 --> 01:19:47.240 ADAM WRIGHT: Yeah, and conversely, there 01:19:47.240 --> 01:19:49.532 are people that are working on machine learning systems 01:19:49.532 --> 01:19:52.940 for dosing insulin or adjusting people's ventilator settings, 01:19:52.940 --> 01:19:54.762 and those are high-- 01:19:54.762 --> 01:19:56.470 PETER SZOLOVITS: Those are the high risk. 01:19:56.470 --> 01:19:57.330 ADAM WRIGHT: --risk jobs. 01:19:57.330 --> 01:19:58.350 PETER SZOLOVITS: Yep. 01:19:58.350 --> 01:20:01.290 All right, last question because we have to wrap up. 01:20:01.290 --> 01:20:04.340 AUDIENCE: You had alluded to some of the [INAUDIBLE] 01:20:04.340 --> 01:20:04.872 problems-- 01:20:04.872 --> 01:20:05.580 ADAM WRIGHT: Yes. 01:20:05.580 --> 01:20:07.080 AUDIENCE: --of some of these models. 01:20:07.080 --> 01:20:12.931 I'm, one, curious how long [INAUDIBLE].. 01:20:12.931 --> 01:20:13.681 ADAM WRIGHT: Yeah. 01:20:13.681 --> 01:20:16.545 AUDIENCE: And I guess, two, once it's 01:20:16.545 --> 01:20:19.819 been determined that actually a significant issue has occurred, 01:20:19.819 --> 01:20:22.110 what are some of the decisions that you made regarding 01:20:22.110 --> 01:20:24.290 tradeoffs of using the out-of-date model that 01:20:24.290 --> 01:20:27.160 looks at [INAUDIBLE] signal versus the cost of retraining? 01:20:27.160 --> 01:20:28.160 ADAM WRIGHT: Retraining? 01:20:28.160 --> 01:20:29.172 Yeah. 01:20:29.172 --> 01:20:29.880 Yeah, absolutely. 01:20:29.880 --> 01:20:32.208 So the question is the set-and-forget, right? 01:20:32.208 --> 01:20:33.000 We build the model. 01:20:33.000 --> 01:20:34.620 The model may become stale. 01:20:34.620 --> 01:20:35.880 Should we update the model? 01:20:35.880 --> 01:20:37.440 And how do we decide to do that? 01:20:37.440 --> 01:20:38.432 I mean, we're using-- 01:20:38.432 --> 01:20:40.140 it depends on what you define as a model. 01:20:40.140 --> 01:20:42.440 We're using tables and rules that we've 01:20:42.440 --> 01:20:45.090 developed since the 1970s. 01:20:45.090 --> 01:20:48.720 I think we have a pretty high desire 01:20:48.720 --> 01:20:50.225 to empirically revisit those. 01:20:50.225 --> 01:20:51.600 There's a problem in the practice 01:20:51.600 --> 01:20:54.100 called knowledge management or knowledge engineering, right? 01:20:54.100 --> 01:20:56.070 How do we remember which of our knowledge bases 01:20:56.070 --> 01:20:58.170 need to be checked again or updated? 01:20:58.170 --> 01:21:02.300 And we'll often, just as a standard, retrain a model 01:21:02.300 --> 01:21:05.460 or re-evaluate a knowledge base every six months or every year 01:21:05.460 --> 01:21:08.810 because it's both harmful to patients 01:21:08.810 --> 01:21:10.713 if this stuff is out-of-date, and it also 01:21:10.713 --> 01:21:11.880 makes us look stupid, right? 01:21:11.880 --> 01:21:13.963 So if there's a new paper that comes out and says, 01:21:13.963 --> 01:21:15.670 beta blockers are terrible poison, 01:21:15.670 --> 01:21:17.970 and we keep suggesting them, then people no longer 01:21:17.970 --> 01:21:21.990 believe the suggestions that we make, that said, 01:21:21.990 --> 01:21:23.257 we still make mistakes, right? 01:21:23.257 --> 01:21:24.840 I mean, things happen all of the time. 01:21:24.840 --> 01:21:27.670 A lot of my work has focused on malfunctions in these systems. 01:21:27.670 --> 01:21:31.170 And so, as an example, empirically, the pharmacy 01:21:31.170 --> 01:21:33.870 might change the code or ID number for a medicine, 01:21:33.870 --> 01:21:36.120 or a new medicine might come on the market, 01:21:36.120 --> 01:21:38.700 and we have to make sure to continually update 01:21:38.700 --> 01:21:40.650 the knowledge base so that we're not suggesting an old medicine 01:21:40.650 --> 01:21:42.817 or overlooking the fact that the patient has already 01:21:42.817 --> 01:21:44.490 been prescribed a new medicine. 01:21:44.490 --> 01:21:47.100 And so we tried to do that prospectively or proactively. 01:21:47.100 --> 01:21:49.770 But then we also tried to listen to feedback from users 01:21:49.770 --> 01:21:51.255 and fix things as we go. 01:21:51.255 --> 01:21:52.410 Cool. 01:21:52.410 --> 01:21:55.560 PETER SZOLOVITS: And just one more comment on that. 01:21:55.560 --> 01:21:58.300 So some things are done in real time. 01:21:58.300 --> 01:21:59.850 There was a system, many years ago, 01:21:59.850 --> 01:22:05.130 at the Intermountain Health in Salt Lake City, where 01:22:05.130 --> 01:22:08.610 they were looking at what bugs were growing out 01:22:08.610 --> 01:22:12.150 of microbiology samples in the laboratory. 01:22:12.150 --> 01:22:14.940 And of course, that can change on an hour-by-hour or 01:22:14.940 --> 01:22:16.520 day-to-day basis. 01:22:16.520 --> 01:22:19.440 And so they were updating those systems that warned you 01:22:19.440 --> 01:22:23.880 about the possibility of that kind of infection in real time 01:22:23.880 --> 01:22:26.358 by taking feeds directly from the laboratory. 01:22:26.358 --> 01:22:27.400 ADAM WRIGHT: That's true. 01:22:27.400 --> 01:22:28.680 PETER SZOLOVITS: All right, thank you very much. 01:22:28.680 --> 01:22:30.055 ADAM WRIGHT: No, thank you, guys. 01:22:30.055 --> 01:22:32.990 [APPLAUSE]