WEBVTT 00:00:00.000 --> 00:00:02.190 align:middle line:84% STEVEN A. GREENBERG: And did you go 00:00:02.190 --> 00:00:08.100 align:middle line:84% through the methods used to classify disease in that or-- 00:00:08.100 --> 00:00:10.410 align:middle line:90% here and there, OK. 00:00:10.410 --> 00:00:15.240 align:middle line:84% Well, that's what I'm focused on in the next two blocks. 00:00:15.240 --> 00:00:18.210 align:middle line:84% And I guess, this is a block of four lectures that are 00:00:18.210 --> 00:00:19.860 align:middle line:90% going to focus on this area. 00:00:19.860 --> 00:00:22.260 align:middle line:84% And this is the Use of Microarrays 00:00:22.260 --> 00:00:24.650 align:middle line:90% for Disease Classification. 00:00:24.650 --> 00:00:25.380 align:middle line:90% Oh, thank you. 00:00:25.380 --> 00:00:29.340 align:middle line:90% 00:00:29.340 --> 00:00:32.780 align:middle line:84% And I think it's important to keep it straight in your mind 00:00:32.780 --> 00:00:35.540 align:middle line:84% when looking at papers that use microarray technology, 00:00:35.540 --> 00:00:39.740 align:middle line:84% or in your own use of it, which of these two 00:00:39.740 --> 00:00:41.990 align:middle line:84% very distinct applications you're 00:00:41.990 --> 00:00:45.410 align:middle line:84% using it for because there is some overlap between the two. 00:00:45.410 --> 00:00:47.870 align:middle line:84% But they are fundamentally distinct uses 00:00:47.870 --> 00:00:49.370 align:middle line:90% of the technology. 00:00:49.370 --> 00:00:52.370 align:middle line:84% And the two uses are for the understanding of disease 00:00:52.370 --> 00:00:53.480 align:middle line:90% pathophysiology. 00:00:53.480 --> 00:00:55.580 align:middle line:84% And that's fairly straightforward 00:00:55.580 --> 00:00:57.080 align:middle line:90% in a certain way. 00:00:57.080 --> 00:01:00.890 align:middle line:84% You look at expression of genes, and a tissue of interest, 00:01:00.890 --> 00:01:03.290 align:middle line:84% and a disease of interest, and try and understand 00:01:03.290 --> 00:01:06.840 align:middle line:84% what the mechanism of disease is in that tissue. 00:01:06.840 --> 00:01:10.220 align:middle line:84% The other approach is simply disease classification, 00:01:10.220 --> 00:01:14.900 align:middle line:84% which can be done completely independently of understanding 00:01:14.900 --> 00:01:20.930 align:middle line:84% anything about particular genes but is essentially a strictly 00:01:20.930 --> 00:01:23.240 align:middle line:84% computational approach to looking 00:01:23.240 --> 00:01:26.270 align:middle line:84% at a large sequence of numbers that 00:01:26.270 --> 00:01:29.750 align:middle line:84% are generated by an experiment for a particular tissue. 00:01:29.750 --> 00:01:32.390 align:middle line:84% And doing this in multiple tissues 00:01:32.390 --> 00:01:36.170 align:middle line:84% and trying to classify disease, again, 00:01:36.170 --> 00:01:41.330 align:middle line:84% without reference to what the underlying biology is at all. 00:01:41.330 --> 00:01:44.790 align:middle line:90% So that's where we're focused. 00:01:44.790 --> 00:01:46.700 align:middle line:84% And I think the title of this block 00:01:46.700 --> 00:01:48.890 align:middle line:90% is The New Histopathology. 00:01:48.890 --> 00:01:53.120 align:middle line:84% And there's a number of papers that 00:01:53.120 --> 00:01:55.250 align:middle line:84% have come out over the last couple of years 00:01:55.250 --> 00:01:56.780 align:middle line:90% that are representative. 00:01:56.780 --> 00:02:02.235 align:middle line:84% Is that going to put anyone to sleep, you think? 00:02:02.235 --> 00:02:02.735 align:middle line:90% OK. 00:02:02.735 --> 00:02:05.990 align:middle line:90% 00:02:05.990 --> 00:02:08.870 align:middle line:84% So there's been a number of approaches 00:02:08.870 --> 00:02:13.530 align:middle line:84% to using microarray in this way and particularly in cancer. 00:02:13.530 --> 00:02:17.810 align:middle line:84% And I guess, you did hear about the approach in leukemia here. 00:02:17.810 --> 00:02:20.450 align:middle line:84% But there's a few different applications 00:02:20.450 --> 00:02:21.710 align:middle line:90% within classification. 00:02:21.710 --> 00:02:26.720 align:middle line:84% They include things like prediction over here, 00:02:26.720 --> 00:02:33.108 align:middle line:84% trying to predict outcome, another predictive one. 00:02:33.108 --> 00:02:35.150 align:middle line:84% This one is more of just straight classification. 00:02:35.150 --> 00:02:37.520 align:middle line:84% But we're going to talk about these different methods 00:02:37.520 --> 00:02:40.070 align:middle line:90% and approaches here. 00:02:40.070 --> 00:02:43.220 align:middle line:84% Fundamentally, I think this is similar to what 00:02:43.220 --> 00:02:48.470 align:middle line:84% happened when medical scientists started using microscopes 00:02:48.470 --> 00:02:50.430 align:middle line:90% to analyze disease. 00:02:50.430 --> 00:02:54.230 align:middle line:84% And this is a quote that microarrays have the potential 00:02:54.230 --> 00:02:58.130 align:middle line:84% to serve as microscopes to see a comprehensive dynamic molecular 00:02:58.130 --> 00:02:59.872 align:middle line:90% picture of a living cell. 00:02:59.872 --> 00:03:02.330 align:middle line:84% And so I'm going to carry that analogy through a little bit 00:03:02.330 --> 00:03:03.320 align:middle line:90% more. 00:03:03.320 --> 00:03:04.940 align:middle line:84% And we'll take a group of diseases 00:03:04.940 --> 00:03:09.500 align:middle line:84% that are muscle diseases, which is an area I work in, 00:03:09.500 --> 00:03:12.320 align:middle line:90% and look at it's classification. 00:03:12.320 --> 00:03:19.340 align:middle line:84% And so before microscopes were invented and used in medicine, 00:03:19.340 --> 00:03:22.010 align:middle line:84% physicians did certainly know about muscle diseases 00:03:22.010 --> 00:03:24.360 align:middle line:84% and had a certain classification of them. 00:03:24.360 --> 00:03:27.770 align:middle line:84% But having the microscope and looking at muscle tissue 00:03:27.770 --> 00:03:30.080 align:middle line:84% from patients under the microscope 00:03:30.080 --> 00:03:32.840 align:middle line:84% allowed for refinements in that classification. 00:03:32.840 --> 00:03:35.660 align:middle line:84% And the way that works is you look at tissue 00:03:35.660 --> 00:03:38.600 align:middle line:84% under a microscope, and this is a cross section of muscle. 00:03:38.600 --> 00:03:42.020 align:middle line:84% These are muscle fibers, the red one. 00:03:42.020 --> 00:03:44.000 align:middle line:84% There's some inflammatory cells here. 00:03:44.000 --> 00:03:47.900 align:middle line:84% But we can essentially make a list of the abnormal findings 00:03:47.900 --> 00:03:51.560 align:middle line:84% that we see in the muscle and use that 00:03:51.560 --> 00:03:54.740 align:middle line:90% to then classify disease. 00:03:54.740 --> 00:03:58.880 align:middle line:90% So other cases-- yeah. 00:03:58.880 --> 00:04:01.280 align:middle line:84% AUDIENCE: I've got a quick question So 00:04:01.280 --> 00:04:05.330 align:middle line:84% is the defining character of the inflammatory cells the fact 00:04:05.330 --> 00:04:06.830 align:middle line:84% that there are these little bodies, 00:04:06.830 --> 00:04:09.800 align:middle line:84% these little dark-staining bodies inside of them? 00:04:09.800 --> 00:04:12.093 align:middle line:90% Or are those the actual-- 00:04:12.093 --> 00:04:14.510 align:middle line:84% STEVEN A. GREENBERG: So some of these dark-staining bodies 00:04:14.510 --> 00:04:15.350 align:middle line:90% are nuclei. 00:04:15.350 --> 00:04:17.540 align:middle line:90% This is a nucleus, here. 00:04:17.540 --> 00:04:19.790 align:middle line:90% Muscle is a syncytium. 00:04:19.790 --> 00:04:23.600 align:middle line:84% So muscle fibers are tubes that have 00:04:23.600 --> 00:04:27.950 align:middle line:84% fused myoblasts that join their cytoplasm 00:04:27.950 --> 00:04:30.420 align:middle line:90% and have multiple nuclei. 00:04:30.420 --> 00:04:33.710 align:middle line:84% And so in this particular picture here, what you're 00:04:33.710 --> 00:04:34.970 align:middle line:90% looking at, that's a nucleus. 00:04:34.970 --> 00:04:37.190 align:middle line:84% That's a nucleus within a muscle fiber. 00:04:37.190 --> 00:04:39.110 align:middle line:90% These are inflammatory cells. 00:04:39.110 --> 00:04:40.430 align:middle line:90% They're a little bit larger. 00:04:40.430 --> 00:04:43.070 align:middle line:90% They stain a little bit darker. 00:04:43.070 --> 00:04:44.690 align:middle line:90% They have other characteristics. 00:04:44.690 --> 00:04:45.190 align:middle line:90% Yeah? 00:04:45.190 --> 00:04:46.857 align:middle line:84% AUDIENCE: They're also much smaller too. 00:04:46.857 --> 00:04:50.898 align:middle line:84% [INAUDIBLE] So what's that surrounding the pink region? 00:04:50.898 --> 00:04:52.190 align:middle line:90% STEVEN A. GREENBERG: This here? 00:04:52.190 --> 00:04:55.430 align:middle line:84% What this is, this is a degenerating-- a necrotic part 00:04:55.430 --> 00:05:00.020 align:middle line:84% of this muscle fiber that these inflammatory cells 00:05:00.020 --> 00:05:01.850 align:middle line:90% are invading. 00:05:01.850 --> 00:05:05.510 align:middle line:84% I can't tell you which cells they are from this view here. 00:05:05.510 --> 00:05:09.110 align:middle line:84% But they're likely to be macrophages and CD8-positive T 00:05:09.110 --> 00:05:14.060 align:middle line:90% cells that are attacking muscle. 00:05:14.060 --> 00:05:16.100 align:middle line:84% But the reason I use this example 00:05:16.100 --> 00:05:18.620 align:middle line:84% is, we can look at a variety of features 00:05:18.620 --> 00:05:21.750 align:middle line:84% when we look at muscle under the microscope. 00:05:21.750 --> 00:05:24.320 align:middle line:84% We have-- I better turn off my auto-- 00:05:24.320 --> 00:05:27.230 align:middle line:84% we have a lot of different features 00:05:27.230 --> 00:05:29.000 align:middle line:84% that we see that are characteristic 00:05:29.000 --> 00:05:30.470 align:middle line:90% of different diseases. 00:05:30.470 --> 00:05:32.570 align:middle line:84% Inflammatory cells are characteristic 00:05:32.570 --> 00:05:35.000 align:middle line:90% of inflammatory myopathies. 00:05:35.000 --> 00:05:37.460 align:middle line:84% Other myopathies, like muscular dystrophies, 00:05:37.460 --> 00:05:38.780 align:middle line:90% have a different pattern. 00:05:38.780 --> 00:05:39.890 align:middle line:90% There's no inflammation. 00:05:39.890 --> 00:05:43.070 align:middle line:90% There are large muscle fibers. 00:05:43.070 --> 00:05:45.950 align:middle line:84% Other diseases, like toxic myopathies 00:05:45.950 --> 00:05:49.310 align:middle line:84% have yet another characteristic feature when 00:05:49.310 --> 00:05:52.040 align:middle line:90% we look at it pathologically. 00:05:52.040 --> 00:05:54.980 align:middle line:84% And so the point I'm trying to make here 00:05:54.980 --> 00:05:57.890 align:middle line:84% is that looking under the microscope 00:05:57.890 --> 00:06:01.430 align:middle line:84% has allowed us to see different patterns within muscle 00:06:01.430 --> 00:06:04.910 align:middle line:84% and to realize that some of the things that were just called 00:06:04.910 --> 00:06:09.140 align:middle line:84% muscle diseases in the past are actually 00:06:09.140 --> 00:06:10.800 align:middle line:84% two different types of muscle disease. 00:06:10.800 --> 00:06:12.175 align:middle line:84% So what you're seeing on the left 00:06:12.175 --> 00:06:15.020 align:middle line:84% is a sample from a patient with polymyositis, 00:06:15.020 --> 00:06:17.490 align:middle line:84% which is a type of inflammatory muscle disease. 00:06:17.490 --> 00:06:20.360 align:middle line:84% And on the right is something called inclusion body myositis, 00:06:20.360 --> 00:06:23.630 align:middle line:84% and they're mainly distinguished by the presence of vacuoles 00:06:23.630 --> 00:06:26.330 align:middle line:84% that you can see under the microscope there. 00:06:26.330 --> 00:06:27.950 align:middle line:84% And they're very different diseases. 00:06:27.950 --> 00:06:30.320 align:middle line:84% Polymyositis responds to treatment 00:06:30.320 --> 00:06:33.080 align:middle line:84% with immunosuppressive agents, and inclusion body myositis 00:06:33.080 --> 00:06:34.590 align:middle line:90% does not. 00:06:34.590 --> 00:06:37.760 align:middle line:84% And so the microscope allows for a very meaningful 00:06:37.760 --> 00:06:42.860 align:middle line:84% classification of diseases that we were previously unaware 00:06:42.860 --> 00:06:47.340 align:middle line:84% of represented subtypes of disease. 00:06:47.340 --> 00:06:51.620 align:middle line:84% And we take that for granted now that part 00:06:51.620 --> 00:06:54.250 align:middle line:84% of accurate diagnosis for many diseases 00:06:54.250 --> 00:07:00.010 align:middle line:84% involves examination of tissue specimens under the microscope. 00:07:00.010 --> 00:07:01.722 align:middle line:84% But it's really-- the same analogy 00:07:01.722 --> 00:07:03.430 align:middle line:84% goes through with microarrays, and I just 00:07:03.430 --> 00:07:07.450 align:middle line:84% wanted to carry that through because we're not yet 00:07:07.450 --> 00:07:09.440 align:middle line:90% taking that for granted. 00:07:09.440 --> 00:07:12.760 align:middle line:84% So in a way, the microscope lets us look at tissue 00:07:12.760 --> 00:07:16.930 align:middle line:84% and enumerate a list of features in the tissue, 00:07:16.930 --> 00:07:18.340 align:middle line:90% and they're limited. 00:07:18.340 --> 00:07:21.430 align:middle line:84% The sort of things that we can see in diseases of muscle 00:07:21.430 --> 00:07:26.980 align:middle line:84% compared to normal muscle is a small list of different things 00:07:26.980 --> 00:07:29.260 align:middle line:90% that we might see. 00:07:29.260 --> 00:07:33.250 align:middle line:84% Microarrays, similarly, allow us to take a tissue of interest 00:07:33.250 --> 00:07:38.830 align:middle line:84% and to measure the expression of genes at the level of messenger 00:07:38.830 --> 00:07:44.200 align:middle line:84% RNA for thousands of genes simultaneously 00:07:44.200 --> 00:07:47.560 align:middle line:84% and similarly provides us with a quantitative list 00:07:47.560 --> 00:07:51.530 align:middle line:84% of expression levels for all these different genes. 00:07:51.530 --> 00:07:54.353 align:middle line:84% And so just how we-- similarly, how 00:07:54.353 --> 00:07:56.020 align:middle line:84% looking at the pattern of the microscope 00:07:56.020 --> 00:07:59.530 align:middle line:84% has helped to classify disease, looking 00:07:59.530 --> 00:08:03.070 align:middle line:84% at the pattern of numbers from a microarray experiment 00:08:03.070 --> 00:08:08.642 align:middle line:84% is also being used to classify disease. 00:08:08.642 --> 00:08:09.225 align:middle line:90% Is that clear? 00:08:09.225 --> 00:08:14.090 align:middle line:90% 00:08:14.090 --> 00:08:15.320 align:middle line:90% Yes? 00:08:15.320 --> 00:08:16.430 align:middle line:90% AUDIENCE: You can also-- 00:08:16.430 --> 00:08:20.930 align:middle line:84% so the example you gave with the microscope, 00:08:20.930 --> 00:08:23.450 align:middle line:84% if you combine that with what we know 00:08:23.450 --> 00:08:27.740 align:middle line:84% about immunological inflammation process, 00:08:27.740 --> 00:08:30.350 align:middle line:84% it could also be used in that other branch 00:08:30.350 --> 00:08:32.929 align:middle line:84% that you mentioned at the start, the pathophysiology. 00:08:32.929 --> 00:08:38.320 align:middle line:90% 00:08:38.320 --> 00:08:42.980 align:middle line:84% So that's where the analogy is coming from, is that you can-- 00:08:42.980 --> 00:08:47.190 align:middle line:84% the microscopic classification of disease, 00:08:47.190 --> 00:08:49.925 align:middle line:84% it's more that the microscopes are the direct analogies 00:08:49.925 --> 00:08:52.250 align:middle line:90% [INAUDIBLE]? 00:08:52.250 --> 00:08:54.170 align:middle line:84% STEVEN A. GREENBERG: Well, so what 00:08:54.170 --> 00:08:56.870 align:middle line:84% we see under the microscope also helps us to understand disease 00:08:56.870 --> 00:08:59.440 align:middle line:90% as well as to classify it. 00:08:59.440 --> 00:09:02.150 align:middle line:84% I mean, so going back to this here, 00:09:02.150 --> 00:09:05.450 align:middle line:84% this process is completely independent of understanding 00:09:05.450 --> 00:09:06.710 align:middle line:90% disease here. 00:09:06.710 --> 00:09:09.920 align:middle line:84% We don't know what vacuoles are due to an IBM. 00:09:09.920 --> 00:09:14.600 align:middle line:84% Nobody really has much idea about why vacuoles form. 00:09:14.600 --> 00:09:16.430 align:middle line:84% Similarly with polymyositis, we know 00:09:16.430 --> 00:09:17.900 align:middle line:90% they're inflammatory cells. 00:09:17.900 --> 00:09:19.880 align:middle line:84% It's believed to be an autoimmune disorder, 00:09:19.880 --> 00:09:22.160 align:middle line:84% but we don't really have any antigens. 00:09:22.160 --> 00:09:24.110 align:middle line:84% We don't really know how it starts. 00:09:24.110 --> 00:09:27.590 align:middle line:84% We don't if it's antigen-specific, even. 00:09:27.590 --> 00:09:29.690 align:middle line:84% But even putting that completely aside, 00:09:29.690 --> 00:09:34.130 align:middle line:84% if you just describe features, which are internal nuclei, 00:09:34.130 --> 00:09:38.030 align:middle line:84% inflammatory cells, et cetera-- just describing features 00:09:38.030 --> 00:09:40.520 align:middle line:84% allows one to distinguish meaningful categories 00:09:40.520 --> 00:09:43.710 align:middle line:84% of disease that are clinically important. 00:09:43.710 --> 00:09:46.100 align:middle line:84% There are many patients with inclusion body myositis 00:09:46.100 --> 00:09:47.810 align:middle line:84% who are being treated with steroids 00:09:47.810 --> 00:09:51.680 align:middle line:84% and get labeled as so-called refractory polymyositis 00:09:51.680 --> 00:09:54.470 align:middle line:84% before somebody pays sufficient attention to their muscle 00:09:54.470 --> 00:09:57.890 align:middle line:84% biopsy slides and realizes they don't have polymyositis. 00:09:57.890 --> 00:10:01.010 align:middle line:84% So it's a very clinically relevant area, 00:10:01.010 --> 00:10:06.080 align:middle line:84% but it's completely independent of any biological knowledge 00:10:06.080 --> 00:10:07.620 align:middle line:90% here. 00:10:07.620 --> 00:10:11.360 align:middle line:84% And so that's why I stress this extreme. 00:10:11.360 --> 00:10:13.880 align:middle line:84% They certainly-- both microscopes and microarrays 00:10:13.880 --> 00:10:17.060 align:middle line:90% have applications in both areas. 00:10:17.060 --> 00:10:19.550 align:middle line:84% But this one area of disease classification 00:10:19.550 --> 00:10:21.830 align:middle line:84% can take place completely independently 00:10:21.830 --> 00:10:26.840 align:middle line:84% of understanding mechanism through this analogy. 00:10:26.840 --> 00:10:31.780 align:middle line:90% 00:10:31.780 --> 00:10:35.400 align:middle line:84% So disease classification, the area of disease classification 00:10:35.400 --> 00:10:39.760 align:middle line:84% actually has a few different areas within it. 00:10:39.760 --> 00:10:41.680 align:middle line:84% And I want to formalize that a bit. 00:10:41.680 --> 00:10:46.110 align:middle line:84% And they include discovery, diagnostics, 00:10:46.110 --> 00:10:48.810 align:middle line:84% but they all have in common the concept of an expression 00:10:48.810 --> 00:10:51.990 align:middle line:84% profile representing a signature. 00:10:51.990 --> 00:10:54.030 align:middle line:84% So the expression profile is a term 00:10:54.030 --> 00:10:57.270 align:middle line:84% we use for this vector of numbers 00:10:57.270 --> 00:11:00.970 align:middle line:84% that we get from a microarray experiment on a single tissue. 00:11:00.970 --> 00:11:03.360 align:middle line:90% So you take one muscle sample. 00:11:03.360 --> 00:11:05.370 align:middle line:90% You extract its RNA. 00:11:05.370 --> 00:11:08.970 align:middle line:84% You hybridize it to a microarray experiment overnight, 00:11:08.970 --> 00:11:11.790 align:middle line:84% and you get back a list of 10,000 00:11:11.790 --> 00:11:15.540 align:middle line:84% or 20,000 genes and their expression levels. 00:11:15.540 --> 00:11:18.900 align:middle line:84% And we can view that list as an ordered vector, 00:11:18.900 --> 00:11:21.540 align:middle line:84% and we can compare that to other muscle samples that 00:11:21.540 --> 00:11:26.160 align:middle line:84% had a microarray experiment and also gave back a list of 20,000 00:11:26.160 --> 00:11:28.170 align:middle line:90% ordered numbers. 00:11:28.170 --> 00:11:31.020 align:middle line:84% And we can ask questions about how similar 00:11:31.020 --> 00:11:35.235 align:middle line:84% these two ordered lists of numbers are to each other. 00:11:35.235 --> 00:11:38.917 align:middle line:90% 00:11:38.917 --> 00:11:39.500 align:middle line:90% Is that clear? 00:11:39.500 --> 00:11:42.410 align:middle line:90% 00:11:42.410 --> 00:11:44.540 align:middle line:90% OK, we'll go through that more. 00:11:44.540 --> 00:11:47.570 align:middle line:84% So but the basic idea is that the expression-- so 00:11:47.570 --> 00:11:52.610 align:middle line:84% an expression profile is this vector of numbers. 00:11:52.610 --> 00:11:55.910 align:middle line:84% And it's the expression profile for a single tissue 00:11:55.910 --> 00:11:57.920 align:middle line:90% under a single experiment. 00:11:57.920 --> 00:12:01.340 align:middle line:84% And they're often viewed as signatures, 00:12:01.340 --> 00:12:05.670 align:middle line:84% as unique to this tissue or this to disease. 00:12:05.670 --> 00:12:08.840 align:middle line:84% And so we just want to explore that concept of signatures 00:12:08.840 --> 00:12:10.190 align:middle line:90% a little bit. 00:12:10.190 --> 00:12:12.980 align:middle line:84% As an aside, this is the disease pathophysiology, 00:12:12.980 --> 00:12:14.510 align:middle line:90% the other branch. 00:12:14.510 --> 00:12:19.280 align:middle line:84% And I'm contrasting a bit back and forth, just to be clear. 00:12:19.280 --> 00:12:22.190 align:middle line:84% I don't if you guys have done this 00:12:22.190 --> 00:12:24.170 align:middle line:84% or had lectures about doing this and how 00:12:24.170 --> 00:12:27.500 align:middle line:84% to identify differentially expressed genes and tissues 00:12:27.500 --> 00:12:28.820 align:middle line:90% versus controls. 00:12:28.820 --> 00:12:33.230 align:middle line:84% They're fairly simple methods of just comparing ratios 00:12:33.230 --> 00:12:35.240 align:middle line:90% for doing statistical analyses. 00:12:35.240 --> 00:12:38.820 align:middle line:84% And I'm going to skip that part and get back to classic. 00:12:38.820 --> 00:12:41.990 align:middle line:84% So staying back in classification now, 00:12:41.990 --> 00:12:46.010 align:middle line:84% the first question formalizes what is an expression profile 00:12:46.010 --> 00:12:52.130 align:middle line:84% signature and to ask whether it truly 00:12:52.130 --> 00:12:54.143 align:middle line:90% exists in a given situation. 00:12:54.143 --> 00:12:56.060 align:middle line:84% There are many papers out there that will say, 00:12:56.060 --> 00:12:58.790 align:middle line:84% we looked at an expression profile signature. 00:12:58.790 --> 00:13:02.240 align:middle line:84% But they fail to demonstrate that what they got out 00:13:02.240 --> 00:13:05.780 align:middle line:84% of a microarray experiment was a signature of that tissue other 00:13:05.780 --> 00:13:09.380 align:middle line:90% than, say, random noise. 00:13:09.380 --> 00:13:13.370 align:middle line:84% And so there are some important aspects to a signature. 00:13:13.370 --> 00:13:17.270 align:middle line:84% And that analogy goes to that signature we use when we write. 00:13:17.270 --> 00:13:20.960 align:middle line:84% And signatures that we use are useful because they're 00:13:20.960 --> 00:13:21.560 align:middle line:90% distinct. 00:13:21.560 --> 00:13:24.200 align:middle line:84% My signature is different than yours 00:13:24.200 --> 00:13:25.580 align:middle line:90% because it's reproducible. 00:13:25.580 --> 00:13:27.960 align:middle line:84% Whenever I write my name down and my signature, 00:13:27.960 --> 00:13:32.000 align:middle line:84% it's pretty similar in that it's readable in a certain way. 00:13:32.000 --> 00:13:36.430 align:middle line:84% That's not always the case with signatures. 00:13:36.430 --> 00:13:39.100 align:middle line:84% This is similarly true of what we would call classes. 00:13:39.100 --> 00:13:45.850 align:middle line:84% And classes are groups of conditions that fit 00:13:45.850 --> 00:13:48.520 align:middle line:90% into one category of some sort. 00:13:48.520 --> 00:13:51.130 align:middle line:84% In this context, in the context of using microarrays 00:13:51.130 --> 00:13:55.840 align:middle line:84% to classify disease, they are, say, the expression pattern-- 00:13:55.840 --> 00:14:01.300 align:middle line:84% the expression profile from 10 patients with a single disease. 00:14:01.300 --> 00:14:05.890 align:middle line:84% Does that form a class, those 10 expression profiles together? 00:14:05.890 --> 00:14:10.936 align:middle line:84% And classes need to have similar properties as signatures. 00:14:10.936 --> 00:14:13.700 align:middle line:84% OK, I don't know why I put that up there for that one. 00:14:13.700 --> 00:14:19.150 align:middle line:84% OK, so this signature idea, again, is, that's my signature, 00:14:19.150 --> 00:14:22.360 align:middle line:84% and that expression profile would represent 00:14:22.360 --> 00:14:28.630 align:middle line:84% the profile of a tissue specimen under microarray experiments. 00:14:28.630 --> 00:14:31.870 align:middle line:84% And so the first question is, is it reproducible? 00:14:31.870 --> 00:14:34.983 align:middle line:84% And so when you do look at papers in this area 00:14:34.983 --> 00:14:36.400 align:middle line:84% and you're interested in data, you 00:14:36.400 --> 00:14:40.480 align:middle line:84% need to see that somebody can do the same thing twice 00:14:40.480 --> 00:14:44.200 align:middle line:84% on a given tissue and get the same pattern out. 00:14:44.200 --> 00:14:46.840 align:middle line:84% I could take bucket of paint and throw it on the wall 00:14:46.840 --> 00:14:50.290 align:middle line:84% and call that a signature, but it only is meaningful 00:14:50.290 --> 00:14:53.560 align:middle line:84% if I can reproduce that and get the same pattern when 00:14:53.560 --> 00:14:55.210 align:middle line:90% I throw the bucket of paint on. 00:14:55.210 --> 00:14:57.970 align:middle line:84% So you do need to show that you can, 00:14:57.970 --> 00:15:00.040 align:middle line:84% and, certainly, it is possible to do it. 00:15:00.040 --> 00:15:02.530 align:middle line:84% This is data from our experiments. 00:15:02.530 --> 00:15:04.660 align:middle line:84% But you should see this type of data 00:15:04.660 --> 00:15:06.910 align:middle line:84% when you look at papers that people 00:15:06.910 --> 00:15:11.010 align:middle line:84% have reproduced the profile they get out, 00:15:11.010 --> 00:15:13.180 align:middle line:84% done two experiments on their tissue sample 00:15:13.180 --> 00:15:19.000 align:middle line:84% and gotten the same pattern of numbers out each time. 00:15:19.000 --> 00:15:20.950 align:middle line:84% It needs to be clear that this is distinct, 00:15:20.950 --> 00:15:23.380 align:middle line:84% that the signature from, say, one disease 00:15:23.380 --> 00:15:26.580 align:middle line:90% is different from another. 00:15:26.580 --> 00:15:27.830 align:middle line:90% And there are ways to do that. 00:15:27.830 --> 00:15:30.820 align:middle line:84% And then the question of readability comes out. 00:15:30.820 --> 00:15:34.010 align:middle line:84% And that's, how do we read a signature? 00:15:34.010 --> 00:15:37.630 align:middle line:84% So if I do an experiment, I have these vectors of numbers. 00:15:37.630 --> 00:15:40.120 align:middle line:84% And I'm going to do this for 10 different tissues 00:15:40.120 --> 00:15:43.090 align:middle line:84% and compare and ask, how are these vectors 00:15:43.090 --> 00:15:45.230 align:middle line:90% similar to each other? 00:15:45.230 --> 00:15:50.210 align:middle line:84% We have to define some way of comparing vectors to each other 00:15:50.210 --> 00:15:52.330 align:middle line:90% and judging their similarity. 00:15:52.330 --> 00:15:57.770 align:middle line:84% These are 20,000-dimensional vectors here. 00:15:57.770 --> 00:15:59.860 align:middle line:84% And there's a number of different measures 00:15:59.860 --> 00:16:05.410 align:middle line:84% that are used to look at similarity of these profiles. 00:16:05.410 --> 00:16:07.910 align:middle line:84% And they include use of correlation coefficient-- 00:16:07.910 --> 00:16:11.950 align:middle line:84% so just taking the Pearson correlation coefficient 00:16:11.950 --> 00:16:15.260 align:middle line:90% of these two vectors. 00:16:15.260 --> 00:16:18.040 align:middle line:90% And that's what I did here. 00:16:18.040 --> 00:16:20.080 align:middle line:84% This is just the Pearson correlation coefficient 00:16:20.080 --> 00:16:22.375 align:middle line:84% of one set of microarray numbers against another. 00:16:22.375 --> 00:16:25.120 align:middle line:90% 00:16:25.120 --> 00:16:27.970 align:middle line:84% Euclidean distance-- mutual information 00:16:27.970 --> 00:16:30.670 align:middle line:84% has just about dropped off the scene at this point 00:16:30.670 --> 00:16:33.160 align:middle line:90% as a use in this field. 00:16:33.160 --> 00:16:35.530 align:middle line:84% But there are a number of similarity measures, 00:16:35.530 --> 00:16:39.520 align:middle line:84% and this will change how one reads these profiles 00:16:39.520 --> 00:16:40.660 align:middle line:90% and compares them. 00:16:40.660 --> 00:16:44.540 align:middle line:90% 00:16:44.540 --> 00:16:48.550 align:middle line:84% And so the same principles that apply to signatures also 00:16:48.550 --> 00:16:50.050 align:middle line:90% apply to classes. 00:16:50.050 --> 00:16:52.660 align:middle line:84% If we define classes, the classes also 00:16:52.660 --> 00:16:56.020 align:middle line:90% have to have the same features. 00:16:56.020 --> 00:17:00.220 align:middle line:84% And the way we look at distinctness of classes 00:17:00.220 --> 00:17:04.119 align:middle line:84% are mostly through clustering methods. 00:17:04.119 --> 00:17:07.510 align:middle line:84% So clustering methods, which you may have had some exposure 00:17:07.510 --> 00:17:10.510 align:middle line:84% to here, what clustering methods are 00:17:10.510 --> 00:17:14.020 align:middle line:84% a way of organizing tissue samples by similarity 00:17:14.020 --> 00:17:16.599 align:middle line:90% into classes. 00:17:16.599 --> 00:17:19.630 align:middle line:84% And they differ substantially in terms of the way 00:17:19.630 --> 00:17:23.079 align:middle line:84% that they create a organizational class 00:17:23.079 --> 00:17:24.819 align:middle line:90% structure or classification. 00:17:24.819 --> 00:17:28.780 align:middle line:84% So hierarchical classification is used quite a lot, 00:17:28.780 --> 00:17:34.820 align:middle line:84% and these are trees, basically, of relationships. 00:17:34.820 --> 00:17:37.763 align:middle line:84% They're not actually true classes in the sense 00:17:37.763 --> 00:17:38.680 align:middle line:90% that they're distinct. 00:17:38.680 --> 00:17:41.050 align:middle line:84% Everything is related to everything else 00:17:41.050 --> 00:17:42.940 align:middle line:90% under hierarchical clustering. 00:17:42.940 --> 00:17:46.420 align:middle line:84% But the degree of relationship is just greater 00:17:46.420 --> 00:17:52.330 align:middle line:84% for profiles that get classified together 00:17:52.330 --> 00:17:57.220 align:middle line:90% on the same tree, the same node. 00:17:57.220 --> 00:17:59.260 align:middle line:84% There's different types of hierarchical classes. 00:17:59.260 --> 00:18:02.420 align:middle line:84% There's non-hierarchical classification. 00:18:02.420 --> 00:18:04.150 align:middle line:84% And so the point here is to emphasize 00:18:04.150 --> 00:18:07.030 align:middle line:84% there's a variety of mathematical methods 00:18:07.030 --> 00:18:10.390 align:middle line:84% that are still used in this field 00:18:10.390 --> 00:18:12.760 align:middle line:84% to try and define distinctness of classes. 00:18:12.760 --> 00:18:15.780 align:middle line:90% 00:18:15.780 --> 00:18:18.420 align:middle line:84% Reproducibility-- so again, the signature issue 00:18:18.420 --> 00:18:20.430 align:middle line:84% was distinctness, reproducibility, 00:18:20.430 --> 00:18:21.810 align:middle line:90% and readability. 00:18:21.810 --> 00:18:25.860 align:middle line:84% Reproducibility of classes is a big issue here, 00:18:25.860 --> 00:18:30.420 align:middle line:84% that when you see papers which will look to say 00:18:30.420 --> 00:18:32.190 align:middle line:84% we have a certain classification. 00:18:32.190 --> 00:18:36.180 align:middle line:84% And like the leukemia one, we have ALL and AML, 00:18:36.180 --> 00:18:38.280 align:middle line:90% separable into two classes. 00:18:38.280 --> 00:18:40.050 align:middle line:84% It's important to ask whether this 00:18:40.050 --> 00:18:43.230 align:middle line:84% is a reproducible classification system, 00:18:43.230 --> 00:18:47.370 align:middle line:84% that if you do this under some slightly perturbed way, 00:18:47.370 --> 00:18:51.730 align:middle line:84% whether you end up with the same classification. 00:18:51.730 --> 00:18:54.960 align:middle line:84% And this is a somewhat neglected area 00:18:54.960 --> 00:18:59.070 align:middle line:84% still within this field of demonstrating reproducibility 00:18:59.070 --> 00:19:01.620 align:middle line:90% of classes. 00:19:01.620 --> 00:19:04.830 align:middle line:84% Readability-- we mentioned the different similarity measures 00:19:04.830 --> 00:19:08.620 align:middle line:84% that are used to read out expression profiles. 00:19:08.620 --> 00:19:12.560 align:middle line:84% This is just the correlation coefficient. 00:19:12.560 --> 00:19:14.310 align:middle line:84% OK, so we're going to go into a little bit 00:19:14.310 --> 00:19:16.560 align:middle line:84% more detail about a hierarchical clustering 00:19:16.560 --> 00:19:20.400 align:middle line:84% because it's such a commonly used technique now 00:19:20.400 --> 00:19:24.150 align:middle line:84% and how tissues are organized by similarity. 00:19:24.150 --> 00:19:27.750 align:middle line:84% And the basic method for this is to-- 00:19:27.750 --> 00:19:31.390 align:middle line:84% so again, our basic data set consists of, let's say, 00:19:31.390 --> 00:19:33.990 align:middle line:84% 20 different tissue samples and the expression 00:19:33.990 --> 00:19:37.320 align:middle line:90% profile for each one-- 00:19:37.320 --> 00:19:43.020 align:middle line:84% so 10,000 genes measured for each of 20 different profiles. 00:19:43.020 --> 00:19:45.030 align:middle line:84% And we're going to try and organize 00:19:45.030 --> 00:19:49.170 align:middle line:84% these 20 different tissue samples into groups 00:19:49.170 --> 00:19:53.910 align:middle line:84% or some structure based on their similarity of their expression 00:19:53.910 --> 00:19:56.310 align:middle line:90% profiles. 00:19:56.310 --> 00:19:58.680 align:middle line:84% And so the way hierarchical clustering works 00:19:58.680 --> 00:20:01.960 align:middle line:84% is to first look at all pairwise correlation coefficients 00:20:01.960 --> 00:20:03.810 align:middle line:90% of these 20 samples. 00:20:03.810 --> 00:20:06.180 align:middle line:84% So sample number 1 and sample number 4 00:20:06.180 --> 00:20:08.250 align:middle line:84% have a certain correlation with each other. 00:20:08.250 --> 00:20:10.708 align:middle line:84% And 1 and 5 have a certain correlation with them. 00:20:10.708 --> 00:20:12.750 align:middle line:84% And we're just going to calculate all correlation 00:20:12.750 --> 00:20:14.610 align:middle line:90% coefficients. 00:20:14.610 --> 00:20:16.050 align:middle line:90% That's the first step. 00:20:16.050 --> 00:20:18.940 align:middle line:84% And then for the single highest correlation coefficient-- 00:20:18.940 --> 00:20:21.210 align:middle line:84% so the two samples that are most correlated, 00:20:21.210 --> 00:20:24.090 align:middle line:84% it might be sample 8 and sample 15-- 00:20:24.090 --> 00:20:29.430 align:middle line:84% we're going to define those as being together, as 00:20:29.430 --> 00:20:33.030 align:middle line:84% close together and being two leaves, basically, 00:20:33.030 --> 00:20:37.950 align:middle line:84% on the end of a branch of a tree. 00:20:37.950 --> 00:20:42.360 align:middle line:84% And then we're going to repeat this for the remaining samples 00:20:42.360 --> 00:20:46.230 align:middle line:84% and build a tree up, basically, from the leaves. 00:20:46.230 --> 00:20:49.720 align:middle line:84% This is one type of hierarchical clustering. 00:20:49.720 --> 00:20:52.560 align:middle line:84% So here, there's scatterplot diagrams 00:20:52.560 --> 00:20:58.470 align:middle line:84% for three of the pairs within a data set that are shown. 00:20:58.470 --> 00:21:03.510 align:middle line:84% But you do this for all pairs here. 00:21:03.510 --> 00:21:08.490 align:middle line:84% And then you start-- let's say, sample 1 and 2 00:21:08.490 --> 00:21:11.580 align:middle line:84% were the most correlated from the whole sample. 00:21:11.580 --> 00:21:14.910 align:middle line:84% You join them together as leaves. 00:21:14.910 --> 00:21:19.560 align:middle line:84% And then, let's say, the next ones with the highest 00:21:19.560 --> 00:21:23.520 align:middle line:84% correlation coefficient are sample 7 and 5. 00:21:23.520 --> 00:21:25.540 align:middle line:90% They get joined together. 00:21:25.540 --> 00:21:27.840 align:middle line:84% And then the next highest correlation coefficient 00:21:27.840 --> 00:21:33.090 align:middle line:84% might be the average coefficient of the S1/S2 group and the 75, 00:21:33.090 --> 00:21:34.540 align:middle line:90% so they get joined together. 00:21:34.540 --> 00:21:37.200 align:middle line:90% So this tree gets built up. 00:21:37.200 --> 00:21:41.580 align:middle line:84% And I guess, I'm assuming you've seen 00:21:41.580 --> 00:21:44.310 align:middle line:84% some of these pictures of hierarchical clustering 00:21:44.310 --> 00:21:46.560 align:middle line:90% diagrams. 00:21:46.560 --> 00:21:50.700 align:middle line:90% Have you seen any pictures? 00:21:50.700 --> 00:21:54.000 align:middle line:84% OK, well, maybe I'll show you one. 00:21:54.000 --> 00:21:59.100 align:middle line:84% Another way to look at this process is to switch to colors. 00:21:59.100 --> 00:22:02.430 align:middle line:84% And this is a pretty common visual representation 00:22:02.430 --> 00:22:06.310 align:middle line:84% in this field of how to represent these data sets. 00:22:06.310 --> 00:22:09.360 align:middle line:84% So what I have here is, my columns 00:22:09.360 --> 00:22:11.980 align:middle line:84% are each a different tissue specimen. 00:22:11.980 --> 00:22:15.150 align:middle line:84% So they're different, let's say, muscle biopsy specimens 00:22:15.150 --> 00:22:16.770 align:middle line:90% from patients. 00:22:16.770 --> 00:22:20.500 align:middle line:84% And for each one, we did a microarray experiment. 00:22:20.500 --> 00:22:22.680 align:middle line:84% So there were seven microarray experiments done. 00:22:22.680 --> 00:22:27.300 align:middle line:84% And we measured 12,000 genes and their expression here. 00:22:27.300 --> 00:22:31.710 align:middle line:84% And we put this into a table and then color code it 00:22:31.710 --> 00:22:35.880 align:middle line:84% according to some color scale so that the greens represent 00:22:35.880 --> 00:22:41.280 align:middle line:84% the lowest expression levels and the reds, the highest. 00:22:41.280 --> 00:22:45.400 align:middle line:84% And so we end up in this color diagram here. 00:22:45.400 --> 00:22:49.470 align:middle line:84% And so the process of them classifying our tissues 00:22:49.470 --> 00:22:52.200 align:middle line:84% is a process whereby we try and find 00:22:52.200 --> 00:22:57.405 align:middle line:84% specimens that have similar colors at each gene. 00:22:57.405 --> 00:23:01.830 align:middle line:90% 00:23:01.830 --> 00:23:08.010 align:middle line:84% And so classification involves shuffling these columns. 00:23:08.010 --> 00:23:09.690 align:middle line:90% One can also classify rows. 00:23:09.690 --> 00:23:12.660 align:middle line:84% But for the purpose of classifying disease, 00:23:12.660 --> 00:23:15.290 align:middle line:84% we'll shuffle the columns accordingly. 00:23:15.290 --> 00:23:20.180 align:middle line:84% And when you do that using hierarchical clustering or 00:23:20.180 --> 00:23:24.320 align:middle line:84% other algorithms to do that, this contrived example, 00:23:24.320 --> 00:23:28.350 align:middle line:84% there were three different classes represented in here. 00:23:28.350 --> 00:23:30.410 align:middle line:84% And that's what disease classification 00:23:30.410 --> 00:23:32.630 align:middle line:90% is using microarrays. 00:23:32.630 --> 00:23:35.400 align:middle line:90% 00:23:35.400 --> 00:23:39.660 align:middle line:84% One can also shuffle the rows and classify the gene 00:23:39.660 --> 00:23:42.810 align:middle line:90% or cluster the genes here. 00:23:42.810 --> 00:23:45.920 align:middle line:84% What that tends to do is to bring genes together 00:23:45.920 --> 00:23:50.372 align:middle line:84% of similar function within the rows 00:23:50.372 --> 00:23:52.080 align:middle line:84% so that genes that are behaving similarly 00:23:52.080 --> 00:23:55.890 align:middle line:84% across this data set of tissues will 00:23:55.890 --> 00:23:58.500 align:middle line:90% tend to get grouped together. 00:23:58.500 --> 00:24:00.660 align:middle line:90% And that's a different-- 00:24:00.660 --> 00:24:02.160 align:middle line:84% there's things you can do with that. 00:24:02.160 --> 00:24:05.580 align:middle line:84% You can infer gene function and other things. 00:24:05.580 --> 00:24:07.165 align:middle line:84% But it's not what we're focused on. 00:24:07.165 --> 00:24:07.665 align:middle line:90% Yes? 00:24:07.665 --> 00:24:09.730 align:middle line:84% AUDIENCE: I got a quick question. 00:24:09.730 --> 00:24:14.785 align:middle line:84% So this is just a representation of a microarray experiment? 00:24:14.785 --> 00:24:16.410 align:middle line:84% STEVEN A. GREENBERG: Of a group, a data 00:24:16.410 --> 00:24:19.740 align:middle line:84% set made up of seven microarray experiments. 00:24:19.740 --> 00:24:21.540 align:middle line:90% AUDIENCE: Right. 00:24:21.540 --> 00:24:26.640 align:middle line:84% And when you're choosing your tissue specimens, 00:24:26.640 --> 00:24:28.650 align:middle line:84% the seven that you're looking at, 00:24:28.650 --> 00:24:35.340 align:middle line:84% are those the actual, I guess, specimens from tissues 00:24:35.340 --> 00:24:39.870 align:middle line:84% that you suspect for now has a particular clinical phenotype 00:24:39.870 --> 00:24:41.130 align:middle line:90% of a disease? 00:24:41.130 --> 00:24:49.470 align:middle line:84% Or do you throw in your control tissue specimens within this, 00:24:49.470 --> 00:24:52.290 align:middle line:84% basically, to demonstrate that there isn't? 00:24:52.290 --> 00:24:54.750 align:middle line:84% STEVEN A. GREENBERG: It depends on what you want to study, 00:24:54.750 --> 00:24:56.500 align:middle line:90% what you want to demonstrate. 00:24:56.500 --> 00:24:57.930 align:middle line:90% And so there certainly are-- 00:24:57.930 --> 00:25:00.360 align:middle line:84% in the diseases I study, there are patients whose muscle 00:25:00.360 --> 00:25:04.950 align:middle line:84% biopsy looks normal under the microscope, 00:25:04.950 --> 00:25:07.180 align:middle line:84% even though they have a muscle disease. 00:25:07.180 --> 00:25:12.570 align:middle line:84% And so that's of interest to us then to do a classification 00:25:12.570 --> 00:25:15.480 align:middle line:84% experiment here, where we take that data from, say, five 00:25:15.480 --> 00:25:18.810 align:middle line:84% patients with a disease, but they have normal muscle 00:25:18.810 --> 00:25:22.200 align:middle line:84% biopsies, and five patients with normal muscle biopsies 00:25:22.200 --> 00:25:25.860 align:middle line:84% who don't have a disease, and to shuffle them, and see if they 00:25:25.860 --> 00:25:29.880 align:middle line:90% fall into two classes properly. 00:25:29.880 --> 00:25:30.540 align:middle line:90% And they do. 00:25:30.540 --> 00:25:34.740 align:middle line:84% I'll show you an example of that. 00:25:34.740 --> 00:25:37.860 align:middle line:84% So in that way, we can demonstrate that, sometimes, 00:25:37.860 --> 00:25:39.630 align:middle line:84% microarrays can see things that we 00:25:39.630 --> 00:25:42.240 align:middle line:90% don't see under the microscope. 00:25:42.240 --> 00:25:46.230 align:middle line:84% But it strictly depends on a particular application. 00:25:46.230 --> 00:25:49.743 align:middle line:84% The principles are what we're focused on here. 00:25:49.743 --> 00:25:51.660 align:middle line:84% AUDIENCE: And when you're doing these samples, 00:25:51.660 --> 00:25:53.400 align:middle line:84% are you doing them more in the sense 00:25:53.400 --> 00:25:56.670 align:middle line:84% that you're organizing these according 00:25:56.670 --> 00:26:01.410 align:middle line:84% to either row or column before you know what's going on? 00:26:01.410 --> 00:26:03.660 align:middle line:84% STEVEN A. GREENBERG: Yeah, they're being done strictly 00:26:03.660 --> 00:26:08.700 align:middle line:84% through, say, an algorithm like this, hierarchical clustering, 00:26:08.700 --> 00:26:11.610 align:middle line:84% where you're just looking at the columns of numbers, 00:26:11.610 --> 00:26:14.790 align:middle line:84% and you're saying, this vector is similar to this vector. 00:26:14.790 --> 00:26:18.030 align:middle line:90% Let's put them together. 00:26:18.030 --> 00:26:20.580 align:middle line:84% This other vector is the next similar most. 00:26:20.580 --> 00:26:22.560 align:middle line:90% Let's put that together. 00:26:22.560 --> 00:26:25.740 align:middle line:84% So it's completely independent of the labels. 00:26:25.740 --> 00:26:30.870 align:middle line:84% So a lot of people think of this field as supervised or-- 00:26:30.870 --> 00:26:36.420 align:middle line:84% so this particular application is unsupervised classification. 00:26:36.420 --> 00:26:38.880 align:middle line:84% Have those terms come up, supervised versus un? 00:26:38.880 --> 00:26:41.880 align:middle line:84% It's the way a lot of the artificial intelligence 00:26:41.880 --> 00:26:44.610 align:middle line:84% and machine learning community thinks about this area. 00:26:44.610 --> 00:26:47.910 align:middle line:84% I don't like to think about this area in that way 00:26:47.910 --> 00:26:52.080 align:middle line:84% because that's secondary whether a method is 00:26:52.080 --> 00:26:53.790 align:middle line:90% unsupervised or supervised. 00:26:53.790 --> 00:26:56.430 align:middle line:90% 00:26:56.430 --> 00:26:57.240 align:middle line:90% That hasn't helped. 00:26:57.240 --> 00:27:01.985 align:middle line:84% That's confused me so many times that I 00:27:01.985 --> 00:27:03.360 align:middle line:84% like to think about it in the way 00:27:03.360 --> 00:27:05.430 align:middle line:90% that I'm presenting it to you. 00:27:05.430 --> 00:27:11.100 align:middle line:84% So this is disease classification. 00:27:11.100 --> 00:27:14.520 align:middle line:84% So there is a group of distinct methods 00:27:14.520 --> 00:27:17.520 align:middle line:84% that we call class methods that are 00:27:17.520 --> 00:27:21.900 align:middle line:84% applicable for a number of different types of applications 00:27:21.900 --> 00:27:23.820 align:middle line:90% within disease classification. 00:27:23.820 --> 00:27:29.100 align:middle line:84% And the three methods are class comparison, class prediction, 00:27:29.100 --> 00:27:30.420 align:middle line:90% and class discovery. 00:27:30.420 --> 00:27:33.640 align:middle line:90% 00:27:33.640 --> 00:27:40.870 align:middle line:84% And so class comparison is an area in which we are simply 00:27:40.870 --> 00:27:48.820 align:middle line:84% describing the different classes that might exist within one 00:27:48.820 --> 00:27:53.800 align:middle line:84% of these data sets that we put together from-- let's take, 00:27:53.800 --> 00:27:58.750 align:middle line:84% for an example, 50 samples and a microarray experiment 00:27:58.750 --> 00:28:02.390 align:middle line:84% for each of those, and consider that one data set. 00:28:02.390 --> 00:28:07.150 align:middle line:84% And so class comparison is a exploratory method 00:28:07.150 --> 00:28:10.870 align:middle line:84% to just compare and contrast whether there 00:28:10.870 --> 00:28:14.920 align:middle line:84% are classes within this data set, what the differences are. 00:28:14.920 --> 00:28:17.710 align:middle line:84% There's a variety of computational tools 00:28:17.710 --> 00:28:22.180 align:middle line:84% that are used, and these include cluster analysis, 00:28:22.180 --> 00:28:27.190 align:middle line:84% the so-called supervised learning, fold analysis. 00:28:27.190 --> 00:28:30.280 align:middle line:84% But I'm focused more on not the computational tool that's 00:28:30.280 --> 00:28:33.040 align:middle line:84% being used, whether we're doing unsupervised or supervised 00:28:33.040 --> 00:28:36.040 align:middle line:84% classification, but the broader concept about what 00:28:36.040 --> 00:28:37.970 align:middle line:90% we're trying to do. 00:28:37.970 --> 00:28:39.640 align:middle line:84% And so one group of applications, 00:28:39.640 --> 00:28:44.890 align:middle line:84% we're just trying to explore the data, and its class structure. 00:28:44.890 --> 00:28:47.440 align:middle line:90% And that's called comparison. 00:28:47.440 --> 00:28:52.000 align:middle line:84% Prediction is a whole different area. 00:28:52.000 --> 00:28:57.010 align:middle line:84% And prediction is a method by which 00:28:57.010 --> 00:29:01.810 align:middle line:84% we want to predict the class of a new sample that's given 00:29:01.810 --> 00:29:04.870 align:middle line:90% to us based on our prior data. 00:29:04.870 --> 00:29:09.610 align:middle line:84% So that involves using data that we have 00:29:09.610 --> 00:29:14.020 align:middle line:84% and constructing some function, essentially, 00:29:14.020 --> 00:29:18.310 align:middle line:84% or some method based on that data set so that if we 00:29:18.310 --> 00:29:22.510 align:middle line:84% encounter a new sample, that we can then predict something 00:29:22.510 --> 00:29:29.860 align:middle line:84% about that new sample based on the prior data that we had. 00:29:29.860 --> 00:29:32.980 align:middle line:84% That's prediction-- very different 00:29:32.980 --> 00:29:37.400 align:middle line:84% than class comparison, although some of them 00:29:37.400 --> 00:29:38.900 align:middle line:90% use similar methods. 00:29:38.900 --> 00:29:43.760 align:middle line:84% So the tools that might be used to do class prediction 00:29:43.760 --> 00:29:51.110 align:middle line:84% can also be used to do class comparison, some of them. 00:29:51.110 --> 00:29:54.890 align:middle line:84% Class discovery is yet a third application. 00:29:54.890 --> 00:29:58.460 align:middle line:84% And class discovery is focused on discovering new disease 00:29:58.460 --> 00:29:59.870 align:middle line:90% subtypes. 00:29:59.870 --> 00:30:03.710 align:middle line:84% So it's most analogous to what I started 00:30:03.710 --> 00:30:07.250 align:middle line:84% to talk with in terms of looking at things we learned when 00:30:07.250 --> 00:30:09.260 align:middle line:84% we looked under the microscope, initially, 00:30:09.260 --> 00:30:11.090 align:middle line:90% about muscle disease. 00:30:11.090 --> 00:30:14.690 align:middle line:84% We learned that there are different types of diseases 00:30:14.690 --> 00:30:18.230 align:middle line:84% that truly are subtypes of muscle disease 00:30:18.230 --> 00:30:21.330 align:middle line:84% that we didn't know about beforehand. 00:30:21.330 --> 00:30:23.390 align:middle line:84% And so class discovery is focused 00:30:23.390 --> 00:30:28.040 align:middle line:84% on looking at microarray data from tissue samples 00:30:28.040 --> 00:30:30.440 align:middle line:84% and trying to understand if there might 00:30:30.440 --> 00:30:35.420 align:middle line:84% be some subtype that's defined by the microarray data 00:30:35.420 --> 00:30:38.270 align:middle line:84% that we weren't aware of beforehand. 00:30:38.270 --> 00:30:41.200 align:middle line:90% 00:30:41.200 --> 00:30:44.870 align:middle line:84% Such a subtype has to have some meaning to it. 00:30:44.870 --> 00:30:48.640 align:middle line:84% It can't just be that we found, looking 00:30:48.640 --> 00:30:51.730 align:middle line:84% at these very undetermined data sets, 00:30:51.730 --> 00:30:56.170 align:middle line:84% that we found different classes of these vectors. 00:30:56.170 --> 00:30:58.430 align:middle line:84% It has to translate to something meaningful. 00:30:58.430 --> 00:31:01.000 align:middle line:84% So one has to define classes and then 00:31:01.000 --> 00:31:05.440 align:middle line:84% go back and say whether this class actually means something, 00:31:05.440 --> 00:31:07.690 align:middle line:84% whether these patients who fell into this class 00:31:07.690 --> 00:31:10.000 align:middle line:84% are really different than the other patients, 00:31:10.000 --> 00:31:14.420 align:middle line:84% and people just never noticed that before, all right? 00:31:14.420 --> 00:31:17.060 align:middle line:90% 00:31:17.060 --> 00:31:19.580 align:middle line:84% So going through those, class comparison, 00:31:19.580 --> 00:31:23.150 align:middle line:84% again, is used to establish distinctness of classes 00:31:23.150 --> 00:31:25.520 align:middle line:90% and just compare and contrast. 00:31:25.520 --> 00:31:30.110 align:middle line:84% There's no direct medical applications of this tool. 00:31:30.110 --> 00:31:32.090 align:middle line:90% There are indirect applications. 00:31:32.090 --> 00:31:35.900 align:middle line:84% It can be useful for generating ideas about classification 00:31:35.900 --> 00:31:37.850 align:middle line:90% and pathophysiology. 00:31:37.850 --> 00:31:40.610 align:middle line:84% And there's a variety of computational tools 00:31:40.610 --> 00:31:44.995 align:middle line:84% that can be used to compare classes here. 00:31:44.995 --> 00:31:51.050 align:middle line:90% 00:31:51.050 --> 00:31:53.660 align:middle line:84% This is an example of class comparison 00:31:53.660 --> 00:31:57.110 align:middle line:84% where we looked at 45 muscle biopsy tissue specimens 00:31:57.110 --> 00:31:58.970 align:middle line:90% from four different groups. 00:31:58.970 --> 00:32:00.260 align:middle line:90% We had normals. 00:32:00.260 --> 00:32:03.140 align:middle line:84% We had patients with Duchenne muscular dystrophy, 00:32:03.140 --> 00:32:05.150 align:middle line:84% nemaline myopathies, and then a group 00:32:05.150 --> 00:32:08.180 align:middle line:84% of inflammatory myopathies, of which there's 00:32:08.180 --> 00:32:11.510 align:middle line:90% a variety of subtypes. 00:32:11.510 --> 00:32:14.810 align:middle line:84% And one of the tools for class comparison 00:32:14.810 --> 00:32:17.720 align:middle line:84% is hierarchical clustering, which we did. 00:32:17.720 --> 00:32:20.210 align:middle line:84% And when we used hierarchical clustering applied 00:32:20.210 --> 00:32:25.010 align:middle line:84% to this data set using one of these color schemes, 00:32:25.010 --> 00:32:28.250 align:middle line:84% this is the raw data here in this column. 00:32:28.250 --> 00:32:32.760 align:middle line:84% And again, this is 45 columns here. 00:32:32.760 --> 00:32:35.690 align:middle line:90% And this is just a portion. 00:32:35.690 --> 00:32:38.360 align:middle line:84% Maybe this is 3,000 or 4,000 genes that 00:32:38.360 --> 00:32:40.460 align:middle line:90% are represented in rows here. 00:32:40.460 --> 00:32:45.650 align:middle line:84% And you just look at it, and you can see streaks of green or red 00:32:45.650 --> 00:32:51.800 align:middle line:84% going down, suggesting that there are certain classes here, 00:32:51.800 --> 00:32:56.990 align:middle line:84% that certain tissues seem to have, say, low expression 00:32:56.990 --> 00:33:00.200 align:middle line:84% levels for these genes and high expression 00:33:00.200 --> 00:33:02.580 align:middle line:90% levels for these genes. 00:33:02.580 --> 00:33:08.630 align:middle line:84% This, here, is a blow-up diagram of this particular section 00:33:08.630 --> 00:33:09.650 align:middle line:90% here. 00:33:09.650 --> 00:33:10.820 align:middle line:90% And it has the labels. 00:33:10.820 --> 00:33:16.010 align:middle line:84% And so the hierarchical tree here has four branches, 00:33:16.010 --> 00:33:18.320 align:middle line:90% essentially, here. 00:33:18.320 --> 00:33:20.660 align:middle line:84% This branch is all of the normal specimens. 00:33:20.660 --> 00:33:25.790 align:middle line:84% So all the normal specimens got classified correctly here. 00:33:25.790 --> 00:33:28.100 align:middle line:84% These are all the inflammatory myopathies. 00:33:28.100 --> 00:33:31.730 align:middle line:84% These are mostly patients from a third group, nemaline. 00:33:31.730 --> 00:33:34.190 align:middle line:84% And these are mostly patients with Duchenne 00:33:34.190 --> 00:33:37.340 align:middle line:90% muscular dystrophy. 00:33:37.340 --> 00:33:39.290 align:middle line:90% But this is class comparison. 00:33:39.290 --> 00:33:43.010 align:middle line:84% We just applied hierarchical clustering to this data set 00:33:43.010 --> 00:33:47.300 align:middle line:84% and found classes here that we're 00:33:47.300 --> 00:33:50.240 align:middle line:90% interested in just looking at. 00:33:50.240 --> 00:33:51.770 align:middle line:84% We knew about these classes already. 00:33:51.770 --> 00:33:56.866 align:middle line:90% 00:33:56.866 --> 00:33:57.780 align:middle line:90% Yeah? 00:33:57.780 --> 00:33:59.322 align:middle line:84% AUDIENCE: The reason that this is not 00:33:59.322 --> 00:34:03.090 align:middle line:84% a direct point of application is because you're not 00:34:03.090 --> 00:34:05.040 align:middle line:90% going in and trying to-- 00:34:05.040 --> 00:34:11.030 align:middle line:84% I'm trying to make connections between these different 00:34:11.030 --> 00:34:14.409 align:middle line:84% patterns that have allowed you to classify these or make 00:34:14.409 --> 00:34:15.280 align:middle line:90% these different-- 00:34:15.280 --> 00:34:17.980 align:middle line:84% STEVEN A. GREENBERG: Well, it has indirect applications. 00:34:17.980 --> 00:34:20.270 align:middle line:84% But by direct, I mean, very direct. 00:34:20.270 --> 00:34:23.320 align:middle line:84% So that if I have a new patient, and I 00:34:23.320 --> 00:34:25.570 align:middle line:84% do a microarray experiment on it, 00:34:25.570 --> 00:34:33.400 align:middle line:84% can I make any predictions of that based on this approach 00:34:33.400 --> 00:34:34.900 align:middle line:90% that I've used so far? 00:34:34.900 --> 00:34:40.929 align:middle line:84% Or have I discovered any new classes of disease 00:34:40.929 --> 00:34:43.870 align:middle line:90% here amongst these groups? 00:34:43.870 --> 00:34:46.900 align:middle line:84% Those would be fairly direct applications. 00:34:46.900 --> 00:34:50.139 align:middle line:84% There are certainly indirect applications here. 00:34:50.139 --> 00:34:51.699 align:middle line:84% So there are things that come up. 00:34:51.699 --> 00:34:57.040 align:middle line:84% Like, the only misclassification for the inflammatory myopathies 00:34:57.040 --> 00:35:01.120 align:middle line:84% was this one, one patient with an inflammatory myopathy. 00:35:01.120 --> 00:35:04.510 align:middle line:84% This one, who had this disease called IBM, 00:35:04.510 --> 00:35:09.800 align:middle line:84% got classified in this group instead of this group. 00:35:09.800 --> 00:35:14.380 align:middle line:84% And it turns out that this one was really different than all 00:35:14.380 --> 00:35:16.240 align:middle line:84% of the others and that it was a patient who 00:35:16.240 --> 00:35:20.140 align:middle line:90% had a familial form of IBM. 00:35:20.140 --> 00:35:22.728 align:middle line:84% We don't know what genes are involved, if any. 00:35:22.728 --> 00:35:24.020 align:middle line:90% And we don't know what this is. 00:35:24.020 --> 00:35:27.310 align:middle line:84% It's very rare for familial instances of IBM. 00:35:27.310 --> 00:35:30.850 align:middle line:84% But this particular patient had familial IBM. 00:35:30.850 --> 00:35:33.220 align:middle line:84% So this might tell me that there really 00:35:33.220 --> 00:35:36.910 align:middle line:84% is something different about that disease 00:35:36.910 --> 00:35:39.880 align:middle line:84% because under the microscope, we don't 00:35:39.880 --> 00:35:41.890 align:middle line:84% see any difference in that patient 00:35:41.890 --> 00:35:45.070 align:middle line:84% compared to the other patients with inflammatory myopathies. 00:35:45.070 --> 00:35:49.960 align:middle line:84% So it helps you generate hypotheses and ideas. 00:35:49.960 --> 00:35:52.090 align:middle line:84% AUDIENCE: I guess, I'm just confused because, 00:35:52.090 --> 00:35:54.610 align:middle line:84% in this particular example that you've given us, 00:35:54.610 --> 00:35:59.480 align:middle line:84% it started as something that you knew [INAUDIBLE] your data set. 00:35:59.480 --> 00:36:03.370 align:middle line:84% So there was no new identification of the disease-- 00:36:03.370 --> 00:36:05.930 align:middle line:84% STEVEN A. GREENBERG: Correct, right. 00:36:05.930 --> 00:36:12.910 align:middle line:84% AUDIENCE: But given a sample of tissue samples 00:36:12.910 --> 00:36:15.430 align:middle line:84% that were from patients that had diseases that affected 00:36:15.430 --> 00:36:20.440 align:middle line:84% something else, you didn't know about this [INAUDIBLE],, 00:36:20.440 --> 00:36:27.020 align:middle line:84% it seems like could have this classification of new disease. 00:36:27.020 --> 00:36:29.270 align:middle line:84% STEVEN A. GREENBERG: You can use this tool to do that, 00:36:29.270 --> 00:36:32.230 align:middle line:84% but you have to add in a few other things. 00:36:32.230 --> 00:36:36.070 align:middle line:84% Yeah, so I mean, this tool of hierarchical clustering 00:36:36.070 --> 00:36:41.170 align:middle line:84% does cut across the three different major class methods-- 00:36:41.170 --> 00:36:45.640 align:middle line:84% exploration, prediction, and discovery. 00:36:45.640 --> 00:36:48.590 align:middle line:90% But you have to add more to it. 00:36:48.590 --> 00:36:51.040 align:middle line:84% So I didn't mean to confuse you and say that way. 00:36:51.040 --> 00:36:55.690 align:middle line:84% But you do want to keep the tools separate 00:36:55.690 --> 00:37:02.800 align:middle line:84% from the method, the overall approach that you're doing. 00:37:02.800 --> 00:37:05.470 align:middle line:84% If you're just exploring data, there's a bunch of tools. 00:37:05.470 --> 00:37:08.080 align:middle line:84% If you're trying to make predictions, 00:37:08.080 --> 00:37:13.390 align:middle line:84% diagnostic or prognostic predictions of new patients, 00:37:13.390 --> 00:37:16.780 align:middle line:84% it's an overlapping set of tools but a totally different 00:37:16.780 --> 00:37:18.610 align:middle line:84% approach or a different procedure. 00:37:18.610 --> 00:37:26.070 align:middle line:90% 00:37:26.070 --> 00:37:29.880 align:middle line:90% So yeah, let's skip this one. 00:37:29.880 --> 00:37:30.990 align:middle line:90% And that one too. 00:37:30.990 --> 00:37:35.970 align:middle line:84% OK, so class discovery, then, is the next area of those three 00:37:35.970 --> 00:37:39.270 align:middle line:90% methods that I outlined. 00:37:39.270 --> 00:37:41.190 align:middle line:84% And this one uses expression data 00:37:41.190 --> 00:37:44.760 align:middle line:84% to discover previously unrecognized but clinically 00:37:44.760 --> 00:37:48.510 align:middle line:90% relevant disease subtypes. 00:37:48.510 --> 00:37:53.610 align:middle line:84% It also doesn't have direct medical applications. 00:37:53.610 --> 00:37:56.460 align:middle line:84% And by direct, I mean things that would immediately 00:37:56.460 --> 00:38:02.160 align:middle line:84% be applicable to patients-- and I 00:38:02.160 --> 00:38:06.300 align:middle line:84% mean, patients who you might see in the office. 00:38:06.300 --> 00:38:08.850 align:middle line:84% And maybe I should revise that one here. 00:38:08.850 --> 00:38:10.530 align:middle line:84% I haven't over the last year, but maybe. 00:38:10.530 --> 00:38:13.860 align:middle line:84% But certainly, indirect, it does advance the field 00:38:13.860 --> 00:38:17.850 align:middle line:84% because it improves our ability to recognize diagnostic 00:38:17.850 --> 00:38:20.520 align:middle line:84% and potentially prognostic subtypes 00:38:20.520 --> 00:38:23.130 align:middle line:84% and potentially treat patients differently 00:38:23.130 --> 00:38:27.480 align:middle line:84% based on this subtype of disease that they may have that we 00:38:27.480 --> 00:38:28.770 align:middle line:90% didn't previously know about. 00:38:28.770 --> 00:38:33.250 align:middle line:90% 00:38:33.250 --> 00:38:36.420 align:middle line:84% And so the basic method of class discovery 00:38:36.420 --> 00:38:38.670 align:middle line:84% often starts with cluster analysis 00:38:38.670 --> 00:38:40.680 align:middle line:90% to try and define classes. 00:38:40.680 --> 00:38:43.530 align:middle line:84% And it can be any hierarchical or K 00:38:43.530 --> 00:38:45.900 align:middle line:84% means-- any variety of approaches 00:38:45.900 --> 00:38:48.510 align:middle line:90% to defining classes here. 00:38:48.510 --> 00:38:53.520 align:middle line:84% But after that, one explores the phenotypic variations 00:38:53.520 --> 00:38:58.500 align:middle line:84% within the defined classes for a number of different types 00:38:58.500 --> 00:39:00.060 align:middle line:90% of phenotypic variables. 00:39:00.060 --> 00:39:04.690 align:middle line:84% So you take 50 patients with, let's say, 00:39:04.690 --> 00:39:07.770 align:middle line:84% a type of cancer, who you think all 00:39:07.770 --> 00:39:09.120 align:middle line:90% have the same type of cancer. 00:39:09.120 --> 00:39:12.210 align:middle line:90% 00:39:12.210 --> 00:39:15.330 align:middle line:84% Our best diagnostic classification system now 00:39:15.330 --> 00:39:20.100 align:middle line:84% of looking at history, and physical exam, and doing tests 00:39:20.100 --> 00:39:24.330 align:middle line:84% says that all these patients are indistinguishable. 00:39:24.330 --> 00:39:27.540 align:middle line:84% And you do microarray experiments, classify the data, 00:39:27.540 --> 00:39:29.970 align:middle line:84% and then you'll will certainly get out 00:39:29.970 --> 00:39:34.265 align:middle line:84% classes, whether they're real or not or meaningful, 00:39:34.265 --> 00:39:34.890 align:middle line:90% doesn't matter. 00:39:34.890 --> 00:39:37.200 align:middle line:84% But the algorithms are guaranteed 00:39:37.200 --> 00:39:39.700 align:middle line:90% to spit out classes for you. 00:39:39.700 --> 00:39:42.340 align:middle line:84% And if you then want to go back and ask, 00:39:42.340 --> 00:39:45.120 align:middle line:84% well, is there something different about this class 00:39:45.120 --> 00:39:49.060 align:middle line:84% compared to that class that's clinically meaningful? 00:39:49.060 --> 00:39:52.350 align:middle line:84% And one thing might be time to some endpoint, 00:39:52.350 --> 00:39:54.270 align:middle line:90% such as survival. 00:39:54.270 --> 00:39:58.050 align:middle line:84% Do patients in one class, based on their expression profiles, 00:39:58.050 --> 00:40:02.760 align:middle line:84% live longer than patients in a different class? 00:40:02.760 --> 00:40:05.130 align:middle line:90% Or any other endpoint-- 00:40:05.130 --> 00:40:07.505 align:middle line:84% do they not respond to treatment? 00:40:07.505 --> 00:40:08.880 align:middle line:84% So I go back and I see there were 00:40:08.880 --> 00:40:14.530 align:middle line:84% 20 patients in this class and 10 in another class and say, 00:40:14.530 --> 00:40:16.020 align:middle line:84% how did the patients in this class 00:40:16.020 --> 00:40:21.510 align:middle line:84% respond to treatment compared to the patients in this class? 00:40:21.510 --> 00:40:24.240 align:middle line:84% And this is actually a fairly powerful method 00:40:24.240 --> 00:40:34.470 align:middle line:84% to discover new disease subtypes that we don't know about. 00:40:34.470 --> 00:40:36.735 align:middle line:84% Yeah, I think it's an excellent method to use. 00:40:36.735 --> 00:40:40.890 align:middle line:90% 00:40:40.890 --> 00:40:43.800 align:middle line:84% It certainly has been successful in a variety 00:40:43.800 --> 00:40:47.970 align:middle line:90% of areas and cancer, for sure. 00:40:47.970 --> 00:40:50.010 align:middle line:84% Its application to other diseases 00:40:50.010 --> 00:40:52.560 align:middle line:84% is lagging behind that of cancer. 00:40:52.560 --> 00:40:59.010 align:middle line:90% 00:40:59.010 --> 00:41:02.220 align:middle line:84% There's a few examples where it's been done. 00:41:02.220 --> 00:41:06.510 align:middle line:84% I think I was planning on going through a couple. 00:41:06.510 --> 00:41:10.230 align:middle line:90% I don't if we need to. 00:41:10.230 --> 00:41:12.250 align:middle line:84% Do want me to go through an example? 00:41:12.250 --> 00:41:15.720 align:middle line:84% This is a lymphoma paper as an example. 00:41:15.720 --> 00:41:17.370 align:middle line:90% OK. 00:41:17.370 --> 00:41:20.760 align:middle line:84% In this paper, the investigators had 00:41:20.760 --> 00:41:27.222 align:middle line:84% 96 patients who they knew about nine classes ahead of time. 00:41:27.222 --> 00:41:28.680 align:middle line:84% And these are all patients who have 00:41:28.680 --> 00:41:31.140 align:middle line:90% different types of lymphoma-- 00:41:31.140 --> 00:41:36.570 align:middle line:90% 1, 2, 3, 4, 5, 6, 7, 8, 9. 00:41:36.570 --> 00:41:43.230 align:middle line:84% Actually, there's eight lymphomas in one normal class 00:41:43.230 --> 00:41:44.370 align:middle line:90% here. 00:41:44.370 --> 00:41:48.540 align:middle line:84% And they used lymph node tissue from patients who have lymphoma 00:41:48.540 --> 00:41:50.610 align:middle line:90% from some normals. 00:41:50.610 --> 00:41:51.950 align:middle line:90% They did microarray experiments. 00:41:51.950 --> 00:41:56.040 align:middle line:84% So they did 96 microarray experiments, 00:41:56.040 --> 00:41:59.520 align:middle line:84% measured something like 4,000 or so genes 00:41:59.520 --> 00:42:04.560 align:middle line:84% for each of 96 patients, and did clustering 00:42:04.560 --> 00:42:07.950 align:middle line:84% of the data, hierarchical clustering, 00:42:07.950 --> 00:42:11.610 align:middle line:84% and then for purposes of presentation, colored things 00:42:11.610 --> 00:42:12.880 align:middle line:90% in for us here. 00:42:12.880 --> 00:42:17.130 align:middle line:84% So this is the dendrogram structure that they obtained. 00:42:17.130 --> 00:42:19.050 align:middle line:84% And they've nicely colored in the picture 00:42:19.050 --> 00:42:24.630 align:middle line:84% so that all patients who had one type of lymphoma, CLL, 00:42:24.630 --> 00:42:25.470 align:middle line:90% are yellow. 00:42:25.470 --> 00:42:28.230 align:middle line:90% That's this group of patients. 00:42:28.230 --> 00:42:32.160 align:middle line:84% And the patients with this type of lymphoma, 00:42:32.160 --> 00:42:37.740 align:middle line:84% diffuse large B-cell lymphoma, are coded in purples. 00:42:37.740 --> 00:42:40.840 align:middle line:84% I think one or two-- one of them ended up out here. 00:42:40.840 --> 00:42:42.010 align:middle line:90% One is here. 00:42:42.010 --> 00:42:43.170 align:middle line:90% One is here. 00:42:43.170 --> 00:42:47.160 align:middle line:84% Most of them are in this section of the tree with a couple 00:42:47.160 --> 00:42:50.860 align:middle line:90% of other diseases mixed in. 00:42:50.860 --> 00:42:54.870 align:middle line:84% So at this point, what would you call this 00:42:54.870 --> 00:42:58.740 align:middle line:84% at this point of the class methods? 00:42:58.740 --> 00:43:01.140 align:middle line:84% Exploration, discovery, or prediction? 00:43:01.140 --> 00:43:03.780 align:middle line:90% 00:43:03.780 --> 00:43:04.770 align:middle line:90% Comparison, right. 00:43:04.770 --> 00:43:08.610 align:middle line:84% So this is class comparison they're just looking at here. 00:43:08.610 --> 00:43:14.730 align:middle line:84% But they go on, and they focus on this structure 00:43:14.730 --> 00:43:16.660 align:middle line:90% here for this disease. 00:43:16.660 --> 00:43:20.700 align:middle line:84% So looking at the patients with diffuse large B-cell lymphoma, 00:43:20.700 --> 00:43:26.820 align:middle line:84% they noticed two different branches at this level. 00:43:26.820 --> 00:43:29.640 align:middle line:84% So there's this group of patients starting 00:43:29.640 --> 00:43:33.000 align:middle line:90% from this one here out to here. 00:43:33.000 --> 00:43:37.440 align:middle line:84% And then there's another group starting from here out to here, 00:43:37.440 --> 00:43:39.540 align:middle line:90% OK? 00:43:39.540 --> 00:43:42.450 align:middle line:84% And so then they did class discovery. 00:43:42.450 --> 00:43:46.770 align:middle line:84% So they asked, that's curious, why do I have 00:43:46.770 --> 00:43:48.720 align:middle line:90% two different branches here? 00:43:48.720 --> 00:43:52.050 align:middle line:84% And it may just be an artifact of the algorithm. 00:43:52.050 --> 00:43:54.000 align:middle line:84% In fact, again, hierarchical clustering 00:43:54.000 --> 00:43:56.430 align:middle line:90% doesn't really give you classes. 00:43:56.430 --> 00:43:59.910 align:middle line:84% Everything is continually divided into a binary tree 00:43:59.910 --> 00:44:01.950 align:middle line:90% structure. 00:44:01.950 --> 00:44:05.070 align:middle line:84% And so it's always going to split things down 00:44:05.070 --> 00:44:07.470 align:middle line:84% and you have to decide what level 00:44:07.470 --> 00:44:09.240 align:middle line:90% you want to look at similarity. 00:44:09.240 --> 00:44:10.960 align:middle line:90% But they ask the question. 00:44:10.960 --> 00:44:12.910 align:middle line:84% So they then went and said, well, 00:44:12.910 --> 00:44:16.200 align:middle line:84% what about survival of these two groups? 00:44:16.200 --> 00:44:19.500 align:middle line:84% Do the patients in this group here 00:44:19.500 --> 00:44:23.370 align:middle line:84% have a different survival than patients in this group? 00:44:23.370 --> 00:44:27.660 align:middle line:90% 00:44:27.660 --> 00:44:32.840 align:middle line:84% And so let's make sure that I have that. 00:44:32.840 --> 00:44:37.000 align:middle line:90% 00:44:37.000 --> 00:44:43.630 align:middle line:84% So what they show here is that the diffuse-- 00:44:43.630 --> 00:44:50.470 align:middle line:90% so I've lost the indicator. 00:44:50.470 --> 00:44:51.490 align:middle line:90% Hang on one second. 00:44:51.490 --> 00:44:54.220 align:middle line:84% Let me make sure that I know what I'm talking about. 00:44:54.220 --> 00:44:55.210 align:middle line:90% Yeah, that's it. 00:44:55.210 --> 00:44:57.430 align:middle line:90% Yeah, right. 00:44:57.430 --> 00:45:05.410 align:middle line:84% So they found, if you looked at survival based-- 00:45:05.410 --> 00:45:08.620 align:middle line:84% so let's ignore this one on the left and look at these two 00:45:08.620 --> 00:45:09.790 align:middle line:90% here. 00:45:09.790 --> 00:45:13.690 align:middle line:84% So survival and decisions about chemotherapy 00:45:13.690 --> 00:45:16.420 align:middle line:84% are based on largely on something 00:45:16.420 --> 00:45:18.700 align:middle line:84% called the International prognostic index 00:45:18.700 --> 00:45:20.150 align:middle line:90% in this disease. 00:45:20.150 --> 00:45:22.930 align:middle line:84% And that's a measure of clinical risk. 00:45:22.930 --> 00:45:25.450 align:middle line:90% 00:45:25.450 --> 00:45:27.903 align:middle line:90% And they found that-- 00:45:27.903 --> 00:45:29.320 align:middle line:84% actually, let's do this one first. 00:45:29.320 --> 00:45:33.460 align:middle line:84% So they did look at survival for the two groups 00:45:33.460 --> 00:45:38.200 align:middle line:84% and found that they did have very different survival here, 00:45:38.200 --> 00:45:45.580 align:middle line:84% that, one group had much better survival expectancy 00:45:45.580 --> 00:45:46.455 align:middle line:90% than the other group. 00:45:46.455 --> 00:45:49.200 align:middle line:90% 00:45:49.200 --> 00:45:54.600 align:middle line:84% They then further looked at the International prognostic index 00:45:54.600 --> 00:45:56.850 align:middle line:90% and how it grouped patients. 00:45:56.850 --> 00:45:59.010 align:middle line:84% And according to that scheme, there 00:45:59.010 --> 00:46:02.310 align:middle line:84% is a group of low and high clinical risk patients. 00:46:02.310 --> 00:46:05.970 align:middle line:84% And the high ones will generally get more aggressive treatment 00:46:05.970 --> 00:46:07.890 align:middle line:90% than the low ones. 00:46:07.890 --> 00:46:10.860 align:middle line:84% And so they looked at this low-risk group 00:46:10.860 --> 00:46:14.820 align:middle line:84% according to their microarray patterns 00:46:14.820 --> 00:46:19.320 align:middle line:84% and asked, how did they fall into these two different groups 00:46:19.320 --> 00:46:20.610 align:middle line:90% that we found? 00:46:20.610 --> 00:46:23.940 align:middle line:84% And they found that a number of those patients-- 00:46:23.940 --> 00:46:30.390 align:middle line:84% 14 of them-- were in the Better prognostic group of the two 00:46:30.390 --> 00:46:31.290 align:middle line:90% they had defined. 00:46:31.290 --> 00:46:34.450 align:middle line:84% And another 10 were in the worst. 00:46:34.450 --> 00:46:36.810 align:middle line:84% And so they were able to refine and say 00:46:36.810 --> 00:46:40.680 align:middle line:84% that this group that's previously 00:46:40.680 --> 00:46:43.140 align:middle line:84% been called low clinical risk based 00:46:43.140 --> 00:46:45.360 align:middle line:84% on the international prognostic index 00:46:45.360 --> 00:46:48.990 align:middle line:84% actually has two subgroups within it. 00:46:48.990 --> 00:46:51.600 align:middle line:84% And one subgroup has a pretty good prognosis, 00:46:51.600 --> 00:46:55.230 align:middle line:84% and the other has quite a bit more aggressive disease. 00:46:55.230 --> 00:46:58.710 align:middle line:84% And they averaged out to this, right here. 00:46:58.710 --> 00:47:02.010 align:middle line:84% But the microarray data suggests that these patients are 00:47:02.010 --> 00:47:04.350 align:middle line:84% at greater risk and should probably 00:47:04.350 --> 00:47:09.480 align:middle line:84% be treated more aggressively based on their microarray data 00:47:09.480 --> 00:47:11.190 align:middle line:90% generate expression profile. 00:47:11.190 --> 00:47:14.050 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:47:14.050 --> 00:47:18.640 align:middle line:84% STEVEN A. GREENBERG: That takes age, and gender, and stage 00:47:18.640 --> 00:47:22.060 align:middle line:84% of disease at presentation, whether it's just in the lymph 00:47:22.060 --> 00:47:25.150 align:middle line:90% node, whether it's metastatic. 00:47:25.150 --> 00:47:27.730 align:middle line:84% There's a few other variables in there. 00:47:27.730 --> 00:47:29.685 align:middle line:84% And I'm not an oncologist, so I'm not sure. 00:47:29.685 --> 00:47:31.060 align:middle line:84% But there are clinical variables. 00:47:31.060 --> 00:47:35.770 align:middle line:84% There are patterns under the microscope as well. 00:47:35.770 --> 00:47:39.910 align:middle line:84% But it's the best that medicine has. 00:47:39.910 --> 00:47:44.020 align:middle line:84% And the microarray work suggests that we could do better. 00:47:44.020 --> 00:47:45.640 align:middle line:84% And that's not an unreasonable thing 00:47:45.640 --> 00:47:49.870 align:middle line:84% to suspect in the first place, particularly in cancer, 00:47:49.870 --> 00:47:53.500 align:middle line:84% where, if we can generate 20,000 numbers 00:47:53.500 --> 00:47:56.005 align:middle line:90% that represent physiology-- 00:47:56.005 --> 00:47:57.880 align:middle line:84% and that's the other thing is, the microscope 00:47:57.880 --> 00:48:01.150 align:middle line:90% really doesn't show physiology. 00:48:01.150 --> 00:48:03.130 align:middle line:90% It shows anatomy there. 00:48:03.130 --> 00:48:05.065 align:middle line:84% The microarray is-- and that's what 00:48:05.065 --> 00:48:06.440 align:middle line:84% the quote said at the beginning-- 00:48:06.440 --> 00:48:11.860 align:middle line:84% a physiological microscope looking at living processes. 00:48:11.860 --> 00:48:13.050 align:middle line:90% Yes? 00:48:13.050 --> 00:48:14.300 align:middle line:90% AUDIENCE: I've got a question. 00:48:14.300 --> 00:48:17.340 align:middle line:90% 00:48:17.340 --> 00:48:20.770 align:middle line:84% I get the impression that there are a couple of marks there. 00:48:20.770 --> 00:48:23.490 align:middle line:84% When I look at the y-axis and I see probability, 00:48:23.490 --> 00:48:26.108 align:middle line:84% could you comment on what that measure actually is? 00:48:26.108 --> 00:48:27.900 align:middle line:84% STEVEN A. GREENBERG: That's the probability 00:48:27.900 --> 00:48:32.740 align:middle line:84% of being alive at a given year, at a given period. 00:48:32.740 --> 00:48:39.420 align:middle line:84% So at four years on the x-axis, you have a 20% probability. 00:48:39.420 --> 00:48:41.370 align:middle line:84% Oh, no-- and this curve being alive, yeah. 00:48:41.370 --> 00:48:46.690 align:middle line:84% These are actually censored data here. 00:48:46.690 --> 00:48:49.530 align:middle line:84% And that's a complex thing about data representation. 00:48:49.530 --> 00:48:55.170 align:middle line:84% I didn't want to get into that, except to the point to take out 00:48:55.170 --> 00:48:57.360 align:middle line:84% of this, I think, is that this has 00:48:57.360 --> 00:49:02.430 align:middle line:84% enormous potential for disease discovery 00:49:02.430 --> 00:49:07.020 align:middle line:90% of subtypes, disease subtypes. 00:49:07.020 --> 00:49:08.910 align:middle line:84% And again, particularly in cancer, 00:49:08.910 --> 00:49:11.700 align:middle line:84% where the physiology of these tumors, what genes are 00:49:11.700 --> 00:49:16.810 align:middle line:84% turned on is probably very much related to the disease course. 00:49:16.810 --> 00:49:19.180 align:middle line:84% Do these have metastatic potential? 00:49:19.180 --> 00:49:21.810 align:middle line:84% Are they growing rapidly or slowly 00:49:21.810 --> 00:49:24.720 align:middle line:84% based on the expression of genes in the tumor? 00:49:24.720 --> 00:49:32.270 align:middle line:90% 00:49:32.270 --> 00:49:35.975 align:middle line:84% Yeah, that was the point about adding to the IPI. 00:49:35.975 --> 00:49:38.570 align:middle line:84% OK, I won't do another example of class discovery. 00:49:38.570 --> 00:49:42.470 align:middle line:84% There's plenty out there right now 00:49:42.470 --> 00:49:45.520 align:middle line:90% in a variety of different areas. 00:49:45.520 --> 00:49:48.090 align:middle line:90% OK. 00:49:48.090 --> 00:49:50.560 align:middle line:90% And there's class prediction. 00:49:50.560 --> 00:49:53.820 align:middle line:84% So class prediction is a method of using expression data 00:49:53.820 --> 00:49:57.000 align:middle line:84% to build a model that will then predict 00:49:57.000 --> 00:50:02.820 align:middle line:84% class assignment of a new sample presented to that model. 00:50:02.820 --> 00:50:07.260 align:middle line:84% This one has potential very direct medical applications. 00:50:07.260 --> 00:50:09.930 align:middle line:84% If one can build such a model for breast cancer, 00:50:09.930 --> 00:50:12.330 align:middle line:84% then when you see a new patient with breast cancer 00:50:12.330 --> 00:50:15.120 align:middle line:84% and you do a microarray experiment on the cancer 00:50:15.120 --> 00:50:19.110 align:middle line:84% tissue, and apply it to the model, and the model 00:50:19.110 --> 00:50:21.240 align:middle line:90% spits out some prediction-- 00:50:21.240 --> 00:50:26.130 align:middle line:84% high risk of death within the next year. 00:50:26.130 --> 00:50:28.560 align:middle line:84% That will help you to decide how to treat the patient, 00:50:28.560 --> 00:50:31.810 align:middle line:90% if it's accurate. 00:50:31.810 --> 00:50:34.090 align:middle line:84% This can be used to establish a diagnosis, 00:50:34.090 --> 00:50:37.150 align:middle line:84% to make predictions of outcome is one of them. 00:50:37.150 --> 00:50:39.640 align:middle line:84% But it could be things like predicting response 00:50:39.640 --> 00:50:40.900 align:middle line:90% to a medication. 00:50:40.900 --> 00:50:42.880 align:middle line:84% This patient is more likely to respond 00:50:42.880 --> 00:50:46.210 align:middle line:84% to this medication as opposed to that one, based 00:50:46.210 --> 00:50:49.600 align:middle line:90% on their expression profile. 00:50:49.600 --> 00:50:52.900 align:middle line:84% Indirectly, it does tell us about disease pathophysiology, 00:50:52.900 --> 00:50:56.000 align:middle line:90% to some extent. 00:50:56.000 --> 00:51:01.030 align:middle line:84% So this method is a bit more involved, 00:51:01.030 --> 00:51:05.270 align:middle line:84% but it's pretty standard in the way people do this now. 00:51:05.270 --> 00:51:07.630 align:middle line:84% And so the approach is to take a data 00:51:07.630 --> 00:51:10.180 align:middle line:90% set that's sufficiently large-- 00:51:10.180 --> 00:51:14.620 align:middle line:84% let's say, 100 patients with breast cancer-- 00:51:14.620 --> 00:51:18.160 align:middle line:84% microarray experiments on each of the 100 cancer 00:51:18.160 --> 00:51:24.290 align:middle line:84% specimens for 20,000 genes, and that's the data set. 00:51:24.290 --> 00:51:27.070 align:middle line:84% So the first approach that's usually taken 00:51:27.070 --> 00:51:30.940 align:middle line:84% is from, within that data, to choose a gene set that will 00:51:30.940 --> 00:51:33.710 align:middle line:90% discriminate amongst classes. 00:51:33.710 --> 00:51:36.040 align:middle line:84% So in this approach, one decides ahead 00:51:36.040 --> 00:51:40.850 align:middle line:84% of time what one hopes to predict. 00:51:40.850 --> 00:51:43.910 align:middle line:84% So let's say, one wants to predict good outcome. 00:51:43.910 --> 00:51:48.490 align:middle line:84% A good outcome might be survival five years from now 00:51:48.490 --> 00:51:51.970 align:middle line:84% without metastases-- does a disease-free survival in five 00:51:51.970 --> 00:51:53.497 align:middle line:90% years from now. 00:51:53.497 --> 00:51:54.580 align:middle line:90% And that's a good outcome. 00:51:54.580 --> 00:51:56.410 align:middle line:84% If patients have that, we call that good. 00:51:56.410 --> 00:51:58.780 align:middle line:90% If they don't, that's bad. 00:51:58.780 --> 00:52:01.330 align:middle line:84% And so we're going to try and build 00:52:01.330 --> 00:52:05.260 align:middle line:84% a model based on our data set to predict 00:52:05.260 --> 00:52:08.180 align:middle line:90% good versus bad outcome. 00:52:08.180 --> 00:52:10.570 align:middle line:84% So this will be a binary predictor. 00:52:10.570 --> 00:52:12.700 align:middle line:84% We're trying to predict one outcome or the other. 00:52:12.700 --> 00:52:14.560 align:middle line:84% One can build predictors that are not 00:52:14.560 --> 00:52:18.430 align:middle line:84% binary that put people into one of three classes or four 00:52:18.430 --> 00:52:21.680 align:middle line:84% classes and say, you know, so forth. 00:52:21.680 --> 00:52:25.970 align:middle line:84% But this is the simplest type of binary prediction. 00:52:25.970 --> 00:52:28.600 align:middle line:84% So instead of using 20,000 genes, 00:52:28.600 --> 00:52:32.380 align:middle line:84% we're going to whittle them down to a more representative set 00:52:32.380 --> 00:52:36.130 align:middle line:84% of genes that are more meaningful to build this model, 00:52:36.130 --> 00:52:39.530 align:middle line:90% and that's called the gene set. 00:52:39.530 --> 00:52:42.910 align:middle line:84% Next, we have to construct or choose a prediction function 00:52:42.910 --> 00:52:45.490 align:middle line:84% that when applied to a new expression profile 00:52:45.490 --> 00:52:49.180 align:middle line:90% will produce a real number. 00:52:49.180 --> 00:52:50.680 align:middle line:84% So this prediction function is going 00:52:50.680 --> 00:52:53.200 align:middle line:90% to be a mathematical function. 00:52:53.200 --> 00:52:55.600 align:middle line:84% Take this times that, add this, square 00:52:55.600 --> 00:52:57.500 align:middle line:90% it, subtract something else. 00:52:57.500 --> 00:53:00.370 align:middle line:84% And it's going to spit out a single number-- 00:53:00.370 --> 00:53:05.280 align:middle line:90% 8, 8.5, 6.2. 00:53:05.280 --> 00:53:07.920 align:middle line:84% And then we need to choose a prediction rule that 00:53:07.920 --> 00:53:11.580 align:middle line:84% will classify a sample based on the output of the prediction 00:53:11.580 --> 00:53:14.230 align:middle line:90% function after application. 00:53:14.230 --> 00:53:16.980 align:middle line:84% So we take our new sample, our new vector, 00:53:16.980 --> 00:53:18.210 align:middle line:90% we feed it to the function. 00:53:18.210 --> 00:53:22.740 align:middle line:84% We get at 8.5, and we decide if the number is greater than 5, 00:53:22.740 --> 00:53:24.930 align:middle line:90% it's good. 00:53:24.930 --> 00:53:26.385 align:middle line:90% If it's less than 5, it's bad. 00:53:26.385 --> 00:53:30.110 align:middle line:90% 00:53:30.110 --> 00:53:34.520 align:middle line:84% And then the last method is to validate 00:53:34.520 --> 00:53:38.510 align:middle line:90% this model and its application. 00:53:38.510 --> 00:53:40.710 align:middle line:90% And that's class prediction. 00:53:40.710 --> 00:53:44.000 align:middle line:84% So going through it in a little bit of detail, 00:53:44.000 --> 00:53:47.610 align:middle line:84% we start with our data set on the left. 00:53:47.610 --> 00:53:50.910 align:middle line:90% We have columns. 00:53:50.910 --> 00:53:55.760 align:middle line:84% So we have 20,000 rows, whatever number of rows here-- 00:53:55.760 --> 00:53:59.750 align:middle line:84% I guess, 7,000-- some number of rows here. 00:53:59.750 --> 00:54:03.320 align:middle line:84% And we have our columns, which represent individual expression 00:54:03.320 --> 00:54:06.260 align:middle line:84% profiles from different tissue samples in patients 00:54:06.260 --> 00:54:08.420 align:middle line:90% with breast cancer. 00:54:08.420 --> 00:54:12.317 align:middle line:84% We figure out some discriminating set-- 00:54:12.317 --> 00:54:14.150 align:middle line:84% and there's a variety of methods to do that. 00:54:14.150 --> 00:54:15.860 align:middle line:90% We'll go through a couple. 00:54:15.860 --> 00:54:17.360 align:middle line:84% And we decide, well, we're not going 00:54:17.360 --> 00:54:22.880 align:middle line:84% to use all 20,000 genes but maybe just 500 of them. 00:54:22.880 --> 00:54:25.790 align:middle line:90% And we pick those genes. 00:54:25.790 --> 00:54:28.130 align:middle line:84% And then we construct our prediction function. 00:54:28.130 --> 00:54:30.390 align:middle line:84% And the prediction function looks like that. 00:54:30.390 --> 00:54:32.840 align:middle line:84% So if I feed it a vector, we're going 00:54:32.840 --> 00:54:37.010 align:middle line:84% to take the expression level for gene 8 00:54:37.010 --> 00:54:40.640 align:middle line:84% and multiply by 2 and the expression level for gene 33, 00:54:40.640 --> 00:54:43.350 align:middle line:84% and square it, and do all these other things. 00:54:43.350 --> 00:54:47.600 align:middle line:84% And the prediction function is going to give us a number, 00:54:47.600 --> 00:54:48.980 align:middle line:90% like 8.5. 00:54:48.980 --> 00:54:50.330 align:middle line:90% And then we need a rule. 00:54:50.330 --> 00:54:53.690 align:middle line:84% And one type of rule is a simple threshold rule. 00:54:53.690 --> 00:54:55.730 align:middle line:84% If it's less than 10, it's in one class. 00:54:55.730 --> 00:54:58.280 align:middle line:84% Greater than 10, it's another class. 00:54:58.280 --> 00:54:58.910 align:middle line:90% All right? 00:54:58.910 --> 00:54:59.450 align:middle line:90% Make sense? 00:54:59.450 --> 00:55:05.210 align:middle line:90% 00:55:05.210 --> 00:55:07.880 align:middle line:84% Now, there's plenty of options for building 00:55:07.880 --> 00:55:11.630 align:middle line:90% these models along each step. 00:55:11.630 --> 00:55:17.870 align:middle line:84% And there are papers using all of these different options 00:55:17.870 --> 00:55:19.710 align:middle line:90% to build these models. 00:55:19.710 --> 00:55:24.590 align:middle line:84% It's still rather ad-hoc approach to model building. 00:55:24.590 --> 00:55:28.130 align:middle line:84% One approach is to cluster the genes-- 00:55:28.130 --> 00:55:30.350 align:middle line:90% so to do class exploration-- 00:55:30.350 --> 00:55:31.880 align:middle line:84% cluster the genes and the columns. 00:55:31.880 --> 00:55:33.980 align:middle line:84% And look at it and say, there's a lot 00:55:33.980 --> 00:55:37.460 align:middle line:84% of red for these genes in one class 00:55:37.460 --> 00:55:38.960 align:middle line:90% and a lot of green over here. 00:55:38.960 --> 00:55:41.180 align:middle line:84% I think this set of genes are important. 00:55:41.180 --> 00:55:45.650 align:middle line:84% I'm going to use that in my discriminating gene set. 00:55:45.650 --> 00:55:49.460 align:middle line:84% That's a pretty successful way of doing it, actually. 00:55:49.460 --> 00:55:51.530 align:middle line:84% Another method is something called correlation 00:55:51.530 --> 00:55:53.690 align:middle line:90% to ideal outcome. 00:55:53.690 --> 00:55:55.880 align:middle line:84% There are other principled methods 00:55:55.880 --> 00:56:02.310 align:middle line:84% for doing this, for finding the discriminating gene set. 00:56:02.310 --> 00:56:05.280 align:middle line:84% This is one where, from my data set, 00:56:05.280 --> 00:56:07.460 align:middle line:84% if I was trying to build some of predictive model, 00:56:07.460 --> 00:56:11.450 align:middle line:84% this is the clustering of both the muscle samples 00:56:11.450 --> 00:56:13.730 align:middle line:90% and the genes going that way. 00:56:13.730 --> 00:56:17.150 align:middle line:84% And I'm looking at IBM and polymyositis, 00:56:17.150 --> 00:56:20.960 align:middle line:84% and I see a lot of red and a whole cluster 00:56:20.960 --> 00:56:23.900 align:middle line:84% of genes that are overexpressed in those diseases 00:56:23.900 --> 00:56:25.580 align:middle line:90% compared to others. 00:56:25.580 --> 00:56:27.620 align:middle line:84% The genes have a lot of similar function. 00:56:27.620 --> 00:56:32.630 align:middle line:84% They're immunoglobulin-related genes, mostly. 00:56:32.630 --> 00:56:35.210 align:middle line:84% And so if I was trying to build a model that would predict 00:56:35.210 --> 00:56:39.950 align:middle line:84% a new sample as either being within this group 00:56:39.950 --> 00:56:42.860 align:middle line:84% versus something else, I'd probably use these genes 00:56:42.860 --> 00:56:44.420 align:middle line:90% in the discriminating gene set. 00:56:44.420 --> 00:56:46.350 align:middle line:90% That would be one way to do it. 00:56:46.350 --> 00:56:50.610 align:middle line:84% Same thing here, and one can do that. 00:56:50.610 --> 00:56:54.800 align:middle line:84% Another option is what's called correlation to ideal outcome. 00:56:54.800 --> 00:56:56.720 align:middle line:84% And this one is used fairly often 00:56:56.720 --> 00:56:58.880 align:middle line:84% and seems like a pretty good method. 00:56:58.880 --> 00:57:02.900 align:middle line:84% And the way this works is, we have our samples, 00:57:02.900 --> 00:57:06.510 align:middle line:90% and we have our genes. 00:57:06.510 --> 00:57:10.190 align:middle line:84% And we will first organize our samples 00:57:10.190 --> 00:57:14.720 align:middle line:84% into the classes that we want them to be in, 00:57:14.720 --> 00:57:16.047 align:middle line:90% the good versus the bad. 00:57:16.047 --> 00:57:17.630 align:middle line:84% So take all the breast cancer patients 00:57:17.630 --> 00:57:20.060 align:middle line:84% who had a good outcome at five years, 00:57:20.060 --> 00:57:22.670 align:middle line:84% and we'll put them in one group, and the ones that 00:57:22.670 --> 00:57:25.590 align:middle line:90% had a bad outcome in the other. 00:57:25.590 --> 00:57:27.890 align:middle line:84% Then we'll create this ideal vector 00:57:27.890 --> 00:57:33.200 align:middle line:84% in which ones represent a good class, and zero, the bad class. 00:57:33.200 --> 00:57:36.740 align:middle line:84% And we'll look at pairwise correlation coefficients 00:57:36.740 --> 00:57:40.440 align:middle line:84% for all of our genes compared to this ideal vector. 00:57:40.440 --> 00:57:43.010 align:middle line:84% So we essentially-- correlation coefficients 00:57:43.010 --> 00:57:46.430 align:middle line:84% test linear direction of correlation. 00:57:46.430 --> 00:57:50.610 align:middle line:84% And so we'll be asking, which genes behave like this vector? 00:57:50.610 --> 00:57:56.120 align:middle line:84% Which genes are up in this class and then change 00:57:56.120 --> 00:57:58.190 align:middle line:90% and down in these other class? 00:57:58.190 --> 00:58:03.410 align:middle line:84% And that's how we'll find our gene set. 00:58:03.410 --> 00:58:07.290 align:middle line:84% AUDIENCE: [INAUDIBLE] question of 1 or negative 1 [INAUDIBLE]?? 00:58:07.290 --> 00:58:09.930 align:middle line:84% STEVEN A. GREENBERG: It doesn't matter what you use here. 00:58:09.930 --> 00:58:13.020 align:middle line:84% You just use two different numbers. 00:58:13.020 --> 00:58:14.600 align:middle line:90% And you can do the math. 00:58:14.600 --> 00:58:17.280 align:middle line:84% I mean, linear correlation coefficients 00:58:17.280 --> 00:58:23.190 align:middle line:84% are invariant to stretch transformations. 00:58:23.190 --> 00:58:28.700 align:middle line:90% So I don't think 1 or 0-- 00:58:28.700 --> 00:58:29.240 align:middle line:90% well, 1 or-- 00:58:29.240 --> 00:58:30.907 align:middle line:84% AUDIENCE: [INAUDIBLE] in the other case. 00:58:30.907 --> 00:58:33.390 align:middle line:84% So if you separately tiering down, 00:58:33.390 --> 00:58:37.850 align:middle line:84% you could also [INAUDIBLE] exactly how it [INAUDIBLE].. 00:58:37.850 --> 00:58:39.110 align:middle line:90% STEVEN A. GREENBERG: Right. 00:58:39.110 --> 00:58:41.570 align:middle line:84% You can use any ideal vector as long 00:58:41.570 --> 00:58:44.810 align:middle line:84% as it's one numbers represent one class, 00:58:44.810 --> 00:58:46.700 align:middle line:84% and a different number represent the others. 00:58:46.700 --> 00:58:47.825 align:middle line:90% Yeah? 00:58:47.825 --> 00:58:49.575 align:middle line:84% AUDIENCE: In the beginning of the lecture, 00:58:49.575 --> 00:58:52.330 align:middle line:84% you mentioned the binary classification method. 00:58:52.330 --> 00:58:58.140 align:middle line:84% Is there any way to expand it to fuzzy logic or multiples? 00:58:58.140 --> 00:59:01.410 align:middle line:84% In your own sample, it tests several of the test [INAUDIBLE] 00:59:01.410 --> 00:59:02.850 align:middle line:90% STEVEN A. GREENBERG: Yes. 00:59:02.850 --> 00:59:06.210 align:middle line:84% You certainly-- it just matters how you build the predictor 00:59:06.210 --> 00:59:07.630 align:middle line:90% function. 00:59:07.630 --> 00:59:10.110 align:middle line:84% So at this step for discriminating gene set, 00:59:10.110 --> 00:59:12.060 align:middle line:84% this could certainly be extended. 00:59:12.060 --> 00:59:16.020 align:middle line:84% This particular method of finding an optimal gene set, 00:59:16.020 --> 00:59:17.940 align:middle line:84% correlation to ideal outcome, could 00:59:17.940 --> 00:59:21.960 align:middle line:84% be established for any number of groupings of classes. 00:59:21.960 --> 00:59:26.760 align:middle line:84% You could use 2, 1, 0, different numbers in this ideal vector. 00:59:26.760 --> 00:59:30.570 align:middle line:84% And you will end up with a discriminating gene 00:59:30.570 --> 00:59:34.800 align:middle line:84% set, a set of genes that do have some differences in expression 00:59:34.800 --> 00:59:38.560 align:middle line:90% across the three classes. 00:59:38.560 --> 00:59:39.310 align:middle line:90% But I don't know. 00:59:39.310 --> 00:59:42.250 align:middle line:84% I'm sure that people have applied fuzzy logic. 00:59:42.250 --> 00:59:46.990 align:middle line:84% People have applied just about every mathematical method 00:59:46.990 --> 00:59:50.710 align:middle line:84% available to doing classification and model 00:59:50.710 --> 00:59:52.060 align:middle line:90% building. 00:59:52.060 --> 00:59:52.780 align:middle line:90% But I don't know. 00:59:52.780 --> 00:59:54.488 align:middle line:84% If that's something you're interested in, 00:59:54.488 --> 00:59:57.040 align:middle line:84% you might want to look at that and see what people have done. 00:59:57.040 --> 01:00:02.500 align:middle line:84% So but this basic method is to correlate all of our genes 01:00:02.500 --> 01:00:05.440 align:middle line:84% to this ideal vector to compute for each one 01:00:05.440 --> 01:00:08.800 align:middle line:84% the correlation coefficient with our ideal vector. 01:00:08.800 --> 01:00:11.320 align:middle line:84% And I listed fictitious examples there. 01:00:11.320 --> 01:00:14.110 align:middle line:84% And we see that, say, for gene 2, 01:00:14.110 --> 01:00:17.870 align:middle line:84% it has a very high correlation coefficient with the vector. 01:00:17.870 --> 01:00:20.740 align:middle line:84% So we might want that to be part of our gene-discriminating set 01:00:20.740 --> 01:00:22.030 align:middle line:90% to build this model. 01:00:22.030 --> 01:00:27.460 align:middle line:84% So we might take the highest 100 genes out of 10,000 01:00:27.460 --> 01:00:30.670 align:middle line:84% here, the ones with the highest correlation coefficients, 01:00:30.670 --> 01:00:33.610 align:middle line:84% and use that as our discriminating gene set 01:00:33.610 --> 01:00:34.480 align:middle line:90% to build this model. 01:00:34.480 --> 01:00:37.310 align:middle line:90% 01:00:37.310 --> 01:00:39.950 align:middle line:90% Let's say, these here, OK? 01:00:39.950 --> 01:00:42.800 align:middle line:84% OK, so then we want to make a prediction function once we 01:00:42.800 --> 01:00:44.180 align:middle line:90% have a discriminating gene set. 01:00:44.180 --> 01:00:48.510 align:middle line:84% And again, lots of ways to do this. 01:00:48.510 --> 01:00:55.370 align:middle line:84% How to compute a function that will give us back a number that 01:00:55.370 --> 01:00:59.660 align:middle line:84% differs significantly between the two classes, 01:00:59.660 --> 01:01:00.635 align:middle line:90% like that function. 01:01:00.635 --> 01:01:03.470 align:middle line:90% 01:01:03.470 --> 01:01:07.220 align:middle line:84% And then lastly, a prediction rule. 01:01:07.220 --> 01:01:11.900 align:middle line:84% The rule is often done as a threshold, saying, 01:01:11.900 --> 01:01:13.070 align:middle line:90% like that example there. 01:01:13.070 --> 01:01:15.410 align:middle line:84% If it's greater than or less than one number, 01:01:15.410 --> 01:01:16.760 align:middle line:90% it's in a different class. 01:01:16.760 --> 01:01:18.350 align:middle line:84% There are certainly prediction rules 01:01:18.350 --> 01:01:21.445 align:middle line:84% that can be ambiguous as well as deterministic ones. 01:01:21.445 --> 01:01:22.820 align:middle line:84% So you can have a prediction rule 01:01:22.820 --> 01:01:25.670 align:middle line:84% that says, if it's greater than 10, it's in one class. 01:01:25.670 --> 01:01:28.040 align:middle line:84% If it's less than 5, it's in another class. 01:01:28.040 --> 01:01:32.450 align:middle line:84% And between 5 and 10, I can't decide. 01:01:32.450 --> 01:01:35.960 align:middle line:84% And the choice of the rule is the classic trade-off 01:01:35.960 --> 01:01:39.110 align:middle line:84% of sensitivity versus specificity, 01:01:39.110 --> 01:01:43.040 align:middle line:84% that once you build your model and your prediction function, 01:01:43.040 --> 01:01:46.340 align:middle line:84% you can then arbitrarily vary your rule 01:01:46.340 --> 01:01:49.808 align:middle line:84% to optimize sensitivity and specificity. 01:01:49.808 --> 01:01:51.350 align:middle line:84% Depending on, let's say, a threshold, 01:01:51.350 --> 01:01:53.100 align:middle line:84% depending on where you put this threshold, 01:01:53.100 --> 01:01:56.600 align:middle line:84% you might get all of them correct in one class, 01:01:56.600 --> 01:01:59.720 align:middle line:84% but there are some from that class that were 01:01:59.720 --> 01:02:03.290 align:middle line:90% in the other class incorrectly. 01:02:03.290 --> 01:02:05.370 align:middle line:84% Or you might move it down some other way, 01:02:05.370 --> 01:02:07.628 align:middle line:84% and then you capture everybody who belongs in class 1, 01:02:07.628 --> 01:02:09.170 align:middle line:84% but there's a few people from class 2 01:02:09.170 --> 01:02:12.748 align:middle line:84% who are also being classified in class 1. 01:02:12.748 --> 01:02:15.345 align:middle line:90% 01:02:15.345 --> 01:02:16.710 align:middle line:90% Everyone OK with that? 01:02:16.710 --> 01:02:21.040 align:middle line:90% 01:02:21.040 --> 01:02:24.720 align:middle line:84% And that's an example here, without going 01:02:24.720 --> 01:02:27.690 align:middle line:84% into too many details, of classification, 01:02:27.690 --> 01:02:31.530 align:middle line:84% that the black is one type of sample, the white is another. 01:02:31.530 --> 01:02:32.700 align:middle line:90% They built a model. 01:02:32.700 --> 01:02:36.030 align:middle line:84% They used a discriminator gene set, the function. 01:02:36.030 --> 01:02:39.390 align:middle line:84% And then depending on whether they put their threshold here 01:02:39.390 --> 01:02:42.060 align:middle line:84% or here, if you put it down here, 01:02:42.060 --> 01:02:45.510 align:middle line:84% you get more of the blacks into this classification, 01:02:45.510 --> 01:02:49.240 align:middle line:84% but you pick up a couple of extra whites here, 01:02:49.240 --> 01:02:50.040 align:middle line:90% which don't belong. 01:02:50.040 --> 01:02:52.900 align:middle line:84% The blacks belong up here, and the whites belong here. 01:02:52.900 --> 01:02:57.180 align:middle line:84% And so depending on where you move this line, up here, 01:02:57.180 --> 01:03:02.430 align:middle line:84% you get different values for sensitivity and specificity 01:03:02.430 --> 01:03:04.095 align:middle line:90% of your predictive model. 01:03:04.095 --> 01:03:13.680 align:middle line:90% 01:03:13.680 --> 01:03:14.180 align:middle line:90% OK. 01:03:14.180 --> 01:03:19.820 align:middle line:84% Now, the last part of this for class prediction is validation. 01:03:19.820 --> 01:03:21.570 align:middle line:84% And this is a very important part. 01:03:21.570 --> 01:03:24.890 align:middle line:84% So the question is, why do validation? 01:03:24.890 --> 01:03:29.220 align:middle line:84% And that's my only fun slide there. 01:03:29.220 --> 01:03:31.980 align:middle line:84% And that's because of overfitting. 01:03:31.980 --> 01:03:38.660 align:middle line:84% So the problem is, let's say, you have a data set here. 01:03:38.660 --> 01:03:42.050 align:middle line:84% And I want to build a prediction function which will tell me 01:03:42.050 --> 01:03:43.680 align:middle line:90% what to do with new data. 01:03:43.680 --> 01:03:46.190 align:middle line:84% So I could draw a straight line through it. 01:03:46.190 --> 01:03:48.080 align:middle line:90% And now it's-- 01:03:48.080 --> 01:03:50.240 align:middle line:84% I'm going to use a simple model like that. 01:03:50.240 --> 01:03:53.930 align:middle line:84% Now if you give me an x value here, 01:03:53.930 --> 01:03:57.230 align:middle line:84% I'm going to predict the y value using the function, 01:03:57.230 --> 01:03:59.900 align:middle line:90% and that's right there. 01:03:59.900 --> 01:04:02.810 align:middle line:84% Now, let's say, instead, I start with this data set, 01:04:02.810 --> 01:04:06.410 align:middle line:84% and I use a very complicated model instead of 01:04:06.410 --> 01:04:09.440 align:middle line:90% a straight line-- 01:04:09.440 --> 01:04:12.630 align:middle line:84% one that works perfectly, in fact. 01:04:12.630 --> 01:04:19.730 align:middle line:84% Every data point ends up on the 100% correctly predicted 01:04:19.730 --> 01:04:22.250 align:middle line:90% by this function. 01:04:22.250 --> 01:04:23.770 align:middle line:84% But this is some complex function. 01:04:23.770 --> 01:04:25.520 align:middle line:84% This is more than a cubic here, but that's 01:04:25.520 --> 01:04:28.050 align:middle line:90% what I put up as an example. 01:04:28.050 --> 01:04:31.100 align:middle line:84% So if you get a new point now and the x-coordinate and you 01:04:31.100 --> 01:04:34.310 align:middle line:84% ask for a prediction of y, you might get something down here 01:04:34.310 --> 01:04:38.270 align:middle line:84% from this function, which probably is not the best match. 01:04:38.270 --> 01:04:41.570 align:middle line:84% There's no one really to say what the best match is here, 01:04:41.570 --> 01:04:43.650 align:middle line:84% but it's probably not the best match. 01:04:43.650 --> 01:04:48.080 align:middle line:84% And this is the issue that continues to plague 01:04:48.080 --> 01:04:50.030 align:middle line:90% model building in this field. 01:04:50.030 --> 01:04:55.220 align:middle line:84% And that's the overfitting because what these models are 01:04:55.220 --> 01:05:00.680 align:middle line:84% doing are taking 100 samples, let's 01:05:00.680 --> 01:05:07.550 align:middle line:84% say, 10,000 genes for each, and picking out of these 10,000 01:05:07.550 --> 01:05:11.780 align:middle line:84% genes a set of genes and a predictive function 01:05:11.780 --> 01:05:17.850 align:middle line:84% that correctly classifies this set of 100 samples. 01:05:17.850 --> 01:05:21.410 align:middle line:84% So you essentially have 10,000 or 20,000 variables 01:05:21.410 --> 01:05:27.970 align:middle line:84% to use to define 100 samples into two classes accurately. 01:05:27.970 --> 01:05:32.220 align:middle line:84% And it's a highly underdetermined data set. 01:05:32.220 --> 01:05:33.810 align:middle line:84% If you allow yourself the freedom 01:05:33.810 --> 01:05:36.210 align:middle line:84% to choose any one over number of ways 01:05:36.210 --> 01:05:41.130 align:middle line:84% to pick the discriminator gene set to build the function, 01:05:41.130 --> 01:05:43.800 align:middle line:84% one can take random noise and build 01:05:43.800 --> 01:05:47.490 align:middle line:84% a perfect predictive function that will classify it. 01:05:47.490 --> 01:05:49.440 align:middle line:84% And so that's where validation comes in. 01:05:49.440 --> 01:05:54.060 align:middle line:84% Validation is an approach in which one has built the model 01:05:54.060 --> 01:05:56.640 align:middle line:84% and then needs to test it before deciding, 01:05:56.640 --> 01:05:58.290 align:middle line:84% I'm going to use this in patients 01:05:58.290 --> 01:06:02.250 align:middle line:84% to decide how to correctly predict new samples. 01:06:02.250 --> 01:06:05.040 align:middle line:84% And the correct way to test it is to, 01:06:05.040 --> 01:06:08.190 align:middle line:84% once you have your model, is to then look 01:06:08.190 --> 01:06:10.980 align:middle line:84% at your next 100 patients who come in, 01:06:10.980 --> 01:06:15.150 align:middle line:84% and do their gene expressions, make a prediction, 01:06:15.150 --> 01:06:18.270 align:middle line:84% wait five years, and see who is really 01:06:18.270 --> 01:06:21.250 align:middle line:84% in the good versus the bad outcome. 01:06:21.250 --> 01:06:24.030 align:middle line:84% And that's the way this will need to be done. 01:06:24.030 --> 01:06:26.700 align:middle line:84% But in the meantime, nobody really 01:06:26.700 --> 01:06:32.280 align:middle line:84% has time to wait five years or to do this 01:06:32.280 --> 01:06:36.000 align:middle line:84% on another 100 samples of patients 01:06:36.000 --> 01:06:37.140 align:middle line:90% and to look that far ahead. 01:06:37.140 --> 01:06:38.723 align:middle line:84% Although I suspect people are starting 01:06:38.723 --> 01:06:41.280 align:middle line:84% to take that perspective in this. 01:06:41.280 --> 01:06:43.770 align:middle line:84% So what people have usually done is 01:06:43.770 --> 01:06:45.780 align:middle line:84% they go back to the original data set 01:06:45.780 --> 01:06:51.000 align:middle line:84% they used to build the model and validate the model 01:06:51.000 --> 01:06:55.664 align:middle line:90% on that data set, sort of. 01:06:55.664 --> 01:06:58.520 align:middle line:84% Sometimes, this is done through the combination of a training 01:06:58.520 --> 01:07:00.030 align:middle line:90% set and a validation set. 01:07:00.030 --> 01:07:02.930 align:middle line:90% 01:07:02.930 --> 01:07:04.290 align:middle line:90% So you had 100 samples. 01:07:04.290 --> 01:07:07.760 align:middle line:84% So why don't we just build the model on 66 of the samples 01:07:07.760 --> 01:07:11.060 align:middle line:84% and save the other 33 to test it? 01:07:11.060 --> 01:07:12.930 align:middle line:90% And that's fine to do that. 01:07:12.930 --> 01:07:15.620 align:middle line:84% So people build the model based on two thirds of the data 01:07:15.620 --> 01:07:18.650 align:middle line:90% and call that the training set. 01:07:18.650 --> 01:07:21.710 align:middle line:84% And then the other third is used to validate the model 01:07:21.710 --> 01:07:24.420 align:middle line:90% and test its accuracy. 01:07:24.420 --> 01:07:26.390 align:middle line:84% Another approach when people don't 01:07:26.390 --> 01:07:28.850 align:middle line:84% have a separate validation set and use 01:07:28.850 --> 01:07:31.160 align:middle line:84% all specimens to create their model 01:07:31.160 --> 01:07:33.770 align:middle line:90% is called cross-validation. 01:07:33.770 --> 01:07:37.370 align:middle line:84% And that's a way of validating the model on the original data 01:07:37.370 --> 01:07:41.270 align:middle line:84% set that was used to construct it. 01:07:41.270 --> 01:07:44.480 align:middle line:84% And there is a potential way to do this OK. 01:07:44.480 --> 01:07:47.660 align:middle line:84% It's inherently got problems with it. 01:07:47.660 --> 01:07:51.120 align:middle line:84% The most common method is leave-one-out cross-validation. 01:07:51.120 --> 01:07:54.140 align:middle line:84% So in that method, you have 100 samples. 01:07:54.140 --> 01:07:56.880 align:middle line:90% First, you remove one sample. 01:07:56.880 --> 01:07:58.850 align:middle line:90% So now you have 99 samples. 01:07:58.850 --> 01:08:01.910 align:middle line:84% You build the model based on those 99 samples, 01:08:01.910 --> 01:08:05.810 align:middle line:84% and then you test it on this one sample that's left. 01:08:05.810 --> 01:08:08.870 align:middle line:84% And you see whether it makes the correct prediction. 01:08:08.870 --> 01:08:10.080 align:middle line:90% Then you do it again. 01:08:10.080 --> 01:08:12.620 align:middle line:84% You take another sample and remove it, 01:08:12.620 --> 01:08:16.069 align:middle line:84% build another model based on those 99 samples, 01:08:16.069 --> 01:08:20.240 align:middle line:84% test its prediction for that sample. 01:08:20.240 --> 01:08:22.520 align:middle line:84% And you do that 100 times, and you 01:08:22.520 --> 01:08:26.420 align:middle line:84% ask, what is the accuracy of this approach? 01:08:26.420 --> 01:08:30.170 align:middle line:84% And did it accurately predict all 100 times 01:08:30.170 --> 01:08:33.175 align:middle line:84% correctly the class or not for the sample? 01:08:33.175 --> 01:08:37.859 align:middle line:90% 01:08:37.859 --> 01:08:41.370 align:middle line:90% So that's validation options. 01:08:41.370 --> 01:08:45.410 align:middle line:84% So that's the leave-one-out cross-validation construct 01:08:45.410 --> 01:08:46.229 align:middle line:90% predictor. 01:08:46.229 --> 01:08:49.760 align:middle line:84% Apply predictor to one left-out sample, repeat for each sample, 01:08:49.760 --> 01:08:54.479 align:middle line:84% calculate an error rate as present misclassifications. 01:08:54.479 --> 01:09:02.750 align:middle line:84% So the way that cross-validation works is, again, the underline 01:09:02.750 --> 01:09:07.670 align:middle line:84% there is construct predictor using all data except one. 01:09:07.670 --> 01:09:10.850 align:middle line:84% If we go back to the method we use 01:09:10.850 --> 01:09:13.700 align:middle line:84% to construct a predictor is, from the data, 01:09:13.700 --> 01:09:16.850 align:middle line:84% we choose a gene set, we choose a prediction function, 01:09:16.850 --> 01:09:20.750 align:middle line:84% and we choose a prediction rule, right? 01:09:20.750 --> 01:09:25.550 align:middle line:84% So a common problem in this area is that people don't do this. 01:09:25.550 --> 01:09:28.399 align:middle line:90% They skip step one. 01:09:28.399 --> 01:09:31.609 align:middle line:84% So they don't go back and rechoose 01:09:31.609 --> 01:09:37.066 align:middle line:84% a new gene set each time they apply cross-validation. 01:09:37.066 --> 01:09:39.649 align:middle line:84% If you think about it, you will have to choose a new gene set, 01:09:39.649 --> 01:09:43.160 align:middle line:84% and you might get a different set of genes each time. 01:09:43.160 --> 01:09:44.210 align:middle line:90% And that's a problem. 01:09:44.210 --> 01:09:46.760 align:middle line:84% If you're trying to build a model 01:09:46.760 --> 01:09:50.300 align:middle line:84% and say that these are the genes that are important in my model, 01:09:50.300 --> 01:09:53.180 align:middle line:84% I'm going to use them for new patients, 01:09:53.180 --> 01:09:57.740 align:middle line:84% you can't really test that hypothesis using this approach. 01:09:57.740 --> 01:10:01.060 align:middle line:90% 01:10:01.060 --> 01:10:04.230 align:middle line:84% And so people have looked at simulations 01:10:04.230 --> 01:10:06.720 align:middle line:84% of doing incorrect cross-validation of not 01:10:06.720 --> 01:10:08.640 align:middle line:90% repeating that first step. 01:10:08.640 --> 01:10:10.740 align:middle line:84% And essentially, the simulations show 01:10:10.740 --> 01:10:13.080 align:middle line:84% that you can build predictors that 01:10:13.080 --> 01:10:18.270 align:middle line:84% are 100% accurate, even in a data set 01:10:18.270 --> 01:10:21.648 align:middle line:90% that's completely random. 01:10:21.648 --> 01:10:29.730 align:middle line:84% All right, now, I want to wrap this up. 01:10:29.730 --> 01:10:32.730 align:middle line:84% But if you repeat the process correctly 01:10:32.730 --> 01:10:36.000 align:middle line:84% by reselecting the informative gene set each time 01:10:36.000 --> 01:10:39.270 align:middle line:84% you leave one out, then the predictor's accuracy 01:10:39.270 --> 01:10:43.380 align:middle line:84% is no better than chance, as expected in a random data set. 01:10:43.380 --> 01:10:47.115 align:middle line:84% So it makes a huge difference here. 01:10:47.115 --> 01:10:51.710 align:middle line:90% 01:10:51.710 --> 01:10:57.760 align:middle line:84% So many papers have incorrect cross-validation in them. 01:10:57.760 --> 01:11:01.517 align:middle line:84% That paper on breast cancer and nature 01:11:01.517 --> 01:11:03.850 align:middle line:84% and its follow-up in The New England Journal of Medicine 01:11:03.850 --> 01:11:08.680 align:middle line:84% as well reported an accuracy of 73% of its predicted. 01:11:08.680 --> 01:11:12.340 align:middle line:84% They built a predictor to predict good versus bad outcome 01:11:12.340 --> 01:11:17.200 align:middle line:90% and reported a 73% accuracy. 01:11:17.200 --> 01:11:19.300 align:middle line:84% This is based on leave-one-out cross-validation, 01:11:19.300 --> 01:11:22.990 align:middle line:84% but they did not recalculate the informative gene set. 01:11:22.990 --> 01:11:27.270 align:middle line:84% If one does that, the corrected actually is only 59%. 01:11:27.270 --> 01:11:29.020 align:middle line:84% It's not much better than flipping a coin. 01:11:29.020 --> 01:11:32.260 align:middle line:90% 01:11:32.260 --> 01:11:37.435 align:middle line:84% Other papers, even the ALL/AML did incorrect cross-validation. 01:11:37.435 --> 01:11:40.330 align:middle line:90% 01:11:40.330 --> 01:11:42.600 align:middle line:90% Well, that's not correct. 01:11:42.600 --> 01:11:44.350 align:middle line:84% They didn't do incorrect cross-validation, 01:11:44.350 --> 01:11:47.470 align:middle line:84% but they had a classifier which was ambiguous. 01:11:47.470 --> 01:11:50.170 align:middle line:84% That's another issue in reporting accuracy, 01:11:50.170 --> 01:11:53.560 align:middle line:84% is they did not use a classifier that said yes or no. 01:11:53.560 --> 01:11:55.330 align:middle line:90% There was a gray zone. 01:11:55.330 --> 01:11:58.750 align:middle line:84% And so they reported the accuracy as 36 out of 38, 01:11:58.750 --> 01:12:03.040 align:middle line:84% but the other two were uncertain. 01:12:03.040 --> 01:12:11.040 align:middle line:90% 01:12:11.040 --> 01:12:14.820 align:middle line:84% Similarly, with another paper on medulloblastoma, 01:12:14.820 --> 01:12:17.460 align:middle line:84% in terms of using ambiguous classifiers, 01:12:17.460 --> 01:12:22.685 align:middle line:84% reporting an accuracy of 72%, but it's 01:12:22.685 --> 01:12:24.060 align:middle line:84% not really the right way to do it 01:12:24.060 --> 01:12:25.990 align:middle line:90% with ambiguous classification. 01:12:25.990 --> 01:12:28.170 align:middle line:90% So that's a separate one. 01:12:28.170 --> 01:12:31.800 align:middle line:84% So I think I'll stop here, so I have a little something 01:12:31.800 --> 01:12:35.440 align:middle line:90% to say next time I see you. 01:12:35.440 --> 01:12:37.050 align:middle line:90% OK, thanks. 01:12:37.050 --> 01:12:40.060 align:middle line:90% Any questions right now? 01:12:40.060 --> 01:12:44.730 align:middle line:84% So is this-- I'm trying to keep it fairly simple. 01:12:44.730 --> 01:12:47.340 align:middle line:90% 01:12:47.340 --> 01:12:53.310 align:middle line:84% Does this seem appropriate, not too complex but not too slow 01:12:53.310 --> 01:12:53.948 align:middle line:90% either? 01:12:53.948 --> 01:12:55.365 align:middle line:84% Or does it seem a little too slow? 01:12:55.365 --> 01:13:00.730 align:middle line:90% 01:13:00.730 --> 01:13:01.380 align:middle line:90% Whatever, yeah. 01:13:01.380 --> 01:13:03.797 align:middle line:84% AUDIENCE: Is this going to be up on the myCourses website? 01:13:03.797 --> 01:13:05.500 align:middle line:84% STEVEN A. GREENBERG: Yeah, yeah, these 01:13:05.500 --> 01:13:07.090 align:middle line:84% are the same ones I used last year. 01:13:07.090 --> 01:13:10.477 align:middle line:84% So is the stuff that we put up last year still on there? 01:13:10.477 --> 01:13:11.560 align:middle line:90% Does it need to be put up? 01:13:11.560 --> 01:13:14.290 align:middle line:84% Yeah, so that's still on there, yep. 01:13:14.290 --> 01:13:18.550 align:middle line:84% Have guys been getting problems sets to do? 01:13:18.550 --> 01:13:19.630 align:middle line:90% Hoe's that been working? 01:13:19.630 --> 01:13:28.256 align:middle line:84% AUDIENCE: [INAUDIBLE] presentation of basic colors. 01:13:28.256 --> 01:13:30.950 align:middle line:90% [INAUDIBLE] 01:13:30.950 --> 01:13:32.990 align:middle line:84% STEVEN A. GREENBERG: There's the Eisen Cluster 01:13:32.990 --> 01:13:35.130 align:middle line:90% and TreeView software. 01:13:35.130 --> 01:13:36.090 align:middle line:90% AUDIENCE: OK. 01:13:36.090 --> 01:13:38.363 align:middle line:84% Is it TreeView making the graphs? 01:13:38.363 --> 01:13:40.280 align:middle line:84% STEVEN A. GREENBERG: TreeView makes the graph. 01:13:40.280 --> 01:13:44.422 align:middle line:84% AUDIENCE: [INAUDIBLE] if I do something 01:13:44.422 --> 01:13:47.980 align:middle line:84% with another tool, then I can [INAUDIBLE]?? 01:13:47.980 --> 01:13:49.730 align:middle line:84% STEVEN A. GREENBERG: As long as you put it 01:13:49.730 --> 01:13:52.660 align:middle line:84% in the right format for TreeView to look at, yeah. 01:13:52.660 --> 01:13:53.950 align:middle line:90% AUDIENCE: OK, thank you. 01:13:53.950 --> 01:13:54.300 align:middle line:90% STEVEN A. GREENBERG: Yeah. 01:13:54.300 --> 01:13:57.140 align:middle line:84% AUDIENCE: There is another system that used pink and blue, 01:13:57.140 --> 01:13:57.770 align:middle line:90% right? 01:13:57.770 --> 01:14:00.187 align:middle line:84% STEVEN A. GREENBERG: There's the pink and blue system too. 01:14:00.187 --> 01:14:07.370 align:middle line:84% So some of those are like Peter Park, who's 01:14:07.370 --> 01:14:09.470 align:middle line:84% going to talk to you, I think, next week, he just 01:14:09.470 --> 01:14:12.920 align:middle line:84% uses our programs that he wrote to do that. 01:14:12.920 --> 01:14:15.980 align:middle line:84% There's commercial software called GeneSpring 01:14:15.980 --> 01:14:18.980 align:middle line:90% that uses pink and blue. 01:14:18.980 --> 01:14:27.230 align:middle line:84% There's a program called TGEV, T-G-E-V, out of the TIGR, 01:14:27.230 --> 01:14:29.390 align:middle line:84% The Institute for Genomic Research, 01:14:29.390 --> 01:14:33.670 align:middle line:84% and they have free clustering programs. 01:14:33.670 --> 01:14:35.000 align:middle line:90%