WEBVTT 00:00:00.000 --> 00:00:02.430 The following content is provided under a Creative 00:00:02.430 --> 00:00:03.730 Commons license. 00:00:03.730 --> 00:00:06.030 Your support will help MIT OpenCourseWare 00:00:06.030 --> 00:00:10.060 continue to offer high-quality educational resources for free. 00:00:10.060 --> 00:00:12.660 To make a donation or to view additional materials 00:00:12.660 --> 00:00:16.560 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:16.560 --> 00:00:17.874 at ocw.mit.edu. 00:00:32.409 --> 00:00:36.070 DUANE BONING: OK, so today we're going to talk about some 00:00:36.070 --> 00:00:38.800 of the probability models associated with variation 00:00:38.800 --> 00:00:41.260 in manufacturing processes. 00:00:41.260 --> 00:00:42.880 The last couple of classes, we've 00:00:42.880 --> 00:00:46.040 tried to go and do two things. 00:00:46.040 --> 00:00:48.070 One is give you a little bit of perspective 00:00:48.070 --> 00:00:50.370 on a couple of different kinds of-- 00:00:50.370 --> 00:00:53.140 or actually, a fairly wide family 00:00:53.140 --> 00:00:55.930 of manufacturing processes, give you 00:00:55.930 --> 00:00:58.420 a little bit of exposure to semiconductor processes 00:00:58.420 --> 00:01:03.070 as well as a couple of different mechanical-- 00:01:03.070 --> 00:01:06.100 mechanically-oriented processes, and give you 00:01:06.100 --> 00:01:10.340 some ideas or ways to think about those processes, 00:01:10.340 --> 00:01:13.250 including just the physical action that's going on, 00:01:13.250 --> 00:01:15.350 as well as some of the sources of variation. 00:01:15.350 --> 00:01:17.200 So we're going to dive in a little bit 00:01:17.200 --> 00:01:23.530 today on dealing with data taken from some manufacturing 00:01:23.530 --> 00:01:26.230 processes, and start to look at random components of that 00:01:26.230 --> 00:01:28.330 and systematic components of that. 00:01:28.330 --> 00:01:31.210 But back on the theme of trying to make sure 00:01:31.210 --> 00:01:34.120 that you have at least some exposure, given everybody's 00:01:34.120 --> 00:01:36.730 different backgrounds, to these processes, 00:01:36.730 --> 00:01:39.970 we would like to make sure you have the opportunity 00:01:39.970 --> 00:01:45.370 to at least see a couple of the kinds of tools 00:01:45.370 --> 00:01:47.360 and equipment in the facilities-- 00:01:47.360 --> 00:01:51.010 especially those associated with semiconductor manufacturing, 00:01:51.010 --> 00:01:53.985 if you haven't previously seen those. 00:01:53.985 --> 00:01:55.360 So let me turn it over to Hayden, 00:01:55.360 --> 00:01:57.520 who can check in with you well the problem 00:01:57.520 --> 00:02:03.100 is that as well, but also about arranging opportunities 00:02:03.100 --> 00:02:05.770 for you to see a little bit more, if you would like. 00:02:08.175 --> 00:02:11.320 HAYDEN TAYLOR: So yes, on the problem set, well, 00:02:11.320 --> 00:02:13.990 if you still have questions about it, 00:02:13.990 --> 00:02:17.350 then feel free to email or call today. 00:02:17.350 --> 00:02:19.810 See me afterwards. 00:02:19.810 --> 00:02:23.890 And problem set 2 is now up on the website. 00:02:23.890 --> 00:02:28.330 That's due on Tuesday, the 26th of this month. 00:02:28.330 --> 00:02:30.260 So there's a bit longer for that, 00:02:30.260 --> 00:02:34.960 and I'm willing to start helping you with that too. 00:02:34.960 --> 00:02:35.860 So thank you. 00:02:39.040 --> 00:02:41.050 DUANE BONING: Any questions for Hayden? 00:02:41.050 --> 00:02:45.170 Any big issues with the problem set? 00:02:45.170 --> 00:02:46.970 AUDIENCE: Actually, I have a-- 00:02:46.970 --> 00:02:47.928 DUANE BONING: Go ahead. 00:02:48.920 --> 00:02:49.670 HAYDEN TAYLOR: OK. 00:02:49.670 --> 00:02:51.212 AUDIENCE: So I have a quick question. 00:02:51.212 --> 00:02:54.080 So if you can refer to the Question 3 of the Problem Set 00:02:54.080 --> 00:03:01.090 1, and you say it's proportional to the water and the time-- 00:03:01.090 --> 00:03:02.390 the KOH. 00:03:02.390 --> 00:03:04.880 And when you talk about these [INAUDIBLE],, 00:03:04.880 --> 00:03:07.010 you're talking about the concentration 00:03:07.010 --> 00:03:08.835 or you're talking about the molarity? 00:03:13.237 --> 00:03:15.530 HAYDEN TAYLOR: What am I saying is proportional to KOH. 00:03:15.530 --> 00:03:16.170 DUANE BONING: The square brackets around the KOH. 00:03:16.670 --> 00:03:18.120 HAYDEN TAYLOR: Oh, I see. 00:03:18.120 --> 00:03:20.245 AUDIENCE: The very last figure [INAUDIBLE].. 00:03:20.245 --> 00:03:20.730 HAYDEN TAYLOR: Yeah, yeah. 00:03:20.730 --> 00:03:21.030 AUDIENCE: --proportional. 00:03:21.030 --> 00:03:23.140 The [? edge 3 ?] is proportional. 00:03:23.140 --> 00:03:25.395 So it's proportional to the concentration, 00:03:25.395 --> 00:03:27.825 or it's proportional to the molarity? 00:03:30.127 --> 00:03:36.060 HAYDEN TAYLOR: Well, for a fixed volume, it's the same. 00:03:36.060 --> 00:03:38.070 There are two lines on that plot, 00:03:38.070 --> 00:03:41.970 and the models that are put forward by [? Seidel ?] 00:03:41.970 --> 00:03:43.080 to explain the data. 00:03:43.080 --> 00:03:47.070 The data are the open circles on that plot. 00:03:47.070 --> 00:03:52.950 And there are two models that have plotted. 00:03:52.950 --> 00:03:57.720 And what I want you to use are primarily 00:03:57.720 --> 00:03:59.530 the experimental data. 00:03:59.530 --> 00:04:05.850 So look at the curved line that is plotted on that graph. 00:04:05.850 --> 00:04:08.150 AUDIENCE: OK and for the curved line that you 00:04:08.150 --> 00:04:13.350 want us to look at, the x-axis should be the concentration, 00:04:13.350 --> 00:04:13.850 right? 00:04:14.285 --> 00:04:14.570 HAYDEN TAYLOR: That's right. 00:04:14.570 --> 00:04:15.050 Yes, the x-axis-- 00:04:15.050 --> 00:04:15.925 AUDIENCE: [INAUDIBLE] 00:04:15.925 --> 00:04:18.355 HAYDEN TAYLOR: --is the weight of KOH. 00:04:18.355 --> 00:04:19.730 In other words, that's the number 00:04:19.730 --> 00:04:24.315 of grams of KOH per gram of water. 00:04:24.315 --> 00:04:25.940 AUDIENCE: Yeah, I just want to clarify. 00:04:25.940 --> 00:04:26.350 That's it. 00:04:26.350 --> 00:04:26.876 HAYDEN TAYLOR: Yeah. 00:04:26.876 --> 00:04:27.312 Thank you. 00:04:27.312 --> 00:04:27.750 AUDIENCE: Thanks. 00:04:27.750 --> 00:04:28.250 Thank you. 00:04:28.445 --> 00:04:29.612 HAYDEN TAYLOR: Anybody else? 00:04:31.610 --> 00:04:32.110 OK. 00:04:35.168 --> 00:04:36.960 DUANE BONING: I was just also going to say, 00:04:36.960 --> 00:04:38.670 it's sort of a manufacturing process 00:04:38.670 --> 00:04:40.950 that we use for making up these problem sets. 00:04:40.950 --> 00:04:46.360 And they are not expected to be completely defect-free, 00:04:46.360 --> 00:04:49.740 so we expect part of the process is 00:04:49.740 --> 00:04:53.670 detecting errors or questions. 00:04:53.670 --> 00:04:58.350 And then we try to correct them. 00:04:58.350 --> 00:05:02.910 Also, as a follow-up to that the schedule for the next problem 00:05:02.910 --> 00:05:07.090 set, just looking ahead for the next couple of weeks, 00:05:07.090 --> 00:05:12.090 the reason that the problem set is due not next Thursday, 00:05:12.090 --> 00:05:14.040 but rather, the Tuesday after that-- 00:05:14.040 --> 00:05:15.840 a little bit longer than a week-- 00:05:15.840 --> 00:05:22.220 is that here at MIT, next Monday, this coming Monday 00:05:22.220 --> 00:05:25.670 is President's Day, a holiday. 00:05:25.670 --> 00:05:29.060 And then Tuesday is one of these strange days. 00:05:29.060 --> 00:05:31.940 You folks out in Singapore-- you may recall this. 00:05:31.940 --> 00:05:34.810 Tuesday is a Monday. 00:05:34.810 --> 00:05:36.370 So it's on a Monday class schedule, 00:05:36.370 --> 00:05:38.490 so we will not be meeting. 00:05:38.490 --> 00:05:40.540 OK? 00:05:40.540 --> 00:05:43.090 Go to whatever classes you would normally have on Monday. 00:05:43.090 --> 00:05:48.020 Go to those on Tuesday out here to MIT. 00:05:48.020 --> 00:05:51.790 And so then our next lecture will be a week from today, 00:05:51.790 --> 00:05:54.160 on Thursday. 00:05:54.160 --> 00:06:01.350 OK, so also going with the material 00:06:01.350 --> 00:06:02.940 that we're starting to talk about now 00:06:02.940 --> 00:06:07.500 and the second problem set that's just been released 00:06:07.500 --> 00:06:10.230 is another reading assignment. 00:06:10.230 --> 00:06:12.150 And in particular, what we're trying 00:06:12.150 --> 00:06:15.090 to do this year is focus-- 00:06:15.090 --> 00:06:19.980 do sort of the concentrated dive of May and Spanos Chapter 4. 00:06:19.980 --> 00:06:21.840 In the past, sometimes we've assigned 00:06:21.840 --> 00:06:25.110 instead Chapters 2 and 3 of Montgomery, 00:06:25.110 --> 00:06:28.140 which are much longer. 00:06:28.140 --> 00:06:32.580 Chapter 4 in Spanos is a shorter overview 00:06:32.580 --> 00:06:36.000 of basic statistical distributions 00:06:36.000 --> 00:06:42.030 and some very basic manipulation and use of statistics 00:06:42.030 --> 00:06:44.300 that we'll talk about today. 00:06:44.300 --> 00:06:47.640 So we're not formally assigning chapters 2 and 3 in Montgomery, 00:06:47.640 --> 00:06:49.380 but the intent here-- 00:06:49.380 --> 00:06:51.840 what I mean by this reading assignment is, 00:06:51.840 --> 00:06:54.210 if you don't have very much background on statistics, 00:06:54.210 --> 00:06:56.760 you may want to read Chapters 2 and 3. 00:06:56.760 --> 00:06:58.710 Or you can skim them to know what's 00:06:58.710 --> 00:07:02.790 in there, and in particular, that's probably the right place 00:07:02.790 --> 00:07:07.440 to go for a longer explanation, if you need more material. 00:07:07.440 --> 00:07:11.370 If May and Spanos is too condensed 00:07:11.370 --> 00:07:13.950 for you or you'd like additional examples, 00:07:13.950 --> 00:07:19.630 there's a very nice articulation in Montgomery. 00:07:19.630 --> 00:07:23.340 Now, I have not made a formal assignment of May and Spanos 00:07:23.340 --> 00:07:29.640 Chapter 3, but that is where you should 00:07:29.640 --> 00:07:32.400 go if you want to read more or need 00:07:32.400 --> 00:07:36.990 to read more about semiconductor fabrication process physics. 00:07:36.990 --> 00:07:39.570 That chapter is a very nice, again, 00:07:39.570 --> 00:07:43.170 relatively condensed description, 00:07:43.170 --> 00:07:51.180 articulation of the key process steps used in microfabrication. 00:07:51.180 --> 00:07:54.510 We'll, throughout the term, be popping in and out 00:07:54.510 --> 00:07:56.340 of a few of those unit processes, 00:07:56.340 --> 00:07:59.640 and I just want you to be aware that's what's in that chapter. 00:07:59.640 --> 00:08:05.310 And you will quite likely want to go in and skim that or read 00:08:05.310 --> 00:08:07.050 that, and dive in on some of those 00:08:07.050 --> 00:08:10.080 processes if you need more background 00:08:10.080 --> 00:08:11.230 on some of those things. 00:08:11.230 --> 00:08:14.220 And I think, especially before this tour 00:08:14.220 --> 00:08:19.290 that Hayden is organizing, it'd be at least really good 00:08:19.290 --> 00:08:22.470 to skim chapter 3. 00:08:22.470 --> 00:08:24.240 Again, it's mostly on the process physics, 00:08:24.240 --> 00:08:25.907 but every now and then, it does at least 00:08:25.907 --> 00:08:30.030 show you a little bit about the equipment that's used. 00:08:30.030 --> 00:08:32.070 And it's very nice to be able-- 00:08:32.070 --> 00:08:34.860 for Hayden to be able to point to some of the equipment 00:08:34.860 --> 00:08:39.000 and components of that, and have you visualize what's going on 00:08:39.000 --> 00:08:41.400 in the process physics, or be able to ask questions 00:08:41.400 --> 00:08:44.120 about that. 00:08:44.120 --> 00:08:48.260 OK, so what I'm going to do here at the start 00:08:48.260 --> 00:08:53.120 is just run through some example processes. 00:08:53.120 --> 00:08:59.190 And in particular, these are little not quite 00:08:59.190 --> 00:09:00.960 manufacturing processes. 00:09:00.960 --> 00:09:04.950 They are more like research scale implementations 00:09:04.950 --> 00:09:10.650 of some basic manufacturing processes that we have actually 00:09:10.650 --> 00:09:13.650 generated some data from, or students have generated data 00:09:13.650 --> 00:09:15.460 from over the years. 00:09:15.460 --> 00:09:17.880 So I'm going to show you some actual measurements 00:09:17.880 --> 00:09:20.580 from a few simple cases. 00:09:20.580 --> 00:09:23.040 And we're going to look at that data, 00:09:23.040 --> 00:09:29.330 and what I think we're going to see is variation. 00:09:29.330 --> 00:09:33.860 And the puzzle for you and for us-- 00:09:33.860 --> 00:09:36.680 and to some extent, for me, because 00:09:36.680 --> 00:09:38.930 in many of these processes, I was not 00:09:38.930 --> 00:09:41.280 involved in the generation of the data-- 00:09:41.280 --> 00:09:43.520 our puzzle here is to look at the data 00:09:43.520 --> 00:09:48.620 and try to think about what might be going on in the data. 00:09:48.620 --> 00:09:51.990 So here's some-- a basic turning process. 00:09:51.990 --> 00:09:53.660 So we've got a work piece. 00:09:53.660 --> 00:09:59.990 We've got under computerized numerical controls, CNC. 00:09:59.990 --> 00:10:06.620 We've got the ability to cut into that work piece, which 00:10:06.620 --> 00:10:10.220 is rotating, to try to get to diameter work pieces 00:10:10.220 --> 00:10:15.120 of a specified dimension. 00:10:15.120 --> 00:10:19.880 So here's the diameter measured on a particular location 00:10:19.880 --> 00:10:23.750 on that work piece as a function of 47-- 00:10:23.750 --> 00:10:26.570 I think it's 47 or 48-- 00:10:26.570 --> 00:10:28.370 different repetitions. 00:10:28.370 --> 00:10:32.300 Now, I believe this may have involved 00:10:32.300 --> 00:10:36.230 more than one set of students, or more than one class. 00:10:36.230 --> 00:10:39.020 We can think of that as a shift change 00:10:39.020 --> 00:10:41.600 in a manufacturing process. 00:10:41.600 --> 00:10:47.870 We're simply gathering here a measurement of a diameter 00:10:47.870 --> 00:10:50.120 on that work piece. 00:10:50.120 --> 00:10:51.390 So talk to me. 00:10:51.390 --> 00:10:52.520 Tell me what you see. 00:10:55.767 --> 00:10:56.350 Is it uniform? 00:10:58.915 --> 00:11:00.700 Not especially-- although you always 00:11:00.700 --> 00:11:02.620 have to be really careful looking, 00:11:02.620 --> 00:11:05.950 because on any kind of plot, you want 00:11:05.950 --> 00:11:07.600 to really look at the spread. 00:11:07.600 --> 00:11:11.410 Here it's kind of magnified, because here's 0.7 inches, 00:11:11.410 --> 00:11:22.510 and we're down to about 0.697 up to 0.702. 00:11:22.510 --> 00:11:24.790 That's not full scale. 00:11:24.790 --> 00:11:28.460 But if we zoom in around whatever nominal is-- 00:11:28.460 --> 00:11:32.380 and in fact, it's not even clear what the target dimension was-- 00:11:32.380 --> 00:11:34.690 I might guess and think that it was 0.7 inches, 00:11:34.690 --> 00:11:38.410 because that's a nice round number, but who knows? 00:11:38.410 --> 00:11:40.780 So it's definitely not uniform. 00:11:40.780 --> 00:11:42.000 What else do you see? 00:11:42.000 --> 00:11:43.262 Yeah? 00:11:43.262 --> 00:11:46.542 AUDIENCE: The data is sort of drifting up over time. 00:11:46.542 --> 00:11:47.250 DUANE BONING: OK. 00:11:47.250 --> 00:11:50.910 So it looks like there might be a long-term trend of a drift 00:11:50.910 --> 00:11:54.840 over the number of runs. 00:11:54.840 --> 00:11:57.330 What might lead you to think that that would 00:11:57.330 --> 00:12:00.264 be a physically reasonable-- 00:12:00.264 --> 00:12:04.240 AUDIENCE: [INAUDIBLE] 00:12:07.277 --> 00:12:08.110 DUANE BONING: Right. 00:12:08.110 --> 00:12:12.240 So if the tool wears-- and in fact, if the tool wears 00:12:12.240 --> 00:12:16.380 or some long-term change in the state of the equipment-- 00:12:16.380 --> 00:12:22.220 that's often the source of these long-term trends. 00:12:22.220 --> 00:12:23.780 Other things people see-- 00:12:28.400 --> 00:12:29.210 Yeah? 00:12:29.210 --> 00:12:30.710 AUDIENCE: It's fairly systematic. 00:12:30.710 --> 00:12:33.110 It constantly goes up and down, and up and down. 00:12:33.110 --> 00:12:36.080 There's only two or three points before it goes back up. 00:12:36.080 --> 00:12:38.450 DUANE BONING: Yeah, that's very interesting. 00:12:38.450 --> 00:12:41.030 Even the variation here-- 00:12:41.030 --> 00:12:46.220 it seems to be cyclical, or periodic, or something 00:12:46.220 --> 00:12:48.980 systematic going on in there. 00:12:51.600 --> 00:12:54.270 Again, I wasn't there for the generation of this data, 00:12:54.270 --> 00:12:56.190 but what might some reasons for that be? 00:12:58.495 --> 00:12:59.995 AUDIENCE: The worker might be trying 00:12:59.995 --> 00:13:03.260 to correct with the subsequent one, 00:13:03.260 --> 00:13:05.467 and he overshoots constantly. 00:13:05.467 --> 00:13:06.800 DUANE BONING: Interesting-- yes. 00:13:09.990 --> 00:13:11.100 That's a good idea. 00:13:11.100 --> 00:13:15.000 I guess, in fact, one can imagine, especially 00:13:15.000 --> 00:13:17.670 if you're taking a measurement after every tool, 00:13:17.670 --> 00:13:25.240 and the target was 0.7, say, then maybe, near the beginning, 00:13:25.240 --> 00:13:26.917 we're starting to see a deviation. 00:13:26.917 --> 00:13:28.750 Maybe this one was a little low, and they're 00:13:28.750 --> 00:13:32.903 trying to get back up, making an overcorrection-- because 00:13:32.903 --> 00:13:35.320 near the end here, it looks like there's a little bit more 00:13:35.320 --> 00:13:37.464 centered around 0.7. 00:13:37.464 --> 00:13:39.089 AUDIENCE: I think that might be because 00:13:39.089 --> 00:13:41.570 of backlash [INAUDIBLE]. 00:13:44.922 --> 00:13:45.630 DUANE BONING: OK. 00:13:45.630 --> 00:13:49.220 AUDIENCE: [INAUDIBLE] 00:13:49.220 --> 00:13:53.030 DUANE BONING: So backlash being an operator adjustment 00:13:53.030 --> 00:13:56.620 that is made as perhaps part of this compensation strategy? 00:13:56.620 --> 00:13:57.500 AUDIENCE: [INAUDIBLE] 00:13:57.500 --> 00:13:58.625 DUANE BONING: Interesting-- 00:13:58.625 --> 00:14:00.593 AUDIENCE: [INAUDIBLE] 00:14:04.690 --> 00:14:06.190 DUANE BONING: Anybody in Singapore-- 00:14:06.190 --> 00:14:08.410 any other observations or ideas? 00:14:11.858 --> 00:14:14.180 AUDIENCE: Since measurement is measured 00:14:14.180 --> 00:14:17.070 at a different place of the things, 00:14:17.070 --> 00:14:22.090 so maybe you can see every three point 00:14:22.090 --> 00:14:24.500 will be a larger diameter. 00:14:24.500 --> 00:14:30.980 I think maybe this is measured at the other side 00:14:30.980 --> 00:14:33.050 of the thing [INAUDIBLE]. 00:14:33.050 --> 00:14:36.280 And maybe for the lower-- for the smaller diameter, 00:14:36.280 --> 00:14:39.320 it's measured in the inner side. 00:14:39.320 --> 00:14:41.600 DUANE BONING: Yes, it's quite possible-- 00:14:41.600 --> 00:14:42.960 at least on this plot. 00:14:42.960 --> 00:14:46.940 We might learn more if we look at more data. 00:14:46.940 --> 00:14:51.380 I read this as run number, rather than measurement number. 00:14:51.380 --> 00:14:55.520 So somehow, perhaps, this is intended 00:14:55.520 --> 00:14:57.620 to be representative of the whole part, 00:14:57.620 --> 00:15:00.590 but it's possible those are multiple measurements 00:15:00.590 --> 00:15:02.060 on different part-- 00:15:02.060 --> 00:15:04.520 portions of an overall turning. 00:15:07.190 --> 00:15:11.490 You could start to look and hypothesize on that. 00:15:11.490 --> 00:15:16.440 Is the 1, 2, 3-- is that the same pattern you see each time? 00:15:16.440 --> 00:15:19.610 And it's not clear whether we see exactly that same pattern 00:15:19.610 --> 00:15:21.140 or not. 00:15:21.140 --> 00:15:22.375 Other ideas? 00:15:22.375 --> 00:15:26.740 AUDIENCE: [INAUDIBLE] 00:15:29.857 --> 00:15:30.690 DUANE BONING: Right. 00:15:30.690 --> 00:15:35.440 So run number is a stand-in for time, and these are occurring. 00:15:35.440 --> 00:15:37.770 So your point here is-- 00:15:37.770 --> 00:15:38.910 AUDIENCE: [INAUDIBLE] 00:15:38.910 --> 00:15:39.702 DUANE BONING: Sure. 00:15:39.702 --> 00:15:40.597 AUDIENCE: [INAUDIBLE] 00:15:40.597 --> 00:15:41.430 DUANE BONING: Right. 00:15:41.430 --> 00:15:43.560 So it may not be equipment state, 00:15:43.560 --> 00:15:47.220 or indirectly, it may be also the rest of the facility 00:15:47.220 --> 00:15:49.470 environment or the line. 00:15:49.470 --> 00:15:51.370 And over time, who knows? 00:15:51.370 --> 00:15:53.760 Maybe this thing starts heating up, 00:15:53.760 --> 00:15:56.700 or the work pieces that are coming in 00:15:56.700 --> 00:15:58.560 have a different temperature-- 00:15:58.560 --> 00:16:02.880 quite possible that there could be some other long-term state 00:16:02.880 --> 00:16:05.040 change. 00:16:05.040 --> 00:16:08.760 We do notice there's this shift change. 00:16:08.760 --> 00:16:13.770 And referring back to this long-term trend, 00:16:13.770 --> 00:16:17.520 if we actually look at this early part, 00:16:17.520 --> 00:16:20.550 it may or may not be drifting that much. 00:16:20.550 --> 00:16:22.440 It may actually be fairly steady. 00:16:22.440 --> 00:16:24.570 And what we're actually seeing in here 00:16:24.570 --> 00:16:29.010 is an overall mean shift, but maybe not even a long-term 00:16:29.010 --> 00:16:29.710 trend. 00:16:29.710 --> 00:16:33.600 So one might actually pose, maybe even apply 00:16:33.600 --> 00:16:36.370 some statistical methods to look and say, 00:16:36.370 --> 00:16:39.480 is there a statistically significant mean difference 00:16:39.480 --> 00:16:44.020 between this set and this set of data? 00:16:44.020 --> 00:16:47.040 So it may look like a long-term trend, 00:16:47.040 --> 00:16:50.670 or it may simply be a reflection that there 00:16:50.670 --> 00:16:54.360 is a setup change on the equipment when 00:16:54.360 --> 00:16:56.160 a new shift comes in, or there may 00:16:56.160 --> 00:16:59.850 be just inherent differences in how the operator interacts 00:16:59.850 --> 00:17:01.050 with the equipment. 00:17:01.050 --> 00:17:04.202 How they load the part might be slightly different. 00:17:08.270 --> 00:17:11.470 Any other ideas? 00:17:11.470 --> 00:17:12.525 Let's go back to the-- 00:17:12.525 --> 00:17:13.900 because I think you had mentioned 00:17:13.900 --> 00:17:20.170 something interesting about possible oscillation in here. 00:17:20.170 --> 00:17:26.680 What might other sources or causes of oscillatory behavior, 00:17:26.680 --> 00:17:34.290 or what looks like two distinct sets of data here-- 00:17:34.290 --> 00:17:37.447 in fact, I'm not completely sure that it's purely oscillatory, 00:17:37.447 --> 00:17:39.030 because you can come down here, and it 00:17:39.030 --> 00:17:42.150 seems like sometimes you've got two things that 00:17:42.150 --> 00:17:46.380 are down low, and then one up, or three things that are low, 00:17:46.380 --> 00:17:47.250 and one up. 00:17:47.250 --> 00:17:49.250 AUDIENCE: Well, this is a little bit farfetched, 00:17:49.250 --> 00:17:51.570 but maybe you could have [? stock ?] material 00:17:51.570 --> 00:17:55.260 of maybe 1 meter, and they always 00:17:55.260 --> 00:17:57.310 cut it into three pieces. 00:17:57.310 --> 00:18:00.390 So they always have the first piece of stock material, 00:18:00.390 --> 00:18:03.630 and they had some material properties that 00:18:03.630 --> 00:18:06.240 led to have [INAUDIBLE] maybe. 00:18:06.240 --> 00:18:08.550 And the middle piece led to middle dimensions, 00:18:08.550 --> 00:18:11.520 and the last piece led to smaller dimensions. 00:18:11.520 --> 00:18:13.270 DUANE BONING: I like that. 00:18:13.270 --> 00:18:14.980 That might be very interesting. 00:18:14.980 --> 00:18:19.050 So if I were generalizing that, it may well be-- 00:18:19.050 --> 00:18:22.920 very often, there may be more than one set 00:18:22.920 --> 00:18:25.200 of starting material. 00:18:25.200 --> 00:18:28.600 So it may have been all one piece and cut into three, 00:18:28.600 --> 00:18:31.500 and so they're randomly picking one of the three, 00:18:31.500 --> 00:18:34.170 and therefore, you're getting a different mix of those three 00:18:34.170 --> 00:18:36.330 points in here. 00:18:36.330 --> 00:18:39.420 But in general, it may well be in many cases 00:18:39.420 --> 00:18:45.090 that you've got two different incoming streams. 00:18:45.090 --> 00:18:48.950 If I were looking at this, instead of thinking of three, 00:18:48.950 --> 00:18:54.240 it kind of almost looks to me like this set of data here-- 00:18:54.240 --> 00:18:57.060 it's kind of randomly around some value, 00:18:57.060 --> 00:19:00.850 and this is a different set of data around some other value. 00:19:00.850 --> 00:19:05.010 So it may be that there's just two boxes apart sitting there, 00:19:05.010 --> 00:19:08.340 and they're just pulling, randomly, starting material out 00:19:08.340 --> 00:19:10.060 of one of the two. 00:19:10.060 --> 00:19:11.100 Yeah? 00:19:11.100 --> 00:19:12.683 AUDIENCE: I also don't know if there's 00:19:12.683 --> 00:19:15.455 any control on how far from the [INAUDIBLE] the materials 00:19:15.455 --> 00:19:18.270 [INAUDIBLE] you could put it in, be a little bit farther, 00:19:18.270 --> 00:19:20.220 and then it would have a larger cantilever 00:19:20.220 --> 00:19:22.200 arm than the next user. 00:19:22.200 --> 00:19:23.390 DUANE BONING: Right, right. 00:19:23.390 --> 00:19:24.810 So there could be lots of sources 00:19:24.810 --> 00:19:29.630 of both random and deterministic variations. 00:19:29.630 --> 00:19:34.980 So we talked in here about things like a mean shift. 00:19:34.980 --> 00:19:38.850 Maybe there's a deviation between these two shift 00:19:38.850 --> 00:19:39.400 changes. 00:19:39.400 --> 00:19:44.580 And that would be a either deterministic or-- 00:19:44.580 --> 00:19:49.290 systematic maybe is a better word of a deviation. 00:19:49.290 --> 00:19:52.080 And then, within different sets, we 00:19:52.080 --> 00:19:56.550 may have no detailed explanation of why 00:19:56.550 --> 00:19:59.970 we have these small deviations, and a random model 00:19:59.970 --> 00:20:02.850 for some of those components might be appropriate. 00:20:02.850 --> 00:20:06.460 So that's part of the idea that we're after. 00:20:06.460 --> 00:20:13.980 So over here on the side, we're just referring back again to 00:20:13.980 --> 00:20:15.300 of our variation equation. 00:20:15.300 --> 00:20:18.803 Maybe I'll put that up so that we can also 00:20:18.803 --> 00:20:20.970 refer to that, because I think we've touched on some 00:20:20.970 --> 00:20:24.906 of these different components. 00:20:24.906 --> 00:20:28.080 We said that our deviations in our output 00:20:28.080 --> 00:20:33.510 y you can often characterize as some component that 00:20:33.510 --> 00:20:39.060 has a sensitivity to disturbances in our process 00:20:39.060 --> 00:20:44.250 parameters, and those disturbances, 00:20:44.250 --> 00:20:50.370 and some component that relates to our controllability 00:20:50.370 --> 00:20:53.370 when we make small changes [INAUDIBLE] 00:20:53.370 --> 00:20:55.230 and then control inputs. 00:20:58.220 --> 00:21:00.460 Among other things, we said, well, 00:21:00.460 --> 00:21:04.320 maybe some of our input feedstock, maybe delta a's-- 00:21:04.320 --> 00:21:06.547 changes in the parameters. 00:21:06.547 --> 00:21:08.880 We also-- and I thought it was very interesting, talking 00:21:08.880 --> 00:21:13.080 about overcompensation in the control scenario. 00:21:13.080 --> 00:21:14.640 It might be, in fact, that there are 00:21:14.640 --> 00:21:19.928 deviations or errors in active control, changes 00:21:19.928 --> 00:21:20.595 to the settings. 00:21:23.790 --> 00:21:26.280 There may also be-- and we also talked about it here-- 00:21:26.280 --> 00:21:35.400 is recognition that the function y of our alpha may, in fact, 00:21:35.400 --> 00:21:42.800 be time-dependent, or that-- 00:21:42.800 --> 00:21:44.190 put it another way-- 00:21:44.190 --> 00:21:49.760 some of the delta a's have time components and time trends 00:21:49.760 --> 00:21:52.460 to them. 00:21:52.460 --> 00:21:56.270 We'll come back, and I'm sure come up with more examples 00:21:56.270 --> 00:21:58.680 as we look a little bit further. 00:21:58.680 --> 00:22:01.850 And I think this is some of the same process. 00:22:01.850 --> 00:22:07.880 It has different run numbers on it, but this is explicitly-- 00:22:07.880 --> 00:22:10.370 it looks like there's, again, a dimension and run 00:22:10.370 --> 00:22:12.870 number along the horizontal axis, but in this case, 00:22:12.870 --> 00:22:16.940 there's the yellow, our inner; the purple, our middle; 00:22:16.940 --> 00:22:20.310 and the blue, our outer dimensions. 00:22:20.310 --> 00:22:23.180 So you can imagine here that perhaps measurements 00:22:23.180 --> 00:22:30.160 are being made at different locations on the work piece. 00:22:30.160 --> 00:22:34.650 And I believe that this line right here 00:22:34.650 --> 00:22:38.970 is, again, a shift change. 00:22:38.970 --> 00:22:40.920 So talk to me a little bit about this. 00:22:40.920 --> 00:22:44.610 What else do we notice in this case? 00:22:44.610 --> 00:22:49.070 There's one important difference in this data compared 00:22:49.070 --> 00:22:54.530 to this data that I'm looking for in particular. 00:22:54.530 --> 00:22:57.350 What additional characteristics-- or what else 00:22:57.350 --> 00:23:00.170 could you say about this data? 00:23:00.170 --> 00:23:04.510 AUDIENCE: [INAUDIBLE] I guess the spread on the [INAUDIBLE].. 00:23:11.512 --> 00:23:12.220 DUANE BONING: OK. 00:23:12.220 --> 00:23:16.150 So one observation is, if we were just looking-- 00:23:16.150 --> 00:23:18.130 well, right here, this is a good one, 00:23:18.130 --> 00:23:20.860 where inner, middle, and outer are all close together. 00:23:20.860 --> 00:23:23.200 But your point is, generally, we're 00:23:23.200 --> 00:23:27.770 seeing a very huge spread in the measurement 00:23:27.770 --> 00:23:33.590 within that particular part than we do over here. 00:23:33.590 --> 00:23:36.460 AUDIENCE: [INAUDIBLE] 00:23:41.400 --> 00:23:42.930 DUANE BONING: Much tighter band-- 00:23:42.930 --> 00:23:45.693 so now you're talking like a band here? 00:23:45.693 --> 00:23:49.410 AUDIENCE: Yeah, that [INAUDIBLE].. 00:23:49.410 --> 00:23:51.700 DUANE BONING: Good, good-- 00:23:51.700 --> 00:23:52.540 yep? 00:23:52.540 --> 00:23:54.460 AUDIENCE: [INAUDIBLE] 00:24:07.405 --> 00:24:08.780 DUANE BONING: Interesting-- yeah, 00:24:08.780 --> 00:24:11.540 so it's a situation here where we don't know 00:24:11.540 --> 00:24:15.950 the specifications, so it's a little hard to actually know 00:24:15.950 --> 00:24:17.000 which is better-- 00:24:17.000 --> 00:24:19.950 because your point you were making is, in this case, 00:24:19.950 --> 00:24:22.160 the blue's always above-- 00:24:22.160 --> 00:24:24.860 or almost always above the purple, 00:24:24.860 --> 00:24:25.940 always above the yellow. 00:24:25.940 --> 00:24:28.190 And it may well be that it's meant 00:24:28.190 --> 00:24:30.620 to have a slight taper to it. 00:24:30.620 --> 00:24:31.505 We'd need to more. 00:24:31.505 --> 00:24:33.380 And actually, that's a really important point 00:24:33.380 --> 00:24:35.310 in manufacturing processes. 00:24:35.310 --> 00:24:37.580 You can't just look at the data and always know 00:24:37.580 --> 00:24:38.310 what's going on. 00:24:38.310 --> 00:24:40.880 You also have to have information from design, 00:24:40.880 --> 00:24:46.190 know what the intended results are. 00:24:46.190 --> 00:24:49.310 So certainly, just to highlight a couple 00:24:49.310 --> 00:24:55.730 of the things [INAUDIBLE] mentioned, 00:24:55.730 --> 00:24:58.580 one here is, very often, we are also 00:24:58.580 --> 00:25:02.700 interested in within part variation. 00:25:02.700 --> 00:25:04.820 So assuming that maybe they all really 00:25:04.820 --> 00:25:06.740 were supposed to have the same dimension, 00:25:06.740 --> 00:25:11.780 then this spread here is very different than this 00:25:11.780 --> 00:25:12.530 spread here. 00:25:12.530 --> 00:25:15.770 So characterization of within part variation 00:25:15.770 --> 00:25:18.170 may be very important in understanding that. 00:25:20.870 --> 00:25:25.390 And then the other thing here is, unlike this-- 00:25:25.390 --> 00:25:27.220 we still have all that gook on here, 00:25:27.220 --> 00:25:31.960 but in here we were basically having a similar band, 00:25:31.960 --> 00:25:35.590 and it was all just kind of moving or shifting. 00:25:35.590 --> 00:25:39.730 But the part-to-part variation did not seem to change. 00:25:39.730 --> 00:25:42.740 The variance of the process did not seem to change. 00:25:42.740 --> 00:25:44.770 The mean shifted. 00:25:44.770 --> 00:25:46.690 So that's a very common characteristic 00:25:46.690 --> 00:25:48.080 that we're worried about. 00:25:48.080 --> 00:25:51.680 And your point here is that, in this case, 00:25:51.680 --> 00:25:55.960 this looks like a much tighter band, or the standard deviation 00:25:55.960 --> 00:26:00.640 of the invariance of the process in a run-to-run sense 00:26:00.640 --> 00:26:04.080 is very different than in this process. 00:26:04.080 --> 00:26:07.040 And so it may be somebody with a slightly steadier hands, 00:26:07.040 --> 00:26:11.540 or a more consistent process, or a more consistent operator, 00:26:11.540 --> 00:26:14.660 or the equipment-- maybe some change was made-- 00:26:14.660 --> 00:26:16.610 improvement to the equipment saying, 00:26:16.610 --> 00:26:19.640 hm, that little screw is a little loose. 00:26:19.640 --> 00:26:22.160 Let me tighten that down. 00:26:22.160 --> 00:26:23.120 Who knows? 00:26:23.120 --> 00:26:28.980 But something clearly changed in order 00:26:28.980 --> 00:26:31.973 to apparently improve the process. 00:26:31.973 --> 00:26:33.765 So those are a couple more characteristics. 00:26:33.765 --> 00:26:34.640 AUDIENCE: [INAUDIBLE] 00:26:34.640 --> 00:26:35.870 DUANE BONING: Yes-- question? 00:26:35.870 --> 00:26:36.745 AUDIENCE: Professor-- 00:26:36.745 --> 00:26:37.650 DUANE BONING: Yes? 00:26:37.650 --> 00:26:39.770 AUDIENCE: Since you mentioned that maybe 00:26:39.770 --> 00:26:42.740 with the shift change there's some improvement 00:26:42.740 --> 00:26:45.350 in the [INAUDIBLE] of data, and that's 00:26:45.350 --> 00:26:49.390 why it gives us a smaller band right now. 00:26:49.390 --> 00:26:53.510 But I noticed that the outer, middle, and inner data-- 00:26:53.510 --> 00:26:56.710 they are pretty close to each other, 00:26:56.710 --> 00:26:59.600 as in the difference between these three sets of data 00:26:59.600 --> 00:27:03.320 are much wider in the previous shift than in this right now. 00:27:03.320 --> 00:27:08.300 So can we still say that maybe this-- 00:27:08.300 --> 00:27:11.870 the object we are measuring is tapered? 00:27:11.870 --> 00:27:14.780 Because it doesn't look like it's tapered right now. 00:27:14.780 --> 00:27:18.740 DUANE BONING: Absolutely-- well, but the point 00:27:18.740 --> 00:27:22.770 was made that we don't actually know the design intent, 00:27:22.770 --> 00:27:24.710 so this was a hypothesis in here. 00:27:24.710 --> 00:27:27.740 I think it's probably unlikely that there 00:27:27.740 --> 00:27:30.890 was an intended taper, and I'll agree with you. 00:27:30.890 --> 00:27:35.450 But in here, it looks like there is 00:27:35.450 --> 00:27:40.430 somewhat closer within part correspondence 00:27:40.430 --> 00:27:43.560 in these dimensions. 00:27:43.560 --> 00:27:46.848 OK, so let me go to one more. 00:27:46.848 --> 00:27:47.640 Here's another one. 00:27:47.640 --> 00:27:51.220 This is brake bending of a metal sheet. 00:27:51.220 --> 00:27:53.610 The basic idea in this process is 00:27:53.610 --> 00:27:57.840 we're trying to change the geometry under the application 00:27:57.840 --> 00:28:02.310 of force in a bending process. 00:28:02.310 --> 00:28:05.660 So we might have some tooling. 00:28:05.660 --> 00:28:09.980 We have some piece of metal, say, and we press down, 00:28:09.980 --> 00:28:13.880 and the intent is to create some permanent deformation 00:28:13.880 --> 00:28:14.990 in the part. 00:28:14.990 --> 00:28:16.940 An important component here is that, when 00:28:16.940 --> 00:28:20.700 we remove that force, remove that pressure, 00:28:20.700 --> 00:28:25.120 there can be spring back in that part-- 00:28:25.120 --> 00:28:31.040 so little bit of the inherent characteristics of this. 00:28:31.040 --> 00:28:35.320 And so here's some data looking at using the same tooling, 00:28:35.320 --> 00:28:42.880 and I believe the same set of downforce, with-- 00:28:42.880 --> 00:28:48.420 I believe this is a measurement of the resulting angle. 00:28:48.420 --> 00:28:50.670 Alpha is not a good variable to use here, is it? 00:28:53.790 --> 00:29:02.790 We call that just angle of the resulting component. 00:29:02.790 --> 00:29:06.660 And what else is changing here is that the source material 00:29:06.660 --> 00:29:08.170 is changing. 00:29:08.170 --> 00:29:12.630 So in some cases, we're using aluminum, some cases steel-- 00:29:12.630 --> 00:29:16.900 and also with different thicknesses. 00:29:16.900 --> 00:29:20.520 So I won't even ask you to share-- 00:29:20.520 --> 00:29:26.040 the most obvious observation is, depending on the material type, 00:29:26.040 --> 00:29:31.050 we certainly get different angles, different degrees 00:29:31.050 --> 00:29:33.120 of spring back. 00:29:36.580 --> 00:29:37.870 That's the most obvious thing. 00:29:37.870 --> 00:29:41.200 What else do you see in this? 00:29:41.200 --> 00:29:42.280 Yeah? 00:29:42.280 --> 00:29:45.142 AUDIENCE: It depends on the thickness of the material. 00:29:45.142 --> 00:29:45.850 DUANE BONING: OK. 00:29:48.610 --> 00:29:51.040 AUDIENCE: So for the same material [INAUDIBLE] 00:29:51.040 --> 00:29:52.570 the springback, it will be more. 00:29:55.330 --> 00:29:56.470 DUANE BONING: Right. 00:29:56.470 --> 00:30:00.130 OK, and one could certainly imagine good physical reasons 00:30:00.130 --> 00:30:04.870 for that, that thicker material is 00:30:04.870 --> 00:30:11.590 going to perhaps spring back more readily. 00:30:11.590 --> 00:30:13.810 And one could even go off and imagine 00:30:13.810 --> 00:30:17.380 trying to build a first principle physical model that 00:30:17.380 --> 00:30:20.140 would tell you what the resulting angle 00:30:20.140 --> 00:30:24.080 or degree of spring back would be as a function of that. 00:30:24.080 --> 00:30:26.440 And in fact, it's probably worth it 00:30:26.440 --> 00:30:28.150 to go and consult some literature 00:30:28.150 --> 00:30:32.320 and see if there is physical insight. 00:30:32.320 --> 00:30:36.850 But if I were to ask you to actually tell me what you think 00:30:36.850 --> 00:30:44.660 the result might be if we had a 0.45-inch steel part, 00:30:44.660 --> 00:30:47.090 what would your first inclination be? 00:30:47.090 --> 00:30:50.540 Would it be to go to the technical literature 00:30:50.540 --> 00:30:52.820 and look up a physical model for this? 00:30:57.510 --> 00:31:01.080 AUDIENCE: [INAUDIBLE] averaging between the [INAUDIBLE].. 00:31:01.080 --> 00:31:02.500 DUANE BONING: Yeah. 00:31:02.500 --> 00:31:04.870 You would be basically using some averaging 00:31:04.870 --> 00:31:07.970 to deal with noise and manufacturing variation, 00:31:07.970 --> 00:31:12.370 but you would be building a very simple empirical model based 00:31:12.370 --> 00:31:16.090 on the data that you have, and then interpolating in some way. 00:31:16.090 --> 00:31:18.230 And here we've only got two data points, 00:31:18.230 --> 00:31:20.890 so you might pick the middle. 00:31:20.890 --> 00:31:22.030 We don't know. 00:31:22.030 --> 00:31:25.090 You might want to actually go and design some experiment 00:31:25.090 --> 00:31:27.840 where you add a third point so you 00:31:27.840 --> 00:31:31.070 know if there's some non-linear dependence, and so on. 00:31:31.070 --> 00:31:33.410 So that's certainly an important thing. 00:31:33.410 --> 00:31:35.560 There is a very clear input-output, 00:31:35.560 --> 00:31:40.180 a deterministic effect that, again, you 00:31:40.180 --> 00:31:42.730 would like to be able to understand enough to be 00:31:42.730 --> 00:31:46.160 able to deal with that. 00:31:46.160 --> 00:31:51.090 And the same may also be true of other discrete design choices-- 00:31:51.090 --> 00:31:53.408 discrete choice being the material, 00:31:53.408 --> 00:31:54.700 whether it's aluminum or steel. 00:31:54.700 --> 00:31:58.895 You would like to know, is it a similar trend? 00:31:58.895 --> 00:32:00.270 One thing that's very interesting 00:32:00.270 --> 00:32:03.850 here is it appears that the delta 00:32:03.850 --> 00:32:05.500 as a function of those thicknesses 00:32:05.500 --> 00:32:08.830 is about the same-- very nearly the same 00:32:08.830 --> 00:32:11.300 in the two material cases. 00:32:11.300 --> 00:32:14.470 And then there's a material delta. 00:32:14.470 --> 00:32:16.850 So I'm already starting to think, OK, 00:32:16.850 --> 00:32:21.990 I can have a very simple additive empirical model that 00:32:21.990 --> 00:32:25.960 has a delta effect due to the material type. 00:32:25.960 --> 00:32:28.270 It's a binary choice-- 00:32:28.270 --> 00:32:30.550 or a binary coefficient, depending on that. 00:32:30.550 --> 00:32:33.690 And then an additional component is a function of thickness, 00:32:33.690 --> 00:32:36.960 and maybe that function of the thickness is independent. 00:32:36.960 --> 00:32:40.080 That delta from amine is independent [INAUDIBLE] 00:32:40.080 --> 00:32:42.140 material type. 00:32:42.140 --> 00:32:44.620 OK, anything else going on? 00:32:44.620 --> 00:32:45.375 Yes? 00:32:45.375 --> 00:32:51.260 AUDIENCE: [INAUDIBLE] there is kind 00:32:51.260 --> 00:32:53.248 of a drop in the middle of the set. 00:32:55.996 --> 00:32:56.820 [INAUDIBLE] 00:33:02.440 --> 00:33:04.240 DUANE BONING: Yes-- very interesting, 00:33:04.240 --> 00:33:07.480 and maybe a hint almost of something, but then it-- 00:33:07.480 --> 00:33:08.020 yeah. 00:33:08.020 --> 00:33:11.810 So there might be something systematic going on there. 00:33:11.810 --> 00:33:14.500 The human eye is wonderful for looking for patterns, isn't it? 00:33:17.010 --> 00:33:18.750 Actually, one of the challenges often 00:33:18.750 --> 00:33:23.640 is using both engineering judgment and when 00:33:23.640 --> 00:33:27.030 the data is simple enough to detect trends, but also 00:33:27.030 --> 00:33:30.930 have statistical methods when the volume of data is huge 00:33:30.930 --> 00:33:33.270 or there are many things changing at the same time 00:33:33.270 --> 00:33:35.880 to detect possible shifts and trends. 00:33:38.990 --> 00:33:39.760 Anything else? 00:33:44.380 --> 00:33:45.570 Yeah, you had more. 00:33:45.570 --> 00:33:47.388 AUDIENCE: OK. 00:33:47.388 --> 00:33:49.810 [INAUDIBLE] 00:33:49.810 --> 00:33:50.560 DUANE BONING: Yes. 00:33:50.560 --> 00:33:53.871 AUDIENCE: [INAUDIBLE] 00:33:57.763 --> 00:34:00.180 DUANE BONING: So you're worried about this, and maybe this 00:34:00.180 --> 00:34:00.660 also. 00:34:00.660 --> 00:34:00.750 AUDIENCE: Yeah. 00:34:00.750 --> 00:34:02.250 DUANE BONING: In fact, we don't even 00:34:02.250 --> 00:34:06.090 know if that's a still component or an aluminum. 00:34:06.090 --> 00:34:07.290 Yeah. 00:34:07.290 --> 00:34:10.590 So in some sense here, if I were looking at this data, 00:34:10.590 --> 00:34:14.940 I almost start to think these are relatively-- 00:34:14.940 --> 00:34:18.810 well, you'd have to ask as a function in comparison 00:34:18.810 --> 00:34:20.940 to specifications to actually say 00:34:20.940 --> 00:34:26.850 how well-controlled that is, but there is relatively tight bands 00:34:26.850 --> 00:34:36.239 with a couple of these shifts, and then almost outlier points. 00:34:36.239 --> 00:34:38.100 Now, would you think that these points 00:34:38.100 --> 00:34:42.659 are coming from the same source of deviation that 00:34:42.659 --> 00:34:47.270 might be at work in this band? 00:34:47.270 --> 00:34:52.659 Might be-- but it's such a large deviation that it 00:34:52.659 --> 00:34:54.730 starts to feel unlikely. 00:34:54.730 --> 00:34:56.620 And another use of statistical methods 00:34:56.620 --> 00:34:59.200 will actually be to quantify, how likely is it 00:34:59.200 --> 00:35:02.080 that we would observe by chance alone 00:35:02.080 --> 00:35:07.750 a deviation within the natural variation band of such a point? 00:35:10.700 --> 00:35:13.820 And the key reason is, if you see something big like that, 00:35:13.820 --> 00:35:15.740 there might be a [? point cause. ?] 00:35:15.740 --> 00:35:19.220 And it's quite possible in here that the operator is 00:35:19.220 --> 00:35:21.980 making a measurement on each part and said, whoops-- 00:35:21.980 --> 00:35:23.660 something just happened here. 00:35:28.050 --> 00:35:29.760 Something got misadjusted on the tool. 00:35:29.760 --> 00:35:32.640 I need to readjust it and get it back in. 00:35:32.640 --> 00:35:36.180 Maybe there's even a little bit of something going on there. 00:35:36.180 --> 00:35:39.180 Or it might have been a single event that 00:35:39.180 --> 00:35:41.250 wasn't-- didn't require adjustment. 00:35:41.250 --> 00:35:42.840 Maybe you go in, you investigate, 00:35:42.840 --> 00:35:46.740 and you go, that one piece of source material coming in, 00:35:46.740 --> 00:35:50.940 that piece of metal coming in was much thicker 00:35:50.940 --> 00:35:53.160 than it was intended to be. 00:35:53.160 --> 00:35:55.882 AUDIENCE: [INAUDIBLE] 00:35:57.300 --> 00:35:59.760 DUANE BONING: Yeah, maybe that's a piece of aluminum right 00:35:59.760 --> 00:36:02.430 in there, or this was-- exactly-- 00:36:02.430 --> 00:36:06.030 this was a steel [? 0.3 ?] in the aluminum set, 00:36:06.030 --> 00:36:09.390 or steel [? 0.3 ?] in the first one of those. 00:36:09.390 --> 00:36:12.900 Absolutely-- that's a cool observation. 00:36:12.900 --> 00:36:15.570 Yeah, that point seems to fit down in that distribution, 00:36:15.570 --> 00:36:16.650 doesn't it? 00:36:16.650 --> 00:36:18.810 Very interesting-- so a couple of the points 00:36:18.810 --> 00:36:21.450 here-- the kinds of things we might be looking for here 00:36:21.450 --> 00:36:23.970 are deterministic and systematic kinds 00:36:23.970 --> 00:36:27.030 of effects that we might want to use modeling for. 00:36:27.030 --> 00:36:29.010 And then we also are starting to get 00:36:29.010 --> 00:36:33.600 close to the statistical process control thinking, which 00:36:33.600 --> 00:36:37.020 is to say we would like to be able to know what 00:36:37.020 --> 00:36:41.130 the inherent or natural variation of the process 00:36:41.130 --> 00:36:44.010 is having to do with these sorts of bands, 00:36:44.010 --> 00:36:47.630 and then be able to detect with high degree of confidence 00:36:47.630 --> 00:36:49.760 that something strange has happened, 00:36:49.760 --> 00:36:51.890 and I better take action. 00:36:51.890 --> 00:36:55.070 And that SPC, or statistical process control. 00:36:58.165 --> 00:37:00.790 Maybe we're going to see many of the same characteristics here. 00:37:00.790 --> 00:37:03.070 This is just a little observation 00:37:03.070 --> 00:37:05.720 from injection molding. 00:37:05.720 --> 00:37:09.370 So this is a plastic molded part. 00:37:11.900 --> 00:37:15.200 We've got a wit, the part along this dimension, 00:37:15.200 --> 00:37:18.590 and down here we've got a number of the run. 00:37:18.590 --> 00:37:20.000 And also notice-- 00:37:20.000 --> 00:37:21.860 I'll just draw your attention here-- 00:37:21.860 --> 00:37:25.580 there are clear intentional bands here, 00:37:25.580 --> 00:37:28.980 where this first segment here says holding time 00:37:28.980 --> 00:37:32.630 was five seconds, injection press 40%. 00:37:32.630 --> 00:37:34.760 And we're changing here-- this is 00:37:34.760 --> 00:37:36.907 a holding time of 10 seconds. 00:37:36.907 --> 00:37:38.990 Here's another holding [INAUDIBLE] of five seconds 00:37:38.990 --> 00:37:40.640 and 60%-- 00:37:40.640 --> 00:37:42.200 and whatever is lurking under there. 00:37:45.092 --> 00:37:49.870 I think it's a holding time of 10 seconds. 00:37:49.870 --> 00:37:53.080 We could find out, but-- 00:37:53.080 --> 00:37:55.470 talk to me about this one. 00:37:55.470 --> 00:37:57.570 Oh, I guess also here, notice, we're 00:37:57.570 --> 00:38:00.360 plotting two pieces of data. 00:38:00.360 --> 00:38:04.150 We've got a width and then some average. 00:38:04.150 --> 00:38:06.420 So it's quite possible that what we're doing here 00:38:06.420 --> 00:38:09.690 is this is a point width at a well-known location, 00:38:09.690 --> 00:38:12.270 and than the average is we take that measurement 00:38:12.270 --> 00:38:17.040 on many, many locations around the part. 00:38:17.040 --> 00:38:21.330 Or it's possible we also actually run multiple parts, 00:38:21.330 --> 00:38:24.810 little mini batches of-- who knows-- maybe five parts, 00:38:24.810 --> 00:38:30.000 and I'm plotting the average across that little batch-- 00:38:30.000 --> 00:38:33.390 that five-part batch, which is another very common thing 00:38:33.390 --> 00:38:38.783 to do in SPC. So tell me a little bit about this. 00:38:38.783 --> 00:38:39.450 What do you see? 00:38:39.450 --> 00:38:40.170 Yeah? 00:38:40.170 --> 00:38:43.781 AUDIENCE: [INAUDIBLE] 00:38:50.268 --> 00:38:51.060 DUANE BONING: Yeah. 00:38:51.060 --> 00:38:55.500 So you're talking about this band here to here. 00:38:55.500 --> 00:38:57.210 Right. 00:38:57.210 --> 00:38:59.400 In fact, this might be the result 00:38:59.400 --> 00:39:02.730 of a designed experiment, where somebody 00:39:02.730 --> 00:39:07.230 is trying to explore the effective different process 00:39:07.230 --> 00:39:11.380 conditions, these four different cases. 00:39:11.380 --> 00:39:13.930 And part of the goal of that may be 00:39:13.930 --> 00:39:18.130 to identify not only things that are closer to a mean target, 00:39:18.130 --> 00:39:19.840 but are also more robust-- 00:39:19.840 --> 00:39:25.930 inherently robust, meaning they have a smaller delta a, 00:39:25.930 --> 00:39:30.310 or are the process itself has a smaller 00:39:30.310 --> 00:39:36.400 sensitivity among at that operating point to whatever 00:39:36.400 --> 00:39:39.370 inherent disturbance, whether it be temperature deviations 00:39:39.370 --> 00:39:40.880 or what have you. 00:39:40.880 --> 00:39:44.490 So that's a good observation. 00:39:44.490 --> 00:39:50.890 It's also kind of amazing to me here that the average looks 00:39:50.890 --> 00:39:54.700 rock steady in these cases, but these individual width 00:39:54.700 --> 00:39:58.576 measurements have a substantial deviation. 00:39:58.576 --> 00:40:01.212 AUDIENCE: Is that the average of all [INAUDIBLE]?? 00:40:01.212 --> 00:40:02.670 DUANE BONING: We don't really know. 00:40:02.670 --> 00:40:03.290 I don't know. 00:40:09.900 --> 00:40:12.730 Yes, it's quite possible that this is just a plot. 00:40:12.730 --> 00:40:14.340 So that's your point. 00:40:14.340 --> 00:40:19.710 This may be [INAUDIBLE] suspiciously so, right? 00:40:19.710 --> 00:40:22.560 This may be the average across that ensemble, 00:40:22.560 --> 00:40:24.570 and all that is doing it's not saying 00:40:24.570 --> 00:40:26.490 it's an average of multiple measurements 00:40:26.490 --> 00:40:29.910 or multiple samples, but that's just highlighting 00:40:29.910 --> 00:40:31.850 that there's this delta. 00:40:31.850 --> 00:40:33.570 And that's got to be it, because-- 00:40:33.570 --> 00:40:35.070 exactly. 00:40:35.070 --> 00:40:37.910 There's zero deviation in those average points. 00:40:41.930 --> 00:40:44.690 OK, but these are some of the same kinds of themes. 00:40:44.690 --> 00:40:50.150 We're, again, looking at offsets from deterministic process 00:40:50.150 --> 00:40:50.750 conditions. 00:40:50.750 --> 00:40:54.260 We're looking at inherent variation, ranges. 00:40:54.260 --> 00:40:56.450 We may be doing design experiments. 00:40:56.450 --> 00:40:59.270 You could start to think of empirical models that 00:40:59.270 --> 00:41:05.870 would help to guide us to an optimal point, the point being 00:41:05.870 --> 00:41:08.690 there that maybe these four conditions give you 00:41:08.690 --> 00:41:12.260 a little bit of an exploration across two different process 00:41:12.260 --> 00:41:13.700 parameters. 00:41:13.700 --> 00:41:20.840 One approach would be maybe your target is 40.85 or 40.87. 00:41:20.840 --> 00:41:24.170 You might pick the one that was closest to it. 00:41:24.170 --> 00:41:26.530 But if you actually built a model, including 00:41:26.530 --> 00:41:28.690 a model with continuous parameters, 00:41:28.690 --> 00:41:34.252 you could then interpolate and drive even closer to a target. 00:41:34.252 --> 00:41:35.710 Yeah-- another question or comment? 00:41:35.710 --> 00:41:39.908 AUDIENCE: [INAUDIBLE] 00:41:39.908 --> 00:41:41.450 DUANE BONING: Right, one would guess. 00:41:41.450 --> 00:41:44.697 AUDIENCE: [INAUDIBLE] 00:41:44.697 --> 00:41:45.530 DUANE BONING: Right. 00:41:45.530 --> 00:41:49.290 AUDIENCE: [INAUDIBLE] 00:42:01.680 --> 00:42:02.430 DUANE BONING: Yes. 00:42:02.430 --> 00:42:04.372 So that's actually a very interesting point. 00:42:04.372 --> 00:42:06.330 When we start talking a little about more about 00:42:06.330 --> 00:42:11.490 design of experiments, about the synergistic effects, 00:42:11.490 --> 00:42:13.890 it may not be a simple additive model. 00:42:13.890 --> 00:42:18.910 You're saying, as we go here, from 5 to 10, 00:42:18.910 --> 00:42:20.350 we got an increase. 00:42:20.350 --> 00:42:23.330 Here, when we went to 5 to 10, we got a decrease. 00:42:23.330 --> 00:42:26.560 So the effect of holding time may be completely 00:42:26.560 --> 00:42:30.158 in the opposite direction at two different pressures-- 00:42:32.800 --> 00:42:34.150 very good point. 00:42:34.150 --> 00:42:36.420 So that means there is some strong interaction 00:42:36.420 --> 00:42:39.360 between those two-- or mutual dependence between those two 00:42:39.360 --> 00:42:40.442 parameters. 00:42:44.300 --> 00:42:47.090 Man, we're getting very abstract in this data. 00:42:47.090 --> 00:42:50.210 We've got measurements on the output and we've got-- 00:42:50.210 --> 00:42:53.712 some number of run number on the side. 00:42:53.712 --> 00:42:56.170 This is actually looking very similar to some other things. 00:42:56.170 --> 00:42:58.230 We see definite shift effects. 00:42:58.230 --> 00:43:00.180 We see mean effects. 00:43:00.180 --> 00:43:05.090 Again, we see perhaps an outlier. 00:43:05.090 --> 00:43:08.840 The point here was that-- and this didn't come up earlier-- 00:43:08.840 --> 00:43:11.390 we keep looking and trying to put blame back 00:43:11.390 --> 00:43:14.990 on the manufacturing equipment or the source material. 00:43:18.000 --> 00:43:22.250 You also have to look carefully at your measurement apparatus. 00:43:22.250 --> 00:43:24.680 There's inherent measurement error. 00:43:24.680 --> 00:43:29.180 Even if the parts were almost perfect, 00:43:29.180 --> 00:43:32.860 very often, you're going to have an inherent measurement 00:43:32.860 --> 00:43:36.310 limitation and deviations due to measurement error. 00:43:36.310 --> 00:43:39.970 And you may have loading problems 00:43:39.970 --> 00:43:42.610 into the measurement apparatus. 00:43:42.610 --> 00:43:49.503 So it's not always an indication that there's part deviations. 00:43:49.503 --> 00:43:51.170 In some sense, the measurement equipment 00:43:51.170 --> 00:43:54.890 is manufacturing process, tool, or equipment 00:43:54.890 --> 00:43:58.880 itself that is also subject to deviations. 00:43:58.880 --> 00:44:01.550 And in fact, part of the applications of some 00:44:01.550 --> 00:44:04.550 of the statistical methods we'll talk about seek 00:44:04.550 --> 00:44:08.540 to characterize or do gauge studies of the accuracy 00:44:08.540 --> 00:44:10.880 and capability of your measurement equipment 00:44:10.880 --> 00:44:14.130 and the components of variation that affect that. 00:44:14.130 --> 00:44:20.750 So for example, there's a pure reproducibility 00:44:20.750 --> 00:44:22.820 and there's a repeatability component 00:44:22.820 --> 00:44:24.770 to typical gauge studies. 00:44:24.770 --> 00:44:26.690 One component, just real quickly, 00:44:26.690 --> 00:44:30.410 would be I've loaded the part on the measurement apparatus, 00:44:30.410 --> 00:44:34.070 and then I just fire whatever the measurement sequence is 00:44:34.070 --> 00:44:38.960 multiple times, and think of that as a pure repeatability-- 00:44:38.960 --> 00:44:44.190 somehow the physical-- maybe there's-- 00:44:44.190 --> 00:44:45.560 it's an optical measurement. 00:44:45.560 --> 00:44:50.720 There's scatter from extraneous light sources nearby off 00:44:50.720 --> 00:44:51.665 of that. 00:44:51.665 --> 00:44:53.540 And then there might be a separate component, 00:44:53.540 --> 00:44:56.990 if I pulled that same part out, took it out, and put it back 00:44:56.990 --> 00:45:01.830 on, and did that sequence of operations, 00:45:01.830 --> 00:45:04.550 including the operator operations. 00:45:04.550 --> 00:45:08.210 Then you have a reproducibility component that might be, 00:45:08.210 --> 00:45:10.230 in fact, much larger. 00:45:10.230 --> 00:45:14.780 So you'd need to understand, and maybe decompose and break down 00:45:14.780 --> 00:45:16.700 to those different components. 00:45:16.700 --> 00:45:19.220 And then, if you started to see that one 00:45:19.220 --> 00:45:21.320 was a source of deviation, then you 00:45:21.320 --> 00:45:25.180 might look at strategies to control or compensate for that. 00:45:25.180 --> 00:45:25.680 Yeah? 00:45:25.680 --> 00:45:27.233 AUDIENCE: [INAUDIBLE] 00:45:27.233 --> 00:45:28.025 DUANE BONING: Yeah. 00:45:28.025 --> 00:45:30.344 AUDIENCE: At one of my internships, 00:45:30.344 --> 00:45:34.440 the company was getting a lot of [INAUDIBLE].. 00:45:34.440 --> 00:45:36.840 They were really trying to figure out 00:45:36.840 --> 00:45:38.450 [INAUDIBLE],, and when it went bad, 00:45:38.450 --> 00:45:40.355 at the supplier [INAUDIBLE]. 00:45:47.960 --> 00:45:50.960 DUANE BONING: Absolutely, absolutely-- 00:45:50.960 --> 00:45:56.940 in some sense, it's often the analogy of it's not working-- 00:45:56.940 --> 00:45:59.390 oh, check the plug on the wall. 00:45:59.390 --> 00:46:04.610 Always check your measurement equipment and apparatus 00:46:04.610 --> 00:46:08.780 before you start diving and tearing your hair out, going 00:46:08.780 --> 00:46:11.690 too deep into debugging of the manufacturing process. 00:46:11.690 --> 00:46:14.450 Check each step along the way-- 00:46:14.450 --> 00:46:17.460 glad they discovered that pin problem back in the-- 00:46:17.460 --> 00:46:20.440 at the case. 00:46:20.440 --> 00:46:21.970 Oh, here's more. 00:46:21.970 --> 00:46:22.990 What do we see here? 00:46:27.350 --> 00:46:31.620 Again, some output as a function of run number [INAUDIBLE] 00:46:31.620 --> 00:46:35.870 again, maybe this is another sheet 00:46:35.870 --> 00:46:40.910 bending kind of component. 00:46:40.910 --> 00:46:46.770 We see effects of shift change multiple times here. 00:46:46.770 --> 00:46:49.430 These are different shifts, maybe different shifts 00:46:49.430 --> 00:46:51.360 or different parts. 00:46:51.360 --> 00:46:54.150 What's very interesting here is that, under some conditions, 00:46:54.150 --> 00:46:58.770 we have a nice stable, not drifting in time-- 00:46:58.770 --> 00:47:01.850 fairly nice, stable, or stationary mean, 00:47:01.850 --> 00:47:03.512 with some deviation-- 00:47:03.512 --> 00:47:04.970 every now and then, maybe something 00:47:04.970 --> 00:47:07.550 that looks outside of the distribution. 00:47:07.550 --> 00:47:11.170 But in some cases, holy cow-- 00:47:11.170 --> 00:47:15.430 that's a very different effect than what's going on in here. 00:47:15.430 --> 00:47:22.420 And it's back to this is a very clear drift that is definitely 00:47:22.420 --> 00:47:25.210 not just a one-time shift. 00:47:25.210 --> 00:47:26.120 Yeah? 00:47:26.120 --> 00:47:31.776 AUDIENCE: When I looked at the [INAUDIBLE],, and the first 00:47:31.776 --> 00:47:32.720 [INAUDIBLE]. 00:47:37.190 --> 00:47:40.192 I'm wondering if maybe the [INAUDIBLE].. 00:47:42.940 --> 00:47:48.330 DUANE BONING: Perhaps-- or maybe this worker 00:47:48.330 --> 00:47:54.060 finds a way that corrects for it in a comparatively effective 00:47:54.060 --> 00:47:59.040 fashion, whereas the strategy used in this case over here 00:47:59.040 --> 00:48:05.190 is not nearly as effective or quick at trying to get back, 00:48:05.190 --> 00:48:07.690 if that was the target. 00:48:07.690 --> 00:48:12.390 And so in fact, one of the challenges often in SPC 00:48:12.390 --> 00:48:16.350 or other control algorithms-- or not even algorithms, 00:48:16.350 --> 00:48:19.140 but control practices, especially involving people-- 00:48:19.140 --> 00:48:26.340 is one shift or one set of operators or engineers 00:48:26.340 --> 00:48:30.070 in charge of a process may learn things 00:48:30.070 --> 00:48:32.500 that don't necessarily get captured 00:48:32.500 --> 00:48:37.060 and conveyed to other shifts. 00:48:37.060 --> 00:48:43.880 OK, so what is going on in lots of these examples is perhaps 00:48:43.880 --> 00:48:45.500 the most important point here-- 00:48:45.500 --> 00:48:49.460 is that, very often, there are two highly different 00:48:49.460 --> 00:48:51.650 characteristics that keep coming up. 00:48:51.650 --> 00:48:54.440 There's systematic or deterministic effects 00:48:54.440 --> 00:48:57.440 having to do with the type of the material, settings 00:48:57.440 --> 00:49:00.750 on the equipment, choice of process conditions. 00:49:00.750 --> 00:49:04.220 And then there's also random components. 00:49:04.220 --> 00:49:06.030 There's the inherent spread. 00:49:06.030 --> 00:49:10.700 And a key point here is inherent spread you can almost never 00:49:10.700 --> 00:49:12.470 completely get rid of. 00:49:12.470 --> 00:49:15.500 I guess, philosophically, you could ultimately maybe appeal 00:49:15.500 --> 00:49:19.460 to quantum mechanics, that there would always be some deviation. 00:49:19.460 --> 00:49:20.840 I think very rarely-- 00:49:20.840 --> 00:49:22.730 perhaps in some semiconductor processes, 00:49:22.730 --> 00:49:31.690 we do approach truly inherent physical randomness. 00:49:31.690 --> 00:49:35.630 But generally, it's that the understanding is limited 00:49:35.630 --> 00:49:39.550 in what is contributing to that small degree of spread, 00:49:39.550 --> 00:49:44.410 or it's impossible to completely control those-- 00:49:44.410 --> 00:49:46.450 that inherent randomness. 00:49:46.450 --> 00:49:49.780 But then we also have big disturbances. 00:49:49.780 --> 00:49:54.550 They may still be random events, but they are not 00:49:54.550 --> 00:50:02.200 this little band, but they are big events with a clear cause. 00:50:02.200 --> 00:50:04.780 We may not always be able to discern the cause, 00:50:04.780 --> 00:50:08.260 or may not always be able to discover it, but in many cases, 00:50:08.260 --> 00:50:10.300 we've been hypothesizing about many 00:50:10.300 --> 00:50:14.560 of these big shift changes or a big drift. 00:50:14.560 --> 00:50:16.930 And a bit of investigation and more knowledge 00:50:16.930 --> 00:50:21.010 about the process would readily show us 00:50:21.010 --> 00:50:22.930 what's going on in those cases. 00:50:22.930 --> 00:50:25.210 And in those, when there's something 00:50:25.210 --> 00:50:28.580 systematic or deterministic going on, 00:50:28.580 --> 00:50:32.540 the whole idea is you make some changes to the equipment 00:50:32.540 --> 00:50:34.770 to stamp those out. 00:50:34.770 --> 00:50:37.680 And what we're often going to be dealing with 00:50:37.680 --> 00:50:40.320 is designing experiments and whatnot 00:50:40.320 --> 00:50:43.950 to try to understand systematic components and eliminate them, 00:50:43.950 --> 00:50:46.500 maybe as part of the initial design 00:50:46.500 --> 00:50:48.360 implementation of a process. 00:50:48.360 --> 00:50:53.010 Or in an ongoing process, when we see a deviation, 00:50:53.010 --> 00:50:55.860 we've got to go and debug it, find out what caused that, 00:50:55.860 --> 00:50:57.730 and eliminate it. 00:50:57.730 --> 00:51:00.460 And the natural occurrence then is 00:51:00.460 --> 00:51:04.120 that, over time, as you have learning about your process 00:51:04.120 --> 00:51:08.560 and manufacturing system, you're generally squeezing out 00:51:08.560 --> 00:51:10.630 these deterministic sources. 00:51:10.630 --> 00:51:13.960 It's something-- was repeatable, systematic, 00:51:13.960 --> 00:51:17.050 and therefore, you can come up with strategies 00:51:17.050 --> 00:51:19.750 either to eliminate it or to compensate for it. 00:51:19.750 --> 00:51:23.140 Maybe you have to have a control strategy that 00:51:23.140 --> 00:51:25.090 recognizes that source of variation 00:51:25.090 --> 00:51:26.930 and keep squeezing it out. 00:51:26.930 --> 00:51:30.730 And what you generally are left with 00:51:30.730 --> 00:51:33.100 are these random components, hopefully 00:51:33.100 --> 00:51:35.410 in a very nice, narrow band. 00:51:35.410 --> 00:51:39.140 But we still need to understand and model those, 00:51:39.140 --> 00:51:41.230 and so that's actually where-- 00:51:41.230 --> 00:51:44.860 another place where the statistical analysis tools are 00:51:44.860 --> 00:51:48.640 really crucial, because then we want to be able to understand, 00:51:48.640 --> 00:51:52.570 what is the natural or reducible-- 00:51:52.570 --> 00:51:56.920 irreducible component that we don't know how to stamp out one 00:51:56.920 --> 00:51:58.030 at a time-- 00:51:58.030 --> 00:52:01.730 and use that information to be able to detect 00:52:01.730 --> 00:52:07.500 in that small scatter when something unusual happens. 00:52:07.500 --> 00:52:11.020 And that's the idea of statistical process control. 00:52:11.020 --> 00:52:15.640 So what we want to do is characterize the process-- 00:52:15.640 --> 00:52:19.140 particularly just the random component, 00:52:19.140 --> 00:52:21.060 assuming that we've been able to squeeze out 00:52:21.060 --> 00:52:27.220 many of those other systematic components. 00:52:27.220 --> 00:52:31.550 Why do we want to characterize that process? 00:52:31.550 --> 00:52:33.950 There's some of the things that I've mentioned, 00:52:33.950 --> 00:52:36.570 and that you've mentioned. 00:52:36.570 --> 00:52:38.760 If I see one of these points that 00:52:38.760 --> 00:52:41.970 looks a little bit different than the distribution, 00:52:41.970 --> 00:52:46.410 we said that [? by i, ?] but very often, the question is, 00:52:46.410 --> 00:52:48.840 maybe there's a point that's a little bit above some 00:52:48.840 --> 00:52:49.950 of the other points. 00:52:49.950 --> 00:52:52.770 Then you would often want to ask the question, 00:52:52.770 --> 00:52:58.520 do I really believe the output changed, or is it 00:52:58.520 --> 00:53:02.170 part of that natural variation? 00:53:02.170 --> 00:53:04.620 And if we start to think about some natural variation 00:53:04.620 --> 00:53:06.760 and models for that, including distributions-- 00:53:06.760 --> 00:53:08.617 which we'll get to in just a moment-- 00:53:08.617 --> 00:53:10.200 then we can start to ask that question 00:53:10.200 --> 00:53:16.000 in a very systematic way, in terms of actual probabilities. 00:53:16.000 --> 00:53:18.490 Other reasons is we may go back to, 00:53:18.490 --> 00:53:20.480 did the input actually cause the change? 00:53:20.480 --> 00:53:25.680 Did I make some output change? 00:53:25.680 --> 00:53:29.680 A very important point here is, how confident 00:53:29.680 --> 00:53:31.600 are we have these answers? 00:53:31.600 --> 00:53:33.760 Very often here, we were appealing to it 00:53:33.760 --> 00:53:37.330 looks like that's a bigger change, or this is-- 00:53:37.330 --> 00:53:42.490 it was a mean offset. 00:53:42.490 --> 00:53:48.100 Well, that's OK to try to convince your management 00:53:48.100 --> 00:53:52.750 or convince your operators that, look, something changed, 00:53:52.750 --> 00:53:54.460 but in many cases, you'd actually 00:53:54.460 --> 00:54:00.840 like to be able to quantify how confident you are that that was 00:54:00.840 --> 00:54:06.480 either a deviation point or that some specific decision that you 00:54:06.480 --> 00:54:08.920 made had a true effect. 00:54:08.920 --> 00:54:12.630 And so the probability and statistical tools 00:54:12.630 --> 00:54:14.960 are really, really important for that. 00:54:14.960 --> 00:54:16.710 So what are some ways that we can actually 00:54:16.710 --> 00:54:20.190 characterize and model the random components? 00:54:20.190 --> 00:54:22.440 Well, that's where we're getting into random processes 00:54:22.440 --> 00:54:24.270 and random variables. 00:54:24.270 --> 00:54:27.060 And I hope you're seeing this is the basis 00:54:27.060 --> 00:54:31.530 for statistical process control I've referred to. 00:54:31.530 --> 00:54:37.510 And we'll dive into some of the tools and techniques, as well. 00:54:37.510 --> 00:54:39.530 Design of experiments-- so if we were 00:54:39.530 --> 00:54:42.050 making that pressure change or that time 00:54:42.050 --> 00:54:45.080 change, how big of an effect is it? 00:54:45.080 --> 00:54:49.190 How certain are we within the random natural variation 00:54:49.190 --> 00:54:52.100 that the deviation-- or the controlled decision I 00:54:52.100 --> 00:54:54.870 made actually had a big effect? 00:54:54.870 --> 00:54:59.420 And then finally, even with compensation strategies-- 00:54:59.420 --> 00:55:02.960 like this-- one of the very first things that we said-- 00:55:02.960 --> 00:55:06.920 in that oscillating case, there might be inherent noise. 00:55:06.920 --> 00:55:08.570 And it's quite possible that somebody 00:55:08.570 --> 00:55:11.840 could be making process control feedback 00:55:11.840 --> 00:55:15.180 changes to try to compensate. 00:55:15.180 --> 00:55:17.880 But if they're trying to compensate for something that 00:55:17.880 --> 00:55:21.870 doesn't have, in fact, a connection between the control 00:55:21.870 --> 00:55:25.990 decision and the true source of noise, 00:55:25.990 --> 00:55:29.730 they may, in fact, be injecting more noise in the process. 00:55:29.730 --> 00:55:33.470 So understanding actually how feedback control works 00:55:33.470 --> 00:55:37.610 when we've got random, and not just the systematic component, 00:55:37.610 --> 00:55:39.632 is very important. 00:55:42.290 --> 00:55:45.120 What are some ways of describing randomness? 00:55:48.060 --> 00:55:50.280 Actually, philosophically, there's 00:55:50.280 --> 00:55:53.460 a couple of very different ways of thinking-- 00:55:53.460 --> 00:55:56.520 if you open up a statistic book, in the first few pages, 00:55:56.520 --> 00:55:59.850 we'll talk about this. 00:55:59.850 --> 00:56:03.280 And from a manufacturing perspective, 00:56:03.280 --> 00:56:05.490 well, we're really dealing with data. 00:56:05.490 --> 00:56:09.440 In many ways, the data tells us a lot. 00:56:09.440 --> 00:56:14.210 And one approach here is to look at actual large ensembles 00:56:14.210 --> 00:56:16.730 or collections of data, and look at histograms, 00:56:16.730 --> 00:56:18.080 frequency distributions. 00:56:18.080 --> 00:56:20.270 And we'll look at some examples of that. 00:56:20.270 --> 00:56:24.080 A very different approach is to start with a pure probability 00:56:24.080 --> 00:56:29.060 model that assumes there is a universe of 00:56:29.060 --> 00:56:31.900 and an infinite number of samples, 00:56:31.900 --> 00:56:34.360 and there are characteristics of that infinite number 00:56:34.360 --> 00:56:36.670 of samples. 00:56:36.670 --> 00:56:41.350 And that gives us the tools to put empirical observations, 00:56:41.350 --> 00:56:45.880 like frequency histograms, on a mathematical basis that we 00:56:45.880 --> 00:56:46.870 can reason with. 00:56:50.300 --> 00:56:53.190 So for example, a way of-- 00:56:53.190 --> 00:56:58.230 let me jump forward here-- this is some thermoforming data. 00:56:58.230 --> 00:57:00.750 Again, this is some dimension, and then, 00:57:00.750 --> 00:57:04.500 over this collection of parts, we have a-- 00:57:04.500 --> 00:57:07.770 this bar chart, this frequency histogram. 00:57:07.770 --> 00:57:11.160 And what that's really saying is, for each of the bends 00:57:11.160 --> 00:57:14.610 in that histogram, I'm simply plotting the frequency-- 00:57:14.610 --> 00:57:17.520 or relative frequency that some output was 00:57:17.520 --> 00:57:19.380 between the bounds of that bin. 00:57:22.570 --> 00:57:28.810 And so what we are doing is converting from historical data 00:57:28.810 --> 00:57:30.820 into bins. 00:57:30.820 --> 00:57:33.160 This is still discrete bins, but then we're 00:57:33.160 --> 00:57:36.070 associating that and interpolating forward 00:57:36.070 --> 00:57:39.880 to say, well, the probability in most applications-- 00:57:39.880 --> 00:57:42.820 if I were to do some more part, we 00:57:42.820 --> 00:57:45.670 think that this histogram, in some sense, 00:57:45.670 --> 00:57:49.150 represents a larger set of data, and I 00:57:49.150 --> 00:57:53.050 can use it to talk about the probabilities of occurrences 00:57:53.050 --> 00:57:56.620 of new data in here. 00:57:56.620 --> 00:57:58.240 So what do you see in here? 00:57:58.240 --> 00:58:08.280 Whoops-- there's a little insert here with the raw data. 00:58:08.280 --> 00:58:11.360 Now, if I hid that from you-- 00:58:11.360 --> 00:58:12.400 it's hard for me-- 00:58:12.400 --> 00:58:14.560 I'm trying to put my hand over this. 00:58:14.560 --> 00:58:17.230 If I hid that from you here, and I hid that from you here, 00:58:17.230 --> 00:58:21.130 and you just looked at this data, typical things you would 00:58:21.130 --> 00:58:21.790 say about it-- 00:58:24.330 --> 00:58:29.850 it looks normal, or Gaussian distributed. 00:58:29.850 --> 00:58:33.102 Maybe you would think almost it's completely random. 00:58:33.102 --> 00:58:34.560 It's got to mean and the deviation. 00:58:34.560 --> 00:58:37.510 It's got a normal distribution. 00:58:37.510 --> 00:58:40.232 Now look back at the data. 00:58:40.232 --> 00:58:41.440 It's kind of drifting around. 00:58:44.070 --> 00:58:48.660 Just a peek forward to other things to be cautious about-- 00:58:48.660 --> 00:58:51.840 don't always believe or just look at your statistics. 00:58:51.840 --> 00:58:54.990 Always go back to the raw data, because if this 00:58:54.990 --> 00:58:56.730 were a characteristic of my process, 00:58:56.730 --> 00:58:59.490 I would say there is a random component, 00:58:59.490 --> 00:59:04.440 I think, here, but then I'm also seeing maybe 00:59:04.440 --> 00:59:07.020 some systematic drift that-- 00:59:07.020 --> 00:59:11.440 I'd like to go in and squeeze that out. 00:59:11.440 --> 00:59:15.180 And then, if I were able to get that out of my process, 00:59:15.180 --> 00:59:17.700 then I might have a random component 00:59:17.700 --> 00:59:20.190 that doesn't have that deterministic or systematic 00:59:20.190 --> 00:59:20.735 component. 00:59:20.735 --> 00:59:22.110 And it might have a distribution, 00:59:22.110 --> 00:59:24.750 but it might be much, much tighter. 00:59:24.750 --> 00:59:29.030 So an important point here is purely 00:59:29.030 --> 00:59:31.910 empirical data doesn't always-- 00:59:31.910 --> 00:59:35.660 it may look random, but it may have embedded in it-- 00:59:35.660 --> 00:59:39.560 if you were to deconvolve all of the sources of variation, 00:59:39.560 --> 00:59:41.180 this big random-- 00:59:41.180 --> 00:59:44.180 or this big normal distribution may have mixed into it 00:59:44.180 --> 00:59:46.610 lots of different systematic components, 00:59:46.610 --> 00:59:51.050 as well as underlying natural variation. 00:59:51.050 --> 00:59:53.270 Imagine, for example, that I actually went in 00:59:53.270 --> 00:59:55.835 and I got rid of this long-term drift. 00:59:58.420 --> 01:00:00.700 Would you believe that the remaining variation is all 01:00:00.700 --> 01:00:01.200 random? 01:00:05.920 --> 01:00:09.370 I bet I could go and I could plot it, and I bet it'd look-- 01:00:09.370 --> 01:00:12.570 the remaining deviations would look mostly normal as well. 01:00:16.280 --> 01:00:17.300 You think so? 01:00:17.300 --> 01:00:18.970 Well, who knows? 01:00:18.970 --> 01:00:22.930 You might go in and do some of the other kinds of hypotheses 01:00:22.930 --> 01:00:25.108 about systematic sources that we were talking 01:00:25.108 --> 01:00:26.150 about in the other cases. 01:00:26.150 --> 01:00:29.620 For example, it looks like there may be a bit of an up, 01:00:29.620 --> 01:00:34.000 down-- there might be something cyclical or periodic going on. 01:00:34.000 --> 01:00:36.070 That might be another systematic source. 01:00:36.070 --> 01:00:40.090 It could masquerade as a random normal distribution. 01:00:40.090 --> 01:00:44.900 So there's always that mix in. 01:00:44.900 --> 01:00:50.990 However, in many cases, maybe you also want to live with-- 01:00:50.990 --> 01:00:53.600 it's not worth it to you, or too expensive to go 01:00:53.600 --> 01:00:56.270 in and eliminate, or you tried and we're not 01:00:56.270 --> 01:01:00.320 able to eliminate also some of these wandering drifts. 01:01:00.320 --> 01:01:03.200 And in that case, maybe the best you can do 01:01:03.200 --> 01:01:07.130 is at least characterize that remaining variation, which 01:01:07.130 --> 01:01:08.690 has a systematic component. 01:01:08.690 --> 01:01:13.160 But you would still like to have a probability model for it 01:01:13.160 --> 01:01:17.720 so that you could detect deviations that were outside 01:01:17.720 --> 01:01:20.550 of that variation source. 01:01:20.550 --> 01:01:23.480 So if I saw a big shift up, I would 01:01:23.480 --> 01:01:25.070 say I've got a point way out here 01:01:25.070 --> 01:01:27.500 in the tail of the distribution that's highly unlikely. 01:01:27.500 --> 01:01:30.920 And you'd still want to use that. 01:01:30.920 --> 01:01:40.270 So now, we might want more compact ways of describing 01:01:40.270 --> 01:01:41.810 this distribution. 01:01:41.810 --> 01:01:43.630 Actually, in the age of computers, 01:01:43.630 --> 01:01:47.860 I think, with lots of big disks and lots of great data 01:01:47.860 --> 01:01:53.290 collection, one of the most underutilized opportunities 01:01:53.290 --> 01:01:55.240 is keeping around all the raw data, 01:01:55.240 --> 01:01:59.280 and actually using empirical distribution. 01:01:59.280 --> 01:02:03.230 So in some sense, it's kind of a holdover from the days 01:02:03.230 --> 01:02:06.770 when it was really tough to keep track and use 01:02:06.770 --> 01:02:08.240 vast amounts of data. 01:02:08.240 --> 01:02:11.280 And we almost always would start to say, 01:02:11.280 --> 01:02:13.880 OK, I'd like a compact way of describing this, 01:02:13.880 --> 01:02:17.600 and I want to describe it as something that only has 01:02:17.600 --> 01:02:19.080 maybe a couple of parameters. 01:02:19.080 --> 01:02:27.110 It has a mean and some variance component. 01:02:27.110 --> 01:02:30.540 But that's a different philosophical point. 01:02:30.540 --> 01:02:32.330 There's, actually, I think, a huge amount 01:02:32.330 --> 01:02:35.720 of statistical methods that are underutilized 01:02:35.720 --> 01:02:40.670 that deal with sampling or resampling from raw data 01:02:40.670 --> 01:02:45.290 that actually can capture subtle things going on in the data. 01:02:45.290 --> 01:02:48.410 We're, in this course, not going to be using those things. 01:02:48.410 --> 01:02:52.520 We're pretty much going to seek compact descriptions 01:02:52.520 --> 01:02:56.060 of randomness in ways that we can reason about. 01:02:56.060 --> 01:02:59.330 There are data-intensive ways of doing 01:02:59.330 --> 01:03:01.230 the same kinds of reasoning. 01:03:01.230 --> 01:03:04.400 So one important point here is we're often 01:03:04.400 --> 01:03:09.980 going to take this discrete binned kind of information-- 01:03:09.980 --> 01:03:15.110 we want to characterize it with perhaps a continuous parameter 01:03:15.110 --> 01:03:19.710 variable and a compact distribution, 01:03:19.710 --> 01:03:22.320 such as a normal or Gaussian distribution. 01:03:22.320 --> 01:03:26.720 And this is especially good when the outputs are themselves 01:03:26.720 --> 01:03:29.960 truly continuous parameters, like a width, 01:03:29.960 --> 01:03:32.930 a dimension of the part. 01:03:32.930 --> 01:03:35.870 And one point here, just to remind you-- especially 01:03:35.870 --> 01:03:38.138 when we go to continuous parameters, 01:03:38.138 --> 01:03:39.680 you've got to be a little bit careful 01:03:39.680 --> 01:03:45.710 about talking about the probability of your part 01:03:45.710 --> 01:03:49.430 or your variable taking on any specific value 01:03:49.430 --> 01:03:52.220 in that continuous distribution. 01:03:52.220 --> 01:03:53.820 Back here when we had things binned, 01:03:53.820 --> 01:03:55.550 I could talk about the probability 01:03:55.550 --> 01:04:01.250 of measuring something in that bin, 01:04:01.250 --> 01:04:07.470 within that range, 1.485 to 1.487. 01:04:07.470 --> 01:04:09.770 But now, if I were to ask, what is the likelihood 01:04:09.770 --> 01:04:16.790 that I measure exactly 1.486, be careful. 01:04:16.790 --> 01:04:22.520 That is a very, very discrete value that-- 01:04:22.520 --> 01:04:24.890 when we talk about continuous probabilities, 01:04:24.890 --> 01:04:27.800 the probability of getting any one value-- 01:04:27.800 --> 01:04:31.430 and it's truly continuous-- 01:04:31.430 --> 01:04:33.120 is 0. 01:04:33.120 --> 01:04:34.890 You're never going to see exactly that. 01:04:34.890 --> 01:04:40.360 Now, with real measurement tools, 01:04:40.360 --> 01:04:43.990 we never have perfect continuous measurements either. 01:04:43.990 --> 01:04:47.140 There is always a precision and a discreetness 01:04:47.140 --> 01:04:51.040 capable from the tool. 01:04:51.040 --> 01:04:53.410 So this is an abstraction dealing 01:04:53.410 --> 01:04:57.220 with the mathematical niceties. 01:04:57.220 --> 01:05:01.450 I'd just point out it's not exactly true 01:05:01.450 --> 01:05:06.470 and measured reality and data. 01:05:06.470 --> 01:05:09.965 So very often, when we talk about probabilities, 01:05:09.965 --> 01:05:10.840 we got to be careful. 01:05:10.840 --> 01:05:14.980 The probability of any one value is 01:05:14.980 --> 01:05:17.380 0 what we talk about really is the probability 01:05:17.380 --> 01:05:21.130 of our measurement or whatever being in some range that's 01:05:21.130 --> 01:05:24.100 inherent in the distribution. 01:05:24.100 --> 01:05:27.820 But we can also talk about, for example, the probability 01:05:27.820 --> 01:05:31.000 that the value be in the range from minus 01:05:31.000 --> 01:05:34.390 infinity up to some value y star, 01:05:34.390 --> 01:05:37.420 and that is a reasonable question 01:05:37.420 --> 01:05:39.880 to ask in terms of probabilities. 01:05:39.880 --> 01:05:43.690 And that is embedded in this notion 01:05:43.690 --> 01:05:48.400 of a cumulative probability function, or cumulative density 01:05:48.400 --> 01:05:51.160 function, or CDF. 01:05:51.160 --> 01:05:53.230 Just to illustrate that a little bit more, 01:05:53.230 --> 01:05:56.380 we can do the same thing, have a cumulative frequency 01:05:56.380 --> 01:05:58.640 with discrete data. 01:05:58.640 --> 01:06:02.470 So if I had that underlying histogram here, 01:06:02.470 --> 01:06:07.090 I could also build a cumulative frequency histogram 01:06:07.090 --> 01:06:10.120 that simply was giving me the probability 01:06:10.120 --> 01:06:15.280 that a part or measured value was up to, but less 01:06:15.280 --> 01:06:17.890 than some particular value. 01:06:17.890 --> 01:06:19.450 And that's simply the integration 01:06:19.450 --> 01:06:24.940 across the PDF up to that cumulative density function. 01:06:24.940 --> 01:06:26.680 And we can do the same thing that we 01:06:26.680 --> 01:06:34.170 did with the discrete probability frequency 01:06:34.170 --> 01:06:35.200 relationship. 01:06:35.200 --> 01:06:39.870 So we can approximate the frequency with a PDF. 01:06:39.870 --> 01:06:42.450 We can also look at the cumulative frequency 01:06:42.450 --> 01:06:46.220 and approximate that with a cumulative density 01:06:46.220 --> 01:06:49.520 function, or a CDF. 01:06:49.520 --> 01:06:52.850 So we can have that kind of an equivalent. 01:06:52.850 --> 01:06:56.600 And those equivalents also, again, help us interpolate 01:06:56.600 --> 01:06:58.670 and ask probability questions also 01:06:58.670 --> 01:07:04.140 on values that may not have been exactly lined up with our bins. 01:07:04.140 --> 01:07:07.010 So very often, we're going to be, I guess, 01:07:07.010 --> 01:07:10.020 talking either about the density function-- 01:07:10.020 --> 01:07:12.090 the PDF of some variable-- 01:07:12.090 --> 01:07:14.970 or the cumulative probability. 01:07:14.970 --> 01:07:19.080 And there is this inherent relationship between the two. 01:07:19.080 --> 01:07:22.710 I describe the cumulative probability as an integration 01:07:22.710 --> 01:07:23.340 up. 01:07:23.340 --> 01:07:26.970 The probability that some value is less than or equal to 01:07:26.970 --> 01:07:33.090 would be an integral up to some x star. 01:07:33.090 --> 01:07:38.080 And so that value right there corresponds to that. 01:07:38.080 --> 01:07:39.570 And then the inverse is-- 01:07:39.570 --> 01:07:41.290 relationship is true as well. 01:07:41.290 --> 01:07:44.790 If I had a cumulative density function, 01:07:44.790 --> 01:07:48.480 I can differentiate that to get down to the probability density 01:07:48.480 --> 01:07:49.750 function. 01:07:49.750 --> 01:07:54.000 So this is some basic statistical concepts 01:07:54.000 --> 01:07:57.630 that you probably have seen before. 01:07:57.630 --> 01:08:04.830 Many of you have seen it in much more detail in 2.853. 01:08:04.830 --> 01:08:06.690 OK, so very often, the histogram does 01:08:06.690 --> 01:08:12.450 suggest an underlying approximate or convenient 01:08:12.450 --> 01:08:14.550 probability density function. 01:08:14.550 --> 01:08:18.210 Very often, we've talked about normal distributions, 01:08:18.210 --> 01:08:21.720 but a key thing to look at is always look at your raw data, 01:08:21.720 --> 01:08:27.540 because there may be other perfectly reasonable continuous 01:08:27.540 --> 01:08:31.800 or discrete probability density or probability mass 01:08:31.800 --> 01:08:34.140 functions that apply in the discrete case, 01:08:34.140 --> 01:08:38.135 such as a uniform distribution. 01:08:38.135 --> 01:08:39.760 And so one of the things that we'll see 01:08:39.760 --> 01:08:46.540 is the family of some typically emerging or typically observed 01:08:46.540 --> 01:08:48.534 kinds of distribution models. 01:08:51.350 --> 01:08:53.060 So very often, what we're doing is 01:08:53.060 --> 01:08:55.970 we're gathering data, looking at these histograms, 01:08:55.970 --> 01:09:00.580 and looking to see, is there a consistent pattern, such 01:09:00.580 --> 01:09:07.850 as we saw with the underlying normal distribution? 01:09:07.850 --> 01:09:13.090 And we're often then trying to intuit 01:09:13.090 --> 01:09:17.020 what that reasonable underlying distribution may be. 01:09:17.020 --> 01:09:20.170 There are ways-- and we may touch on them a little bit-- 01:09:20.170 --> 01:09:23.800 to be able to test whether or how 01:09:23.800 --> 01:09:30.370 good a particular distribution model may fit with that data. 01:09:30.370 --> 01:09:33.010 And on the problem set, in fact, you'll 01:09:33.010 --> 01:09:36.010 explore things like a qqnorm plot. 01:09:36.010 --> 01:09:38.240 And I'm not going to go into detail here, 01:09:38.240 --> 01:09:39.939 but essentially, that-- 01:09:39.939 --> 01:09:43.090 assume a model-- for example, what the probability density 01:09:43.090 --> 01:09:46.660 function probabilities would be associated with certain kinds 01:09:46.660 --> 01:09:50.200 of observations, if it were a normal distribution-- 01:09:50.200 --> 01:09:52.180 and then you plot your data against that 01:09:52.180 --> 01:09:55.300 and start to see how well the data fits 01:09:55.300 --> 01:09:56.740 that particular model. 01:10:01.030 --> 01:10:03.360 Let's go back to that original set 01:10:03.360 --> 01:10:05.130 of data, the [? c and c turning. ?] 01:10:05.130 --> 01:10:08.220 If I were to plot that as a distribution, 01:10:08.220 --> 01:10:09.840 I might get something like this. 01:10:13.180 --> 01:10:16.630 I might argue that my bins may be too coarse here, 01:10:16.630 --> 01:10:17.860 but what do you see? 01:10:20.550 --> 01:10:22.410 This is a little tricky, right? 01:10:22.410 --> 01:10:24.630 If I were to plot that data as a distribution, 01:10:24.630 --> 01:10:29.010 I might be tempted to say, OK, that's one big Gaussian 01:10:29.010 --> 01:10:31.760 distribution. 01:10:31.760 --> 01:10:34.490 You could try that. 01:10:34.490 --> 01:10:36.510 But again, if we look back at the data, 01:10:36.510 --> 01:10:39.830 another interpretation here is, really-- 01:10:39.830 --> 01:10:41.330 you can start to see it-- 01:10:44.318 --> 01:10:45.690 it's bimodal. 01:10:45.690 --> 01:10:49.380 It's a mix of two distributions. 01:10:49.380 --> 01:10:54.090 So I guess the simple point here is don't always 01:10:54.090 --> 01:10:58.350 assume it's going to be just a single distribution that's 01:10:58.350 --> 01:10:58.982 underlying it. 01:10:58.982 --> 01:10:59.940 There may be a mixture. 01:11:03.770 --> 01:11:05.940 Similarly, here's another set. 01:11:05.940 --> 01:11:10.450 And very often, again, plotting your empirical data 01:11:10.450 --> 01:11:13.660 in time order tells you a lot more, but also 01:11:13.660 --> 01:11:16.480 be sensitive to those kinds of questions. 01:11:16.480 --> 01:11:18.940 I could have started the whole class off and never 01:11:18.940 --> 01:11:22.390 shown you the raw data, and just started plotting distributions. 01:11:22.390 --> 01:11:25.570 And in some cases, things might be more observable there. 01:11:25.570 --> 01:11:31.060 Usually it's easier to see perhaps lurking up here, 01:11:31.060 --> 01:11:33.760 when there's such a very clear time shift. 01:11:33.760 --> 01:11:38.650 But here, this is the histogram associated with these, 01:11:38.650 --> 01:11:43.780 and it sure looks like one normal distribution here 01:11:43.780 --> 01:11:45.760 and another normal distribution here. 01:11:49.320 --> 01:11:51.780 Now, by the way, if these things were ping 01:11:51.780 --> 01:11:54.400 ponging back and forth, or they were randomly picking 01:11:54.400 --> 01:11:58.560 and it was one of those source material deviations, 01:11:58.560 --> 01:12:01.350 it might actually be hard to detect that, 01:12:01.350 --> 01:12:03.390 whereas the bimodality-- the fact 01:12:03.390 --> 01:12:06.300 that I've actually got what looks like two mixtures-- may 01:12:06.300 --> 01:12:14.990 actually appear more easily in a histogram kind of form. 01:12:14.990 --> 01:12:19.670 There's another example. 01:12:19.670 --> 01:12:22.700 Here we start to see, again, these two shifts, 01:12:22.700 --> 01:12:26.450 and now you can start to convert these perhaps 01:12:26.450 --> 01:12:29.570 to two statistical distributions, two 01:12:29.570 --> 01:12:31.850 Gaussians with different means. 01:12:31.850 --> 01:12:35.210 And now you can also estimate parameters, 01:12:35.210 --> 01:12:37.100 like what the mean shift is. 01:12:37.100 --> 01:12:38.900 So once you've got distributions, 01:12:38.900 --> 01:12:41.870 now you've got mathematical approaches 01:12:41.870 --> 01:12:45.810 for being able to estimate those parameters. 01:12:45.810 --> 01:12:50.230 So this is just going back to some of that other data. 01:12:50.230 --> 01:12:55.690 OK, so some of the key points here out of this portion 01:12:55.690 --> 01:13:00.250 is, even if there's no strong input effect, 01:13:00.250 --> 01:13:03.970 there may still be disturbances. 01:13:03.970 --> 01:13:08.080 And we'll often see a consistent histogram pattern emerge. 01:13:08.080 --> 01:13:12.130 And what we often want to do is use 01:13:12.130 --> 01:13:15.490 that either to detect deterministic deviations 01:13:15.490 --> 01:13:20.930 and squeeze them out, or once we've done all those process 01:13:20.930 --> 01:13:23.960 improvements and gotten those all squeezed out, 01:13:23.960 --> 01:13:28.710 we may still have an underlying inherent distribution. 01:13:28.710 --> 01:13:32.300 Maybe it's random variation associated 01:13:32.300 --> 01:13:34.280 with a Gaussian distribution. 01:13:34.280 --> 01:13:36.410 And the key idea is we can even use, 01:13:36.410 --> 01:13:39.620 and importantly, we want to use knowledge of that underlying 01:13:39.620 --> 01:13:41.170 pattern. 01:13:41.170 --> 01:13:42.590 It's not a systematic pattern. 01:13:42.590 --> 01:13:45.460 It's a random pattern associated with that distribution-- 01:13:45.460 --> 01:13:48.430 so that we can predict behavior, and in particular, set 01:13:48.430 --> 01:13:50.480 limits on normal behavior. 01:13:53.050 --> 01:13:54.640 This is a nice case. 01:13:54.640 --> 01:13:58.780 Maybe this is what we'll consider normal behavior. 01:13:58.780 --> 01:14:01.180 And again, I want to be able to statistically 01:14:01.180 --> 01:14:04.930 say, what's the likelihood that some 01:14:04.930 --> 01:14:09.190 of these points or a new point belong to the natural variation 01:14:09.190 --> 01:14:11.170 or not? 01:14:11.170 --> 01:14:15.360 So we're going to typically use analytic probability density 01:14:15.360 --> 01:14:17.690 functions to do that. 01:14:17.690 --> 01:14:22.550 I think we've already talked about most of these things. 01:14:22.550 --> 01:14:26.030 I've been emphasizing, by the way, continuous values, 01:14:26.030 --> 01:14:31.220 these width or geometry parameters, 01:14:31.220 --> 01:14:34.550 but there's also a set of different statistical 01:14:34.550 --> 01:14:38.150 distribution than the normal distribution associated 01:14:38.150 --> 01:14:41.060 especially with the occurrence-- random occurrence of point 01:14:41.060 --> 01:14:41.990 defects. 01:14:41.990 --> 01:14:43.940 And a little later in the term, we'll 01:14:43.940 --> 01:14:46.580 actually do some yield modeling dealing 01:14:46.580 --> 01:14:49.280 with things like Poisson distributions 01:14:49.280 --> 01:14:51.830 and in these sorts of discrete distributions 01:14:51.830 --> 01:14:56.720 that are more appropriate to discrete events 01:14:56.720 --> 01:14:58.310 or discrete occurrences. 01:14:58.310 --> 01:15:00.620 We're going to be, in the first part here, 01:15:00.620 --> 01:15:04.710 dealing pretty much with continuous parameters. 01:15:04.710 --> 01:15:06.920 So this is just laying out, again, a little bit. 01:15:06.920 --> 01:15:08.420 We've talked about some of these. 01:15:08.420 --> 01:15:10.880 I want to build up just very briefly, 01:15:10.880 --> 01:15:13.850 get you going on some of the basic definitions. 01:15:13.850 --> 01:15:16.670 And then Chapter 4 in Spanos really articulates 01:15:16.670 --> 01:15:17.970 these much more. 01:15:17.970 --> 01:15:20.000 So we have our probability density function 01:15:20.000 --> 01:15:23.360 and cumulative density function, and there is that relationship 01:15:23.360 --> 01:15:25.920 between the two. 01:15:25.920 --> 01:15:28.160 And then we can ask about certain moments 01:15:28.160 --> 01:15:31.910 or characteristics of these continuous probability density 01:15:31.910 --> 01:15:32.980 functions. 01:15:32.980 --> 01:15:34.310 These should be familiar to. 01:15:34.310 --> 01:15:38.540 We can ask, what is the mean or the expected value of one 01:15:38.540 --> 01:15:39.980 of these distributions? 01:15:39.980 --> 01:15:45.290 And here I'm using t as a stand-in for time or run 01:15:45.290 --> 01:15:46.310 number. 01:15:46.310 --> 01:15:50.540 And if I have a probability density function associated 01:15:50.540 --> 01:15:56.000 with the random variable, and even time, then 01:15:56.000 --> 01:16:00.260 there we have a definition of the expected value 01:16:00.260 --> 01:16:02.880 of x as a function of time. 01:16:02.880 --> 01:16:05.810 Now, I'm kind of being pedantic here and giving 01:16:05.810 --> 01:16:10.660 this definition, because we also saw trends-- 01:16:10.660 --> 01:16:12.220 long-term trends in our data. 01:16:12.220 --> 01:16:16.195 And we could ask what the mean is as a function of time. 01:16:19.330 --> 01:16:22.060 Again, we're in a situation where I'm trying 01:16:22.060 --> 01:16:24.070 to squeeze out those drifts. 01:16:24.070 --> 01:16:30.610 So usually we're in a situation where we want to avoid 01:16:30.610 --> 01:16:35.740 non-stationary drifting processes or analyses where 01:16:35.740 --> 01:16:37.450 some of these functions-- 01:16:37.450 --> 01:16:40.390 the probability density function and its parameters, 01:16:40.390 --> 01:16:41.470 like the mean-- 01:16:41.470 --> 01:16:43.030 drift over time. 01:16:43.030 --> 01:16:48.400 We typically want to be having stationary processes where 01:16:48.400 --> 01:16:54.430 the mean and other moments, like variance, are 01:16:54.430 --> 01:16:57.940 independent of time. 01:17:03.350 --> 01:17:06.400 So in the stationary case, which is what we're mostly 01:17:06.400 --> 01:17:08.560 going to be dealing with, now I've 01:17:08.560 --> 01:17:11.680 removed the time dependence on the underlying 01:17:11.680 --> 01:17:13.600 statistical model. 01:17:13.600 --> 01:17:16.520 Let's say, from time to time, it should be stable. 01:17:16.520 --> 01:17:20.770 And then we have a mean, this mu of x-- the true, 01:17:20.770 --> 01:17:24.070 or theoretical true mean-- 01:17:24.070 --> 01:17:26.830 that we would calculate with the same formula. 01:17:26.830 --> 01:17:29.950 But again, the idea of the stationary process-- 01:17:29.950 --> 01:17:33.770 important characteristic there is that the mean is a constant, 01:17:33.770 --> 01:17:35.455 and not wandering or drifting. 01:17:38.680 --> 01:17:42.820 Similarly, in the case when it's a stationary process, 01:17:42.820 --> 01:17:47.570 the variance, defined here as the second moment-- 01:17:47.570 --> 01:17:52.430 the expectation of the distance squared of your data 01:17:52.430 --> 01:17:53.990 from the mean-- 01:17:53.990 --> 01:17:56.600 [INAUDIBLE] expectation of that-- 01:17:56.600 --> 01:17:57.830 is the variance. 01:17:57.830 --> 01:17:59.480 That's just the definition. 01:17:59.480 --> 01:18:02.750 And in the stationary case, that is also 01:18:02.750 --> 01:18:04.400 a constant of the process-- 01:18:04.400 --> 01:18:08.060 has a nice alternative little description, 01:18:08.060 --> 01:18:09.560 if you work through the mathematics, 01:18:09.560 --> 01:18:15.080 where it can also be calculated as the expectation 01:18:15.080 --> 01:18:18.230 of the square of your data minus the mean squared. 01:18:21.360 --> 01:18:25.190 By the way, you can ask the same questions-- variance, mean-- 01:18:25.190 --> 01:18:27.530 for any continuous distribution. 01:18:27.530 --> 01:18:29.930 They're perfectly good moments. 01:18:29.930 --> 01:18:33.830 In some cases, they completely characterize the distribution. 01:18:33.830 --> 01:18:37.670 In other cases, there may need to be additional moments. 01:18:37.670 --> 01:18:40.800 And an example here is we could ask, 01:18:40.800 --> 01:18:43.910 what is the mean of a uniform distribution-- uniform, where 01:18:43.910 --> 01:18:50.190 the probability density is equal across an entire range? 01:18:50.190 --> 01:18:55.310 And so one could calculate those as well. 01:18:55.310 --> 01:18:59.320 Everybody is seeing the normal distribution. 01:18:59.320 --> 01:19:01.990 Again, we can actually have an explicit description 01:19:01.990 --> 01:19:05.020 for what the PDF is. 01:19:05.020 --> 01:19:11.780 We'll often be talking about the unit standard normal or unit 01:19:11.780 --> 01:19:13.670 normal distribution. 01:19:13.670 --> 01:19:18.440 The full distribution here actually has the mean in it, 01:19:18.440 --> 01:19:21.150 and it has the variance in it. 01:19:21.150 --> 01:19:23.330 So for your particular process, those 01:19:23.330 --> 01:19:27.500 may be very different for whatever collection of data. 01:19:27.500 --> 01:19:33.520 In order to be able to talk generically about properties 01:19:33.520 --> 01:19:37.480 of the normal distribution, and to use tabulated values and so 01:19:37.480 --> 01:19:42.040 on, this normalization, where we subtract off the mean 01:19:42.040 --> 01:19:45.430 to get a 0 mean description, and then we 01:19:45.430 --> 01:19:48.460 divide by the standard deviation, 01:19:48.460 --> 01:19:51.890 gives us the unit normal distribution. 01:19:51.890 --> 01:19:54.580 And that's exactly always this distribution 01:19:54.580 --> 01:19:58.840 in terms of z units. 01:19:58.840 --> 01:20:01.750 Now we've made it independent to the mean, independent 01:20:01.750 --> 01:20:04.000 of the variance. 01:20:04.000 --> 01:20:08.320 And the exact values, the probability densities 01:20:08.320 --> 01:20:13.450 for any values of z are well-known and tabulated. 01:20:13.450 --> 01:20:16.990 So we've made it 0 mean and made it unit norm-- 01:20:16.990 --> 01:20:18.950 unit variance. 01:20:18.950 --> 01:20:21.860 There is also a cumulative distribution here. 01:20:21.860 --> 01:20:25.550 I'm describing this as p of z, so now we 01:20:25.550 --> 01:20:27.440 can ask the cumulative distribution 01:20:27.440 --> 01:20:32.900 function for that unit normal, and plot that. 01:20:35.530 --> 01:20:38.530 For the normal PDF, what's nice is 01:20:38.530 --> 01:20:40.690 this is the whole distribution, and all 01:20:40.690 --> 01:20:44.470 we need are those two parameters, mean and variance, 01:20:44.470 --> 01:20:48.010 and it completely characterizes the process. 01:20:48.010 --> 01:20:50.050 And then there's some statistical operations 01:20:50.050 --> 01:20:53.680 that we can do for when we-- that are really convenient when 01:20:53.680 --> 01:20:55.200 we have normal distributions. 01:20:55.200 --> 01:21:00.160 For example, if we have multiple effects, we can look, 01:21:00.160 --> 01:21:03.580 and the sum of multiple normal random variables 01:21:03.580 --> 01:21:06.555 still has a normal distribution. 01:21:06.555 --> 01:21:08.680 So I'm just giving you a little bit of a peek here. 01:21:08.680 --> 01:21:10.450 You can do some reading, which is probably 01:21:10.450 --> 01:21:15.130 more effective than me just flashing by these things. 01:21:15.130 --> 01:21:17.680 There are important operations, for example, 01:21:17.680 --> 01:21:20.260 where, if we have an output y that 01:21:20.260 --> 01:21:23.170 is a sum of multiple random variables, 01:21:23.170 --> 01:21:26.410 there are things like the mean of the overall y 01:21:26.410 --> 01:21:28.780 is simply the sum of the mean of each 01:21:28.780 --> 01:21:30.730 of these individual random variables. 01:21:30.730 --> 01:21:34.270 And if there's a scaling in the mean, that simply scales. 01:21:34.270 --> 01:21:37.360 There are also operations when these-- 01:21:37.360 --> 01:21:42.880 if these are independent random variables, the variances add. 01:21:42.880 --> 01:21:44.890 And you've got to be careful any constant. 01:21:44.890 --> 01:21:47.230 If you run that through the definition of variance, 01:21:47.230 --> 01:21:48.950 those are also squared. 01:21:48.950 --> 01:21:51.580 So you'll see the basic description 01:21:51.580 --> 01:21:53.260 of some of these mathematical operations 01:21:53.260 --> 01:21:57.850 and these distributions in May and Spanos. 01:21:57.850 --> 01:22:00.040 And you'll also start to get a feel 01:22:00.040 --> 01:22:03.820 for using these distributions to ask 01:22:03.820 --> 01:22:07.980 some very standard questions, like what 01:22:07.980 --> 01:22:09.630 is the probability that you would 01:22:09.630 --> 01:22:13.600 observe a value in some range? 01:22:13.600 --> 01:22:15.490 So I think, with that, we'll close. 01:22:15.490 --> 01:22:17.620 If you want to get started on problem set 2, 01:22:17.620 --> 01:22:20.200 there's some problems that-- 01:22:20.200 --> 01:22:24.550 this is just basically giving you some familiarity 01:22:24.550 --> 01:22:28.330 with manipulating things with Gaussian distributions 01:22:28.330 --> 01:22:29.470 and other statistics. 01:22:29.470 --> 01:22:34.380 And then we'll start building up more detail next Thursday.