1 00:00:00,000 --> 00:00:02,430 The following content is provided under a Creative 2 00:00:02,430 --> 00:00:03,730 Commons license. 3 00:00:03,730 --> 00:00:06,030 Your support will help MIT OpenCourseWare 4 00:00:06,030 --> 00:00:10,060 continue to offer high-quality educational resources for free. 5 00:00:10,060 --> 00:00:12,660 To make a donation or to view additional materials 6 00:00:12,660 --> 00:00:16,560 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:16,560 --> 00:00:17,874 at ocw.mit.edu. 8 00:00:32,409 --> 00:00:36,070 DUANE BONING: OK, so today we're going to talk about some 9 00:00:36,070 --> 00:00:38,800 of the probability models associated with variation 10 00:00:38,800 --> 00:00:41,260 in manufacturing processes. 11 00:00:41,260 --> 00:00:42,880 The last couple of classes, we've 12 00:00:42,880 --> 00:00:46,040 tried to go and do two things. 13 00:00:46,040 --> 00:00:48,070 One is give you a little bit of perspective 14 00:00:48,070 --> 00:00:50,370 on a couple of different kinds of-- 15 00:00:50,370 --> 00:00:53,140 or actually, a fairly wide family 16 00:00:53,140 --> 00:00:55,930 of manufacturing processes, give you 17 00:00:55,930 --> 00:00:58,420 a little bit of exposure to semiconductor processes 18 00:00:58,420 --> 00:01:03,070 as well as a couple of different mechanical-- 19 00:01:03,070 --> 00:01:06,100 mechanically-oriented processes, and give you 20 00:01:06,100 --> 00:01:10,340 some ideas or ways to think about those processes, 21 00:01:10,340 --> 00:01:13,250 including just the physical action that's going on, 22 00:01:13,250 --> 00:01:15,350 as well as some of the sources of variation. 23 00:01:15,350 --> 00:01:17,200 So we're going to dive in a little bit 24 00:01:17,200 --> 00:01:23,530 today on dealing with data taken from some manufacturing 25 00:01:23,530 --> 00:01:26,230 processes, and start to look at random components of that 26 00:01:26,230 --> 00:01:28,330 and systematic components of that. 27 00:01:28,330 --> 00:01:31,210 But back on the theme of trying to make sure 28 00:01:31,210 --> 00:01:34,120 that you have at least some exposure, given everybody's 29 00:01:34,120 --> 00:01:36,730 different backgrounds, to these processes, 30 00:01:36,730 --> 00:01:39,970 we would like to make sure you have the opportunity 31 00:01:39,970 --> 00:01:45,370 to at least see a couple of the kinds of tools 32 00:01:45,370 --> 00:01:47,360 and equipment in the facilities-- 33 00:01:47,360 --> 00:01:51,010 especially those associated with semiconductor manufacturing, 34 00:01:51,010 --> 00:01:53,985 if you haven't previously seen those. 35 00:01:53,985 --> 00:01:55,360 So let me turn it over to Hayden, 36 00:01:55,360 --> 00:01:57,520 who can check in with you well the problem 37 00:01:57,520 --> 00:02:03,100 is that as well, but also about arranging opportunities 38 00:02:03,100 --> 00:02:05,770 for you to see a little bit more, if you would like. 39 00:02:08,175 --> 00:02:11,320 HAYDEN TAYLOR: So yes, on the problem set, well, 40 00:02:11,320 --> 00:02:13,990 if you still have questions about it, 41 00:02:13,990 --> 00:02:17,350 then feel free to email or call today. 42 00:02:17,350 --> 00:02:19,810 See me afterwards. 43 00:02:19,810 --> 00:02:23,890 And problem set 2 is now up on the website. 44 00:02:23,890 --> 00:02:28,330 That's due on Tuesday, the 26th of this month. 45 00:02:28,330 --> 00:02:30,260 So there's a bit longer for that, 46 00:02:30,260 --> 00:02:34,960 and I'm willing to start helping you with that too. 47 00:02:34,960 --> 00:02:35,860 So thank you. 48 00:02:39,040 --> 00:02:41,050 DUANE BONING: Any questions for Hayden? 49 00:02:41,050 --> 00:02:45,170 Any big issues with the problem set? 50 00:02:45,170 --> 00:02:46,970 AUDIENCE: Actually, I have a-- 51 00:02:46,970 --> 00:02:47,928 DUANE BONING: Go ahead. 52 00:02:48,920 --> 00:02:49,670 HAYDEN TAYLOR: OK. 53 00:02:49,670 --> 00:02:51,212 AUDIENCE: So I have a quick question. 54 00:02:51,212 --> 00:02:54,080 So if you can refer to the Question 3 of the Problem Set 55 00:02:54,080 --> 00:03:01,090 1, and you say it's proportional to the water and the time-- 56 00:03:01,090 --> 00:03:02,390 the KOH. 57 00:03:02,390 --> 00:03:04,880 And when you talk about these [INAUDIBLE],, 58 00:03:04,880 --> 00:03:07,010 you're talking about the concentration 59 00:03:07,010 --> 00:03:08,835 or you're talking about the molarity? 60 00:03:13,237 --> 00:03:15,530 HAYDEN TAYLOR: What am I saying is proportional to KOH. 61 00:03:15,530 --> 00:03:16,170 DUANE BONING: The square brackets around the KOH. 62 00:03:16,670 --> 00:03:18,120 HAYDEN TAYLOR: Oh, I see. 63 00:03:18,120 --> 00:03:20,245 AUDIENCE: The very last figure [INAUDIBLE].. 64 00:03:20,245 --> 00:03:20,730 HAYDEN TAYLOR: Yeah, yeah. 65 00:03:20,730 --> 00:03:21,030 AUDIENCE: --proportional. 66 00:03:21,030 --> 00:03:23,140 The [? edge 3 ?] is proportional. 67 00:03:23,140 --> 00:03:25,395 So it's proportional to the concentration, 68 00:03:25,395 --> 00:03:27,825 or it's proportional to the molarity? 69 00:03:30,127 --> 00:03:36,060 HAYDEN TAYLOR: Well, for a fixed volume, it's the same. 70 00:03:36,060 --> 00:03:38,070 There are two lines on that plot, 71 00:03:38,070 --> 00:03:41,970 and the models that are put forward by [? Seidel ?] 72 00:03:41,970 --> 00:03:43,080 to explain the data. 73 00:03:43,080 --> 00:03:47,070 The data are the open circles on that plot. 74 00:03:47,070 --> 00:03:52,950 And there are two models that have plotted. 75 00:03:52,950 --> 00:03:57,720 And what I want you to use are primarily 76 00:03:57,720 --> 00:03:59,530 the experimental data. 77 00:03:59,530 --> 00:04:05,850 So look at the curved line that is plotted on that graph. 78 00:04:05,850 --> 00:04:08,150 AUDIENCE: OK and for the curved line that you 79 00:04:08,150 --> 00:04:13,350 want us to look at, the x-axis should be the concentration, 80 00:04:13,350 --> 00:04:13,850 right? 81 00:04:14,285 --> 00:04:14,570 HAYDEN TAYLOR: That's right. 82 00:04:14,570 --> 00:04:15,050 Yes, the x-axis-- 83 00:04:15,050 --> 00:04:15,925 AUDIENCE: [INAUDIBLE] 84 00:04:15,925 --> 00:04:18,355 HAYDEN TAYLOR: --is the weight of KOH. 85 00:04:18,355 --> 00:04:19,730 In other words, that's the number 86 00:04:19,730 --> 00:04:24,315 of grams of KOH per gram of water. 87 00:04:24,315 --> 00:04:25,940 AUDIENCE: Yeah, I just want to clarify. 88 00:04:25,940 --> 00:04:26,350 That's it. 89 00:04:26,350 --> 00:04:26,876 HAYDEN TAYLOR: Yeah. 90 00:04:26,876 --> 00:04:27,312 Thank you. 91 00:04:27,312 --> 00:04:27,750 AUDIENCE: Thanks. 92 00:04:27,750 --> 00:04:28,250 Thank you. 93 00:04:28,445 --> 00:04:29,612 HAYDEN TAYLOR: Anybody else? 94 00:04:31,610 --> 00:04:32,110 OK. 95 00:04:35,168 --> 00:04:36,960 DUANE BONING: I was just also going to say, 96 00:04:36,960 --> 00:04:38,670 it's sort of a manufacturing process 97 00:04:38,670 --> 00:04:40,950 that we use for making up these problem sets. 98 00:04:40,950 --> 00:04:46,360 And they are not expected to be completely defect-free, 99 00:04:46,360 --> 00:04:49,740 so we expect part of the process is 100 00:04:49,740 --> 00:04:53,670 detecting errors or questions. 101 00:04:53,670 --> 00:04:58,350 And then we try to correct them. 102 00:04:58,350 --> 00:05:02,910 Also, as a follow-up to that the schedule for the next problem 103 00:05:02,910 --> 00:05:07,090 set, just looking ahead for the next couple of weeks, 104 00:05:07,090 --> 00:05:12,090 the reason that the problem set is due not next Thursday, 105 00:05:12,090 --> 00:05:14,040 but rather, the Tuesday after that-- 106 00:05:14,040 --> 00:05:15,840 a little bit longer than a week-- 107 00:05:15,840 --> 00:05:22,220 is that here at MIT, next Monday, this coming Monday 108 00:05:22,220 --> 00:05:25,670 is President's Day, a holiday. 109 00:05:25,670 --> 00:05:29,060 And then Tuesday is one of these strange days. 110 00:05:29,060 --> 00:05:31,940 You folks out in Singapore-- you may recall this. 111 00:05:31,940 --> 00:05:34,810 Tuesday is a Monday. 112 00:05:34,810 --> 00:05:36,370 So it's on a Monday class schedule, 113 00:05:36,370 --> 00:05:38,490 so we will not be meeting. 114 00:05:38,490 --> 00:05:40,540 OK? 115 00:05:40,540 --> 00:05:43,090 Go to whatever classes you would normally have on Monday. 116 00:05:43,090 --> 00:05:48,020 Go to those on Tuesday out here to MIT. 117 00:05:48,020 --> 00:05:51,790 And so then our next lecture will be a week from today, 118 00:05:51,790 --> 00:05:54,160 on Thursday. 119 00:05:54,160 --> 00:06:01,350 OK, so also going with the material 120 00:06:01,350 --> 00:06:02,940 that we're starting to talk about now 121 00:06:02,940 --> 00:06:07,500 and the second problem set that's just been released 122 00:06:07,500 --> 00:06:10,230 is another reading assignment. 123 00:06:10,230 --> 00:06:12,150 And in particular, what we're trying 124 00:06:12,150 --> 00:06:15,090 to do this year is focus-- 125 00:06:15,090 --> 00:06:19,980 do sort of the concentrated dive of May and Spanos Chapter 4. 126 00:06:19,980 --> 00:06:21,840 In the past, sometimes we've assigned 127 00:06:21,840 --> 00:06:25,110 instead Chapters 2 and 3 of Montgomery, 128 00:06:25,110 --> 00:06:28,140 which are much longer. 129 00:06:28,140 --> 00:06:32,580 Chapter 4 in Spanos is a shorter overview 130 00:06:32,580 --> 00:06:36,000 of basic statistical distributions 131 00:06:36,000 --> 00:06:42,030 and some very basic manipulation and use of statistics 132 00:06:42,030 --> 00:06:44,300 that we'll talk about today. 133 00:06:44,300 --> 00:06:47,640 So we're not formally assigning chapters 2 and 3 in Montgomery, 134 00:06:47,640 --> 00:06:49,380 but the intent here-- 135 00:06:49,380 --> 00:06:51,840 what I mean by this reading assignment is, 136 00:06:51,840 --> 00:06:54,210 if you don't have very much background on statistics, 137 00:06:54,210 --> 00:06:56,760 you may want to read Chapters 2 and 3. 138 00:06:56,760 --> 00:06:58,710 Or you can skim them to know what's 139 00:06:58,710 --> 00:07:02,790 in there, and in particular, that's probably the right place 140 00:07:02,790 --> 00:07:07,440 to go for a longer explanation, if you need more material. 141 00:07:07,440 --> 00:07:11,370 If May and Spanos is too condensed 142 00:07:11,370 --> 00:07:13,950 for you or you'd like additional examples, 143 00:07:13,950 --> 00:07:19,630 there's a very nice articulation in Montgomery. 144 00:07:19,630 --> 00:07:23,340 Now, I have not made a formal assignment of May and Spanos 145 00:07:23,340 --> 00:07:29,640 Chapter 3, but that is where you should 146 00:07:29,640 --> 00:07:32,400 go if you want to read more or need 147 00:07:32,400 --> 00:07:36,990 to read more about semiconductor fabrication process physics. 148 00:07:36,990 --> 00:07:39,570 That chapter is a very nice, again, 149 00:07:39,570 --> 00:07:43,170 relatively condensed description, 150 00:07:43,170 --> 00:07:51,180 articulation of the key process steps used in microfabrication. 151 00:07:51,180 --> 00:07:54,510 We'll, throughout the term, be popping in and out 152 00:07:54,510 --> 00:07:56,340 of a few of those unit processes, 153 00:07:56,340 --> 00:07:59,640 and I just want you to be aware that's what's in that chapter. 154 00:07:59,640 --> 00:08:05,310 And you will quite likely want to go in and skim that or read 155 00:08:05,310 --> 00:08:07,050 that, and dive in on some of those 156 00:08:07,050 --> 00:08:10,080 processes if you need more background 157 00:08:10,080 --> 00:08:11,230 on some of those things. 158 00:08:11,230 --> 00:08:14,220 And I think, especially before this tour 159 00:08:14,220 --> 00:08:19,290 that Hayden is organizing, it'd be at least really good 160 00:08:19,290 --> 00:08:22,470 to skim chapter 3. 161 00:08:22,470 --> 00:08:24,240 Again, it's mostly on the process physics, 162 00:08:24,240 --> 00:08:25,907 but every now and then, it does at least 163 00:08:25,907 --> 00:08:30,030 show you a little bit about the equipment that's used. 164 00:08:30,030 --> 00:08:32,070 And it's very nice to be able-- 165 00:08:32,070 --> 00:08:34,860 for Hayden to be able to point to some of the equipment 166 00:08:34,860 --> 00:08:39,000 and components of that, and have you visualize what's going on 167 00:08:39,000 --> 00:08:41,400 in the process physics, or be able to ask questions 168 00:08:41,400 --> 00:08:44,120 about that. 169 00:08:44,120 --> 00:08:48,260 OK, so what I'm going to do here at the start 170 00:08:48,260 --> 00:08:53,120 is just run through some example processes. 171 00:08:53,120 --> 00:08:59,190 And in particular, these are little not quite 172 00:08:59,190 --> 00:09:00,960 manufacturing processes. 173 00:09:00,960 --> 00:09:04,950 They are more like research scale implementations 174 00:09:04,950 --> 00:09:10,650 of some basic manufacturing processes that we have actually 175 00:09:10,650 --> 00:09:13,650 generated some data from, or students have generated data 176 00:09:13,650 --> 00:09:15,460 from over the years. 177 00:09:15,460 --> 00:09:17,880 So I'm going to show you some actual measurements 178 00:09:17,880 --> 00:09:20,580 from a few simple cases. 179 00:09:20,580 --> 00:09:23,040 And we're going to look at that data, 180 00:09:23,040 --> 00:09:29,330 and what I think we're going to see is variation. 181 00:09:29,330 --> 00:09:33,860 And the puzzle for you and for us-- 182 00:09:33,860 --> 00:09:36,680 and to some extent, for me, because 183 00:09:36,680 --> 00:09:38,930 in many of these processes, I was not 184 00:09:38,930 --> 00:09:41,280 involved in the generation of the data-- 185 00:09:41,280 --> 00:09:43,520 our puzzle here is to look at the data 186 00:09:43,520 --> 00:09:48,620 and try to think about what might be going on in the data. 187 00:09:48,620 --> 00:09:51,990 So here's some-- a basic turning process. 188 00:09:51,990 --> 00:09:53,660 So we've got a work piece. 189 00:09:53,660 --> 00:09:59,990 We've got under computerized numerical controls, CNC. 190 00:09:59,990 --> 00:10:06,620 We've got the ability to cut into that work piece, which 191 00:10:06,620 --> 00:10:10,220 is rotating, to try to get to diameter work pieces 192 00:10:10,220 --> 00:10:15,120 of a specified dimension. 193 00:10:15,120 --> 00:10:19,880 So here's the diameter measured on a particular location 194 00:10:19,880 --> 00:10:23,750 on that work piece as a function of 47-- 195 00:10:23,750 --> 00:10:26,570 I think it's 47 or 48-- 196 00:10:26,570 --> 00:10:28,370 different repetitions. 197 00:10:28,370 --> 00:10:32,300 Now, I believe this may have involved 198 00:10:32,300 --> 00:10:36,230 more than one set of students, or more than one class. 199 00:10:36,230 --> 00:10:39,020 We can think of that as a shift change 200 00:10:39,020 --> 00:10:41,600 in a manufacturing process. 201 00:10:41,600 --> 00:10:47,870 We're simply gathering here a measurement of a diameter 202 00:10:47,870 --> 00:10:50,120 on that work piece. 203 00:10:50,120 --> 00:10:51,390 So talk to me. 204 00:10:51,390 --> 00:10:52,520 Tell me what you see. 205 00:10:55,767 --> 00:10:56,350 Is it uniform? 206 00:10:58,915 --> 00:11:00,700 Not especially-- although you always 207 00:11:00,700 --> 00:11:02,620 have to be really careful looking, 208 00:11:02,620 --> 00:11:05,950 because on any kind of plot, you want 209 00:11:05,950 --> 00:11:07,600 to really look at the spread. 210 00:11:07,600 --> 00:11:11,410 Here it's kind of magnified, because here's 0.7 inches, 211 00:11:11,410 --> 00:11:22,510 and we're down to about 0.697 up to 0.702. 212 00:11:22,510 --> 00:11:24,790 That's not full scale. 213 00:11:24,790 --> 00:11:28,460 But if we zoom in around whatever nominal is-- 214 00:11:28,460 --> 00:11:32,380 and in fact, it's not even clear what the target dimension was-- 215 00:11:32,380 --> 00:11:34,690 I might guess and think that it was 0.7 inches, 216 00:11:34,690 --> 00:11:38,410 because that's a nice round number, but who knows? 217 00:11:38,410 --> 00:11:40,780 So it's definitely not uniform. 218 00:11:40,780 --> 00:11:42,000 What else do you see? 219 00:11:42,000 --> 00:11:43,262 Yeah? 220 00:11:43,262 --> 00:11:46,542 AUDIENCE: The data is sort of drifting up over time. 221 00:11:46,542 --> 00:11:47,250 DUANE BONING: OK. 222 00:11:47,250 --> 00:11:50,910 So it looks like there might be a long-term trend of a drift 223 00:11:50,910 --> 00:11:54,840 over the number of runs. 224 00:11:54,840 --> 00:11:57,330 What might lead you to think that that would 225 00:11:57,330 --> 00:12:00,264 be a physically reasonable-- 226 00:12:00,264 --> 00:12:04,240 AUDIENCE: [INAUDIBLE] 227 00:12:07,277 --> 00:12:08,110 DUANE BONING: Right. 228 00:12:08,110 --> 00:12:12,240 So if the tool wears-- and in fact, if the tool wears 229 00:12:12,240 --> 00:12:16,380 or some long-term change in the state of the equipment-- 230 00:12:16,380 --> 00:12:22,220 that's often the source of these long-term trends. 231 00:12:22,220 --> 00:12:23,780 Other things people see-- 232 00:12:28,400 --> 00:12:29,210 Yeah? 233 00:12:29,210 --> 00:12:30,710 AUDIENCE: It's fairly systematic. 234 00:12:30,710 --> 00:12:33,110 It constantly goes up and down, and up and down. 235 00:12:33,110 --> 00:12:36,080 There's only two or three points before it goes back up. 236 00:12:36,080 --> 00:12:38,450 DUANE BONING: Yeah, that's very interesting. 237 00:12:38,450 --> 00:12:41,030 Even the variation here-- 238 00:12:41,030 --> 00:12:46,220 it seems to be cyclical, or periodic, or something 239 00:12:46,220 --> 00:12:48,980 systematic going on in there. 240 00:12:51,600 --> 00:12:54,270 Again, I wasn't there for the generation of this data, 241 00:12:54,270 --> 00:12:56,190 but what might some reasons for that be? 242 00:12:58,495 --> 00:12:59,995 AUDIENCE: The worker might be trying 243 00:12:59,995 --> 00:13:03,260 to correct with the subsequent one, 244 00:13:03,260 --> 00:13:05,467 and he overshoots constantly. 245 00:13:05,467 --> 00:13:06,800 DUANE BONING: Interesting-- yes. 246 00:13:09,990 --> 00:13:11,100 That's a good idea. 247 00:13:11,100 --> 00:13:15,000 I guess, in fact, one can imagine, especially 248 00:13:15,000 --> 00:13:17,670 if you're taking a measurement after every tool, 249 00:13:17,670 --> 00:13:25,240 and the target was 0.7, say, then maybe, near the beginning, 250 00:13:25,240 --> 00:13:26,917 we're starting to see a deviation. 251 00:13:26,917 --> 00:13:28,750 Maybe this one was a little low, and they're 252 00:13:28,750 --> 00:13:32,903 trying to get back up, making an overcorrection-- because 253 00:13:32,903 --> 00:13:35,320 near the end here, it looks like there's a little bit more 254 00:13:35,320 --> 00:13:37,464 centered around 0.7. 255 00:13:37,464 --> 00:13:39,089 AUDIENCE: I think that might be because 256 00:13:39,089 --> 00:13:41,570 of backlash [INAUDIBLE]. 257 00:13:44,922 --> 00:13:45,630 DUANE BONING: OK. 258 00:13:45,630 --> 00:13:49,220 AUDIENCE: [INAUDIBLE] 259 00:13:49,220 --> 00:13:53,030 DUANE BONING: So backlash being an operator adjustment 260 00:13:53,030 --> 00:13:56,620 that is made as perhaps part of this compensation strategy? 261 00:13:56,620 --> 00:13:57,500 AUDIENCE: [INAUDIBLE] 262 00:13:57,500 --> 00:13:58,625 DUANE BONING: Interesting-- 263 00:13:58,625 --> 00:14:00,593 AUDIENCE: [INAUDIBLE] 264 00:14:04,690 --> 00:14:06,190 DUANE BONING: Anybody in Singapore-- 265 00:14:06,190 --> 00:14:08,410 any other observations or ideas? 266 00:14:11,858 --> 00:14:14,180 AUDIENCE: Since measurement is measured 267 00:14:14,180 --> 00:14:17,070 at a different place of the things, 268 00:14:17,070 --> 00:14:22,090 so maybe you can see every three point 269 00:14:22,090 --> 00:14:24,500 will be a larger diameter. 270 00:14:24,500 --> 00:14:30,980 I think maybe this is measured at the other side 271 00:14:30,980 --> 00:14:33,050 of the thing [INAUDIBLE]. 272 00:14:33,050 --> 00:14:36,280 And maybe for the lower-- for the smaller diameter, 273 00:14:36,280 --> 00:14:39,320 it's measured in the inner side. 274 00:14:39,320 --> 00:14:41,600 DUANE BONING: Yes, it's quite possible-- 275 00:14:41,600 --> 00:14:42,960 at least on this plot. 276 00:14:42,960 --> 00:14:46,940 We might learn more if we look at more data. 277 00:14:46,940 --> 00:14:51,380 I read this as run number, rather than measurement number. 278 00:14:51,380 --> 00:14:55,520 So somehow, perhaps, this is intended 279 00:14:55,520 --> 00:14:57,620 to be representative of the whole part, 280 00:14:57,620 --> 00:15:00,590 but it's possible those are multiple measurements 281 00:15:00,590 --> 00:15:02,060 on different part-- 282 00:15:02,060 --> 00:15:04,520 portions of an overall turning. 283 00:15:07,190 --> 00:15:11,490 You could start to look and hypothesize on that. 284 00:15:11,490 --> 00:15:16,440 Is the 1, 2, 3-- is that the same pattern you see each time? 285 00:15:16,440 --> 00:15:19,610 And it's not clear whether we see exactly that same pattern 286 00:15:19,610 --> 00:15:21,140 or not. 287 00:15:21,140 --> 00:15:22,375 Other ideas? 288 00:15:22,375 --> 00:15:26,740 AUDIENCE: [INAUDIBLE] 289 00:15:29,857 --> 00:15:30,690 DUANE BONING: Right. 290 00:15:30,690 --> 00:15:35,440 So run number is a stand-in for time, and these are occurring. 291 00:15:35,440 --> 00:15:37,770 So your point here is-- 292 00:15:37,770 --> 00:15:38,910 AUDIENCE: [INAUDIBLE] 293 00:15:38,910 --> 00:15:39,702 DUANE BONING: Sure. 294 00:15:39,702 --> 00:15:40,597 AUDIENCE: [INAUDIBLE] 295 00:15:40,597 --> 00:15:41,430 DUANE BONING: Right. 296 00:15:41,430 --> 00:15:43,560 So it may not be equipment state, 297 00:15:43,560 --> 00:15:47,220 or indirectly, it may be also the rest of the facility 298 00:15:47,220 --> 00:15:49,470 environment or the line. 299 00:15:49,470 --> 00:15:51,370 And over time, who knows? 300 00:15:51,370 --> 00:15:53,760 Maybe this thing starts heating up, 301 00:15:53,760 --> 00:15:56,700 or the work pieces that are coming in 302 00:15:56,700 --> 00:15:58,560 have a different temperature-- 303 00:15:58,560 --> 00:16:02,880 quite possible that there could be some other long-term state 304 00:16:02,880 --> 00:16:05,040 change. 305 00:16:05,040 --> 00:16:08,760 We do notice there's this shift change. 306 00:16:08,760 --> 00:16:13,770 And referring back to this long-term trend, 307 00:16:13,770 --> 00:16:17,520 if we actually look at this early part, 308 00:16:17,520 --> 00:16:20,550 it may or may not be drifting that much. 309 00:16:20,550 --> 00:16:22,440 It may actually be fairly steady. 310 00:16:22,440 --> 00:16:24,570 And what we're actually seeing in here 311 00:16:24,570 --> 00:16:29,010 is an overall mean shift, but maybe not even a long-term 312 00:16:29,010 --> 00:16:29,710 trend. 313 00:16:29,710 --> 00:16:33,600 So one might actually pose, maybe even apply 314 00:16:33,600 --> 00:16:36,370 some statistical methods to look and say, 315 00:16:36,370 --> 00:16:39,480 is there a statistically significant mean difference 316 00:16:39,480 --> 00:16:44,020 between this set and this set of data? 317 00:16:44,020 --> 00:16:47,040 So it may look like a long-term trend, 318 00:16:47,040 --> 00:16:50,670 or it may simply be a reflection that there 319 00:16:50,670 --> 00:16:54,360 is a setup change on the equipment when 320 00:16:54,360 --> 00:16:56,160 a new shift comes in, or there may 321 00:16:56,160 --> 00:16:59,850 be just inherent differences in how the operator interacts 322 00:16:59,850 --> 00:17:01,050 with the equipment. 323 00:17:01,050 --> 00:17:04,202 How they load the part might be slightly different. 324 00:17:08,270 --> 00:17:11,470 Any other ideas? 325 00:17:11,470 --> 00:17:12,525 Let's go back to the-- 326 00:17:12,525 --> 00:17:13,900 because I think you had mentioned 327 00:17:13,900 --> 00:17:20,170 something interesting about possible oscillation in here. 328 00:17:20,170 --> 00:17:26,680 What might other sources or causes of oscillatory behavior, 329 00:17:26,680 --> 00:17:34,290 or what looks like two distinct sets of data here-- 330 00:17:34,290 --> 00:17:37,447 in fact, I'm not completely sure that it's purely oscillatory, 331 00:17:37,447 --> 00:17:39,030 because you can come down here, and it 332 00:17:39,030 --> 00:17:42,150 seems like sometimes you've got two things that 333 00:17:42,150 --> 00:17:46,380 are down low, and then one up, or three things that are low, 334 00:17:46,380 --> 00:17:47,250 and one up. 335 00:17:47,250 --> 00:17:49,250 AUDIENCE: Well, this is a little bit farfetched, 336 00:17:49,250 --> 00:17:51,570 but maybe you could have [? stock ?] material 337 00:17:51,570 --> 00:17:55,260 of maybe 1 meter, and they always 338 00:17:55,260 --> 00:17:57,310 cut it into three pieces. 339 00:17:57,310 --> 00:18:00,390 So they always have the first piece of stock material, 340 00:18:00,390 --> 00:18:03,630 and they had some material properties that 341 00:18:03,630 --> 00:18:06,240 led to have [INAUDIBLE] maybe. 342 00:18:06,240 --> 00:18:08,550 And the middle piece led to middle dimensions, 343 00:18:08,550 --> 00:18:11,520 and the last piece led to smaller dimensions. 344 00:18:11,520 --> 00:18:13,270 DUANE BONING: I like that. 345 00:18:13,270 --> 00:18:14,980 That might be very interesting. 346 00:18:14,980 --> 00:18:19,050 So if I were generalizing that, it may well be-- 347 00:18:19,050 --> 00:18:22,920 very often, there may be more than one set 348 00:18:22,920 --> 00:18:25,200 of starting material. 349 00:18:25,200 --> 00:18:28,600 So it may have been all one piece and cut into three, 350 00:18:28,600 --> 00:18:31,500 and so they're randomly picking one of the three, 351 00:18:31,500 --> 00:18:34,170 and therefore, you're getting a different mix of those three 352 00:18:34,170 --> 00:18:36,330 points in here. 353 00:18:36,330 --> 00:18:39,420 But in general, it may well be in many cases 354 00:18:39,420 --> 00:18:45,090 that you've got two different incoming streams. 355 00:18:45,090 --> 00:18:48,950 If I were looking at this, instead of thinking of three, 356 00:18:48,950 --> 00:18:54,240 it kind of almost looks to me like this set of data here-- 357 00:18:54,240 --> 00:18:57,060 it's kind of randomly around some value, 358 00:18:57,060 --> 00:19:00,850 and this is a different set of data around some other value. 359 00:19:00,850 --> 00:19:05,010 So it may be that there's just two boxes apart sitting there, 360 00:19:05,010 --> 00:19:08,340 and they're just pulling, randomly, starting material out 361 00:19:08,340 --> 00:19:10,060 of one of the two. 362 00:19:10,060 --> 00:19:11,100 Yeah? 363 00:19:11,100 --> 00:19:12,683 AUDIENCE: I also don't know if there's 364 00:19:12,683 --> 00:19:15,455 any control on how far from the [INAUDIBLE] the materials 365 00:19:15,455 --> 00:19:18,270 [INAUDIBLE] you could put it in, be a little bit farther, 366 00:19:18,270 --> 00:19:20,220 and then it would have a larger cantilever 367 00:19:20,220 --> 00:19:22,200 arm than the next user. 368 00:19:22,200 --> 00:19:23,390 DUANE BONING: Right, right. 369 00:19:23,390 --> 00:19:24,810 So there could be lots of sources 370 00:19:24,810 --> 00:19:29,630 of both random and deterministic variations. 371 00:19:29,630 --> 00:19:34,980 So we talked in here about things like a mean shift. 372 00:19:34,980 --> 00:19:38,850 Maybe there's a deviation between these two shift 373 00:19:38,850 --> 00:19:39,400 changes. 374 00:19:39,400 --> 00:19:44,580 And that would be a either deterministic or-- 375 00:19:44,580 --> 00:19:49,290 systematic maybe is a better word of a deviation. 376 00:19:49,290 --> 00:19:52,080 And then, within different sets, we 377 00:19:52,080 --> 00:19:56,550 may have no detailed explanation of why 378 00:19:56,550 --> 00:19:59,970 we have these small deviations, and a random model 379 00:19:59,970 --> 00:20:02,850 for some of those components might be appropriate. 380 00:20:02,850 --> 00:20:06,460 So that's part of the idea that we're after. 381 00:20:06,460 --> 00:20:13,980 So over here on the side, we're just referring back again to 382 00:20:13,980 --> 00:20:15,300 of our variation equation. 383 00:20:15,300 --> 00:20:18,803 Maybe I'll put that up so that we can also 384 00:20:18,803 --> 00:20:20,970 refer to that, because I think we've touched on some 385 00:20:20,970 --> 00:20:24,906 of these different components. 386 00:20:24,906 --> 00:20:28,080 We said that our deviations in our output 387 00:20:28,080 --> 00:20:33,510 y you can often characterize as some component that 388 00:20:33,510 --> 00:20:39,060 has a sensitivity to disturbances in our process 389 00:20:39,060 --> 00:20:44,250 parameters, and those disturbances, 390 00:20:44,250 --> 00:20:50,370 and some component that relates to our controllability 391 00:20:50,370 --> 00:20:53,370 when we make small changes [INAUDIBLE] 392 00:20:53,370 --> 00:20:55,230 and then control inputs. 393 00:20:58,220 --> 00:21:00,460 Among other things, we said, well, 394 00:21:00,460 --> 00:21:04,320 maybe some of our input feedstock, maybe delta a's-- 395 00:21:04,320 --> 00:21:06,547 changes in the parameters. 396 00:21:06,547 --> 00:21:08,880 We also-- and I thought it was very interesting, talking 397 00:21:08,880 --> 00:21:13,080 about overcompensation in the control scenario. 398 00:21:13,080 --> 00:21:14,640 It might be, in fact, that there are 399 00:21:14,640 --> 00:21:19,928 deviations or errors in active control, changes 400 00:21:19,928 --> 00:21:20,595 to the settings. 401 00:21:23,790 --> 00:21:26,280 There may also be-- and we also talked about it here-- 402 00:21:26,280 --> 00:21:35,400 is recognition that the function y of our alpha may, in fact, 403 00:21:35,400 --> 00:21:42,800 be time-dependent, or that-- 404 00:21:42,800 --> 00:21:44,190 put it another way-- 405 00:21:44,190 --> 00:21:49,760 some of the delta a's have time components and time trends 406 00:21:49,760 --> 00:21:52,460 to them. 407 00:21:52,460 --> 00:21:56,270 We'll come back, and I'm sure come up with more examples 408 00:21:56,270 --> 00:21:58,680 as we look a little bit further. 409 00:21:58,680 --> 00:22:01,850 And I think this is some of the same process. 410 00:22:01,850 --> 00:22:07,880 It has different run numbers on it, but this is explicitly-- 411 00:22:07,880 --> 00:22:10,370 it looks like there's, again, a dimension and run 412 00:22:10,370 --> 00:22:12,870 number along the horizontal axis, but in this case, 413 00:22:12,870 --> 00:22:16,940 there's the yellow, our inner; the purple, our middle; 414 00:22:16,940 --> 00:22:20,310 and the blue, our outer dimensions. 415 00:22:20,310 --> 00:22:23,180 So you can imagine here that perhaps measurements 416 00:22:23,180 --> 00:22:30,160 are being made at different locations on the work piece. 417 00:22:30,160 --> 00:22:34,650 And I believe that this line right here 418 00:22:34,650 --> 00:22:38,970 is, again, a shift change. 419 00:22:38,970 --> 00:22:40,920 So talk to me a little bit about this. 420 00:22:40,920 --> 00:22:44,610 What else do we notice in this case? 421 00:22:44,610 --> 00:22:49,070 There's one important difference in this data compared 422 00:22:49,070 --> 00:22:54,530 to this data that I'm looking for in particular. 423 00:22:54,530 --> 00:22:57,350 What additional characteristics-- or what else 424 00:22:57,350 --> 00:23:00,170 could you say about this data? 425 00:23:00,170 --> 00:23:04,510 AUDIENCE: [INAUDIBLE] I guess the spread on the [INAUDIBLE].. 426 00:23:11,512 --> 00:23:12,220 DUANE BONING: OK. 427 00:23:12,220 --> 00:23:16,150 So one observation is, if we were just looking-- 428 00:23:16,150 --> 00:23:18,130 well, right here, this is a good one, 429 00:23:18,130 --> 00:23:20,860 where inner, middle, and outer are all close together. 430 00:23:20,860 --> 00:23:23,200 But your point is, generally, we're 431 00:23:23,200 --> 00:23:27,770 seeing a very huge spread in the measurement 432 00:23:27,770 --> 00:23:33,590 within that particular part than we do over here. 433 00:23:33,590 --> 00:23:36,460 AUDIENCE: [INAUDIBLE] 434 00:23:41,400 --> 00:23:42,930 DUANE BONING: Much tighter band-- 435 00:23:42,930 --> 00:23:45,693 so now you're talking like a band here? 436 00:23:45,693 --> 00:23:49,410 AUDIENCE: Yeah, that [INAUDIBLE].. 437 00:23:49,410 --> 00:23:51,700 DUANE BONING: Good, good-- 438 00:23:51,700 --> 00:23:52,540 yep? 439 00:23:52,540 --> 00:23:54,460 AUDIENCE: [INAUDIBLE] 440 00:24:07,405 --> 00:24:08,780 DUANE BONING: Interesting-- yeah, 441 00:24:08,780 --> 00:24:11,540 so it's a situation here where we don't know 442 00:24:11,540 --> 00:24:15,950 the specifications, so it's a little hard to actually know 443 00:24:15,950 --> 00:24:17,000 which is better-- 444 00:24:17,000 --> 00:24:19,950 because your point you were making is, in this case, 445 00:24:19,950 --> 00:24:22,160 the blue's always above-- 446 00:24:22,160 --> 00:24:24,860 or almost always above the purple, 447 00:24:24,860 --> 00:24:25,940 always above the yellow. 448 00:24:25,940 --> 00:24:28,190 And it may well be that it's meant 449 00:24:28,190 --> 00:24:30,620 to have a slight taper to it. 450 00:24:30,620 --> 00:24:31,505 We'd need to more. 451 00:24:31,505 --> 00:24:33,380 And actually, that's a really important point 452 00:24:33,380 --> 00:24:35,310 in manufacturing processes. 453 00:24:35,310 --> 00:24:37,580 You can't just look at the data and always know 454 00:24:37,580 --> 00:24:38,310 what's going on. 455 00:24:38,310 --> 00:24:40,880 You also have to have information from design, 456 00:24:40,880 --> 00:24:46,190 know what the intended results are. 457 00:24:46,190 --> 00:24:49,310 So certainly, just to highlight a couple 458 00:24:49,310 --> 00:24:55,730 of the things [INAUDIBLE] mentioned, 459 00:24:55,730 --> 00:24:58,580 one here is, very often, we are also 460 00:24:58,580 --> 00:25:02,700 interested in within part variation. 461 00:25:02,700 --> 00:25:04,820 So assuming that maybe they all really 462 00:25:04,820 --> 00:25:06,740 were supposed to have the same dimension, 463 00:25:06,740 --> 00:25:11,780 then this spread here is very different than this 464 00:25:11,780 --> 00:25:12,530 spread here. 465 00:25:12,530 --> 00:25:15,770 So characterization of within part variation 466 00:25:15,770 --> 00:25:18,170 may be very important in understanding that. 467 00:25:20,870 --> 00:25:25,390 And then the other thing here is, unlike this-- 468 00:25:25,390 --> 00:25:27,220 we still have all that gook on here, 469 00:25:27,220 --> 00:25:31,960 but in here we were basically having a similar band, 470 00:25:31,960 --> 00:25:35,590 and it was all just kind of moving or shifting. 471 00:25:35,590 --> 00:25:39,730 But the part-to-part variation did not seem to change. 472 00:25:39,730 --> 00:25:42,740 The variance of the process did not seem to change. 473 00:25:42,740 --> 00:25:44,770 The mean shifted. 474 00:25:44,770 --> 00:25:46,690 So that's a very common characteristic 475 00:25:46,690 --> 00:25:48,080 that we're worried about. 476 00:25:48,080 --> 00:25:51,680 And your point here is that, in this case, 477 00:25:51,680 --> 00:25:55,960 this looks like a much tighter band, or the standard deviation 478 00:25:55,960 --> 00:26:00,640 of the invariance of the process in a run-to-run sense 479 00:26:00,640 --> 00:26:04,080 is very different than in this process. 480 00:26:04,080 --> 00:26:07,040 And so it may be somebody with a slightly steadier hands, 481 00:26:07,040 --> 00:26:11,540 or a more consistent process, or a more consistent operator, 482 00:26:11,540 --> 00:26:14,660 or the equipment-- maybe some change was made-- 483 00:26:14,660 --> 00:26:16,610 improvement to the equipment saying, 484 00:26:16,610 --> 00:26:19,640 hm, that little screw is a little loose. 485 00:26:19,640 --> 00:26:22,160 Let me tighten that down. 486 00:26:22,160 --> 00:26:23,120 Who knows? 487 00:26:23,120 --> 00:26:28,980 But something clearly changed in order 488 00:26:28,980 --> 00:26:31,973 to apparently improve the process. 489 00:26:31,973 --> 00:26:33,765 So those are a couple more characteristics. 490 00:26:33,765 --> 00:26:34,640 AUDIENCE: [INAUDIBLE] 491 00:26:34,640 --> 00:26:35,870 DUANE BONING: Yes-- question? 492 00:26:35,870 --> 00:26:36,745 AUDIENCE: Professor-- 493 00:26:36,745 --> 00:26:37,650 DUANE BONING: Yes? 494 00:26:37,650 --> 00:26:39,770 AUDIENCE: Since you mentioned that maybe 495 00:26:39,770 --> 00:26:42,740 with the shift change there's some improvement 496 00:26:42,740 --> 00:26:45,350 in the [INAUDIBLE] of data, and that's 497 00:26:45,350 --> 00:26:49,390 why it gives us a smaller band right now. 498 00:26:49,390 --> 00:26:53,510 But I noticed that the outer, middle, and inner data-- 499 00:26:53,510 --> 00:26:56,710 they are pretty close to each other, 500 00:26:56,710 --> 00:26:59,600 as in the difference between these three sets of data 501 00:26:59,600 --> 00:27:03,320 are much wider in the previous shift than in this right now. 502 00:27:03,320 --> 00:27:08,300 So can we still say that maybe this-- 503 00:27:08,300 --> 00:27:11,870 the object we are measuring is tapered? 504 00:27:11,870 --> 00:27:14,780 Because it doesn't look like it's tapered right now. 505 00:27:14,780 --> 00:27:18,740 DUANE BONING: Absolutely-- well, but the point 506 00:27:18,740 --> 00:27:22,770 was made that we don't actually know the design intent, 507 00:27:22,770 --> 00:27:24,710 so this was a hypothesis in here. 508 00:27:24,710 --> 00:27:27,740 I think it's probably unlikely that there 509 00:27:27,740 --> 00:27:30,890 was an intended taper, and I'll agree with you. 510 00:27:30,890 --> 00:27:35,450 But in here, it looks like there is 511 00:27:35,450 --> 00:27:40,430 somewhat closer within part correspondence 512 00:27:40,430 --> 00:27:43,560 in these dimensions. 513 00:27:43,560 --> 00:27:46,848 OK, so let me go to one more. 514 00:27:46,848 --> 00:27:47,640 Here's another one. 515 00:27:47,640 --> 00:27:51,220 This is brake bending of a metal sheet. 516 00:27:51,220 --> 00:27:53,610 The basic idea in this process is 517 00:27:53,610 --> 00:27:57,840 we're trying to change the geometry under the application 518 00:27:57,840 --> 00:28:02,310 of force in a bending process. 519 00:28:02,310 --> 00:28:05,660 So we might have some tooling. 520 00:28:05,660 --> 00:28:09,980 We have some piece of metal, say, and we press down, 521 00:28:09,980 --> 00:28:13,880 and the intent is to create some permanent deformation 522 00:28:13,880 --> 00:28:14,990 in the part. 523 00:28:14,990 --> 00:28:16,940 An important component here is that, when 524 00:28:16,940 --> 00:28:20,700 we remove that force, remove that pressure, 525 00:28:20,700 --> 00:28:25,120 there can be spring back in that part-- 526 00:28:25,120 --> 00:28:31,040 so little bit of the inherent characteristics of this. 527 00:28:31,040 --> 00:28:35,320 And so here's some data looking at using the same tooling, 528 00:28:35,320 --> 00:28:42,880 and I believe the same set of downforce, with-- 529 00:28:42,880 --> 00:28:48,420 I believe this is a measurement of the resulting angle. 530 00:28:48,420 --> 00:28:50,670 Alpha is not a good variable to use here, is it? 531 00:28:53,790 --> 00:29:02,790 We call that just angle of the resulting component. 532 00:29:02,790 --> 00:29:06,660 And what else is changing here is that the source material 533 00:29:06,660 --> 00:29:08,170 is changing. 534 00:29:08,170 --> 00:29:12,630 So in some cases, we're using aluminum, some cases steel-- 535 00:29:12,630 --> 00:29:16,900 and also with different thicknesses. 536 00:29:16,900 --> 00:29:20,520 So I won't even ask you to share-- 537 00:29:20,520 --> 00:29:26,040 the most obvious observation is, depending on the material type, 538 00:29:26,040 --> 00:29:31,050 we certainly get different angles, different degrees 539 00:29:31,050 --> 00:29:33,120 of spring back. 540 00:29:36,580 --> 00:29:37,870 That's the most obvious thing. 541 00:29:37,870 --> 00:29:41,200 What else do you see in this? 542 00:29:41,200 --> 00:29:42,280 Yeah? 543 00:29:42,280 --> 00:29:45,142 AUDIENCE: It depends on the thickness of the material. 544 00:29:45,142 --> 00:29:45,850 DUANE BONING: OK. 545 00:29:48,610 --> 00:29:51,040 AUDIENCE: So for the same material [INAUDIBLE] 546 00:29:51,040 --> 00:29:52,570 the springback, it will be more. 547 00:29:55,330 --> 00:29:56,470 DUANE BONING: Right. 548 00:29:56,470 --> 00:30:00,130 OK, and one could certainly imagine good physical reasons 549 00:30:00,130 --> 00:30:04,870 for that, that thicker material is 550 00:30:04,870 --> 00:30:11,590 going to perhaps spring back more readily. 551 00:30:11,590 --> 00:30:13,810 And one could even go off and imagine 552 00:30:13,810 --> 00:30:17,380 trying to build a first principle physical model that 553 00:30:17,380 --> 00:30:20,140 would tell you what the resulting angle 554 00:30:20,140 --> 00:30:24,080 or degree of spring back would be as a function of that. 555 00:30:24,080 --> 00:30:26,440 And in fact, it's probably worth it 556 00:30:26,440 --> 00:30:28,150 to go and consult some literature 557 00:30:28,150 --> 00:30:32,320 and see if there is physical insight. 558 00:30:32,320 --> 00:30:36,850 But if I were to ask you to actually tell me what you think 559 00:30:36,850 --> 00:30:44,660 the result might be if we had a 0.45-inch steel part, 560 00:30:44,660 --> 00:30:47,090 what would your first inclination be? 561 00:30:47,090 --> 00:30:50,540 Would it be to go to the technical literature 562 00:30:50,540 --> 00:30:52,820 and look up a physical model for this? 563 00:30:57,510 --> 00:31:01,080 AUDIENCE: [INAUDIBLE] averaging between the [INAUDIBLE].. 564 00:31:01,080 --> 00:31:02,500 DUANE BONING: Yeah. 565 00:31:02,500 --> 00:31:04,870 You would be basically using some averaging 566 00:31:04,870 --> 00:31:07,970 to deal with noise and manufacturing variation, 567 00:31:07,970 --> 00:31:12,370 but you would be building a very simple empirical model based 568 00:31:12,370 --> 00:31:16,090 on the data that you have, and then interpolating in some way. 569 00:31:16,090 --> 00:31:18,230 And here we've only got two data points, 570 00:31:18,230 --> 00:31:20,890 so you might pick the middle. 571 00:31:20,890 --> 00:31:22,030 We don't know. 572 00:31:22,030 --> 00:31:25,090 You might want to actually go and design some experiment 573 00:31:25,090 --> 00:31:27,840 where you add a third point so you 574 00:31:27,840 --> 00:31:31,070 know if there's some non-linear dependence, and so on. 575 00:31:31,070 --> 00:31:33,410 So that's certainly an important thing. 576 00:31:33,410 --> 00:31:35,560 There is a very clear input-output, 577 00:31:35,560 --> 00:31:40,180 a deterministic effect that, again, you 578 00:31:40,180 --> 00:31:42,730 would like to be able to understand enough to be 579 00:31:42,730 --> 00:31:46,160 able to deal with that. 580 00:31:46,160 --> 00:31:51,090 And the same may also be true of other discrete design choices-- 581 00:31:51,090 --> 00:31:53,408 discrete choice being the material, 582 00:31:53,408 --> 00:31:54,700 whether it's aluminum or steel. 583 00:31:54,700 --> 00:31:58,895 You would like to know, is it a similar trend? 584 00:31:58,895 --> 00:32:00,270 One thing that's very interesting 585 00:32:00,270 --> 00:32:03,850 here is it appears that the delta 586 00:32:03,850 --> 00:32:05,500 as a function of those thicknesses 587 00:32:05,500 --> 00:32:08,830 is about the same-- very nearly the same 588 00:32:08,830 --> 00:32:11,300 in the two material cases. 589 00:32:11,300 --> 00:32:14,470 And then there's a material delta. 590 00:32:14,470 --> 00:32:16,850 So I'm already starting to think, OK, 591 00:32:16,850 --> 00:32:21,990 I can have a very simple additive empirical model that 592 00:32:21,990 --> 00:32:25,960 has a delta effect due to the material type. 593 00:32:25,960 --> 00:32:28,270 It's a binary choice-- 594 00:32:28,270 --> 00:32:30,550 or a binary coefficient, depending on that. 595 00:32:30,550 --> 00:32:33,690 And then an additional component is a function of thickness, 596 00:32:33,690 --> 00:32:36,960 and maybe that function of the thickness is independent. 597 00:32:36,960 --> 00:32:40,080 That delta from amine is independent [INAUDIBLE] 598 00:32:40,080 --> 00:32:42,140 material type. 599 00:32:42,140 --> 00:32:44,620 OK, anything else going on? 600 00:32:44,620 --> 00:32:45,375 Yes? 601 00:32:45,375 --> 00:32:51,260 AUDIENCE: [INAUDIBLE] there is kind 602 00:32:51,260 --> 00:32:53,248 of a drop in the middle of the set. 603 00:32:55,996 --> 00:32:56,820 [INAUDIBLE] 604 00:33:02,440 --> 00:33:04,240 DUANE BONING: Yes-- very interesting, 605 00:33:04,240 --> 00:33:07,480 and maybe a hint almost of something, but then it-- 606 00:33:07,480 --> 00:33:08,020 yeah. 607 00:33:08,020 --> 00:33:11,810 So there might be something systematic going on there. 608 00:33:11,810 --> 00:33:14,500 The human eye is wonderful for looking for patterns, isn't it? 609 00:33:17,010 --> 00:33:18,750 Actually, one of the challenges often 610 00:33:18,750 --> 00:33:23,640 is using both engineering judgment and when 611 00:33:23,640 --> 00:33:27,030 the data is simple enough to detect trends, but also 612 00:33:27,030 --> 00:33:30,930 have statistical methods when the volume of data is huge 613 00:33:30,930 --> 00:33:33,270 or there are many things changing at the same time 614 00:33:33,270 --> 00:33:35,880 to detect possible shifts and trends. 615 00:33:38,990 --> 00:33:39,760 Anything else? 616 00:33:44,380 --> 00:33:45,570 Yeah, you had more. 617 00:33:45,570 --> 00:33:47,388 AUDIENCE: OK. 618 00:33:47,388 --> 00:33:49,810 [INAUDIBLE] 619 00:33:49,810 --> 00:33:50,560 DUANE BONING: Yes. 620 00:33:50,560 --> 00:33:53,871 AUDIENCE: [INAUDIBLE] 621 00:33:57,763 --> 00:34:00,180 DUANE BONING: So you're worried about this, and maybe this 622 00:34:00,180 --> 00:34:00,660 also. 623 00:34:00,660 --> 00:34:00,750 AUDIENCE: Yeah. 624 00:34:00,750 --> 00:34:02,250 DUANE BONING: In fact, we don't even 625 00:34:02,250 --> 00:34:06,090 know if that's a still component or an aluminum. 626 00:34:06,090 --> 00:34:07,290 Yeah. 627 00:34:07,290 --> 00:34:10,590 So in some sense here, if I were looking at this data, 628 00:34:10,590 --> 00:34:14,940 I almost start to think these are relatively-- 629 00:34:14,940 --> 00:34:18,810 well, you'd have to ask as a function in comparison 630 00:34:18,810 --> 00:34:20,940 to specifications to actually say 631 00:34:20,940 --> 00:34:26,850 how well-controlled that is, but there is relatively tight bands 632 00:34:26,850 --> 00:34:36,239 with a couple of these shifts, and then almost outlier points. 633 00:34:36,239 --> 00:34:38,100 Now, would you think that these points 634 00:34:38,100 --> 00:34:42,659 are coming from the same source of deviation that 635 00:34:42,659 --> 00:34:47,270 might be at work in this band? 636 00:34:47,270 --> 00:34:52,659 Might be-- but it's such a large deviation that it 637 00:34:52,659 --> 00:34:54,730 starts to feel unlikely. 638 00:34:54,730 --> 00:34:56,620 And another use of statistical methods 639 00:34:56,620 --> 00:34:59,200 will actually be to quantify, how likely is it 640 00:34:59,200 --> 00:35:02,080 that we would observe by chance alone 641 00:35:02,080 --> 00:35:07,750 a deviation within the natural variation band of such a point? 642 00:35:10,700 --> 00:35:13,820 And the key reason is, if you see something big like that, 643 00:35:13,820 --> 00:35:15,740 there might be a [? point cause. ?] 644 00:35:15,740 --> 00:35:19,220 And it's quite possible in here that the operator is 645 00:35:19,220 --> 00:35:21,980 making a measurement on each part and said, whoops-- 646 00:35:21,980 --> 00:35:23,660 something just happened here. 647 00:35:28,050 --> 00:35:29,760 Something got misadjusted on the tool. 648 00:35:29,760 --> 00:35:32,640 I need to readjust it and get it back in. 649 00:35:32,640 --> 00:35:36,180 Maybe there's even a little bit of something going on there. 650 00:35:36,180 --> 00:35:39,180 Or it might have been a single event that 651 00:35:39,180 --> 00:35:41,250 wasn't-- didn't require adjustment. 652 00:35:41,250 --> 00:35:42,840 Maybe you go in, you investigate, 653 00:35:42,840 --> 00:35:46,740 and you go, that one piece of source material coming in, 654 00:35:46,740 --> 00:35:50,940 that piece of metal coming in was much thicker 655 00:35:50,940 --> 00:35:53,160 than it was intended to be. 656 00:35:53,160 --> 00:35:55,882 AUDIENCE: [INAUDIBLE] 657 00:35:57,300 --> 00:35:59,760 DUANE BONING: Yeah, maybe that's a piece of aluminum right 658 00:35:59,760 --> 00:36:02,430 in there, or this was-- exactly-- 659 00:36:02,430 --> 00:36:06,030 this was a steel [? 0.3 ?] in the aluminum set, 660 00:36:06,030 --> 00:36:09,390 or steel [? 0.3 ?] in the first one of those. 661 00:36:09,390 --> 00:36:12,900 Absolutely-- that's a cool observation. 662 00:36:12,900 --> 00:36:15,570 Yeah, that point seems to fit down in that distribution, 663 00:36:15,570 --> 00:36:16,650 doesn't it? 664 00:36:16,650 --> 00:36:18,810 Very interesting-- so a couple of the points 665 00:36:18,810 --> 00:36:21,450 here-- the kinds of things we might be looking for here 666 00:36:21,450 --> 00:36:23,970 are deterministic and systematic kinds 667 00:36:23,970 --> 00:36:27,030 of effects that we might want to use modeling for. 668 00:36:27,030 --> 00:36:29,010 And then we also are starting to get 669 00:36:29,010 --> 00:36:33,600 close to the statistical process control thinking, which 670 00:36:33,600 --> 00:36:37,020 is to say we would like to be able to know what 671 00:36:37,020 --> 00:36:41,130 the inherent or natural variation of the process 672 00:36:41,130 --> 00:36:44,010 is having to do with these sorts of bands, 673 00:36:44,010 --> 00:36:47,630 and then be able to detect with high degree of confidence 674 00:36:47,630 --> 00:36:49,760 that something strange has happened, 675 00:36:49,760 --> 00:36:51,890 and I better take action. 676 00:36:51,890 --> 00:36:55,070 And that SPC, or statistical process control. 677 00:36:58,165 --> 00:37:00,790 Maybe we're going to see many of the same characteristics here. 678 00:37:00,790 --> 00:37:03,070 This is just a little observation 679 00:37:03,070 --> 00:37:05,720 from injection molding. 680 00:37:05,720 --> 00:37:09,370 So this is a plastic molded part. 681 00:37:11,900 --> 00:37:15,200 We've got a wit, the part along this dimension, 682 00:37:15,200 --> 00:37:18,590 and down here we've got a number of the run. 683 00:37:18,590 --> 00:37:20,000 And also notice-- 684 00:37:20,000 --> 00:37:21,860 I'll just draw your attention here-- 685 00:37:21,860 --> 00:37:25,580 there are clear intentional bands here, 686 00:37:25,580 --> 00:37:28,980 where this first segment here says holding time 687 00:37:28,980 --> 00:37:32,630 was five seconds, injection press 40%. 688 00:37:32,630 --> 00:37:34,760 And we're changing here-- this is 689 00:37:34,760 --> 00:37:36,907 a holding time of 10 seconds. 690 00:37:36,907 --> 00:37:38,990 Here's another holding [INAUDIBLE] of five seconds 691 00:37:38,990 --> 00:37:40,640 and 60%-- 692 00:37:40,640 --> 00:37:42,200 and whatever is lurking under there. 693 00:37:45,092 --> 00:37:49,870 I think it's a holding time of 10 seconds. 694 00:37:49,870 --> 00:37:53,080 We could find out, but-- 695 00:37:53,080 --> 00:37:55,470 talk to me about this one. 696 00:37:55,470 --> 00:37:57,570 Oh, I guess also here, notice, we're 697 00:37:57,570 --> 00:38:00,360 plotting two pieces of data. 698 00:38:00,360 --> 00:38:04,150 We've got a width and then some average. 699 00:38:04,150 --> 00:38:06,420 So it's quite possible that what we're doing here 700 00:38:06,420 --> 00:38:09,690 is this is a point width at a well-known location, 701 00:38:09,690 --> 00:38:12,270 and than the average is we take that measurement 702 00:38:12,270 --> 00:38:17,040 on many, many locations around the part. 703 00:38:17,040 --> 00:38:21,330 Or it's possible we also actually run multiple parts, 704 00:38:21,330 --> 00:38:24,810 little mini batches of-- who knows-- maybe five parts, 705 00:38:24,810 --> 00:38:30,000 and I'm plotting the average across that little batch-- 706 00:38:30,000 --> 00:38:33,390 that five-part batch, which is another very common thing 707 00:38:33,390 --> 00:38:38,783 to do in SPC. So tell me a little bit about this. 708 00:38:38,783 --> 00:38:39,450 What do you see? 709 00:38:39,450 --> 00:38:40,170 Yeah? 710 00:38:40,170 --> 00:38:43,781 AUDIENCE: [INAUDIBLE] 711 00:38:50,268 --> 00:38:51,060 DUANE BONING: Yeah. 712 00:38:51,060 --> 00:38:55,500 So you're talking about this band here to here. 713 00:38:55,500 --> 00:38:57,210 Right. 714 00:38:57,210 --> 00:38:59,400 In fact, this might be the result 715 00:38:59,400 --> 00:39:02,730 of a designed experiment, where somebody 716 00:39:02,730 --> 00:39:07,230 is trying to explore the effective different process 717 00:39:07,230 --> 00:39:11,380 conditions, these four different cases. 718 00:39:11,380 --> 00:39:13,930 And part of the goal of that may be 719 00:39:13,930 --> 00:39:18,130 to identify not only things that are closer to a mean target, 720 00:39:18,130 --> 00:39:19,840 but are also more robust-- 721 00:39:19,840 --> 00:39:25,930 inherently robust, meaning they have a smaller delta a, 722 00:39:25,930 --> 00:39:30,310 or are the process itself has a smaller 723 00:39:30,310 --> 00:39:36,400 sensitivity among at that operating point to whatever 724 00:39:36,400 --> 00:39:39,370 inherent disturbance, whether it be temperature deviations 725 00:39:39,370 --> 00:39:40,880 or what have you. 726 00:39:40,880 --> 00:39:44,490 So that's a good observation. 727 00:39:44,490 --> 00:39:50,890 It's also kind of amazing to me here that the average looks 728 00:39:50,890 --> 00:39:54,700 rock steady in these cases, but these individual width 729 00:39:54,700 --> 00:39:58,576 measurements have a substantial deviation. 730 00:39:58,576 --> 00:40:01,212 AUDIENCE: Is that the average of all [INAUDIBLE]?? 731 00:40:01,212 --> 00:40:02,670 DUANE BONING: We don't really know. 732 00:40:02,670 --> 00:40:03,290 I don't know. 733 00:40:09,900 --> 00:40:12,730 Yes, it's quite possible that this is just a plot. 734 00:40:12,730 --> 00:40:14,340 So that's your point. 735 00:40:14,340 --> 00:40:19,710 This may be [INAUDIBLE] suspiciously so, right? 736 00:40:19,710 --> 00:40:22,560 This may be the average across that ensemble, 737 00:40:22,560 --> 00:40:24,570 and all that is doing it's not saying 738 00:40:24,570 --> 00:40:26,490 it's an average of multiple measurements 739 00:40:26,490 --> 00:40:29,910 or multiple samples, but that's just highlighting 740 00:40:29,910 --> 00:40:31,850 that there's this delta. 741 00:40:31,850 --> 00:40:33,570 And that's got to be it, because-- 742 00:40:33,570 --> 00:40:35,070 exactly. 743 00:40:35,070 --> 00:40:37,910 There's zero deviation in those average points. 744 00:40:41,930 --> 00:40:44,690 OK, but these are some of the same kinds of themes. 745 00:40:44,690 --> 00:40:50,150 We're, again, looking at offsets from deterministic process 746 00:40:50,150 --> 00:40:50,750 conditions. 747 00:40:50,750 --> 00:40:54,260 We're looking at inherent variation, ranges. 748 00:40:54,260 --> 00:40:56,450 We may be doing design experiments. 749 00:40:56,450 --> 00:40:59,270 You could start to think of empirical models that 750 00:40:59,270 --> 00:41:05,870 would help to guide us to an optimal point, the point being 751 00:41:05,870 --> 00:41:08,690 there that maybe these four conditions give you 752 00:41:08,690 --> 00:41:12,260 a little bit of an exploration across two different process 753 00:41:12,260 --> 00:41:13,700 parameters. 754 00:41:13,700 --> 00:41:20,840 One approach would be maybe your target is 40.85 or 40.87. 755 00:41:20,840 --> 00:41:24,170 You might pick the one that was closest to it. 756 00:41:24,170 --> 00:41:26,530 But if you actually built a model, including 757 00:41:26,530 --> 00:41:28,690 a model with continuous parameters, 758 00:41:28,690 --> 00:41:34,252 you could then interpolate and drive even closer to a target. 759 00:41:34,252 --> 00:41:35,710 Yeah-- another question or comment? 760 00:41:35,710 --> 00:41:39,908 AUDIENCE: [INAUDIBLE] 761 00:41:39,908 --> 00:41:41,450 DUANE BONING: Right, one would guess. 762 00:41:41,450 --> 00:41:44,697 AUDIENCE: [INAUDIBLE] 763 00:41:44,697 --> 00:41:45,530 DUANE BONING: Right. 764 00:41:45,530 --> 00:41:49,290 AUDIENCE: [INAUDIBLE] 765 00:42:01,680 --> 00:42:02,430 DUANE BONING: Yes. 766 00:42:02,430 --> 00:42:04,372 So that's actually a very interesting point. 767 00:42:04,372 --> 00:42:06,330 When we start talking a little about more about 768 00:42:06,330 --> 00:42:11,490 design of experiments, about the synergistic effects, 769 00:42:11,490 --> 00:42:13,890 it may not be a simple additive model. 770 00:42:13,890 --> 00:42:18,910 You're saying, as we go here, from 5 to 10, 771 00:42:18,910 --> 00:42:20,350 we got an increase. 772 00:42:20,350 --> 00:42:23,330 Here, when we went to 5 to 10, we got a decrease. 773 00:42:23,330 --> 00:42:26,560 So the effect of holding time may be completely 774 00:42:26,560 --> 00:42:30,158 in the opposite direction at two different pressures-- 775 00:42:32,800 --> 00:42:34,150 very good point. 776 00:42:34,150 --> 00:42:36,420 So that means there is some strong interaction 777 00:42:36,420 --> 00:42:39,360 between those two-- or mutual dependence between those two 778 00:42:39,360 --> 00:42:40,442 parameters. 779 00:42:44,300 --> 00:42:47,090 Man, we're getting very abstract in this data. 780 00:42:47,090 --> 00:42:50,210 We've got measurements on the output and we've got-- 781 00:42:50,210 --> 00:42:53,712 some number of run number on the side. 782 00:42:53,712 --> 00:42:56,170 This is actually looking very similar to some other things. 783 00:42:56,170 --> 00:42:58,230 We see definite shift effects. 784 00:42:58,230 --> 00:43:00,180 We see mean effects. 785 00:43:00,180 --> 00:43:05,090 Again, we see perhaps an outlier. 786 00:43:05,090 --> 00:43:08,840 The point here was that-- and this didn't come up earlier-- 787 00:43:08,840 --> 00:43:11,390 we keep looking and trying to put blame back 788 00:43:11,390 --> 00:43:14,990 on the manufacturing equipment or the source material. 789 00:43:18,000 --> 00:43:22,250 You also have to look carefully at your measurement apparatus. 790 00:43:22,250 --> 00:43:24,680 There's inherent measurement error. 791 00:43:24,680 --> 00:43:29,180 Even if the parts were almost perfect, 792 00:43:29,180 --> 00:43:32,860 very often, you're going to have an inherent measurement 793 00:43:32,860 --> 00:43:36,310 limitation and deviations due to measurement error. 794 00:43:36,310 --> 00:43:39,970 And you may have loading problems 795 00:43:39,970 --> 00:43:42,610 into the measurement apparatus. 796 00:43:42,610 --> 00:43:49,503 So it's not always an indication that there's part deviations. 797 00:43:49,503 --> 00:43:51,170 In some sense, the measurement equipment 798 00:43:51,170 --> 00:43:54,890 is manufacturing process, tool, or equipment 799 00:43:54,890 --> 00:43:58,880 itself that is also subject to deviations. 800 00:43:58,880 --> 00:44:01,550 And in fact, part of the applications of some 801 00:44:01,550 --> 00:44:04,550 of the statistical methods we'll talk about seek 802 00:44:04,550 --> 00:44:08,540 to characterize or do gauge studies of the accuracy 803 00:44:08,540 --> 00:44:10,880 and capability of your measurement equipment 804 00:44:10,880 --> 00:44:14,130 and the components of variation that affect that. 805 00:44:14,130 --> 00:44:20,750 So for example, there's a pure reproducibility 806 00:44:20,750 --> 00:44:22,820 and there's a repeatability component 807 00:44:22,820 --> 00:44:24,770 to typical gauge studies. 808 00:44:24,770 --> 00:44:26,690 One component, just real quickly, 809 00:44:26,690 --> 00:44:30,410 would be I've loaded the part on the measurement apparatus, 810 00:44:30,410 --> 00:44:34,070 and then I just fire whatever the measurement sequence is 811 00:44:34,070 --> 00:44:38,960 multiple times, and think of that as a pure repeatability-- 812 00:44:38,960 --> 00:44:44,190 somehow the physical-- maybe there's-- 813 00:44:44,190 --> 00:44:45,560 it's an optical measurement. 814 00:44:45,560 --> 00:44:50,720 There's scatter from extraneous light sources nearby off 815 00:44:50,720 --> 00:44:51,665 of that. 816 00:44:51,665 --> 00:44:53,540 And then there might be a separate component, 817 00:44:53,540 --> 00:44:56,990 if I pulled that same part out, took it out, and put it back 818 00:44:56,990 --> 00:45:01,830 on, and did that sequence of operations, 819 00:45:01,830 --> 00:45:04,550 including the operator operations. 820 00:45:04,550 --> 00:45:08,210 Then you have a reproducibility component that might be, 821 00:45:08,210 --> 00:45:10,230 in fact, much larger. 822 00:45:10,230 --> 00:45:14,780 So you'd need to understand, and maybe decompose and break down 823 00:45:14,780 --> 00:45:16,700 to those different components. 824 00:45:16,700 --> 00:45:19,220 And then, if you started to see that one 825 00:45:19,220 --> 00:45:21,320 was a source of deviation, then you 826 00:45:21,320 --> 00:45:25,180 might look at strategies to control or compensate for that. 827 00:45:25,180 --> 00:45:25,680 Yeah? 828 00:45:25,680 --> 00:45:27,233 AUDIENCE: [INAUDIBLE] 829 00:45:27,233 --> 00:45:28,025 DUANE BONING: Yeah. 830 00:45:28,025 --> 00:45:30,344 AUDIENCE: At one of my internships, 831 00:45:30,344 --> 00:45:34,440 the company was getting a lot of [INAUDIBLE].. 832 00:45:34,440 --> 00:45:36,840 They were really trying to figure out 833 00:45:36,840 --> 00:45:38,450 [INAUDIBLE],, and when it went bad, 834 00:45:38,450 --> 00:45:40,355 at the supplier [INAUDIBLE]. 835 00:45:47,960 --> 00:45:50,960 DUANE BONING: Absolutely, absolutely-- 836 00:45:50,960 --> 00:45:56,940 in some sense, it's often the analogy of it's not working-- 837 00:45:56,940 --> 00:45:59,390 oh, check the plug on the wall. 838 00:45:59,390 --> 00:46:04,610 Always check your measurement equipment and apparatus 839 00:46:04,610 --> 00:46:08,780 before you start diving and tearing your hair out, going 840 00:46:08,780 --> 00:46:11,690 too deep into debugging of the manufacturing process. 841 00:46:11,690 --> 00:46:14,450 Check each step along the way-- 842 00:46:14,450 --> 00:46:17,460 glad they discovered that pin problem back in the-- 843 00:46:17,460 --> 00:46:20,440 at the case. 844 00:46:20,440 --> 00:46:21,970 Oh, here's more. 845 00:46:21,970 --> 00:46:22,990 What do we see here? 846 00:46:27,350 --> 00:46:31,620 Again, some output as a function of run number [INAUDIBLE] 847 00:46:31,620 --> 00:46:35,870 again, maybe this is another sheet 848 00:46:35,870 --> 00:46:40,910 bending kind of component. 849 00:46:40,910 --> 00:46:46,770 We see effects of shift change multiple times here. 850 00:46:46,770 --> 00:46:49,430 These are different shifts, maybe different shifts 851 00:46:49,430 --> 00:46:51,360 or different parts. 852 00:46:51,360 --> 00:46:54,150 What's very interesting here is that, under some conditions, 853 00:46:54,150 --> 00:46:58,770 we have a nice stable, not drifting in time-- 854 00:46:58,770 --> 00:47:01,850 fairly nice, stable, or stationary mean, 855 00:47:01,850 --> 00:47:03,512 with some deviation-- 856 00:47:03,512 --> 00:47:04,970 every now and then, maybe something 857 00:47:04,970 --> 00:47:07,550 that looks outside of the distribution. 858 00:47:07,550 --> 00:47:11,170 But in some cases, holy cow-- 859 00:47:11,170 --> 00:47:15,430 that's a very different effect than what's going on in here. 860 00:47:15,430 --> 00:47:22,420 And it's back to this is a very clear drift that is definitely 861 00:47:22,420 --> 00:47:25,210 not just a one-time shift. 862 00:47:25,210 --> 00:47:26,120 Yeah? 863 00:47:26,120 --> 00:47:31,776 AUDIENCE: When I looked at the [INAUDIBLE],, and the first 864 00:47:31,776 --> 00:47:32,720 [INAUDIBLE]. 865 00:47:37,190 --> 00:47:40,192 I'm wondering if maybe the [INAUDIBLE].. 866 00:47:42,940 --> 00:47:48,330 DUANE BONING: Perhaps-- or maybe this worker 867 00:47:48,330 --> 00:47:54,060 finds a way that corrects for it in a comparatively effective 868 00:47:54,060 --> 00:47:59,040 fashion, whereas the strategy used in this case over here 869 00:47:59,040 --> 00:48:05,190 is not nearly as effective or quick at trying to get back, 870 00:48:05,190 --> 00:48:07,690 if that was the target. 871 00:48:07,690 --> 00:48:12,390 And so in fact, one of the challenges often in SPC 872 00:48:12,390 --> 00:48:16,350 or other control algorithms-- or not even algorithms, 873 00:48:16,350 --> 00:48:19,140 but control practices, especially involving people-- 874 00:48:19,140 --> 00:48:26,340 is one shift or one set of operators or engineers 875 00:48:26,340 --> 00:48:30,070 in charge of a process may learn things 876 00:48:30,070 --> 00:48:32,500 that don't necessarily get captured 877 00:48:32,500 --> 00:48:37,060 and conveyed to other shifts. 878 00:48:37,060 --> 00:48:43,880 OK, so what is going on in lots of these examples is perhaps 879 00:48:43,880 --> 00:48:45,500 the most important point here-- 880 00:48:45,500 --> 00:48:49,460 is that, very often, there are two highly different 881 00:48:49,460 --> 00:48:51,650 characteristics that keep coming up. 882 00:48:51,650 --> 00:48:54,440 There's systematic or deterministic effects 883 00:48:54,440 --> 00:48:57,440 having to do with the type of the material, settings 884 00:48:57,440 --> 00:49:00,750 on the equipment, choice of process conditions. 885 00:49:00,750 --> 00:49:04,220 And then there's also random components. 886 00:49:04,220 --> 00:49:06,030 There's the inherent spread. 887 00:49:06,030 --> 00:49:10,700 And a key point here is inherent spread you can almost never 888 00:49:10,700 --> 00:49:12,470 completely get rid of. 889 00:49:12,470 --> 00:49:15,500 I guess, philosophically, you could ultimately maybe appeal 890 00:49:15,500 --> 00:49:19,460 to quantum mechanics, that there would always be some deviation. 891 00:49:19,460 --> 00:49:20,840 I think very rarely-- 892 00:49:20,840 --> 00:49:22,730 perhaps in some semiconductor processes, 893 00:49:22,730 --> 00:49:31,690 we do approach truly inherent physical randomness. 894 00:49:31,690 --> 00:49:35,630 But generally, it's that the understanding is limited 895 00:49:35,630 --> 00:49:39,550 in what is contributing to that small degree of spread, 896 00:49:39,550 --> 00:49:44,410 or it's impossible to completely control those-- 897 00:49:44,410 --> 00:49:46,450 that inherent randomness. 898 00:49:46,450 --> 00:49:49,780 But then we also have big disturbances. 899 00:49:49,780 --> 00:49:54,550 They may still be random events, but they are not 900 00:49:54,550 --> 00:50:02,200 this little band, but they are big events with a clear cause. 901 00:50:02,200 --> 00:50:04,780 We may not always be able to discern the cause, 902 00:50:04,780 --> 00:50:08,260 or may not always be able to discover it, but in many cases, 903 00:50:08,260 --> 00:50:10,300 we've been hypothesizing about many 904 00:50:10,300 --> 00:50:14,560 of these big shift changes or a big drift. 905 00:50:14,560 --> 00:50:16,930 And a bit of investigation and more knowledge 906 00:50:16,930 --> 00:50:21,010 about the process would readily show us 907 00:50:21,010 --> 00:50:22,930 what's going on in those cases. 908 00:50:22,930 --> 00:50:25,210 And in those, when there's something 909 00:50:25,210 --> 00:50:28,580 systematic or deterministic going on, 910 00:50:28,580 --> 00:50:32,540 the whole idea is you make some changes to the equipment 911 00:50:32,540 --> 00:50:34,770 to stamp those out. 912 00:50:34,770 --> 00:50:37,680 And what we're often going to be dealing with 913 00:50:37,680 --> 00:50:40,320 is designing experiments and whatnot 914 00:50:40,320 --> 00:50:43,950 to try to understand systematic components and eliminate them, 915 00:50:43,950 --> 00:50:46,500 maybe as part of the initial design 916 00:50:46,500 --> 00:50:48,360 implementation of a process. 917 00:50:48,360 --> 00:50:53,010 Or in an ongoing process, when we see a deviation, 918 00:50:53,010 --> 00:50:55,860 we've got to go and debug it, find out what caused that, 919 00:50:55,860 --> 00:50:57,730 and eliminate it. 920 00:50:57,730 --> 00:51:00,460 And the natural occurrence then is 921 00:51:00,460 --> 00:51:04,120 that, over time, as you have learning about your process 922 00:51:04,120 --> 00:51:08,560 and manufacturing system, you're generally squeezing out 923 00:51:08,560 --> 00:51:10,630 these deterministic sources. 924 00:51:10,630 --> 00:51:13,960 It's something-- was repeatable, systematic, 925 00:51:13,960 --> 00:51:17,050 and therefore, you can come up with strategies 926 00:51:17,050 --> 00:51:19,750 either to eliminate it or to compensate for it. 927 00:51:19,750 --> 00:51:23,140 Maybe you have to have a control strategy that 928 00:51:23,140 --> 00:51:25,090 recognizes that source of variation 929 00:51:25,090 --> 00:51:26,930 and keep squeezing it out. 930 00:51:26,930 --> 00:51:30,730 And what you generally are left with 931 00:51:30,730 --> 00:51:33,100 are these random components, hopefully 932 00:51:33,100 --> 00:51:35,410 in a very nice, narrow band. 933 00:51:35,410 --> 00:51:39,140 But we still need to understand and model those, 934 00:51:39,140 --> 00:51:41,230 and so that's actually where-- 935 00:51:41,230 --> 00:51:44,860 another place where the statistical analysis tools are 936 00:51:44,860 --> 00:51:48,640 really crucial, because then we want to be able to understand, 937 00:51:48,640 --> 00:51:52,570 what is the natural or reducible-- 938 00:51:52,570 --> 00:51:56,920 irreducible component that we don't know how to stamp out one 939 00:51:56,920 --> 00:51:58,030 at a time-- 940 00:51:58,030 --> 00:52:01,730 and use that information to be able to detect 941 00:52:01,730 --> 00:52:07,500 in that small scatter when something unusual happens. 942 00:52:07,500 --> 00:52:11,020 And that's the idea of statistical process control. 943 00:52:11,020 --> 00:52:15,640 So what we want to do is characterize the process-- 944 00:52:15,640 --> 00:52:19,140 particularly just the random component, 945 00:52:19,140 --> 00:52:21,060 assuming that we've been able to squeeze out 946 00:52:21,060 --> 00:52:27,220 many of those other systematic components. 947 00:52:27,220 --> 00:52:31,550 Why do we want to characterize that process? 948 00:52:31,550 --> 00:52:33,950 There's some of the things that I've mentioned, 949 00:52:33,950 --> 00:52:36,570 and that you've mentioned. 950 00:52:36,570 --> 00:52:38,760 If I see one of these points that 951 00:52:38,760 --> 00:52:41,970 looks a little bit different than the distribution, 952 00:52:41,970 --> 00:52:46,410 we said that [? by i, ?] but very often, the question is, 953 00:52:46,410 --> 00:52:48,840 maybe there's a point that's a little bit above some 954 00:52:48,840 --> 00:52:49,950 of the other points. 955 00:52:49,950 --> 00:52:52,770 Then you would often want to ask the question, 956 00:52:52,770 --> 00:52:58,520 do I really believe the output changed, or is it 957 00:52:58,520 --> 00:53:02,170 part of that natural variation? 958 00:53:02,170 --> 00:53:04,620 And if we start to think about some natural variation 959 00:53:04,620 --> 00:53:06,760 and models for that, including distributions-- 960 00:53:06,760 --> 00:53:08,617 which we'll get to in just a moment-- 961 00:53:08,617 --> 00:53:10,200 then we can start to ask that question 962 00:53:10,200 --> 00:53:16,000 in a very systematic way, in terms of actual probabilities. 963 00:53:16,000 --> 00:53:18,490 Other reasons is we may go back to, 964 00:53:18,490 --> 00:53:20,480 did the input actually cause the change? 965 00:53:20,480 --> 00:53:25,680 Did I make some output change? 966 00:53:25,680 --> 00:53:29,680 A very important point here is, how confident 967 00:53:29,680 --> 00:53:31,600 are we have these answers? 968 00:53:31,600 --> 00:53:33,760 Very often here, we were appealing to it 969 00:53:33,760 --> 00:53:37,330 looks like that's a bigger change, or this is-- 970 00:53:37,330 --> 00:53:42,490 it was a mean offset. 971 00:53:42,490 --> 00:53:48,100 Well, that's OK to try to convince your management 972 00:53:48,100 --> 00:53:52,750 or convince your operators that, look, something changed, 973 00:53:52,750 --> 00:53:54,460 but in many cases, you'd actually 974 00:53:54,460 --> 00:54:00,840 like to be able to quantify how confident you are that that was 975 00:54:00,840 --> 00:54:06,480 either a deviation point or that some specific decision that you 976 00:54:06,480 --> 00:54:08,920 made had a true effect. 977 00:54:08,920 --> 00:54:12,630 And so the probability and statistical tools 978 00:54:12,630 --> 00:54:14,960 are really, really important for that. 979 00:54:14,960 --> 00:54:16,710 So what are some ways that we can actually 980 00:54:16,710 --> 00:54:20,190 characterize and model the random components? 981 00:54:20,190 --> 00:54:22,440 Well, that's where we're getting into random processes 982 00:54:22,440 --> 00:54:24,270 and random variables. 983 00:54:24,270 --> 00:54:27,060 And I hope you're seeing this is the basis 984 00:54:27,060 --> 00:54:31,530 for statistical process control I've referred to. 985 00:54:31,530 --> 00:54:37,510 And we'll dive into some of the tools and techniques, as well. 986 00:54:37,510 --> 00:54:39,530 Design of experiments-- so if we were 987 00:54:39,530 --> 00:54:42,050 making that pressure change or that time 988 00:54:42,050 --> 00:54:45,080 change, how big of an effect is it? 989 00:54:45,080 --> 00:54:49,190 How certain are we within the random natural variation 990 00:54:49,190 --> 00:54:52,100 that the deviation-- or the controlled decision I 991 00:54:52,100 --> 00:54:54,870 made actually had a big effect? 992 00:54:54,870 --> 00:54:59,420 And then finally, even with compensation strategies-- 993 00:54:59,420 --> 00:55:02,960 like this-- one of the very first things that we said-- 994 00:55:02,960 --> 00:55:06,920 in that oscillating case, there might be inherent noise. 995 00:55:06,920 --> 00:55:08,570 And it's quite possible that somebody 996 00:55:08,570 --> 00:55:11,840 could be making process control feedback 997 00:55:11,840 --> 00:55:15,180 changes to try to compensate. 998 00:55:15,180 --> 00:55:17,880 But if they're trying to compensate for something that 999 00:55:17,880 --> 00:55:21,870 doesn't have, in fact, a connection between the control 1000 00:55:21,870 --> 00:55:25,990 decision and the true source of noise, 1001 00:55:25,990 --> 00:55:29,730 they may, in fact, be injecting more noise in the process. 1002 00:55:29,730 --> 00:55:33,470 So understanding actually how feedback control works 1003 00:55:33,470 --> 00:55:37,610 when we've got random, and not just the systematic component, 1004 00:55:37,610 --> 00:55:39,632 is very important. 1005 00:55:42,290 --> 00:55:45,120 What are some ways of describing randomness? 1006 00:55:48,060 --> 00:55:50,280 Actually, philosophically, there's 1007 00:55:50,280 --> 00:55:53,460 a couple of very different ways of thinking-- 1008 00:55:53,460 --> 00:55:56,520 if you open up a statistic book, in the first few pages, 1009 00:55:56,520 --> 00:55:59,850 we'll talk about this. 1010 00:55:59,850 --> 00:56:03,280 And from a manufacturing perspective, 1011 00:56:03,280 --> 00:56:05,490 well, we're really dealing with data. 1012 00:56:05,490 --> 00:56:09,440 In many ways, the data tells us a lot. 1013 00:56:09,440 --> 00:56:14,210 And one approach here is to look at actual large ensembles 1014 00:56:14,210 --> 00:56:16,730 or collections of data, and look at histograms, 1015 00:56:16,730 --> 00:56:18,080 frequency distributions. 1016 00:56:18,080 --> 00:56:20,270 And we'll look at some examples of that. 1017 00:56:20,270 --> 00:56:24,080 A very different approach is to start with a pure probability 1018 00:56:24,080 --> 00:56:29,060 model that assumes there is a universe of 1019 00:56:29,060 --> 00:56:31,900 and an infinite number of samples, 1020 00:56:31,900 --> 00:56:34,360 and there are characteristics of that infinite number 1021 00:56:34,360 --> 00:56:36,670 of samples. 1022 00:56:36,670 --> 00:56:41,350 And that gives us the tools to put empirical observations, 1023 00:56:41,350 --> 00:56:45,880 like frequency histograms, on a mathematical basis that we 1024 00:56:45,880 --> 00:56:46,870 can reason with. 1025 00:56:50,300 --> 00:56:53,190 So for example, a way of-- 1026 00:56:53,190 --> 00:56:58,230 let me jump forward here-- this is some thermoforming data. 1027 00:56:58,230 --> 00:57:00,750 Again, this is some dimension, and then, 1028 00:57:00,750 --> 00:57:04,500 over this collection of parts, we have a-- 1029 00:57:04,500 --> 00:57:07,770 this bar chart, this frequency histogram. 1030 00:57:07,770 --> 00:57:11,160 And what that's really saying is, for each of the bends 1031 00:57:11,160 --> 00:57:14,610 in that histogram, I'm simply plotting the frequency-- 1032 00:57:14,610 --> 00:57:17,520 or relative frequency that some output was 1033 00:57:17,520 --> 00:57:19,380 between the bounds of that bin. 1034 00:57:22,570 --> 00:57:28,810 And so what we are doing is converting from historical data 1035 00:57:28,810 --> 00:57:30,820 into bins. 1036 00:57:30,820 --> 00:57:33,160 This is still discrete bins, but then we're 1037 00:57:33,160 --> 00:57:36,070 associating that and interpolating forward 1038 00:57:36,070 --> 00:57:39,880 to say, well, the probability in most applications-- 1039 00:57:39,880 --> 00:57:42,820 if I were to do some more part, we 1040 00:57:42,820 --> 00:57:45,670 think that this histogram, in some sense, 1041 00:57:45,670 --> 00:57:49,150 represents a larger set of data, and I 1042 00:57:49,150 --> 00:57:53,050 can use it to talk about the probabilities of occurrences 1043 00:57:53,050 --> 00:57:56,620 of new data in here. 1044 00:57:56,620 --> 00:57:58,240 So what do you see in here? 1045 00:57:58,240 --> 00:58:08,280 Whoops-- there's a little insert here with the raw data. 1046 00:58:08,280 --> 00:58:11,360 Now, if I hid that from you-- 1047 00:58:11,360 --> 00:58:12,400 it's hard for me-- 1048 00:58:12,400 --> 00:58:14,560 I'm trying to put my hand over this. 1049 00:58:14,560 --> 00:58:17,230 If I hid that from you here, and I hid that from you here, 1050 00:58:17,230 --> 00:58:21,130 and you just looked at this data, typical things you would 1051 00:58:21,130 --> 00:58:21,790 say about it-- 1052 00:58:24,330 --> 00:58:29,850 it looks normal, or Gaussian distributed. 1053 00:58:29,850 --> 00:58:33,102 Maybe you would think almost it's completely random. 1054 00:58:33,102 --> 00:58:34,560 It's got to mean and the deviation. 1055 00:58:34,560 --> 00:58:37,510 It's got a normal distribution. 1056 00:58:37,510 --> 00:58:40,232 Now look back at the data. 1057 00:58:40,232 --> 00:58:41,440 It's kind of drifting around. 1058 00:58:44,070 --> 00:58:48,660 Just a peek forward to other things to be cautious about-- 1059 00:58:48,660 --> 00:58:51,840 don't always believe or just look at your statistics. 1060 00:58:51,840 --> 00:58:54,990 Always go back to the raw data, because if this 1061 00:58:54,990 --> 00:58:56,730 were a characteristic of my process, 1062 00:58:56,730 --> 00:58:59,490 I would say there is a random component, 1063 00:58:59,490 --> 00:59:04,440 I think, here, but then I'm also seeing maybe 1064 00:59:04,440 --> 00:59:07,020 some systematic drift that-- 1065 00:59:07,020 --> 00:59:11,440 I'd like to go in and squeeze that out. 1066 00:59:11,440 --> 00:59:15,180 And then, if I were able to get that out of my process, 1067 00:59:15,180 --> 00:59:17,700 then I might have a random component 1068 00:59:17,700 --> 00:59:20,190 that doesn't have that deterministic or systematic 1069 00:59:20,190 --> 00:59:20,735 component. 1070 00:59:20,735 --> 00:59:22,110 And it might have a distribution, 1071 00:59:22,110 --> 00:59:24,750 but it might be much, much tighter. 1072 00:59:24,750 --> 00:59:29,030 So an important point here is purely 1073 00:59:29,030 --> 00:59:31,910 empirical data doesn't always-- 1074 00:59:31,910 --> 00:59:35,660 it may look random, but it may have embedded in it-- 1075 00:59:35,660 --> 00:59:39,560 if you were to deconvolve all of the sources of variation, 1076 00:59:39,560 --> 00:59:41,180 this big random-- 1077 00:59:41,180 --> 00:59:44,180 or this big normal distribution may have mixed into it 1078 00:59:44,180 --> 00:59:46,610 lots of different systematic components, 1079 00:59:46,610 --> 00:59:51,050 as well as underlying natural variation. 1080 00:59:51,050 --> 00:59:53,270 Imagine, for example, that I actually went in 1081 00:59:53,270 --> 00:59:55,835 and I got rid of this long-term drift. 1082 00:59:58,420 --> 01:00:00,700 Would you believe that the remaining variation is all 1083 01:00:00,700 --> 01:00:01,200 random? 1084 01:00:05,920 --> 01:00:09,370 I bet I could go and I could plot it, and I bet it'd look-- 1085 01:00:09,370 --> 01:00:12,570 the remaining deviations would look mostly normal as well. 1086 01:00:16,280 --> 01:00:17,300 You think so? 1087 01:00:17,300 --> 01:00:18,970 Well, who knows? 1088 01:00:18,970 --> 01:00:22,930 You might go in and do some of the other kinds of hypotheses 1089 01:00:22,930 --> 01:00:25,108 about systematic sources that we were talking 1090 01:00:25,108 --> 01:00:26,150 about in the other cases. 1091 01:00:26,150 --> 01:00:29,620 For example, it looks like there may be a bit of an up, 1092 01:00:29,620 --> 01:00:34,000 down-- there might be something cyclical or periodic going on. 1093 01:00:34,000 --> 01:00:36,070 That might be another systematic source. 1094 01:00:36,070 --> 01:00:40,090 It could masquerade as a random normal distribution. 1095 01:00:40,090 --> 01:00:44,900 So there's always that mix in. 1096 01:00:44,900 --> 01:00:50,990 However, in many cases, maybe you also want to live with-- 1097 01:00:50,990 --> 01:00:53,600 it's not worth it to you, or too expensive to go 1098 01:00:53,600 --> 01:00:56,270 in and eliminate, or you tried and we're not 1099 01:00:56,270 --> 01:01:00,320 able to eliminate also some of these wandering drifts. 1100 01:01:00,320 --> 01:01:03,200 And in that case, maybe the best you can do 1101 01:01:03,200 --> 01:01:07,130 is at least characterize that remaining variation, which 1102 01:01:07,130 --> 01:01:08,690 has a systematic component. 1103 01:01:08,690 --> 01:01:13,160 But you would still like to have a probability model for it 1104 01:01:13,160 --> 01:01:17,720 so that you could detect deviations that were outside 1105 01:01:17,720 --> 01:01:20,550 of that variation source. 1106 01:01:20,550 --> 01:01:23,480 So if I saw a big shift up, I would 1107 01:01:23,480 --> 01:01:25,070 say I've got a point way out here 1108 01:01:25,070 --> 01:01:27,500 in the tail of the distribution that's highly unlikely. 1109 01:01:27,500 --> 01:01:30,920 And you'd still want to use that. 1110 01:01:30,920 --> 01:01:40,270 So now, we might want more compact ways of describing 1111 01:01:40,270 --> 01:01:41,810 this distribution. 1112 01:01:41,810 --> 01:01:43,630 Actually, in the age of computers, 1113 01:01:43,630 --> 01:01:47,860 I think, with lots of big disks and lots of great data 1114 01:01:47,860 --> 01:01:53,290 collection, one of the most underutilized opportunities 1115 01:01:53,290 --> 01:01:55,240 is keeping around all the raw data, 1116 01:01:55,240 --> 01:01:59,280 and actually using empirical distribution. 1117 01:01:59,280 --> 01:02:03,230 So in some sense, it's kind of a holdover from the days 1118 01:02:03,230 --> 01:02:06,770 when it was really tough to keep track and use 1119 01:02:06,770 --> 01:02:08,240 vast amounts of data. 1120 01:02:08,240 --> 01:02:11,280 And we almost always would start to say, 1121 01:02:11,280 --> 01:02:13,880 OK, I'd like a compact way of describing this, 1122 01:02:13,880 --> 01:02:17,600 and I want to describe it as something that only has 1123 01:02:17,600 --> 01:02:19,080 maybe a couple of parameters. 1124 01:02:19,080 --> 01:02:27,110 It has a mean and some variance component. 1125 01:02:27,110 --> 01:02:30,540 But that's a different philosophical point. 1126 01:02:30,540 --> 01:02:32,330 There's, actually, I think, a huge amount 1127 01:02:32,330 --> 01:02:35,720 of statistical methods that are underutilized 1128 01:02:35,720 --> 01:02:40,670 that deal with sampling or resampling from raw data 1129 01:02:40,670 --> 01:02:45,290 that actually can capture subtle things going on in the data. 1130 01:02:45,290 --> 01:02:48,410 We're, in this course, not going to be using those things. 1131 01:02:48,410 --> 01:02:52,520 We're pretty much going to seek compact descriptions 1132 01:02:52,520 --> 01:02:56,060 of randomness in ways that we can reason about. 1133 01:02:56,060 --> 01:02:59,330 There are data-intensive ways of doing 1134 01:02:59,330 --> 01:03:01,230 the same kinds of reasoning. 1135 01:03:01,230 --> 01:03:04,400 So one important point here is we're often 1136 01:03:04,400 --> 01:03:09,980 going to take this discrete binned kind of information-- 1137 01:03:09,980 --> 01:03:15,110 we want to characterize it with perhaps a continuous parameter 1138 01:03:15,110 --> 01:03:19,710 variable and a compact distribution, 1139 01:03:19,710 --> 01:03:22,320 such as a normal or Gaussian distribution. 1140 01:03:22,320 --> 01:03:26,720 And this is especially good when the outputs are themselves 1141 01:03:26,720 --> 01:03:29,960 truly continuous parameters, like a width, 1142 01:03:29,960 --> 01:03:32,930 a dimension of the part. 1143 01:03:32,930 --> 01:03:35,870 And one point here, just to remind you-- especially 1144 01:03:35,870 --> 01:03:38,138 when we go to continuous parameters, 1145 01:03:38,138 --> 01:03:39,680 you've got to be a little bit careful 1146 01:03:39,680 --> 01:03:45,710 about talking about the probability of your part 1147 01:03:45,710 --> 01:03:49,430 or your variable taking on any specific value 1148 01:03:49,430 --> 01:03:52,220 in that continuous distribution. 1149 01:03:52,220 --> 01:03:53,820 Back here when we had things binned, 1150 01:03:53,820 --> 01:03:55,550 I could talk about the probability 1151 01:03:55,550 --> 01:04:01,250 of measuring something in that bin, 1152 01:04:01,250 --> 01:04:07,470 within that range, 1.485 to 1.487. 1153 01:04:07,470 --> 01:04:09,770 But now, if I were to ask, what is the likelihood 1154 01:04:09,770 --> 01:04:16,790 that I measure exactly 1.486, be careful. 1155 01:04:16,790 --> 01:04:22,520 That is a very, very discrete value that-- 1156 01:04:22,520 --> 01:04:24,890 when we talk about continuous probabilities, 1157 01:04:24,890 --> 01:04:27,800 the probability of getting any one value-- 1158 01:04:27,800 --> 01:04:31,430 and it's truly continuous-- 1159 01:04:31,430 --> 01:04:33,120 is 0. 1160 01:04:33,120 --> 01:04:34,890 You're never going to see exactly that. 1161 01:04:34,890 --> 01:04:40,360 Now, with real measurement tools, 1162 01:04:40,360 --> 01:04:43,990 we never have perfect continuous measurements either. 1163 01:04:43,990 --> 01:04:47,140 There is always a precision and a discreetness 1164 01:04:47,140 --> 01:04:51,040 capable from the tool. 1165 01:04:51,040 --> 01:04:53,410 So this is an abstraction dealing 1166 01:04:53,410 --> 01:04:57,220 with the mathematical niceties. 1167 01:04:57,220 --> 01:05:01,450 I'd just point out it's not exactly true 1168 01:05:01,450 --> 01:05:06,470 and measured reality and data. 1169 01:05:06,470 --> 01:05:09,965 So very often, when we talk about probabilities, 1170 01:05:09,965 --> 01:05:10,840 we got to be careful. 1171 01:05:10,840 --> 01:05:14,980 The probability of any one value is 1172 01:05:14,980 --> 01:05:17,380 0 what we talk about really is the probability 1173 01:05:17,380 --> 01:05:21,130 of our measurement or whatever being in some range that's 1174 01:05:21,130 --> 01:05:24,100 inherent in the distribution. 1175 01:05:24,100 --> 01:05:27,820 But we can also talk about, for example, the probability 1176 01:05:27,820 --> 01:05:31,000 that the value be in the range from minus 1177 01:05:31,000 --> 01:05:34,390 infinity up to some value y star, 1178 01:05:34,390 --> 01:05:37,420 and that is a reasonable question 1179 01:05:37,420 --> 01:05:39,880 to ask in terms of probabilities. 1180 01:05:39,880 --> 01:05:43,690 And that is embedded in this notion 1181 01:05:43,690 --> 01:05:48,400 of a cumulative probability function, or cumulative density 1182 01:05:48,400 --> 01:05:51,160 function, or CDF. 1183 01:05:51,160 --> 01:05:53,230 Just to illustrate that a little bit more, 1184 01:05:53,230 --> 01:05:56,380 we can do the same thing, have a cumulative frequency 1185 01:05:56,380 --> 01:05:58,640 with discrete data. 1186 01:05:58,640 --> 01:06:02,470 So if I had that underlying histogram here, 1187 01:06:02,470 --> 01:06:07,090 I could also build a cumulative frequency histogram 1188 01:06:07,090 --> 01:06:10,120 that simply was giving me the probability 1189 01:06:10,120 --> 01:06:15,280 that a part or measured value was up to, but less 1190 01:06:15,280 --> 01:06:17,890 than some particular value. 1191 01:06:17,890 --> 01:06:19,450 And that's simply the integration 1192 01:06:19,450 --> 01:06:24,940 across the PDF up to that cumulative density function. 1193 01:06:24,940 --> 01:06:26,680 And we can do the same thing that we 1194 01:06:26,680 --> 01:06:34,170 did with the discrete probability frequency 1195 01:06:34,170 --> 01:06:35,200 relationship. 1196 01:06:35,200 --> 01:06:39,870 So we can approximate the frequency with a PDF. 1197 01:06:39,870 --> 01:06:42,450 We can also look at the cumulative frequency 1198 01:06:42,450 --> 01:06:46,220 and approximate that with a cumulative density 1199 01:06:46,220 --> 01:06:49,520 function, or a CDF. 1200 01:06:49,520 --> 01:06:52,850 So we can have that kind of an equivalent. 1201 01:06:52,850 --> 01:06:56,600 And those equivalents also, again, help us interpolate 1202 01:06:56,600 --> 01:06:58,670 and ask probability questions also 1203 01:06:58,670 --> 01:07:04,140 on values that may not have been exactly lined up with our bins. 1204 01:07:04,140 --> 01:07:07,010 So very often, we're going to be, I guess, 1205 01:07:07,010 --> 01:07:10,020 talking either about the density function-- 1206 01:07:10,020 --> 01:07:12,090 the PDF of some variable-- 1207 01:07:12,090 --> 01:07:14,970 or the cumulative probability. 1208 01:07:14,970 --> 01:07:19,080 And there is this inherent relationship between the two. 1209 01:07:19,080 --> 01:07:22,710 I describe the cumulative probability as an integration 1210 01:07:22,710 --> 01:07:23,340 up. 1211 01:07:23,340 --> 01:07:26,970 The probability that some value is less than or equal to 1212 01:07:26,970 --> 01:07:33,090 would be an integral up to some x star. 1213 01:07:33,090 --> 01:07:38,080 And so that value right there corresponds to that. 1214 01:07:38,080 --> 01:07:39,570 And then the inverse is-- 1215 01:07:39,570 --> 01:07:41,290 relationship is true as well. 1216 01:07:41,290 --> 01:07:44,790 If I had a cumulative density function, 1217 01:07:44,790 --> 01:07:48,480 I can differentiate that to get down to the probability density 1218 01:07:48,480 --> 01:07:49,750 function. 1219 01:07:49,750 --> 01:07:54,000 So this is some basic statistical concepts 1220 01:07:54,000 --> 01:07:57,630 that you probably have seen before. 1221 01:07:57,630 --> 01:08:04,830 Many of you have seen it in much more detail in 2.853. 1222 01:08:04,830 --> 01:08:06,690 OK, so very often, the histogram does 1223 01:08:06,690 --> 01:08:12,450 suggest an underlying approximate or convenient 1224 01:08:12,450 --> 01:08:14,550 probability density function. 1225 01:08:14,550 --> 01:08:18,210 Very often, we've talked about normal distributions, 1226 01:08:18,210 --> 01:08:21,720 but a key thing to look at is always look at your raw data, 1227 01:08:21,720 --> 01:08:27,540 because there may be other perfectly reasonable continuous 1228 01:08:27,540 --> 01:08:31,800 or discrete probability density or probability mass 1229 01:08:31,800 --> 01:08:34,140 functions that apply in the discrete case, 1230 01:08:34,140 --> 01:08:38,135 such as a uniform distribution. 1231 01:08:38,135 --> 01:08:39,760 And so one of the things that we'll see 1232 01:08:39,760 --> 01:08:46,540 is the family of some typically emerging or typically observed 1233 01:08:46,540 --> 01:08:48,534 kinds of distribution models. 1234 01:08:51,350 --> 01:08:53,060 So very often, what we're doing is 1235 01:08:53,060 --> 01:08:55,970 we're gathering data, looking at these histograms, 1236 01:08:55,970 --> 01:09:00,580 and looking to see, is there a consistent pattern, such 1237 01:09:00,580 --> 01:09:07,850 as we saw with the underlying normal distribution? 1238 01:09:07,850 --> 01:09:13,090 And we're often then trying to intuit 1239 01:09:13,090 --> 01:09:17,020 what that reasonable underlying distribution may be. 1240 01:09:17,020 --> 01:09:20,170 There are ways-- and we may touch on them a little bit-- 1241 01:09:20,170 --> 01:09:23,800 to be able to test whether or how 1242 01:09:23,800 --> 01:09:30,370 good a particular distribution model may fit with that data. 1243 01:09:30,370 --> 01:09:33,010 And on the problem set, in fact, you'll 1244 01:09:33,010 --> 01:09:36,010 explore things like a qqnorm plot. 1245 01:09:36,010 --> 01:09:38,240 And I'm not going to go into detail here, 1246 01:09:38,240 --> 01:09:39,939 but essentially, that-- 1247 01:09:39,939 --> 01:09:43,090 assume a model-- for example, what the probability density 1248 01:09:43,090 --> 01:09:46,660 function probabilities would be associated with certain kinds 1249 01:09:46,660 --> 01:09:50,200 of observations, if it were a normal distribution-- 1250 01:09:50,200 --> 01:09:52,180 and then you plot your data against that 1251 01:09:52,180 --> 01:09:55,300 and start to see how well the data fits 1252 01:09:55,300 --> 01:09:56,740 that particular model. 1253 01:10:01,030 --> 01:10:03,360 Let's go back to that original set 1254 01:10:03,360 --> 01:10:05,130 of data, the [? c and c turning. ?] 1255 01:10:05,130 --> 01:10:08,220 If I were to plot that as a distribution, 1256 01:10:08,220 --> 01:10:09,840 I might get something like this. 1257 01:10:13,180 --> 01:10:16,630 I might argue that my bins may be too coarse here, 1258 01:10:16,630 --> 01:10:17,860 but what do you see? 1259 01:10:20,550 --> 01:10:22,410 This is a little tricky, right? 1260 01:10:22,410 --> 01:10:24,630 If I were to plot that data as a distribution, 1261 01:10:24,630 --> 01:10:29,010 I might be tempted to say, OK, that's one big Gaussian 1262 01:10:29,010 --> 01:10:31,760 distribution. 1263 01:10:31,760 --> 01:10:34,490 You could try that. 1264 01:10:34,490 --> 01:10:36,510 But again, if we look back at the data, 1265 01:10:36,510 --> 01:10:39,830 another interpretation here is, really-- 1266 01:10:39,830 --> 01:10:41,330 you can start to see it-- 1267 01:10:44,318 --> 01:10:45,690 it's bimodal. 1268 01:10:45,690 --> 01:10:49,380 It's a mix of two distributions. 1269 01:10:49,380 --> 01:10:54,090 So I guess the simple point here is don't always 1270 01:10:54,090 --> 01:10:58,350 assume it's going to be just a single distribution that's 1271 01:10:58,350 --> 01:10:58,982 underlying it. 1272 01:10:58,982 --> 01:10:59,940 There may be a mixture. 1273 01:11:03,770 --> 01:11:05,940 Similarly, here's another set. 1274 01:11:05,940 --> 01:11:10,450 And very often, again, plotting your empirical data 1275 01:11:10,450 --> 01:11:13,660 in time order tells you a lot more, but also 1276 01:11:13,660 --> 01:11:16,480 be sensitive to those kinds of questions. 1277 01:11:16,480 --> 01:11:18,940 I could have started the whole class off and never 1278 01:11:18,940 --> 01:11:22,390 shown you the raw data, and just started plotting distributions. 1279 01:11:22,390 --> 01:11:25,570 And in some cases, things might be more observable there. 1280 01:11:25,570 --> 01:11:31,060 Usually it's easier to see perhaps lurking up here, 1281 01:11:31,060 --> 01:11:33,760 when there's such a very clear time shift. 1282 01:11:33,760 --> 01:11:38,650 But here, this is the histogram associated with these, 1283 01:11:38,650 --> 01:11:43,780 and it sure looks like one normal distribution here 1284 01:11:43,780 --> 01:11:45,760 and another normal distribution here. 1285 01:11:49,320 --> 01:11:51,780 Now, by the way, if these things were ping 1286 01:11:51,780 --> 01:11:54,400 ponging back and forth, or they were randomly picking 1287 01:11:54,400 --> 01:11:58,560 and it was one of those source material deviations, 1288 01:11:58,560 --> 01:12:01,350 it might actually be hard to detect that, 1289 01:12:01,350 --> 01:12:03,390 whereas the bimodality-- the fact 1290 01:12:03,390 --> 01:12:06,300 that I've actually got what looks like two mixtures-- may 1291 01:12:06,300 --> 01:12:14,990 actually appear more easily in a histogram kind of form. 1292 01:12:14,990 --> 01:12:19,670 There's another example. 1293 01:12:19,670 --> 01:12:22,700 Here we start to see, again, these two shifts, 1294 01:12:22,700 --> 01:12:26,450 and now you can start to convert these perhaps 1295 01:12:26,450 --> 01:12:29,570 to two statistical distributions, two 1296 01:12:29,570 --> 01:12:31,850 Gaussians with different means. 1297 01:12:31,850 --> 01:12:35,210 And now you can also estimate parameters, 1298 01:12:35,210 --> 01:12:37,100 like what the mean shift is. 1299 01:12:37,100 --> 01:12:38,900 So once you've got distributions, 1300 01:12:38,900 --> 01:12:41,870 now you've got mathematical approaches 1301 01:12:41,870 --> 01:12:45,810 for being able to estimate those parameters. 1302 01:12:45,810 --> 01:12:50,230 So this is just going back to some of that other data. 1303 01:12:50,230 --> 01:12:55,690 OK, so some of the key points here out of this portion 1304 01:12:55,690 --> 01:13:00,250 is, even if there's no strong input effect, 1305 01:13:00,250 --> 01:13:03,970 there may still be disturbances. 1306 01:13:03,970 --> 01:13:08,080 And we'll often see a consistent histogram pattern emerge. 1307 01:13:08,080 --> 01:13:12,130 And what we often want to do is use 1308 01:13:12,130 --> 01:13:15,490 that either to detect deterministic deviations 1309 01:13:15,490 --> 01:13:20,930 and squeeze them out, or once we've done all those process 1310 01:13:20,930 --> 01:13:23,960 improvements and gotten those all squeezed out, 1311 01:13:23,960 --> 01:13:28,710 we may still have an underlying inherent distribution. 1312 01:13:28,710 --> 01:13:32,300 Maybe it's random variation associated 1313 01:13:32,300 --> 01:13:34,280 with a Gaussian distribution. 1314 01:13:34,280 --> 01:13:36,410 And the key idea is we can even use, 1315 01:13:36,410 --> 01:13:39,620 and importantly, we want to use knowledge of that underlying 1316 01:13:39,620 --> 01:13:41,170 pattern. 1317 01:13:41,170 --> 01:13:42,590 It's not a systematic pattern. 1318 01:13:42,590 --> 01:13:45,460 It's a random pattern associated with that distribution-- 1319 01:13:45,460 --> 01:13:48,430 so that we can predict behavior, and in particular, set 1320 01:13:48,430 --> 01:13:50,480 limits on normal behavior. 1321 01:13:53,050 --> 01:13:54,640 This is a nice case. 1322 01:13:54,640 --> 01:13:58,780 Maybe this is what we'll consider normal behavior. 1323 01:13:58,780 --> 01:14:01,180 And again, I want to be able to statistically 1324 01:14:01,180 --> 01:14:04,930 say, what's the likelihood that some 1325 01:14:04,930 --> 01:14:09,190 of these points or a new point belong to the natural variation 1326 01:14:09,190 --> 01:14:11,170 or not? 1327 01:14:11,170 --> 01:14:15,360 So we're going to typically use analytic probability density 1328 01:14:15,360 --> 01:14:17,690 functions to do that. 1329 01:14:17,690 --> 01:14:22,550 I think we've already talked about most of these things. 1330 01:14:22,550 --> 01:14:26,030 I've been emphasizing, by the way, continuous values, 1331 01:14:26,030 --> 01:14:31,220 these width or geometry parameters, 1332 01:14:31,220 --> 01:14:34,550 but there's also a set of different statistical 1333 01:14:34,550 --> 01:14:38,150 distribution than the normal distribution associated 1334 01:14:38,150 --> 01:14:41,060 especially with the occurrence-- random occurrence of point 1335 01:14:41,060 --> 01:14:41,990 defects. 1336 01:14:41,990 --> 01:14:43,940 And a little later in the term, we'll 1337 01:14:43,940 --> 01:14:46,580 actually do some yield modeling dealing 1338 01:14:46,580 --> 01:14:49,280 with things like Poisson distributions 1339 01:14:49,280 --> 01:14:51,830 and in these sorts of discrete distributions 1340 01:14:51,830 --> 01:14:56,720 that are more appropriate to discrete events 1341 01:14:56,720 --> 01:14:58,310 or discrete occurrences. 1342 01:14:58,310 --> 01:15:00,620 We're going to be, in the first part here, 1343 01:15:00,620 --> 01:15:04,710 dealing pretty much with continuous parameters. 1344 01:15:04,710 --> 01:15:06,920 So this is just laying out, again, a little bit. 1345 01:15:06,920 --> 01:15:08,420 We've talked about some of these. 1346 01:15:08,420 --> 01:15:10,880 I want to build up just very briefly, 1347 01:15:10,880 --> 01:15:13,850 get you going on some of the basic definitions. 1348 01:15:13,850 --> 01:15:16,670 And then Chapter 4 in Spanos really articulates 1349 01:15:16,670 --> 01:15:17,970 these much more. 1350 01:15:17,970 --> 01:15:20,000 So we have our probability density function 1351 01:15:20,000 --> 01:15:23,360 and cumulative density function, and there is that relationship 1352 01:15:23,360 --> 01:15:25,920 between the two. 1353 01:15:25,920 --> 01:15:28,160 And then we can ask about certain moments 1354 01:15:28,160 --> 01:15:31,910 or characteristics of these continuous probability density 1355 01:15:31,910 --> 01:15:32,980 functions. 1356 01:15:32,980 --> 01:15:34,310 These should be familiar to. 1357 01:15:34,310 --> 01:15:38,540 We can ask, what is the mean or the expected value of one 1358 01:15:38,540 --> 01:15:39,980 of these distributions? 1359 01:15:39,980 --> 01:15:45,290 And here I'm using t as a stand-in for time or run 1360 01:15:45,290 --> 01:15:46,310 number. 1361 01:15:46,310 --> 01:15:50,540 And if I have a probability density function associated 1362 01:15:50,540 --> 01:15:56,000 with the random variable, and even time, then 1363 01:15:56,000 --> 01:16:00,260 there we have a definition of the expected value 1364 01:16:00,260 --> 01:16:02,880 of x as a function of time. 1365 01:16:02,880 --> 01:16:05,810 Now, I'm kind of being pedantic here and giving 1366 01:16:05,810 --> 01:16:10,660 this definition, because we also saw trends-- 1367 01:16:10,660 --> 01:16:12,220 long-term trends in our data. 1368 01:16:12,220 --> 01:16:16,195 And we could ask what the mean is as a function of time. 1369 01:16:19,330 --> 01:16:22,060 Again, we're in a situation where I'm trying 1370 01:16:22,060 --> 01:16:24,070 to squeeze out those drifts. 1371 01:16:24,070 --> 01:16:30,610 So usually we're in a situation where we want to avoid 1372 01:16:30,610 --> 01:16:35,740 non-stationary drifting processes or analyses where 1373 01:16:35,740 --> 01:16:37,450 some of these functions-- 1374 01:16:37,450 --> 01:16:40,390 the probability density function and its parameters, 1375 01:16:40,390 --> 01:16:41,470 like the mean-- 1376 01:16:41,470 --> 01:16:43,030 drift over time. 1377 01:16:43,030 --> 01:16:48,400 We typically want to be having stationary processes where 1378 01:16:48,400 --> 01:16:54,430 the mean and other moments, like variance, are 1379 01:16:54,430 --> 01:16:57,940 independent of time. 1380 01:17:03,350 --> 01:17:06,400 So in the stationary case, which is what we're mostly 1381 01:17:06,400 --> 01:17:08,560 going to be dealing with, now I've 1382 01:17:08,560 --> 01:17:11,680 removed the time dependence on the underlying 1383 01:17:11,680 --> 01:17:13,600 statistical model. 1384 01:17:13,600 --> 01:17:16,520 Let's say, from time to time, it should be stable. 1385 01:17:16,520 --> 01:17:20,770 And then we have a mean, this mu of x-- the true, 1386 01:17:20,770 --> 01:17:24,070 or theoretical true mean-- 1387 01:17:24,070 --> 01:17:26,830 that we would calculate with the same formula. 1388 01:17:26,830 --> 01:17:29,950 But again, the idea of the stationary process-- 1389 01:17:29,950 --> 01:17:33,770 important characteristic there is that the mean is a constant, 1390 01:17:33,770 --> 01:17:35,455 and not wandering or drifting. 1391 01:17:38,680 --> 01:17:42,820 Similarly, in the case when it's a stationary process, 1392 01:17:42,820 --> 01:17:47,570 the variance, defined here as the second moment-- 1393 01:17:47,570 --> 01:17:52,430 the expectation of the distance squared of your data 1394 01:17:52,430 --> 01:17:53,990 from the mean-- 1395 01:17:53,990 --> 01:17:56,600 [INAUDIBLE] expectation of that-- 1396 01:17:56,600 --> 01:17:57,830 is the variance. 1397 01:17:57,830 --> 01:17:59,480 That's just the definition. 1398 01:17:59,480 --> 01:18:02,750 And in the stationary case, that is also 1399 01:18:02,750 --> 01:18:04,400 a constant of the process-- 1400 01:18:04,400 --> 01:18:08,060 has a nice alternative little description, 1401 01:18:08,060 --> 01:18:09,560 if you work through the mathematics, 1402 01:18:09,560 --> 01:18:15,080 where it can also be calculated as the expectation 1403 01:18:15,080 --> 01:18:18,230 of the square of your data minus the mean squared. 1404 01:18:21,360 --> 01:18:25,190 By the way, you can ask the same questions-- variance, mean-- 1405 01:18:25,190 --> 01:18:27,530 for any continuous distribution. 1406 01:18:27,530 --> 01:18:29,930 They're perfectly good moments. 1407 01:18:29,930 --> 01:18:33,830 In some cases, they completely characterize the distribution. 1408 01:18:33,830 --> 01:18:37,670 In other cases, there may need to be additional moments. 1409 01:18:37,670 --> 01:18:40,800 And an example here is we could ask, 1410 01:18:40,800 --> 01:18:43,910 what is the mean of a uniform distribution-- uniform, where 1411 01:18:43,910 --> 01:18:50,190 the probability density is equal across an entire range? 1412 01:18:50,190 --> 01:18:55,310 And so one could calculate those as well. 1413 01:18:55,310 --> 01:18:59,320 Everybody is seeing the normal distribution. 1414 01:18:59,320 --> 01:19:01,990 Again, we can actually have an explicit description 1415 01:19:01,990 --> 01:19:05,020 for what the PDF is. 1416 01:19:05,020 --> 01:19:11,780 We'll often be talking about the unit standard normal or unit 1417 01:19:11,780 --> 01:19:13,670 normal distribution. 1418 01:19:13,670 --> 01:19:18,440 The full distribution here actually has the mean in it, 1419 01:19:18,440 --> 01:19:21,150 and it has the variance in it. 1420 01:19:21,150 --> 01:19:23,330 So for your particular process, those 1421 01:19:23,330 --> 01:19:27,500 may be very different for whatever collection of data. 1422 01:19:27,500 --> 01:19:33,520 In order to be able to talk generically about properties 1423 01:19:33,520 --> 01:19:37,480 of the normal distribution, and to use tabulated values and so 1424 01:19:37,480 --> 01:19:42,040 on, this normalization, where we subtract off the mean 1425 01:19:42,040 --> 01:19:45,430 to get a 0 mean description, and then we 1426 01:19:45,430 --> 01:19:48,460 divide by the standard deviation, 1427 01:19:48,460 --> 01:19:51,890 gives us the unit normal distribution. 1428 01:19:51,890 --> 01:19:54,580 And that's exactly always this distribution 1429 01:19:54,580 --> 01:19:58,840 in terms of z units. 1430 01:19:58,840 --> 01:20:01,750 Now we've made it independent to the mean, independent 1431 01:20:01,750 --> 01:20:04,000 of the variance. 1432 01:20:04,000 --> 01:20:08,320 And the exact values, the probability densities 1433 01:20:08,320 --> 01:20:13,450 for any values of z are well-known and tabulated. 1434 01:20:13,450 --> 01:20:16,990 So we've made it 0 mean and made it unit norm-- 1435 01:20:16,990 --> 01:20:18,950 unit variance. 1436 01:20:18,950 --> 01:20:21,860 There is also a cumulative distribution here. 1437 01:20:21,860 --> 01:20:25,550 I'm describing this as p of z, so now we 1438 01:20:25,550 --> 01:20:27,440 can ask the cumulative distribution 1439 01:20:27,440 --> 01:20:32,900 function for that unit normal, and plot that. 1440 01:20:35,530 --> 01:20:38,530 For the normal PDF, what's nice is 1441 01:20:38,530 --> 01:20:40,690 this is the whole distribution, and all 1442 01:20:40,690 --> 01:20:44,470 we need are those two parameters, mean and variance, 1443 01:20:44,470 --> 01:20:48,010 and it completely characterizes the process. 1444 01:20:48,010 --> 01:20:50,050 And then there's some statistical operations 1445 01:20:50,050 --> 01:20:53,680 that we can do for when we-- that are really convenient when 1446 01:20:53,680 --> 01:20:55,200 we have normal distributions. 1447 01:20:55,200 --> 01:21:00,160 For example, if we have multiple effects, we can look, 1448 01:21:00,160 --> 01:21:03,580 and the sum of multiple normal random variables 1449 01:21:03,580 --> 01:21:06,555 still has a normal distribution. 1450 01:21:06,555 --> 01:21:08,680 So I'm just giving you a little bit of a peek here. 1451 01:21:08,680 --> 01:21:10,450 You can do some reading, which is probably 1452 01:21:10,450 --> 01:21:15,130 more effective than me just flashing by these things. 1453 01:21:15,130 --> 01:21:17,680 There are important operations, for example, 1454 01:21:17,680 --> 01:21:20,260 where, if we have an output y that 1455 01:21:20,260 --> 01:21:23,170 is a sum of multiple random variables, 1456 01:21:23,170 --> 01:21:26,410 there are things like the mean of the overall y 1457 01:21:26,410 --> 01:21:28,780 is simply the sum of the mean of each 1458 01:21:28,780 --> 01:21:30,730 of these individual random variables. 1459 01:21:30,730 --> 01:21:34,270 And if there's a scaling in the mean, that simply scales. 1460 01:21:34,270 --> 01:21:37,360 There are also operations when these-- 1461 01:21:37,360 --> 01:21:42,880 if these are independent random variables, the variances add. 1462 01:21:42,880 --> 01:21:44,890 And you've got to be careful any constant. 1463 01:21:44,890 --> 01:21:47,230 If you run that through the definition of variance, 1464 01:21:47,230 --> 01:21:48,950 those are also squared. 1465 01:21:48,950 --> 01:21:51,580 So you'll see the basic description 1466 01:21:51,580 --> 01:21:53,260 of some of these mathematical operations 1467 01:21:53,260 --> 01:21:57,850 and these distributions in May and Spanos. 1468 01:21:57,850 --> 01:22:00,040 And you'll also start to get a feel 1469 01:22:00,040 --> 01:22:03,820 for using these distributions to ask 1470 01:22:03,820 --> 01:22:07,980 some very standard questions, like what 1471 01:22:07,980 --> 01:22:09,630 is the probability that you would 1472 01:22:09,630 --> 01:22:13,600 observe a value in some range? 1473 01:22:13,600 --> 01:22:15,490 So I think, with that, we'll close. 1474 01:22:15,490 --> 01:22:17,620 If you want to get started on problem set 2, 1475 01:22:17,620 --> 01:22:20,200 there's some problems that-- 1476 01:22:20,200 --> 01:22:24,550 this is just basically giving you some familiarity 1477 01:22:24,550 --> 01:22:28,330 with manipulating things with Gaussian distributions 1478 01:22:28,330 --> 01:22:29,470 and other statistics. 1479 01:22:29,470 --> 01:22:34,380 And then we'll start building up more detail next Thursday.