WEBVTT 00:00:00.000 --> 00:00:02.430 The following content is provided under a Creative 00:00:02.430 --> 00:00:03.730 Commons license. 00:00:03.730 --> 00:00:06.030 Your support will help MIT OpenCourseWare 00:00:06.030 --> 00:00:10.060 continue to offer high-quality educational resources for free. 00:00:10.060 --> 00:00:12.690 To make a donation or to view additional materials 00:00:12.690 --> 00:00:16.560 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:16.560 --> 00:00:17.892 at ocw.mit.edu. 00:00:22.140 --> 00:00:25.500 PROFESSOR: Well, while people are filing in, 00:00:25.500 --> 00:00:27.720 we can chat a little bit about the quiz overall. 00:00:27.720 --> 00:00:31.050 I was very, very impressed, at least with the question 1 00:00:31.050 --> 00:00:33.450 and question 2, which I graded. 00:00:33.450 --> 00:00:36.840 I haven't graded question 3. 00:00:36.840 --> 00:00:38.820 So here's all the statistics. 00:00:38.820 --> 00:00:41.650 You can see that this was a pretty difficult quiz, 00:00:41.650 --> 00:00:42.150 actually. 00:00:46.140 --> 00:00:47.520 We do agree. 00:00:47.520 --> 00:00:49.800 You have evidence to-- 00:00:49.800 --> 00:00:50.810 AUDIENCE: [INAUDIBLE] 00:00:50.810 --> 00:00:53.040 PROFESSOR: --to overcome that null hypothesis 00:00:53.040 --> 00:00:57.070 that it was a difficult quiz. 00:00:57.070 --> 00:01:03.870 So the average there about 78, 79, with a fairly wide spread-- 00:01:03.870 --> 00:01:09.208 what we will do is post both the quiz and the solutions-- which 00:01:09.208 --> 00:01:11.250 may be more interesting to you, since you already 00:01:11.250 --> 00:01:13.830 have the quiz. 00:01:13.830 --> 00:01:16.980 And if you do see anything, feel free to chat with us 00:01:16.980 --> 00:01:21.090 about the question or whatever. 00:01:21.090 --> 00:01:24.390 If you're concerned about a point here or point there, 00:01:24.390 --> 00:01:26.040 I really wouldn't worry about it. 00:01:28.950 --> 00:01:35.220 Our grading is-- we attempt to be as accurate as possible, 00:01:35.220 --> 00:01:39.240 but we reserve the right to have a little bit of variation, 00:01:39.240 --> 00:01:43.470 given the nature of the course, in the grading. 00:01:43.470 --> 00:01:49.410 And we will, of course, be treating everything 00:01:49.410 --> 00:01:52.290 a little bit as a mix in at the end, 00:01:52.290 --> 00:01:55.140 from this quiz grade, the second quiz grade, class 00:01:55.140 --> 00:01:57.210 participation, homeworks-- 00:01:57.210 --> 00:01:58.420 all of that. 00:01:58.420 --> 00:02:03.750 So I think the main thing to do is look at the solutions 00:02:03.750 --> 00:02:08.100 and make sure that you understand 00:02:08.100 --> 00:02:11.642 at least the approaches that we took in the solutions-- 00:02:11.642 --> 00:02:14.100 which is not to say that some of the approaches that people 00:02:14.100 --> 00:02:17.980 took on the quiz is wrong. 00:02:17.980 --> 00:02:21.180 In fact, on problem 2, I think the biggest thing 00:02:21.180 --> 00:02:25.050 that we observed is people answered the third part, if you 00:02:25.050 --> 00:02:28.420 remember, really the hard way. 00:02:28.420 --> 00:02:32.430 And unfortunately, you may have burned a fair amount of time 00:02:32.430 --> 00:02:36.210 on that with very detailed t-tests, and f-tests, 00:02:36.210 --> 00:02:41.190 and complicated hypothesis tests-- 00:02:41.190 --> 00:02:45.000 whereas what you had was already the confidence intervals 00:02:45.000 --> 00:02:46.740 from parts A and B that you could 00:02:46.740 --> 00:02:50.830 use directly, in many cases, to make inferences. 00:02:50.830 --> 00:02:54.150 So were there any questions on the quiz-- 00:02:54.150 --> 00:02:58.320 top level, not detailed? 00:02:58.320 --> 00:02:59.770 OK, well, take a look at those. 00:02:59.770 --> 00:03:01.650 And again, talk to Hayden or I if-- 00:03:01.650 --> 00:03:06.330 more if you have any confusions, I think, is the main thing. 00:03:06.330 --> 00:03:12.300 OK, so here's the plan for today. 00:03:12.300 --> 00:03:15.810 We started on full factorial models design of experiments 00:03:15.810 --> 00:03:16.480 last time. 00:03:16.480 --> 00:03:18.960 I'm going to pick up where we left off on some of that, 00:03:18.960 --> 00:03:23.250 talk a little bit more about the extension of that to generalize 00:03:23.250 --> 00:03:25.080 to the k experiments. 00:03:25.080 --> 00:03:28.200 The key idea here is, again, picking out experimental design 00:03:28.200 --> 00:03:31.380 points so that you can fit particular kinds of models. 00:03:31.380 --> 00:03:34.740 And we'll start, again, with ANOVA for linear models, 00:03:34.740 --> 00:03:39.000 and then move on to asking the question, what 00:03:39.000 --> 00:03:40.950 if a linear model is not sufficient? 00:03:40.950 --> 00:03:45.630 How do if a linear model is not sufficient? 00:03:45.630 --> 00:03:49.170 How do you check for the adequacy of the model form? 00:03:49.170 --> 00:03:53.340 Is there evidence that you need higher order terms, 00:03:53.340 --> 00:03:56.350 like a quadratic or other curvature? 00:03:56.350 --> 00:03:59.580 So we'll spend the first half really focused on that. 00:03:59.580 --> 00:04:02.550 Then we'll start moving into some additional issues 00:04:02.550 --> 00:04:03.930 in experimental design-- 00:04:03.930 --> 00:04:09.240 these ideas of blocks and confounding, as well as-- 00:04:09.240 --> 00:04:11.010 those same issues come out very strongly 00:04:11.010 --> 00:04:12.940 in fractional factorial designs. 00:04:12.940 --> 00:04:14.040 So we'll get to that. 00:04:14.040 --> 00:04:18.810 Now, I posted on the website reading assignment. 00:04:18.810 --> 00:04:21.750 The key is probably Montgomery chapter 12. 00:04:21.750 --> 00:04:24.540 I find that a little bit better description 00:04:24.540 --> 00:04:28.860 than May and Spanos. 00:04:28.860 --> 00:04:30.510 But there are some additional topics 00:04:30.510 --> 00:04:32.280 at the end of the May and Spanos chapter. 00:04:32.280 --> 00:04:33.947 I can't remember which chapter that is-- 00:04:33.947 --> 00:04:36.760 7, 13-- can't remember. 00:04:36.760 --> 00:04:38.322 But it's posted on the website. 00:04:38.322 --> 00:04:39.780 And so there are some things you'll 00:04:39.780 --> 00:04:43.570 want to scan in May and Spanos as well. 00:04:43.570 --> 00:04:46.485 So recall that last time we started talking a little bit 00:04:46.485 --> 00:04:48.360 about the kinds of models that are associated 00:04:48.360 --> 00:04:50.860 with different experimental designs. 00:04:50.860 --> 00:04:55.260 So for example, we start with the simple model-- 00:04:55.260 --> 00:04:58.170 it's not the simplest, because we already have two factors, 00:04:58.170 --> 00:05:03.300 but a very simple 2-by-2 or 2 to the 2 model 00:05:03.300 --> 00:05:08.430 where, again, what we've got are two levels 00:05:08.430 --> 00:05:11.830 in each of two factors. 00:05:11.830 --> 00:05:14.760 So our factors here are x1 and x2, 00:05:14.760 --> 00:05:16.300 and we've got two different levels. 00:05:16.300 --> 00:05:18.780 So we're just picking the corner points or extremal points 00:05:18.780 --> 00:05:19.720 for that. 00:05:19.720 --> 00:05:21.340 And we talked last time about how 00:05:21.340 --> 00:05:25.300 this can generate a regression model that 00:05:25.300 --> 00:05:30.400 has both mean and first-order terms, 00:05:30.400 --> 00:05:32.710 as well as an interaction term. 00:05:32.710 --> 00:05:34.900 We also talked about the way that you 00:05:34.900 --> 00:05:37.960 can estimate these using contrast, 00:05:37.960 --> 00:05:40.090 and I'm going to pick up on that a little bit, 00:05:40.090 --> 00:05:45.190 where we talk about the effective varying x1-- 00:05:45.190 --> 00:05:49.510 the total effect being a movement in the output, 00:05:49.510 --> 00:05:51.400 or an average movement in the output 00:05:51.400 --> 00:05:58.660 by a magnitude A in these coded coefficients for x1-- 00:05:58.660 --> 00:06:03.250 as well as effects for B and the interaction effect. 00:06:03.250 --> 00:06:04.720 Basically, we're seeing that there 00:06:04.720 --> 00:06:07.780 is this correspondence between a regression approach 00:06:07.780 --> 00:06:09.280 or a contrast approach. 00:06:09.280 --> 00:06:13.930 It's a very fast, quick way to estimate 00:06:13.930 --> 00:06:18.310 these coefficients using these kinds of design of experiments. 00:06:29.260 --> 00:06:30.810 OK, that's not good. 00:06:35.880 --> 00:06:37.560 That's very strange. 00:06:37.560 --> 00:06:41.490 All right, so a little bit of the terminology 00:06:41.490 --> 00:06:44.430 that we talked about just to remind us, and then 00:06:44.430 --> 00:06:47.340 expand on that a little bit-- 00:06:47.340 --> 00:06:50.550 there is this notion of a contrast that's 00:06:50.550 --> 00:06:53.790 going to get us to being able to estimate 00:06:53.790 --> 00:07:01.770 the effect of, say, coefficient A, or beta 1, or beta-- 00:07:01.770 --> 00:07:06.910 each of our two main effects or the interaction effects. 00:07:06.910 --> 00:07:12.090 And these condensed down tables summarizing the design 00:07:12.090 --> 00:07:14.850 of your experiment are really useful tools 00:07:14.850 --> 00:07:18.120 for being able to generate these contrasts, 00:07:18.120 --> 00:07:21.690 for being able to identify what the mix of level settings 00:07:21.690 --> 00:07:25.630 are for each of the trial runs that you do, and so on. 00:07:25.630 --> 00:07:28.710 So a little bit of terminology-- again, we 00:07:28.710 --> 00:07:35.190 said this column here, the trial column, is essentially our run. 00:07:35.190 --> 00:07:38.400 And we can label them in a shorthand that tells us 00:07:38.400 --> 00:07:41.190 what our factor settings are, our level settings 00:07:41.190 --> 00:07:44.550 are for each of our two factors-- 00:07:44.550 --> 00:07:49.090 our x1 factor-- that's one-- and our x2 factor. 00:07:49.090 --> 00:07:53.700 So the first label here is just the low setting 00:07:53.700 --> 00:07:57.990 for both of our factors, and we'll label that with a 1. 00:07:57.990 --> 00:08:00.300 Our high setting for the a factor-- 00:08:00.300 --> 00:08:04.230 we label that one with a high setting, or a plus. 00:08:04.230 --> 00:08:06.960 Sometimes we'll label that as a plus 1-- 00:08:06.960 --> 00:08:10.440 and our low setting then, for everything else, for b. 00:08:10.440 --> 00:08:13.440 Our b factor-- that's where we're set that one high. 00:08:13.440 --> 00:08:15.270 And when we set both of them high, 00:08:15.270 --> 00:08:18.940 that's our plus setting for the two of them. 00:08:18.940 --> 00:08:24.660 Now, we can also think about, what is the factor 00:08:24.660 --> 00:08:26.610 level for the interaction? 00:08:26.610 --> 00:08:31.320 And we do that with this very funky shorthand algebra, just 00:08:31.320 --> 00:08:33.929 saying, OK, AB is really-- 00:08:33.929 --> 00:08:36.510 think of that as a multiplication of the A and B 00:08:36.510 --> 00:08:37.600 factors. 00:08:37.600 --> 00:08:41.110 So a minus 1 and a minus 1 gives a plus 1 there-- 00:08:41.110 --> 00:08:43.919 and similarly for all of the other columns. 00:08:43.919 --> 00:08:47.620 That generates the level setting, if you will, 00:08:47.620 --> 00:08:50.710 for each of those combinations. 00:08:50.710 --> 00:08:52.420 Now, in the actual experiment, of course, 00:08:52.420 --> 00:08:55.950 we're only setting x1 and x2 for each of our trials. 00:08:55.950 --> 00:09:01.500 These experimental interaction columns are essentially giving 00:09:01.500 --> 00:09:06.840 us information that we need in order to form the contrast 00:09:06.840 --> 00:09:10.470 in the shorthand way of estimating effects, 00:09:10.470 --> 00:09:13.530 as well as-- and we'll see this a little bit later-- 00:09:13.530 --> 00:09:19.210 as well as telling us what the interaction term should be, 00:09:19.210 --> 00:09:21.310 if we're going to form regression-- 00:09:21.310 --> 00:09:24.860 including do the regression on the interaction factor. 00:09:24.860 --> 00:09:27.420 So it's pretty funky, all intertwined here. 00:09:30.040 --> 00:09:34.300 Something strange in our slides-- 00:09:34.300 --> 00:09:35.890 I don't know what that is. 00:09:35.890 --> 00:09:39.700 That's supposed to be a dot, just a multiplication-- 00:09:39.700 --> 00:09:42.430 because now we can say, OK, how do we 00:09:42.430 --> 00:09:49.510 form the contrast between our different combinations 00:09:49.510 --> 00:09:53.890 of settings in order to estimate the effect of A? 00:09:53.890 --> 00:09:55.810 And essentially what we're going to be doing 00:09:55.810 --> 00:10:01.060 is forming the average between the high for A-- 00:10:01.060 --> 00:10:02.695 this column and this column-- 00:10:05.850 --> 00:10:08.730 or the difference between the average of those two 00:10:08.730 --> 00:10:12.880 columns and these two settings. 00:10:12.880 --> 00:10:15.060 So I'm basically forming the contrast 00:10:15.060 --> 00:10:17.730 in each case for each factor as the difference 00:10:17.730 --> 00:10:19.650 between the high and low setting. 00:10:22.330 --> 00:10:25.230 And so that's actually why we need the interaction factor, 00:10:25.230 --> 00:10:28.890 so we can look at the high settings for the interaction, 00:10:28.890 --> 00:10:34.860 the AB and the 1, and then the minus A and the minus B. 00:10:34.860 --> 00:10:36.420 Where are these coming from? 00:10:36.420 --> 00:10:39.000 A much better way to picture this 00:10:39.000 --> 00:10:44.560 is if we look at this graphically. 00:10:44.560 --> 00:10:48.450 So what I'm going to do here first is generalize. 00:10:48.450 --> 00:10:52.410 This is a 2 to the 3 design. 00:10:52.410 --> 00:10:55.320 In more general terms, we can do a full factorial 00:10:55.320 --> 00:10:59.250 with any number of factors and form this sort of hypercube, 00:10:59.250 --> 00:11:01.770 again, with corner points at all combinations 00:11:01.770 --> 00:11:04.170 at the high and low values-- 00:11:04.170 --> 00:11:06.430 or levels for each of our setting. 00:11:06.430 --> 00:11:08.040 And on that hypercube-- so now I've 00:11:08.040 --> 00:11:12.360 got A factor, B factor, and C factor. 00:11:12.360 --> 00:11:14.340 This is a nice picture for forming 00:11:14.340 --> 00:11:16.950 what that contrast is doing and how 00:11:16.950 --> 00:11:22.230 we're estimating the average effect of changing one factor 00:11:22.230 --> 00:11:24.840 at a time on this picture. 00:11:24.840 --> 00:11:28.440 So just conceptually, or intuitively, 00:11:28.440 --> 00:11:33.390 if you were trying to form an estimate of the factor effect 00:11:33.390 --> 00:11:37.830 A associated with some variable x1, 00:11:37.830 --> 00:11:39.720 notice that-- what we've got. 00:11:39.720 --> 00:11:44.520 At all of these settings on the left, shaded with the blue, 00:11:44.520 --> 00:11:48.390 those are all low settings for our a factor. 00:11:48.390 --> 00:11:52.350 I'm varying B and C also, but for all of those, 00:11:52.350 --> 00:11:55.470 I'm holding the a factor at the low value. 00:11:55.470 --> 00:11:57.450 And then similarly, on the red side, 00:11:57.450 --> 00:12:00.960 I'm holding the a factor at the high value 00:12:00.960 --> 00:12:02.460 in all of those cases-- 00:12:02.460 --> 00:12:05.010 maybe varying other things too. 00:12:05.010 --> 00:12:08.760 But if I think that those other effects are basically 00:12:08.760 --> 00:12:11.700 balancing out, now what I've got is a difference 00:12:11.700 --> 00:12:16.620 between the A high and the A low as a net overall estimate 00:12:16.620 --> 00:12:20.370 of the total effective changing variable x1, 00:12:20.370 --> 00:12:22.590 changing the variable associated with A. 00:12:22.590 --> 00:12:25.080 And that's all that the contrasts are doing. 00:12:25.080 --> 00:12:29.250 They're simply saying, OK, the high contrast 00:12:29.250 --> 00:12:31.920 is all of the places where A is set 00:12:31.920 --> 00:12:38.340 high minus all of the places where A is set low. 00:12:38.340 --> 00:12:41.010 Form that difference. 00:12:41.010 --> 00:12:43.020 Recall that we've got-- 00:12:43.020 --> 00:12:44.530 I need to take the average. 00:12:44.530 --> 00:12:47.400 So in each of these cases, I've got four points at high 00:12:47.400 --> 00:12:49.650 and four points at low, so I'm taking the average 00:12:49.650 --> 00:12:51.060 and taking the difference. 00:12:51.060 --> 00:12:56.070 That's my estimate of the total net effect. 00:12:56.070 --> 00:12:56.723 Question-- 00:12:56.723 --> 00:12:58.140 AUDIENCE: What happens if I locate 00:12:58.140 --> 00:13:04.610 AC and I have a diagram of four points of the same [INAUDIBLE]?? 00:13:07.700 --> 00:13:09.710 PROFESSOR: OK, say that question again. 00:13:09.710 --> 00:13:10.530 Here's AC. 00:13:10.530 --> 00:13:11.440 AUDIENCE: Yeah. 00:13:11.440 --> 00:13:17.510 It's like then I think if I move from 0.1 to AC, 00:13:17.510 --> 00:13:24.170 and then I have that points AC, BC, and AB are in between. 00:13:24.170 --> 00:13:26.665 [INAUDIBLE] 00:13:26.665 --> 00:13:28.040 PROFESSOR: So far what we've done 00:13:28.040 --> 00:13:29.990 is only estimated one of the contrasts. 00:13:29.990 --> 00:13:32.630 It looks like you're actually getting close to estimating 00:13:32.630 --> 00:13:37.850 a different term in a model. 00:13:37.850 --> 00:13:42.920 So what we did here was just to estimate the a effect. 00:13:42.920 --> 00:13:47.360 I would also need to do the same thing for the B effect. 00:13:47.360 --> 00:13:50.240 So that's telling me, if I vary x2-- 00:13:50.240 --> 00:13:53.300 and I can similarly do that for the C effect. 00:13:53.300 --> 00:13:55.340 But then there's the question, well, 00:13:55.340 --> 00:13:58.910 what if there's an interaction between two of these terms? 00:13:58.910 --> 00:14:03.830 And one can also form a contrast for an interaction effect. 00:14:03.830 --> 00:14:06.810 And maybe this is not exactly what you are describing, 00:14:06.810 --> 00:14:09.590 but it's getting close to other ways of combining 00:14:09.590 --> 00:14:11.580 some of these effects. 00:14:11.580 --> 00:14:17.030 So it turns out what this does is makes a balanced combination 00:14:17.030 --> 00:14:20.690 of our data points to be able to take advantage of that AB 00:14:20.690 --> 00:14:22.160 column. 00:14:22.160 --> 00:14:25.790 This is just basically taking the high and low combinations 00:14:25.790 --> 00:14:29.330 and forming that estimate of the AB interaction 00:14:29.330 --> 00:14:35.480 effect using the contrast between these points. 00:14:35.480 --> 00:14:38.390 Now, there would also be another contrast for the BC 00:14:38.390 --> 00:14:43.250 interaction, if you rotated these around and formed 00:14:43.250 --> 00:14:50.280 the plane where you took the high-low combinations of B 00:14:50.280 --> 00:14:54.010 and C, instead of A and B. OK? 00:14:56.990 --> 00:14:59.540 Does that get at your question, or was there-- 00:14:59.540 --> 00:15:01.890 you can always form different combinations of things. 00:15:01.890 --> 00:15:03.557 The question is, what combination do you 00:15:03.557 --> 00:15:05.420 need in order to be able to estimate 00:15:05.420 --> 00:15:07.250 the coefficient in the model? 00:15:07.250 --> 00:15:08.360 AUDIENCE: Yeah. 00:15:08.360 --> 00:15:11.030 I mean, [INAUDIBLE]. 00:15:11.030 --> 00:15:12.830 I don't know. 00:15:12.830 --> 00:15:15.992 It does not like derivatives of A, B, and C. 00:15:15.992 --> 00:15:16.700 PROFESSOR: Right. 00:15:16.700 --> 00:15:20.000 AUDIENCE: You just take [INAUDIBLE] and cross them. 00:15:20.000 --> 00:15:22.670 PROFESSOR: Yes. 00:15:22.670 --> 00:15:25.670 That cross thing is intuitive in the sense-- 00:15:25.670 --> 00:15:30.590 only in the sense that it corresponds nicely-- 00:15:30.590 --> 00:15:34.490 you can generate it once you have this column. 00:15:34.490 --> 00:15:36.930 So now I can connect up the high points, 00:15:36.930 --> 00:15:38.270 and that would be one plane. 00:15:38.270 --> 00:15:39.560 Connect up the low points-- 00:15:39.560 --> 00:15:40.670 that's my other plane. 00:15:40.670 --> 00:15:42.330 And then I take the difference. 00:15:42.330 --> 00:15:47.390 But it's not entirely intuitive why that particular combination 00:15:47.390 --> 00:15:49.670 gives you the estimate of that coefficient. 00:15:49.670 --> 00:15:51.120 I agree with that. 00:15:51.120 --> 00:15:54.080 So this is the shorthand that gives the answer. 00:15:57.780 --> 00:16:01.650 In fact, I don't have a good intuitive explanation, 00:16:01.650 --> 00:16:06.180 just looking purely at this picture, 00:16:06.180 --> 00:16:08.800 why that estimates the interaction. 00:16:11.370 --> 00:16:17.430 The closest I can come is almost in two dimensions, instead 00:16:17.430 --> 00:16:24.130 of three, with these interaction plots that we've drawn before. 00:16:24.130 --> 00:16:28.840 So the basic interaction plot would look and say, 00:16:28.840 --> 00:16:32.440 if I just have x1 and x2 values, I just 00:16:32.440 --> 00:16:38.380 draw them as here's my output as I vary x1 from high to low, 00:16:38.380 --> 00:16:44.480 holding x2 low, that would be one effect. 00:16:44.480 --> 00:16:53.550 If now I let x2 be high, I don't have just an offset, 00:16:53.550 --> 00:16:58.230 but I've got a different delta here than here. 00:16:58.230 --> 00:17:04.500 That means that, in combination, x1 and x2 work together 00:17:04.500 --> 00:17:07.890 to do something different than purely x1 or x3. 00:17:07.890 --> 00:17:10.290 There is not just purely an additive 00:17:10.290 --> 00:17:14.290 A effect and a B effect, but there is an AB effect. 00:17:17.770 --> 00:17:19.589 So in two dimensions, essentially 00:17:19.589 --> 00:17:21.780 what you're doing is looking at forming 00:17:21.780 --> 00:17:26.369 the delta from this combination and this combination, 00:17:26.369 --> 00:17:30.420 and saying there is a difference-- 00:17:30.420 --> 00:17:34.140 an extra delta between the two. 00:17:34.140 --> 00:17:36.630 In 3D, it's a little harder to see, 00:17:36.630 --> 00:17:39.720 but I think it's doing conceptually that same thing. 00:17:39.720 --> 00:17:40.710 Yeah? 00:17:40.710 --> 00:17:43.500 AUDIENCE: Why do you take the average divide it by 4 00:17:43.500 --> 00:17:47.740 but not do the same in the two factor? 00:17:47.740 --> 00:17:51.268 PROFESSOR: Oh, I think we did in the two factor. 00:17:51.268 --> 00:17:53.310 AUDIENCE: No, no, in the slide previous to that-- 00:17:57.498 --> 00:17:59.040 PROFESSOR: So the 4 is basically just 00:17:59.040 --> 00:18:03.392 because I'm taking the average from each of the-- 00:18:03.392 --> 00:18:06.550 this plane has four points, so I'm just taking the average 00:18:06.550 --> 00:18:09.430 from that the difference from that, so the 4 00:18:09.430 --> 00:18:12.520 is coming from the number of points I had in each average. 00:18:12.520 --> 00:18:15.970 That's an estimate of the total effect. 00:18:15.970 --> 00:18:17.900 Then there's one additional subtle point, 00:18:17.900 --> 00:18:21.190 which is, how do I get from that factor-- 00:18:21.190 --> 00:18:25.950 total of net effect, main effect A, or interaction effect AB-- 00:18:25.950 --> 00:18:29.150 to an estimate of the model coefficient? 00:18:29.150 --> 00:18:32.230 And that's where some scaling comes into play. 00:18:32.230 --> 00:18:35.260 What we've done in these pictures 00:18:35.260 --> 00:18:43.820 is scale our x1 and x2 to a low of minus 1 and a high of 1, 00:18:43.820 --> 00:18:45.750 and intermediate point of 0. 00:18:51.340 --> 00:18:54.700 Let's say this was the overall a effect. 00:18:54.700 --> 00:18:57.010 What I'd like is something that goes like y 00:18:57.010 --> 00:19:01.990 equals beta 0 plus beta 1 x1. 00:19:01.990 --> 00:19:06.040 And so what I've got is a total range in x1 of 2. 00:19:06.040 --> 00:19:15.400 So I have to basically scale this so that I've got A/2 x1. 00:19:15.400 --> 00:19:18.580 As x goes from 0 to 1, I've got half of the effect, 00:19:18.580 --> 00:19:20.440 and as x goes from 0 to minus 1, I've 00:19:20.440 --> 00:19:22.540 got the other half of the effective. 00:19:22.540 --> 00:19:25.030 So that's where beta 1 is equal to the A/2. 00:19:28.480 --> 00:19:32.050 Again, a lot of what I'm saying is only true 00:19:32.050 --> 00:19:35.560 if we're using the scaled coefficients for x1, 00:19:35.560 --> 00:19:37.570 from minus 1 to plus 1. 00:19:37.570 --> 00:19:42.400 And then all of this magical shorthand of these column 00:19:42.400 --> 00:19:44.590 tables, the multiplication column tables, 00:19:44.590 --> 00:19:48.717 the formation of these contrasts works out really nicely. 00:19:48.717 --> 00:19:50.800 And so that's where you'll look in the literature, 00:19:50.800 --> 00:19:53.950 and there's so much machinery around this that makes 00:19:53.950 --> 00:19:56.890 it relatively quick and easy. 00:20:04.250 --> 00:20:07.160 So we can extend all of this, again, to-- 00:20:07.160 --> 00:20:09.620 beyond one factor, or two factors, 00:20:09.620 --> 00:20:13.830 to three factors, or any k factors. 00:20:13.830 --> 00:20:15.935 So we can consider, for example, a 2 00:20:15.935 --> 00:20:19.760 to the third experiment, where now we have three factors-- 00:20:19.760 --> 00:20:22.100 an x1, x2, x3-- 00:20:22.100 --> 00:20:28.070 or going with an A effect, B effect, and C effect. 00:20:28.070 --> 00:20:31.160 And then we can just extend this to the different treatment 00:20:31.160 --> 00:20:32.540 combinations. 00:20:32.540 --> 00:20:34.280 And essentially, what we're doing here 00:20:34.280 --> 00:20:38.450 is formulating every possible corner point, 00:20:38.450 --> 00:20:41.930 every possible combination of the high and low 00:20:41.930 --> 00:20:43.460 for each of the settings. 00:20:43.460 --> 00:20:44.990 And the same idea applies. 00:20:44.990 --> 00:20:48.650 Here I've indicated them with a plus 1, minus 1, 00:20:48.650 --> 00:20:52.740 rather than just plus minus, but it's exactly the same idea. 00:20:52.740 --> 00:20:55.830 And you can see C already lurking here. 00:20:55.830 --> 00:21:00.660 Which runs would you use to estimate the C effect? 00:21:00.660 --> 00:21:02.670 That one sort of leaps out at us, 00:21:02.670 --> 00:21:06.690 as we happen to have ordered this table. 00:21:06.690 --> 00:21:09.750 I would average these runs-- 00:21:09.750 --> 00:21:12.630 runs 5, 6, 7, and 8-- 00:21:12.630 --> 00:21:16.230 average runs 1, 2, 3, and 4, take the difference, 00:21:16.230 --> 00:21:20.670 and that's my estimate of the C effect. 00:21:20.670 --> 00:21:22.760 That makes good intuitive sense. 00:21:25.980 --> 00:21:31.570 Now, that's the contrast that I would do, again, 00:21:31.570 --> 00:21:36.700 for C. If I also want to form contrasts 00:21:36.700 --> 00:21:39.400 for the other coefficients, it's exactly the same thing, 00:21:39.400 --> 00:21:43.330 but the table is not quite oriented 00:21:43.330 --> 00:21:45.020 to make it leap out at me. 00:21:45.020 --> 00:21:46.900 But essentially, for the A contrast, 00:21:46.900 --> 00:21:52.570 I'm just forming those for my A, AB. 00:21:52.570 --> 00:21:56.650 That's my plus 1's, all my pluses, and all my minuses, 00:21:56.650 --> 00:21:58.240 forming that as my contrast. 00:21:58.240 --> 00:22:03.130 I can very quickly then estimate that main effect 00:22:03.130 --> 00:22:06.050 using this big column. 00:22:06.050 --> 00:22:10.510 I can also estimate my interaction terms. 00:22:10.510 --> 00:22:13.480 We already talked about an AB interaction. 00:22:13.480 --> 00:22:14.950 If I have three model-- 00:22:14.950 --> 00:22:20.470 or three factors, I can have three-way interactions where, 00:22:20.470 --> 00:22:24.460 depending on all of the settings of A, B, and C, 00:22:24.460 --> 00:22:27.430 I might get a little boost up or down, 00:22:27.430 --> 00:22:30.760 different than I would just from A, B, C, 00:22:30.760 --> 00:22:32.320 and even different from the boost 00:22:32.320 --> 00:22:35.200 I got by having A and B in combination. 00:22:35.200 --> 00:22:38.440 I can have these subtle three-way interactions, 00:22:38.440 --> 00:22:43.180 and you can form the contrast and estimate that effect 00:22:43.180 --> 00:22:48.550 in a similar fashion using the ABC interaction column, 00:22:48.550 --> 00:22:51.640 and form the contrast using that. 00:22:51.640 --> 00:22:53.710 Two steps to get to here-- 00:22:53.710 --> 00:22:59.150 one step is, how do I generate this column? 00:22:59.150 --> 00:23:02.320 And then the second step is then using that column 00:23:02.320 --> 00:23:06.710 to form an estimate of the interaction effect. 00:23:06.710 --> 00:23:10.470 So how do we get to ABC column? 00:23:10.470 --> 00:23:13.140 By extension of what I told you about the AB column-- 00:23:13.140 --> 00:23:14.220 how do you generate that? 00:23:17.040 --> 00:23:18.040 AUDIENCE: Just multiply. 00:23:18.040 --> 00:23:19.082 PROFESSOR: Just multiply. 00:23:19.082 --> 00:23:21.160 It's kind of this repeated multiplication. 00:23:21.160 --> 00:23:24.025 I just take the A, B, and C columns 00:23:24.025 --> 00:23:26.940 and pointwise multiply out those factors. 00:23:26.940 --> 00:23:29.560 So minus 1 minus 1 minus 1 gives me-- 00:23:29.560 --> 00:23:32.680 I overwrote it-- there we go-- 00:23:32.680 --> 00:23:33.790 gives me a minus 1. 00:23:33.790 --> 00:23:36.830 And I similarly do that for all of the columns. 00:23:36.830 --> 00:23:40.330 Once I have that, then doing the second step 00:23:40.330 --> 00:23:43.435 is pretty simple to estimate that contrast. 00:23:47.220 --> 00:23:49.430 Once we've got the contrast, just the differences 00:23:49.430 --> 00:23:51.830 in the columns, then I do have to do a little bit 00:23:51.830 --> 00:23:54.170 of that normalization. 00:23:54.170 --> 00:23:58.610 Actually, I probably am not completely consistent here. 00:23:58.610 --> 00:24:02.180 The contrast is just the differences between the points. 00:24:02.180 --> 00:24:04.715 I then have to do the average over the number of points. 00:24:04.715 --> 00:24:09.080 Oops-- well, two things-- 00:24:09.080 --> 00:24:12.110 to form the estimate of the overall effect, 00:24:12.110 --> 00:24:15.380 first off, I have to divide by the number of points 00:24:15.380 --> 00:24:20.270 I had in each of those contrasts, 00:24:20.270 --> 00:24:23.810 which is basically just 2 to the k minus 1. 00:24:23.810 --> 00:24:25.730 When we had that cube with eight points, 00:24:25.730 --> 00:24:30.350 it was 2 to the third 8 divided by 2, 00:24:30.350 --> 00:24:33.500 because I'm forming half of the points in one contrast, half 00:24:33.500 --> 00:24:35.640 of the points in the other. 00:24:35.640 --> 00:24:39.020 And then this n, which we haven't talked about very much, 00:24:39.020 --> 00:24:40.490 is just the replicates. 00:24:40.490 --> 00:24:44.390 I can do multiple runs at each experimental point, 00:24:44.390 --> 00:24:48.080 so I can also average those in. 00:24:48.080 --> 00:24:51.560 So it's also possible that we've got more than one 00:24:51.560 --> 00:24:54.990 run at every experimental combination point. 00:24:54.990 --> 00:24:58.070 So that's how we can estimate those effects. 00:24:58.070 --> 00:25:02.390 And then, once we have that, for example, here, that's 00:25:02.390 --> 00:25:04.640 how we get the-- in the 2 to the third, 00:25:04.640 --> 00:25:12.200 we get a 1/4 factor that we saw earlier for the three factor 00:25:12.200 --> 00:25:15.260 cube in three-dimensional space as an estimate 00:25:15.260 --> 00:25:16.370 for the overall effect. 00:25:19.720 --> 00:25:25.780 OK, now, another point is here I'm 00:25:25.780 --> 00:25:30.160 talking about estimating our model using these contrasts 00:25:30.160 --> 00:25:33.190 and factor effects. 00:25:33.190 --> 00:25:35.830 Another approach, and probably the approach 00:25:35.830 --> 00:25:40.270 that you would end up using more often if you actually 00:25:40.270 --> 00:25:45.670 had computers handy, is to do a regression. 00:25:45.670 --> 00:25:50.770 Take your data, put it in, solve the regression 00:25:50.770 --> 00:25:53.680 equations, and regress to get your estimate 00:25:53.680 --> 00:25:55.510 of your coefficients. 00:25:55.510 --> 00:25:59.620 And essentially, these big tables that we talked about, 00:25:59.620 --> 00:26:04.180 these combinations are still critical and very, very useful, 00:26:04.180 --> 00:26:08.980 because that's basically how you form the x matrix that 00:26:08.980 --> 00:26:14.740 describes the combinations of all of your factor levels 00:26:14.740 --> 00:26:20.086 that you feed into the regression. 00:26:20.086 --> 00:26:24.250 In fact, your coded values from minus 1 to plus 1, 00:26:24.250 --> 00:26:27.130 this is your x matrix right there, 00:26:27.130 --> 00:26:36.580 because what you're trying to do is estimate a beta 0, a beta 1, 00:26:36.580 --> 00:26:39.100 and a beta 2, and a beta 3-- 00:26:39.100 --> 00:26:43.720 oops-- beta 3, and then also trying to estimate these 00:26:43.720 --> 00:26:44.920 interaction terms-- 00:26:44.920 --> 00:26:52.270 a beta 1, 3; a beta 1, 3; beta 2, 3; and even a beta 1, 2, 3. 00:26:52.270 --> 00:26:55.210 And so you're basically fictitiously creating 00:26:55.210 --> 00:27:02.740 these additional variables for the product of x1, x2, x3 00:27:02.740 --> 00:27:04.880 in your regression. 00:27:04.880 --> 00:27:08.470 So you need this table, even if you're doing regression, 00:27:08.470 --> 00:27:10.870 in order to be able to set up the regression 00:27:10.870 --> 00:27:13.030 if you want to also include these interaction 00:27:13.030 --> 00:27:15.610 terms in your regression model. 00:27:15.610 --> 00:27:20.440 You have to create fictitious columns of data 00:27:20.440 --> 00:27:25.390 based on those interactions, the product of your three input 00:27:25.390 --> 00:27:26.110 settings. 00:27:26.110 --> 00:27:29.650 And this is basically telling you exactly what those were. 00:27:29.650 --> 00:27:32.590 So this is essentially showing that same thing 00:27:32.590 --> 00:27:34.310 I just described in words. 00:27:34.310 --> 00:27:39.250 What we're talking about here is basically setting up 00:27:39.250 --> 00:27:41.200 to do the pseudo-inverse when I've 00:27:41.200 --> 00:27:45.580 got my data, with my x matrix being my setup of my input 00:27:45.580 --> 00:27:50.320 conditions, and then y being my output, 00:27:50.320 --> 00:27:53.980 and then beta being my vector of coefficients 00:27:53.980 --> 00:27:58.990 with both the mean, the main effects, and the interaction 00:27:58.990 --> 00:28:00.710 effects. 00:28:00.710 --> 00:28:05.230 So I would have to create the column of the x1, x2 product 00:28:05.230 --> 00:28:07.610 in order to be able to do that. 00:28:07.610 --> 00:28:10.240 And then you can see again here the relationship 00:28:10.240 --> 00:28:15.220 between the contrasts idea and the regression idea 00:28:15.220 --> 00:28:18.430 that comes out. 00:28:18.430 --> 00:28:20.410 OK, relatively clear? 00:28:20.410 --> 00:28:22.880 Questions on that? 00:28:22.880 --> 00:28:25.180 OK. 00:28:25.180 --> 00:28:29.490 So now we've estimated coefficients. 00:28:29.490 --> 00:28:32.760 We've estimated effects, either in the regression 00:28:32.760 --> 00:28:37.230 or the contrast approach. 00:28:37.230 --> 00:28:38.700 A key question is-- 00:28:38.700 --> 00:28:39.810 you do that. 00:28:39.810 --> 00:28:41.560 You get a value for A-- 00:28:41.560 --> 00:28:45.130 the A effect that's non-zero. 00:28:45.130 --> 00:28:49.050 Do you think it's significant or not? 00:28:49.050 --> 00:28:53.910 Is it real or not, or would you have observed that amount of A 00:28:53.910 --> 00:28:57.450 by chance alone, given random noise in your experiment? 00:28:57.450 --> 00:29:00.240 This is exactly back to the ANOVA question 00:29:00.240 --> 00:29:02.610 that we were asking earlier. 00:29:02.610 --> 00:29:06.630 I would like to know the significance of these effects. 00:29:06.630 --> 00:29:11.010 Is it large enough to give me 90% confidence 00:29:11.010 --> 00:29:14.100 or 95% confidence that it's real, that it wouldn't 00:29:14.100 --> 00:29:16.090 have occurred by chance alone? 00:29:16.090 --> 00:29:18.720 And so essentially, what we want to do 00:29:18.720 --> 00:29:24.360 is run the ANOVA for each of our effects. 00:29:24.360 --> 00:29:27.180 If I have three factors, I'd like 00:29:27.180 --> 00:29:30.390 to know, is the main effect associated with factor two-- 00:29:30.390 --> 00:29:32.370 is that significant? 00:29:32.370 --> 00:29:35.640 I can also do that for interaction effects. 00:29:35.640 --> 00:29:39.690 And very often, you might see large main effects, 00:29:39.690 --> 00:29:41.940 but you want to ask the question, 00:29:41.940 --> 00:29:44.820 is there really an interaction between these two? 00:29:44.820 --> 00:29:47.490 Can I think of them as separable coefficients 00:29:47.490 --> 00:29:49.410 or separable knobs on my equipment, 00:29:49.410 --> 00:29:52.050 or do I have to think of them as highly coupled? 00:29:52.050 --> 00:29:54.420 I'd like to know the significance of the interaction 00:29:54.420 --> 00:29:55.630 effect. 00:29:55.630 --> 00:30:00.240 So we essentially just use the ANOVA approach fairly directly, 00:30:00.240 --> 00:30:02.850 where what we're doing is forming 00:30:02.850 --> 00:30:06.360 an estimate of the total deviations associated 00:30:06.360 --> 00:30:11.190 with that effect, the sum of squared deviations, which turns 00:30:11.190 --> 00:30:15.450 out to be the contrast squared, and then 00:30:15.450 --> 00:30:20.970 divided by the n over 2k factor. 00:30:20.970 --> 00:30:22.560 There's a homework problem set where 00:30:22.560 --> 00:30:26.220 you get to help derive why that it's true, 00:30:26.220 --> 00:30:28.050 or convince yourself why that is true. 00:30:31.003 --> 00:30:32.670 You're looking forward to it, I can see. 00:30:40.210 --> 00:30:42.270 But what that lets us do is look again 00:30:42.270 --> 00:30:46.620 at the total amount of magnitude of sum of squared deviations 00:30:46.620 --> 00:30:49.440 around a mean associated with that effect, 00:30:49.440 --> 00:30:51.270 and then look at that in comparison 00:30:51.270 --> 00:30:54.060 to some estimate of the random noise, 00:30:54.060 --> 00:30:58.530 and do an F-test, just like we normally do with ANOVA. 00:30:58.530 --> 00:31:00.750 So you can start to, again, do that. 00:31:00.750 --> 00:31:04.470 If you've got two effects or two factors, 00:31:04.470 --> 00:31:07.650 you would look for the question, is there 00:31:07.650 --> 00:31:10.920 something significant in the A effect, in the B effect, 00:31:10.920 --> 00:31:12.720 and in the interaction? 00:31:12.720 --> 00:31:16.530 And then the remaining sum of squared deviations 00:31:16.530 --> 00:31:19.830 or the residual gives you your error estimate 00:31:19.830 --> 00:31:23.070 that you use in the denominator, and that's 00:31:23.070 --> 00:31:27.870 how you decompose the total observed variations. 00:31:27.870 --> 00:31:31.080 So then you can form that into an ANOVA table, 00:31:31.080 --> 00:31:32.820 again, looking at the main effects 00:31:32.820 --> 00:31:37.800 A and B, the interaction effect, and an estimate of the error. 00:31:37.800 --> 00:31:40.410 Again, we get this little funky sine. 00:31:40.410 --> 00:31:42.930 That's supposed to just be a multiplication. 00:31:42.930 --> 00:31:44.130 That's just a little dot-- 00:31:46.830 --> 00:31:50.540 where, again, you're forming estimates of the mean squares, 00:31:50.540 --> 00:31:52.140 and then ratios of those. 00:31:52.140 --> 00:31:55.280 And then you do your F-test to see 00:31:55.280 --> 00:31:57.690 if the F-is higher than that. 00:31:57.690 --> 00:32:01.650 So this is one way of looking at it, in terms of contrast. 00:32:01.650 --> 00:32:06.230 If, instead of forming contrast, you had used the regression 00:32:06.230 --> 00:32:12.980 and come up with regression coefficients associated 00:32:12.980 --> 00:32:16.490 with each of the terms, you can similarly 00:32:16.490 --> 00:32:21.420 form the sum of squares and the mean squares 00:32:21.420 --> 00:32:23.490 from that perspective. 00:32:23.490 --> 00:32:27.360 And in the regression form, by the way, that turns out nicely, 00:32:27.360 --> 00:32:31.200 because it's-- also gives you the chance to not subtract off 00:32:31.200 --> 00:32:33.600 the mean from everything, but actually estimate whether 00:32:33.600 --> 00:32:37.500 the mean may be significantly different than 0. 00:32:37.500 --> 00:32:40.770 But it's essentially the same thing. 00:32:40.770 --> 00:32:43.710 OK, so what I've tried to convey to 00:32:43.710 --> 00:32:48.720 you here is that you can use ANOVA to estimate-- 00:32:48.720 --> 00:32:50.430 that's a tool we already know-- 00:32:50.430 --> 00:32:53.850 to estimate whether the effects are significant 00:32:53.850 --> 00:32:55.650 or not, and whether you should include 00:32:55.650 --> 00:32:58.090 that term in your model. 00:32:58.090 --> 00:33:02.550 If you don't have enough evidence that that value-- 00:33:02.550 --> 00:33:05.790 the coefficient associated with x1-- 00:33:05.790 --> 00:33:08.970 might be non-zero, you're probably better off not 00:33:08.970 --> 00:33:11.340 including it in your model-- 00:33:11.340 --> 00:33:13.450 in which case, those-- 00:33:13.450 --> 00:33:15.660 any little squiggles that might be there, 00:33:15.660 --> 00:33:18.020 but you're not confident really are there 00:33:18.020 --> 00:33:20.730 contribute to your error term. 00:33:20.730 --> 00:33:23.280 And you treat them basically as noise, rather 00:33:23.280 --> 00:33:25.720 than trying to fit them. 00:33:25.720 --> 00:33:29.710 OK, so let's try this out for an example. 00:33:29.710 --> 00:33:32.680 This is one of the sets of data that we looked 00:33:32.680 --> 00:33:35.240 at very early in the term. 00:33:35.240 --> 00:33:36.520 It's the break forming. 00:33:36.520 --> 00:33:40.870 Remember, the press-- we have either aluminum 00:33:40.870 --> 00:33:44.260 or stainless steel, and we've got an angled press, 00:33:44.260 --> 00:33:46.420 and we're pressing it down into some depth 00:33:46.420 --> 00:33:50.440 and trying to bend that piece of sheet metal. 00:33:50.440 --> 00:33:53.020 And that was actually a designed experiment. 00:33:53.020 --> 00:33:56.530 In fact, the two factors that we were looking at here 00:33:56.530 --> 00:34:02.620 is how deep the punch goes in and what 00:34:02.620 --> 00:34:04.660 the material type was-- 00:34:04.660 --> 00:34:06.160 either aluminum or steel. 00:34:06.160 --> 00:34:10.780 And this is just identifying the factor levels 00:34:10.780 --> 00:34:15.080 that I'm going to associate with these terms. 00:34:15.080 --> 00:34:16.810 Now, this is actually an interesting one, 00:34:16.810 --> 00:34:21.889 because this is a discrete factor level. 00:34:21.889 --> 00:34:24.370 Well, this is a more of a continuous factor level. 00:34:27.499 --> 00:34:30.000 And you'll catch a little bit later-- 00:34:30.000 --> 00:34:31.699 there's something a little bit odd, 00:34:31.699 --> 00:34:35.929 where we talk about the zero factor sitting 00:34:35.929 --> 00:34:37.639 between aluminum and steel. 00:34:37.639 --> 00:34:39.840 That actually doesn't make any sense. 00:34:39.840 --> 00:34:43.100 But I can talk about perhaps the zero factor setting 00:34:43.100 --> 00:34:47.780 of 0.45 inches halfway between the low 00:34:47.780 --> 00:34:50.239 and the high of a continuous parameter. 00:34:50.239 --> 00:34:52.370 So I like this because it's mixing or showing 00:34:52.370 --> 00:34:55.370 that the same tools and technology mostly 00:34:55.370 --> 00:34:58.970 work for both discrete decisions and design of experiments, 00:34:58.970 --> 00:35:00.590 and continuous ones. 00:35:00.590 --> 00:35:05.350 And we'll see where it doesn't a little bit later. 00:35:05.350 --> 00:35:08.670 So let's try out a very simple designed experiment, a 2 00:35:08.670 --> 00:35:15.810 to the 2 factor where the final angle after removing the punch 00:35:15.810 --> 00:35:18.690 is our output. 00:35:18.690 --> 00:35:20.190 This is the actual data. 00:35:20.190 --> 00:35:23.220 What we've got is, for each of the level settings, 00:35:23.220 --> 00:35:24.990 we've got replicates. 00:35:24.990 --> 00:35:31.200 So I'm doing 10 replicates at each test 00:35:31.200 --> 00:35:35.310 setting of the high and low combination for x1 and x2. 00:35:35.310 --> 00:35:37.020 So I've got four different corners, 00:35:37.020 --> 00:35:40.140 four different combinations, 10 replicates. 00:35:40.140 --> 00:35:44.250 And here's the data for that. 00:35:44.250 --> 00:35:47.470 By the way-- we'll come back to this a little bit later-- 00:35:47.470 --> 00:35:53.190 but if this is run number and if those are done in time, 00:35:53.190 --> 00:35:55.290 there's actually something a little risky 00:35:55.290 --> 00:35:56.830 in this experimental design. 00:35:59.520 --> 00:36:01.800 This is foreshadowing to coming back 00:36:01.800 --> 00:36:05.610 to additional issues of nasty things 00:36:05.610 --> 00:36:08.580 you've got to keep in mind when you do design experiments, 00:36:08.580 --> 00:36:13.410 but what if there was some time degradation, some time where 00:36:13.410 --> 00:36:15.520 in the tool? 00:36:15.520 --> 00:36:17.010 The problem is you wouldn't really 00:36:17.010 --> 00:36:20.700 be able to distinguish a time effect from a change 00:36:20.700 --> 00:36:22.990 in the factor setting. 00:36:22.990 --> 00:36:24.720 So one approach that you might-- 00:36:24.720 --> 00:36:27.600 if you can cheaply do it-- 00:36:27.600 --> 00:36:30.750 here you might actually have had to make maybe 00:36:30.750 --> 00:36:32.880 some kind of machine setup change, 00:36:32.880 --> 00:36:35.760 and you wouldn't want to do this. 00:36:35.760 --> 00:36:39.270 But you might randomize the order of your runs 00:36:39.270 --> 00:36:45.070 in time to block against the effective time. 00:36:45.070 --> 00:36:46.500 And we'll come back to that. 00:36:46.500 --> 00:36:48.870 But here, it's a simple-- 00:36:48.870 --> 00:36:53.520 I do all of my low-low combinations first, 00:36:53.520 --> 00:36:59.340 then I did my low-high, then my high-low, than my high-high. 00:36:59.340 --> 00:37:01.170 So that's our data. 00:37:01.170 --> 00:37:03.570 Now, what we're doing here, first off, 00:37:03.570 --> 00:37:06.540 is I'm just going to form the mean-- 00:37:06.540 --> 00:37:09.840 I take that the mean of those 10 points, and that's 00:37:09.840 --> 00:37:15.260 going to be my y bar at each of my level settings. 00:37:15.260 --> 00:37:17.970 So I'm reducing it down from the 10 replicates 00:37:17.970 --> 00:37:21.030 down to four means. 00:37:21.030 --> 00:37:24.690 So this is my mean at each of those four 00:37:24.690 --> 00:37:26.760 different conditions. 00:37:26.760 --> 00:37:32.730 And now I have my output, my y vector-- 00:37:32.730 --> 00:37:38.790 setting up for a regression-- as being the observed means. 00:37:38.790 --> 00:37:42.150 What's my x matrix setting up for my regression? 00:37:42.150 --> 00:37:50.220 Oops-- it's based on the level settings. 00:37:50.220 --> 00:37:53.460 But notice, we also add in that one factor. 00:37:53.460 --> 00:37:56.430 That one factor is basically giving us-- 00:37:56.430 --> 00:37:58.260 when I multiply out my matrix, that's 00:37:58.260 --> 00:38:01.990 corresponding to my beta 0, my overall mean, 00:38:01.990 --> 00:38:07.230 so that what that's going to do is be able to add all of those 00:38:07.230 --> 00:38:10.320 up, divide by 4, and give me an estimate 00:38:10.320 --> 00:38:15.310 of the overall grand mean of my output. 00:38:15.310 --> 00:38:19.840 And we also create the x1, x2 cross factor 00:38:19.840 --> 00:38:23.750 in order to be able to estimate and interaction term. 00:38:23.750 --> 00:38:27.670 So overall then, this here is my x matrix. 00:38:27.670 --> 00:38:32.380 I can then solve it. 00:38:32.380 --> 00:38:34.930 Now, notice we're using the direct inverse, and not 00:38:34.930 --> 00:38:38.010 the pseudo-inverse. 00:38:38.010 --> 00:38:38.510 Why? 00:38:46.610 --> 00:38:51.890 Remember, ultimately, we've got y equals beta x. 00:38:51.890 --> 00:38:54.020 That's the model we're trying to form-- 00:38:54.020 --> 00:38:58.070 beta 0 plus beta 1, beta 2 plus an interaction term. 00:39:02.370 --> 00:39:04.290 In this approach, how many data points 00:39:04.290 --> 00:39:09.280 do I have fitting into this regression? 00:39:09.280 --> 00:39:12.170 My y vector was four data points, 00:39:12.170 --> 00:39:13.420 because it was just the means. 00:39:16.220 --> 00:39:18.730 How many coefficients? 00:39:18.730 --> 00:39:23.500 Four-- I can exactly fit my four coefficients, 00:39:23.500 --> 00:39:25.850 and that's why I can do the direct inverse. 00:39:25.850 --> 00:39:28.150 And when I do that, it directly pops out 00:39:28.150 --> 00:39:31.030 these values from my data coefficients, which feed 00:39:31.030 --> 00:39:34.390 or give me my regression model. 00:39:34.390 --> 00:39:36.040 What's epsilon in this model? 00:39:42.300 --> 00:39:44.580 0-- it's 0. 00:39:44.580 --> 00:39:46.950 I exactly fit my four data points 00:39:46.950 --> 00:39:49.470 with my four coefficients. 00:39:49.470 --> 00:39:54.840 I don't even have any extra data to estimate epsilon, 00:39:54.840 --> 00:39:58.870 if I just use the mean values. 00:39:58.870 --> 00:40:01.170 However, if I go back and expand out-- 00:40:01.170 --> 00:40:05.220 realize I had 10 replicates at each of those mean values, 00:40:05.220 --> 00:40:06.960 that's one form-- 00:40:06.960 --> 00:40:12.040 or one source of information about pure replication error-- 00:40:12.040 --> 00:40:17.560 that is to say, noise factors in the data. 00:40:17.560 --> 00:40:19.440 So in this approach, where I just 00:40:19.440 --> 00:40:20.910 use the mean and the regression, I 00:40:20.910 --> 00:40:22.920 don't have an estimate for epsilon. 00:40:22.920 --> 00:40:25.140 I would have to look back and form 00:40:25.140 --> 00:40:29.430 a separate pool of estimate across my four 00:40:29.430 --> 00:40:38.200 experimental data points for the noise variance in the system. 00:40:38.200 --> 00:40:40.540 OK, let's see. 00:40:40.540 --> 00:40:44.180 What else do I want to say here? 00:40:44.180 --> 00:40:45.770 I thought I had another example. 00:40:45.770 --> 00:40:48.020 OK, we'll come back to that. 00:40:48.020 --> 00:40:53.150 So once I've got the model, this is essentially the model 00:40:53.150 --> 00:40:57.510 that I have of the true underlying process-- 00:40:57.510 --> 00:41:01.250 again, first-order, linear, plus an interaction-- 00:41:01.250 --> 00:41:04.040 plus maybe I have lingering in my head, 00:41:04.040 --> 00:41:08.270 there might still be some higher order quadratic dependence, 00:41:08.270 --> 00:41:10.730 for example, on punch depth. 00:41:10.730 --> 00:41:14.670 I haven't really sampled enough to be able to estimate that. 00:41:14.670 --> 00:41:16.460 And I also have some noise. 00:41:16.460 --> 00:41:21.470 The model that I have fit right here, my y hat estimate, 00:41:21.470 --> 00:41:23.360 includes estimates-- 00:41:23.360 --> 00:41:26.290 I guess I should say estimates here-- 00:41:26.290 --> 00:41:29.710 for these different terms. 00:41:29.710 --> 00:41:32.320 And what we're basically saying is 00:41:32.320 --> 00:41:35.530 we're lumping in together all of the errors that 00:41:35.530 --> 00:41:40.070 might be lurking as two effects together. 00:41:40.070 --> 00:41:43.390 One is random noise and the other 00:41:43.390 --> 00:41:45.865 is neglected higher order terms. 00:41:48.460 --> 00:41:50.650 That's our acronym for higher order terms. 00:41:54.950 --> 00:41:55.820 Question-- 00:41:55.820 --> 00:41:58.310 AUDIENCE: What if you put the order of [INAUDIBLE]?? 00:41:58.310 --> 00:41:59.970 Then you can actually get epsilon 0. 00:42:02.945 --> 00:42:05.080 PROFESSOR: Right. 00:42:05.080 --> 00:42:07.580 So the question was, what if you add additional higher order 00:42:07.580 --> 00:42:11.880 terms to drive epsilon to 0? 00:42:11.880 --> 00:42:15.540 This is a very good point, because if I 00:42:15.540 --> 00:42:18.330 don't have any replicates, I could always 00:42:18.330 --> 00:42:20.640 add model terms until I finally get 00:42:20.640 --> 00:42:23.580 to where I have a perfect fit-- 00:42:23.580 --> 00:42:26.640 just like we saw with the four data points and four-- 00:42:26.640 --> 00:42:30.300 in which case, my residual then is 0. 00:42:30.300 --> 00:42:32.400 But I've added model terms. 00:42:32.400 --> 00:42:34.200 But I also have no way, in that case, 00:42:34.200 --> 00:42:38.040 of knowing whether the model terms are significant or not. 00:42:38.040 --> 00:42:40.680 If I have replicates, on the other hand, 00:42:40.680 --> 00:42:43.290 then we will almost never-- 00:42:43.290 --> 00:42:46.410 if there's any noise in the replicates, where 00:42:46.410 --> 00:42:48.930 the replicates do not lie exactly 00:42:48.930 --> 00:42:52.620 on top of each other for every rerun at the same condition, 00:42:52.620 --> 00:42:56.430 then there's no way to drive the residuals to 0, because I will 00:42:56.430 --> 00:42:59.490 have pure process error at least, 00:42:59.490 --> 00:43:02.250 even if I've added all the model terms I need to drive 00:43:02.250 --> 00:43:05.670 model error, the HOT, to 0. 00:43:05.670 --> 00:43:09.070 So we've got actually these two things going on. 00:43:09.070 --> 00:43:18.120 We've got two key sources of residuals, lack of perfection 00:43:18.120 --> 00:43:21.480 in our model that we want to talk about how 00:43:21.480 --> 00:43:23.460 we distinguish and detect-- 00:43:23.460 --> 00:43:29.820 because one of those is whether we have an adequate model form. 00:43:29.820 --> 00:43:33.330 Do I have the right terms and enough of the terms in there? 00:43:33.330 --> 00:43:38.440 Or is there just replication variance, noise-- 00:43:38.440 --> 00:43:40.320 which I can't fit? 00:43:40.320 --> 00:43:43.410 All I can do is estimate the stigma associated 00:43:43.410 --> 00:43:46.560 with that replicate noise. 00:43:46.560 --> 00:43:54.570 That's the key question that we want to get to. 00:43:54.570 --> 00:43:57.690 Before we get to breaking down the residuals, 00:43:57.690 --> 00:43:59.250 one thing that you should always do 00:43:59.250 --> 00:44:03.180 after you fit your model is actually look at the residuals 00:44:03.180 --> 00:44:07.230 to see if there are trends qualitatively that 00:44:07.230 --> 00:44:09.390 may be missing, because that will often give you 00:44:09.390 --> 00:44:15.240 a lot of insight into whether it's a model structure problem 00:44:15.240 --> 00:44:16.090 or not. 00:44:16.090 --> 00:44:18.570 And so for this data, what we've actually done here 00:44:18.570 --> 00:44:22.260 is look at the residuals associated with the 10 00:44:22.260 --> 00:44:24.480 replicates that we've got. 00:44:24.480 --> 00:44:28.350 And what you'll see is, first off, the mean of your residuals 00:44:28.350 --> 00:44:29.310 had better be 0. 00:44:29.310 --> 00:44:33.810 Otherwise, you've actually got a screw up somewhere in your fit, 00:44:33.810 --> 00:44:35.790 just mechanically. 00:44:35.790 --> 00:44:39.000 But what you would like to see is, 00:44:39.000 --> 00:44:42.510 essentially, the residuals ought to not 00:44:42.510 --> 00:44:47.980 depend on the level setting or the experimental data point. 00:44:47.980 --> 00:44:50.310 Especially with continuous parameters, 00:44:50.310 --> 00:44:54.270 if what you see is the residuals for small values are very tiny 00:44:54.270 --> 00:44:57.930 and the residuals for big values are really, really large, 00:44:57.930 --> 00:45:00.480 that actually violates some of the assumptions we've 00:45:00.480 --> 00:45:02.850 made in the model fitting. 00:45:02.850 --> 00:45:06.120 And you might need to go and do a transformation of your data-- 00:45:06.120 --> 00:45:09.930 perhaps something like a logarithmic transformation 00:45:09.930 --> 00:45:12.600 or some other transformation of your y output data-- 00:45:12.600 --> 00:45:18.840 because your residuals are not all n 0 sigma squared, 00:45:18.840 --> 00:45:19.920 as we assume-- 00:45:19.920 --> 00:45:23.690 the same variance for each of your data points. 00:45:23.690 --> 00:45:25.548 Question-- 00:45:25.548 --> 00:45:29.900 AUDIENCE: [INAUDIBLE] mentioned if we have four thumbs 00:45:29.900 --> 00:45:32.610 and four data points to error, thumb moves to 0. 00:45:32.610 --> 00:45:36.110 But isn't that supposing the x1 and x2, 00:45:36.110 --> 00:45:43.870 that we've chosen our only two factors which [INAUDIBLE].. 00:45:43.870 --> 00:45:47.270 We could have [INAUDIBLE] and x3 [INAUDIBLE].. 00:45:47.270 --> 00:45:48.060 PROFESSOR: Yes. 00:45:48.060 --> 00:45:51.130 AUDIENCE: In which case you're still [INAUDIBLE] 00:45:51.130 --> 00:45:53.355 and it should still be pretty significant. 00:45:53.355 --> 00:45:53.980 PROFESSOR: Yes. 00:45:53.980 --> 00:45:57.020 So the question, for folks in Singapore 00:45:57.020 --> 00:46:00.590 who may not have heard that, was, 00:46:00.590 --> 00:46:06.200 aren't there other possible sources of residual or error-- 00:46:06.200 --> 00:46:08.870 including, for example other factors, 00:46:08.870 --> 00:46:12.110 like an x3 factor that we didn't even consider 00:46:12.110 --> 00:46:13.580 is lurking underneath? 00:46:13.580 --> 00:46:14.870 And that's absolutely true. 00:46:14.870 --> 00:46:20.720 Those are all potential additional sources of-- 00:46:20.720 --> 00:46:23.960 I guess you could look at it as either model error or noise. 00:46:23.960 --> 00:46:27.950 What we're doing is treating it all as noise. 00:46:27.950 --> 00:46:30.770 And we'll talk a little bit more about nuisance factors 00:46:30.770 --> 00:46:32.880 and how we might walk against that. 00:46:32.880 --> 00:46:35.150 But essentially, we're lumping all of those things 00:46:35.150 --> 00:46:40.520 into just the sigma squared, unless we're explicitly 00:46:40.520 --> 00:46:42.590 breaking them out, treating them in our model. 00:46:45.280 --> 00:46:46.030 AUDIENCE: Hold on. 00:46:46.030 --> 00:46:47.120 There's a question. 00:46:47.120 --> 00:46:49.545 PROFESSOR: Yes-- question-- 00:46:49.545 --> 00:46:51.920 AUDIENCE: What if I don't have enough data-- for example, 00:46:51.920 --> 00:46:54.460 I want to have my replicate, but I still 00:46:54.460 --> 00:46:56.950 want to estimate the residual? 00:46:56.950 --> 00:47:01.300 Can I just drop the higher orders 00:47:01.300 --> 00:47:04.600 and use the data points to estimate the residual? 00:47:04.600 --> 00:47:05.750 PROFESSOR: Yes. 00:47:05.750 --> 00:47:14.200 So you did, essentially, the special case where, 00:47:14.200 --> 00:47:17.710 if you believe really the higher order term, 00:47:17.710 --> 00:47:21.010 or even the interaction term in our 2-by-2-- 00:47:21.010 --> 00:47:23.560 let's say I really believe that really is not 00:47:23.560 --> 00:47:27.970 any reason for there to be an AB interaction-- 00:47:27.970 --> 00:47:32.950 you can use that as essentially a replicate, 00:47:32.950 --> 00:47:35.110 and not include that in the model term-- 00:47:35.110 --> 00:47:37.960 in which case, then I've got four data points, three model 00:47:37.960 --> 00:47:38.990 terms-- 00:47:38.990 --> 00:47:41.260 beta 0, beta 1, beta 2. 00:47:41.260 --> 00:47:45.670 And I've got the barest bones of additional data 00:47:45.670 --> 00:47:49.030 to be able to form an error estimate, with basically 00:47:49.030 --> 00:47:51.160 treating the interaction as noise, 00:47:51.160 --> 00:47:53.870 because I don't think that it's real. 00:47:53.870 --> 00:47:57.190 And then you can do-- you've got just enough 00:47:57.190 --> 00:48:00.790 to be able to at least ask minimal questions 00:48:00.790 --> 00:48:04.360 about significance. 00:48:04.360 --> 00:48:07.060 One last thing that we always want to do with our residuals 00:48:07.060 --> 00:48:11.410 is do a quick test or check to see, 00:48:11.410 --> 00:48:14.560 is it reasonable to think that these things are normally 00:48:14.560 --> 00:48:15.680 distributed? 00:48:15.680 --> 00:48:19.390 So you can, again, do a qq plot and see 00:48:19.390 --> 00:48:24.670 if there's strong evidence of something strange going on. 00:48:24.670 --> 00:48:30.880 Now, I said here we had to do-- if I used my y 00:48:30.880 --> 00:48:33.370 average as my output, I can do the regression 00:48:33.370 --> 00:48:36.380 and solve directly. 00:48:36.380 --> 00:48:38.830 You could also formulate the regression problem 00:48:38.830 --> 00:48:42.430 using all of the data, and not pre-calculating 00:48:42.430 --> 00:48:47.020 the average for each of my four settings, in which case 00:48:47.020 --> 00:48:51.070 we set up our data little bit larger. 00:48:51.070 --> 00:48:52.510 I still have my-- 00:48:52.510 --> 00:48:54.640 I don't know why it became an eta. 00:48:54.640 --> 00:48:56.050 I guess there's my eta. 00:48:56.050 --> 00:48:57.070 That's still a y. 00:48:57.070 --> 00:48:59.740 This is my y data. 00:48:59.740 --> 00:49:03.160 I still have my x, which has my main effects 00:49:03.160 --> 00:49:05.090 and my interaction. 00:49:05.090 --> 00:49:07.580 But now each of the replicates-- 00:49:07.580 --> 00:49:09.640 for example-- I don't know if you can see it, 00:49:09.640 --> 00:49:11.860 but this is a 1, 1, 1 setting, and that's 00:49:11.860 --> 00:49:14.060 one of those outputs. 00:49:14.060 --> 00:49:19.190 Here's another 1, 1, 1, and the output for that particular run. 00:49:19.190 --> 00:49:21.970 In other words, I have multiple rows in my data. 00:49:21.970 --> 00:49:24.850 I have 10 rows with the same factor settings, 00:49:24.850 --> 00:49:28.600 and each one of which has a different output, y. 00:49:28.600 --> 00:49:32.500 And now I can set up and do my regression. 00:49:32.500 --> 00:49:35.710 Now I have to use the pseudo-inverse, 00:49:35.710 --> 00:49:40.640 because I've got more data than I've got coefficients. 00:49:40.640 --> 00:49:43.000 So I'm doing my least squares fitting 00:49:43.000 --> 00:49:45.700 using my pseudo-inverse. 00:49:45.700 --> 00:49:47.800 And what pops out, if you actually 00:49:47.800 --> 00:49:51.440 do that with this data, is exactly the same data. 00:49:51.440 --> 00:49:54.010 Thank God, right? 00:49:54.010 --> 00:50:00.230 So it's the same estimates of beta as before. 00:50:00.230 --> 00:50:05.200 Now, you might also have directly a residual 00:50:05.200 --> 00:50:10.240 based on the difference between your predicted y-- 00:50:10.240 --> 00:50:15.550 your y hat-- and your actual measured. 00:50:15.550 --> 00:50:17.920 I've got 40 of those values, and I 00:50:17.920 --> 00:50:20.050 can use that as my overall estimate 00:50:20.050 --> 00:50:23.240 for my overall residual in that-- in this case. 00:50:23.240 --> 00:50:27.190 So it sort of pops right out as well, 00:50:27.190 --> 00:50:30.040 and you can form the same residual. 00:50:30.040 --> 00:50:32.440 So if you actually plot this for the response 00:50:32.440 --> 00:50:36.610 surface for our very simple linear model with interaction, 00:50:36.610 --> 00:50:37.720 this is what you see. 00:50:37.720 --> 00:50:40.060 As you change the material axis from your high 00:50:40.060 --> 00:50:44.320 to low in your depth axis, you've got this-- 00:50:44.320 --> 00:50:48.835 looks like mostly a slanted plane kind of effect. 00:50:51.760 --> 00:50:53.200 Oh, let me ask you. 00:50:55.727 --> 00:50:57.310 Just looking at the plot, do you think 00:50:57.310 --> 00:51:01.300 there's a significant effect from factor x1? 00:51:05.690 --> 00:51:07.580 Significant effect from factor x2? 00:51:14.310 --> 00:51:15.815 I see some heads shaking. 00:51:19.000 --> 00:51:20.690 Oh, looks like it's a nice trend-- 00:51:20.690 --> 00:51:25.420 there's lots of movement in the output values. 00:51:25.420 --> 00:51:27.010 Sure-- why not? 00:51:27.010 --> 00:51:32.050 Just looking at this plot, I argue, you can't tell. 00:51:32.050 --> 00:51:36.460 This is just your best fit based on your betas, 00:51:36.460 --> 00:51:38.320 but you don't actually know whether they're 00:51:38.320 --> 00:51:40.690 significant or not, just plotting 00:51:40.690 --> 00:51:42.130 the output of your model. 00:51:45.930 --> 00:51:46.680 Here's why. 00:51:46.680 --> 00:51:49.620 For example, if I were to tell you, OK, 00:51:49.620 --> 00:51:53.220 at this combination of depth and material, at the plus 1, 00:51:53.220 --> 00:51:57.360 plus 1, I actually plotted the 10 different values you saw, 00:51:57.360 --> 00:52:00.430 and they all looked like this. 00:52:00.430 --> 00:52:03.870 And if I plotted the low-low and they all look like this, 00:52:03.870 --> 00:52:07.740 now you'd start to say, yeah, my model output-- 00:52:07.740 --> 00:52:10.020 there's a big separation between those two. 00:52:10.020 --> 00:52:12.130 I think that's significant. 00:52:12.130 --> 00:52:14.610 On the other hand, if I plotted and showed you 00:52:14.610 --> 00:52:18.330 a scatter around here that was huge 00:52:18.330 --> 00:52:20.220 and a scatter of your actual data 00:52:20.220 --> 00:52:24.420 points around this condition that was also huge, 00:52:24.420 --> 00:52:27.340 and there was big overlap between the two, 00:52:27.340 --> 00:52:29.070 now you'd be a little more leery, right? 00:52:29.070 --> 00:52:31.800 You'd say, well, that was my best bet, 00:52:31.800 --> 00:52:35.590 but I'm not sure that they're actually significant. 00:52:35.590 --> 00:52:39.180 So this highlights, again, you've got to use the ANOVA 00:52:39.180 --> 00:52:43.620 as well to be able to decide, is there actually 00:52:43.620 --> 00:52:46.650 significance to the model coefficient 00:52:46.650 --> 00:52:48.950 that we were looking at? 00:52:48.950 --> 00:52:51.660 It's just another picture. 00:52:51.660 --> 00:52:53.720 This is looking a little bit at the side. 00:52:53.720 --> 00:52:57.080 This is pointing out, if I believe that the terms are 00:52:57.080 --> 00:52:59.880 significant, you could ask the question, 00:52:59.880 --> 00:53:02.250 is there an interaction effect? 00:53:02.250 --> 00:53:08.390 It looks like there's not quite parallelism between these two 00:53:08.390 --> 00:53:09.110 lines. 00:53:09.110 --> 00:53:11.930 There may be an AB interaction, but I'd also 00:53:11.930 --> 00:53:15.660 need to ask the question about significance on that. 00:53:15.660 --> 00:53:17.210 So that's the key question. 00:53:17.210 --> 00:53:21.410 The effects of these terms-- are they significant or not? 00:53:21.410 --> 00:53:23.810 We would do the ANOVA analysis. 00:53:23.810 --> 00:53:27.950 By the way, the ANOVA and the t-test-- 00:53:27.950 --> 00:53:29.370 almost the same. 00:53:29.370 --> 00:53:32.720 In fact, I think another problem on the problem set 00:53:32.720 --> 00:53:35.090 is understanding a little bit the relationship 00:53:35.090 --> 00:53:38.810 between these two ways of looking at the problem. 00:53:38.810 --> 00:53:42.950 If I were to just look at this data and look at the effect 00:53:42.950 --> 00:53:46.610 of depth only on the aluminum-- so I'm just holding-- 00:53:46.610 --> 00:53:50.570 just looking at the aluminum parts-- 00:53:50.570 --> 00:53:53.660 for the depth of 0.6 and the depth of 0.3, 00:53:53.660 --> 00:53:55.440 there's the spread in my data. 00:53:55.440 --> 00:53:57.410 I can look and say, what's the variation 00:53:57.410 --> 00:54:02.450 within the test compared to the variation between tests? 00:54:02.450 --> 00:54:05.360 I could then think of that as my F-test. 00:54:05.360 --> 00:54:11.030 Or I could do a pairwise t-test, or do a t-test looking 00:54:11.030 --> 00:54:16.340 at the difference in the means, given the spread between 00:54:16.340 --> 00:54:17.090 the two-- 00:54:17.090 --> 00:54:19.610 turns out to be the same test-- 00:54:19.610 --> 00:54:22.070 gives the same significance level 00:54:22.070 --> 00:54:24.260 whether you do it with an F or a t. 00:54:24.260 --> 00:54:26.240 But essentially, this is the same question 00:54:26.240 --> 00:54:30.290 I was just describing qualitatively on the 3D plot, 00:54:30.290 --> 00:54:32.870 looking at one factor at a time. 00:54:32.870 --> 00:54:35.930 I can ask the question, do I think the effect of depth 00:54:35.930 --> 00:54:37.520 is significant? 00:54:37.520 --> 00:54:39.620 This is looking at it qualitatively. 00:54:39.620 --> 00:54:44.480 I can go and chug, and do the ANOVA or the t-test machinery, 00:54:44.480 --> 00:54:46.280 but it gives the same idea. 00:54:46.280 --> 00:54:49.320 I can do the same thing for the material effect. 00:54:49.320 --> 00:54:53.930 Say, if I look at aluminum versus the stainless steel, 00:54:53.930 --> 00:54:56.878 is that effect significant? 00:54:56.878 --> 00:54:59.420 By the way, you can go, and chug through, and do all of that. 00:54:59.420 --> 00:55:00.560 What do you think? 00:55:00.560 --> 00:55:03.320 Are these factors significant? 00:55:03.320 --> 00:55:05.370 I have high confidence. 00:55:05.370 --> 00:55:08.480 I'm pretty confident that those are real effects. 00:55:08.480 --> 00:55:10.090 There's a huge spread between these, 00:55:10.090 --> 00:55:13.370 so the intuition is very clear. 00:55:13.370 --> 00:55:19.050 The interaction effect-- that's a little tricky. 00:55:19.050 --> 00:55:20.700 So we can go in-- 00:55:20.700 --> 00:55:24.080 I would actually want to run the ANOVA test in that case. 00:55:24.080 --> 00:55:25.910 Here's my raw data. 00:55:25.910 --> 00:55:28.880 There's the model I fit, again, using the ANOVA 00:55:28.880 --> 00:55:30.980 table that we saw a minute ago. 00:55:30.980 --> 00:55:34.700 I could by hand form that, or I could put it in JMP 00:55:34.700 --> 00:55:37.760 and come out with the output. 00:55:37.760 --> 00:55:40.850 But now here, this is very nice, because I can look and say, 00:55:40.850 --> 00:55:48.410 OK, associated with the mean x1, x2, and the interaction x1, x2, 00:55:48.410 --> 00:55:51.680 what are the mean squares and the F compared 00:55:51.680 --> 00:55:54.800 to our estimate of the noise? 00:55:54.800 --> 00:55:57.260 Remember, we had 40 data points. 00:55:57.260 --> 00:55:59.300 I'm estimating four model coefficients. 00:55:59.300 --> 00:56:03.570 That's leaving 36 replicate degrees of freedom, 00:56:03.570 --> 00:56:07.590 so I have a very good estimate of noise in the system. 00:56:07.590 --> 00:56:12.560 That's the denominator in all of my F ratios. 00:56:12.560 --> 00:56:18.140 And I'm seeing F's from 10,000 down to 77, 00:56:18.140 --> 00:56:23.000 and my F critical for 95% confidence is 4.1. 00:56:23.000 --> 00:56:25.910 So it looks like all of my F's are much bigger than that. 00:56:25.910 --> 00:56:30.200 I have high confidence that the mean is significant, 00:56:30.200 --> 00:56:31.890 the main effects are significant, 00:56:31.890 --> 00:56:33.980 and the interaction is real. 00:56:33.980 --> 00:56:37.400 It is bigger than I would expect, 00:56:37.400 --> 00:56:39.570 given the noise in the process. 00:56:39.570 --> 00:56:41.600 So I would accept all of those model terms. 00:56:46.780 --> 00:56:50.010 So so far, we used the idea of replicates 00:56:50.010 --> 00:56:52.590 to ask the significant question. 00:56:52.590 --> 00:56:55.800 We said our residual could have two sources of error in it. 00:56:55.800 --> 00:57:00.300 One is pure replication noise, but the other thing 00:57:00.300 --> 00:57:03.250 that we haven't really assessed yet is, 00:57:03.250 --> 00:57:06.810 is the model form adequate? 00:57:06.810 --> 00:57:10.410 You might still be nervous that maybe there's 00:57:10.410 --> 00:57:17.030 a quadratic relationship between output and the punch depth. 00:57:17.030 --> 00:57:19.610 With those two data points, the best I could do 00:57:19.610 --> 00:57:21.470 is get a linear model. 00:57:21.470 --> 00:57:23.780 How would you possibly check to see 00:57:23.780 --> 00:57:25.940 if there might be curvature in the model, 00:57:25.940 --> 00:57:27.290 or a quadratic effect? 00:57:30.070 --> 00:57:31.690 Do you have enough data to do that? 00:57:34.983 --> 00:57:36.650 AUDIENCE: There could be a center point. 00:57:36.650 --> 00:57:37.280 PROFESSOR: What? 00:57:37.280 --> 00:57:38.630 AUDIENCE: There could be a center point. 00:57:38.630 --> 00:57:40.338 PROFESSOR: There could be a center point. 00:57:40.338 --> 00:57:42.110 Let's add a center point. 00:57:42.110 --> 00:57:45.305 Let's consider additional experimental center points. 00:57:47.910 --> 00:57:52.110 OK, so what you would do then is run some additional things 00:57:52.110 --> 00:57:55.110 at the center level, the zero setting of those two data 00:57:55.110 --> 00:57:57.630 points, and then look at the deviation 00:57:57.630 --> 00:57:59.910 between what the linear model suggests 00:57:59.910 --> 00:58:04.110 and what the actual data comes up with. 00:58:04.110 --> 00:58:07.890 Now, you've got to ask the significance of the delta. 00:58:07.890 --> 00:58:10.680 Again, your model may not be perfect in predicting 00:58:10.680 --> 00:58:13.830 what that center point is, even if there 00:58:13.830 --> 00:58:16.590 is no curvature in the model, because there's noise also 00:58:16.590 --> 00:58:17.740 in the model. 00:58:17.740 --> 00:58:19.990 So we have to do two things. 00:58:19.990 --> 00:58:23.910 One is we have to estimate the deviation from our simpler 00:58:23.910 --> 00:58:26.010 model-- the linear model-- 00:58:26.010 --> 00:58:30.390 versus the estimation with the quadratic term 00:58:30.390 --> 00:58:33.060 that would go through, say, the center point-- 00:58:33.060 --> 00:58:35.910 go through that additional center point, 00:58:35.910 --> 00:58:38.940 and then compare the size of that to the natural spread 00:58:38.940 --> 00:58:44.730 in our data, and do essentially an F-test or an ANOVA test 00:58:44.730 --> 00:58:47.100 on that quadratic model term-- 00:58:47.100 --> 00:58:49.344 same thing. 00:58:49.344 --> 00:58:51.450 Anything bother you about this picture? 00:58:54.500 --> 00:58:57.970 We can set up and do this center point, and run it. 00:59:03.070 --> 00:59:04.390 So I'm down on the two-all. 00:59:04.390 --> 00:59:06.190 I'm setting the depth factor. 00:59:06.190 --> 00:59:08.230 That's my x to factor, I guess. 00:59:08.230 --> 00:59:11.020 And I put it at 0.45 inches. 00:59:11.020 --> 00:59:13.770 I pick my part. 00:59:13.770 --> 00:59:15.420 I got my aluminum. 00:59:15.420 --> 00:59:16.780 I've got my stainless steel. 00:59:16.780 --> 00:59:19.110 I need my 0-th level. 00:59:22.660 --> 00:59:25.660 There is no center point on that, right? 00:59:25.660 --> 00:59:29.590 So you'd have to actually be very careful. 00:59:29.590 --> 00:59:31.990 If you've got only discrete values that 00:59:31.990 --> 00:59:38.530 don't have some ordering or some continuous degree to them, 00:59:38.530 --> 00:59:41.890 you can't do a center point. 00:59:41.890 --> 00:59:43.630 So what would you do in this case? 00:59:46.977 --> 00:59:48.060 AUDIENCE: Physical model-- 00:59:48.060 --> 00:59:49.268 PROFESSOR: A physical model-- 00:59:51.670 --> 00:59:53.630 it's so complicated, it's hopeless. 00:59:53.630 --> 00:59:56.400 We don't have a physical model. 00:59:56.400 --> 00:59:58.530 You might try a physical model. 00:59:58.530 --> 01:00:00.540 There's something simpler you could do, still 01:00:00.540 --> 01:00:02.340 using design of experiments. 01:00:02.340 --> 01:00:03.270 Yeah? 01:00:03.270 --> 01:00:05.192 AUDIENCE: You just use the easiest one. 01:00:05.192 --> 01:00:06.150 PROFESSOR: What's that? 01:00:06.150 --> 01:00:08.168 AUDIENCE: Just use the simplest one. 01:00:08.168 --> 01:00:09.793 PROFESSOR: Just use the simplest point? 01:00:09.793 --> 01:00:16.580 AUDIENCE: [INAUDIBLE] 01:00:16.580 --> 01:00:18.740 PROFESSOR: Maybe-- I'm thinking, what 01:00:18.740 --> 01:00:20.240 additional experimental point should 01:00:20.240 --> 01:00:25.347 I run to be able to decide if there is additional curvature? 01:00:25.347 --> 01:00:26.930 Remember, we were asking the question, 01:00:26.930 --> 01:00:29.390 is there curvature or a higher order term 01:00:29.390 --> 01:00:33.380 associated with what variable? 01:00:33.380 --> 01:00:35.300 Just punch depth, right? 01:00:35.300 --> 01:00:38.510 So why don't I still pick this, but now I 01:00:38.510 --> 01:00:41.150 can pick data points right here, and I might also 01:00:41.150 --> 01:00:43.350 pick data points right here? 01:00:43.350 --> 01:00:46.070 I don't have to pick central point. 01:00:46.070 --> 01:00:49.640 I could actually pick a couple. 01:00:49.640 --> 01:00:53.220 In fact, I could even just do the runs right here and say, 01:00:53.220 --> 01:00:55.530 this is what the linear model would predict. 01:00:55.530 --> 01:00:59.090 But if there's curvature in my surface, 01:00:59.090 --> 01:01:01.520 then I'm going to get a deviation, just 01:01:01.520 --> 01:01:04.083 at the stainless steel setting. 01:01:04.083 --> 01:01:06.500 Now, if I were careful and trying to keep things balanced, 01:01:06.500 --> 01:01:11.810 I might do the stainless steel and the aluminum, 01:01:11.810 --> 01:01:17.510 and get an estimate across both of those at that deviation. 01:01:17.510 --> 01:01:18.890 So essentially, what we're trying 01:01:18.890 --> 01:01:22.190 to do here is add a data point and then look 01:01:22.190 --> 01:01:28.010 at a lack of fit test in particular, where I want 01:01:28.010 --> 01:01:30.260 to say, is the model error-- 01:01:30.260 --> 01:01:33.320 the deviation because of this extra higher order term-- 01:01:33.320 --> 01:01:38.000 is it explainable by pure replicate variance, 01:01:38.000 --> 01:01:41.990 or is it larger than that, and therefore not explainable 01:01:41.990 --> 01:01:43.970 by my noise factor? 01:01:43.970 --> 01:01:53.810 If we've got more deviation because of the quadratic term-- 01:01:53.810 --> 01:01:56.180 say, if I'm adding an x squared-- 01:01:56.180 --> 01:01:59.030 in this model, an x2 squared term-- 01:01:59.030 --> 01:02:01.760 then I have to reject the null hypothesis that says, 01:02:01.760 --> 01:02:03.740 I've just got random variance [INAUDIBLE].. 01:02:03.740 --> 01:02:05.780 I really need an extra model term. 01:02:05.780 --> 01:02:11.750 So we can still use the basic ANOVA setup. 01:02:11.750 --> 01:02:15.690 So this is our model with higher order terms. 01:02:15.690 --> 01:02:19.100 You might think about trying to add two higher order terms. 01:02:19.100 --> 01:02:22.610 I would argue, if this is the material type, 01:02:22.610 --> 01:02:25.610 it makes no sense to add that term. 01:02:25.610 --> 01:02:26.930 I wouldn't do that. 01:02:26.930 --> 01:02:28.500 That would be bad. 01:02:28.500 --> 01:02:31.650 But if this is associated with the punch depth, 01:02:31.650 --> 01:02:35.820 then I think it does make sense to add the hypothesis of that. 01:02:35.820 --> 01:02:40.190 And then you check for deviation at one or more of those points. 01:02:40.190 --> 01:02:47.180 And the basic question is asking whether beta 2, 2 is equal to 0 01:02:47.180 --> 01:02:48.710 or not. 01:02:48.710 --> 01:02:51.470 Is the effective due to that deviation 01:02:51.470 --> 01:02:55.360 plausibly the same as noise or not? 01:02:55.360 --> 01:02:57.200 So we can go through and do that. 01:02:57.200 --> 01:03:01.730 Here's a simple test, simple example. 01:03:01.730 --> 01:03:05.120 And this is aggregating what we saw 01:03:05.120 --> 01:03:10.400 going back, in fact, to some of the earlier things we talked 01:03:10.400 --> 01:03:13.970 about of, if I don't have replicate data, what can I do? 01:03:13.970 --> 01:03:20.120 If I just do my full factorial four corner point test, 01:03:20.120 --> 01:03:24.560 no replicates, and I use the full linear model with 01:03:24.560 --> 01:03:29.630 interactions, I cannot test for significance and I cannot test 01:03:29.630 --> 01:03:32.450 for lack of fit. 01:03:32.450 --> 01:03:35.570 That's the most bare bones situation you've got. 01:03:35.570 --> 01:03:37.160 I've got a perfect fit. 01:03:37.160 --> 01:03:39.230 But you better be awfully sure that there's not 01:03:39.230 --> 01:03:41.915 noise going on in your model or in your system. 01:03:45.890 --> 01:03:49.400 Even without replicating my corner points, 01:03:49.400 --> 01:03:55.910 I can still step back and ask the question about lack of fit 01:03:55.910 --> 01:03:59.820 by adding center points. 01:03:59.820 --> 01:04:03.825 But if I just add one center point-- 01:04:03.825 --> 01:04:06.740 one replicate at the center point, 01:04:06.740 --> 01:04:10.040 can I really ask the question about the significance 01:04:10.040 --> 01:04:13.410 of the quadratic term? 01:04:13.410 --> 01:04:13.910 No. 01:04:13.910 --> 01:04:16.430 I would be able to perfectly fit the quadratic term 01:04:16.430 --> 01:04:20.010 with that one extra data point, one extra model coefficient. 01:04:20.010 --> 01:04:24.380 So you have to have replicates somewhere in the system. 01:04:24.380 --> 01:04:27.590 And one typical approach is maybe 01:04:27.590 --> 01:04:32.390 I don't want to have 10 replicates at each of my corner 01:04:32.390 --> 01:04:33.920 points. 01:04:33.920 --> 01:04:37.940 I'm just going to add replicates at my center point. 01:04:37.940 --> 01:04:41.090 So now, with the addition of center points 01:04:41.090 --> 01:04:45.500 to potentially check for curvature and replicates, 01:04:45.500 --> 01:04:47.660 I've got an estimate of random-- 01:04:47.660 --> 01:04:51.630 a replicate noise and deviation from the linear. 01:04:51.630 --> 01:04:54.650 So together, by adding replicates at the center point, 01:04:54.650 --> 01:04:56.820 you get a lot of power. 01:04:56.820 --> 01:04:59.150 So then you can use the ANOVA approach. 01:04:59.150 --> 01:05:01.400 And this is just working through and saying, 01:05:01.400 --> 01:05:04.490 I estimate the quadratic term or the deviation 01:05:04.490 --> 01:05:07.250 and sum of squared deviations from that, 01:05:07.250 --> 01:05:09.960 form an estimate of the variance because of that, 01:05:09.960 --> 01:05:12.670 and compare that to the replication error. 01:05:15.680 --> 01:05:18.590 Here's just working that out for our example, 01:05:18.590 --> 01:05:22.400 where I've got the average of my output from 01:05:22.400 --> 01:05:26.420 the linear model, the average at my replicate-- 01:05:26.420 --> 01:05:28.190 if my overall grand average is a little 01:05:28.190 --> 01:05:31.910 different than the average coming from my center point 01:05:31.910 --> 01:05:35.000 replicates, that's the deviation due 01:05:35.000 --> 01:05:39.830 to the hypothesized quadratic term. 01:05:39.830 --> 01:05:42.740 I can form the variance of that, and then compare that 01:05:42.740 --> 01:05:46.580 to my underlying estimate. 01:05:46.580 --> 01:05:49.070 OK, and then this example I'll let you work through. 01:05:49.070 --> 01:05:52.550 This is just doing that for a simple run, where, again, I 01:05:52.550 --> 01:05:56.030 have just one run at each of the four corner points. 01:05:56.030 --> 01:05:58.290 This is a different data set, by the way. 01:05:58.290 --> 01:05:59.970 This is not the break forming anymore. 01:05:59.970 --> 01:06:02.330 And then I've got five replicates here at the center 01:06:02.330 --> 01:06:03.560 point. 01:06:03.560 --> 01:06:04.303 Question-- 01:06:04.303 --> 01:06:06.220 AUDIENCE: Are we assuming that the application 01:06:06.220 --> 01:06:09.215 error and the center point is going to be the same as the-- 01:06:09.215 --> 01:06:09.840 PROFESSOR: Yes. 01:06:09.840 --> 01:06:11.990 So we are making the assumption here-- 01:06:11.990 --> 01:06:15.350 and it's the same one we've made pretty much in most ANOVA 01:06:15.350 --> 01:06:16.580 analyses-- 01:06:16.580 --> 01:06:21.060 that the variance is the same at all run points. 01:06:21.060 --> 01:06:24.290 In other words, we have the n 0 sigma squared assumption, 01:06:24.290 --> 01:06:27.050 where sigma squared is operated at the center point as well 01:06:27.050 --> 01:06:28.100 as all the corner points. 01:06:30.725 --> 01:06:32.600 So this is a neat example where, again, we're 01:06:32.600 --> 01:06:37.260 trying to estimate main effects, perhaps interaction effects. 01:06:37.260 --> 01:06:39.300 We've got an AB interaction column here. 01:06:42.650 --> 01:06:46.490 So just using the corner points, we can fit a model. 01:06:46.490 --> 01:06:49.960 We can ask the ANOVA question on significance of that. 01:06:49.960 --> 01:06:53.480 But I can also form the sum of squares 01:06:53.480 --> 01:06:58.310 due to this quadratic term and use the replicated information 01:06:58.310 --> 01:07:01.760 there to be able to form an estimate of the overall process 01:07:01.760 --> 01:07:02.900 variance. 01:07:02.900 --> 01:07:10.600 And what that lets me do is form the F-test line 01:07:10.600 --> 01:07:15.992 in here for the quadratic term, in addition to the AB 01:07:15.992 --> 01:07:17.780 and the interaction term. 01:07:17.780 --> 01:07:20.560 And in this case, when I look at that F 01:07:20.560 --> 01:07:23.920 ratio, the sum of squared deviations 01:07:23.920 --> 01:07:28.420 from the quadratic estimate versus the linear estimate 01:07:28.420 --> 01:07:31.840 compared to my estimate of noise in the system, 01:07:31.840 --> 01:07:36.550 I get a very small F, well below the critical F. 01:07:36.550 --> 01:07:41.200 And therefore, I conclude there is no evidence of lack of fit. 01:07:41.200 --> 01:07:44.560 I do not include the quadratic term, 01:07:44.560 --> 01:07:46.840 and I stick with the linear model. 01:07:46.840 --> 01:07:51.830 Now, would you keep the interaction term? 01:07:51.830 --> 01:07:54.790 Same thing-- it's also not significant. 01:07:54.790 --> 01:07:56.990 Would you keep the two main effect terms? 01:07:56.990 --> 01:08:01.690 Yeah, sure-- those are above the F0, so I'm happy with those. 01:08:01.690 --> 01:08:05.470 So this is a nice, simple data set-- additive effects, 01:08:05.470 --> 01:08:07.510 no interaction. 01:08:07.510 --> 01:08:11.170 This is the best for ultimately using the model in a control 01:08:11.170 --> 01:08:14.080 kind of setting, because I can presumably 01:08:14.080 --> 01:08:16.720 change A and B independently, and try 01:08:16.720 --> 01:08:19.359 to get the output that I want. 01:08:19.359 --> 01:08:22.366 AUDIENCE: Would the f-test be quite close to the boundary? 01:08:22.366 --> 01:08:25.740 Or do we have to do sensitivity analysis to see-- 01:08:25.740 --> 01:08:30.160 [INAUDIBLE] to 96% or 97% [INAUDIBLE].. 01:08:30.160 --> 01:08:31.630 PROFESSOR: Sure. 01:08:31.630 --> 01:08:33.550 You always got to be a little careful 01:08:33.550 --> 01:08:36.520 and use exactly that knowledge. 01:08:36.520 --> 01:08:41.109 This was a 95% confidence, and if I was anywhere 01:08:41.109 --> 01:08:45.819 near the boundary, I might say, well, my-- 01:08:45.819 --> 01:08:47.740 I want to include that if I wanted to just 01:08:47.740 --> 01:08:52.319 be 90% confidence, because that's probably 01:08:52.319 --> 01:08:56.609 suggesting that that model term really does have an effect. 01:08:56.609 --> 01:08:59.370 In this case, that cut-off-- 01:08:59.370 --> 01:09:00.700 those are very obvious. 01:09:00.700 --> 01:09:02.520 But in other subtle cases, you actually 01:09:02.520 --> 01:09:05.790 have to use some intuition and knowledge. 01:09:05.790 --> 01:09:12.229 One part of that does look at-- maybe you're 01:09:12.229 --> 01:09:15.450 getting at this a little bit with sensitivity. 01:09:15.450 --> 01:09:18.680 There's another sensitivity analysis you can do, 01:09:18.680 --> 01:09:22.910 which has to do with not tweaking the alpha 01:09:22.910 --> 01:09:24.979 level, the significance level-- 01:09:24.979 --> 01:09:27.620 you can tweak that a little bit, but you also 01:09:27.620 --> 01:09:32.080 might ask the question slightly differently 01:09:32.080 --> 01:09:36.040 about whether the model coefficient is real or not. 01:09:36.040 --> 01:09:37.870 That's a significance question. 01:09:37.870 --> 01:09:41.890 You can also ask the question, how big is the effective? 01:09:41.890 --> 01:09:44.920 Is it so big that I would want to include it in the model? 01:09:44.920 --> 01:09:49.600 Or it's real, but it's so tiny compared to my other effects 01:09:49.600 --> 01:09:51.670 that I'm going to neglect it. 01:09:51.670 --> 01:09:55.870 So there is this difference between significant, real, 01:09:55.870 --> 01:09:58.730 and important or not. 01:09:58.730 --> 01:10:01.510 And so that's where your knowledge of the process 01:10:01.510 --> 01:10:03.610 also comes into play, and whether you might want 01:10:03.610 --> 01:10:05.200 to include or drop model terms. 01:10:08.110 --> 01:10:09.160 OK. 01:10:09.160 --> 01:10:12.273 I think what we'll do is I'll go-- 01:10:12.273 --> 01:10:14.440 I might actually only go for about five more minutes 01:10:14.440 --> 01:10:16.900 and give you a little bit of a glimpse of blocks 01:10:16.900 --> 01:10:20.950 and confounding, talk about these nuisance factors 01:10:20.950 --> 01:10:21.670 a little bit. 01:10:21.670 --> 01:10:23.503 And then next time, we'll come back and dive 01:10:23.503 --> 01:10:27.190 in more on fractional factorial designs, 01:10:27.190 --> 01:10:29.200 because there are some additional design 01:10:29.200 --> 01:10:33.620 issues I've only alluded to in experimental design. 01:10:33.620 --> 01:10:37.360 So far, we've really just done full factorial-- 01:10:37.360 --> 01:10:38.500 these corner points-- 01:10:38.500 --> 01:10:40.240 and then also the idea of center points 01:10:40.240 --> 01:10:44.410 to ask the question about curvature. 01:10:44.410 --> 01:10:46.990 But there may be other factors. 01:10:46.990 --> 01:10:52.300 Maybe there was the temperature in our break forming tool 01:10:52.300 --> 01:10:56.140 that we didn't really want to do a design experiment on. 01:10:56.140 --> 01:10:59.410 We're not really sure it's important or not, 01:10:59.410 --> 01:11:00.760 but maybe it's out there. 01:11:00.760 --> 01:11:03.670 There may be all these other nuisance factors 01:11:03.670 --> 01:11:05.200 or nuisance effects. 01:11:05.200 --> 01:11:08.260 They may affect the output, but I didn't really 01:11:08.260 --> 01:11:10.660 want to include the effect in the model. 01:11:10.660 --> 01:11:15.190 And I'd rather suppress them, and for sure suppress them 01:11:15.190 --> 01:11:16.150 from making-- 01:11:16.150 --> 01:11:21.580 from me making a mistake and confusing 01:11:21.580 --> 01:11:25.930 the effect of temperature with, say, the effect of which metal 01:11:25.930 --> 01:11:27.850 I use. 01:11:27.850 --> 01:11:30.700 So what are the approaches you can do? 01:11:30.700 --> 01:11:32.680 Well, in many cases, I may not be 01:11:32.680 --> 01:11:34.300 able to run the whole experiment, 01:11:34.300 --> 01:11:36.100 holding that factor constant. 01:11:36.100 --> 01:11:37.767 In general, you would really want 01:11:37.767 --> 01:11:39.850 to make sure, well, if temperature affects things, 01:11:39.850 --> 01:11:42.100 I want to hold the temperature of the tool as constant 01:11:42.100 --> 01:11:42.640 as I can-- 01:11:42.640 --> 01:11:47.020 try to keep everything else it's constant as possible. 01:11:47.020 --> 01:11:50.320 Time can often be an important substitute 01:11:50.320 --> 01:11:52.360 for nuisance effects. 01:11:52.360 --> 01:11:55.480 Time is going on, and other things may be changing. 01:11:55.480 --> 01:11:58.840 So you can't entirely suppress potentially 01:11:58.840 --> 01:12:00.290 all these other nuisance effects. 01:12:00.290 --> 01:12:03.590 But in general, you want to keep everything else constant. 01:12:03.590 --> 01:12:06.430 But some cases-- you may not be able to hold 01:12:06.430 --> 01:12:08.210 everything else constant. 01:12:08.210 --> 01:12:11.200 And if you know some of these factors-- 01:12:11.200 --> 01:12:15.220 like temperature perhaps throughout the day 01:12:15.220 --> 01:12:17.740 varies in my machine shop, say-- 01:12:17.740 --> 01:12:22.600 then it may be uncontrollable. 01:12:22.600 --> 01:12:25.890 But it's a known factor, in which case, I 01:12:25.890 --> 01:12:28.750 might want to think about that and randomize the order 01:12:28.750 --> 01:12:31.430 of my runs against that. 01:12:31.430 --> 01:12:36.310 So I equally include some high levels of that nuisance factor 01:12:36.310 --> 01:12:38.260 and some low levels of that nuisance factor 01:12:38.260 --> 01:12:40.035 in each of my other settings. 01:12:40.035 --> 01:12:44.860 So I try as much as I can to randomize, 01:12:44.860 --> 01:12:49.600 basically convert that effect from a systematic effect 01:12:49.600 --> 01:12:53.630 into a random noise effect. 01:12:53.630 --> 01:12:58.760 In many cases, if I can actually pick the level of temperature-- 01:12:58.760 --> 01:13:01.530 it's known and controllable-- 01:13:01.530 --> 01:13:04.520 you can actually design your experiment, 01:13:04.520 --> 01:13:07.610 separate out the data into blocks 01:13:07.610 --> 01:13:09.740 where, in each of the two blocks, 01:13:09.740 --> 01:13:14.220 the nuisance factor is held constant. 01:13:14.220 --> 01:13:16.820 And what we have the opportunity to do 01:13:16.820 --> 01:13:21.650 is transform the effect of that nuisance factor 01:13:21.650 --> 01:13:26.270 into what would only be a very higher order interaction 01:13:26.270 --> 01:13:29.910 term in our model. 01:13:29.910 --> 01:13:31.910 And that's the intuition I want to just give you 01:13:31.910 --> 01:13:33.785 a little bit of a glimpse for, and then we'll 01:13:33.785 --> 01:13:35.040 talk about next time. 01:13:35.040 --> 01:13:38.480 First off, if the thing is controllable, 01:13:38.480 --> 01:13:42.290 we would just want to, say, replicate and randomize. 01:13:42.290 --> 01:13:44.000 Here's an example where we're looking 01:13:44.000 --> 01:13:47.330 for the hardness of four different test samples, 01:13:47.330 --> 01:13:50.210 but I'm doing it with an indentation test, 01:13:50.210 --> 01:13:52.460 but I've got four different tips that I'm 01:13:52.460 --> 01:13:54.560 using for the indentation. 01:13:54.560 --> 01:13:59.810 I don't really care about the tip factor, 01:13:59.810 --> 01:14:02.750 but I'm worried that it might perturb 01:14:02.750 --> 01:14:05.000 my estimate of the hardness. 01:14:05.000 --> 01:14:08.240 So you can, first off, basically think of it 01:14:08.240 --> 01:14:11.630 as I'm just going to run with all four tips 01:14:11.630 --> 01:14:14.120 and take the average, and that's my best 01:14:14.120 --> 01:14:17.570 average of-- or estimate of the hardness in the four cases. 01:14:17.570 --> 01:14:21.920 So I'm just replicating around that nuisance factor-- 01:14:21.920 --> 01:14:25.280 not including it in the model explicitly, 01:14:25.280 --> 01:14:29.210 but on randomizing or replicating around it. 01:14:32.040 --> 01:14:34.530 Many cases-- you don't have the opportunity. 01:14:34.530 --> 01:14:38.490 It's too expensive to do lots of replications. 01:14:38.490 --> 01:14:43.590 And what I want to do is arrange things so 01:14:43.590 --> 01:14:46.140 that, if there is a nuisance factor, 01:14:46.140 --> 01:14:50.890 it will have the least damaging effect on my model. 01:14:50.890 --> 01:14:54.210 So here's my one little intuitive example. 01:14:54.210 --> 01:14:56.680 Suppose we've got this 2-by-2 design. 01:14:56.680 --> 01:15:00.720 So I've got two factors, two levels doing four runs. 01:15:00.720 --> 01:15:03.000 But let's say I have two arrange it 01:15:03.000 --> 01:15:06.240 where I do two runs at low temperature 01:15:06.240 --> 01:15:08.875 and two runs at high temperature-- 01:15:08.875 --> 01:15:11.250 have to do two runs in the morning, two in the afternoon, 01:15:11.250 --> 01:15:13.420 and during the day, my machine job heats up. 01:15:13.420 --> 01:15:14.620 That's just the way it is. 01:15:14.620 --> 01:15:15.990 I've got to live with it. 01:15:15.990 --> 01:15:19.740 Which two runs should I pick to do with the high temperature, 01:15:19.740 --> 01:15:22.860 and which two runs should I do at the low temperature? 01:15:22.860 --> 01:15:25.890 Suppose I did it like this, where 01:15:25.890 --> 01:15:30.630 this was my low temperature and this was my high temperature. 01:15:30.630 --> 01:15:31.800 Any problem with that? 01:15:36.330 --> 01:15:38.240 Big problem with that-- 01:15:38.240 --> 01:15:45.280 if x2 has an effect, I can't tell 01:15:45.280 --> 01:15:47.140 whether that's a temperature effect 01:15:47.140 --> 01:15:51.310 or and x2 effect, because I did both of my x2 runs 01:15:51.310 --> 01:15:55.780 at the low temperature and both of my high values of x2 01:15:55.780 --> 01:15:56.900 at the high temperature. 01:15:56.900 --> 01:15:59.560 I don't know whether it's a temperature effect when I form 01:15:59.560 --> 01:16:04.700 that contrast or an x2 effect. 01:16:04.700 --> 01:16:09.940 Is there a better way to arrange or pick 01:16:09.940 --> 01:16:13.270 which two lines I want to do so that I don't get 01:16:13.270 --> 01:16:17.210 confused with the main effects in the experiment? 01:16:17.210 --> 01:16:22.770 And the intuition here is yes, there is a better approach. 01:16:22.770 --> 01:16:29.550 What if I pick block 1 to run the 1 and AB combinations? 01:16:29.550 --> 01:16:31.640 So what if I pick block 1 to run-- 01:16:31.640 --> 01:16:32.570 what'd I say? 01:16:32.570 --> 01:16:34.250 The 1 and the AB? 01:16:34.250 --> 01:16:38.405 1 and AB-- notice what's nice about that. 01:16:38.405 --> 01:16:40.910 The 1 and the AB-- 01:16:40.910 --> 01:16:45.320 so make that block 1 and make this block 2-- 01:16:45.320 --> 01:16:53.240 notice that neither block is corresponding 01:16:53.240 --> 01:16:57.890 to both high settings or both low settings for either factor. 01:16:57.890 --> 01:17:00.170 What, in fact, the blockage is actually 01:17:00.170 --> 01:17:03.905 lining up with is the x1, x2 interaction factor. 01:17:10.227 --> 01:17:12.310 I don't want to get confused about whether there's 01:17:12.310 --> 01:17:16.090 a main effect or this nasty nuisance factor 01:17:16.090 --> 01:17:18.170 is having an effect. 01:17:18.170 --> 01:17:20.650 What I would am willing to do is be 01:17:20.650 --> 01:17:26.680 confused, or confound between my block effect and my interaction 01:17:26.680 --> 01:17:27.760 effect. 01:17:27.760 --> 01:17:29.920 I won't be able to tell if the interaction is real 01:17:29.920 --> 01:17:31.540 or not, but that's OK. 01:17:31.540 --> 01:17:35.930 I believe it's not, and I'd rather mix in my block factor. 01:17:35.930 --> 01:17:42.040 So this gets to this idea of forming patterns-- or blocking, 01:17:42.040 --> 01:17:46.180 in this case-- to confound so that I can detect 01:17:46.180 --> 01:17:47.950 some things, and not others. 01:17:47.950 --> 01:17:50.590 And we'll talk more about that next time-- 01:17:50.590 --> 01:17:54.700 talk a little bit more generally about confounding. 01:17:54.700 --> 01:17:56.710 And this actually leads into the main idea 01:17:56.710 --> 01:17:59.620 of fractional factorial design, where 01:17:59.620 --> 01:18:01.690 I may not want to pick all of my corner points. 01:18:01.690 --> 01:18:04.180 I might subset some of them, because I'm 01:18:04.180 --> 01:18:07.210 willing to be confused about whether there's 01:18:07.210 --> 01:18:11.440 a fourth-order factor that's also the same detection 01:18:11.440 --> 01:18:14.080 or same contrast with a second-order interaction 01:18:14.080 --> 01:18:15.040 factor. 01:18:15.040 --> 01:18:16.425 I don't care. 01:18:16.425 --> 01:18:17.800 And therefore, if I don't care, I 01:18:17.800 --> 01:18:20.200 can actually reduce the number of experimental points 01:18:20.200 --> 01:18:20.800 we want to do. 01:18:20.800 --> 01:18:23.490 So we'll talk about that next time.