WEBVTT 00:00:00.000 --> 00:00:01.491 align:middle line:90% [SQUEAKING] 00:00:01.491 --> 00:00:03.976 align:middle line:90% [RUSTLING] 00:00:03.976 --> 00:00:07.455 align:middle line:90% [CLICKING] 00:00:07.455 --> 00:00:11.255 align:middle line:90% 00:00:11.255 --> 00:00:11.880 align:middle line:90% PETER SHOR: OK. 00:00:11.880 --> 00:00:14.560 align:middle line:84% So, today, we're doing linear programming, 00:00:14.560 --> 00:00:22.400 align:middle line:84% and we are going to basically do a very, I think, 00:00:22.400 --> 00:00:29.360 align:middle line:84% four- or five-lecture unit on linear programming. 00:00:29.360 --> 00:00:34.740 align:middle line:84% And linear programming is one of the big, I guess, 00:00:34.740 --> 00:00:37.760 align:middle line:90% engines behind optimization. 00:00:37.760 --> 00:00:41.320 align:middle line:84% And to show how important it is-- 00:00:41.320 --> 00:00:46.540 align:middle line:84% well, when I started working in 1986, 00:00:46.540 --> 00:00:52.840 align:middle line:84% there were a bunch of linear programming tools on the market. 00:00:52.840 --> 00:00:57.280 align:middle line:84% And Robert Bixby, in 1987, founded a company 00:00:57.280 --> 00:01:04.650 align:middle line:84% and wrote CPLEX, which was, by far, the best linear programming 00:01:04.650 --> 00:01:08.010 align:middle line:84% tool on the market, and he charged an immense amount 00:01:08.010 --> 00:01:09.050 align:middle line:90% for it. 00:01:09.050 --> 00:01:12.630 align:middle line:84% And a lot of people bought it, and he-- 00:01:12.630 --> 00:01:15.850 align:middle line:84% and he sold his company 10 years later for, I don't know, 00:01:15.850 --> 00:01:19.610 align:middle line:90% $40 or $50 million. 00:01:19.610 --> 00:01:24.610 align:middle line:84% And IBM recently bought the company 00:01:24.610 --> 00:01:28.250 align:middle line:84% that he sold it to for $180 million. 00:01:28.250 --> 00:01:33.570 align:middle line:84% So it's big business and very important. 00:01:33.570 --> 00:01:36.010 align:middle line:90% OK. 00:01:36.010 --> 00:01:37.970 align:middle line:90% Just-- yeah. 00:01:37.970 --> 00:01:39.830 align:middle line:90% So what is linear programming? 00:01:39.830 --> 00:01:54.050 align:middle line:90% 00:01:54.050 --> 00:01:58.570 align:middle line:84% So I'm going to give two examples of problems 00:01:58.570 --> 00:02:00.990 align:middle line:84% you might want to solve with linear programming, 00:02:00.990 --> 00:02:08.130 align:middle line:84% and then we will continue with other things. 00:02:08.130 --> 00:02:10.509 align:middle line:90% So what is linear programming? 00:02:10.509 --> 00:02:20.290 align:middle line:84% It's essentially optimize-- either minimize or maximize-- 00:02:20.290 --> 00:02:27.210 align:middle line:90% 00:02:27.210 --> 00:02:37.150 align:middle line:84% a linear function subject to linear constraints. 00:02:37.150 --> 00:02:54.650 align:middle line:90% 00:02:54.650 --> 00:02:56.650 align:middle line:84% And we'll start with a diet model. 00:02:56.650 --> 00:03:00.180 align:middle line:90% 00:03:00.180 --> 00:03:08.540 align:middle line:84% And so this is actually taken straight from the lecture notes. 00:03:08.540 --> 00:03:16.220 align:middle line:84% And I think, given the costs, I think it only 00:03:16.220 --> 00:03:17.977 align:middle line:90% works if you're feeding mice. 00:03:17.977 --> 00:03:19.310 align:middle line:90% So let's say you have some mice. 00:03:19.310 --> 00:03:22.500 align:middle line:90% 00:03:22.500 --> 00:03:24.520 align:middle line:84% And there are nutritional requirements. 00:03:24.520 --> 00:03:29.460 align:middle line:90% 00:03:29.460 --> 00:03:31.150 align:middle line:84% They need a certain amount of starch. 00:03:31.150 --> 00:03:36.300 align:middle line:90% 00:03:36.300 --> 00:03:42.100 align:middle line:84% And there are two grains you can give them, grain 1 and grain 2. 00:03:42.100 --> 00:03:54.590 align:middle line:84% And they need 8 units of starch, 15 units of protein, 00:03:54.590 --> 00:03:56.720 align:middle line:90% and 7 units of vitamins. 00:03:56.720 --> 00:04:02.910 align:middle line:90% 00:04:02.910 --> 00:04:17.390 align:middle line:84% And grain 1 contains 5, 4, and 2 and costs $0.60. 00:04:17.390 --> 00:04:29.550 align:middle line:84% And grain 2 contains 7, 2, and 1, and costs $0.35. 00:04:29.550 --> 00:04:35.310 align:middle line:84% And the question is, how cheaply can you feed the mice 00:04:35.310 --> 00:04:38.010 align:middle line:84% and satisfy their nutritional requirements? 00:04:38.010 --> 00:04:43.470 align:middle line:90% 00:04:43.470 --> 00:04:46.510 align:middle line:84% So what we do with linear programming 00:04:46.510 --> 00:04:59.630 align:middle line:84% is we need to write down a model. 00:04:59.630 --> 00:05:01.130 align:middle line:84% And how do you write down a model? 00:05:01.130 --> 00:05:02.490 align:middle line:90% You choose variables. 00:05:02.490 --> 00:05:08.390 align:middle line:90% 00:05:08.390 --> 00:05:16.470 align:middle line:84% Write objective function and constraints. 00:05:16.470 --> 00:05:18.550 align:middle line:84% And then you feed your linear program 00:05:18.550 --> 00:05:22.830 align:middle line:84% to a linear programming package, and it will give you the answer. 00:05:22.830 --> 00:05:27.330 align:middle line:84% So, any suggestions as to what the variables should be? 00:05:27.330 --> 00:05:35.030 align:middle line:90% 00:05:35.030 --> 00:05:36.670 align:middle line:90% Yeah? 00:05:36.670 --> 00:05:37.170 align:middle line:90% Yeah. 00:05:37.170 --> 00:05:38.390 align:middle line:90% How much of each grain? 00:05:38.390 --> 00:05:46.018 align:middle line:84% So let's let x1, amount of grain 1, 00:05:46.018 --> 00:05:52.400 align:middle line:90% and x2 be the amount of grain 2. 00:05:52.400 --> 00:05:56.540 align:middle line:84% And now, we want to write down an objective function, 00:05:56.540 --> 00:05:59.540 align:middle line:84% so we either need to minimize or maximize something. 00:05:59.540 --> 00:06:01.460 align:middle line:90% So what do we want to minimize? 00:06:01.460 --> 00:06:04.800 align:middle line:90% 00:06:04.800 --> 00:06:06.040 align:middle line:90% The cost. 00:06:06.040 --> 00:06:09.440 align:middle line:90% So let's write it down here. 00:06:09.440 --> 00:06:19.600 align:middle line:84% Minimize-- I guess it's 60x1 plus 35x2. 00:06:19.600 --> 00:06:24.200 align:middle line:90% 00:06:24.200 --> 00:06:26.340 align:middle line:90% And what are the constraints? 00:06:26.340 --> 00:06:29.540 align:middle line:84% Well, the constraints are that they get enough starch, 00:06:29.540 --> 00:06:32.040 align:middle line:84% get enough proteins, and get enough vitamin. 00:06:32.040 --> 00:06:33.050 align:middle line:90% So that would be-- 00:06:33.050 --> 00:06:36.520 align:middle line:90% 00:06:36.520 --> 00:06:40.200 align:middle line:84% and what we do when we write down linear programs 00:06:40.200 --> 00:06:45.960 align:middle line:90% is we say subject to-- 00:06:45.960 --> 00:06:47.460 align:middle line:90% I believe we say subject to. 00:06:47.460 --> 00:06:53.850 align:middle line:84% Let's see if-- yeah, subject to, before we write down 00:06:53.850 --> 00:06:55.850 align:middle line:90% all the constraints. 00:06:55.850 --> 00:07:01.510 align:middle line:84% So subject to-- and now we need some things. 00:07:01.510 --> 00:07:07.210 align:middle line:84% So we need 5 times the grain 1 plus 7 times grain 2 00:07:07.210 --> 00:07:08.660 align:middle line:90% has to be at least 8. 00:07:08.660 --> 00:07:11.210 align:middle line:90% 00:07:11.210 --> 00:07:17.810 align:middle line:84% So 5x1 plus 7x2 is greater than or equal to 8. 00:07:17.810 --> 00:07:23.400 align:middle line:84% And 4x1 plus 2x2 is greater than or equal to 15. 00:07:23.400 --> 00:07:34.850 align:middle line:90% 00:07:34.850 --> 00:07:39.830 align:middle line:84% And finally, 2x1 plus x2 is greater than or equal to 7. 00:07:39.830 --> 00:07:46.730 align:middle line:90% 00:07:46.730 --> 00:07:49.210 align:middle line:84% And I'm not going to solve it in part because it 00:07:49.210 --> 00:07:51.210 align:middle line:90% has an utterly trivial answer. 00:07:51.210 --> 00:07:55.490 align:middle line:84% And also, we're not going to actually tell you 00:07:55.490 --> 00:07:59.597 align:middle line:84% how to solve linear programming in this class. 00:07:59.597 --> 00:08:01.430 align:middle line:84% There's something called the simplex method, 00:08:01.430 --> 00:08:06.850 align:middle line:84% and I will give you a very rough overview of how the simplex 00:08:06.850 --> 00:08:08.850 align:middle line:90% method works at some point. 00:08:08.850 --> 00:08:13.590 align:middle line:84% But actually-- explaining exactly how the details work 00:08:13.590 --> 00:08:15.090 align:middle line:84% is something, first, you'll probably 00:08:15.090 --> 00:08:19.210 align:middle line:84% never need because there are tons of linear programming 00:08:19.210 --> 00:08:23.210 align:middle line:84% packages, and you can just download one from the internet 00:08:23.210 --> 00:08:24.370 align:middle line:90% and use that. 00:08:24.370 --> 00:08:31.670 align:middle line:84% And second, explaining how it works is somewhat interesting, 00:08:31.670 --> 00:08:33.370 align:middle line:84% but it's also rather tedious, and there 00:08:33.370 --> 00:08:36.862 align:middle line:84% are lots of little technicalities 00:08:36.862 --> 00:08:39.279 align:middle line:84% you have to go through, so I don't really want to do that. 00:08:39.279 --> 00:08:44.455 align:middle line:90% 00:08:44.455 --> 00:08:46.890 align:middle line:90% So that is linear programming. 00:08:46.890 --> 00:08:50.140 align:middle line:90% 00:08:50.140 --> 00:08:52.020 align:middle line:90% Oh, wait. 00:08:52.020 --> 00:08:54.900 align:middle line:84% I just realized there are two constraints 00:08:54.900 --> 00:08:56.660 align:middle line:90% I did not write down. 00:08:56.660 --> 00:08:59.360 align:middle line:84% Does anybody-- can anybody tell me what they are? 00:08:59.360 --> 00:09:02.300 align:middle line:90% 00:09:02.300 --> 00:09:02.800 align:middle line:90% Yes? 00:09:02.800 --> 00:09:03.980 align:middle line:90% STUDENT: The non-negative? 00:09:03.980 --> 00:09:05.200 align:middle line:90% PETER SHOR: Non-negativity. 00:09:05.200 --> 00:09:05.700 align:middle line:90% Yeah. 00:09:05.700 --> 00:09:09.900 align:middle line:84% So x1 greater than or 2, x1 greater than or equal to 0, 00:09:09.900 --> 00:09:12.380 align:middle line:84% and x2 greater than or equal to 0. 00:09:12.380 --> 00:09:16.500 align:middle line:84% And these are non-negativity constraints. 00:09:16.500 --> 00:09:20.000 align:middle line:90% Non-negativity constraints. 00:09:20.000 --> 00:09:22.740 align:middle line:90% 00:09:22.740 --> 00:09:27.300 align:middle line:84% And they're actually handled a little bit differently 00:09:27.300 --> 00:09:29.770 align:middle line:90% than most constraints in-- 00:09:29.770 --> 00:09:32.637 align:middle line:90% 00:09:32.637 --> 00:09:34.220 align:middle line:84% are often handled a little differently 00:09:34.220 --> 00:09:37.000 align:middle line:84% from most constraints in linear programming. 00:09:37.000 --> 00:09:37.500 align:middle line:90% OK. 00:09:37.500 --> 00:09:58.660 align:middle line:90% 00:09:58.660 --> 00:09:59.200 align:middle line:90% OK. 00:09:59.200 --> 00:10:04.220 align:middle line:84% And now I'm going to come up with another example. 00:10:04.220 --> 00:10:05.380 align:middle line:90% OK. 00:10:05.380 --> 00:10:07.310 align:middle line:84% So let's say you're having a bake sale. 00:10:07.310 --> 00:10:11.740 align:middle line:90% 00:10:11.740 --> 00:10:15.365 align:middle line:84% And there are two things you know how to bake. 00:10:15.365 --> 00:10:16.990 align:middle line:84% Let's say you know how to bake cookies. 00:10:16.990 --> 00:10:19.620 align:middle line:90% 00:10:19.620 --> 00:10:22.540 align:middle line:84% And we'll simplify the ingredient list. 00:10:22.540 --> 00:10:25.300 align:middle line:84% Don't go home and use this ingredient list 00:10:25.300 --> 00:10:26.780 align:middle line:90% for real cookies. 00:10:26.780 --> 00:10:38.300 align:middle line:84% So eggs, we need 1; sugar, we need 2 cups; 00:10:38.300 --> 00:10:43.180 align:middle line:90% and flour, we need 1 cup-- 00:10:43.180 --> 00:10:44.400 align:middle line:90% no, 1 and 1/2 cups. 00:10:44.400 --> 00:10:49.990 align:middle line:90% 00:10:49.990 --> 00:10:52.490 align:middle line:84% And then you also know how to bake meringues. 00:10:52.490 --> 00:10:58.550 align:middle line:90% 00:10:58.550 --> 00:11:04.270 align:middle line:84% And for those, you need 3 eggs, you need 1 cup of sugar, 00:11:04.270 --> 00:11:05.520 align:middle line:90% and you don't need any flour. 00:11:05.520 --> 00:11:08.950 align:middle line:90% 00:11:08.950 --> 00:11:22.550 align:middle line:84% And you can sell a batch of meringues for $20, 00:11:22.550 --> 00:11:26.850 align:middle line:84% and you can sell a batch of cookies for $20. 00:11:26.850 --> 00:11:37.908 align:middle line:90% 00:11:37.908 --> 00:11:40.450 align:middle line:84% So the first thing we need to do is write down our variables. 00:11:40.450 --> 00:11:41.860 align:middle line:90% So what variables would you use? 00:11:41.860 --> 00:11:49.300 align:middle line:90% 00:11:49.300 --> 00:11:49.800 align:middle line:90% Yeah? 00:11:49.800 --> 00:11:52.600 align:middle line:90% 00:11:52.600 --> 00:11:54.080 align:middle line:90% No? 00:11:54.080 --> 00:11:54.600 align:middle line:90% Yes? 00:11:54.600 --> 00:11:56.100 align:middle line:84% STUDENT: Like, how many cookies and how many-- 00:11:56.100 --> 00:11:56.600 align:middle line:90% PETER SHOR: Right. 00:11:56.600 --> 00:11:59.380 align:middle line:84% How many batches of cookies and how many batches of meringue? 00:11:59.380 --> 00:12:09.880 align:middle line:84% So let's say x1 is the number of batches of cookies 00:12:09.880 --> 00:12:19.740 align:middle line:84% and x2 is the number of batches of meringues. 00:12:19.740 --> 00:12:25.680 align:middle line:90% 00:12:25.680 --> 00:12:28.800 align:middle line:90% And what do we have? 00:12:28.800 --> 00:12:32.300 align:middle line:84% Well, you want to maximize the amount of money, 00:12:32.300 --> 00:12:33.546 align:middle line:90% so you want to maximize-- 00:12:33.546 --> 00:12:36.240 align:middle line:90% 00:12:36.240 --> 00:12:41.400 align:middle line:84% so our notes call the things you're maximizing or minimizing 00:12:41.400 --> 00:12:50.860 align:middle line:84% z, and much of the time you don't actually need to-- 00:12:50.860 --> 00:12:57.742 align:middle line:90% 00:12:57.742 --> 00:12:59.200 align:middle line:84% much of the time you don't actually 00:12:59.200 --> 00:13:01.463 align:middle line:84% need to write down z in your linear program, 00:13:01.463 --> 00:13:02.880 align:middle line:84% but there are some times when it's 00:13:02.880 --> 00:13:06.480 align:middle line:84% very useful to have a variable name for the thing you're 00:13:06.480 --> 00:13:10.760 align:middle line:90% maximizing, so we'll use it. 00:13:10.760 --> 00:13:14.440 align:middle line:84% Well, you don't want to run out of eggs, 00:13:14.440 --> 00:13:17.680 align:middle line:84% so that means that the number of batches of cookies, 00:13:17.680 --> 00:13:27.480 align:middle line:84% x1 times 1 plus 3 times x2, had better 00:13:27.480 --> 00:13:33.040 align:middle line:84% be less than or equal to the total number of eggs you have, 00:13:33.040 --> 00:13:36.880 align:middle line:84% which I didn't tell you how many eggs you had. 00:13:36.880 --> 00:13:38.600 align:middle line:90% OK. 00:13:38.600 --> 00:13:52.690 align:middle line:84% So let's say we have 9 eggs, 8 cups of sugar, 00:13:52.690 --> 00:13:56.100 align:middle line:90% and 15 cups flour. 00:13:56.100 --> 00:13:59.690 align:middle line:90% 00:13:59.690 --> 00:14:02.660 align:middle line:84% So x1 plus 3x2 is less than or equal to 9. 00:14:02.660 --> 00:14:06.090 align:middle line:90% 00:14:06.090 --> 00:14:14.610 align:middle line:84% And 2x1 plus x2 is less than or equal to 8. 00:14:14.610 --> 00:14:16.970 align:middle line:90% And there's another one. 00:14:16.970 --> 00:14:25.700 align:middle line:84% 1.5 times x1 is less than or equal to 15. 00:14:25.700 --> 00:14:28.570 align:middle line:90% 00:14:28.570 --> 00:14:30.650 align:middle line:84% And of course, we have x1 greater than 00:14:30.650 --> 00:14:33.920 align:middle line:84% or equal to 0 and x2 greater than or equal to 0. 00:14:33.920 --> 00:14:41.780 align:middle line:90% 00:14:41.780 --> 00:14:46.680 align:middle line:84% And one nice thing you can do with linear programs, 00:14:46.680 --> 00:14:48.780 align:middle line:84% if there's not very many variables and not very 00:14:48.780 --> 00:14:53.580 align:middle line:84% many constraints, is you can actually draw the feasible 00:14:53.580 --> 00:14:55.740 align:middle line:90% region on the plane. 00:14:55.740 --> 00:15:01.740 align:middle line:84% So I think I need to give you some notation before I go ahead 00:15:01.740 --> 00:15:05.810 align:middle line:90% with drawing this. 00:15:05.810 --> 00:15:29.020 align:middle line:90% 00:15:29.020 --> 00:15:29.520 align:middle line:90% Notation. 00:15:29.520 --> 00:15:33.780 align:middle line:90% 00:15:33.780 --> 00:15:49.340 align:middle line:84% A feasible solution-- it's not necessarily optimal, is a point, 00:15:49.340 --> 00:15:56.330 align:middle line:84% let's say, x1 through xn are our variables, x1, x2, through-- 00:15:56.330 --> 00:16:09.100 align:middle line:90% 00:16:09.100 --> 00:16:12.300 align:middle line:84% I'm looking to see if the notes-- 00:16:12.300 --> 00:16:13.060 align:middle line:90% OK. 00:16:13.060 --> 00:16:14.030 align:middle line:90% Let's say xk. 00:16:14.030 --> 00:16:18.580 align:middle line:90% 00:16:18.580 --> 00:16:20.840 align:middle line:84% That satisfies all the constraints. 00:16:20.840 --> 00:16:32.380 align:middle line:90% 00:16:32.380 --> 00:16:39.390 align:middle line:84% An optimal solution is a feasible solution 00:16:39.390 --> 00:16:41.660 align:middle line:84% that optimizes our objective function. 00:16:41.660 --> 00:16:56.870 align:middle line:90% 00:16:56.870 --> 00:16:59.600 align:middle line:90% So, OK. 00:16:59.600 --> 00:17:32.520 align:middle line:90% 00:17:32.520 --> 00:17:37.640 align:middle line:84% So what does the feasible region look like? 00:17:37.640 --> 00:17:40.140 align:middle line:90% Well, let's write it down. 00:17:40.140 --> 00:17:44.720 align:middle line:84% Let's say this is x1 and this is x2. 00:17:44.720 --> 00:17:47.240 align:middle line:90% And we have x1-- 00:17:47.240 --> 00:17:51.960 align:middle line:84% well, we have 2x1 plus x2 is less than or equal to 8, 00:17:51.960 --> 00:17:58.360 align:middle line:84% which means we have the 0.8 up here, and the 0.4 down here, 00:17:58.360 --> 00:18:01.950 align:middle line:84% and we have the linear region there. 00:18:01.950 --> 00:18:04.560 align:middle line:90% 00:18:04.560 --> 00:18:10.000 align:middle line:90% So that's our first constraint. 00:18:10.000 --> 00:18:14.160 align:middle line:84% Second constraint is x1 plus 3x2 is less than or equal to 9. 00:18:14.160 --> 00:18:21.560 align:middle line:84% So now, let's put 9 here, 3 here. 00:18:21.560 --> 00:18:27.570 align:middle line:84% And our second constraint will be everything under this. 00:18:27.570 --> 00:18:33.240 align:middle line:90% 00:18:33.240 --> 00:18:37.280 align:middle line:84% And the third constraint is 1.5x1 is less than or equal 00:18:37.280 --> 00:18:41.480 align:middle line:84% to 15, which means x1 is less than or equal to 10, 00:18:41.480 --> 00:18:48.760 align:middle line:84% which means the third constraint is everything to the left 00:18:48.760 --> 00:18:49.470 align:middle line:90% of this line. 00:18:49.470 --> 00:18:52.440 align:middle line:90% 00:18:52.440 --> 00:18:57.040 align:middle line:84% So you can see that this third constraint is completely 00:18:57.040 --> 00:18:59.480 align:middle line:84% meaningless because anything that satisfies the first two 00:18:59.480 --> 00:19:04.280 align:middle line:84% constraints must satisfy the third one. 00:19:04.280 --> 00:19:09.200 align:middle line:84% And the actual feasible region is 00:19:09.200 --> 00:19:14.100 align:middle line:84% this region and the intersection of the first two constraints. 00:19:14.100 --> 00:19:16.780 align:middle line:90% 00:19:16.780 --> 00:19:17.280 align:middle line:90% OK. 00:19:17.280 --> 00:19:39.210 align:middle line:90% 00:19:39.210 --> 00:19:41.650 align:middle line:84% I think, before I need to say anything more, 00:19:41.650 --> 00:19:46.130 align:middle line:84% I should say something about the standard form 00:19:46.130 --> 00:19:51.350 align:middle line:84% and the canonical form for linear programming problems 00:19:51.350 --> 00:19:51.850 align:middle line:90% because-- 00:19:51.850 --> 00:20:09.610 align:middle line:90% 00:20:09.610 --> 00:20:24.110 align:middle line:90% so the notes-- form. 00:20:24.110 --> 00:20:28.860 align:middle line:90% 00:20:28.860 --> 00:20:36.540 align:middle line:84% So maximum z is equal to z transpose. 00:20:36.540 --> 00:20:38.340 align:middle line:90% These are our constraints. 00:20:38.340 --> 00:20:40.400 align:middle line:84% No, this is our-- not constraints, 00:20:40.400 --> 00:20:43.300 align:middle line:84% this is our objective function x. 00:20:43.300 --> 00:20:50.573 align:middle line:84% So here, I guess, z transpose is the vector 20, 20. 00:20:50.573 --> 00:20:58.660 align:middle line:90% 00:20:58.660 --> 00:21:07.230 align:middle line:84% Ax equals b, and x greater than or equal to 0. 00:21:07.230 --> 00:21:13.460 align:middle line:90% 00:21:13.460 --> 00:21:14.550 align:middle line:90% Canonical form. 00:21:14.550 --> 00:21:20.260 align:middle line:90% 00:21:20.260 --> 00:21:24.560 align:middle line:84% Maximum-- and it's nearly the same. 00:21:24.560 --> 00:21:28.460 align:middle line:90% c transpose x. 00:21:28.460 --> 00:21:31.640 align:middle line:90% A sub x less than or equal to b. 00:21:31.640 --> 00:21:33.410 align:middle line:90% x greater or equal to 0. 00:21:33.410 --> 00:21:47.820 align:middle line:90% 00:21:47.820 --> 00:21:50.140 align:middle line:90% Theorem. 00:21:50.140 --> 00:22:04.420 align:middle line:84% Any linear program can be put into, well, either standard 00:22:04.420 --> 00:22:06.690 align:middle line:90% or canonical form. 00:22:06.690 --> 00:22:16.140 align:middle line:90% 00:22:16.140 --> 00:22:18.520 align:middle line:90% So, how do we do this? 00:22:18.520 --> 00:22:22.500 align:middle line:90% 00:22:22.500 --> 00:22:25.080 align:middle line:84% OK, so there are some easier ones. 00:22:25.080 --> 00:22:27.990 align:middle line:84% So suppose our objective function-- 00:22:27.990 --> 00:22:36.030 align:middle line:84% suppose objective function-- is minimize 3x plus-- 00:22:36.030 --> 00:22:38.273 align:middle line:90% 3x1 plus 5x2. 00:22:38.273 --> 00:22:40.440 align:middle line:84% How would you turn that into a maximization problem? 00:22:40.440 --> 00:22:46.990 align:middle line:90% 00:22:46.990 --> 00:22:47.490 align:middle line:90% Yes? 00:22:47.490 --> 00:22:48.970 align:middle line:90% STUDENT: You just the negative. 00:22:48.970 --> 00:22:50.053 align:middle line:90% PETER SHOR: Yeah, exactly. 00:22:50.053 --> 00:22:51.190 align:middle line:90% Stick a negative on it. 00:22:51.190 --> 00:22:55.190 align:middle line:90% Maximize minus 3x1 minus 5x2. 00:22:55.190 --> 00:22:58.550 align:middle line:90% 00:22:58.550 --> 00:22:59.050 align:middle line:90% OK. 00:22:59.050 --> 00:23:05.514 align:middle line:90% 00:23:05.514 --> 00:23:21.270 align:middle line:84% Suppose you have unconstrained variable x sub i. 00:23:21.270 --> 00:23:23.450 align:middle line:84% Can be either positive or negative. 00:23:23.450 --> 00:23:30.525 align:middle line:90% 00:23:30.525 --> 00:23:32.150 align:middle line:84% Maybe I should just tell you the answer 00:23:32.150 --> 00:23:36.150 align:middle line:84% to this because it's not quite as obvious as the last one. 00:23:36.150 --> 00:23:37.110 align:middle line:90% OK. 00:23:37.110 --> 00:23:52.830 align:middle line:84% So replace xi with x sub i plus minus x sub i minus. 00:23:52.830 --> 00:23:55.790 align:middle line:84% x sub i plus is greater than or equal to 0, 00:23:55.790 --> 00:24:00.670 align:middle line:84% x sub i minus is greater than or equal to 0. 00:24:00.670 --> 00:24:03.510 align:middle line:90% So whenever you have-- 00:24:03.510 --> 00:24:06.870 align:middle line:84% so, I mean, it doesn't matter whether xi 00:24:06.870 --> 00:24:09.950 align:middle line:84% is positive or negative, there's also ways some xi plus 00:24:09.950 --> 00:24:15.090 align:middle line:84% and some xi minus which will give you any value of x sub i. 00:24:15.090 --> 00:24:18.310 align:middle line:90% 00:24:18.310 --> 00:24:22.980 align:middle line:84% And now these are non-negative variables. 00:24:22.980 --> 00:24:33.360 align:middle line:90% 00:24:33.360 --> 00:24:37.570 align:middle line:84% OK, so there's two more cases I should talk about. 00:24:37.570 --> 00:24:40.560 align:middle line:90% 00:24:40.560 --> 00:24:43.010 align:middle line:84% And they are a little bit more complicated than this. 00:24:43.010 --> 00:25:10.760 align:middle line:90% 00:25:10.760 --> 00:25:16.160 align:middle line:90% Suppose-- so let's look up here. 00:25:16.160 --> 00:25:20.450 align:middle line:84% So canonical form, all the constraints are inequalities. 00:25:20.450 --> 00:25:23.970 align:middle line:84% Staggered form, all the constraints are equalities. 00:25:23.970 --> 00:25:27.270 align:middle line:84% So to go back and forth between canonical form or standard form, 00:25:27.270 --> 00:25:29.250 align:middle line:84% or just to put an arbitrary linear program 00:25:29.250 --> 00:25:32.170 align:middle line:84% in canonical form, we need to turn inequalities 00:25:32.170 --> 00:25:34.730 align:middle line:90% into equalities and vice versa. 00:25:34.730 --> 00:25:49.730 align:middle line:84% So suppose we have x1 plus 3x2 less than or equal to 9, x1, 00:25:49.730 --> 00:25:54.450 align:middle line:84% x2 greater than or equal to 0, how do we make this an-- 00:25:54.450 --> 00:26:01.550 align:middle line:90% how do we make this an equality? 00:26:01.550 --> 00:26:05.330 align:middle line:90% 00:26:05.330 --> 00:26:06.590 align:middle line:90% Any suggestions? 00:26:06.590 --> 00:26:19.410 align:middle line:90% 00:26:19.410 --> 00:26:20.087 align:middle line:90% Yes? 00:26:20.087 --> 00:26:21.670 align:middle line:84% STUDENT: You can add another variable? 00:26:21.670 --> 00:26:23.212 align:middle line:84% PETER SHOR: You add another variable. 00:26:23.212 --> 00:26:24.010 align:middle line:90% Very good. 00:26:24.010 --> 00:26:33.850 align:middle line:84% What we do is we add something that's called a slack variable, 00:26:33.850 --> 00:26:44.530 align:middle line:84% and we get x1 plus 3x2 plus s is equal to 9 with s greater than 00:26:44.530 --> 00:26:47.650 align:middle line:90% or equal to 0. 00:26:47.650 --> 00:26:55.050 align:middle line:84% And if we have x1 plus 3x2 greater or equal to 9, 00:26:55.050 --> 00:27:04.010 align:middle line:84% we would add something called a surplus variable, 00:27:04.010 --> 00:27:10.820 align:middle line:84% x1 plus 3x2 minus s equals 9 s greater than 0. 00:27:10.820 --> 00:27:14.010 align:middle line:90% 00:27:14.010 --> 00:27:17.170 align:middle line:84% And the last thing we need to do is 00:27:17.170 --> 00:27:27.060 align:middle line:84% we need to, say, how to turn any equalities into inequalities. 00:27:27.060 --> 00:27:30.390 align:middle line:84% And that is actually, again, pretty easy. 00:27:30.390 --> 00:27:47.940 align:middle line:90% 00:27:47.940 --> 00:27:56.340 align:middle line:84% So if we have x1 plus 3x2 equals 9, 00:27:56.340 --> 00:28:02.740 align:middle line:84% we can turn that into x1 plus 3x2 greater than or equal to 9 00:28:02.740 --> 00:28:09.540 align:middle line:84% and x1 plus 3x2 less than or equal to 9. 00:28:09.540 --> 00:28:13.180 align:middle line:84% So anything that satisfies this satisfies that. 00:28:13.180 --> 00:28:16.230 align:middle line:90% And now, if you don't like-- 00:28:16.230 --> 00:28:20.230 align:middle line:84% if you want to put your thing in canonical form, 00:28:20.230 --> 00:28:23.790 align:middle line:84% you need to get rid of all the greater than or equal to signs, 00:28:23.790 --> 00:28:25.950 align:middle line:90% but that's easy. 00:28:25.950 --> 00:28:30.110 align:middle line:84% Then you just say minus x1 minus 3x2 is less than or equal 00:28:30.110 --> 00:28:30.880 align:middle line:90% to minus 9. 00:28:30.880 --> 00:28:33.790 align:middle line:90% 00:28:33.790 --> 00:28:35.520 align:middle line:90% And this is canonical form. 00:28:35.520 --> 00:28:44.470 align:middle line:90% 00:28:44.470 --> 00:28:47.750 align:middle line:84% So why do we need to worry about canonical form 00:28:47.750 --> 00:28:51.510 align:middle line:90% and standard form? 00:28:51.510 --> 00:29:09.860 align:middle line:84% Well, standard form use in simplex method. 00:29:09.860 --> 00:29:13.550 align:middle line:90% 00:29:13.550 --> 00:29:16.470 align:middle line:84% So the simplex method is one of the standard ways 00:29:16.470 --> 00:29:19.890 align:middle line:84% of solving linear programs, and we're not going to go into it, 00:29:19.890 --> 00:29:23.630 align:middle line:84% but as I said, I will sketch you the basic ideas for it 00:29:23.630 --> 00:29:24.870 align:middle line:90% at some point. 00:29:24.870 --> 00:29:32.040 align:middle line:90% And we use this in duality. 00:29:32.040 --> 00:29:35.590 align:middle line:90% 00:29:35.590 --> 00:29:44.750 align:middle line:84% And duality is one of the most important, I guess, 00:29:44.750 --> 00:29:48.310 align:middle line:90% aspects of linear programming. 00:29:48.310 --> 00:29:51.030 align:middle line:84% Well, another very important aspect of linear programming 00:29:51.030 --> 00:29:53.470 align:middle line:84% is that there's an efficient method of solving them, 00:29:53.470 --> 00:29:57.790 align:middle line:84% but as I said, we're not going to explain 00:29:57.790 --> 00:30:00.310 align:middle line:90% that in very much detail. 00:30:00.310 --> 00:30:07.070 align:middle line:84% Because if you're an applied mathematician employed 00:30:07.070 --> 00:30:09.430 align:middle line:84% at some company, you never actually solve 00:30:09.430 --> 00:30:15.480 align:middle line:84% linear programming by hand, you use whatever tool you have 00:30:15.480 --> 00:30:17.700 align:middle line:84% available, which is probably CPLEX 00:30:17.700 --> 00:30:19.240 align:middle line:84% if it's a company that solves a ton 00:30:19.240 --> 00:30:27.480 align:middle line:84% of different linear programs, and you 00:30:27.480 --> 00:30:31.560 align:middle line:84% don't need to know what's underneath the hood in CPLEX. 00:30:31.560 --> 00:30:36.140 align:middle line:84% But if you are an applied mathematician, 00:30:36.140 --> 00:30:39.320 align:middle line:84% you really do need to know how to use duality, 00:30:39.320 --> 00:30:42.450 align:middle line:84% and I we will explain duality in this course. 00:30:42.450 --> 00:31:17.159 align:middle line:90% 00:31:17.159 --> 00:31:21.010 align:middle line:90% So let's write down a theorem. 00:31:21.010 --> 00:31:42.270 align:middle line:84% Every linear program that has a solution, an optimal solution, 00:31:42.270 --> 00:31:51.740 align:middle line:84% has an optimal basic feasible solution. 00:31:51.740 --> 00:31:59.290 align:middle line:90% 00:31:59.290 --> 00:32:00.050 align:middle line:90% OK. 00:32:00.050 --> 00:32:03.610 align:middle line:84% So what is a basic feasible solution? 00:32:03.610 --> 00:32:10.030 align:middle line:84% Well, the easiest way to explain is by looking at this diagram. 00:32:10.030 --> 00:32:15.070 align:middle line:84% Of course, real linear programs, this diagram isn't planar, 00:32:15.070 --> 00:32:18.090 align:middle line:84% it's n-dimensional and impossible to draw 00:32:18.090 --> 00:32:23.930 align:middle line:84% on a blackboard, but the basic idea is the same. 00:32:23.930 --> 00:32:35.050 align:middle line:84% So the basic feasible solutions are basically 00:32:35.050 --> 00:32:42.680 align:middle line:84% the intersection are the extreme points of the feasible region. 00:32:42.680 --> 00:32:46.730 align:middle line:90% 00:32:46.730 --> 00:32:52.730 align:middle line:84% So these are basic feasible solutions. 00:32:52.730 --> 00:32:56.370 align:middle line:90% 00:32:56.370 --> 00:33:00.370 align:middle line:84% And the theorem is that one of these is optimal. 00:33:00.370 --> 00:33:03.430 align:middle line:84% So why are they called basic feasible solutions? 00:33:03.430 --> 00:33:07.130 align:middle line:90% 00:33:07.130 --> 00:33:15.140 align:middle line:84% Well, to explain that, I think I need to go into standard form 00:33:15.140 --> 00:33:17.490 align:middle line:84% and put this linear program in standard form. 00:33:17.490 --> 00:33:20.820 align:middle line:90% 00:33:20.820 --> 00:33:43.860 align:middle line:84% The above LP in standard form is maximize z equals 20x1 plus 20x2 00:33:43.860 --> 00:33:53.600 align:middle line:84% subject to x1 plus 3x2 less than or equal to 9. 00:33:53.600 --> 00:33:54.100 align:middle line:90% x2. 00:33:54.100 --> 00:34:00.020 align:middle line:90% 00:34:00.020 --> 00:34:03.360 align:middle line:90% Or rather, plus s1. 00:34:03.360 --> 00:34:05.580 align:middle line:90% This is our slack variable. 00:34:05.580 --> 00:34:13.190 align:middle line:90% And now we need 2x1 plus x2. 00:34:13.190 --> 00:34:15.190 align:middle line:84% And now we need a different Slack variable 00:34:15.190 --> 00:34:17.230 align:middle line:90% for the second equation. 00:34:17.230 --> 00:34:19.310 align:middle line:84% You need a slack variable for each equation 00:34:19.310 --> 00:34:20.909 align:middle line:90% when you're doing this. 00:34:20.909 --> 00:34:26.960 align:middle line:84% And finally, 1.5x1 is less than or equal to 15. 00:34:26.960 --> 00:34:33.201 align:middle line:90% 00:34:33.201 --> 00:34:38.160 align:middle line:84% And so that would be plus s3 equals 15. 00:34:38.160 --> 00:35:17.590 align:middle line:90% 00:35:17.590 --> 00:35:38.360 align:middle line:84% So a basis here is a linearly independent set of columns. 00:35:38.360 --> 00:35:42.110 align:middle line:90% 00:35:42.110 --> 00:35:53.210 align:middle line:90% So, let's just take this one. 00:35:53.210 --> 00:35:59.696 align:middle line:90% 00:35:59.696 --> 00:36:00.196 align:middle line:90% OK. 00:36:00.196 --> 00:36:16.360 align:middle line:90% So that is x1 plus s1 equals 9. 00:36:16.360 --> 00:36:22.666 align:middle line:90% 2x1 plus s2 equals 8. 00:36:22.666 --> 00:36:28.760 align:middle line:90% 00:36:28.760 --> 00:36:31.270 align:middle line:90% And 1.5x1 plus x3-- 00:36:31.270 --> 00:36:35.880 align:middle line:90% 00:36:35.880 --> 00:36:40.620 align:middle line:84% actually, we're just picking these three columns, 00:36:40.620 --> 00:36:42.040 align:middle line:90% so we leave this one. 00:36:42.040 --> 00:36:45.190 align:middle line:90% So that is equal to 15. 00:36:45.190 --> 00:36:52.800 align:middle line:90% 00:36:52.800 --> 00:36:55.960 align:middle line:84% And for every basis, there's a solution 00:36:55.960 --> 00:37:07.500 align:middle line:84% because this is just a matrix times a vector equals a vector. 00:37:07.500 --> 00:37:10.410 align:middle line:90% 00:37:10.410 --> 00:37:18.250 align:middle line:84% I mean, it's just a set of linear equations with k 00:37:18.250 --> 00:37:22.140 align:middle line:84% equations and k variables, so it has a solution. 00:37:22.140 --> 00:37:29.170 align:middle line:90% 00:37:29.170 --> 00:37:33.730 align:middle line:84% And I guess this solution would be x1 equals 10. 00:37:33.730 --> 00:37:38.860 align:middle line:84% And now you could use this to get x2 equals-- 00:37:38.860 --> 00:37:42.090 align:middle line:90% 00:37:42.090 --> 00:37:47.130 align:middle line:84% that's 20, plus s2, equals 8, negative 12, 00:37:47.130 --> 00:37:51.150 align:middle line:90% and s1 equals minus 1. 00:37:51.150 --> 00:37:54.210 align:middle line:90% 00:37:54.210 --> 00:38:06.730 align:middle line:84% So you can see that this basis is not a feasible solution 00:38:06.730 --> 00:38:15.410 align:middle line:84% because it violates the non-negativity constraints. 00:38:15.410 --> 00:38:22.330 align:middle line:84% But every linear program has a feasible solution, 00:38:22.330 --> 00:38:32.490 align:middle line:84% which the feasible points here correspond to the extreme points 00:38:32.490 --> 00:38:41.010 align:middle line:84% of the feasible region, and the optimal one is the 20x1 plus 00:38:41.010 --> 00:38:48.287 align:middle line:84% 20x2 that is furthest in the direction of the vector, 00:38:48.287 --> 00:38:49.620 align:middle line:90% which is the objective function. 00:38:49.620 --> 00:38:54.370 align:middle line:90% 00:38:54.370 --> 00:38:56.810 align:middle line:84% And what does the linear programming-- 00:38:56.810 --> 00:38:59.180 align:middle line:84% what does the simplex algorithm do? 00:38:59.180 --> 00:39:01.500 align:middle line:84% Well, what the simplex algorithm does 00:39:01.500 --> 00:39:08.620 align:middle line:84% is it starts at one basic feasible solution, which, 00:39:08.620 --> 00:39:11.180 align:middle line:84% in this case, you would use 0, 0, 00:39:11.180 --> 00:39:16.100 align:middle line:84% and it walks along the edges until-- 00:39:16.100 --> 00:39:19.860 align:middle line:84% and it chooses an edge which improves the objective function 00:39:19.860 --> 00:39:21.500 align:middle line:90% every time. 00:39:21.500 --> 00:39:24.260 align:middle line:84% And there's always an edge that improves the objective function 00:39:24.260 --> 00:39:27.100 align:middle line:84% unless you're actually at the optimum. 00:39:27.100 --> 00:39:32.020 align:middle line:84% So it just walks along the edges until it reaches the optimum. 00:39:32.020 --> 00:39:35.140 align:middle line:84% So, is this an efficient algorithm? 00:39:35.140 --> 00:39:41.580 align:middle line:84% Well this is a little bit of history, 00:39:41.580 --> 00:39:46.260 align:middle line:90% which we're not going to-- 00:39:46.260 --> 00:39:49.660 align:middle line:84% we're not going to test you on, but this 00:39:49.660 --> 00:39:57.540 align:middle line:84% was Dantzig's simplex algorithm, just to walk along the edges. 00:39:57.540 --> 00:39:59.080 align:middle line:90% And he proposed it-- 00:39:59.080 --> 00:40:03.060 align:middle line:84% it worked really well in practice, but pretty much 00:40:03.060 --> 00:40:05.100 align:middle line:90% for any-- 00:40:05.100 --> 00:40:07.300 align:middle line:84% I mean, you need some way of choosing 00:40:07.300 --> 00:40:09.140 align:middle line:84% which edge you're going to walk along, 00:40:09.140 --> 00:40:12.100 align:middle line:84% and pretty much for any way of choosing which 00:40:12.100 --> 00:40:15.180 align:middle line:84% edge you want to walk along, people 00:40:15.180 --> 00:40:17.420 align:middle line:84% could come up with a counterexample which 00:40:17.420 --> 00:40:21.980 align:middle line:84% it took exponentially many steps along the edges. 00:40:21.980 --> 00:40:27.060 align:middle line:84% So it works really well in practice. 00:40:27.060 --> 00:40:35.420 align:middle line:84% In the worst case, nobody could prove 00:40:35.420 --> 00:40:38.900 align:middle line:90% that it worked really well. 00:40:38.900 --> 00:40:46.540 align:middle line:84% And this lasted until Dan Spielman and Tang-- 00:40:46.540 --> 00:40:49.820 align:middle line:84% Dan Spielman was a professor at MIT at the time-- 00:40:49.820 --> 00:40:53.140 align:middle line:84% invented a new technique called smoothed analysis 00:40:53.140 --> 00:40:59.150 align:middle line:84% that says that it works really well almost all the time. 00:40:59.150 --> 00:41:05.910 align:middle line:84% So, that is probably all we're going to say on the simplex 00:41:05.910 --> 00:41:06.570 align:middle line:90% algorithm. 00:41:06.570 --> 00:41:18.710 align:middle line:90% 00:41:18.710 --> 00:41:20.030 align:middle line:90% OK. 00:41:20.030 --> 00:41:22.660 align:middle line:84% So now I want to talk about duality. 00:41:22.660 --> 00:41:28.550 align:middle line:90% 00:41:28.550 --> 00:41:39.710 align:middle line:84% And we're going to use this linear program as an example. 00:41:39.710 --> 00:41:45.070 align:middle line:84% And actually, as you can see from that diagram, 00:41:45.070 --> 00:41:47.690 align:middle line:84% we could just get rid of this constraint, 00:41:47.690 --> 00:41:49.670 align:middle line:84% it doesn't make any difference, and just 00:41:49.670 --> 00:41:51.770 align:middle line:84% look at these first two constraints. 00:41:51.770 --> 00:41:58.880 align:middle line:90% 00:41:58.880 --> 00:42:01.180 align:middle line:84% So what we're going to do is we're going to ask, 00:42:01.180 --> 00:42:05.200 align:middle line:84% how can we find a lower bound on the optimum 00:42:05.200 --> 00:42:08.600 align:middle line:84% and how can we find an upper bound on the optimum? 00:42:08.600 --> 00:42:14.400 align:middle line:84% And then we will show that these two bounds actually 00:42:14.400 --> 00:42:18.980 align:middle line:84% will end up being equal under some conditions. 00:42:18.980 --> 00:42:19.910 align:middle line:90% So, lower bound. 00:42:19.910 --> 00:42:22.520 align:middle line:90% 00:42:22.520 --> 00:42:31.050 align:middle line:84% Any feasible solution gives a lower bound. 00:42:31.050 --> 00:42:37.880 align:middle line:90% 00:42:37.880 --> 00:42:42.120 align:middle line:84% So let's take a feasible solution for that one. 00:42:42.120 --> 00:42:45.760 align:middle line:90% How about x1 equals x2 equals 2? 00:42:45.760 --> 00:42:48.640 align:middle line:84% That satisfies all the constraints. 00:42:48.640 --> 00:42:51.310 align:middle line:90% x1 equals x2 equals 2. 00:42:51.310 --> 00:42:54.280 align:middle line:90% 00:42:54.280 --> 00:43:05.520 align:middle line:84% And is that 2 plus 6 is 8 and 4 plus 2 is 6, 00:43:05.520 --> 00:43:07.820 align:middle line:90% and those are satisfied. 00:43:07.820 --> 00:43:12.240 align:middle line:90% 00:43:12.240 --> 00:43:20.240 align:middle line:84% And the objective function is 20 times 2 plus 20 times 2 00:43:20.240 --> 00:43:22.190 align:middle line:90% is equal to 80. 00:43:22.190 --> 00:43:26.680 align:middle line:90% 00:43:26.680 --> 00:43:29.620 align:middle line:84% So, that is one way you could do this. 00:43:29.620 --> 00:43:33.480 align:middle line:84% And I should say that for this linear program, 00:43:33.480 --> 00:43:35.840 align:middle line:84% even though I've called them batches, 00:43:35.840 --> 00:43:39.480 align:middle line:84% you have to assume that you can use fractional numbers for x1 00:43:39.480 --> 00:43:43.120 align:middle line:84% and x2, otherwise you get an integer program and not 00:43:43.120 --> 00:43:47.320 align:middle line:84% a linear program, and those are much more complicated. 00:43:47.320 --> 00:43:48.800 align:middle line:90% OK. 00:43:48.800 --> 00:43:52.078 align:middle line:84% So any feasible solution gives you a lower bound. 00:43:52.078 --> 00:43:53.120 align:middle line:90% How about an upper bound? 00:43:53.120 --> 00:44:01.730 align:middle line:90% 00:44:01.730 --> 00:44:06.210 align:middle line:84% So, I'm going to write down something. 00:44:06.210 --> 00:44:09.390 align:middle line:90% 20x1 plus x2. 00:44:09.390 --> 00:44:09.890 align:middle line:90% OK. 00:44:09.890 --> 00:44:12.730 align:middle line:90% 00:44:12.730 --> 00:44:22.410 align:middle line:84% Is certainly less than 20x1 plus 3x2 because we have-- 00:44:22.410 --> 00:44:25.370 align:middle line:90% well, because x2 is positive. 00:44:25.370 --> 00:44:29.330 align:middle line:84% And now we have x1 plus 3x2 is less than or equal to 9. 00:44:29.330 --> 00:44:35.930 align:middle line:84% So this is less than or equal to 20 times 9 equals 180. 00:44:35.930 --> 00:44:41.010 align:middle line:84% And we can do the same thing for the second constraint. 00:44:41.010 --> 00:44:53.580 align:middle line:84% 20x1 plus x2 is less than or equal to 20 times 2x1 plus x2 is 00:44:53.580 --> 00:44:59.460 align:middle line:90% less than or equal to 160. 00:44:59.460 --> 00:45:02.660 align:middle line:84% So these are both good upper bounds, 00:45:02.660 --> 00:45:05.300 align:middle line:90% and this is a good lower bound. 00:45:05.300 --> 00:45:08.100 align:middle line:90% Can we do better? 00:45:08.100 --> 00:45:09.480 align:middle line:90% OK, let's do better. 00:45:09.480 --> 00:45:31.620 align:middle line:90% 00:45:31.620 --> 00:45:34.380 align:middle line:90% Let's add this and this. 00:45:34.380 --> 00:45:38.680 align:middle line:90% We have two-- let's see. 00:45:38.680 --> 00:45:41.500 align:middle line:90% We have 3x1 plus 4x2. 00:45:41.500 --> 00:45:51.140 align:middle line:90% 00:45:51.140 --> 00:45:55.020 align:middle line:84% So we're adding these two constraints. 00:45:55.020 --> 00:45:56.290 align:middle line:90% It's less than or equal to 17. 00:45:56.290 --> 00:46:04.620 align:middle line:90% 00:46:04.620 --> 00:46:08.650 align:middle line:90% And we have 20/3. 00:46:08.650 --> 00:46:13.260 align:middle line:90% 00:46:13.260 --> 00:46:14.660 align:middle line:90% Oh. 00:46:14.660 --> 00:46:26.420 align:middle line:84% 20x1 plus x2 is less than or equal to 20/3 times 3x1 plus 4x2 00:46:26.420 --> 00:46:32.640 align:middle line:84% is less than or equal to 20/3 times 17, 00:46:32.640 --> 00:46:36.820 align:middle line:90% which is equal to 113 and 1/3. 00:46:36.820 --> 00:46:39.460 align:middle line:90% 00:46:39.460 --> 00:46:41.180 align:middle line:90% OK. 00:46:41.180 --> 00:46:43.540 align:middle line:84% So, we've got a much better upper bound 00:46:43.540 --> 00:46:48.070 align:middle line:90% by adding these two constraints. 00:46:48.070 --> 00:46:54.510 align:middle line:84% So what happens if we find the best some of these constraints? 00:46:54.510 --> 00:46:55.530 align:middle line:90% How does it look? 00:46:55.530 --> 00:47:06.110 align:middle line:90% 00:47:06.110 --> 00:47:10.030 align:middle line:84% OK, so I want to go about this more systematically 00:47:10.030 --> 00:47:15.510 align:middle line:84% so we can find the dual-linear program. 00:47:15.510 --> 00:47:18.190 align:middle line:84% Every linear program has a dual-linear program, 00:47:18.190 --> 00:47:21.490 align:middle line:90% and we want to do this. 00:47:21.490 --> 00:47:38.630 align:middle line:84% So, we have-- and the way we will do this is we will assume 00:47:38.630 --> 00:47:42.230 align:middle line:84% that the linear program is canonical-- 00:47:42.230 --> 00:47:44.880 align:middle line:84% in canonical form for the time being. 00:47:44.880 --> 00:47:46.760 align:middle line:84% And luckily, this linear program just 00:47:46.760 --> 00:47:50.400 align:middle line:90% happens to be in canonical form. 00:47:50.400 --> 00:47:53.440 align:middle line:90% So how do we do this? 00:47:53.440 --> 00:47:58.540 align:middle line:84% Well, what we want to do is we want a combination of-- 00:47:58.540 --> 00:48:03.120 align:middle line:90% 00:48:03.120 --> 00:48:10.560 align:middle line:84% a combination of these constraints which will let us 00:48:10.560 --> 00:48:13.120 align:middle line:84% give a best upper bound possible. 00:48:13.120 --> 00:48:20.320 align:middle line:84% So let's say we have y1 times our first constraint. 00:48:20.320 --> 00:48:22.900 align:middle line:84% x1 plus 3x2 is less than or equal to 9. 00:48:22.900 --> 00:48:30.520 align:middle line:90% 00:48:30.520 --> 00:48:34.600 align:middle line:84% And y2 times our second constraint, 00:48:34.600 --> 00:48:38.900 align:middle line:84% that's 2x1 plus x2 is less than or equal to 8. 00:48:38.900 --> 00:48:45.600 align:middle line:90% 00:48:45.600 --> 00:48:56.600 align:middle line:84% And now, we want this combination to be less than 00:48:56.600 --> 00:48:57.740 align:middle line:90% or equal to-- 00:48:57.740 --> 00:49:02.200 align:middle line:90% 00:49:02.200 --> 00:49:04.280 align:middle line:90% all right. 00:49:04.280 --> 00:49:07.760 align:middle line:84% We want 20x1 plus x2 to be bigger than-- 00:49:07.760 --> 00:49:10.880 align:middle line:84% to be less than or equal to our combination. 00:49:10.880 --> 00:49:17.680 align:middle line:84% So we want the combination to be bigger than or equal to 20x1 00:49:17.680 --> 00:49:20.160 align:middle line:90% plus 20x2. 00:49:20.160 --> 00:49:34.320 align:middle line:84% So that tells us that y1 times 1 plus y2 times 2 had better 00:49:34.320 --> 00:49:35.290 align:middle line:90% be bigger than 20. 00:49:35.290 --> 00:49:38.600 align:middle line:90% 00:49:38.600 --> 00:49:46.320 align:middle line:84% And 3y1 plus y2 is also bigger than or equal to 20. 00:49:46.320 --> 00:49:55.210 align:middle line:90% 00:49:55.210 --> 00:49:59.320 align:middle line:84% So suppose we found y's like this, what would we have? 00:49:59.320 --> 00:50:09.130 align:middle line:90% 00:50:09.130 --> 00:50:13.290 align:middle line:90% We would have y1x1-- 00:50:13.290 --> 00:50:19.370 align:middle line:90% 00:50:19.370 --> 00:50:29.890 align:middle line:84% OK, y1 plus 2y2 x1 plus 3y1 plus-- 00:50:29.890 --> 00:50:42.520 align:middle line:84% 3y1 plus y2 x2 is greater than or equal to 20x1 plus 20x2. 00:50:42.520 --> 00:50:46.820 align:middle line:90% 00:50:46.820 --> 00:50:52.540 align:middle line:90% So this would be our things. 00:50:52.540 --> 00:50:58.900 align:middle line:84% And this is, in turn, less than or equal to, I want to say, 00:50:58.900 --> 00:51:00.260 align:middle line:90% 9y1 plus 8y2. 00:51:00.260 --> 00:51:18.900 align:middle line:90% 00:51:18.900 --> 00:51:23.820 align:middle line:84% So to get the best bound on 20x1 plus 20x2, 00:51:23.820 --> 00:51:27.740 align:middle line:84% we need to minimize 9y1 plus 8y2. 00:51:27.740 --> 00:51:40.180 align:middle line:84% So we want to minimize 9y1 plus 8y2 subject to y1 plus 2y2 00:51:40.180 --> 00:51:51.900 align:middle line:84% greater than or equal to 20, and 3y1 plus y2 greater than 00:51:51.900 --> 00:51:54.340 align:middle line:90% or equal to 20. 00:51:54.340 --> 00:51:58.980 align:middle line:84% And y1, y2 greater than or equal to 0. 00:51:58.980 --> 00:52:02.340 align:middle line:84% So why y1, y2 greater than or equal to 0? 00:52:02.340 --> 00:52:07.140 align:middle line:84% Well, I mean, if y2 was less than 0, 00:52:07.140 --> 00:52:11.430 align:middle line:84% then when we add these equalities up, we have-- 00:52:11.430 --> 00:52:18.022 align:middle line:90% 00:52:18.022 --> 00:52:19.980 align:middle line:84% the less than or equal to-- if y2 was less than 00:52:19.980 --> 00:52:22.140 align:middle line:84% or equal to 0, the less than or equal to sign 00:52:22.140 --> 00:52:24.160 align:middle line:84% would turn into a greater than or equal to sign, 00:52:24.160 --> 00:52:27.700 align:middle line:84% and adding these inequalities up would not make any sense. 00:52:27.700 --> 00:52:32.300 align:middle line:84% So we need the yi is greater than or equal to 0. 00:52:32.300 --> 00:52:34.560 align:middle line:84% So what did what did we get here? 00:52:34.560 --> 00:52:37.750 align:middle line:90% 00:52:37.750 --> 00:52:38.850 align:middle line:90% What is this thing? 00:52:38.850 --> 00:52:44.270 align:middle line:90% 00:52:44.270 --> 00:52:46.310 align:middle line:90% It's another linear program. 00:52:46.310 --> 00:52:50.260 align:middle line:84% So this is the dual-linear program. 00:52:50.260 --> 00:52:55.630 align:middle line:90% 00:52:55.630 --> 00:52:57.240 align:middle line:90% And now I want to write down-- 00:52:57.240 --> 00:53:01.110 align:middle line:90% 00:53:01.110 --> 00:53:04.430 align:middle line:84% I want to write down the dual-linear program 00:53:04.430 --> 00:53:12.590 align:middle line:84% for an arbitrary program in canonical form, 00:53:12.590 --> 00:53:20.700 align:middle line:84% and then we will show the theorem of weak duality. 00:53:20.700 --> 00:53:26.910 align:middle line:90% 00:53:26.910 --> 00:53:28.210 align:middle line:90% But maybe I want to-- 00:53:28.210 --> 00:53:32.190 align:middle line:90% 00:53:32.190 --> 00:53:34.480 align:middle line:84% maybe I want to solve these linear problems first. 00:53:34.480 --> 00:53:58.670 align:middle line:90% 00:53:58.670 --> 00:54:17.200 align:middle line:90% So, x1 equals 3, x2 equals 2. 00:54:17.200 --> 00:54:20.150 align:middle line:90% 00:54:20.150 --> 00:54:21.340 align:middle line:90% This is a feasible point. 00:54:21.340 --> 00:54:25.350 align:middle line:90% 00:54:25.350 --> 00:54:28.410 align:middle line:90% x1 equals 3, x2 equals 2. 00:54:28.410 --> 00:54:34.055 align:middle line:84% That's 3 plus 2 times 3 is less than or equal to 9, and 6 plus 2 00:54:34.055 --> 00:54:35.180 align:middle line:90% is less than or equal to 8. 00:54:35.180 --> 00:54:37.400 align:middle line:90% So this is a feasible point. 00:54:37.400 --> 00:54:50.120 align:middle line:84% And objective function is 20 times 5 equals 100. 00:54:50.120 --> 00:54:51.980 align:middle line:90% Now, let's solve this one. 00:54:51.980 --> 00:54:57.440 align:middle line:90% 00:54:57.440 --> 00:55:00.800 align:middle line:84% OK, I did not write down the solution here, 00:55:00.800 --> 00:55:05.140 align:middle line:84% but there is a solution to this linear program, 00:55:05.140 --> 00:55:08.600 align:middle line:84% so the objective function is also 100. 00:55:08.600 --> 00:55:15.180 align:middle line:84% And the dual program gave you an upper bound-- 00:55:15.180 --> 00:55:17.480 align:middle line:84% and the feasible point in the dual program 00:55:17.480 --> 00:55:19.240 align:middle line:84% gives you an upper bound, a feasible point 00:55:19.240 --> 00:55:22.480 align:middle line:84% in the primal program gives you a lower bound. 00:55:22.480 --> 00:55:25.880 align:middle line:84% So because this 100 is equal to that 100, 00:55:25.880 --> 00:55:46.930 align:middle line:84% then we know that the answer is 100. 00:55:46.930 --> 00:55:54.960 align:middle line:84% So let's try y1 equals 4, y2 equals 8 4 plus 16 is 20. 00:55:54.960 --> 00:55:57.930 align:middle line:90% 12 plus 8 is 20. 00:55:57.930 --> 00:56:07.570 align:middle line:84% And 9 times 4 plus 8 times 8 is equal to 100. 00:56:07.570 --> 00:56:11.170 align:middle line:84% So here is a feasible point for the dual program, 00:56:11.170 --> 00:56:13.820 align:middle line:84% which also gives the optimal objective value. 00:56:13.820 --> 00:56:39.410 align:middle line:90% 00:56:39.410 --> 00:56:41.250 align:middle line:90% OK. 00:56:41.250 --> 00:56:59.130 align:middle line:90% So, canonical form dual. 00:56:59.130 --> 00:57:02.330 align:middle line:90% 00:57:02.330 --> 00:57:11.290 align:middle line:84% Well, the primal was maximize z equals 00:57:11.290 --> 00:57:21.075 align:middle line:84% c transpose x subject to Ax less than or equal to b and x 00:57:21.075 --> 00:57:22.200 align:middle line:90% greater than or equal to 0. 00:57:22.200 --> 00:57:26.330 align:middle line:90% 00:57:26.330 --> 00:57:34.500 align:middle line:84% Well, the dual turns out to be minimized y-- 00:57:34.500 --> 00:57:45.540 align:middle line:84% I'm sorry, w equals b transpose y subject to A transpose y 00:57:45.540 --> 00:57:50.700 align:middle line:84% is greater than or equal to c, y greater than or equal to 0. 00:57:50.700 --> 00:57:52.860 align:middle line:84% And you can see why this is maybe, 00:57:52.860 --> 00:57:59.880 align:middle line:84% by looking at this example, when we did this example, 00:57:59.880 --> 00:58:07.868 align:middle line:84% we had the original linear program had the matrix A, 1, 3, 00:58:07.868 --> 00:58:09.900 align:middle line:90% 2, 1. 00:58:09.900 --> 00:58:14.780 align:middle line:84% This now has the matrix A equals 1, 2, 3, 1, 00:58:14.780 --> 00:58:27.640 align:middle line:84% and this 2 came from the Aij A 1, 2 thing-- 00:58:27.640 --> 00:58:39.670 align:middle line:84% A 2, 1 entry of A, and now it's the A 1, 2 entry of the A. 00:58:39.670 --> 00:58:41.470 align:middle line:90% So this is just-- 00:58:41.470 --> 00:58:45.230 align:middle line:84% this matrix is just the transpose of the original. 00:58:45.230 --> 00:58:52.310 align:middle line:84% The right-hand sides are just the original objective function. 00:58:52.310 --> 00:58:59.710 align:middle line:84% And the new objective function is just the right-hand sides 00:58:59.710 --> 00:59:03.350 align:middle line:90% of these inequalities. 00:59:03.350 --> 00:59:17.070 align:middle line:84% So we have A transpose instead of A. 00:59:17.070 --> 00:59:21.990 align:middle line:84% The objective function gets turned into the right-hand side 00:59:21.990 --> 00:59:24.230 align:middle line:84% of the constraints, and the right-hand side 00:59:24.230 --> 00:59:27.280 align:middle line:84% of the constraints gets turned into the new objective function. 00:59:27.280 --> 00:59:30.310 align:middle line:90% 00:59:30.310 --> 00:59:30.810 align:middle line:90% OK. 00:59:30.810 --> 00:59:53.070 align:middle line:90% 00:59:53.070 --> 00:59:53.830 align:middle line:90% OK. 00:59:53.830 --> 00:59:57.960 align:middle line:84% So now, I will prove something called weak duality. 00:59:57.960 --> 01:00:08.510 align:middle line:90% 01:00:08.510 --> 01:00:13.200 align:middle line:90% Theorem, weak duality. 01:00:13.200 --> 01:00:22.110 align:middle line:90% 01:00:22.110 --> 01:00:36.150 align:middle line:84% If x is a feasible solution of the primal 01:00:36.150 --> 01:00:53.640 align:middle line:84% with objective function z equals c transpose x, and y 01:00:53.640 --> 01:01:14.280 align:middle line:84% is a feasible solution of dual with objective function 01:01:14.280 --> 01:01:15.490 align:middle line:90% w equals. 01:01:15.490 --> 01:01:18.600 align:middle line:90% 01:01:18.600 --> 01:01:32.640 align:middle line:84% b transpose y, then z is less than or equal to w. 01:01:32.640 --> 01:01:37.210 align:middle line:90% 01:01:37.210 --> 01:01:40.930 align:middle line:84% So any feasible solution of the primal 01:01:40.930 --> 01:01:45.000 align:middle line:84% is less than any feasible solution of the dual. 01:01:45.000 --> 01:01:53.050 align:middle line:90% 01:01:53.050 --> 01:01:58.770 align:middle line:84% OK, there's also something called strong duality, which 01:01:58.770 --> 01:02:00.910 align:middle line:84% we're not going to have a chance to prove today, 01:02:00.910 --> 01:02:03.610 align:middle line:84% we will do that next time, but I'm 01:02:03.610 --> 01:02:05.930 align:middle line:90% going to tell you what it is. 01:02:05.930 --> 01:02:07.940 align:middle line:84% And then we'll give the proof of weak duality. 01:02:07.940 --> 01:02:10.530 align:middle line:90% 01:02:10.530 --> 01:02:13.850 align:middle line:90% So strong duality is really-- 01:02:13.850 --> 01:02:17.440 align:middle line:84% I want to say this is one of the great-- 01:02:17.440 --> 01:02:21.290 align:middle line:90% 01:02:21.290 --> 01:02:24.370 align:middle line:84% one of the most important theorems 01:02:24.370 --> 01:02:26.410 align:middle line:90% in combinatorial optimization. 01:02:26.410 --> 01:02:37.070 align:middle line:90% Strong duality theorem. 01:02:37.070 --> 01:02:43.130 align:middle line:90% 01:02:43.130 --> 01:03:01.130 align:middle line:84% If the optimum z star is finite, so is w star, 01:03:01.130 --> 01:03:06.490 align:middle line:90% and z star is equal to w star. 01:03:06.490 --> 01:03:11.290 align:middle line:84% So weak duality says that for any feasible solution, 01:03:11.290 --> 01:03:17.490 align:middle line:84% the value for the primal is less than or equal to the value 01:03:17.490 --> 01:03:19.010 align:middle line:90% for the dual. 01:03:19.010 --> 01:03:29.380 align:middle line:84% Strong duality says that if the optimal values are finite, then 01:03:29.380 --> 01:03:30.230 align:middle line:90% they are equal. 01:03:30.230 --> 01:03:47.060 align:middle line:90% 01:03:47.060 --> 01:03:48.660 align:middle line:90% OK. 01:03:48.660 --> 01:03:51.290 align:middle line:84% So, how do we prove weak duality? 01:03:51.290 --> 01:03:54.700 align:middle line:90% 01:03:54.700 --> 01:04:04.950 align:middle line:84% So, we have z equals proof of weak duality. 01:04:04.950 --> 01:04:07.940 align:middle line:90% 01:04:07.940 --> 01:04:13.690 align:middle line:90% We have z equals c transpose x. 01:04:13.690 --> 01:04:17.950 align:middle line:90% 01:04:17.950 --> 01:04:28.070 align:middle line:84% Recall A transpose y was greater than or equal to c. 01:04:28.070 --> 01:04:30.050 align:middle line:90% So that's one of our-- 01:04:30.050 --> 01:04:36.070 align:middle line:90% 01:04:36.070 --> 01:04:38.773 align:middle line:84% that's our constraints of the dual. 01:04:38.773 --> 01:04:40.190 align:middle line:84% So what we're going to do is we're 01:04:40.190 --> 01:04:43.420 align:middle line:84% going to plug this into here, and we get. 01:04:43.420 --> 01:04:47.390 align:middle line:90% 01:04:47.390 --> 01:04:55.710 align:middle line:84% c transpose x is less than or equal to A transpose y 01:04:55.710 --> 01:05:01.610 align:middle line:84% transpose x, which is equal to y transpose Ax. 01:05:01.610 --> 01:05:09.350 align:middle line:90% 01:05:09.350 --> 01:05:16.830 align:middle line:84% And now, well, do we have something about Ax there? 01:05:16.830 --> 01:05:19.290 align:middle line:84% Up there, we have Ax less than or equal to b, 01:05:19.290 --> 01:05:22.190 align:middle line:84% so let's plug that into here, and that's less than 01:05:22.190 --> 01:05:25.350 align:middle line:90% or equal to y transpose b. 01:05:25.350 --> 01:05:27.170 align:middle line:90% And what is y transpose b? 01:05:27.170 --> 01:05:30.750 align:middle line:90% 01:05:30.750 --> 01:05:36.110 align:middle line:84% y transpose b, well, that's just actually 01:05:36.110 --> 01:05:43.830 align:middle line:84% the transpose of b transpose y transpose. 01:05:43.830 --> 01:05:57.650 align:middle line:84% And b transpose y was w, so this is w because w is a scalar, 01:05:57.650 --> 01:06:00.670 align:middle line:84% so transposing it doesn't make any difference. 01:06:00.670 --> 01:06:05.950 align:middle line:84% So that says that z is less than or equal to this thing, which 01:06:05.950 --> 01:06:08.930 align:middle line:84% is less than or equal to this thing, which is equal to w, 01:06:08.930 --> 01:06:10.500 align:middle line:90% so z is less than w. 01:06:10.500 --> 01:06:26.240 align:middle line:90% 01:06:26.240 --> 01:06:27.160 align:middle line:90% OK. 01:06:27.160 --> 01:06:31.200 align:middle line:84% There's two more things I want to do today, 01:06:31.200 --> 01:06:37.360 align:middle line:84% although I'm not entirely sure we'll get by more than one 01:06:37.360 --> 01:06:37.940 align:middle line:90% of them. 01:06:37.940 --> 01:06:48.920 align:middle line:90% 01:06:48.920 --> 01:06:54.490 align:middle line:84% Theorem, the dual of the dual is the primal. 01:06:54.490 --> 01:07:04.960 align:middle line:90% 01:07:04.960 --> 01:07:06.840 align:middle line:84% And really, I think probably all of you 01:07:06.840 --> 01:07:10.560 align:middle line:90% were expecting this to be true. 01:07:10.560 --> 01:07:18.480 align:middle line:90% So the primal is the maximum. 01:07:18.480 --> 01:07:28.080 align:middle line:84% z equals c transpose x such that-- 01:07:28.080 --> 01:07:32.880 align:middle line:84% subject to, sorry, Ax is less than or equal to b and x 01:07:32.880 --> 01:07:35.400 align:middle line:90% greater than or equal to 0. 01:07:35.400 --> 01:07:41.800 align:middle line:84% Now the dual is minimum of w equals 01:07:41.800 --> 01:07:54.120 align:middle line:84% b transpose y subject to A transpose y 01:07:54.120 --> 01:07:57.310 align:middle line:84% greater than or equal to c, y greater than or equal to 0. 01:07:57.310 --> 01:08:13.810 align:middle line:90% 01:08:13.810 --> 01:08:18.050 align:middle line:84% So, before we can go on, we want to put the dual 01:08:18.050 --> 01:08:21.050 align:middle line:84% in the canonical form, because I haven't told you 01:08:21.050 --> 01:08:22.569 align:middle line:84% how to take duals of anything that's 01:08:22.569 --> 01:08:24.069 align:middle line:90% not in the canonical form. 01:08:24.069 --> 01:08:29.029 align:middle line:84% So one way you can take duals of an arbitrary linear program 01:08:29.029 --> 01:08:32.529 align:middle line:84% is, first, put into canonical form and then take the dual. 01:08:32.529 --> 01:08:36.970 align:middle line:84% Another way is to go back and look at the proof 01:08:36.970 --> 01:08:40.090 align:middle line:84% and figure out the formula for taking the dual of an arbitrary 01:08:40.090 --> 01:08:44.170 align:middle line:84% linear program, and we'll do that next time. 01:08:44.170 --> 01:08:49.910 align:middle line:84% But let's put this in canonical form. 01:08:49.910 --> 01:09:00.170 align:middle line:84% So how do we turn a minimization into a maximization? 01:09:00.170 --> 01:09:01.729 align:middle line:90% We saw this already. 01:09:01.729 --> 01:09:03.350 align:middle line:90% We just take the negative. 01:09:03.350 --> 01:09:04.350 align:middle line:90% So we need to-- 01:09:04.350 --> 01:09:08.930 align:middle line:90% 01:09:08.930 --> 01:09:13.500 align:middle line:84% so what we need to do is we need to maximize 01:09:13.500 --> 01:09:21.180 align:middle line:84% minus b transpose y subject to our-- 01:09:21.180 --> 01:09:23.700 align:middle line:90% yeah. 01:09:23.700 --> 01:09:27.660 align:middle line:84% Well, this inequality is the wrong way around 01:09:27.660 --> 01:09:35.340 align:middle line:84% for a canonical form, so let's take the negative of it 01:09:35.340 --> 01:09:40.580 align:middle line:84% subject to minus A transpose y is less than 01:09:40.580 --> 01:09:43.740 align:middle line:90% or equal to minus c. 01:09:43.740 --> 01:09:45.479 align:middle line:84% And y greater than or equal to 0. 01:09:45.479 --> 01:09:48.460 align:middle line:90% 01:09:48.460 --> 01:09:51.420 align:middle line:90% So this is the canonical form. 01:09:51.420 --> 01:10:02.060 align:middle line:84% So all we do is we want to take the transpose of A 01:10:02.060 --> 01:10:08.980 align:middle line:90% and put this up here. 01:10:08.980 --> 01:10:33.865 align:middle line:84% So the dual is min minus c transpose x subject to. 01:10:33.865 --> 01:10:36.780 align:middle line:84% OK, we need to take the transpose of this. 01:10:36.780 --> 01:10:45.260 align:middle line:84% That gives us a minus Ax is greater than 01:10:45.260 --> 01:10:49.160 align:middle line:90% or equal to minus b. 01:10:49.160 --> 01:10:53.740 align:middle line:90% 01:10:53.740 --> 01:10:56.380 align:middle line:84% But now, if we just take the negative of this 01:10:56.380 --> 01:11:00.180 align:middle line:84% and the negative of this, we get back to the original thing 01:11:00.180 --> 01:11:04.140 align:middle line:84% maximize z equals c transpose x such that Ax is less than 01:11:04.140 --> 01:11:06.700 align:middle line:84% or equal to b rather than minus Ax is 01:11:06.700 --> 01:11:09.190 align:middle line:84% greater than or equal to minus b and x greater than 01:11:09.190 --> 01:11:09.780 align:middle line:90% or equal to 0. 01:11:09.780 --> 01:11:14.870 align:middle line:90% 01:11:14.870 --> 01:11:16.710 align:middle line:84% So, the transpose of the transpose 01:11:16.710 --> 01:11:22.860 align:middle line:84% is the dual, which, we know, makes-- 01:11:22.860 --> 01:11:27.590 align:middle line:90% 01:11:27.590 --> 01:11:30.550 align:middle line:90% which justifies the name. 01:11:30.550 --> 01:11:33.610 align:middle line:84% Sorry, the dual of the dual is the primal, 01:11:33.610 --> 01:11:35.000 align:middle line:90% which justifies the name dual. 01:11:35.000 --> 01:11:37.650 align:middle line:90% 01:11:37.650 --> 01:11:38.150 align:middle line:90% OK. 01:11:38.150 --> 01:11:40.910 align:middle line:90% 01:11:40.910 --> 01:11:43.710 align:middle line:84% I have four minutes left, and I don't 01:11:43.710 --> 01:11:45.602 align:middle line:84% know if I can say anything reasonable in four 01:11:45.602 --> 01:11:47.560 align:middle line:84% minutes, so are there any questions about this? 01:11:47.560 --> 01:11:50.930 align:middle line:90% 01:11:50.930 --> 01:11:51.430 align:middle line:90% OK. 01:11:51.430 --> 01:11:58.190 align:middle line:84% So one thing you do when you first 01:11:58.190 --> 01:12:04.310 align:middle line:84% see a linear program, if you're doing research, is take the dual 01:12:04.310 --> 01:12:08.080 align:middle line:84% and stare at it and see if it makes any sense intuitively. 01:12:08.080 --> 01:12:09.600 align:middle line:84% There are lots of linear problems 01:12:09.600 --> 01:12:12.920 align:middle line:84% where if you take the dual and you stare at it, 01:12:12.920 --> 01:12:14.760 align:middle line:84% it makes sense intuitively, and you 01:12:14.760 --> 01:12:18.840 align:middle line:84% understand why the dual is an upper bound 01:12:18.840 --> 01:12:20.880 align:middle line:90% on the optimum of the primal. 01:12:20.880 --> 01:12:24.320 align:middle line:84% There are also lots of other linear problems when you get it. 01:12:24.320 --> 01:12:27.840 align:middle line:84% You take the dual, you stare at it, and you scratch your head 01:12:27.840 --> 01:12:32.200 align:middle line:84% and you say, why is this a bound on the primal? 01:12:32.200 --> 01:12:34.980 align:middle line:84% And, I mean, it is because it's the dual, 01:12:34.980 --> 01:12:37.550 align:middle line:84% but it's not intuitively obvious. 01:12:37.550 --> 01:12:42.656 align:middle line:90% 01:12:42.656 --> 01:12:46.340 align:middle line:84% And some of those, if you stare at it for longer, 01:12:46.340 --> 01:12:48.240 align:middle line:90% you understand what's going on. 01:12:48.240 --> 01:12:51.020 align:middle line:84% And some of them, you can stare at it for as long as you want, 01:12:51.020 --> 01:12:53.720 align:middle line:84% and you probably still won't understand. 01:12:53.720 --> 01:13:00.480 align:middle line:84% So we're going to see illustrations of, 01:13:00.480 --> 01:13:06.640 align:middle line:90% I think, after the quiz-- 01:13:06.640 --> 01:13:09.560 align:middle line:90% or maybe before the quiz-- 01:13:09.560 --> 01:13:10.320 align:middle line:90% yeah. 01:13:10.320 --> 01:13:19.920 align:middle line:84% We will see illustrations of this kind of primal program 01:13:19.920 --> 01:13:21.180 align:middle line:90% making intuitive sense. 01:13:21.180 --> 01:13:27.320 align:middle line:84% And then the dual makes also intuitive sense, 01:13:27.320 --> 01:13:30.480 align:middle line:84% is clearly an upper bound on the primal. 01:13:30.480 --> 01:13:33.960 align:middle line:84% So you're going to see that in, I guess, 01:13:33.960 --> 01:13:40.680 align:middle line:84% zero-sum games and in probably max flow. 01:13:40.680 --> 01:13:44.560 align:middle line:84% So, I will see everybody on Thursday, 01:13:44.560 --> 01:13:50.480 align:middle line:84% and we will talk more about linear programs and duals. 01:13:50.480 --> 01:13:53.800 align:middle line:84% And I will put out a homework tonight, which I 01:13:53.800 --> 01:13:56.670 align:middle line:90% think will be due next Tuesday. 01:13:56.670 --> 01:14:04.000 align:middle line:90%