WEBVTT 00:00:00.000 --> 00:00:01.972 align:middle line:90% [SQUEAKING] 00:00:01.972 --> 00:00:03.944 align:middle line:90% [RUSTLING] 00:00:03.944 --> 00:00:06.409 align:middle line:90% [CLICKING] 00:00:06.409 --> 00:00:11.560 align:middle line:90% 00:00:11.560 --> 00:00:15.680 align:middle line:84% PETER SHOR: So last time we defined canonical forms 00:00:15.680 --> 00:00:26.680 align:middle line:84% of a linear program and their maximize c 00:00:26.680 --> 00:00:32.880 align:middle line:84% transpose x where these are vectors with such that-- 00:00:32.880 --> 00:00:39.800 align:middle line:84% sorry, subject to Ax is less than or equal to b, 00:00:39.800 --> 00:00:42.390 align:middle line:84% and x is greater than or equal to 0. 00:00:42.390 --> 00:00:47.120 align:middle line:90% 00:00:47.120 --> 00:00:49.090 align:middle line:84% And the dual for the canonical form. 00:00:49.090 --> 00:01:00.230 align:middle line:90% 00:01:00.230 --> 00:01:09.510 align:middle line:84% Dual, we showed was minimum b transpose y. 00:01:09.510 --> 00:01:12.270 align:middle line:84% So we've taken the right-hand sides of the equation 00:01:12.270 --> 00:01:16.090 align:middle line:84% and turned them into the objective function. 00:01:16.090 --> 00:01:21.870 align:middle line:84% A transpose y is greater than or equal to c. 00:01:21.870 --> 00:01:26.270 align:middle line:84% So we've taken the objective function of the primal 00:01:26.270 --> 00:01:29.670 align:middle line:84% and turned it into the right-hand side 00:01:29.670 --> 00:01:32.030 align:middle line:90% of the equations of the dual. 00:01:32.030 --> 00:01:34.400 align:middle line:84% And y greater than or equal to 0. 00:01:34.400 --> 00:01:39.230 align:middle line:90% 00:01:39.230 --> 00:01:51.190 align:middle line:84% And last time, we've showed weak duality 00:01:51.190 --> 00:02:02.750 align:middle line:84% for any feasible x in the primal and any feasible 00:02:02.750 --> 00:02:05.930 align:middle line:90% y in the dual solution. 00:02:05.930 --> 00:02:08.910 align:middle line:84% Now, feasible just means they satisfy the constraints. 00:02:08.910 --> 00:02:13.290 align:middle line:90% 00:02:13.290 --> 00:02:20.670 align:middle line:84% c transpose x is less than or equal to b transpose y. 00:02:20.670 --> 00:02:33.050 align:middle line:90% 00:02:33.050 --> 00:02:42.620 align:middle line:84% So this is true for when the optimal value is finite. 00:02:42.620 --> 00:02:45.690 align:middle line:90% 00:02:45.690 --> 00:02:53.880 align:middle line:84% So a linear program can have finite optimum. 00:02:53.880 --> 00:02:59.600 align:middle line:90% 00:02:59.600 --> 00:03:01.360 align:middle line:90% It can be infeasible. 00:03:01.360 --> 00:03:05.840 align:middle line:90% 00:03:05.840 --> 00:03:07.210 align:middle line:90% What's an infeasible program? 00:03:07.210 --> 00:03:10.480 align:middle line:90% 00:03:10.480 --> 00:03:15.720 align:middle line:84% x1 greater than or equal to 5 and x1 less than or equal to 3. 00:03:15.720 --> 00:03:20.460 align:middle line:84% So this is a basically trivial example of infeasible program. 00:03:20.460 --> 00:03:22.720 align:middle line:90% There are no solutions. 00:03:22.720 --> 00:03:26.080 align:middle line:84% Most infeasible programs, it's much harder to-- 00:03:26.080 --> 00:03:29.400 align:middle line:84% you cannot tell that it's infeasible just by looking 00:03:29.400 --> 00:03:30.600 align:middle line:90% at it. 00:03:30.600 --> 00:03:32.580 align:middle line:90% And it can be unbounded. 00:03:32.580 --> 00:03:38.597 align:middle line:90% 00:03:38.597 --> 00:03:40.180 align:middle line:84% And let's give you an example of that. 00:03:40.180 --> 00:03:44.260 align:middle line:84% Maximize x greater than or equal to 3. 00:03:44.260 --> 00:03:46.820 align:middle line:90% 00:03:46.820 --> 00:03:54.320 align:middle line:84% You can see that you can find a feasible x such 00:03:54.320 --> 00:03:58.510 align:middle line:84% that it has any value for the objective function. 00:03:58.510 --> 00:04:01.895 align:middle line:90% And so these are-- 00:04:01.895 --> 00:04:04.270 align:middle line:84% actually, it's pretty clear that these are the only three 00:04:04.270 --> 00:04:05.410 align:middle line:90% possibilities. 00:04:05.410 --> 00:04:11.390 align:middle line:90% 00:04:11.390 --> 00:04:11.890 align:middle line:90% OK. 00:04:11.890 --> 00:04:16.763 align:middle line:90% 00:04:16.763 --> 00:04:28.630 align:middle line:84% Now I'm going to tell you, well, something that follows 00:04:28.630 --> 00:04:30.750 align:middle line:90% immediately from weak duality. 00:04:30.750 --> 00:04:42.890 align:middle line:84% If the primal is unbounded, the dual is infeasible. 00:04:42.890 --> 00:04:48.870 align:middle line:90% 00:04:48.870 --> 00:04:53.740 align:middle line:84% And that's because if there were some feasible 00:04:53.740 --> 00:04:56.780 align:middle line:84% solution to the dual, then you could 00:04:56.780 --> 00:05:00.660 align:middle line:84% find a solution to the primal which is larger than that. 00:05:00.660 --> 00:05:03.740 align:middle line:84% And this is the contradiction of weak duality. 00:05:03.740 --> 00:05:09.620 align:middle line:84% And similarly, if dual is unbounded, 00:05:09.620 --> 00:05:11.480 align:middle line:90% the primal is infeasible. 00:05:11.480 --> 00:05:16.180 align:middle line:90% 00:05:16.180 --> 00:05:19.140 align:middle line:84% And there's one more possibility. 00:05:19.140 --> 00:05:31.560 align:middle line:84% Possible dual, primal are both infeasible. 00:05:31.560 --> 00:05:34.380 align:middle line:90% 00:05:34.380 --> 00:05:38.220 align:middle line:84% So this doesn't happen very often, but it can happen 00:05:38.220 --> 00:05:42.820 align:middle line:84% and you probably should be aware of it. 00:05:42.820 --> 00:05:44.760 align:middle line:84% We're not going to discuss it any further. 00:05:44.760 --> 00:05:47.700 align:middle line:84% Sometimes we give a homework problem 00:05:47.700 --> 00:05:52.160 align:middle line:84% asking you to find an example where this is the case, 00:05:52.160 --> 00:05:55.380 align:middle line:84% and it's not that hard a homework problem. 00:05:55.380 --> 00:05:58.060 align:middle line:84% So if you want to figure out how this can happen, 00:05:58.060 --> 00:06:01.500 align:middle line:84% you can go home and figure it out. 00:06:01.500 --> 00:06:03.680 align:middle line:84% Or you can probably look it up on Wikipedia 00:06:03.680 --> 00:06:05.620 align:middle line:90% and find it that way. 00:06:05.620 --> 00:06:10.840 align:middle line:90% 00:06:10.840 --> 00:06:13.320 align:middle line:90% OK. 00:06:13.320 --> 00:06:15.680 align:middle line:84% So first thing I want to do today 00:06:15.680 --> 00:06:23.240 align:middle line:84% is talk about what the primal and dual look like for things 00:06:23.240 --> 00:06:24.990 align:middle line:90% that are not in canonical form. 00:06:24.990 --> 00:06:27.560 align:middle line:90% 00:06:27.560 --> 00:06:34.140 align:middle line:84% And I'll first give you the treatment and the lecture notes, 00:06:34.140 --> 00:06:36.920 align:middle line:84% and then I will give you my own treatment, which is slight-- 00:06:36.920 --> 00:06:41.298 align:middle line:84% well, it's not really different but it's slightly different. 00:06:41.298 --> 00:06:42.590 align:middle line:90% And I think it makes it easier. 00:06:42.590 --> 00:06:46.240 align:middle line:90% 00:06:46.240 --> 00:06:49.210 align:middle line:84% So let's look at the proof of duality. 00:06:49.210 --> 00:07:00.710 align:middle line:90% 00:07:00.710 --> 00:07:05.590 align:middle line:84% Maybe I should do this by example, 00:07:05.590 --> 00:07:08.730 align:middle line:84% because I like doing things by example, 00:07:08.730 --> 00:07:11.030 align:middle line:90% because it's easier than-- 00:07:11.030 --> 00:07:14.150 align:middle line:84% sometimes it's easier than doing them abstractly. 00:07:14.150 --> 00:07:22.590 align:middle line:84% So let's say max 3x1 plus 5x2 minus 2x3. 00:07:22.590 --> 00:07:29.030 align:middle line:90% 00:07:29.030 --> 00:07:32.610 align:middle line:84% x1 plus x2 less than or equal to 8. 00:07:32.610 --> 00:07:35.630 align:middle line:90% 00:07:35.630 --> 00:07:41.310 align:middle line:84% x1 minus x3 is greater than or equal to 1. 00:07:41.310 --> 00:07:46.190 align:middle line:90% And x2 plus-- 00:07:46.190 --> 00:07:47.460 align:middle line:90% OK, I did this wrong. 00:07:47.460 --> 00:07:50.580 align:middle line:90% 00:07:50.580 --> 00:07:55.820 align:middle line:84% I want to leave a gap here so you can line up the variables. 00:07:55.820 --> 00:07:58.220 align:middle line:90% Greater than or equal to 1. 00:07:58.220 --> 00:08:04.210 align:middle line:90% And x2 plus x3 equals 4. 00:08:04.210 --> 00:08:11.860 align:middle line:90% 00:08:11.860 --> 00:08:16.010 align:middle line:90% Recall proof of duality. 00:08:16.010 --> 00:08:21.340 align:middle line:90% 00:08:21.340 --> 00:08:29.560 align:middle line:84% The dual has a variable for each constraint in the primal, 00:08:29.560 --> 00:08:33.570 align:middle line:84% and a constraint for each variable in the primal dual. 00:08:33.570 --> 00:08:40.100 align:middle line:90% 00:08:40.100 --> 00:08:42.510 align:middle line:90% Each primal variable. 00:08:42.510 --> 00:08:45.180 align:middle line:90% 00:08:45.180 --> 00:08:46.820 align:middle line:90% Oh, I'm sorry. 00:08:46.820 --> 00:08:54.970 align:middle line:84% I left out the non-negativity conditions. 00:08:54.970 --> 00:08:57.610 align:middle line:84% X1 and x3 are greater than or equal to 0, 00:08:57.610 --> 00:08:59.790 align:middle line:90% and x2 is unconstrained. 00:08:59.790 --> 00:09:04.450 align:middle line:90% 00:09:04.450 --> 00:09:14.250 align:middle line:84% So there's a dual variable for each primal constraint. 00:09:14.250 --> 00:09:19.750 align:middle line:90% 00:09:19.750 --> 00:09:20.250 align:middle line:90% Good. 00:09:20.250 --> 00:09:26.770 align:middle line:90% 00:09:26.770 --> 00:09:40.370 align:middle line:84% And now what we did was we took the inequalities, multiplied 00:09:40.370 --> 00:09:46.410 align:middle line:84% them by the new variables and the dual, and added them up. 00:09:46.410 --> 00:09:57.150 align:middle line:84% So we have y1 times x1 plus x2 is less than or equal to 8. 00:09:57.150 --> 00:10:06.230 align:middle line:84% y2 times x1 minus x3 is greater than or equal to 1. 00:10:06.230 --> 00:10:16.050 align:middle line:90% y3 times x2 plus x3 equals 4. 00:10:16.050 --> 00:10:24.830 align:middle line:90% 00:10:24.830 --> 00:10:27.620 align:middle line:90% OK. 00:10:27.620 --> 00:10:35.310 align:middle line:84% And when we add them up, we want these variables to-- 00:10:35.310 --> 00:10:40.430 align:middle line:84% or we want these inequalities to all point the same way. 00:10:40.430 --> 00:10:45.420 align:middle line:84% Because if we add x1 plus x2 less than or equal to 8 and x1 00:10:45.420 --> 00:10:47.280 align:middle line:84% minus x3 greater than or equal to 1, 00:10:47.280 --> 00:10:49.420 align:middle line:90% it doesn't tell us anything. 00:10:49.420 --> 00:10:52.180 align:middle line:84% So that means that here we have to have 00:10:52.180 --> 00:10:55.380 align:middle line:90% y1 greater or equal to 0. 00:10:55.380 --> 00:10:58.680 align:middle line:84% Here we want to flip the sign of this inequality, 00:10:58.680 --> 00:11:02.020 align:middle line:84% so we need y2 less than or equal to 0. 00:11:02.020 --> 00:11:06.100 align:middle line:84% And here, well, it doesn't matter 00:11:06.100 --> 00:11:09.280 align:middle line:84% whether y3 is bigger than or less than or equal to 0. 00:11:09.280 --> 00:11:11.740 align:middle line:84% Because when we multiply it, there'll be an equality. 00:11:11.740 --> 00:11:13.380 align:middle line:84% And when we add them all up, it'll 00:11:13.380 --> 00:11:15.920 align:middle line:90% still be the right inequality. 00:11:15.920 --> 00:11:19.280 align:middle line:84% So here, we have y3 unconstrained. 00:11:19.280 --> 00:11:23.785 align:middle line:90% 00:11:23.785 --> 00:11:24.285 align:middle line:90% OK. 00:11:24.285 --> 00:11:28.005 align:middle line:90% 00:11:28.005 --> 00:11:32.660 align:middle line:84% So going from a less than or equal to 00:11:32.660 --> 00:11:37.540 align:middle line:84% sign, if the primal is a maximization, 00:11:37.540 --> 00:11:40.580 align:middle line:90% gives you a positive variable. 00:11:40.580 --> 00:11:43.540 align:middle line:84% A greater or equal to sign, if the primal is maximization, 00:11:43.540 --> 00:11:47.090 align:middle line:84% gives you a negative variable, and an equal sign gives you 00:11:47.090 --> 00:11:48.360 align:middle line:90% an unconstrained variable. 00:11:48.360 --> 00:11:51.010 align:middle line:90% 00:11:51.010 --> 00:11:53.490 align:middle line:90% OK. 00:11:53.490 --> 00:11:53.990 align:middle line:90% Now. 00:11:53.990 --> 00:12:01.810 align:middle line:90% 00:12:01.810 --> 00:12:07.090 align:middle line:84% Remember, we wanted to maximize 3x1 plus 5x2 minus 2x3. 00:12:07.090 --> 00:12:25.330 align:middle line:84% So the first column gives you y1 plus y2 00:12:25.330 --> 00:12:29.050 align:middle line:90% is greater than or equal to 3. 00:12:29.050 --> 00:12:35.090 align:middle line:84% The second column gives you y1 plus y3 00:12:35.090 --> 00:12:39.320 align:middle line:90% is greater than or equal to 5. 00:12:39.320 --> 00:12:42.170 align:middle line:90% 00:12:42.170 --> 00:12:43.000 align:middle line:90% Is that right? 00:12:43.000 --> 00:12:48.040 align:middle line:90% 00:12:48.040 --> 00:12:59.380 align:middle line:84% And the third column gives you minus y2 plus y3 is-- 00:12:59.380 --> 00:13:02.680 align:middle line:90% 00:13:02.680 --> 00:13:12.080 align:middle line:84% minus y2 plus y3 is greater than root 2 minus 2. 00:13:12.080 --> 00:13:14.550 align:middle line:84% So these are the constraints of the dual. 00:13:14.550 --> 00:13:18.480 align:middle line:90% 00:13:18.480 --> 00:13:23.800 align:middle line:84% And similarly, when you're going from the dual to the primal, 00:13:23.800 --> 00:13:29.800 align:middle line:84% everything works the same way, except less than or equal to-- 00:13:29.800 --> 00:13:34.320 align:middle line:84% greater than or equal to signs turn into positive variables, 00:13:34.320 --> 00:13:37.320 align:middle line:84% less than or equal to signs turn into negative variables, 00:13:37.320 --> 00:13:42.180 align:middle line:84% and again, equal signs turn into unconstrained variables. 00:13:42.180 --> 00:13:45.180 align:middle line:84% So I don't like doing it this way. 00:13:45.180 --> 00:13:46.400 align:middle line:90% Yeah, you have a question? 00:13:46.400 --> 00:13:50.260 align:middle line:84% AUDIENCE: Oh, shouldn't y1 plus y3 be equal then [INAUDIBLE]? 00:13:50.260 --> 00:13:51.020 align:middle line:90% PETER SHOR: Yes. 00:13:51.020 --> 00:13:52.380 align:middle line:90% Yes, it should. 00:13:52.380 --> 00:13:55.200 align:middle line:90% y1 plus y3 equals 5. 00:13:55.200 --> 00:13:55.800 align:middle line:90% Thank you. 00:13:55.800 --> 00:14:01.980 align:middle line:90% 00:14:01.980 --> 00:14:04.752 align:middle line:90% OK. 00:14:04.752 --> 00:14:06.360 align:middle line:90% So what was I going to say? 00:14:06.360 --> 00:14:10.940 align:middle line:84% I was going to say, the reason I don't like this is it 00:14:10.940 --> 00:14:13.580 align:middle line:84% gives you negative variables, and negative variables 00:14:13.580 --> 00:14:15.000 align:middle line:90% are hard to think about. 00:14:15.000 --> 00:14:17.600 align:middle line:84% And when you're thinking about them, 00:14:17.600 --> 00:14:19.700 align:middle line:90% you're liable to make mistakes. 00:14:19.700 --> 00:14:22.480 align:middle line:84% So what I'm going to do is I'm going to-- 00:14:22.480 --> 00:14:27.320 align:middle line:90% 00:14:27.320 --> 00:14:27.820 align:middle line:90% yeah. 00:14:27.820 --> 00:14:31.060 align:middle line:90% 00:14:31.060 --> 00:14:32.430 align:middle line:90% Another way to take the dual. 00:14:32.430 --> 00:14:42.130 align:middle line:90% 00:14:42.130 --> 00:14:53.930 align:middle line:84% First, make all equations the correct less than 00:14:53.930 --> 00:14:57.810 align:middle line:84% or equal to or greater than or equal to, the correct sign. 00:14:57.810 --> 00:15:01.930 align:middle line:84% So here what we want to do is we want to take the second equation 00:15:01.930 --> 00:15:05.170 align:middle line:84% and turn it into a less than or equal sign, because if we have 00:15:05.170 --> 00:15:11.330 align:middle line:84% a maximization and canonical form, you want less than 00:15:11.330 --> 00:15:13.170 align:middle line:90% or equal to signs. 00:15:13.170 --> 00:15:26.330 align:middle line:84% So that turns into 3x1 plus 5x2 minus 2x3, 00:15:26.330 --> 00:15:40.640 align:middle line:84% subject to x1 plus x2 less than or equal to 8 minus x1 plus x3 00:15:40.640 --> 00:15:48.140 align:middle line:84% is less than minus 1 and x2 plus x3 equals 4. 00:15:48.140 --> 00:16:02.200 align:middle line:90% 00:16:02.200 --> 00:16:08.600 align:middle line:84% And I forgot to write down x1, x2 greater 00:16:08.600 --> 00:16:20.560 align:middle line:84% than or equal to 0 No, x1, x3 greater than or equal to 0. 00:16:20.560 --> 00:16:22.540 align:middle line:90% x2 unconstrained. 00:16:22.540 --> 00:16:36.120 align:middle line:90% 00:16:36.120 --> 00:16:39.870 align:middle line:84% So now we can just write this off. 00:16:39.870 --> 00:16:50.950 align:middle line:84% We take the transpose of the A matrix, which is y1 minus y2, y1 00:16:50.950 --> 00:16:56.420 align:middle line:90% plus y3, and y2 plus y3. 00:16:56.420 --> 00:17:02.830 align:middle line:90% 00:17:02.830 --> 00:17:06.670 align:middle line:84% x2 was unconstrained, so this is an equality. 00:17:06.670 --> 00:17:12.230 align:middle line:84% And x2 and x3 and x1 both had greater than 00:17:12.230 --> 00:17:18.900 align:middle line:84% or equal to signs, so these are less than or equal to signs. 00:17:18.900 --> 00:17:22.349 align:middle line:90% 00:17:22.349 --> 00:17:26.349 align:middle line:84% The minimum, because the original, 00:17:26.349 --> 00:17:27.970 align:middle line:90% the primal was maximization. 00:17:27.970 --> 00:17:30.070 align:middle line:90% The dual is minimization. 00:17:30.070 --> 00:17:32.310 align:middle line:84% And we just read off the right-hand sides 00:17:32.310 --> 00:17:37.590 align:middle line:84% of the equations, the inequalities, 00:17:37.590 --> 00:17:43.050 align:middle line:84% and we get 8 y1 minus y2 plus 4y3. 00:17:43.050 --> 00:17:45.930 align:middle line:90% 00:17:45.930 --> 00:17:51.210 align:middle line:84% And we call this w, not that it really matters, 00:17:51.210 --> 00:17:55.210 align:middle line:90% subject to these things. 00:17:55.210 --> 00:18:01.010 align:middle line:84% And we take the original objective functions 00:18:01.010 --> 00:18:03.410 align:middle line:84% and use it for the right-hand side. 00:18:03.410 --> 00:18:07.650 align:middle line:90% So 3, 5, and minus 2. 00:18:07.650 --> 00:18:14.810 align:middle line:84% And finally, a third equation was an inequality, so y1, 00:18:14.810 --> 00:18:21.870 align:middle line:84% y2 greater or equal to 0, and y3 unconstrained. 00:18:21.870 --> 00:18:24.890 align:middle line:90% 00:18:24.890 --> 00:18:33.410 align:middle line:84% And we don't have this hard to think about variable y2 00:18:33.410 --> 00:18:34.620 align:middle line:90% less than or equal to 0. 00:18:34.620 --> 00:18:38.920 align:middle line:90% 00:18:38.920 --> 00:18:40.970 align:middle line:84% So this is the way I like to take duals. 00:18:40.970 --> 00:18:45.120 align:middle line:90% 00:18:45.120 --> 00:18:45.620 align:middle line:90% OK. 00:18:45.620 --> 00:18:48.120 align:middle line:84% So the next thing I want to do is talk 00:18:48.120 --> 00:18:52.400 align:middle line:90% about complementary slackness. 00:18:52.400 --> 00:18:55.862 align:middle line:84% Any questions about the materials so far? 00:18:55.862 --> 00:18:56.362 align:middle line:90% OK. 00:18:56.362 --> 00:19:18.800 align:middle line:90% 00:19:18.800 --> 00:19:22.300 align:middle line:84% Complementary slackness, we are going to-- 00:19:22.300 --> 00:19:37.923 align:middle line:90% 00:19:37.923 --> 00:19:39.590 align:middle line:84% well, why don't I write the theorem down 00:19:39.590 --> 00:19:42.990 align:middle line:90% and then we'll prove it? 00:19:42.990 --> 00:19:47.710 align:middle line:84% So again, we're going to have two linear programs, 00:19:47.710 --> 00:19:53.030 align:middle line:84% and we're only going to prove it for the-- 00:19:53.030 --> 00:19:55.270 align:middle line:84% we're going to prove it when the linear programs are 00:19:55.270 --> 00:19:56.950 align:middle line:90% in canonical form. 00:19:56.950 --> 00:20:02.180 align:middle line:84% But in general, it's true for any linear program. 00:20:02.180 --> 00:20:08.150 align:middle line:90% 00:20:08.150 --> 00:20:17.430 align:middle line:84% Suppose x is a feasible solution in the primal and y 00:20:17.430 --> 00:20:20.290 align:middle line:84% is a feasible solution in the dual. 00:20:20.290 --> 00:20:35.580 align:middle line:90% 00:20:35.580 --> 00:20:56.820 align:middle line:84% Then x is optimal in primal and y is optimal in dual 00:20:56.820 --> 00:21:15.620 align:middle line:84% if for all i, either yi equals 0 or A sub xi equals b sub i. 00:21:15.620 --> 00:21:20.620 align:middle line:84% So this is either saying the i-th variable in the dual is 0 00:21:20.620 --> 00:21:22.620 align:middle line:84% or the i-th equation in the primal 00:21:22.620 --> 00:21:25.900 align:middle line:90% is satisfied with equality. 00:21:25.900 --> 00:21:43.080 align:middle line:84% And for all j, either x sub j equals 0 or A transpose yj 00:21:43.080 --> 00:21:45.040 align:middle line:90% equals b sub j. 00:21:45.040 --> 00:21:51.680 align:middle line:84% So this is saying either the j-th variable is 0 or the j-th 00:21:51.680 --> 00:21:53.990 align:middle line:90% equation in the dual-- 00:21:53.990 --> 00:21:57.000 align:middle line:90% 00:21:57.000 --> 00:21:58.300 align:middle line:90% OK, this is not b. 00:21:58.300 --> 00:21:59.140 align:middle line:90% This is c. 00:21:59.140 --> 00:22:02.200 align:middle line:90% 00:22:02.200 --> 00:22:05.960 align:middle line:84% Either the j-th equation in the j-th variable is 0, 00:22:05.960 --> 00:22:09.980 align:middle line:84% or the j-th equation in the dual is satisfied with equality. 00:22:09.980 --> 00:22:12.637 align:middle line:90% 00:22:12.637 --> 00:22:13.137 align:middle line:90% OK. 00:22:13.137 --> 00:22:16.119 align:middle line:90% 00:22:16.119 --> 00:22:20.400 align:middle line:90% So we'll prove it. 00:22:20.400 --> 00:22:23.000 align:middle line:90% And we'll only prove it for-- 00:22:23.000 --> 00:22:28.510 align:middle line:84% or rather, we will prove it for linear programs 00:22:28.510 --> 00:22:31.012 align:middle line:84% in the canonical form, but because it's 00:22:31.012 --> 00:22:32.970 align:middle line:84% true for linear programs in the canonical form, 00:22:32.970 --> 00:22:35.180 align:middle line:84% it's automatically true for all linear programs. 00:22:35.180 --> 00:22:46.230 align:middle line:90% 00:22:46.230 --> 00:22:47.910 align:middle line:84% There's no reason you have to prove it 00:22:47.910 --> 00:22:51.200 align:middle line:84% for an arbitrary linear program, and it's just extra notation. 00:22:51.200 --> 00:22:54.390 align:middle line:90% 00:22:54.390 --> 00:22:55.050 align:middle line:90% OK. 00:22:55.050 --> 00:23:00.630 align:middle line:84% So we should write down the programs in the canonical form. 00:23:00.630 --> 00:23:11.602 align:middle line:84% And maximum z equals c transpose x, Ax less than or equal to b, 00:23:11.602 --> 00:23:13.790 align:middle line:90% x greater than or equal to 0. 00:23:13.790 --> 00:23:15.710 align:middle line:90% So this is the primal. 00:23:15.710 --> 00:23:25.510 align:middle line:84% And the dual is the minimum w equals b transpose y, 00:23:25.510 --> 00:23:31.460 align:middle line:84% A transpose suppose y equals c is less than or equal to c, 00:23:31.460 --> 00:23:33.680 align:middle line:84% and y greater than or equal to 0. 00:23:33.680 --> 00:23:43.820 align:middle line:90% 00:23:43.820 --> 00:23:45.230 align:middle line:90% OK. 00:23:45.230 --> 00:23:59.020 align:middle line:90% So what we have is, then proof. 00:23:59.020 --> 00:24:14.140 align:middle line:84% Well, we have A transpose y is less than or equal to c. 00:24:14.140 --> 00:24:17.380 align:middle line:84% So let's start with c transpose x, 00:24:17.380 --> 00:24:22.420 align:middle line:84% and now let's plug this equation into here. 00:24:22.420 --> 00:24:26.070 align:middle line:84% c transpose x is less than or equal to-- 00:24:26.070 --> 00:24:29.250 align:middle line:90% 00:24:29.250 --> 00:24:33.815 align:middle line:84% well, c transpose is y transpose A. So this is y transpose Ax. 00:24:33.815 --> 00:24:39.130 align:middle line:90% 00:24:39.130 --> 00:24:47.250 align:middle line:84% Now, y transpose Ax is equal to y transpose Ax because matrix 00:24:47.250 --> 00:24:50.170 align:middle line:90% multiplication is associative. 00:24:50.170 --> 00:24:53.650 align:middle line:84% And we have Ax less than or equal to b. 00:24:53.650 --> 00:24:59.410 align:middle line:84% So this says it's less than or equal to y transpose b. 00:24:59.410 --> 00:25:03.440 align:middle line:90% So this is just weak duality. 00:25:03.440 --> 00:25:07.290 align:middle line:90% 00:25:07.290 --> 00:25:15.570 align:middle line:84% Suppose both of these, both less than or equal to signs 00:25:15.570 --> 00:25:17.070 align:middle line:90% hold with equality. 00:25:17.070 --> 00:25:22.170 align:middle line:90% 00:25:22.170 --> 00:25:26.190 align:middle line:84% Well, then we know that for this to be optimal, 00:25:26.190 --> 00:25:29.110 align:middle line:84% these signs need to be held with equality. 00:25:29.110 --> 00:25:36.470 align:middle line:84% And if this holds with equality, well, 00:25:36.470 --> 00:25:48.022 align:middle line:84% then let's rewrite this equation y transpose A minus c, 00:25:48.022 --> 00:25:49.240 align:middle line:90% x equals 0. 00:25:49.240 --> 00:25:53.030 align:middle line:90% 00:25:53.030 --> 00:25:56.830 align:middle line:84% Well, this should be c transpose. 00:25:56.830 --> 00:25:59.790 align:middle line:90% And let's rewrite this equation. 00:25:59.790 --> 00:26:07.780 align:middle line:84% That's just y transpose Ax minus b equals 0. 00:26:07.780 --> 00:26:11.910 align:middle line:90% 00:26:11.910 --> 00:26:12.410 align:middle line:90% OK. 00:26:12.410 --> 00:26:17.376 align:middle line:90% 00:26:17.376 --> 00:26:23.020 align:middle line:84% So if we have two dot products of vectors are 0 00:26:23.020 --> 00:26:30.420 align:middle line:84% and each of these vectors contains only positive entries, 00:26:30.420 --> 00:26:36.780 align:middle line:84% then every piece of the dot product, one of these two 00:26:36.780 --> 00:26:39.460 align:middle line:90% equals 0. 00:26:39.460 --> 00:26:58.400 align:middle line:84% So y transpose i equals 0 or Ax sub i equals b sub i. 00:26:58.400 --> 00:27:01.220 align:middle line:90% 00:27:01.220 --> 00:27:04.300 align:middle line:84% And here, it's just the same thing. 00:27:04.300 --> 00:27:15.260 align:middle line:84% Either xj equals 0 or y transpose Aj y transpose 00:27:15.260 --> 00:27:23.330 align:middle line:90% A sub j equals c transpose j. 00:27:23.330 --> 00:27:30.010 align:middle line:90% 00:27:30.010 --> 00:27:35.410 align:middle line:84% So this is one way of the inequality. 00:27:35.410 --> 00:27:37.730 align:middle line:90% Now suppose the other way holds. 00:27:37.730 --> 00:27:43.930 align:middle line:84% That means either xj equals 0 or y transpose Aj equals cj. 00:27:43.930 --> 00:27:47.450 align:middle line:84% Well, that means that this product is 0, which 00:27:47.450 --> 00:27:54.770 align:middle line:90% means that this is an equality. 00:27:54.770 --> 00:28:00.250 align:middle line:84% And similarly, on this side, if either this equals 0 00:28:00.250 --> 00:28:04.410 align:middle line:84% or this equals 0, then this dot product is zero, 00:28:04.410 --> 00:28:06.490 align:middle line:84% and that means that this is an equality. 00:28:06.490 --> 00:28:09.890 align:middle line:84% So z transpose x equals y transpose 00:28:09.890 --> 00:28:19.880 align:middle line:84% b, which means that c transpose x and y transpose b 00:28:19.880 --> 00:28:23.160 align:middle line:90% are optimal solutions. 00:28:23.160 --> 00:28:25.830 align:middle line:90% OK. 00:28:25.830 --> 00:28:31.000 align:middle line:84% Now I want to go back here and tell you 00:28:31.000 --> 00:28:35.140 align:middle line:84% something neat you can do with complementary slackness. 00:28:35.140 --> 00:28:45.240 align:middle line:90% 00:28:45.240 --> 00:28:46.500 align:middle line:90% OK. 00:28:46.500 --> 00:28:58.190 align:middle line:84% So x1 equals 4, x3 equals 4 is a feasible solution. 00:28:58.190 --> 00:29:03.960 align:middle line:90% 00:29:03.960 --> 00:29:05.232 align:middle line:90% Right? 00:29:05.232 --> 00:29:10.760 align:middle line:84% Because 4 plus 4 is less than or equal to 8 minus 4 00:29:10.760 --> 00:29:12.430 align:middle line:90% plus 4 is what? 00:29:12.430 --> 00:29:17.120 align:middle line:90% 00:29:17.120 --> 00:29:22.025 align:middle line:84% x1 plus-- this is an x2, not an x3. 00:29:22.025 --> 00:29:26.972 align:middle line:90% 4 plus 4 is less than 8 minus 4. 00:29:26.972 --> 00:29:28.860 align:middle line:90% Oh, I forgot. 00:29:28.860 --> 00:29:32.060 align:middle line:90% x3 equals 0. 00:29:32.060 --> 00:29:36.140 align:middle line:84% Minus 4 plus 0 is less than or equal to minus 1, 00:29:36.140 --> 00:29:40.980 align:middle line:90% and 4 plus 0 equals 4. 00:29:40.980 --> 00:29:44.200 align:middle line:90% So this is a feasible solution. 00:29:44.200 --> 00:29:45.850 align:middle line:84% And in fact, it's the optimum solution. 00:29:45.850 --> 00:29:57.380 align:middle line:90% 00:29:57.380 --> 00:30:00.980 align:middle line:84% And if you know it's the optimum solution to the primal, 00:30:00.980 --> 00:30:03.100 align:middle line:84% you can use complementary slackness 00:30:03.100 --> 00:30:05.570 align:middle line:84% to find the optimum solution to the dual. 00:30:05.570 --> 00:30:16.500 align:middle line:90% 00:30:16.500 --> 00:30:19.930 align:middle line:84% So here, we have two variables which 00:30:19.930 --> 00:30:26.930 align:middle line:84% are not equal to 0, which means we have two equations which 00:30:26.930 --> 00:30:33.210 align:middle line:90% are satisfied with equality. 00:30:33.210 --> 00:30:35.460 align:middle line:84% So that means that we must have-- 00:30:35.460 --> 00:30:39.010 align:middle line:90% 00:30:39.010 --> 00:30:41.210 align:middle line:84% so the two variables which are not equal to 0 00:30:41.210 --> 00:30:43.025 align:middle line:84% are the first and the second, which 00:30:43.025 --> 00:30:44.650 align:middle line:84% means the first and the second equation 00:30:44.650 --> 00:30:46.350 align:middle line:90% are satisfied with equality. 00:30:46.350 --> 00:30:50.230 align:middle line:90% So y1 equals y2. 00:30:50.230 --> 00:30:50.730 align:middle line:90% OK. 00:30:50.730 --> 00:30:54.450 align:middle line:90% I hope I'm doing this right. 00:30:54.450 --> 00:30:55.410 align:middle line:90% No. 00:30:55.410 --> 00:30:58.610 align:middle line:84% I'm not doing it right, because this equation 00:30:58.610 --> 00:31:02.890 align:middle line:90% is y1 minus y2 is less equals-- 00:31:02.890 --> 00:31:04.250 align:middle line:90% equals 3. 00:31:04.250 --> 00:31:14.850 align:middle line:84% So y1 minus y2 equals 3, and y1 plus y3 equals 5. 00:31:14.850 --> 00:31:19.920 align:middle line:84% Now up here there's an equation which 00:31:19.920 --> 00:31:27.100 align:middle line:84% is not satisfied with equality in the optimal solution. 00:31:27.100 --> 00:31:30.950 align:middle line:84% That's the second equation, which means y2 equals 0. 00:31:30.950 --> 00:31:35.480 align:middle line:90% 00:31:35.480 --> 00:31:39.400 align:middle line:84% And now you've written down these three equations, 00:31:39.400 --> 00:31:42.040 align:middle line:84% and there are three unknowns, so you can solve it. 00:31:42.040 --> 00:31:46.350 align:middle line:84% And so y1 equals 3 and y2 equals 5. 00:31:46.350 --> 00:31:49.800 align:middle line:90% 00:31:49.800 --> 00:31:53.200 align:middle line:84% And now you can plug this in and see 00:31:53.200 --> 00:32:00.280 align:middle line:84% whether this equation for the primal or this feasible 00:32:00.280 --> 00:32:03.280 align:middle line:84% solution for the primal and this solution for the dual 00:32:03.280 --> 00:32:06.780 align:middle line:84% give you the same objective function. 00:32:06.780 --> 00:32:07.940 align:middle line:90% So let's do that. 00:32:07.940 --> 00:32:13.720 align:middle line:84% We have x1 plus x2 is 3 times 4 plus 5 times 4. 00:32:13.720 --> 00:32:14.620 align:middle line:90% That's 32. 00:32:14.620 --> 00:32:17.190 align:middle line:90% 00:32:17.190 --> 00:32:22.590 align:middle line:90% So objective function, 32. 00:32:22.590 --> 00:32:26.375 align:middle line:90% And down here, we have 24. 00:32:26.375 --> 00:32:30.030 align:middle line:90% 00:32:30.030 --> 00:32:31.070 align:middle line:90% OK. 00:32:31.070 --> 00:32:32.830 align:middle line:90% y1 equals 3. 00:32:32.830 --> 00:32:36.630 align:middle line:84% I obviously did my equations wrong. y1 equals 3. 00:32:36.630 --> 00:32:38.650 align:middle line:90% y2 equals 0, so y1 equals 3. 00:32:38.650 --> 00:32:40.650 align:middle line:90% And we have 3 plus y3 equals 5. 00:32:40.650 --> 00:32:42.410 align:middle line:90% So y3 equals 2. 00:32:42.410 --> 00:32:46.390 align:middle line:90% 00:32:46.390 --> 00:32:49.310 align:middle line:90% Apologies for my arithmetic. 00:32:49.310 --> 00:32:56.320 align:middle line:84% So that means that we have 24 plus 8 is 32. 00:32:56.320 --> 00:32:58.830 align:middle line:90% 00:32:58.830 --> 00:33:01.360 align:middle line:90% Objective function equals 32. 00:33:01.360 --> 00:33:06.310 align:middle line:90% 00:33:06.310 --> 00:33:08.650 align:middle line:84% So this is an optimal solution for the primal 00:33:08.650 --> 00:33:10.580 align:middle line:84% and this is an optimal solution for the dual. 00:33:10.580 --> 00:33:18.490 align:middle line:90% 00:33:18.490 --> 00:33:18.990 align:middle line:90% OK. 00:33:18.990 --> 00:33:23.850 align:middle line:90% 00:33:23.850 --> 00:33:29.990 align:middle line:84% So the last thing I want to do is give a-- 00:33:29.990 --> 00:33:33.290 align:middle line:84% I don't know if I want to call it a proof or not, 00:33:33.290 --> 00:33:38.250 align:middle line:84% because maybe I give a justification 00:33:38.250 --> 00:33:42.650 align:middle line:90% for strong duality. 00:33:42.650 --> 00:33:51.290 align:middle line:84% If you look online, there are roughly two kinds of proofs. 00:33:51.290 --> 00:33:54.890 align:middle line:84% There are three kinds of proofs of strong duality. 00:33:54.890 --> 00:33:59.850 align:middle line:84% The first, you define the simplex algorithm, 00:33:59.850 --> 00:34:02.970 align:middle line:84% which is a way of getting the optimal solution 00:34:02.970 --> 00:34:04.690 align:middle line:90% for linear programs. 00:34:04.690 --> 00:34:07.630 align:middle line:84% And then when the simplex algorithm finishes, 00:34:07.630 --> 00:34:10.650 align:middle line:84% it gives you what's called a tableau, from which you can read 00:34:10.650 --> 00:34:13.800 align:middle line:84% out the solution to the primal, you can read off 00:34:13.800 --> 00:34:16.080 align:middle line:84% the optimal solution to the dual, 00:34:16.080 --> 00:34:23.880 align:middle line:84% and then you can stare at it and use the-- 00:34:23.880 --> 00:34:30.040 align:middle line:84% well, you can-- it tells you that the optimal solution 00:34:30.040 --> 00:34:36.320 align:middle line:84% to the primal is equal to the optimal solution to the dual. 00:34:36.320 --> 00:34:41.139 align:middle line:84% So that's one way of proving strong duality, 00:34:41.139 --> 00:34:43.920 align:middle line:90% and it's absolutely rigorous. 00:34:43.920 --> 00:34:46.639 align:middle line:84% Another way of proving strong duality 00:34:46.639 --> 00:34:52.480 align:middle line:84% is borrowing this lemma from analysis called Farkas' Lemma 00:34:52.480 --> 00:34:54.159 align:middle line:90% and applying it. 00:34:54.159 --> 00:34:56.280 align:middle line:84% The problem with this is that Farkas' lemma 00:34:56.280 --> 00:35:01.440 align:middle line:84% is basically strong duality in different notation. 00:35:01.440 --> 00:35:05.000 align:middle line:90% So you haven't really-- 00:35:05.000 --> 00:35:08.960 align:middle line:84% I mean, I guess it's a proof if you believe Farkas' lemma, 00:35:08.960 --> 00:35:12.670 align:middle line:84% but it doesn't seem to give you any more real insight 00:35:12.670 --> 00:35:15.630 align:middle line:84% than my just telling you that strong duality holds, 00:35:15.630 --> 00:35:18.420 align:middle line:90% because that's what it's doing. 00:35:18.420 --> 00:35:21.590 align:middle line:90% 00:35:21.590 --> 00:35:27.550 align:middle line:84% And the third uses mechanics in n dimensions. 00:35:27.550 --> 00:35:30.970 align:middle line:84% We need a few basic theorems from mechanics, 00:35:30.970 --> 00:35:34.670 align:middle line:84% and then we can prove strong duality. 00:35:34.670 --> 00:35:39.990 align:middle line:84% So, I mean, we have to accept these theorems from mechanics. 00:35:39.990 --> 00:35:42.910 align:middle line:84% I don't know if there's any way going through mathematics 00:35:42.910 --> 00:35:46.870 align:middle line:84% and proving that these theorems from mechanics 00:35:46.870 --> 00:35:52.910 align:middle line:84% really hold, but I mean, if you accept 00:35:52.910 --> 00:35:56.510 align:middle line:84% the theorems from mechanics, then you absolutely 00:35:56.510 --> 00:36:00.790 align:middle line:84% have a proof of strong duality and showing-- 00:36:00.790 --> 00:36:04.110 align:middle line:84% I mean, I find this really pretty amazing, showing 00:36:04.110 --> 00:36:07.870 align:middle line:84% that mechanics is consistent in high dimensions. 00:36:07.870 --> 00:36:10.020 align:middle line:84% It's the same thing, in some sense, 00:36:10.020 --> 00:36:13.500 align:middle line:84% as proving strong duality of linear programming, which 00:36:13.500 --> 00:36:16.780 align:middle line:84% is a connection I never would have expected. 00:36:16.780 --> 00:36:20.260 align:middle line:84% So we're going to prove give you the justification 00:36:20.260 --> 00:36:21.440 align:middle line:90% from mechanics. 00:36:21.440 --> 00:36:24.700 align:middle line:90% 00:36:24.700 --> 00:36:26.000 align:middle line:90% And let's see. 00:36:26.000 --> 00:36:47.820 align:middle line:90% 00:36:47.820 --> 00:36:56.920 align:middle line:84% So a linear program set of hyperplanes. 00:36:56.920 --> 00:37:03.180 align:middle line:90% 00:37:03.180 --> 00:37:04.280 align:middle line:90% The constraints. 00:37:04.280 --> 00:37:09.560 align:middle line:90% 00:37:09.560 --> 00:37:12.340 align:middle line:84% The constraints say you're on one side of the hyperplane. 00:37:12.340 --> 00:37:24.160 align:middle line:90% 00:37:24.160 --> 00:37:29.280 align:middle line:84% And the objective function, you want 00:37:29.280 --> 00:37:42.400 align:middle line:84% to find feasible point furthest in direction 00:37:42.400 --> 00:37:45.910 align:middle line:90% of objective function. 00:37:45.910 --> 00:37:49.400 align:middle line:90% 00:37:49.400 --> 00:37:50.880 align:middle line:90% So what do we do? 00:37:50.880 --> 00:37:56.050 align:middle line:84% Well, we pretend each of these hyperplanes is a solid surface. 00:37:56.050 --> 00:38:00.640 align:middle line:90% 00:38:00.640 --> 00:38:01.140 align:middle line:90% OK. 00:38:01.140 --> 00:38:02.130 align:middle line:90% This is a bad drawing. 00:38:02.130 --> 00:38:06.710 align:middle line:90% 00:38:06.710 --> 00:38:12.910 align:middle line:84% And what we're going to do is we're going to drop a ball, 00:38:12.910 --> 00:38:15.270 align:middle line:84% and gravity is going to take it in the direction 00:38:15.270 --> 00:38:21.430 align:middle line:84% of the increasing or decreasing objective function, 00:38:21.430 --> 00:38:24.770 align:middle line:84% and it's going to hit this hyperplane. 00:38:24.770 --> 00:38:26.830 align:middle line:84% And now it's going to roll in this direction 00:38:26.830 --> 00:38:28.790 align:middle line:84% and it's going to hit this point. 00:38:28.790 --> 00:38:34.030 align:middle line:90% And now we're going to assume-- 00:38:34.030 --> 00:38:39.830 align:middle line:84% well, actually, maybe we should make another plane here. 00:38:39.830 --> 00:38:43.870 align:middle line:84% It's going to roll down to here, and this is the lowest point 00:38:43.870 --> 00:38:46.270 align:middle line:84% on the proper side of all the hyperplanes so it's just 00:38:46.270 --> 00:38:47.750 align:middle line:90% going to sit there. 00:38:47.750 --> 00:38:51.030 align:middle line:84% And mechanics tells us that the force of gravity on it 00:38:51.030 --> 00:38:57.370 align:middle line:84% is balanced by the force being pushed on this point 00:38:57.370 --> 00:38:59.910 align:middle line:84% perpendicular to each hyperplane. 00:38:59.910 --> 00:39:03.870 align:middle line:84% So the sum of the forces exerted on this ball by the hyperplanes 00:39:03.870 --> 00:39:06.010 align:middle line:84% is equal to the force of gravity. 00:39:06.010 --> 00:39:25.660 align:middle line:90% 00:39:25.660 --> 00:39:30.000 align:middle line:84% And this is a theorem from mechanics we need. 00:39:30.000 --> 00:39:37.860 align:middle line:90% 00:39:37.860 --> 00:39:38.360 align:middle line:90% OK. 00:39:38.360 --> 00:39:47.036 align:middle line:90% 00:39:47.036 --> 00:39:53.980 align:middle line:84% So now let's write down the linear program 00:39:53.980 --> 00:39:55.430 align:middle line:90% that solves this problem. 00:39:55.430 --> 00:40:12.560 align:middle line:90% 00:40:12.560 --> 00:40:35.400 align:middle line:84% Minimize c transpose x subject to Ax 00:40:35.400 --> 00:40:38.920 align:middle line:90% is greater than or equal to b. 00:40:38.920 --> 00:40:42.920 align:middle line:84% And this is just saying the point 00:40:42.920 --> 00:40:46.620 align:middle line:84% is on the correct side of every hyperplane. 00:40:46.620 --> 00:40:56.680 align:middle line:90% 00:40:56.680 --> 00:41:00.640 align:middle line:84% And this is just the force of gravity. 00:41:00.640 --> 00:41:05.990 align:middle line:84% You want to make the point as low as possible. 00:41:05.990 --> 00:41:15.210 align:middle line:84% And, well, x is an arbitrary point, so x is unconstrained. 00:41:15.210 --> 00:41:22.310 align:middle line:90% 00:41:22.310 --> 00:41:27.510 align:middle line:84% So now we can ask, what's the dual of this problem? 00:41:27.510 --> 00:41:30.430 align:middle line:90% So this is the primal. 00:41:30.430 --> 00:41:38.770 align:middle line:84% The dual, well, it's going to be max v transpose y. 00:41:38.770 --> 00:41:41.310 align:middle line:90% 00:41:41.310 --> 00:41:45.650 align:middle line:84% And now all of these x's are unconstrained. 00:41:45.650 --> 00:41:52.110 align:middle line:90% 00:41:52.110 --> 00:41:59.990 align:middle line:84% So that means all of these equations must be equalities. 00:41:59.990 --> 00:42:04.800 align:middle line:84% So we have a transpose y equals c. 00:42:04.800 --> 00:42:08.380 align:middle line:90% 00:42:08.380 --> 00:42:13.980 align:middle line:84% And here, we have a whole bunch of inequalities which means 00:42:13.980 --> 00:42:16.010 align:middle line:90% we get y greater or equal zero. 00:42:16.010 --> 00:42:25.580 align:middle line:90% 00:42:25.580 --> 00:42:26.566 align:middle line:90% OK. 00:42:26.566 --> 00:42:33.540 align:middle line:84% So this is the linear program and its dual, 00:42:33.540 --> 00:42:37.870 align:middle line:84% that we will show how the property of strong duality. 00:42:37.870 --> 00:43:11.970 align:middle line:90% 00:43:11.970 --> 00:43:13.350 align:middle line:90% So a hyperplane. 00:43:13.350 --> 00:43:16.090 align:middle line:90% 00:43:16.090 --> 00:43:18.890 align:middle line:90% The equation was-- 00:43:18.890 --> 00:43:21.110 align:middle line:90% I left out some letters there. 00:43:21.110 --> 00:43:25.210 align:middle line:90% 00:43:25.210 --> 00:43:40.950 align:middle line:84% The equation was Ai1 x1 plus Ai2 x2 plus dot, dot, dot plus Ai n 00:43:40.950 --> 00:43:47.630 align:middle line:84% x sub n is greater than or equal to b sub i. 00:43:47.630 --> 00:43:52.490 align:middle line:90% 00:43:52.490 --> 00:43:57.070 align:middle line:84% So what is the upward force on this hyperplane? 00:43:57.070 --> 00:44:15.630 align:middle line:90% 00:44:15.630 --> 00:44:18.270 align:middle line:84% Well, if you have a hyperplane and you 00:44:18.270 --> 00:44:20.510 align:middle line:84% have a ball resting on the hyperplane, 00:44:20.510 --> 00:44:24.370 align:middle line:84% the force on the ball is perpendicular to the hyperplane. 00:44:24.370 --> 00:44:37.230 align:middle line:90% 00:44:37.230 --> 00:44:40.330 align:middle line:90% So it's in direction. 00:44:40.330 --> 00:44:45.190 align:middle line:90% 00:44:45.190 --> 00:44:51.930 align:middle line:84% I want to say Ai1 Ai2 through a sub i n. 00:44:51.930 --> 00:44:57.100 align:middle line:90% 00:44:57.100 --> 00:45:04.580 align:middle line:84% It has to be positive, so it's some constant c. 00:45:04.580 --> 00:45:12.930 align:middle line:84% So force is c Ai1 through A sub i n with c greater than 0. 00:45:12.930 --> 00:45:17.840 align:middle line:90% 00:45:17.840 --> 00:45:18.340 align:middle line:90% OK. 00:45:18.340 --> 00:45:21.580 align:middle line:90% 00:45:21.580 --> 00:45:26.080 align:middle line:84% c is a bad number for this, bad variable for this. 00:45:26.080 --> 00:45:28.620 align:middle line:90% Let's see. 00:45:28.620 --> 00:45:31.780 align:middle line:90% Let's call it f. 00:45:31.780 --> 00:45:33.600 align:middle line:84% I should probably call this f sub i. 00:45:33.600 --> 00:45:40.280 align:middle line:90% 00:45:40.280 --> 00:45:40.780 align:middle line:90% OK. 00:45:40.780 --> 00:45:55.340 align:middle line:90% 00:45:55.340 --> 00:45:55.840 align:middle line:90% OK. 00:45:55.840 --> 00:46:14.760 align:middle line:90% 00:46:14.760 --> 00:46:15.260 align:middle line:90% OK. 00:46:15.260 --> 00:46:20.770 align:middle line:90% 00:46:20.770 --> 00:46:28.400 align:middle line:84% We have there must be some forces. 00:46:28.400 --> 00:46:34.130 align:middle line:90% 00:46:34.130 --> 00:46:38.810 align:middle line:84% And I'm going to call them y sub i instead of f sub i, 00:46:38.810 --> 00:46:43.450 align:middle line:84% just because y sub i is the right name for the variable 00:46:43.450 --> 00:46:52.640 align:middle line:84% because it's a dual variable, such that summation-- 00:46:52.640 --> 00:46:54.600 align:middle line:90% OK. 00:46:54.600 --> 00:47:02.280 align:middle line:84% Such that if v sub i is equal to this direction, 00:47:02.280 --> 00:47:15.640 align:middle line:84% Ai1, Ai2 through A sub i n, then summation v sub i. 00:47:15.640 --> 00:47:18.000 align:middle line:90% No. 00:47:18.000 --> 00:47:20.160 align:middle line:90% Yeah, that's right. 00:47:20.160 --> 00:47:23.900 align:middle line:84% Then summation y sub i, v sub i is equal to c. 00:47:23.900 --> 00:47:35.840 align:middle line:90% 00:47:35.840 --> 00:47:39.460 align:middle line:84% Actually, maybe we should make this c transpose. 00:47:39.460 --> 00:47:43.560 align:middle line:90% 00:47:43.560 --> 00:47:47.240 align:middle line:84% And because these forces all act away from the hyperplane, 00:47:47.240 --> 00:47:51.430 align:middle line:84% we need y sub i greater or equal to 0. 00:47:51.430 --> 00:48:02.860 align:middle line:90% 00:48:02.860 --> 00:48:03.360 align:middle line:90% OK. 00:48:03.360 --> 00:48:11.964 align:middle line:90% 00:48:11.964 --> 00:48:41.560 align:middle line:84% If the ball touches the i-th hyperplane, then A sub i. 00:48:41.560 --> 00:48:45.620 align:middle line:90% 00:48:45.620 --> 00:48:52.140 align:middle line:84% If the ball touches the i-th hyperplane, I want to say Ax. 00:48:52.140 --> 00:48:57.050 align:middle line:90% 00:48:57.050 --> 00:48:58.330 align:middle line:90% Ax. 00:48:58.330 --> 00:49:04.450 align:middle line:84% The i-th term of Ax is equal to b sub i, 00:49:04.450 --> 00:49:08.130 align:middle line:84% because if the ball touches the i-th hyperplane, 00:49:08.130 --> 00:49:11.570 align:middle line:84% then the constraint is satisfied with equality. 00:49:11.570 --> 00:49:14.550 align:middle line:90% And this is true. 00:49:14.550 --> 00:49:17.930 align:middle line:90% 00:49:17.930 --> 00:49:21.950 align:middle line:84% The ball does not touch the i-th hyperplane. 00:49:21.950 --> 00:49:31.530 align:middle line:90% 00:49:31.530 --> 00:49:32.970 align:middle line:90% Then what's going on? 00:49:32.970 --> 00:49:36.570 align:middle line:84% If the ball does not touch the i-th hyperplane, 00:49:36.570 --> 00:49:41.440 align:middle line:84% then the hyperplane does not exert any force on the ball. 00:49:41.440 --> 00:49:45.250 align:middle line:90% 00:49:45.250 --> 00:49:48.200 align:middle line:90% Then y sub i equals 0. 00:49:48.200 --> 00:49:50.800 align:middle line:90% 00:49:50.800 --> 00:49:57.840 align:middle line:84% And these two things are enough to give you complementary. 00:49:57.840 --> 00:50:00.600 align:middle line:84% Are enough to-- I mean, these two things, first, 00:50:00.600 --> 00:50:02.920 align:middle line:84% look a lot like complementary slackness, 00:50:02.920 --> 00:50:05.890 align:middle line:84% but they're enough to give you strong duality. 00:50:05.890 --> 00:50:37.040 align:middle line:90% 00:50:37.040 --> 00:50:37.540 align:middle line:90% OK. 00:50:37.540 --> 00:50:45.360 align:middle line:90% 00:50:45.360 --> 00:50:49.830 align:middle line:90% OK, we have y sub i. 00:50:49.830 --> 00:50:50.430 align:middle line:90% No. 00:50:50.430 --> 00:51:00.750 align:middle line:84% y transpose Ax minus b is equal to yi, 00:51:00.750 --> 00:51:07.230 align:middle line:84% I guess summation y sub i, Ax minus bi. 00:51:07.230 --> 00:51:12.990 align:middle line:90% 00:51:12.990 --> 00:51:18.230 align:middle line:84% And I want to claim either yi equals 0 or Ax sub i equals b 00:51:18.230 --> 00:51:20.510 align:middle line:90% sub i. 00:51:20.510 --> 00:51:25.950 align:middle line:90% So this equals 0. 00:51:25.950 --> 00:51:33.630 align:middle line:84% Now this goes y transpose Ax minus b equals 0. 00:51:33.630 --> 00:51:37.030 align:middle line:84% And let's look back at the linear program. 00:51:37.030 --> 00:51:47.710 align:middle line:84% And we have A transpose y equals c. 00:51:47.710 --> 00:51:51.960 align:middle line:90% 00:51:51.960 --> 00:51:57.410 align:middle line:90% So y transpose A is c transpose. 00:51:57.410 --> 00:52:05.730 align:middle line:84% So this says-- oh, this should be y transpose b. 00:52:05.730 --> 00:52:10.890 align:middle line:84% So when I took this, I forgot to move the y transpose 00:52:10.890 --> 00:52:13.570 align:middle line:90% into the parentheses. 00:52:13.570 --> 00:52:17.490 align:middle line:84% So this is c transpose x, and so we 00:52:17.490 --> 00:52:23.170 align:middle line:84% have c transpose x minus y transpose b equals 0 00:52:23.170 --> 00:52:28.670 align:middle line:84% and c transpose x equals y transpose b. 00:52:28.670 --> 00:52:44.520 align:middle line:90% 00:52:44.520 --> 00:52:46.860 align:middle line:90% OK. 00:52:46.860 --> 00:52:50.600 align:middle line:84% So that actually is the end of the linear programming 00:52:50.600 --> 00:52:52.480 align:middle line:90% duality lecture notes. 00:52:52.480 --> 00:52:55.380 align:middle line:84% And I suspected I might have a little time left, 00:52:55.380 --> 00:52:58.520 align:middle line:84% so I am going to tell you about Konig's theorem, which 00:52:58.520 --> 00:53:01.810 align:middle line:84% is an application of linear programming duality. 00:53:01.810 --> 00:53:17.720 align:middle line:90% 00:53:17.720 --> 00:53:19.980 align:middle line:84% And I did not prepare this as well, 00:53:19.980 --> 00:53:24.440 align:middle line:84% so hopefully I won't make any mistakes. 00:53:24.440 --> 00:53:25.310 align:middle line:90% Konig's theorem. 00:53:25.310 --> 00:53:30.240 align:middle line:90% 00:53:30.240 --> 00:53:32.050 align:middle line:84% Suppose you have a bipartite graph. 00:53:32.050 --> 00:53:47.470 align:middle line:90% 00:53:47.470 --> 00:53:49.326 align:middle line:90% OK. 00:53:49.326 --> 00:53:50.740 align:middle line:90% So let's draw an example. 00:53:50.740 --> 00:54:24.430 align:middle line:90% 00:54:24.430 --> 00:54:24.930 align:middle line:90% OK. 00:54:24.930 --> 00:54:27.870 align:middle line:90% 00:54:27.870 --> 00:54:46.490 align:middle line:84% Matching is a set of edges that do not-- 00:54:46.490 --> 00:54:49.180 align:middle line:90% 00:54:49.180 --> 00:54:54.230 align:middle line:84% I want to say that do not meet at any vertices. 00:54:54.230 --> 00:55:06.700 align:middle line:90% 00:55:06.700 --> 00:55:11.020 align:middle line:84% So this is one edge of a matching. 00:55:11.020 --> 00:55:12.940 align:middle line:84% This might be another edge of the matching. 00:55:12.940 --> 00:55:15.340 align:middle line:90% This might be another edge. 00:55:15.340 --> 00:55:19.460 align:middle line:90% And here's another edge. 00:55:19.460 --> 00:55:21.190 align:middle line:84% So that's four edges in this matching. 00:55:21.190 --> 00:55:39.000 align:middle line:90% 00:55:39.000 --> 00:55:39.500 align:middle line:90% OK. 00:55:39.500 --> 00:55:40.940 align:middle line:90% Now I want to claim-- 00:55:40.940 --> 00:55:43.030 align:middle line:84% now I want to define a vertex cover. 00:55:43.030 --> 00:55:57.160 align:middle line:90% 00:55:57.160 --> 00:56:07.880 align:middle line:84% Vertex cover is a set of vertices 00:56:07.880 --> 00:56:10.310 align:middle line:90% that intersect all the edges. 00:56:10.310 --> 00:56:16.520 align:middle line:90% 00:56:16.520 --> 00:56:17.530 align:middle line:90% That meet all the edges. 00:56:17.530 --> 00:56:23.280 align:middle line:90% 00:56:23.280 --> 00:56:23.900 align:middle line:90% So let's see. 00:56:23.900 --> 00:56:27.120 align:middle line:84% What would a vertex cover in this graph look like? 00:56:27.120 --> 00:56:32.320 align:middle line:90% Well, you might use this vertex. 00:56:32.320 --> 00:56:35.000 align:middle line:90% You might use this vertex. 00:56:35.000 --> 00:56:37.240 align:middle line:90% You might use this vertex. 00:56:37.240 --> 00:56:38.580 align:middle line:90% And you might use this vertex. 00:56:38.580 --> 00:56:40.550 align:middle line:90% Does this cover all the edges? 00:56:40.550 --> 00:56:42.590 align:middle line:90% No, actually, it doesn't. 00:56:42.590 --> 00:56:43.880 align:middle line:90% So let's not use that vertex. 00:56:43.880 --> 00:56:46.590 align:middle line:90% 00:56:46.590 --> 00:56:47.740 align:middle line:90% You could use this vertex. 00:56:47.740 --> 00:56:50.390 align:middle line:90% 00:56:50.390 --> 00:56:52.780 align:middle line:84% And that actually does cover all the edges because-- 00:56:52.780 --> 00:56:55.470 align:middle line:90% 00:56:55.470 --> 00:56:57.320 align:middle line:90% yeah. 00:56:57.320 --> 00:56:57.820 align:middle line:90% OK. 00:56:57.820 --> 00:57:03.110 align:middle line:90% 00:57:03.110 --> 00:57:03.610 align:middle line:90% Theorem. 00:57:03.610 --> 00:57:07.070 align:middle line:90% 00:57:07.070 --> 00:57:17.190 align:middle line:84% The size of the maximum matching is 00:57:17.190 --> 00:57:20.740 align:middle line:84% equal to the size of the minimum vertex cover. 00:57:20.740 --> 00:57:32.910 align:middle line:90% 00:57:32.910 --> 00:57:33.750 align:middle line:90% OK. 00:57:33.750 --> 00:57:44.300 align:middle line:84% So what we have here is we have four in each of them. 00:57:44.300 --> 00:57:48.620 align:middle line:84% So this is an example where equality holds. 00:57:48.620 --> 00:57:54.860 align:middle line:84% And the Konig's theorem says equality holds in all graphs. 00:57:54.860 --> 00:58:03.050 align:middle line:84% And there's one which one direction is easy. 00:58:03.050 --> 00:58:09.940 align:middle line:90% 00:58:09.940 --> 00:58:19.768 align:middle line:84% Size of any vertex cover has to be 00:58:19.768 --> 00:58:21.810 align:middle line:84% bigger than or equal to the size of any matching. 00:58:21.810 --> 00:58:30.980 align:middle line:90% 00:58:30.980 --> 00:58:33.100 align:middle line:90% And why is this? 00:58:33.100 --> 00:58:39.770 align:middle line:84% Well, I mean, a vertex cover has to cover each of the edges 00:58:39.770 --> 00:58:42.290 align:middle line:84% in the matching, which means it has 00:58:42.290 --> 00:58:45.250 align:middle line:84% to be one of the endpoints of each of the edges of the-- 00:58:45.250 --> 00:58:47.010 align:middle line:84% has to contain one of the endpoints 00:58:47.010 --> 00:58:52.370 align:middle line:84% in each of the edges of the matching, which means that x has 00:58:52.370 --> 00:58:57.850 align:middle line:84% to be the size of the maximum matching, at least. 00:58:57.850 --> 00:59:00.420 align:middle line:84% And Konig's theorem says that they're equal. 00:59:00.420 --> 00:59:11.250 align:middle line:90% 00:59:11.250 --> 00:59:11.750 align:middle line:90% OK. 00:59:11.750 --> 00:59:16.250 align:middle line:90% 00:59:16.250 --> 00:59:19.010 align:middle line:90% So how do we prove equality? 00:59:19.010 --> 00:59:24.770 align:middle line:84% Well, one way to do it is write down a linear program 00:59:24.770 --> 00:59:27.170 align:middle line:90% for the maximum matching. 00:59:27.170 --> 00:59:31.290 align:middle line:84% Write down a linear program for vertex covers. 00:59:31.290 --> 00:59:33.650 align:middle line:84% Show that they're duals, and then 00:59:33.650 --> 00:59:38.830 align:middle line:84% show that there is an integer solution to each of them. 00:59:38.830 --> 00:59:41.410 align:middle line:84% So I do not have time to do all of that, 00:59:41.410 --> 00:59:49.710 align:middle line:84% but we can at least write down the theorem 00:59:49.710 --> 00:59:51.840 align:middle line:90% for the maximum matching. 00:59:51.840 --> 01:00:13.550 align:middle line:90% 01:00:13.550 --> 01:00:15.590 align:middle line:84% How are we going to write down a linear program 01:00:15.590 --> 01:00:18.990 align:middle line:90% for the maximum matching? 01:00:18.990 --> 01:00:20.530 align:middle line:90% What should variables be? 01:00:20.530 --> 01:00:34.340 align:middle line:90% 01:00:34.340 --> 01:00:36.630 align:middle line:90% Edges, maybe? 01:00:36.630 --> 01:00:37.130 align:middle line:90% OK. 01:00:37.130 --> 01:00:53.100 align:middle line:90% 01:00:53.100 --> 01:01:00.760 align:middle line:84% So we want to maximize the sum of x sub e. 01:01:00.760 --> 01:01:03.392 align:middle line:90% 01:01:03.392 --> 01:01:03.892 align:middle line:90% OK. 01:01:03.892 --> 01:01:07.196 align:middle line:90% 01:01:07.196 --> 01:01:14.940 align:middle line:84% And now we want to say that at most one of these edges-- 01:01:14.940 --> 01:01:17.060 align:middle line:84% if we have a vertex here, we want 01:01:17.060 --> 01:01:21.420 align:middle line:84% to say at one of these edges is present in the matching. 01:01:21.420 --> 01:01:24.340 align:middle line:84% And we're not going to be able to do that because these are not 01:01:24.340 --> 01:01:25.400 align:middle line:90% integer variables. 01:01:25.400 --> 01:01:27.580 align:middle line:90% They're real numbers. 01:01:27.580 --> 01:01:30.980 align:middle line:90% But let's write down zero. 01:01:30.980 --> 01:01:43.090 align:middle line:84% That's an xe lesson one and summation v is contained in e. 01:01:43.090 --> 01:01:54.410 align:middle line:84% So for every vertex, the edges containing that vertex x sub e 01:01:54.410 --> 01:01:56.250 align:middle line:90% is less than or equal to 1. 01:01:56.250 --> 01:02:07.450 align:middle line:84% So this is something like there is at most one edge containing 01:02:07.450 --> 01:02:08.000 align:middle line:90% every vertex. 01:02:08.000 --> 01:02:12.170 align:middle line:90% 01:02:12.170 --> 01:02:14.930 align:middle line:84% Which means that we do not have two 01:02:14.930 --> 01:02:18.570 align:middle line:84% vertices, two edges intersecting in vertex and are matching. 01:02:18.570 --> 01:02:21.410 align:middle line:90% 01:02:21.410 --> 01:02:24.450 align:middle line:84% Assuming that all the variables are 0 or 1, 01:02:24.450 --> 01:02:28.020 align:middle line:84% and that's exactly the criterion for the maximum matching. 01:02:28.020 --> 01:02:31.000 align:middle line:90% 01:02:31.000 --> 01:02:31.500 align:middle line:90% OK. 01:02:31.500 --> 01:02:37.920 align:middle line:90% 01:02:37.920 --> 01:02:38.870 align:middle line:90% Let's find the dual. 01:02:38.870 --> 01:02:41.680 align:middle line:90% 01:02:41.680 --> 01:02:43.760 align:middle line:90% Well, the dual-- 01:02:43.760 --> 01:02:49.560 align:middle line:84% I mean, we have an equation for every vertex 01:02:49.560 --> 01:02:54.820 align:middle line:84% and some extra equations and a maximization. 01:02:54.820 --> 01:02:57.080 align:middle line:84% So the dual is going to be a minimization. 01:02:57.080 --> 01:03:03.480 align:middle line:84% It's going to have an equation of variable for every edge, 01:03:03.480 --> 01:03:05.620 align:middle line:84% and it's going to have some extra variables. 01:03:05.620 --> 01:03:09.480 align:middle line:90% 01:03:09.480 --> 01:03:12.110 align:middle line:90% Dual. 01:03:12.110 --> 01:03:12.610 align:middle line:90% OK. 01:03:12.610 --> 01:03:19.080 align:middle line:90% 01:03:19.080 --> 01:03:26.720 align:middle line:90% So one constraint per edge. 01:03:26.720 --> 01:03:31.190 align:middle line:84% And here, we have one per vertex. 01:03:31.190 --> 01:03:33.940 align:middle line:90% 01:03:33.940 --> 01:03:41.670 align:middle line:84% Here we have extra per constraint per edge. 01:03:41.670 --> 01:03:44.740 align:middle line:90% 01:03:44.740 --> 01:03:51.360 align:middle line:84% And I guess we can also have non-negative constraints. 01:03:51.360 --> 01:04:11.780 align:middle line:90% 01:04:11.780 --> 01:04:14.020 align:middle line:90% So let's see. 01:04:14.020 --> 01:04:18.460 align:middle line:90% The dual is going to have-- 01:04:18.460 --> 01:04:24.460 align:middle line:84% I should write down what these equations-- 01:04:24.460 --> 01:04:26.980 align:middle line:90% what these variables are. 01:04:26.980 --> 01:04:27.560 align:middle line:90% Dual. 01:04:27.560 --> 01:04:35.430 align:middle line:84% Let's call this y sub v and z sub e, the equations. 01:04:35.430 --> 01:04:55.690 align:middle line:90% 01:04:55.690 --> 01:05:07.330 align:middle line:90% Now dual equation for each. 01:05:07.330 --> 01:05:08.760 align:middle line:90% Our inequality, I should say. 01:05:08.760 --> 01:05:15.370 align:middle line:90% 01:05:15.370 --> 01:05:19.770 align:middle line:90% For each edge. 01:05:19.770 --> 01:05:24.330 align:middle line:90% 01:05:24.330 --> 01:05:26.760 align:middle line:84% And what are these inequalities going to be? 01:05:26.760 --> 01:05:35.240 align:middle line:84% Well, y sub v appears in the equation for x sub e, 01:05:35.240 --> 01:05:42.760 align:middle line:84% and z sub e appears in the equation for x sub e. 01:05:42.760 --> 01:05:55.860 align:middle line:84% So that means that y sub v plus z sub e. 01:05:55.860 --> 01:06:02.040 align:middle line:90% 01:06:02.040 --> 01:06:03.080 align:middle line:90% Well, let's see. 01:06:03.080 --> 01:06:15.800 align:middle line:84% Minimum summation y sub v. y sub v plus z sub e 01:06:15.800 --> 01:06:20.480 align:middle line:90% is less than or equal to 1. 01:06:20.480 --> 01:06:29.590 align:middle line:90% 01:06:29.590 --> 01:06:30.330 align:middle line:90% Oh, wait. 01:06:30.330 --> 01:06:35.430 align:middle line:90% 01:06:35.430 --> 01:06:37.710 align:middle line:84% This is greater than or equal to. 01:06:37.710 --> 01:06:38.360 align:middle line:90% I'm sorry. 01:06:38.360 --> 01:06:43.430 align:middle line:90% 01:06:43.430 --> 01:06:47.970 align:middle line:90% And yv and z sub e. 01:06:47.970 --> 01:06:59.470 align:middle line:90% 01:06:59.470 --> 01:07:03.270 align:middle line:90% So what does this really mean? 01:07:03.270 --> 01:07:16.910 align:middle line:84% Well, what you want is you want a vertex cover, which 01:07:16.910 --> 01:07:20.210 align:middle line:84% means you want a set of vertices that are adjacent to every edge. 01:07:20.210 --> 01:07:30.610 align:middle line:90% 01:07:30.610 --> 01:07:33.970 align:middle line:84% OK, so actually, I did that wrong. 01:07:33.970 --> 01:07:37.550 align:middle line:90% Minimize y sub v plus z sub e. 01:07:37.550 --> 01:07:49.290 align:middle line:90% 01:07:49.290 --> 01:07:54.190 align:middle line:84% Because the right-hand sides are 1 for y sub v and 1 for z sub e. 01:07:54.190 --> 01:07:59.890 align:middle line:90% 01:07:59.890 --> 01:08:01.570 align:middle line:90% OK. 01:08:01.570 --> 01:08:02.950 align:middle line:90% OK. 01:08:02.950 --> 01:08:06.650 align:middle line:84% I seem to have gotten myself lost here, 01:08:06.650 --> 01:08:14.170 align:middle line:84% but I can go back and talk more about the program 01:08:14.170 --> 01:08:17.580 align:middle line:90% for the primal program. 01:08:17.580 --> 01:08:24.200 align:middle line:90% 01:08:24.200 --> 01:08:26.689 align:middle line:90% And why the primal program is-- 01:08:26.689 --> 01:08:30.939 align:middle line:90% 01:08:30.939 --> 01:08:31.439 align:middle line:90% yeah. 01:08:31.439 --> 01:08:33.380 align:middle line:84% So that's the primal program up here. 01:08:33.380 --> 01:08:43.200 align:middle line:90% 01:08:43.200 --> 01:08:44.310 align:middle line:90% So the primal program. 01:08:44.310 --> 01:08:50.880 align:middle line:90% 01:08:50.880 --> 01:09:00.500 align:middle line:84% A solution is a weight on each edge. 01:09:00.500 --> 01:09:03.479 align:middle line:90% 01:09:03.479 --> 01:09:17.939 align:middle line:84% So some of the weights, we, v in e, is less than or equal to 1. 01:09:17.939 --> 01:09:22.109 align:middle line:90% 01:09:22.109 --> 01:09:28.390 align:middle line:84% So suppose we have fractional solutions. 01:09:28.390 --> 01:09:38.785 align:middle line:84% That means that there's some edge where you might have two-- 01:09:38.785 --> 01:09:40.910 align:middle line:84% some vertices where you might have two edges, which 01:09:40.910 --> 01:09:43.590 align:middle line:90% have 0.5 going out of it. 01:09:43.590 --> 01:09:48.910 align:middle line:84% Now, we might have another edge going out 01:09:48.910 --> 01:09:54.029 align:middle line:84% of 0.5 out of this one, because this plus this is less than 01:09:54.029 --> 01:09:57.450 align:middle line:84% or equal to 1, and we want to maximize it. 01:09:57.450 --> 01:10:01.900 align:middle line:84% So we would have another one going out. 01:10:01.900 --> 01:10:07.590 align:middle line:90% And this might be 0.25. 01:10:07.590 --> 01:10:08.700 align:middle line:90% Let's put one up here. 01:10:08.700 --> 01:10:11.270 align:middle line:90% 01:10:11.270 --> 01:10:16.493 align:middle line:84% Maybe another 0.5 and another 0.5. 01:10:16.493 --> 01:10:16.993 align:middle line:90% OK. 01:10:16.993 --> 01:10:19.891 align:middle line:90% 01:10:19.891 --> 01:10:34.020 align:middle line:84% In a fractional solution, you can always 01:10:34.020 --> 01:10:39.890 align:middle line:84% find a cycle of fractional edges. 01:10:39.890 --> 01:10:45.420 align:middle line:90% 01:10:45.420 --> 01:10:46.749 align:middle line:90% Right? 01:10:46.749 --> 01:10:58.100 align:middle line:84% Because, well, I mean, if you cannot find a-- 01:10:58.100 --> 01:11:00.780 align:middle line:84% if you find a cycle of fractional edges, what 01:11:00.780 --> 01:11:03.900 align:middle line:84% you can do is you can increase the even ones 01:11:03.900 --> 01:11:09.940 align:middle line:84% and decrease the odd ones and set one of them equal to 0 01:11:09.940 --> 01:11:13.390 align:middle line:84% and always get a better solution. 01:11:13.390 --> 01:11:48.440 align:middle line:90% 01:11:48.440 --> 01:11:54.360 align:middle line:84% And first, all of these cycles are even because this 01:11:54.360 --> 01:11:56.320 align:middle line:90% is a bipartite graph. 01:11:56.320 --> 01:11:59.000 align:middle line:84% So that means we can increase the odd ones 01:11:59.000 --> 01:12:06.080 align:middle line:84% and decrease the even ones and we get 1, 0, 1, 0, 1, 0. 01:12:06.080 --> 01:12:12.160 align:middle line:84% And it has the same objective function 01:12:12.160 --> 01:12:13.690 align:middle line:90% and fewer fractional edges. 01:12:13.690 --> 01:12:18.470 align:middle line:90% 01:12:18.470 --> 01:12:23.430 align:middle line:84% In doing this, we can always get rid of the fractional edges. 01:12:23.430 --> 01:12:26.030 align:middle line:84% Now, there's another possibility, which 01:12:26.030 --> 01:12:33.310 align:middle line:90% is you have 0.5, 0.5, 0.5, 0.5. 01:12:33.310 --> 01:12:36.950 align:middle line:84% And what you can do is you can now increase this one to 1, 01:12:36.950 --> 01:12:39.590 align:middle line:84% decrease this to 0, increase this one to 1, 01:12:39.590 --> 01:12:40.640 align:middle line:90% decrease this to 0. 01:12:40.640 --> 01:12:54.350 align:middle line:90% 01:12:54.350 --> 01:12:57.590 align:middle line:84% Because if you can find a path, that 01:12:57.590 --> 01:13:00.430 align:middle line:84% would mean that this vertex does not 01:13:00.430 --> 01:13:04.010 align:middle line:84% have any fractional edges next to it aside from this one, 01:13:04.010 --> 01:13:06.830 align:middle line:84% and this vertex does not have any fractional edges next to it 01:13:06.830 --> 01:13:07.930 align:middle line:90% besides this one. 01:13:07.930 --> 01:13:11.750 align:middle line:84% And that means that you can increase the odd edges 01:13:11.750 --> 01:13:15.620 align:middle line:90% and decrease the even edges. 01:13:15.620 --> 01:13:18.540 align:middle line:84% Either you end up with the same objective function 01:13:18.540 --> 01:13:20.580 align:middle line:84% or you increase the objective function. 01:13:20.580 --> 01:13:22.440 align:middle line:84% But we're assuming this was optimal, 01:13:22.440 --> 01:13:25.760 align:middle line:84% so you always end up with the same objective function. 01:13:25.760 --> 01:13:29.580 align:middle line:84% So you can always decrease the number of fractional edges 01:13:29.580 --> 01:13:33.500 align:middle line:84% and keep the same objective function, which 01:13:33.500 --> 01:13:37.020 align:middle line:84% means that eventually you find a solution where there are 01:13:37.020 --> 01:13:43.380 align:middle line:84% no fractional edges, and that is your maximum matching that you 01:13:43.380 --> 01:13:46.060 align:middle line:90% wanted to find. 01:13:46.060 --> 01:13:50.500 align:middle line:84% And if you take the dual of this and stare at the dual, 01:13:50.500 --> 01:13:54.460 align:middle line:90% you will see that it gives you-- 01:13:54.460 --> 01:13:57.120 align:middle line:84% and I screwed that up when I was trying to do it, 01:13:57.120 --> 01:13:58.420 align:middle line:90% but it will tell you. 01:13:58.420 --> 01:14:01.060 align:middle line:84% You will see that it gives you a vertex cover. 01:14:01.060 --> 01:14:03.200 align:middle line:84% And again, you can have a fractional vertex cover, 01:14:03.200 --> 01:14:05.860 align:middle line:84% but a very similar argument will tell you 01:14:05.860 --> 01:14:13.220 align:middle line:84% that you can find an optimal vertex cover which is an integer 01:14:13.220 --> 01:14:15.170 align:middle line:90% solution to the linear program. 01:14:15.170 --> 01:14:19.090 align:middle line:90% And that shows Konig's theorem. 01:14:19.090 --> 01:14:23.810 align:middle line:84% And we will be doing things that are more like this, taking 01:14:23.810 --> 01:14:28.810 align:middle line:84% linear programs for interesting problems, 01:14:28.810 --> 01:14:35.570 align:middle line:84% and we will do this with zero sum games and maximum flow, 01:14:35.570 --> 01:14:40.770 align:middle line:84% and show first that the fractional solutions can always 01:14:40.770 --> 01:14:49.650 align:middle line:84% be massaged to give you integer solutions for maximum flow. 01:14:49.650 --> 01:14:53.730 align:middle line:84% And this will give us a duality theorem 01:14:53.730 --> 01:14:56.330 align:middle line:84% that the maximum flow equals the minimum cut. 01:14:56.330 --> 01:15:05.770 align:middle line:84% And for zero sum games, a game is a two player things, 01:15:05.770 --> 01:15:10.730 align:middle line:84% and we will show that the primal linear program gives you 01:15:10.730 --> 01:15:13.830 align:middle line:84% the optimal strategy for the first player, 01:15:13.830 --> 01:15:19.350 align:middle line:84% and the dual linear program gives you the optimal strategy 01:15:19.350 --> 01:15:20.910 align:middle line:90% for the second player. 01:15:20.910 --> 01:15:27.270 align:middle line:84% And because they are duals, they have the same objective 01:15:27.270 --> 01:15:27.770 align:middle line:90% function. 01:15:27.770 --> 01:15:31.590 align:middle line:84% And that tells us that if the first player plays 01:15:31.590 --> 01:15:33.870 align:middle line:84% his optimum strategy, he can always 01:15:33.870 --> 01:15:40.750 align:middle line:84% guarantee a win of, say, w on each turn on average. 01:15:40.750 --> 01:15:43.050 align:middle line:84% And if the second player plays his optimal strategy, 01:15:43.050 --> 01:15:49.410 align:middle line:84% he can always achieve a loss of at most w on each turn. 01:15:49.410 --> 01:15:55.470 align:middle line:84% So the first player and the second player, first player 01:15:55.470 --> 01:15:58.550 align:middle line:84% can guarantee he wins plus w, and the second player 01:15:58.550 --> 01:16:01.550 align:middle line:84% can guarantee he loses no more than minus w. 01:16:01.550 --> 01:16:05.740 align:middle line:84% And so this is the value of the game. 01:16:05.740 --> 01:16:12.000 align:middle line:90%