WEBVTT 00:00:00.000 --> 00:00:01.988 align:middle line:90% [SQUEAKING] 00:00:01.988 --> 00:00:03.479 align:middle line:90% [RUSTLING] 00:00:03.479 --> 00:00:06.461 align:middle line:90% [CLICKING] 00:00:06.461 --> 00:00:18.890 align:middle line:90% 00:00:18.890 --> 00:00:22.880 align:middle line:84% ESTHER DUFLO: Today I want to switch back to empirical-- 00:00:22.880 --> 00:00:26.660 align:middle line:90% to examining empirical things. 00:00:26.660 --> 00:00:28.742 align:middle line:90% You guys have read my paper. 00:00:28.742 --> 00:00:30.200 align:middle line:84% I'm going to talk a bit about that. 00:00:30.200 --> 00:00:35.270 align:middle line:84% And I'm going to talk also about it on Monday. 00:00:35.270 --> 00:00:38.558 align:middle line:84% So some of the questions that you 00:00:38.558 --> 00:00:40.100 align:middle line:84% had-- many of the questions, comments 00:00:40.100 --> 00:00:43.460 align:middle line:84% you had were on the returns to education piece. 00:00:43.460 --> 00:00:45.920 align:middle line:84% And those we'll discuss on Monday. 00:00:45.920 --> 00:00:47.540 align:middle line:84% But there are a number of things that 00:00:47.540 --> 00:00:52.350 align:middle line:84% come up that I should be able to address today. 00:00:52.350 --> 00:00:55.985 align:middle line:84% So if you remember, we ended up last time by saying-- 00:00:55.985 --> 00:00:59.450 align:middle line:90% or we at some point last-- 00:00:59.450 --> 00:01:04.430 align:middle line:84% on Monday, we had this equation that the level of education 00:01:04.430 --> 00:01:06.590 align:middle line:84% chosen by a family for their kids 00:01:06.590 --> 00:01:10.070 align:middle line:84% is going to be a function of the cost of education, 00:01:10.070 --> 00:01:13.010 align:middle line:84% the marginal cost for them of acquiring one more 00:01:13.010 --> 00:01:18.080 align:middle line:84% unit of human capital, and of the returns to education, 00:01:18.080 --> 00:01:21.800 align:middle line:84% both the immediate financial return in the form of wages 00:01:21.800 --> 00:01:25.280 align:middle line:90% as well as the human-- 00:01:25.280 --> 00:01:27.530 align:middle line:84% how is it going to acquire the human capital 00:01:27.530 --> 00:01:29.540 align:middle line:90% of the next generation. 00:01:29.540 --> 00:01:35.390 align:middle line:84% So these are all things that people have looked at. 00:01:35.390 --> 00:01:37.550 align:middle line:84% So we should expect people to respond both 00:01:37.550 --> 00:01:39.630 align:middle line:90% to costs and benefits. 00:01:39.630 --> 00:01:44.120 align:middle line:84% So the cost piece seems relatively straightforward. 00:01:44.120 --> 00:01:47.570 align:middle line:84% One would look for a setting where 00:01:47.570 --> 00:01:51.800 align:middle line:84% cost to education direct or indirect went down 00:01:51.800 --> 00:01:56.150 align:middle line:84% and then look at whether people respond 00:01:56.150 --> 00:01:59.030 align:middle line:90% by educating their kids more. 00:01:59.030 --> 00:02:01.550 align:middle line:84% And then, the benefits is whether people respond 00:02:01.550 --> 00:02:02.870 align:middle line:90% to returns to education. 00:02:02.870 --> 00:02:05.870 align:middle line:84% That might be a little bit more tricky. 00:02:05.870 --> 00:02:10.813 align:middle line:84% And we are going to do a bit of that, both the actual benefits 00:02:10.813 --> 00:02:12.230 align:middle line:84% and the perceived benefits, which, 00:02:12.230 --> 00:02:14.120 align:middle line:90% of course, is in this equation. 00:02:14.120 --> 00:02:17.060 align:middle line:84% Implicitly, something we hardly discussed but just a little 00:02:17.060 --> 00:02:20.930 align:middle line:84% bit last time is whether the perceived benefit are 00:02:20.930 --> 00:02:23.833 align:middle line:90% the same as the real benefits. 00:02:23.833 --> 00:02:25.250 align:middle line:84% So let's start with the cost which 00:02:25.250 --> 00:02:28.340 align:middle line:90% are the straightforward piece. 00:02:28.340 --> 00:02:32.510 align:middle line:84% So there are a number of papers that look at the reaction 00:02:32.510 --> 00:02:35.000 align:middle line:90% to the direct cost of education. 00:02:35.000 --> 00:02:40.100 align:middle line:84% So in particular, but this is a bit 00:02:40.100 --> 00:02:41.707 align:middle line:84% of a biased review of the literature, 00:02:41.707 --> 00:02:43.040 align:middle line:90% because it has mostly my papers. 00:02:43.040 --> 00:02:43.540 align:middle line:90% [LAUGHTER] 00:02:43.540 --> 00:02:47.900 align:middle line:84% But I have a paper with Pascaline Dupas and Michael 00:02:47.900 --> 00:02:53.900 align:middle line:84% Kremer where we provide the school uniforms for kids 00:02:53.900 --> 00:02:56.810 align:middle line:84% entering grade seven or grade-- yeah, 00:02:56.810 --> 00:02:59.510 align:middle line:90% grade six, or seven, or eight. 00:02:59.510 --> 00:03:03.060 align:middle line:84% And we found a pretty large effect there. 00:03:03.060 --> 00:03:04.910 align:middle line:84% We then have a literature that we'll 00:03:04.910 --> 00:03:06.750 align:middle line:90% discuss in much more detail. 00:03:06.750 --> 00:03:09.660 align:middle line:84% We have a paper on the secondary school to education, 00:03:09.660 --> 00:03:13.910 align:middle line:84% which are much larger than the school uniform, which 00:03:13.910 --> 00:03:15.800 align:middle line:84% we'll discuss in detail on Monday 00:03:15.800 --> 00:03:21.440 align:middle line:84% and you're going to read, which again show pretty large impact. 00:03:21.440 --> 00:03:23.150 align:middle line:84% In a sense, what is-- and Michael 00:03:23.150 --> 00:03:25.910 align:middle line:84% has other papers with other people on various forms 00:03:25.910 --> 00:03:29.360 align:middle line:90% of scholarships for schools. 00:03:29.360 --> 00:03:33.410 align:middle line:84% So this literature is interesting, 00:03:33.410 --> 00:03:35.720 align:middle line:84% in particular the uniform literature. 00:03:35.720 --> 00:03:39.680 align:middle line:84% Because on the one hand, you're seeing effect-- 00:03:39.680 --> 00:03:43.340 align:middle line:84% impact of paying for uniform on schooling. 00:03:43.340 --> 00:03:46.460 align:middle line:84% So that would suggest that people are appropriately 00:03:46.460 --> 00:03:48.140 align:middle line:90% responding to costs. 00:03:48.140 --> 00:03:53.030 align:middle line:84% On the other hand, these impacts are really large-- 00:03:53.030 --> 00:03:55.230 align:middle line:90% really, really, really large. 00:03:55.230 --> 00:03:57.890 align:middle line:84% So you get, for example, in our uniform paper 00:03:57.890 --> 00:04:02.120 align:middle line:84% you get reduction in dropout of a few percentage point. 00:04:02.120 --> 00:04:06.460 align:middle line:84% And the cost of a school uniform is $5 or so. 00:04:06.460 --> 00:04:10.260 align:middle line:84% So which even for a Kenyan family is not that big. 00:04:10.260 --> 00:04:15.470 align:middle line:84% So that is-- on the one hand, you get the magnitude rate. 00:04:15.470 --> 00:04:18.019 align:middle line:84% On the other hand, it is not really, 00:04:18.019 --> 00:04:20.720 align:middle line:84% I would say, a vindication of this model because the impacts 00:04:20.720 --> 00:04:22.310 align:middle line:90% are too large. 00:04:22.310 --> 00:04:26.120 align:middle line:84% In the sense that if the only reason why 00:04:26.120 --> 00:04:28.460 align:middle line:84% the cost to education enter is you're 00:04:28.460 --> 00:04:31.790 align:middle line:84% comparing them to the interest rate, then clearly-- 00:04:31.790 --> 00:04:34.430 align:middle line:84% or not to-- I mean, to the benefits 00:04:34.430 --> 00:04:37.250 align:middle line:84% and appropriately discounted by the interest rate, 00:04:37.250 --> 00:04:40.370 align:middle line:90% and that seems way too large. 00:04:40.370 --> 00:04:42.950 align:middle line:84% These are indicating huge credit constraint 00:04:42.950 --> 00:04:47.600 align:middle line:84% that they can't even $5, or something else is going on. 00:04:47.600 --> 00:04:48.378 align:middle line:90% Yeah. 00:04:48.378 --> 00:04:50.420 align:middle line:84% STUDENT: When you were designing this experiment, 00:04:50.420 --> 00:04:54.656 align:middle line:84% why did you think about uniforms [INAUDIBLE] 00:04:54.656 --> 00:04:56.650 align:middle line:90% and cost of [INAUDIBLE]? 00:04:56.650 --> 00:04:59.440 align:middle line:84% ESTHER DUFLO: Everybody says it's a big deal. 00:04:59.440 --> 00:05:00.950 align:middle line:90% Everybody says it's a big deal. 00:05:00.950 --> 00:05:03.070 align:middle line:90% In fact, we got to uniform-- 00:05:03.070 --> 00:05:10.190 align:middle line:84% this is a paper where we looked at HIV. 00:05:10.190 --> 00:05:13.810 align:middle line:84% We were very interested in testing 00:05:13.810 --> 00:05:20.840 align:middle line:84% the impact of the government HIV school intervention. 00:05:20.840 --> 00:05:23.950 align:middle line:84% So the government in Kenya was introducing a new program 00:05:23.950 --> 00:05:31.390 align:middle line:84% for HIV, which we actually were reasonably skeptical about. 00:05:31.390 --> 00:05:33.197 align:middle line:84% This is called the ABCD Strategy, 00:05:33.197 --> 00:05:35.530 align:middle line:84% which was followed at the time by most countries in East 00:05:35.530 --> 00:05:37.300 align:middle line:90% Africa. 00:05:37.300 --> 00:05:40.300 align:middle line:84% That stands for abstain, be careful, use a condom, 00:05:40.300 --> 00:05:41.398 align:middle line:90% or you die. 00:05:41.398 --> 00:05:43.270 align:middle line:90% [LAUGHTER] 00:05:43.270 --> 00:05:48.070 align:middle line:84% That was relatively a kind of fear mongering 00:05:48.070 --> 00:05:49.490 align:middle line:90% among-- in this order. 00:05:49.490 --> 00:05:51.010 align:middle line:90% So abstain is the best. 00:05:51.010 --> 00:05:53.230 align:middle line:84% Be safe, use a condom, or you die. 00:05:53.230 --> 00:05:59.530 align:middle line:84% So this was an example of what's called a avoiding all risk-- 00:05:59.530 --> 00:06:01.450 align:middle line:84% risk avoidance strategy as opposed 00:06:01.450 --> 00:06:04.430 align:middle line:90% to risk mitigation strategy. 00:06:04.430 --> 00:06:07.930 align:middle line:84% So this program was introduced among upper primary school 00:06:07.930 --> 00:06:08.470 align:middle line:90% students. 00:06:08.470 --> 00:06:11.230 align:middle line:84% This is grade six to eight in Kenya. 00:06:11.230 --> 00:06:13.460 align:middle line:84% They were interested in evaluating 00:06:13.460 --> 00:06:15.130 align:middle line:84% and we were interested in evaluating it. 00:06:15.130 --> 00:06:17.170 align:middle line:84% But at the same time, we were quite skeptical. 00:06:17.170 --> 00:06:20.590 align:middle line:84% And we wanted to compare that with the strategy that is just 00:06:20.590 --> 00:06:26.920 align:middle line:84% keeping girls in school as a direct protective measure 00:06:26.920 --> 00:06:29.070 align:middle line:84% as opposed to trying to teach them something on HIV 00:06:29.070 --> 00:06:30.070 align:middle line:90% when they are in school. 00:06:30.070 --> 00:06:32.000 align:middle line:90% So we wanted to have that. 00:06:32.000 --> 00:06:37.300 align:middle line:84% And it was pretty obvious to us that for kids their age, 00:06:37.300 --> 00:06:39.040 align:middle line:90% the uniform is a big constraint. 00:06:39.040 --> 00:06:41.110 align:middle line:84% They cannot go to school without the uniform. 00:06:41.110 --> 00:06:44.380 align:middle line:84% The school send them back without the uniform. 00:06:44.380 --> 00:06:46.207 align:middle line:84% Initially, when we started the project, 00:06:46.207 --> 00:06:48.290 align:middle line:84% conceiving of the project, there were school fees. 00:06:48.290 --> 00:06:50.350 align:middle line:84% So we were going to pay for the school fee. 00:06:50.350 --> 00:06:52.870 align:middle line:90% And the school fee went away. 00:06:52.870 --> 00:06:55.550 align:middle line:84% But we realized that the uniform were still a barrier. 00:06:55.550 --> 00:06:57.020 align:middle line:84% So that's why we did the program. 00:06:57.020 --> 00:07:00.190 align:middle line:84% Even though, if you think about it 00:07:00.190 --> 00:07:02.650 align:middle line:84% from the comfort of your office, people 00:07:02.650 --> 00:07:04.690 align:middle line:84% shouldn't really be responsive to the cost-- 00:07:04.690 --> 00:07:06.430 align:middle line:90% to the presence of the uniform. 00:07:06.430 --> 00:07:08.800 align:middle line:90% But as it turns out, they are. 00:07:08.800 --> 00:07:11.500 align:middle line:90% So that's how we came to that. 00:07:11.500 --> 00:07:13.570 align:middle line:90% It was less of-- 00:07:13.570 --> 00:07:17.410 align:middle line:84% and the results are pretty interesting in the sense that 00:07:17.410 --> 00:07:22.180 align:middle line:84% keeping kids in school, keeping girls in school 00:07:22.180 --> 00:07:24.340 align:middle line:90% does reduce HIV. 00:07:24.340 --> 00:07:26.870 align:middle line:90% That program doesn't work. 00:07:26.870 --> 00:07:29.230 align:middle line:84% And in fact, that program undoes the effect 00:07:29.230 --> 00:07:32.170 align:middle line:90% of keeping girls in school. 00:07:32.170 --> 00:07:34.210 align:middle line:84% So it is better to just pay for the uniform 00:07:34.210 --> 00:07:35.740 align:middle line:90% and do nothing else. 00:07:35.740 --> 00:07:36.790 align:middle line:90% That's worse. 00:07:36.790 --> 00:07:39.490 align:middle line:84% It is worse than trying to teach them the ABCD program 00:07:39.490 --> 00:07:41.200 align:middle line:90% while they're in school. 00:07:41.200 --> 00:07:43.720 align:middle line:84% For reasons that we probably will get into 00:07:43.720 --> 00:07:46.070 align:middle line:90% and we talk more about health. 00:07:46.070 --> 00:07:49.490 align:middle line:84% So that that's how this came about. 00:07:49.490 --> 00:07:52.330 align:middle line:84% So this is something that everybody talks about, 00:07:52.330 --> 00:07:55.390 align:middle line:84% the school uniform, even though this is not 00:07:55.390 --> 00:07:57.472 align:middle line:90% consistent with this model. 00:07:57.472 --> 00:07:59.680 align:middle line:84% But that's probably a failure of the model as opposed 00:07:59.680 --> 00:08:02.860 align:middle line:84% to a failure of people's thinking. 00:08:02.860 --> 00:08:04.550 align:middle line:84% So that's the first set of costs. 00:08:04.550 --> 00:08:09.490 align:middle line:84% So I would say, just as a vindication of this model, 00:08:09.490 --> 00:08:12.132 align:middle line:84% it's not very-- it's a bit half-half. 00:08:12.132 --> 00:08:14.590 align:middle line:84% Because on the one hand, you see people responsive to cost. 00:08:14.590 --> 00:08:17.570 align:middle line:84% On the other hand, they're way too responsive. 00:08:17.570 --> 00:08:19.490 align:middle line:84% Then there are the indirect cost, 00:08:19.490 --> 00:08:22.540 align:middle line:84% which is the opportunity cost of being in school. 00:08:22.540 --> 00:08:25.720 align:middle line:84% So while you're in school, you don't make money 00:08:25.720 --> 00:08:27.250 align:middle line:90% for the family. 00:08:27.250 --> 00:08:30.850 align:middle line:84% And it is believed, in particular for slightly 00:08:30.850 --> 00:08:33.280 align:middle line:84% older students, middle school, high school, 00:08:33.280 --> 00:08:37.659 align:middle line:84% that it prevents some kids from being in school. 00:08:37.659 --> 00:08:40.539 align:middle line:84% And this is one of the motivation of conditional cash 00:08:40.539 --> 00:08:42.880 align:middle line:84% transfer programs which were introduced initially 00:08:42.880 --> 00:08:47.320 align:middle line:84% in Latin America, which has a component that 00:08:47.320 --> 00:08:51.550 align:middle line:84% is conditional on sending kids to school. 00:08:51.550 --> 00:08:55.390 align:middle line:84% And the idea is that that's actually a fair amount of money 00:08:55.390 --> 00:08:58.300 align:middle line:84% that people get from the conditional cash transfer. 00:08:58.300 --> 00:09:01.750 align:middle line:84% And it generates-- so as long as your kids goes to school 00:09:01.750 --> 00:09:03.970 align:middle line:84% regularly and is not absent too much-- 00:09:03.970 --> 00:09:05.590 align:middle line:84% is enrolled and not absent too much-- 00:09:05.590 --> 00:09:08.050 align:middle line:90% you get a transfer monthly. 00:09:08.050 --> 00:09:14.170 align:middle line:84% So that sort of compensate for not having their earnings 00:09:14.170 --> 00:09:18.550 align:middle line:84% if they worked on a market or on a farm or something like that. 00:09:18.550 --> 00:09:21.880 align:middle line:84% So that's also a way of manipulating 00:09:21.880 --> 00:09:24.490 align:middle line:84% the cost of education by creating a benefit that 00:09:24.490 --> 00:09:27.350 align:middle line:90% offsets an existing cost. 00:09:27.350 --> 00:09:30.280 align:middle line:84% So that's a form of indirect cash transfer. 00:09:30.280 --> 00:09:33.310 align:middle line:90% 00:09:33.310 --> 00:09:35.170 align:middle line:84% There, again, I would say the results-- 00:09:35.170 --> 00:09:39.520 align:middle line:84% the evidence towards in favor of this model 00:09:39.520 --> 00:09:41.410 align:middle line:90% are relatively-- is mixed. 00:09:41.410 --> 00:09:43.780 align:middle line:84% Because on the one hand, people are 00:09:43.780 --> 00:09:47.740 align:middle line:84% very responsive to conditional cash transfer. 00:09:47.740 --> 00:09:51.160 align:middle line:84% They have now been evaluating in dozens of countries, 00:09:51.160 --> 00:09:54.000 align:middle line:84% starting with the evaluation in Mexico, which 00:09:54.000 --> 00:10:00.580 align:middle line:84% was one of the first big prominent city in development. 00:10:00.580 --> 00:10:03.040 align:middle line:84% It was replicated in lots of countries. 00:10:03.040 --> 00:10:07.360 align:middle line:84% And maybe bizarrely enough, because the first one 00:10:07.360 --> 00:10:08.620 align:middle line:90% was done with an experiment. 00:10:08.620 --> 00:10:11.050 align:middle line:84% Everybody thought they had to have their experiment too. 00:10:11.050 --> 00:10:14.980 align:middle line:84% So it was experimented in many, many, many places. 00:10:14.980 --> 00:10:18.880 align:middle line:84% And you tend to always find an impact 00:10:18.880 --> 00:10:23.350 align:middle line:84% on education, particularly for these older kids. 00:10:23.350 --> 00:10:26.230 align:middle line:84% So that would be good that people respond 00:10:26.230 --> 00:10:27.300 align:middle line:90% to the cost of education. 00:10:27.300 --> 00:10:32.290 align:middle line:84% However, they also responds when the cash transfer 00:10:32.290 --> 00:10:35.160 align:middle line:90% are unconditional. 00:10:35.160 --> 00:10:38.970 align:middle line:84% And there, again, remember, in our model an easier-- 00:10:38.970 --> 00:10:43.617 align:middle line:84% just a lump sum transfer should not have an impact. 00:10:43.617 --> 00:10:46.200 align:middle line:84% Well, you could say maybe it's credit-constrained or something 00:10:46.200 --> 00:10:46.780 align:middle line:90% like that. 00:10:46.780 --> 00:10:54.340 align:middle line:84% But it's certainly not directly a response 00:10:54.340 --> 00:10:55.510 align:middle line:90% to the cost of education. 00:10:55.510 --> 00:10:58.570 align:middle line:90% So people are just about-- 00:10:58.570 --> 00:11:01.210 align:middle line:84% in some studies you find them somewhat more responsive 00:11:01.210 --> 00:11:02.770 align:middle line:90% to conditional cash transfer. 00:11:02.770 --> 00:11:04.940 align:middle line:84% In many studies, there is no difference. 00:11:04.940 --> 00:11:07.090 align:middle line:84% So we did one in Morocco, for example, 00:11:07.090 --> 00:11:12.550 align:middle line:84% where we found that conditional and unconditional transfer are 00:11:12.550 --> 00:11:15.280 align:middle line:90% more or less equivalent. 00:11:15.280 --> 00:11:16.960 align:middle line:84% With an unconditional cash transfer, 00:11:16.960 --> 00:11:18.790 align:middle line:84% possibly a little more effective, actually, 00:11:18.790 --> 00:11:21.690 align:middle line:84% than the conditional cash transfer. 00:11:21.690 --> 00:11:24.060 align:middle line:90% So again, that's kind of mixed. 00:11:24.060 --> 00:11:26.310 align:middle line:84% While people respond to things that 00:11:26.310 --> 00:11:28.470 align:middle line:84% are meant to send them to school, 00:11:28.470 --> 00:11:30.960 align:middle line:90% maybe meant to reduce the cost. 00:11:30.960 --> 00:11:34.860 align:middle line:84% But they seem to do that even though the transfer is not 00:11:34.860 --> 00:11:37.910 align:middle line:90% directly affected to the cost. 00:11:37.910 --> 00:11:39.700 align:middle line:84% And then, of course, there is another form 00:11:39.700 --> 00:11:44.170 align:middle line:84% of reducing the indirect cost is reducing 00:11:44.170 --> 00:11:45.880 align:middle line:90% the distance to school. 00:11:45.880 --> 00:11:47.770 align:middle line:84% So some would ask, how do you think 00:11:47.770 --> 00:11:50.050 align:middle line:84% about this intervention-- the school construction 00:11:50.050 --> 00:11:51.010 align:middle line:90% intervention? 00:11:51.010 --> 00:11:53.740 align:middle line:84% And one very natural way to think about it 00:11:53.740 --> 00:11:56.800 align:middle line:84% is you reduce the cost of going to school by making 00:11:56.800 --> 00:12:00.580 align:middle line:84% the school very nearby so it's much easier to go there 00:12:00.580 --> 00:12:02.110 align:middle line:84% for the child and for the parents. 00:12:02.110 --> 00:12:04.960 align:middle line:90% 00:12:04.960 --> 00:12:09.490 align:middle line:84% So that's the idea of my paper in Indonesia. 00:12:09.490 --> 00:12:15.440 align:middle line:84% So someone asked how I came about with this idea. 00:12:15.440 --> 00:12:17.950 align:middle line:90% So maybe that's relevant. 00:12:17.950 --> 00:12:22.120 align:middle line:84% Second-years I've already said that because I mentioned it 00:12:22.120 --> 00:12:23.770 align:middle line:90% in the paper writing class. 00:12:23.770 --> 00:12:25.822 align:middle line:84% But that turned out to be my job market paper 00:12:25.822 --> 00:12:27.280 align:middle line:84% although I didn't know it was going 00:12:27.280 --> 00:12:29.560 align:middle line:90% to be my job market paper. 00:12:29.560 --> 00:12:33.100 align:middle line:84% But I was sitting in development and in labor. 00:12:33.100 --> 00:12:36.250 align:middle line:84% And in labor we only talked about instrumenting 00:12:36.250 --> 00:12:38.830 align:middle line:84% for education, saying it's so important to instrument 00:12:38.830 --> 00:12:41.110 align:middle line:84% for education because there might be ability biased 00:12:41.110 --> 00:12:44.850 align:middle line:84% people who are more likely to go to school 00:12:44.850 --> 00:12:48.413 align:middle line:84% are also maybe more competent in various ways. 00:12:48.413 --> 00:12:50.330 align:middle line:84% And therefore, they'll earn more money anyway, 00:12:50.330 --> 00:12:51.990 align:middle line:90% even if they don't go to school. 00:12:51.990 --> 00:12:54.610 align:middle line:90% So that's the ability bias. 00:12:54.610 --> 00:12:57.460 align:middle line:84% I thought, well, in development that seems worse. 00:12:57.460 --> 00:13:01.120 align:middle line:84% Because it's not just your ability bias that, in fact, 00:13:01.120 --> 00:13:04.270 align:middle line:84% doesn't seem to really be there in rich countries. 00:13:04.270 --> 00:13:07.120 align:middle line:90% You don't seem to have a-- 00:13:07.120 --> 00:13:08.770 align:middle line:84% to change the return to education 00:13:08.770 --> 00:13:12.340 align:middle line:84% much when you control for this selection bias. 00:13:12.340 --> 00:13:14.230 align:middle line:84% But I thought, probably in development things 00:13:14.230 --> 00:13:15.040 align:middle line:90% would be quite different. 00:13:15.040 --> 00:13:17.650 align:middle line:84% Because there are many reasons why kids would go to school, 00:13:17.650 --> 00:13:19.990 align:middle line:84% like in particular, the connections of their parents, 00:13:19.990 --> 00:13:23.950 align:middle line:84% the money of their parents, that might be related with someone's 00:13:23.950 --> 00:13:27.410 align:middle line:90% ability to get a job later. 00:13:27.410 --> 00:13:32.800 align:middle line:84% So I thought the ability bias would have to be larger. 00:13:32.800 --> 00:13:34.660 align:middle line:84% And then I was very surprised that there 00:13:34.660 --> 00:13:39.223 align:middle line:84% was no study using instrumental variable in development. 00:13:39.223 --> 00:13:41.140 align:middle line:84% So I thought, someone should write this study. 00:13:41.140 --> 00:13:41.650 align:middle line:90% [LAUGHTER] 00:13:41.650 --> 00:13:43.570 align:middle line:84% And I was trying to think about what 00:13:43.570 --> 00:13:45.580 align:middle line:84% would be a good instrument, and thinking 00:13:45.580 --> 00:13:48.363 align:middle line:84% about distance to school being a good instrument. 00:13:48.363 --> 00:13:49.780 align:middle line:84% But I'm thinking, well, that's not 00:13:49.780 --> 00:13:52.240 align:middle line:84% going to work because if people are closer to school, 00:13:52.240 --> 00:13:55.010 align:middle line:84% or probably are in more urban setting. 00:13:55.010 --> 00:13:57.190 align:middle line:84% So what you really need is a change 00:13:57.190 --> 00:13:59.050 align:middle line:90% in the distance to school. 00:13:59.050 --> 00:14:01.820 align:middle line:84% Well, where would that come from? 00:14:01.820 --> 00:14:04.210 align:middle line:84% Well, if a country started building a lot of schools, 00:14:04.210 --> 00:14:07.580 align:middle line:84% that would reduce the distance to school. 00:14:07.580 --> 00:14:11.060 align:middle line:84% Is there a country that has built a lot of schools? 00:14:11.060 --> 00:14:12.970 align:middle line:84% And so, in the back of my mind, I 00:14:12.970 --> 00:14:15.220 align:middle line:84% was looking for that country, which 00:14:15.220 --> 00:14:16.690 align:middle line:90% had built a lot of schools. 00:14:16.690 --> 00:14:20.088 align:middle line:84% Now, separately at the time, we didn't 00:14:20.088 --> 00:14:21.130 align:middle line:90% have a second-year paper. 00:14:21.130 --> 00:14:23.020 align:middle line:84% But we had to write an econometrics paper. 00:14:23.020 --> 00:14:25.150 align:middle line:84% And I had written mine on Indonesia 00:14:25.150 --> 00:14:27.910 align:middle line:84% because there was a good data set that a lot of people 00:14:27.910 --> 00:14:30.520 align:middle line:84% use to do any number of things called the Indonesian family 00:14:30.520 --> 00:14:32.170 align:middle line:90% life survey. 00:14:32.170 --> 00:14:34.330 align:middle line:90% So I started with Indonesia. 00:14:34.330 --> 00:14:35.620 align:middle line:90% And I went to the library. 00:14:35.620 --> 00:14:37.570 align:middle line:84% And they have all the World Bank reports. 00:14:37.570 --> 00:14:40.550 align:middle line:84% One of you found this report that I found-- 00:14:40.550 --> 00:14:44.152 align:middle line:84% 1989 report that-- but they have a bunch on all of the countries 00:14:44.152 --> 00:14:45.610 align:middle line:84% and what they are doing every year. 00:14:45.610 --> 00:14:47.235 align:middle line:84% And I said, let me start with Indonesia 00:14:47.235 --> 00:14:49.870 align:middle line:84% since there is this data set which I already know. 00:14:49.870 --> 00:14:52.385 align:middle line:90% And then I found that. 00:14:52.385 --> 00:14:53.260 align:middle line:90% I said, that's great. 00:14:53.260 --> 00:14:54.430 align:middle line:90% That's a lot of schools. 00:14:54.430 --> 00:14:57.490 align:middle line:84% That should reduce the number of schools a lot. 00:14:57.490 --> 00:15:02.077 align:middle line:84% So let me see whether I can do something with that. 00:15:02.077 --> 00:15:04.160 align:middle line:84% And then I quickly realized that, so first of all, 00:15:04.160 --> 00:15:07.370 align:middle line:84% I had to found a number of schools. 00:15:07.370 --> 00:15:10.390 align:middle line:84% And I had to find a large data set. 00:15:10.390 --> 00:15:12.280 align:middle line:84% I figured that such large data set 00:15:12.280 --> 00:15:17.290 align:middle line:84% exists, maybe from that same report, that they were using 00:15:17.290 --> 00:15:20.950 align:middle line:84% it, this kind of large census and inter-census survey. 00:15:20.950 --> 00:15:23.510 align:middle line:84% So I knew that this data set existed. 00:15:23.510 --> 00:15:26.650 align:middle line:84% I didn't know that the number-- the data 00:15:26.650 --> 00:15:29.970 align:middle line:84% and the number of school existed. 00:15:29.970 --> 00:15:32.100 align:middle line:84% But I thought, probably it's somewhere. 00:15:32.100 --> 00:15:35.830 align:middle line:84% It has to be in some library in Indonesia. 00:15:35.830 --> 00:15:41.400 align:middle line:84% So I went to Indonesia during the summer. 00:15:41.400 --> 00:15:46.260 align:middle line:84% And I hunted for the data which I knew existed. 00:15:46.260 --> 00:15:47.970 align:middle line:90% That's the outcome data. 00:15:47.970 --> 00:15:50.550 align:middle line:84% I had to go to the statistical office to find it. 00:15:50.550 --> 00:15:53.820 align:middle line:90% 00:15:53.820 --> 00:15:58.317 align:middle line:84% I think-- so anyway, I paid some money to someone, 00:15:58.317 --> 00:15:59.400 align:middle line:90% and they gave me the data. 00:15:59.400 --> 00:16:01.260 align:middle line:90% [LAUGHTER] 00:16:01.260 --> 00:16:03.960 align:middle line:90% And I didn't poke too much. 00:16:03.960 --> 00:16:04.890 align:middle line:90% But that happened. 00:16:04.890 --> 00:16:05.390 align:middle line:90% [LAUGHTER] 00:16:05.390 --> 00:16:06.720 align:middle line:90% Now it's much easier. 00:16:06.720 --> 00:16:09.030 align:middle line:84% The data, since then, the data from Indonesia 00:16:09.030 --> 00:16:11.390 align:middle line:90% is very, very easily available. 00:16:11.390 --> 00:16:14.280 align:middle line:84% The Australian National Agency Fund 00:16:14.280 --> 00:16:18.240 align:middle line:84% gave them a lot of data to make their data available. 00:16:18.240 --> 00:16:19.770 align:middle line:90% And then I went to the library. 00:16:19.770 --> 00:16:21.660 align:middle line:84% I knew who had implemented this program. 00:16:21.660 --> 00:16:23.368 align:middle line:90% There was a planning department. 00:16:23.368 --> 00:16:25.410 align:middle line:84% I went to the library of the planning department. 00:16:25.410 --> 00:16:28.200 align:middle line:84% I asked them, would you have this data? 00:16:28.200 --> 00:16:34.410 align:middle line:84% And they gave me books which showed the number of schools 00:16:34.410 --> 00:16:39.360 align:middle line:84% that were built each year, the plan of the schools, et cetera. 00:16:39.360 --> 00:16:41.853 align:middle line:84% I made some photocopy, and I was done. 00:16:41.853 --> 00:16:44.020 align:middle line:84% And then I had two more weeks to spend in Indonesia. 00:16:44.020 --> 00:16:45.030 align:middle line:90% And I got very bored. 00:16:45.030 --> 00:16:47.795 align:middle line:90% That was the end of that. 00:16:47.795 --> 00:16:49.890 align:middle line:90% So that's how I came to that. 00:16:49.890 --> 00:16:52.890 align:middle line:84% Everything lined up quite nicely. 00:16:52.890 --> 00:16:56.610 align:middle line:84% It doesn't have-- anyway, it's not always like that. 00:16:56.610 --> 00:16:59.340 align:middle line:84% There is any number of places where this project could 00:16:59.340 --> 00:17:00.300 align:middle line:90% have collapsed. 00:17:00.300 --> 00:17:05.579 align:middle line:84% But in this instance, I was lucky at every step. 00:17:05.579 --> 00:17:09.510 align:middle line:84% Interestingly, I was initially interested-- 00:17:09.510 --> 00:17:11.880 align:middle line:84% so I had no idea how good an idea it was. 00:17:11.880 --> 00:17:13.859 align:middle line:90% I just thought, this is-- 00:17:13.859 --> 00:17:17.190 align:middle line:84% I gave this advice to someone yesterday. 00:17:17.190 --> 00:17:21.720 align:middle line:84% And I think it's good advice, that it's very hard when 00:17:21.720 --> 00:17:25.020 align:middle line:84% you start working on a research project 00:17:25.020 --> 00:17:27.990 align:middle line:84% to think about whether that's going to be an A-plus 00:17:27.990 --> 00:17:30.180 align:middle line:84% contribution, or a B-minus contribution, 00:17:30.180 --> 00:17:32.730 align:middle line:90% or no contribution at all. 00:17:32.730 --> 00:17:33.790 align:middle line:90% It's hard for you. 00:17:33.790 --> 00:17:36.580 align:middle line:84% And it's even hard for people who talk to you. 00:17:36.580 --> 00:17:38.910 align:middle line:84% So I think the best guide that I have found 00:17:38.910 --> 00:17:43.060 align:middle line:84% is to write the paper that I wish I had read. 00:17:43.060 --> 00:17:45.450 align:middle line:84% I want to write the paper that I wish to read. 00:17:45.450 --> 00:17:49.210 align:middle line:84% And then you know that at least one person will like it. 00:17:49.210 --> 00:17:49.710 align:middle line:90% [LAUGHTER] 00:17:49.710 --> 00:17:52.477 align:middle line:84% And a pretty important person, because that's yourself, 00:17:52.477 --> 00:17:54.810 align:middle line:84% and you're going to spend a lot of time with that paper. 00:17:54.810 --> 00:17:57.100 align:middle line:90% And the rest works itself out. 00:17:57.100 --> 00:18:00.540 align:middle line:84% And I just had no idea that in and of itself doing 00:18:00.540 --> 00:18:02.340 align:middle line:90% all of these step was so key. 00:18:02.340 --> 00:18:03.960 align:middle line:90% It seemed to me very easy. 00:18:03.960 --> 00:18:07.740 align:middle line:84% I thought of it as a scales, like doing scales, 00:18:07.740 --> 00:18:11.040 align:middle line:84% to take a strategy that is very, very well-established somewhere 00:18:11.040 --> 00:18:14.100 align:middle line:84% and transpose it to another context. 00:18:14.100 --> 00:18:16.650 align:middle line:84% And in fact, in my mind, it was going 00:18:16.650 --> 00:18:20.220 align:middle line:84% to help me think about non-parametric returns 00:18:20.220 --> 00:18:21.210 align:middle line:90% to education. 00:18:21.210 --> 00:18:26.070 align:middle line:84% I had read a paper which was very new at the time by Whitney 00:18:26.070 --> 00:18:32.370 align:middle line:84% Newey about estimating non-parametric system with 00:18:32.370 --> 00:18:34.140 align:middle line:90% triangular-- 00:18:34.140 --> 00:18:36.780 align:middle line:84% with this type of situation where you have 00:18:36.780 --> 00:18:39.567 align:middle line:90% an instrument for the variable. 00:18:39.567 --> 00:18:41.400 align:middle line:84% So I thought, that's where I'm going to get. 00:18:41.400 --> 00:18:43.317 align:middle line:84% And at least that's going to be something new. 00:18:43.317 --> 00:18:45.000 align:middle line:90% Now, it turned out-- 00:18:45.000 --> 00:18:47.190 align:middle line:84% for a reason we're going to discuss very soon, 00:18:47.190 --> 00:18:49.680 align:middle line:90% or if not today or on Monday-- 00:18:49.680 --> 00:18:53.790 align:middle line:84% that it was completely inappropriate to do that. 00:18:53.790 --> 00:18:56.850 align:middle line:84% Not only that, but I could have intuited it from the beginning 00:18:56.850 --> 00:19:00.270 align:middle line:84% that if I had an instrument that affected people going 00:19:00.270 --> 00:19:02.220 align:middle line:84% to primary school, I was not going 00:19:02.220 --> 00:19:04.740 align:middle line:84% to be able to use it to extrapolate 00:19:04.740 --> 00:19:07.830 align:middle line:84% to the entire distribution of the returns 00:19:07.830 --> 00:19:10.240 align:middle line:84% across the entire education spectrum. 00:19:10.240 --> 00:19:11.850 align:middle line:84% So I had thought about it, I would 00:19:11.850 --> 00:19:14.280 align:middle line:84% have realized this was not-- this part of the project 00:19:14.280 --> 00:19:15.660 align:middle line:90% couldn't get done. 00:19:15.660 --> 00:19:17.790 align:middle line:84% But it turned out that even without this part 00:19:17.790 --> 00:19:24.140 align:middle line:84% of the project that that was a good paper, or at least 00:19:24.140 --> 00:19:26.310 align:middle line:84% understood to be a good paper, got me a job, 00:19:26.310 --> 00:19:28.810 align:middle line:90% and so on and so forth. 00:19:28.810 --> 00:19:31.020 align:middle line:84% So all that to say that it's hard to-- 00:19:31.020 --> 00:19:34.450 align:middle line:90% 00:19:34.450 --> 00:19:40.490 align:middle line:84% it was a nice project because the steps were so clear to me. 00:19:40.490 --> 00:19:43.270 align:middle line:84% And then, I was lucky to get them done. 00:19:43.270 --> 00:19:45.820 align:middle line:84% I was not stopped by the fact that I didn't 00:19:45.820 --> 00:19:47.090 align:middle line:90% know if I would find the data. 00:19:47.090 --> 00:19:48.250 align:middle line:90% I would take some risk. 00:19:48.250 --> 00:19:52.870 align:middle line:84% And then even though the steps were so clear, 00:19:52.870 --> 00:19:54.830 align:middle line:84% it was simple at the time, it was sufficient. 00:19:54.830 --> 00:19:56.800 align:middle line:84% Now, of course, everything needs to have 00:19:56.800 --> 00:19:57.880 align:middle line:90% more bells and whistles. 00:19:57.880 --> 00:20:01.720 align:middle line:84% But I think the general principle 00:20:01.720 --> 00:20:05.980 align:middle line:84% of not being discouraged in advance by this is too easy 00:20:05.980 --> 00:20:09.320 align:middle line:84% or this is too obvious still holds. 00:20:09.320 --> 00:20:14.990 align:middle line:84% So that's for the autobiography of related to this paper. 00:20:14.990 --> 00:20:15.970 align:middle line:90% So it's a difference. 00:20:15.970 --> 00:20:18.012 align:middle line:84% In difference, we are going to see a lot of that. 00:20:18.012 --> 00:20:20.080 align:middle line:84% You're going to see a lot of that in your career. 00:20:20.080 --> 00:20:22.150 align:middle line:84% If you're an empirical person, at some point 00:20:22.150 --> 00:20:25.090 align:middle line:84% you'll probably estimate some version of an event study graph 00:20:25.090 --> 00:20:27.100 align:middle line:84% or a difference in diff and diff. 00:20:27.100 --> 00:20:30.220 align:middle line:90% 00:20:30.220 --> 00:20:33.200 align:middle line:84% In the simpler setting, it's pretty straightforward. 00:20:33.200 --> 00:20:37.750 align:middle line:84% You have two groups and then two periods. 00:20:37.750 --> 00:20:42.670 align:middle line:84% And then the group gets treated in the post-period and not 00:20:42.670 --> 00:20:44.230 align:middle line:90% in the pre-period. 00:20:44.230 --> 00:20:46.060 align:middle line:84% And your identification assumption 00:20:46.060 --> 00:20:47.450 align:middle line:90% can be written in this way. 00:20:47.450 --> 00:20:51.370 align:middle line:84% Which is if you think of it in terms of your potential outcome 00:20:51.370 --> 00:20:55.270 align:middle line:84% and treat it, it's a function-- put 00:20:55.270 --> 00:20:59.860 align:middle line:84% a possibly very flexible function of anything 00:20:59.860 --> 00:21:02.600 align:middle line:84% could happen with respect to the group. 00:21:02.600 --> 00:21:05.380 align:middle line:84% It could be related to-- sorry, to the group, 00:21:05.380 --> 00:21:06.640 align:middle line:90% to the time period. 00:21:06.640 --> 00:21:08.500 align:middle line:84% But what's not there that's going 00:21:08.500 --> 00:21:11.500 align:middle line:84% to help you for identification and [INAUDIBLE] education. 00:21:11.500 --> 00:21:17.970 align:middle line:90% 00:21:17.970 --> 00:21:24.070 align:middle line:84% What did I propose from this equation that determines-- yes. 00:21:24.070 --> 00:21:26.070 align:middle line:84% STUDENT: Are there interaction between belonging 00:21:26.070 --> 00:21:26.700 align:middle line:90% to the group and-- 00:21:26.700 --> 00:21:26.880 align:middle line:90% [INTERPOSING VOICES] 00:21:26.880 --> 00:21:27.797 align:middle line:90% ESTHER DUFLO: Exactly. 00:21:27.797 --> 00:21:32.370 align:middle line:84% So we assume that maybe people who are younger 00:21:32.370 --> 00:21:35.580 align:middle line:84% will get more education because they are younger. 00:21:35.580 --> 00:21:37.620 align:middle line:90% Countries is richer. 00:21:37.620 --> 00:21:41.100 align:middle line:84% People who are in some region will get more education 00:21:41.100 --> 00:21:42.210 align:middle line:90% than others. 00:21:42.210 --> 00:21:44.130 align:middle line:90% But there is no-- 00:21:44.130 --> 00:21:47.070 align:middle line:84% the younger people in the place that are more educated 00:21:47.070 --> 00:21:49.710 align:middle line:84% would not otherwise have been even more or even 00:21:49.710 --> 00:21:51.310 align:middle line:90% less educated. 00:21:51.310 --> 00:21:53.930 align:middle line:84% So that's the assumption we are making. 00:21:53.930 --> 00:21:55.810 align:middle line:84% And that's the key identification assumption. 00:21:55.810 --> 00:21:58.750 align:middle line:84% And in a lot of difference in difference application, 00:21:58.750 --> 00:22:02.770 align:middle line:84% the trick is going to both justify this assumption 00:22:02.770 --> 00:22:08.830 align:middle line:84% from a priori principle and gather 00:22:08.830 --> 00:22:11.690 align:middle line:84% as much evidence as possible that the assumption is, 00:22:11.690 --> 00:22:13.660 align:middle line:90% in fact real. 00:22:13.660 --> 00:22:16.630 align:middle line:84% Once we have that, then we're in good shape. 00:22:16.630 --> 00:22:19.150 align:middle line:84% So again, if we have two groups, two period, 00:22:19.150 --> 00:22:22.330 align:middle line:84% then the diff and diff estimates are simply-- 00:22:22.330 --> 00:22:25.870 align:middle line:84% or so the object that we are trying to estimate 00:22:25.870 --> 00:22:32.260 align:middle line:84% is the difference in mean between post and pre 00:22:32.260 --> 00:22:35.770 align:middle line:84% for the treated group minus the same thing for the control 00:22:35.770 --> 00:22:36.460 align:middle line:90% group. 00:22:36.460 --> 00:22:39.410 align:middle line:84% And there are the identification assumption that I laid down. 00:22:39.410 --> 00:22:42.310 align:middle line:84% That's going to give you the impact-- 00:22:42.310 --> 00:22:45.177 align:middle line:84% the average impact of the program. 00:22:45.177 --> 00:22:46.760 align:middle line:84% Of course, we don't have these things. 00:22:46.760 --> 00:22:53.500 align:middle line:84% So we replace them by population averages or by sample averages. 00:22:53.500 --> 00:22:57.940 align:middle line:84% And the definitive estimator is now going to be-- 00:22:57.940 --> 00:22:59.290 align:middle line:90% this should be a hat there-- 00:22:59.290 --> 00:23:04.540 align:middle line:84% going to be the average treated group post minus 00:23:04.540 --> 00:23:09.130 align:middle line:84% treated group pre minus control group post minus control group 00:23:09.130 --> 00:23:11.384 align:middle line:90% pre. 00:23:11.384 --> 00:23:15.930 align:middle line:84% I would like you to do three lines of algebra when 00:23:15.930 --> 00:23:24.360 align:middle line:84% you go home which will convince you that this and this is 00:23:24.360 --> 00:23:27.900 align:middle line:84% numerically equivalent to estimating an OLS 00:23:27.900 --> 00:23:32.790 align:middle line:84% regression on a group-- a treated dummy, 00:23:32.790 --> 00:23:37.140 align:middle line:84% a constant treated dummy, a post dummy, 00:23:37.140 --> 00:23:38.660 align:middle line:90% and the interaction of the two. 00:23:38.660 --> 00:23:41.590 align:middle line:90% 00:23:41.590 --> 00:23:45.500 align:middle line:84% With undergraduates, I make the calculations. 00:23:45.500 --> 00:23:47.650 align:middle line:84% But I'm going to assume that you can make it. 00:23:47.650 --> 00:23:51.050 align:middle line:90% That's from the OLS formula. 00:23:51.050 --> 00:23:53.590 align:middle line:84% So it takes the lines of algebra. 00:23:53.590 --> 00:23:56.260 align:middle line:84% But it's interesting-- it's good to have in mind. 00:23:56.260 --> 00:23:58.300 align:middle line:90% You really need to know. 00:23:58.300 --> 00:24:00.550 align:middle line:84% And sometimes we ask this question in the exam 00:24:00.550 --> 00:24:02.980 align:middle line:90% and not everybody gets it. 00:24:02.980 --> 00:24:05.160 align:middle line:84% You really need to know what this is. 00:24:05.160 --> 00:24:10.990 align:middle line:84% This is the difference between post and pre, beta 1. 00:24:10.990 --> 00:24:15.340 align:middle line:84% This is the difference between treated and control 00:24:15.340 --> 00:24:19.900 align:middle line:90% in the pre period numerically. 00:24:19.900 --> 00:24:23.830 align:middle line:90% And this is the diff and diff. 00:24:23.830 --> 00:24:28.255 align:middle line:84% This is something one needs to know 00:24:28.255 --> 00:24:30.820 align:middle line:90% in life because you get to. 00:24:30.820 --> 00:24:35.500 align:middle line:84% So in our setting, we have a relatively swift school 00:24:35.500 --> 00:24:38.110 align:middle line:84% construction building program financed 00:24:38.110 --> 00:24:40.960 align:middle line:90% by the oil boom in 1973. 00:24:40.960 --> 00:24:43.000 align:middle line:84% So someone asked how it came about 00:24:43.000 --> 00:24:46.240 align:middle line:84% that they decided to use their oil money to do that. 00:24:46.240 --> 00:24:49.760 align:middle line:84% What's the political economy of that? 00:24:49.760 --> 00:24:53.170 align:middle line:84% So the political economy of that is they were-- 00:24:53.170 --> 00:24:58.150 align:middle line:84% Suharto came to power after a pretty bloody civil war. 00:24:58.150 --> 00:25:01.870 align:middle line:84% Millions of communists were killed. 00:25:01.870 --> 00:25:05.350 align:middle line:84% The country was not in good shape either politically, 00:25:05.350 --> 00:25:08.500 align:middle line:84% or economically when he took and consolidated power. 00:25:08.500 --> 00:25:13.150 align:middle line:84% And his view for fixing that was to try and create 00:25:13.150 --> 00:25:18.710 align:middle line:84% a national identity, which was uniform for the whole country. 00:25:18.710 --> 00:25:22.210 align:middle line:84% And to do that, he invented a whole ideology 00:25:22.210 --> 00:25:24.730 align:middle line:90% called the Pancasila. 00:25:24.730 --> 00:25:27.922 align:middle line:84% Really promoted the Bahasa Indonesia language, 00:25:27.922 --> 00:25:29.380 align:middle line:84% which interestingly is the language 00:25:29.380 --> 00:25:30.640 align:middle line:90% of no one in particular. 00:25:30.640 --> 00:25:33.400 align:middle line:84% It's a group-- it's a trader language that's 00:25:33.400 --> 00:25:35.590 align:middle line:84% not spoken or was not spoken, in particular, 00:25:35.590 --> 00:25:37.470 align:middle line:90% by the main ethnic group. 00:25:37.470 --> 00:25:39.220 align:middle line:84% And he thought schools would be a good way 00:25:39.220 --> 00:25:42.430 align:middle line:84% to diffuse that ideology and to diffuse the language, 00:25:42.430 --> 00:25:45.400 align:middle line:84% and therefore to create a sense of national unity 00:25:45.400 --> 00:25:49.330 align:middle line:90% after what they had experienced. 00:25:49.330 --> 00:25:54.860 align:middle line:84% And before that, Indonesia is an oil-producing country. 00:25:54.860 --> 00:25:56.800 align:middle line:84% But before that, the oil proceeds 00:25:56.800 --> 00:26:00.100 align:middle line:84% used to go to the places that produced the oil. 00:26:00.100 --> 00:26:01.720 align:middle line:84% And another thing that he thought 00:26:01.720 --> 00:26:04.750 align:middle line:84% that he would consolidate both his power and the whole place 00:26:04.750 --> 00:26:10.000 align:middle line:84% was to use this proceeds in a more equitable way. 00:26:10.000 --> 00:26:12.700 align:middle line:84% So he decided that all of the money from the oil 00:26:12.700 --> 00:26:17.290 align:middle line:84% had to be redistributed by the center in social programs. 00:26:17.290 --> 00:26:20.050 align:middle line:84% And one of-- and the school construction program. 00:26:20.050 --> 00:26:22.990 align:middle line:84% They are called this in-pres, in presidential instruction, 00:26:22.990 --> 00:26:25.030 align:middle line:90% by presidential instruction. 00:26:25.030 --> 00:26:28.480 align:middle line:84% So the in-pres-- the education-- the primary school program 00:26:28.480 --> 00:26:29.500 align:middle line:90% was the first of those. 00:26:29.500 --> 00:26:32.350 align:middle line:84% Then there was a water and sanitation program. 00:26:32.350 --> 00:26:35.500 align:middle line:84% So but when they started, it was very small, 00:26:35.500 --> 00:26:37.510 align:middle line:90% because they started to-- 00:26:37.510 --> 00:26:41.350 align:middle line:90% the first year of this was 1972. 00:26:41.350 --> 00:26:42.490 align:middle line:90% And then the oil boom came. 00:26:42.490 --> 00:26:44.530 align:middle line:84% And then it became, I think, much bigger 00:26:44.530 --> 00:26:46.870 align:middle line:90% than what they had anticipated. 00:26:46.870 --> 00:26:49.090 align:middle line:84% And they stuck with it, both in terms 00:26:49.090 --> 00:26:53.410 align:middle line:84% of how to allocate the rules, how to allocate the schools, 00:26:53.410 --> 00:26:58.070 align:middle line:84% and to continue using all money to this purpose. 00:26:58.070 --> 00:26:59.450 align:middle line:84% So it's very different to Brazil, 00:26:59.450 --> 00:27:00.970 align:middle line:84% for example, where the oil money is 00:27:00.970 --> 00:27:04.840 align:middle line:84% being spent in the provinces that extract it. 00:27:04.840 --> 00:27:07.000 align:middle line:84% Or in fact, a lot of the oil is in the sea. 00:27:07.000 --> 00:27:10.480 align:middle line:84% So it's extracted-- it's used in the provinces that 00:27:10.480 --> 00:27:13.420 align:middle line:90% are in line to the sea. 00:27:13.420 --> 00:27:16.390 align:middle line:84% And that money-- there is a whole literature 00:27:16.390 --> 00:27:18.610 align:middle line:84% showing that that money is not very well spent 00:27:18.610 --> 00:27:20.600 align:middle line:90% by those places. 00:27:20.600 --> 00:27:22.285 align:middle line:84% So that's the political economy of that. 00:27:22.285 --> 00:27:24.790 align:middle line:90% 00:27:24.790 --> 00:27:26.650 align:middle line:84% Then they decided that they were going 00:27:26.650 --> 00:27:31.270 align:middle line:84% to allocate the schools because the objective was to get 00:27:31.270 --> 00:27:33.310 align:middle line:90% to 85% enrollment everywhere. 00:27:33.310 --> 00:27:35.050 align:middle line:84% So they were going to allocate the school 00:27:35.050 --> 00:27:37.990 align:middle line:84% as a function of the difference between 85% 00:27:37.990 --> 00:27:40.870 align:middle line:90% and wherever the place was. 00:27:40.870 --> 00:27:44.020 align:middle line:90% They had their census in 1971. 00:27:44.020 --> 00:27:47.590 align:middle line:84% So they were basically use that to say-- 00:27:47.590 --> 00:27:50.290 align:middle line:84% to [? vertically ?] linear increase to 85%. 00:27:50.290 --> 00:27:53.420 align:middle line:84% And in principle, it should have stuck at 85%. 00:27:53.420 --> 00:27:55.510 align:middle line:84% That's another thing that I tried to use 00:27:55.510 --> 00:27:58.690 align:middle line:84% and didn't use to see whether after 85% 00:27:58.690 --> 00:28:00.460 align:middle line:90% it should have flattened. 00:28:00.460 --> 00:28:05.110 align:middle line:84% And so, one could use that for a neater identification. 00:28:05.110 --> 00:28:06.370 align:middle line:90% But it didn't quite-- 00:28:06.370 --> 00:28:09.890 align:middle line:84% it wasn't it was too smooth, so it didn't work. 00:28:09.890 --> 00:28:13.240 align:middle line:84% So with that, just it depends on the pre-campaign enrollment. 00:28:13.240 --> 00:28:16.610 align:middle line:84% That gives me my difference in difference. 00:28:16.610 --> 00:28:19.750 align:middle line:84% So I'm going to define a group of treated people 00:28:19.750 --> 00:28:24.265 align:middle line:84% which are the young guys, and then treated in control region. 00:28:24.265 --> 00:28:28.300 align:middle line:90% 00:28:28.300 --> 00:28:33.250 align:middle line:84% To start with, we can say, well, let's be low above the median. 00:28:33.250 --> 00:28:38.560 align:middle line:84% And then we can do a simple diff and diff for year of education. 00:28:38.560 --> 00:28:44.620 align:middle line:84% So these are sample average for the high program intensity 00:28:44.620 --> 00:28:49.040 align:middle line:84% and the young people, high intensity and the old people. 00:28:49.040 --> 00:28:52.240 align:middle line:84% The first thing we note is among the old people 00:28:52.240 --> 00:28:54.670 align:middle line:84% there was the education level, this 00:28:54.670 --> 00:28:59.050 align:middle line:84% is years of education attained by adults 00:28:59.050 --> 00:29:04.640 align:middle line:84% who were kids during this period is lower in the high intensity 00:29:04.640 --> 00:29:05.140 align:middle line:90% region. 00:29:05.140 --> 00:29:10.960 align:middle line:84% That makes sense, because they were putting more schools 00:29:10.960 --> 00:29:14.410 align:middle line:84% in places that had less education to start with. 00:29:14.410 --> 00:29:20.330 align:middle line:84% And they are still lower in for the young kids, but less so. 00:29:20.330 --> 00:29:30.870 align:middle line:84% So the difference is positive although it's insignificant. 00:29:30.870 --> 00:29:33.365 align:middle line:84% So this is illustrative, but it's not very precise. 00:29:33.365 --> 00:29:36.170 align:middle line:90% 00:29:36.170 --> 00:29:37.050 align:middle line:90% So that's good. 00:29:37.050 --> 00:29:38.690 align:middle line:90% So then now we can look at-- 00:29:38.690 --> 00:29:41.600 align:middle line:90% 00:29:41.600 --> 00:29:44.570 align:middle line:84% we can be a bit concerned that in even 00:29:44.570 --> 00:29:51.020 align:middle line:84% in the absence of the program, the young cohort 00:29:51.020 --> 00:29:54.455 align:middle line:84% would have converged maybe to the old cohorts because-- 00:29:54.455 --> 00:29:57.650 align:middle line:90% 00:29:57.650 --> 00:29:59.000 align:middle line:90% go back to our basic model. 00:29:59.000 --> 00:30:00.080 align:middle line:90% Things converge. 00:30:00.080 --> 00:30:02.750 align:middle line:84% There should be one steady state to where everybody is going 00:30:02.750 --> 00:30:05.310 align:middle line:90% and it's within the country. 00:30:05.310 --> 00:30:07.400 align:middle line:84% So we would expect convergence anywhere. 00:30:07.400 --> 00:30:11.060 align:middle line:90% So am I just capturing that? 00:30:11.060 --> 00:30:13.630 align:middle line:84% So one easy way to do that is to compare the old 00:30:13.630 --> 00:30:15.700 align:middle line:90% to the very old. 00:30:15.700 --> 00:30:21.580 align:middle line:84% And we can do that with that control experiment. 00:30:21.580 --> 00:30:23.260 align:middle line:84% And then we can see that at least 00:30:23.260 --> 00:30:26.740 align:middle line:84% before it seems to be that the trend were 00:30:26.740 --> 00:30:30.890 align:middle line:84% in fact parallel between the two places. 00:30:30.890 --> 00:30:33.790 align:middle line:90% So that's encouraging. 00:30:33.790 --> 00:30:36.500 align:middle line:84% The old and-- these are already my old. 00:30:36.500 --> 00:30:37.655 align:middle line:90% And this is my very old. 00:30:37.655 --> 00:30:39.530 align:middle line:84% And if I had done the diff and diff for them, 00:30:39.530 --> 00:30:43.200 align:middle line:84% I would have found a much smaller number. 00:30:43.200 --> 00:30:48.614 align:middle line:84% And so, that goes in the right direction. 00:30:48.614 --> 00:30:51.640 align:middle line:84% Now, this is all-- this is quite good. 00:30:51.640 --> 00:30:53.360 align:middle line:90% And we can do better than that. 00:30:53.360 --> 00:30:55.660 align:middle line:90% And that's where we have-- 00:30:55.660 --> 00:30:58.670 align:middle line:84% first of all, we have many groups. 00:30:58.670 --> 00:31:02.470 align:middle line:84% So in some places, they built one school per 2,000 kids, 00:31:02.470 --> 00:31:05.590 align:middle line:84% and two school per 1,000 kids, and 100 schools per 1,000 kids. 00:31:05.590 --> 00:31:07.300 align:middle line:90% Or well, it doesn't go to 100. 00:31:07.300 --> 00:31:13.720 align:middle line:84% But so we could think of several treatments. 00:31:13.720 --> 00:31:16.490 align:middle line:84% So in our case, you could have one to three 00:31:16.490 --> 00:31:17.950 align:middle line:90% if you have not very many. 00:31:17.950 --> 00:31:19.540 align:middle line:84% Or you could have, in our case, we 00:31:19.540 --> 00:31:23.390 align:middle line:84% can say this is something discrete. 00:31:23.390 --> 00:31:26.770 align:middle line:84% And I'm going to replace my simple diff 00:31:26.770 --> 00:31:32.530 align:middle line:84% and diff by basically a regression pre 00:31:32.530 --> 00:31:37.390 align:middle line:84% and post on the number of education differences 00:31:37.390 --> 00:31:43.530 align:middle line:84% on the number of kids that were built. 00:31:43.530 --> 00:31:48.470 align:middle line:84% So again, if I had only two cohorts and I only relax that, 00:31:48.470 --> 00:31:49.400 align:middle line:90% it gives me-- 00:31:49.400 --> 00:31:51.980 align:middle line:90% 00:31:51.980 --> 00:31:53.520 align:middle line:90% I'm now need to control. 00:31:53.520 --> 00:31:56.580 align:middle line:84% I need to put region-fixed effect. 00:31:56.580 --> 00:31:59.480 align:middle line:84% So that's control for any differences. 00:31:59.480 --> 00:32:01.743 align:middle line:84% And then I'm going to assume here-- 00:32:01.743 --> 00:32:03.410 align:middle line:84% I make the assumption that the effect is 00:32:03.410 --> 00:32:06.350 align:middle line:90% linear in the number of school. 00:32:06.350 --> 00:32:11.690 align:middle line:84% So I'm regressing on an interaction, 00:32:11.690 --> 00:32:15.110 align:middle line:84% not anymore on a treated dummy, but on a number 00:32:15.110 --> 00:32:19.700 align:middle line:90% of school per 1,000 kids dummy. 00:32:19.700 --> 00:32:22.220 align:middle line:84% And that gives me this regression, again, 00:32:22.220 --> 00:32:29.480 align:middle line:84% that I can do by comparing just the young to the old. 00:32:29.480 --> 00:32:32.120 align:middle line:84% And so, now this is interpreted as saying 00:32:32.120 --> 00:32:38.540 align:middle line:84% that in the places that got one more school-- 00:32:38.540 --> 00:32:42.410 align:middle line:84% for each extra school per 1,000 kids, 00:32:42.410 --> 00:32:45.230 align:middle line:84% the increase in education between the young 00:32:45.230 --> 00:32:50.380 align:middle line:84% and the old cohort is 0.12 years faster. 00:32:50.380 --> 00:32:52.370 align:middle line:90% Does that make sense? 00:32:52.370 --> 00:32:54.790 align:middle line:90% So this is how we read that. 00:32:54.790 --> 00:32:56.980 align:middle line:90% That's a regression. 00:32:56.980 --> 00:32:59.440 align:middle line:84% You could have read that-- you could have run that 00:32:59.440 --> 00:33:03.940 align:middle line:84% as a regression of the difference between the young 00:33:03.940 --> 00:33:06.820 align:middle line:84% and the old on the number of schools per capita 00:33:06.820 --> 00:33:11.290 align:middle line:84% that were built in their region of birth 00:33:11.290 --> 00:33:16.390 align:middle line:84% controlling for any differences between the region. 00:33:16.390 --> 00:33:23.640 align:middle line:84% And we can do the same experiment between the old 00:33:23.640 --> 00:33:26.190 align:middle line:90% and the super old. 00:33:26.190 --> 00:33:30.060 align:middle line:84% And things are now much more precise, 00:33:30.060 --> 00:33:32.550 align:middle line:84% although they are too precise for reasons 00:33:32.550 --> 00:33:35.790 align:middle line:90% that I will come in a minute. 00:33:35.790 --> 00:33:38.295 align:middle line:84% My standard errors are completely off in that paper. 00:33:38.295 --> 00:33:40.820 align:middle line:90% 00:33:40.820 --> 00:33:43.450 align:middle line:90% I didn't know any better. 00:33:43.450 --> 00:33:44.400 align:middle line:90% But I now do know. 00:33:44.400 --> 00:33:48.300 align:middle line:84% So this [INAUDIBLE] off for reasons I'll discuss in a bit. 00:33:48.300 --> 00:33:53.010 align:middle line:84% But at least it seems quite a bit more precise, or a lot more 00:33:53.010 --> 00:33:54.875 align:middle line:90% precise. 00:33:54.875 --> 00:33:55.500 align:middle line:90% So that's good. 00:33:55.500 --> 00:33:57.840 align:middle line:84% It looks like every 1,000 schools increase the number 00:33:57.840 --> 00:34:00.270 align:middle line:90% of year by 0.12 years. 00:34:00.270 --> 00:34:03.900 align:middle line:90% And now we can do the-- 00:34:03.900 --> 00:34:08.130 align:middle line:84% we can use the fact that in fact, we have many cohorts 00:34:08.130 --> 00:34:10.960 align:middle line:84% and also that perhaps we have a prediction. 00:34:10.960 --> 00:34:13.060 align:middle line:84% So instead of controlling, first of all, 00:34:13.060 --> 00:34:16.560 align:middle line:84% instead of controlling for just the post, 00:34:16.560 --> 00:34:20.340 align:middle line:84% I could control for one demi-period. 00:34:20.340 --> 00:34:22.949 align:middle line:84% That's not going to change anything. 00:34:22.949 --> 00:34:26.580 align:middle line:84% But more interestingly, I could, instead 00:34:26.580 --> 00:34:29.250 align:middle line:84% of having just an interaction for post 00:34:29.250 --> 00:34:34.710 align:middle line:84% and the number of schools, I'm going to now control-- 00:34:34.710 --> 00:34:40.199 align:middle line:84% I'm going to have an interaction for every year dummy 00:34:40.199 --> 00:34:44.730 align:middle line:84% and the number of schools, which you can think-- 00:34:44.730 --> 00:34:49.920 align:middle line:84% the way this is written, which starts from the absolutely 00:34:49.920 --> 00:34:53.429 align:middle line:84% youngest people or oldest people until the very end, 00:34:53.429 --> 00:34:56.159 align:middle line:84% you can think of running a series of diff and diff, 00:34:56.159 --> 00:34:58.470 align:middle line:84% where every time you're comparing 00:34:58.470 --> 00:35:00.900 align:middle line:90% all the people born in 19-- 00:35:00.900 --> 00:35:05.700 align:middle line:84% I think the oldest people are born 1950 or something. 00:35:05.700 --> 00:35:08.070 align:middle line:84% You compare them to people who are born-- 00:35:08.070 --> 00:35:11.190 align:middle line:84% you do a diff and diff between the 51 cohort to 50-- 00:35:11.190 --> 00:35:14.670 align:middle line:84% the difference between 51 and 50 regressed 00:35:14.670 --> 00:35:16.740 align:middle line:84% on the number of schools that were built. 00:35:16.740 --> 00:35:18.840 align:middle line:84% And then the difference between 52 00:35:18.840 --> 00:35:20.940 align:middle line:84% and 50 and the number of schools that were built, 00:35:20.940 --> 00:35:22.560 align:middle line:90% et cetera, et cetera. 00:35:22.560 --> 00:35:25.860 align:middle line:90% And that builds you your-- 00:35:25.860 --> 00:35:27.570 align:middle line:84% so this first one is just treated. 00:35:27.570 --> 00:35:30.810 align:middle line:84% And this second one is combined with the treatment of schools. 00:35:30.810 --> 00:35:33.960 align:middle line:84% And that gives you this typical event study graph. 00:35:33.960 --> 00:35:37.050 align:middle line:90% 00:35:37.050 --> 00:35:40.400 align:middle line:84% So this is the one for education where 00:35:40.400 --> 00:35:47.190 align:middle line:84% I started here at zero, and for 24, for the oldest people. 00:35:47.190 --> 00:35:51.360 align:middle line:84% And so, this is a diff and diff, the continuous diff 00:35:51.360 --> 00:35:55.310 align:middle line:84% and diff for the 23, for the 22, et cetera. 00:35:55.310 --> 00:35:56.900 align:middle line:90% This is run as one regression. 00:35:56.900 --> 00:36:02.540 align:middle line:84% But this is each of these dot compares to the oldest people. 00:36:02.540 --> 00:36:08.000 align:middle line:84% And I do have a prediction that there wasn't any school built 00:36:08.000 --> 00:36:12.590 align:middle line:84% that these guys could go to if they are older than 12 in 1974. 00:36:12.590 --> 00:36:15.260 align:middle line:84% So I shouldn't see-- all of these guys should be zero. 00:36:15.260 --> 00:36:18.170 align:middle line:84% And then as schools most schools get built 00:36:18.170 --> 00:36:21.110 align:middle line:84% and people are younger, so they get to enjoy them 00:36:21.110 --> 00:36:24.020 align:middle line:84% for a longer period of time, the prediction 00:36:24.020 --> 00:36:29.440 align:middle line:84% is that the treatment effect should increase. 00:36:29.440 --> 00:36:34.890 align:middle line:84% So at that time, I think people really liked that graph. 00:36:34.890 --> 00:36:40.710 align:middle line:84% Since then they have become very, very, very common, 00:36:40.710 --> 00:36:42.490 align:middle line:90% and fashionable, and important. 00:36:42.490 --> 00:36:46.190 align:middle line:84% So this graph has a lot of issues. 00:36:46.190 --> 00:36:48.750 align:middle line:84% It's not really-- it's not the way that one would 00:36:48.750 --> 00:36:50.910 align:middle line:90% construct such a graph today. 00:36:50.910 --> 00:36:55.260 align:middle line:84% And I'll show you all of the things we would need to add. 00:36:55.260 --> 00:37:01.990 align:middle line:84% But at this time, I was pretty satisfied that if you eyeball, 00:37:01.990 --> 00:37:03.610 align:middle line:90% this looks like a straight line. 00:37:03.610 --> 00:37:05.820 align:middle line:84% This looks like an increasing line. 00:37:05.820 --> 00:37:09.190 align:middle line:90% And that's what we would expect. 00:37:09.190 --> 00:37:12.490 align:middle line:90% So that's encouraging. 00:37:12.490 --> 00:37:16.400 align:middle line:84% Once we have found this, we can say, well, 00:37:16.400 --> 00:37:21.250 align:middle line:84% now that I have convinced myself that this is, in fact, zero, 00:37:21.250 --> 00:37:26.080 align:middle line:84% I can impose it, which gives me a large control cohort. 00:37:26.080 --> 00:37:30.460 align:middle line:84% And now I can construct my diff and diff all with respect 00:37:30.460 --> 00:37:31.840 align:middle line:90% to the control cohort. 00:37:31.840 --> 00:37:35.650 align:middle line:84% So all of this is going to be normalized as having 00:37:35.650 --> 00:37:39.220 align:middle line:90% zero effect compared to them. 00:37:39.220 --> 00:37:42.740 align:middle line:84% And then, this is going to be compared to the entire group. 00:37:42.740 --> 00:37:44.690 align:middle line:84% So someone asked, why would you do that? 00:37:44.690 --> 00:37:48.130 align:middle line:84% Well, you do that because you get much more precise results 00:37:48.130 --> 00:37:51.610 align:middle line:84% if you have a control group that's much larger. 00:37:51.610 --> 00:37:55.690 align:middle line:90% 00:37:55.690 --> 00:37:56.493 align:middle line:90% And oh. 00:37:56.493 --> 00:38:00.840 align:middle line:90% 00:38:00.840 --> 00:38:03.870 align:middle line:84% So that's what I did in that paper where I still-- 00:38:03.870 --> 00:38:07.410 align:middle line:84% I estimate where-- that's what's done here. 00:38:07.410 --> 00:38:10.980 align:middle line:84% We are here running a regression with only 10 dummy 00:38:10.980 --> 00:38:14.010 align:middle line:84% or 11 dummies, which are all the people treated 00:38:14.010 --> 00:38:16.680 align:middle line:84% and their various treatment effect as they age. 00:38:16.680 --> 00:38:22.760 align:middle line:90% 00:38:22.760 --> 00:38:26.120 align:middle line:84% And then, once we've done it for education, 00:38:26.120 --> 00:38:27.770 align:middle line:84% we can also do it for wages, which 00:38:27.770 --> 00:38:30.440 align:middle line:84% will lead to what we'll discuss next time for the. 00:38:30.440 --> 00:38:33.470 align:middle line:90% 00:38:33.470 --> 00:38:35.510 align:middle line:90% So that's all nice. 00:38:35.510 --> 00:38:38.630 align:middle line:84% This is a very typical example of a diff 00:38:38.630 --> 00:38:41.870 align:middle line:84% and diff two-way fixed effect that have become super popular 00:38:41.870 --> 00:38:42.890 align:middle line:90% in applied economics. 00:38:42.890 --> 00:38:45.000 align:middle line:84% And you're going to keep seeing them. 00:38:45.000 --> 00:38:48.140 align:middle line:84% This one is a simple one because everyone 00:38:48.140 --> 00:38:50.840 align:middle line:90% got treated at the same time. 00:38:50.840 --> 00:38:55.130 align:middle line:84% The only complication is that the effect-- 00:38:55.130 --> 00:38:57.890 align:middle line:84% the treatment is introduced progressively. 00:38:57.890 --> 00:39:02.930 align:middle line:84% So we would not expect no effect and then a program effect. 00:39:02.930 --> 00:39:05.000 align:middle line:84% Because we expect-- the treatment itself, 00:39:05.000 --> 00:39:06.600 align:middle line:90% the intensity keeps increasing. 00:39:06.600 --> 00:39:09.320 align:middle line:84% So the impact should also keep increasing. 00:39:09.320 --> 00:39:13.160 align:middle line:84% There are also many cases where states are adopting policies 00:39:13.160 --> 00:39:15.170 align:middle line:84% in a staggered fashion, or regions 00:39:15.170 --> 00:39:17.400 align:middle line:84% are adopting a policy in a staggered fashion. 00:39:17.400 --> 00:39:24.740 align:middle line:84% So for example, you could have a change in the minimum wage laws 00:39:24.740 --> 00:39:27.920 align:middle line:84% that one state approves, and then another one, 00:39:27.920 --> 00:39:30.600 align:middle line:84% and then another one, and another one. 00:39:30.600 --> 00:39:32.910 align:middle line:90% And then you have-- 00:39:32.910 --> 00:39:35.720 align:middle line:84% you can also compare, even in these settings, 00:39:35.720 --> 00:39:39.770 align:middle line:84% even if everyone eventually adopts, you can still 00:39:39.770 --> 00:39:42.360 align:middle line:84% estimate the impact of the policy by comparing, 00:39:42.360 --> 00:39:45.950 align:middle line:84% for example, people who haven't adopted yet to people 00:39:45.950 --> 00:39:49.610 align:middle line:90% who have already adopted. 00:39:49.610 --> 00:39:52.950 align:middle line:84% And that group keeps changing over time. 00:39:52.950 --> 00:39:57.060 align:middle line:84% So in this setting, so that's called the staggered design. 00:39:57.060 --> 00:40:00.530 align:middle line:84% So this is a one-time, one shot policy stuff. 00:40:00.530 --> 00:40:03.690 align:middle line:84% Those examples are staggered design. 00:40:03.690 --> 00:40:05.840 align:middle line:84% And there is a whole literature for how 00:40:05.840 --> 00:40:07.598 align:middle line:90% to do these things right. 00:40:07.598 --> 00:40:09.140 align:middle line:84% The first thing with staggered design 00:40:09.140 --> 00:40:15.110 align:middle line:84% is that you want to put them in the right place. 00:40:15.110 --> 00:40:17.870 align:middle line:84% You want to relabel them such that zero 00:40:17.870 --> 00:40:21.830 align:middle line:90% is the beginning of time-- 00:40:21.830 --> 00:40:26.750 align:middle line:84% is the time where a particular state introduces the policy. 00:40:26.750 --> 00:40:33.380 align:middle line:84% And everything is-- you will draw your event study graph not 00:40:33.380 --> 00:40:40.580 align:middle line:84% in calendar time but in pre and post the event time, event 00:40:40.580 --> 00:40:43.560 align:middle line:84% being the policies being adopted in a particular place. 00:40:43.560 --> 00:40:44.897 align:middle line:90% Does that make sense? 00:40:44.897 --> 00:40:46.730 align:middle line:84% I mention that because I'm going to show you 00:40:46.730 --> 00:40:50.090 align:middle line:84% an event that is graphed before, [INAUDIBLE] understand 00:40:50.090 --> 00:40:53.960 align:middle line:84% what it means when the policies are staggered. 00:40:53.960 --> 00:40:57.170 align:middle line:84% So there is an active literature on how to do it right, 00:40:57.170 --> 00:41:02.720 align:middle line:84% which means that this paper is not right anymore. 00:41:02.720 --> 00:41:06.410 align:middle line:84% Fortunately, the findings are still mostly right. 00:41:06.410 --> 00:41:11.210 align:middle line:84% But there is a lot of things you would do differently. 00:41:11.210 --> 00:41:13.325 align:middle line:84% First of all is the standard error. 00:41:13.325 --> 00:41:17.700 align:middle line:90% 00:41:17.700 --> 00:41:19.880 align:middle line:84% So we discussed at some point very briefly 00:41:19.880 --> 00:41:22.170 align:middle line:84% the idea of clustering standard errors. 00:41:22.170 --> 00:41:26.000 align:middle line:84% So at a minimum, how would you cluster the standard error 00:41:26.000 --> 00:41:29.990 align:middle line:84% in a diff and diff program like this one where 00:41:29.990 --> 00:41:32.690 align:middle line:84% you have individual data but the treatment, 00:41:32.690 --> 00:41:35.600 align:middle line:84% the program is allocated at the district level. 00:41:35.600 --> 00:41:39.983 align:middle line:90% 00:41:39.983 --> 00:41:41.275 align:middle line:90% STUDENT: [INAUDIBLE] 00:41:41.275 --> 00:41:42.650 align:middle line:84% ESTHER DUFLO: At the minimum, you 00:41:42.650 --> 00:41:46.890 align:middle line:84% want to cluster at the district [INAUDIBLE] by 10 times year. 00:41:46.890 --> 00:41:50.990 align:middle line:84% And that because that's where everyone 00:41:50.990 --> 00:41:53.720 align:middle line:84% in a particular district in a particular year 00:41:53.720 --> 00:41:57.150 align:middle line:90% has the same number of schools. 00:41:57.150 --> 00:42:01.360 align:middle line:84% So that's one, which is not done in that paper. 00:42:01.360 --> 00:42:05.300 align:middle line:84% But is easy to do, doesn't change much, 00:42:05.300 --> 00:42:09.670 align:middle line:84% but could have if the outcomes between people in a given 00:42:09.670 --> 00:42:12.020 align:middle line:84% district were [INAUDIBLE] recorded. 00:42:12.020 --> 00:42:14.920 align:middle line:84% Then you need to-- then there is one more 00:42:14.920 --> 00:42:19.300 align:middle line:84% worry, which is that suppose that 00:42:19.300 --> 00:42:21.740 align:middle line:90% instead of having annual data. 00:42:21.740 --> 00:42:23.950 align:middle line:84% In fact, I think in that data set, 00:42:23.950 --> 00:42:30.920 align:middle line:90% I have people exact birth date. 00:42:30.920 --> 00:42:34.590 align:middle line:84% So I aggregated it by year of birth. 00:42:34.590 --> 00:42:40.830 align:middle line:84% But in truth people's months of births also affected. 00:42:40.830 --> 00:42:47.570 align:middle line:84% So instead of aggregating the data, 00:42:47.570 --> 00:42:50.900 align:middle line:84% running the regression at the district time year level, 00:42:50.900 --> 00:42:53.710 align:middle line:84% I could have said, let me run it at the district time month. 00:42:53.710 --> 00:42:56.310 align:middle line:90% 00:42:56.310 --> 00:42:57.060 align:middle line:90% Why not? 00:42:57.060 --> 00:42:59.865 align:middle line:84% It should be more precise and going to give you more data. 00:42:59.865 --> 00:43:02.970 align:middle line:90% 00:43:02.970 --> 00:43:06.630 align:middle line:84% And suppose that I very seriously 00:43:06.630 --> 00:43:10.620 align:middle line:84% cluster my standard error at the district time month. 00:43:10.620 --> 00:43:12.570 align:middle line:84% Suddenly, I have many more clusters 00:43:12.570 --> 00:43:13.830 align:middle line:90% than in district time year. 00:43:13.830 --> 00:43:16.800 align:middle line:84% So I'm going to get results that should be much more precise. 00:43:16.800 --> 00:43:17.895 align:middle line:90% Does that sound right? 00:43:17.895 --> 00:43:30.290 align:middle line:90% 00:43:30.290 --> 00:43:31.270 align:middle line:90% We could move further. 00:43:31.270 --> 00:43:33.853 align:middle line:84% I could say, well, I'm going to aggregate at the district time 00:43:33.853 --> 00:43:35.530 align:middle line:90% day of birth. 00:43:35.530 --> 00:43:38.410 align:middle line:84% Probably I'm back to very small clusters now. 00:43:38.410 --> 00:43:42.250 align:middle line:84% A lot of very small clusters should become very precise. 00:43:42.250 --> 00:43:43.330 align:middle line:90% Should I want to do that? 00:43:43.330 --> 00:43:48.210 align:middle line:90% 00:43:48.210 --> 00:43:51.660 align:middle line:90% What sounds odd? 00:43:51.660 --> 00:43:54.013 align:middle line:90% Yeah. 00:43:54.013 --> 00:43:54.846 align:middle line:90% STUDENT: Never mind. 00:43:54.846 --> 00:43:58.120 align:middle line:90% 00:43:58.120 --> 00:43:59.680 align:middle line:90% ESTHER DUFLO: Yes. 00:43:59.680 --> 00:44:02.920 align:middle line:84% STUDENT: You eventually run out of freedom degrees, degrees 00:44:02.920 --> 00:44:03.790 align:middle line:90% of freedom? 00:44:03.790 --> 00:44:04.870 align:middle line:84% ESTHER DUFLO: No, because I'm not running-- 00:44:04.870 --> 00:44:07.100 align:middle line:84% I'm still running exactly the same regression. 00:44:07.100 --> 00:44:09.040 align:middle line:84% So in terms of the number of-- or I'm 00:44:09.040 --> 00:44:11.815 align:middle line:84% going to have more dummies for day of birth. 00:44:11.815 --> 00:44:14.440 align:middle line:84% But that's not really a problem, because I also have a lot of-- 00:44:14.440 --> 00:44:15.273 align:middle line:90% [INTERPOSING VOICES] 00:44:15.273 --> 00:44:17.320 align:middle line:84% STUDENT: But not have treated and control people 00:44:17.320 --> 00:44:18.320 align:middle line:90% inside the same cluster? 00:44:18.320 --> 00:44:20.528 align:middle line:84% ESTHER DUFLO: Yeah, I could have very small clusters. 00:44:20.528 --> 00:44:21.550 align:middle line:90% But that's fine. 00:44:21.550 --> 00:44:22.780 align:middle line:90% I'm going to have-- 00:44:22.780 --> 00:44:26.230 align:middle line:84% the smaller the cluster, the better, per se. 00:44:26.230 --> 00:44:26.830 align:middle line:90% Yeah. 00:44:26.830 --> 00:44:28.747 align:middle line:84% STUDENT: Because you should have the treatment 00:44:28.747 --> 00:44:30.690 align:middle line:84% is correlated across time [INAUDIBLE].. 00:44:30.690 --> 00:44:31.607 align:middle line:90% ESTHER DUFLO: Exactly. 00:44:31.607 --> 00:44:33.480 align:middle line:84% It sounds a bit like I'm manufacturing data. 00:44:33.480 --> 00:44:35.310 align:middle line:84% I'm not manufacturing data because a number 00:44:35.310 --> 00:44:37.050 align:middle line:84% of observations will be the same. 00:44:37.050 --> 00:44:39.120 align:middle line:84% But I'm reducing the cluster size. 00:44:39.120 --> 00:44:40.890 align:middle line:84% We agreed that I needed to cluster 00:44:40.890 --> 00:44:42.943 align:middle line:84% at the level of the unit of the treatment. 00:44:42.943 --> 00:44:44.610 align:middle line:84% And we said, well, that's about-- that's 00:44:44.610 --> 00:44:46.080 align:middle line:90% year time district. 00:44:46.080 --> 00:44:48.050 align:middle line:84% And then now I'm making it-- well, 00:44:48.050 --> 00:44:50.370 align:middle line:90% year times month time district. 00:44:50.370 --> 00:44:52.590 align:middle line:84% So I have more cluster, basically more effective 00:44:52.590 --> 00:44:54.180 align:middle line:90% observation. 00:44:54.180 --> 00:44:59.610 align:middle line:84% It sounds like I'm creating precision 00:44:59.610 --> 00:45:01.110 align:middle line:90% that seems too good to be true. 00:45:01.110 --> 00:45:04.140 align:middle line:84% And the issue is that the treatment 00:45:04.140 --> 00:45:10.980 align:middle line:84% is very correlated by your year of birth in particular. 00:45:10.980 --> 00:45:12.780 align:middle line:84% Someone born in February or March 00:45:12.780 --> 00:45:15.150 align:middle line:90% is treated almost the same. 00:45:15.150 --> 00:45:18.850 align:middle line:84% Their level of treatment is almost the same. 00:45:18.850 --> 00:45:24.850 align:middle line:84% So that would in and of itself be fine except, 00:45:24.850 --> 00:45:27.010 align:middle line:84% that the outcomes might also be correlated 00:45:27.010 --> 00:45:28.420 align:middle line:90% because they are in the state-- 00:45:28.420 --> 00:45:29.920 align:middle line:84% they are in the labor market roughly 00:45:29.920 --> 00:45:31.630 align:middle line:90% at the same time, et cetera. 00:45:31.630 --> 00:45:32.647 align:middle line:90% So yeah. 00:45:32.647 --> 00:45:33.730 align:middle line:90% STUDENT: But what if the-- 00:45:33.730 --> 00:45:35.200 align:middle line:84% it's, I guess, a similar scenario. 00:45:35.200 --> 00:45:37.750 align:middle line:84% What if we observe multiple observations 00:45:37.750 --> 00:45:38.770 align:middle line:90% of the same person? 00:45:38.770 --> 00:45:40.240 align:middle line:90% ESTHER DUFLO: Yeah, likewise. 00:45:40.240 --> 00:45:41.890 align:middle line:84% If we observe multiple observations 00:45:41.890 --> 00:45:44.410 align:middle line:84% of the same person, we wouldn't want to treat them 00:45:44.410 --> 00:45:45.610 align:middle line:90% as individual observations. 00:45:45.610 --> 00:45:47.553 align:middle line:84% STUDENT: So we cluster by individuals? 00:45:47.553 --> 00:45:48.470 align:middle line:90% ESTHER DUFLO: Exactly. 00:45:48.470 --> 00:45:51.260 align:middle line:84% So the question-- in that case, we would cluster by individual. 00:45:51.260 --> 00:45:53.930 align:middle line:84% We would say, they all come from the same person. 00:45:53.930 --> 00:45:55.940 align:middle line:84% We would either make an assumption 00:45:55.940 --> 00:45:59.300 align:middle line:84% about what the time series of this era is. 00:45:59.300 --> 00:46:01.573 align:middle line:84% Or we would just cluster by individual saying, 00:46:01.573 --> 00:46:02.990 align:middle line:84% I'm acknowledging that they're all 00:46:02.990 --> 00:46:04.910 align:middle line:84% coming from the same individual they are not 00:46:04.910 --> 00:46:06.020 align:middle line:90% independent observation. 00:46:06.020 --> 00:46:07.910 align:middle line:84% Therefore, I need to take that into account 00:46:07.910 --> 00:46:09.170 align:middle line:90% in my standard error. 00:46:09.170 --> 00:46:12.380 align:middle line:84% Similarly, for people who live in the same district year 00:46:12.380 --> 00:46:13.790 align:middle line:84% we say, well, they live together, 00:46:13.790 --> 00:46:17.070 align:middle line:84% or they do a lot of things in common. 00:46:17.070 --> 00:46:19.760 align:middle line:84% We need to cluster at that level because the treatment is also 00:46:19.760 --> 00:46:21.890 align:middle line:90% cluster at that level. 00:46:21.890 --> 00:46:27.200 align:middle line:84% And then, similarly, as you both are getting to, 00:46:27.200 --> 00:46:33.440 align:middle line:90% people who are living in-- 00:46:33.440 --> 00:46:35.060 align:middle line:84% who are from the same year of birth, 00:46:35.060 --> 00:46:37.280 align:middle line:84% similar years of births, similar months of births 00:46:37.280 --> 00:46:40.430 align:middle line:84% have experienced similar education condition, 00:46:40.430 --> 00:46:41.460 align:middle line:90% labor market condition. 00:46:41.460 --> 00:46:43.500 align:middle line:84% Their outcome is likely to be correlated. 00:46:43.500 --> 00:46:45.350 align:middle line:84% And their treatment is very correlated. 00:46:45.350 --> 00:46:48.320 align:middle line:84% In fact, with diff and diff, it's really extreme. 00:46:48.320 --> 00:46:52.040 align:middle line:84% Because once someone is treated, they stay treated forever. 00:46:52.040 --> 00:46:54.530 align:middle line:84% And therefore, there is a mechanical very strong 00:46:54.530 --> 00:46:56.570 align:middle line:84% correlation between treatment status 00:46:56.570 --> 00:47:01.610 align:middle line:84% of people who are born in related years 00:47:01.610 --> 00:47:03.050 align:middle line:90% because it's not-- 00:47:03.050 --> 00:47:05.180 align:middle line:90% it doesn't go on and off. 00:47:05.180 --> 00:47:08.390 align:middle line:84% Therefore, we need to take that serial correlation over time 00:47:08.390 --> 00:47:09.395 align:middle line:90% into account. 00:47:09.395 --> 00:47:11.270 align:middle line:84% The simplest way is to do exactly what you're 00:47:11.270 --> 00:47:12.890 align:middle line:84% saying, which is instead of clustering 00:47:12.890 --> 00:47:14.450 align:middle line:84% at the district times year level, 00:47:14.450 --> 00:47:18.800 align:middle line:84% we are going to cluster at the district level 00:47:18.800 --> 00:47:22.400 align:middle line:84% to take into account the correlation in the outcomes 00:47:22.400 --> 00:47:24.110 align:middle line:84% of people who live in the same district 00:47:24.110 --> 00:47:25.910 align:middle line:90% and are born in different years. 00:47:25.910 --> 00:47:28.850 align:middle line:84% The clustering-- so the correlation is not one. 00:47:28.850 --> 00:47:31.830 align:middle line:84% But the clustering-- we let the data 00:47:31.830 --> 00:47:33.770 align:middle line:90% figure out roughly what it is. 00:47:33.770 --> 00:47:35.930 align:middle line:84% Unless we have very few clusters, in which case 00:47:35.930 --> 00:47:40.070 align:middle line:90% you need to do other things. 00:47:40.070 --> 00:47:40.910 align:middle line:90% Yeah. 00:47:40.910 --> 00:47:43.250 align:middle line:84% STUDENT: And if you have data, like 00:47:43.250 --> 00:47:45.560 align:middle line:84% monthly data of the construction of schools, 00:47:45.560 --> 00:47:47.600 align:middle line:84% would it make sense to have clusters that 00:47:47.600 --> 00:47:49.580 align:middle line:90% depend on the time as well? 00:47:49.580 --> 00:47:50.330 align:middle line:90% ESTHER DUFLO: Yes. 00:47:50.330 --> 00:47:51.690 align:middle line:90% You could do that. 00:47:51.690 --> 00:47:53.690 align:middle line:84% You could say, if you had monthly data on school 00:47:53.690 --> 00:47:57.120 align:middle line:84% construction, you would have your treatment defined 00:47:57.120 --> 00:47:59.062 align:middle line:90% at the month time place. 00:47:59.062 --> 00:48:01.520 align:middle line:84% But then you would also want to cluster at the place level, 00:48:01.520 --> 00:48:04.340 align:middle line:84% because for sure there is a lot of autocorrelation 00:48:04.340 --> 00:48:06.347 align:middle line:84% in the number of schools constructed, 00:48:06.347 --> 00:48:07.430 align:middle line:90% because it keeps going up. 00:48:07.430 --> 00:48:08.330 align:middle line:90% It doesn't go down. 00:48:08.330 --> 00:48:11.240 align:middle line:84% And the outcomes might also be correlated for people 00:48:11.240 --> 00:48:13.970 align:middle line:90% in similar cohort over times. 00:48:13.970 --> 00:48:15.650 align:middle line:84% So that's the first thing we have to do. 00:48:15.650 --> 00:48:16.930 align:middle line:90% But this we've known-- yeah. 00:48:16.930 --> 00:48:17.930 align:middle line:90% STUDENT: Would it also-- 00:48:17.930 --> 00:48:19.550 align:middle line:84% I mean, this would still be, I think, 00:48:19.550 --> 00:48:21.530 align:middle line:84% you would still have the autocorrelation 00:48:21.530 --> 00:48:24.650 align:middle line:84% that you can handle once you [INAUDIBLE] the district level. 00:48:24.650 --> 00:48:27.390 align:middle line:84% But would it be slightly comparable to calendar year 00:48:27.390 --> 00:48:31.980 align:middle line:84% to use a school year so you're getting the cohort born 00:48:31.980 --> 00:48:34.938 align:middle line:90% into the same school-year? 00:48:34.938 --> 00:48:35.730 align:middle line:90% ESTHER DUFLO: Yeah. 00:48:35.730 --> 00:48:37.950 align:middle line:84% You could have a bit more precision that, maybe 00:48:37.950 --> 00:48:40.950 align:middle line:84% if you could try playing with that. 00:48:40.950 --> 00:48:43.860 align:middle line:84% I do think the calendar year is-- 00:48:43.860 --> 00:48:47.340 align:middle line:84% the school year is calendar in this particular instance. 00:48:47.340 --> 00:48:48.750 align:middle line:90% But I'm not positive. 00:48:48.750 --> 00:48:49.630 align:middle line:90% Yes. 00:48:49.630 --> 00:48:51.180 align:middle line:84% So that doesn't-- that's not about-- 00:48:51.180 --> 00:48:52.680 align:middle line:84% that's not a standard error problem. 00:48:52.680 --> 00:48:56.200 align:middle line:84% It's define your treatment in a more precise way. 00:48:56.200 --> 00:49:00.240 align:middle line:84% So that's going to give you more precision that 00:49:00.240 --> 00:49:03.070 align:middle line:84% is going to be reflected in your scenario. 00:49:03.070 --> 00:49:07.125 align:middle line:84% So first thing that I didn't do that I now know, 00:49:07.125 --> 00:49:10.240 align:middle line:84% you have to cluster at the district level. 00:49:10.240 --> 00:49:13.530 align:middle line:84% And when you do that, actually, the estimates are less precise. 00:49:13.530 --> 00:49:16.080 align:middle line:84% You still have a significant impact on education. 00:49:16.080 --> 00:49:20.565 align:middle line:84% But the impact on wages are a bit dicey. 00:49:20.565 --> 00:49:23.960 align:middle line:90% 00:49:23.960 --> 00:49:27.980 align:middle line:84% Then you have this series of people 00:49:27.980 --> 00:49:29.600 align:middle line:84% that are much more recent that you're 00:49:29.600 --> 00:49:33.980 align:middle line:84% going to see more in recitation and in the problem set. 00:49:33.980 --> 00:49:36.510 align:middle line:84% But I'm going to make the list for you. 00:49:36.510 --> 00:49:38.870 align:middle line:84% These are my understanding of the papers that 00:49:38.870 --> 00:49:42.980 align:middle line:84% needs to be read now if you're interested in ever 00:49:42.980 --> 00:49:44.990 align:middle line:84% doing an event study or difference in difference 00:49:44.990 --> 00:49:46.160 align:middle line:90% in your life. 00:49:46.160 --> 00:49:48.530 align:middle line:84% One is the Goodman-Bacon point, which 00:49:48.530 --> 00:49:52.010 align:middle line:84% makes this point of stacking the data appropriately 00:49:52.010 --> 00:49:55.430 align:middle line:90% if the event is staggered. 00:49:55.430 --> 00:50:00.350 align:middle line:84% One is-- so this idea that it's a bit loose that, oh, 00:50:00.350 --> 00:50:04.760 align:middle line:84% is there a pre-trend that everybody is testing? 00:50:04.760 --> 00:50:08.795 align:middle line:84% This paper takes that idea more seriously in saying, 00:50:08.795 --> 00:50:10.670 align:middle line:84% first of all, how would I test for pre-trend. 00:50:10.670 --> 00:50:14.510 align:middle line:84% Second of all, you could not reject the pre-trend, 00:50:14.510 --> 00:50:16.640 align:middle line:84% but they are there anyway, so we should 00:50:16.640 --> 00:50:17.870 align:middle line:90% control for them anyways. 00:50:17.870 --> 00:50:23.360 align:middle line:84% So it gives you some tips for controlling for any pre-trends. 00:50:23.360 --> 00:50:26.010 align:middle line:90% 00:50:26.010 --> 00:50:27.300 align:middle line:90% This paper is great. 00:50:27.300 --> 00:50:29.460 align:middle line:84% I'm going to show you two graphs from this paper. 00:50:29.460 --> 00:50:31.590 align:middle line:90% But it's fantastic. 00:50:31.590 --> 00:50:35.340 align:middle line:84% I think eventually it probably everything you need to know 00:50:35.340 --> 00:50:37.000 align:middle line:90% is in here. 00:50:37.000 --> 00:50:39.090 align:middle line:84% I'm willing to bet that that becomes 00:50:39.090 --> 00:50:41.340 align:middle line:84% the standard and everybody else is forgotten. 00:50:41.340 --> 00:50:45.300 align:middle line:84% It's a review paper on how to do an event study graph 00:50:45.300 --> 00:50:49.050 align:middle line:84% as well as how to control for the pre-trend. 00:50:49.050 --> 00:50:51.330 align:middle line:84% And I'll show you their event study graph. 00:50:51.330 --> 00:50:53.790 align:middle line:90% We can compare it to mine. 00:50:53.790 --> 00:50:57.820 align:middle line:84% But it's a review for the annual review or something like that. 00:50:57.820 --> 00:50:58.950 align:middle line:90% So it's very pedagogical. 00:50:58.950 --> 00:51:01.810 align:middle line:84% It's well done, and there are [INAUDIBLE] codes to it. 00:51:01.810 --> 00:51:04.500 align:middle line:84% So my sense is it's your one-stop shop. 00:51:04.500 --> 00:51:06.810 align:middle line:84% I just found out about it recently, 00:51:06.810 --> 00:51:11.670 align:middle line:84% otherwise it would be in the syllabus with three stars. 00:51:11.670 --> 00:51:14.400 align:middle line:84% Then another issue that you might get into is when-- 00:51:14.400 --> 00:51:20.310 align:middle line:84% if the treatment effect are heterogeneous, 00:51:20.310 --> 00:51:24.210 align:middle line:84% then what you get from just a simple difference in difference 00:51:24.210 --> 00:51:26.130 align:middle line:84% in the stack design is a weighted effect 00:51:26.130 --> 00:51:28.710 align:middle line:90% of all the treatment effects. 00:51:28.710 --> 00:51:30.340 align:middle line:90% And usually, that's fine. 00:51:30.340 --> 00:51:31.920 align:middle line:84% But if you're unlucky, some weights 00:51:31.920 --> 00:51:35.470 align:middle line:84% could be negative, in which case you're a bit screwed. 00:51:35.470 --> 00:51:40.020 align:middle line:84% So there is some work to correct for that-- 00:51:40.020 --> 00:51:43.940 align:middle line:84% construct the right standard error. 00:51:43.940 --> 00:51:46.490 align:middle line:90% It's an issue probably to check. 00:51:46.490 --> 00:51:49.700 align:middle line:84% I would, in my sense, probably less a central 00:51:49.700 --> 00:51:53.160 align:middle line:90% issue than doing that properly. 00:51:53.160 --> 00:51:56.960 align:middle line:84% So here is their event study graph 00:51:56.960 --> 00:52:01.332 align:middle line:84% after all of the suggestions have been implemented. 00:52:01.332 --> 00:52:03.290 align:middle line:84% So maybe you don't even need to read the paper. 00:52:03.290 --> 00:52:08.480 align:middle line:84% You can refer to this very neat summary of it. 00:52:08.480 --> 00:52:10.340 align:middle line:84% I do think you should read the paper. 00:52:10.340 --> 00:52:11.880 align:middle line:90% But what do we need to do? 00:52:11.880 --> 00:52:13.310 align:middle line:90% Well, this is our event study. 00:52:13.310 --> 00:52:15.290 align:middle line:90% So it's zero is the point. 00:52:15.290 --> 00:52:17.780 align:middle line:84% In our case-- in my case, it would 00:52:17.780 --> 00:52:22.220 align:middle line:90% be 12 in 1974 would be my zero. 00:52:22.220 --> 00:52:23.930 align:middle line:90% So what did I do wrong? 00:52:23.930 --> 00:52:26.300 align:middle line:84% So first of all, they suggest that it is better 00:52:26.300 --> 00:52:28.880 align:middle line:90% to normalize the year before. 00:52:28.880 --> 00:52:31.580 align:middle line:84% I normalized here, at the beginning of my period. 00:52:31.580 --> 00:52:33.800 align:middle line:84% But of course, that doesn't make much sense. 00:52:33.800 --> 00:52:36.710 align:middle line:84% You would want to normalize the year before the study. 00:52:36.710 --> 00:52:39.980 align:middle line:84% So I should have used as my omitted category 00:52:39.980 --> 00:52:42.680 align:middle line:90% people who were 11 in 1972. 00:52:42.680 --> 00:52:46.370 align:middle line:90% So you normalize at minus 1. 00:52:46.370 --> 00:52:49.340 align:middle line:84% Then I have no confidence interval in any of my graphs. 00:52:49.340 --> 00:52:51.840 align:middle line:84% Actually, some of them have, but not too much. 00:52:51.840 --> 00:52:54.350 align:middle line:84% So you should have the confidence interval 00:52:54.350 --> 00:52:56.300 align:middle line:90% that everybody is now doing. 00:52:56.300 --> 00:53:01.790 align:middle line:84% What they suggest adding as well are these longer confidence 00:53:01.790 --> 00:53:04.860 align:middle line:84% interval, which is a uniform confidence band. 00:53:04.860 --> 00:53:07.340 align:middle line:84% So it takes into account the fact that you are not just 00:53:07.340 --> 00:53:11.790 align:middle line:84% testing one by one by one, but you are testing all of them. 00:53:11.790 --> 00:53:16.280 align:middle line:84% So the standard error are a bit wider. 00:53:16.280 --> 00:53:21.840 align:middle line:84% Then here, they put the average of all this point. 00:53:21.840 --> 00:53:25.850 align:middle line:84% So it's basically the average of your outcome in the-- 00:53:25.850 --> 00:53:27.800 align:middle line:84% it's normalized as zero in the regression. 00:53:27.800 --> 00:53:29.550 align:middle line:90% But it's not zero, of course. 00:53:29.550 --> 00:53:32.510 align:middle line:84% There is a-- so it's an average of your outcome 00:53:32.510 --> 00:53:34.200 align:middle line:90% in the pre-period. 00:53:34.200 --> 00:53:36.740 align:middle line:84% So it gives you a sense of magnitude 00:53:36.740 --> 00:53:39.875 align:middle line:84% and how the magnitude of the point estimate compare. 00:53:39.875 --> 00:53:43.310 align:middle line:90% 00:53:43.310 --> 00:53:44.480 align:middle line:90% What else do they propose? 00:53:44.480 --> 00:53:47.900 align:middle line:84% They propose also to plot what you think 00:53:47.900 --> 00:53:49.710 align:middle line:90% is the more restricted model. 00:53:49.710 --> 00:53:53.540 align:middle line:84% So in this case, the assumption was that there is an effect, 00:53:53.540 --> 00:53:57.780 align:middle line:84% there is a pre-period, and then there is an impact. 00:53:57.780 --> 00:53:59.280 align:middle line:90% So that's the restricted model. 00:53:59.280 --> 00:54:01.910 align:middle line:84% So you can see the extent to which there is deviation 00:54:01.910 --> 00:54:04.070 align:middle line:90% from that restricted model. 00:54:04.070 --> 00:54:05.630 align:middle line:90% In my case, I would have built-- 00:54:05.630 --> 00:54:07.490 align:middle line:90% I would have drawn this. 00:54:07.490 --> 00:54:10.250 align:middle line:84% Or in fact, I could have drawn something somewhere. 00:54:10.250 --> 00:54:12.440 align:middle line:84% At some point in the paper, I had drawn 00:54:12.440 --> 00:54:14.270 align:middle line:90% the number of school built. 00:54:14.270 --> 00:54:16.280 align:middle line:84% So you could build-- you could construct 00:54:16.280 --> 00:54:19.280 align:middle line:84% for every child how many years of effective school 00:54:19.280 --> 00:54:20.810 align:middle line:90% they would enjoy. 00:54:20.810 --> 00:54:22.550 align:middle line:84% And that would be your restricted effect 00:54:22.550 --> 00:54:25.760 align:middle line:84% if the effect were only proportional to the number 00:54:25.760 --> 00:54:27.830 align:middle line:84% of school multiplied by the number of years 00:54:27.830 --> 00:54:29.180 align:middle line:90% you have to enjoy. 00:54:29.180 --> 00:54:33.650 align:middle line:84% So you could compare it to the shape of the coefficient. 00:54:33.650 --> 00:54:36.690 align:middle line:84% That makes it easier for the reader to do that. 00:54:36.690 --> 00:54:39.110 align:middle line:84% Finally, you could introduce-- you could put in the legend 00:54:39.110 --> 00:54:41.100 align:middle line:90% of the graph-- 00:54:41.100 --> 00:54:44.390 align:middle line:84% so here, what they suggest in the paper but for some reason 00:54:44.390 --> 00:54:49.440 align:middle line:84% it's not here, is a P-value of all these guys being negative. 00:54:49.440 --> 00:54:51.260 align:middle line:84% So here you would not be able to reject. 00:54:51.260 --> 00:54:52.940 align:middle line:84% I think they are nice and negative. 00:54:52.940 --> 00:54:55.800 align:middle line:84% So a valid test of all these things being negative. 00:54:55.800 --> 00:54:56.750 align:middle line:90% So that's the test-- 00:54:56.750 --> 00:54:59.300 align:middle line:84% a formal test of-- sorry, being zero-- 00:54:59.300 --> 00:55:04.310 align:middle line:90% formal test of zero pre-trend. 00:55:04.310 --> 00:55:08.083 align:middle line:84% And then, a formal test of the restricting model. 00:55:08.083 --> 00:55:09.500 align:middle line:84% In this case, the restricted model 00:55:09.500 --> 00:55:12.877 align:middle line:84% is that the treatment effect is constant once it starts. 00:55:12.877 --> 00:55:14.960 align:middle line:84% And here, you can see that it looks like it's not. 00:55:14.960 --> 00:55:18.980 align:middle line:84% And in fact, they introduce here the test for constant effect 00:55:18.980 --> 00:55:21.540 align:middle line:90% post and the rejected. 00:55:21.540 --> 00:55:23.426 align:middle line:90% So that's in the-- 00:55:23.426 --> 00:55:27.290 align:middle line:84% that you can put that in the legend. 00:55:27.290 --> 00:55:30.830 align:middle line:84% So I put both the pre-trend and the constant treatment effect. 00:55:30.830 --> 00:55:34.120 align:middle line:84% But for some reason, on this graph they only have that. 00:55:34.120 --> 00:55:38.140 align:middle line:84% So that's your state of the art event study graph. 00:55:38.140 --> 00:55:44.020 align:middle line:84% If you're a second-year student, you're writing your second-year 00:55:44.020 --> 00:55:46.120 align:middle line:90% paper, you want-- 00:55:46.120 --> 00:55:49.390 align:middle line:84% and you are planning on an event study graph, 00:55:49.390 --> 00:55:51.413 align:middle line:90% it needs to looks like that. 00:55:51.413 --> 00:55:52.830 align:middle line:84% And that's another suggestion they 00:55:52.830 --> 00:56:02.820 align:middle line:84% have is to plot the least wiggly line that fits 00:56:02.820 --> 00:56:05.500 align:middle line:90% all the point in the graph. 00:56:05.500 --> 00:56:07.260 align:middle line:84% So here are the least wiggly line 00:56:07.260 --> 00:56:10.300 align:middle line:84% that fits all the point in the graph is a straight line. 00:56:10.300 --> 00:56:13.710 align:middle line:84% So that basically is a way to tell you it doesn't fit. 00:56:13.710 --> 00:56:17.340 align:middle line:84% It looks like this is all pre-trend. 00:56:17.340 --> 00:56:21.270 align:middle line:84% And here, this one, the least wiggly line possible 00:56:21.270 --> 00:56:22.260 align:middle line:90% is quite wiggly. 00:56:22.260 --> 00:56:24.030 align:middle line:84% So it seems very unlikely that this 00:56:24.030 --> 00:56:30.210 align:middle line:84% is due to some other thing that is moving in this funny way. 00:56:30.210 --> 00:56:32.930 align:middle line:84% So in our case, in the case of the in-pres program, 00:56:32.930 --> 00:56:35.030 align:middle line:84% the least wiggly line that fits all the point 00:56:35.030 --> 00:56:38.600 align:middle line:84% would probably have been nice and like that, and therefore 00:56:38.600 --> 00:56:41.420 align:middle line:84% quite unlikely to be explainable by something else. 00:56:41.420 --> 00:56:45.200 align:middle line:84% But it's a way of making this argument in a less-- 00:56:45.200 --> 00:56:49.260 align:middle line:90% in an impressionistic way. 00:56:49.260 --> 00:56:53.820 align:middle line:84% So that's the state of the art of difference in difference. 00:56:53.820 --> 00:56:55.005 align:middle line:90% Any question on that? 00:56:55.005 --> 00:56:59.310 align:middle line:90% 00:56:59.310 --> 00:57:01.950 align:middle line:84% I thought it would be a good occasion 00:57:01.950 --> 00:57:04.020 align:middle line:90% to put all of this on slides. 00:57:04.020 --> 00:57:07.260 align:middle line:84% Then you'll have a convenient way to refer to them later. 00:57:07.260 --> 00:57:11.080 align:middle line:90% 00:57:11.080 --> 00:57:14.370 align:middle line:84% So that's also what I have to say also 00:57:14.370 --> 00:57:17.850 align:middle line:84% on substance in terms of what's concerned the people's reaction 00:57:17.850 --> 00:57:19.800 align:middle line:90% to the cost of education. 00:57:19.800 --> 00:57:21.610 align:middle line:90% We'll go back to this paper. 00:57:21.610 --> 00:57:24.130 align:middle line:84% So many people have asked about general equilibrium effect. 00:57:24.130 --> 00:57:30.370 align:middle line:84% And I'll get back to that next time. 00:57:30.370 --> 00:57:32.770 align:middle line:84% Actually, I did write a paper about it. 00:57:32.770 --> 00:57:38.130 align:middle line:84% So I have a paper that looks at the impact of school 00:57:38.130 --> 00:57:39.600 align:middle line:84% construction on people who are not 00:57:39.600 --> 00:57:42.750 align:middle line:84% affected by school construction to get at the get 00:57:42.750 --> 00:57:45.740 align:middle line:90% at the equilibrium effect. 00:57:45.740 --> 00:57:48.350 align:middle line:84% Let me spend the last 10 minutes we have-- 00:57:48.350 --> 00:57:52.820 align:middle line:84% 10, 15 minutes we have thinking about the other aspects of what 00:57:52.820 --> 00:57:56.330 align:middle line:84% else enters in the education production function, 00:57:56.330 --> 00:57:59.180 align:middle line:84% what people seem to be responding to. 00:57:59.180 --> 00:58:02.810 align:middle line:84% So one very nice feature of a very specific feature 00:58:02.810 --> 00:58:05.750 align:middle line:84% of this model is people should respond to the perceived 00:58:05.750 --> 00:58:09.410 align:middle line:90% return to education. 00:58:09.410 --> 00:58:13.860 align:middle line:84% So how one would go about testing that? 00:58:13.860 --> 00:58:17.250 align:middle line:84% So how Jensen first exploited the fact 00:58:17.250 --> 00:58:19.290 align:middle line:84% that people make mistakes on what they think 00:58:19.290 --> 00:58:20.610 align:middle line:90% the return to education are. 00:58:20.610 --> 00:58:22.703 align:middle line:84% And therefore, you can just tell them the truth. 00:58:22.703 --> 00:58:24.120 align:middle line:84% And by telling them the truth, you 00:58:24.120 --> 00:58:27.810 align:middle line:84% are affecting their perceived return. 00:58:27.810 --> 00:58:30.510 align:middle line:84% And you can see whether therefore they 00:58:30.510 --> 00:58:36.120 align:middle line:90% react by going to school more. 00:58:36.120 --> 00:58:40.230 align:middle line:84% So he did this project in the Dominican Republic 00:58:40.230 --> 00:58:43.290 align:middle line:84% where the completion rate for high school is low. 00:58:43.290 --> 00:58:46.650 align:middle line:84% And he first went to the students and asked them 00:58:46.650 --> 00:58:50.970 align:middle line:84% their view of return to education by comparing-- 00:58:50.970 --> 00:58:54.648 align:middle line:84% by asking them to say how much someone with-- 00:58:54.648 --> 00:58:56.940 align:middle line:84% for those who have complete primary school versus those 00:58:56.940 --> 00:59:00.970 align:middle line:84% who have completed high school, what's the difference? 00:59:00.970 --> 00:59:07.100 align:middle line:84% By the way, so let me finish that, and then I ask you. 00:59:07.100 --> 00:59:10.040 align:middle line:84% At baseline, people seem to be-- people 00:59:10.040 --> 00:59:14.020 align:middle line:84% here seem to be underestimating the return to education. 00:59:14.020 --> 00:59:19.560 align:middle line:84% Which, by the way, is really not a generalized finding. 00:59:19.560 --> 00:59:23.610 align:middle line:84% In my experience, people widely overestimate the returns 00:59:23.610 --> 00:59:26.010 align:middle line:84% to education, at least to secondary education. 00:59:26.010 --> 00:59:31.430 align:middle line:84% But in this instance, they were underestimating them. 00:59:31.430 --> 00:59:33.770 align:middle line:84% They saw that you would get only a 10% increase 00:59:33.770 --> 00:59:38.030 align:middle line:84% from moving from the eighth to the 12th when, in fact, it's 00:59:38.030 --> 00:59:39.780 align:middle line:90% much bigger than that. 00:59:39.780 --> 00:59:42.320 align:middle line:84% And then what they did is that they told them 00:59:42.320 --> 00:59:47.210 align:middle line:84% the truth in some schools to the Mincerian return 00:59:47.210 --> 00:59:49.260 align:middle line:90% are more like 10% per year. 00:59:49.260 --> 00:59:53.740 align:middle line:84% So 40% for going from eighth grade to high school. 00:59:53.740 --> 00:59:56.968 align:middle line:84% And then, after some time they compared who stays in school. 00:59:56.968 --> 00:59:59.260 align:middle line:84% First of all, they show that they update their beliefs. 00:59:59.260 --> 01:00:01.070 align:middle line:84% And I'll show you that in a minute. 01:00:01.070 --> 01:00:04.600 align:middle line:84% And then they investigate the impact on education. 01:00:04.600 --> 01:00:08.350 align:middle line:84% So before I go further, what's the issue with doing that? 01:00:08.350 --> 01:00:10.980 align:middle line:90% 01:00:10.980 --> 01:00:14.230 align:middle line:84% Or in fact, even with asking them in this way, 01:00:14.230 --> 01:00:16.680 align:middle line:84% the difference they thought between in the income 01:00:16.680 --> 01:00:19.140 align:middle line:84% of someone who has completed versus not completed? 01:00:19.140 --> 01:00:23.830 align:middle line:90% 01:00:23.830 --> 01:00:26.560 align:middle line:84% What is potential concern in doing this exercise? 01:00:26.560 --> 01:00:27.640 align:middle line:90% Yeah. 01:00:27.640 --> 01:00:30.160 align:middle line:84% STUDENT: Well, I just wanted [INAUDIBLE] don't totally 01:00:30.160 --> 01:00:36.478 align:middle line:84% understand what that means, and I just also [INAUDIBLE].. 01:00:36.478 --> 01:00:37.270 align:middle line:90% ESTHER DUFLO: Yeah. 01:00:37.270 --> 01:00:40.375 align:middle line:84% They might not understand what that means in some sense. 01:00:40.375 --> 01:00:43.320 align:middle line:90% 01:00:43.320 --> 01:00:44.696 align:middle line:90% Yeah. 01:00:44.696 --> 01:00:47.670 align:middle line:84% STUDENT: It's depending on how the question was asked, 01:00:47.670 --> 01:00:51.656 align:middle line:84% it's not clear if it's like the return for an individual who 01:00:51.656 --> 01:00:54.420 align:middle line:84% chooses to [INAUDIBLE] primary school [INAUDIBLE] 01:00:54.420 --> 01:00:56.775 align:middle line:84% but also taking into account the ability. 01:00:56.775 --> 01:00:57.900 align:middle line:90% ESTHER DUFLO: Yes, exactly. 01:00:57.900 --> 01:00:58.560 align:middle line:90% Exactly. 01:00:58.560 --> 01:01:02.160 align:middle line:84% And in what is given to them is full 01:01:02.160 --> 01:01:04.860 align:middle line:84% of the ability bias potentially, or the selection bias 01:01:04.860 --> 01:01:06.563 align:middle line:90% of any kind. 01:01:06.563 --> 01:01:07.980 align:middle line:84% You're telling them the difference 01:01:07.980 --> 01:01:10.380 align:middle line:84% between learning or someone who has completed and not 01:01:10.380 --> 01:01:11.340 align:middle line:90% completed education. 01:01:11.340 --> 01:01:13.050 align:middle line:84% That's not the returns to education. 01:01:13.050 --> 01:01:15.850 align:middle line:84% That's not the causal effect of education. 01:01:15.850 --> 01:01:17.630 align:middle line:84% So potentially they are lying to people, 01:01:17.630 --> 01:01:19.540 align:middle line:90% which is a bit problematic. 01:01:19.540 --> 01:01:21.910 align:middle line:84% Or not lying, but giving them an information 01:01:21.910 --> 01:01:24.250 align:middle line:90% that's not accurate. 01:01:24.250 --> 01:01:25.780 align:middle line:84% It's not accurate for the average. 01:01:25.780 --> 01:01:30.070 align:middle line:84% It might-- and then the other part of your answer is that 01:01:30.070 --> 01:01:31.720 align:middle line:90% it's not even the-- 01:01:31.720 --> 01:01:33.380 align:middle line:90% it's about an individual. 01:01:33.380 --> 01:01:35.300 align:middle line:84% It's not necessarily the relevant information 01:01:35.300 --> 01:01:37.300 align:middle line:84% for any particular individual, which, of course, 01:01:37.300 --> 01:01:38.980 align:middle line:90% you don't know. 01:01:38.980 --> 01:01:43.040 align:middle line:84% So that's a little bit of a dicey project. 01:01:43.040 --> 01:01:46.480 align:middle line:84% But again, maybe we didn't know until-- this 01:01:46.480 --> 01:01:48.640 align:middle line:90% might be 2020 hindsight. 01:01:48.640 --> 01:01:52.540 align:middle line:84% But that suddenly something that people criticized. 01:01:52.540 --> 01:01:56.000 align:middle line:84% They do find an impact on education. 01:01:56.000 --> 01:02:00.050 align:middle line:84% They find that people generally update their belief positively. 01:02:00.050 --> 01:02:02.137 align:middle line:84% And they find an average impact on education. 01:02:02.137 --> 01:02:03.220 align:middle line:90% Let me show you the table. 01:02:03.220 --> 01:02:06.220 align:middle line:90% 01:02:06.220 --> 01:02:10.600 align:middle line:84% So perceived return have increased. 01:02:10.600 --> 01:02:13.480 align:middle line:84% On average these treatments regress for a bunch of stuff, 01:02:13.480 --> 01:02:14.200 align:middle line:90% doesn't matter. 01:02:14.200 --> 01:02:17.000 align:middle line:84% This is RCT, you shouldn't need any of that. 01:02:17.000 --> 01:02:20.950 align:middle line:84% And then, so this is the average effect on passive return. 01:02:20.950 --> 01:02:22.990 align:middle line:84% And then the average effect on various things, 01:02:22.990 --> 01:02:26.050 align:middle line:84% whether they return, whether they 01:02:26.050 --> 01:02:27.888 align:middle line:90% finish the years of schooling. 01:02:27.888 --> 01:02:29.680 align:middle line:84% And then they actually find a larger effect 01:02:29.680 --> 01:02:34.540 align:middle line:84% for the rich household than the poor household, 01:02:34.540 --> 01:02:37.650 align:middle line:84% possibly because in addition, people 01:02:37.650 --> 01:02:39.490 align:middle line:84% they are also maybe credit-constrained, 01:02:39.490 --> 01:02:41.480 align:middle line:90% people can't do it immediately. 01:02:41.480 --> 01:02:45.350 align:middle line:84% So return is not the only thing that enter the decision. 01:02:45.350 --> 01:02:49.470 align:middle line:84% So if you had to run this regression-- 01:02:49.470 --> 01:02:52.590 align:middle line:84% having done this experiment, if you had to run this regression, 01:02:52.590 --> 01:02:54.570 align:middle line:84% would you run it this way, or would there 01:02:54.570 --> 01:02:58.360 align:middle line:84% be another way to run it that would make more sense. 01:02:58.360 --> 01:02:58.900 align:middle line:90% Yes. 01:02:58.900 --> 01:03:00.275 align:middle line:84% STUDENT: Wouldn't we need to take 01:03:00.275 --> 01:03:03.140 align:middle line:84% into account that the treatment effect is different 01:03:03.140 --> 01:03:06.133 align:middle line:90% depending on how [INAUDIBLE]? 01:03:06.133 --> 01:03:07.050 align:middle line:90% ESTHER DUFLO: Exactly. 01:03:07.050 --> 01:03:08.870 align:middle line:84% So at the minimum, you should think 01:03:08.870 --> 01:03:12.830 align:middle line:84% that the treatment doesn't have a monotonous effect 01:03:12.830 --> 01:03:14.120 align:middle line:90% on these other things. 01:03:14.120 --> 01:03:18.500 align:middle line:84% Because some people, even though on average people 01:03:18.500 --> 01:03:20.180 align:middle line:84% were underestimating, it is likely 01:03:20.180 --> 01:03:22.010 align:middle line:84% that some people were overestimating 01:03:22.010 --> 01:03:24.700 align:middle line:84% and some people were underestimating. 01:03:24.700 --> 01:03:27.340 align:middle line:84% So at the minimum, when you are doing this kind of debiasing 01:03:27.340 --> 01:03:29.755 align:middle line:84% treatment, and they have become popular 01:03:29.755 --> 01:03:32.650 align:middle line:84% and we are going to see other examples of that, 01:03:32.650 --> 01:03:37.540 align:middle line:84% you should look at-- you should separate the sample 01:03:37.540 --> 01:03:41.952 align:middle line:84% by the people whose prior you're moving up to the people 01:03:41.952 --> 01:03:43.660 align:middle line:84% who prior you're moving down, because you 01:03:43.660 --> 01:03:47.750 align:middle line:84% would expect the effect to go in opposite direction. 01:03:47.750 --> 01:03:52.980 align:middle line:84% Well, that's just a note of how one could have done that. 01:03:52.980 --> 01:03:58.660 align:middle line:84% So given-- so this paper was criticized for what I'm telling 01:03:58.660 --> 01:04:01.270 align:middle line:84% you, which is that basically you're 01:04:01.270 --> 01:04:02.650 align:middle line:90% telling people some junk. 01:04:02.650 --> 01:04:04.570 align:middle line:90% That's a little bit troubling. 01:04:04.570 --> 01:04:07.360 align:middle line:90% Maybe we should do that. 01:04:07.360 --> 01:04:08.980 align:middle line:84% And then there are other paper that 01:04:08.980 --> 01:04:13.060 align:middle line:84% are trying to use paper that are trying to use the real data-- 01:04:13.060 --> 01:04:18.460 align:middle line:84% data from the real world, to look at how people respond 01:04:18.460 --> 01:04:21.570 align:middle line:90% to the return to education. 01:04:21.570 --> 01:04:26.250 align:middle line:84% An older paper by Foster and Rosenzweig 01:04:26.250 --> 01:04:29.640 align:middle line:84% in the [INAUDIBLE] quite nice, looks 01:04:29.640 --> 01:04:33.120 align:middle line:84% at the impact of high-yielding variety, 01:04:33.120 --> 01:04:37.260 align:middle line:84% and argue that high-yielding variety 01:04:37.260 --> 01:04:40.862 align:middle line:84% grain has a large impact on returns to education. 01:04:40.862 --> 01:04:43.320 align:middle line:84% Because basically, you need to be able to read the package, 01:04:43.320 --> 01:04:44.710 align:middle line:90% and understand what it is. 01:04:44.710 --> 01:04:47.130 align:middle line:84% And then they have some suggestive evidence 01:04:47.130 --> 01:04:50.010 align:middle line:84% that the return to high-yielding variety 01:04:50.010 --> 01:04:52.350 align:middle line:84% is very low for people who are not educated and very 01:04:52.350 --> 01:04:54.370 align:middle line:84% high for people who are educated. 01:04:54.370 --> 01:04:57.870 align:middle line:84% So the introduction, which is a bit sequential, 01:04:57.870 --> 01:05:02.820 align:middle line:84% as well of high-yielding variety across villages has different 01:05:02.820 --> 01:05:04.050 align:middle line:90% impact as-- 01:05:04.050 --> 01:05:06.550 align:middle line:84% increases the return to education. 01:05:06.550 --> 01:05:08.940 align:middle line:84% So that's the first thing that they say. 01:05:08.940 --> 01:05:14.430 align:middle line:84% And then, as a result, when you see high-yielding varieties 01:05:14.430 --> 01:05:16.500 align:middle line:84% being introduced, that should increase 01:05:16.500 --> 01:05:21.570 align:middle line:84% people educating their kids because the return to education 01:05:21.570 --> 01:05:24.700 align:middle line:90% have now gone up. 01:05:24.700 --> 01:05:25.810 align:middle line:90% And they find that. 01:05:25.810 --> 01:05:27.420 align:middle line:84% Now, one issue with that exercise 01:05:27.420 --> 01:05:29.170 align:middle line:84% is that at the same time that it increased 01:05:29.170 --> 01:05:30.628 align:middle line:84% the returns to education, it should 01:05:30.628 --> 01:05:32.560 align:middle line:90% increase people's income. 01:05:32.560 --> 01:05:36.460 align:middle line:84% And we've seen that in a richer model of the world, 01:05:36.460 --> 01:05:39.220 align:middle line:84% income effect would also increase 01:05:39.220 --> 01:05:41.442 align:middle line:90% education in and of itself. 01:05:41.442 --> 01:05:43.150 align:middle line:84% That's not an issue for them because they 01:05:43.150 --> 01:05:44.450 align:middle line:90% don't believe in income effect. 01:05:44.450 --> 01:05:47.530 align:middle line:84% So the only thing that could possibly have happened 01:05:47.530 --> 01:05:52.780 align:middle line:84% is the response to returns to education. 01:05:52.780 --> 01:05:55.700 align:middle line:84% But one could be worried about that. 01:05:55.700 --> 01:06:02.270 align:middle line:84% Two more recent paper that have the same-- 01:06:02.270 --> 01:06:04.910 align:middle line:84% those two papers have a little bit the same mechanism 01:06:04.910 --> 01:06:05.810 align:middle line:90% at hand-- 01:06:05.810 --> 01:06:10.050 align:middle line:84% is a David Atkin paper and Sara Hernandez paper. 01:06:10.050 --> 01:06:12.470 align:middle line:84% Sara was a graduate student here and now teaches 01:06:12.470 --> 01:06:15.810 align:middle line:90% at Northwestern. 01:06:15.810 --> 01:06:19.010 align:middle line:84% They look at expert jobs and export 01:06:19.010 --> 01:06:22.100 align:middle line:84% jobs coming from the garment sector, mostly in Mexico, 01:06:22.100 --> 01:06:24.810 align:middle line:84% and the flower sector in Colombia. 01:06:24.810 --> 01:06:27.630 align:middle line:84% And they show that when these jobs are introduced 01:06:27.630 --> 01:06:30.810 align:middle line:84% it leads to dropouts of girls despite the fact 01:06:30.810 --> 01:06:32.910 align:middle line:90% that people aren't richer. 01:06:32.910 --> 01:06:36.570 align:middle line:84% Which they explain by the fact that the return to education 01:06:36.570 --> 01:06:38.700 align:middle line:84% are now lower because you don't need to be educated 01:06:38.700 --> 01:06:40.270 align:middle line:90% to get any of these jobs. 01:06:40.270 --> 01:06:43.410 align:middle line:84% So you can get a good job working in a garment factory, 01:06:43.410 --> 01:06:45.960 align:middle line:84% or you can get a good job working in a flower sector 01:06:45.960 --> 01:06:46.840 align:middle line:90% without education. 01:06:46.840 --> 01:06:49.110 align:middle line:90% So people dropped out. 01:06:49.110 --> 01:06:51.732 align:middle line:84% In the example of David, they got screwed for it 01:06:51.732 --> 01:06:53.190 align:middle line:84% because then the sector disappeared 01:06:53.190 --> 01:06:54.773 align:middle line:84% and they found themselves without jobs 01:06:54.773 --> 01:06:57.780 align:middle line:90% and without an education. 01:06:57.780 --> 01:06:59.820 align:middle line:84% That's the full story of that paper. 01:06:59.820 --> 01:07:01.690 align:middle line:84% But so the advantage-- "advantage"-- 01:07:01.690 --> 01:07:05.520 align:middle line:84% the advantage of that example is that the return to education go 01:07:05.520 --> 01:07:07.660 align:middle line:84% down at the same time as the income go up. 01:07:07.660 --> 01:07:11.520 align:middle line:84% So we don't have the income effect and the substitution 01:07:11.520 --> 01:07:13.840 align:middle line:84% effect going in the same direction. 01:07:13.840 --> 01:07:15.480 align:middle line:84% So if we find education dropping, 01:07:15.480 --> 01:07:20.790 align:middle line:84% it seems likely that it's because of that. 01:07:20.790 --> 01:07:23.300 align:middle line:84% But then Jensen went back one step further 01:07:23.300 --> 01:07:27.000 align:middle line:84% and say, well, you didn't like my change in the belief. 01:07:27.000 --> 01:07:30.750 align:middle line:84% So I'm going to change the return. 01:07:30.750 --> 01:07:33.810 align:middle line:84% That seems ambitious to say, are you 01:07:33.810 --> 01:07:35.580 align:middle line:84% able to change the return to education. 01:07:35.580 --> 01:07:38.970 align:middle line:84% And he had a great, great idea, which 01:07:38.970 --> 01:07:42.720 align:middle line:84% is to participate in a recruiting campaign for call 01:07:42.720 --> 01:07:45.750 align:middle line:84% centers, these type of things when you call 01:07:45.750 --> 01:07:47.790 align:middle line:84% for your computer, and typically it's 01:07:47.790 --> 01:07:52.260 align:middle line:84% a nice young person from the suburb of Delhi, where 01:07:52.260 --> 01:07:56.510 align:middle line:84% suburb is intended in a very broad sense who 01:07:56.510 --> 01:07:58.760 align:middle line:90% is answering the phone. 01:07:58.760 --> 01:08:02.150 align:middle line:84% So early on they were really recruiting mostly in Goregaon, 01:08:02.150 --> 01:08:03.710 align:middle line:90% very close to Delhi. 01:08:03.710 --> 01:08:06.920 align:middle line:90% And then they started expanding. 01:08:06.920 --> 01:08:09.560 align:middle line:84% And he worked with them at the beginning of expansion 01:08:09.560 --> 01:08:14.750 align:middle line:84% to lead them to villages where they had never been in Haryana, 01:08:14.750 --> 01:08:21.130 align:middle line:84% or in UP, so basically a broader sector are in UP. 01:08:21.130 --> 01:08:27.279 align:middle line:84% He connects the BPO, the back office processing 01:08:27.279 --> 01:08:29.439 align:middle line:90% operation with the villages. 01:08:29.439 --> 01:08:33.880 align:middle line:84% Someone goes to the village, advertises the possibility, 01:08:33.880 --> 01:08:37.060 align:middle line:90% and then sees what happens. 01:08:37.060 --> 01:08:40.229 align:middle line:84% So some girls who were the right age 01:08:40.229 --> 01:08:42.029 align:middle line:84% and had the right education, which 01:08:42.029 --> 01:08:45.840 align:middle line:84% is secondary education with some English went. 01:08:45.840 --> 01:08:48.270 align:middle line:84% And he's interested in what happens to the younger 01:08:48.270 --> 01:08:51.479 align:middle line:90% girls who could not go yet. 01:08:51.479 --> 01:08:52.439 align:middle line:90% They were little. 01:08:52.439 --> 01:08:56.069 align:middle line:84% Are they more likely to stay in school? 01:08:56.069 --> 01:09:01.140 align:middle line:84% So what he finds is that women are indeed-- 01:09:01.140 --> 01:09:04.770 align:middle line:84% younger women are indeed more likely to be working in a BPO. 01:09:04.770 --> 01:09:07.470 align:middle line:90% So that's good. 01:09:07.470 --> 01:09:09.194 align:middle line:84% There was no such effect for men. 01:09:09.194 --> 01:09:11.819 align:middle line:90% 01:09:11.819 --> 01:09:17.189 align:middle line:84% And that both younger and-- both these women 01:09:17.189 --> 01:09:21.899 align:middle line:84% but also the younger women, the girls 01:09:21.899 --> 01:09:26.850 align:middle line:84% were more likely to be in school. 01:09:26.850 --> 01:09:30.465 align:middle line:90% And even younger are better fed. 01:09:30.465 --> 01:09:33.260 align:middle line:90% 01:09:33.260 --> 01:09:35.630 align:middle line:90% So basically, the parents-- 01:09:35.630 --> 01:09:38.200 align:middle line:84% the story is, parents realized that-- 01:09:38.200 --> 01:09:39.680 align:middle line:90% there is more in terms to girls. 01:09:39.680 --> 01:09:42.140 align:middle line:84% In general, they become more valuable in general, 01:09:42.140 --> 01:09:44.880 align:middle line:84% and in particular are more valuable if they are educated. 01:09:44.880 --> 01:09:47.899 align:middle line:84% So the return to education for women have gone up. 01:09:47.899 --> 01:09:52.100 align:middle line:84% And they start educating them more. 01:09:52.100 --> 01:09:54.229 align:middle line:84% So interestingly, there is no effect for men. 01:09:54.229 --> 01:09:56.000 align:middle line:84% It's not discussed much in this paper. 01:09:56.000 --> 01:09:58.490 align:middle line:84% But I have another paper where they separate 01:09:58.490 --> 01:10:03.320 align:middle line:84% the boys between people who at baseline were described 01:10:03.320 --> 01:10:06.090 align:middle line:84% to be the one who are going to take care of the family farm. 01:10:06.090 --> 01:10:09.470 align:middle line:84% So usually the oldest boy, versus the younger boy. 01:10:09.470 --> 01:10:10.910 align:middle line:84% And when they separate that, they 01:10:10.910 --> 01:10:13.910 align:middle line:84% find that actually when the BPO equipment started 01:10:13.910 --> 01:10:17.480 align:middle line:84% for the boys who are supposed to work in the family farm 01:10:17.480 --> 01:10:19.670 align:middle line:90% the education went down. 01:10:19.670 --> 01:10:22.520 align:middle line:84% And for other boys, it's more like girls. 01:10:22.520 --> 01:10:25.100 align:middle line:84% And so the idea there is that you're very scared 01:10:25.100 --> 01:10:27.223 align:middle line:84% to have return to education go up for the boys 01:10:27.223 --> 01:10:29.390 align:middle line:84% because you don't want them to leave because someone 01:10:29.390 --> 01:10:31.542 align:middle line:84% needs to stay to take care of you 01:10:31.542 --> 01:10:32.750 align:middle line:90% and to take care of the farm. 01:10:32.750 --> 01:10:35.540 align:middle line:84% And therefore, you're going to suppress education so that they 01:10:35.540 --> 01:10:36.620 align:middle line:90% don't-- 01:10:36.620 --> 01:10:40.090 align:middle line:84% here the return to education for you 01:10:40.090 --> 01:10:42.380 align:middle line:84% are negative from the point of view of the parents. 01:10:42.380 --> 01:10:44.450 align:middle line:84% So that leads a little bit to the conversation 01:10:44.450 --> 01:10:48.350 align:middle line:84% we had with [INAUDIBLE] and others 01:10:48.350 --> 01:10:51.150 align:middle line:84% about the contract between the family. 01:10:51.150 --> 01:10:53.430 align:middle line:84% That if you cannot contract to get the money back from 01:10:53.430 --> 01:10:55.230 align:middle line:84% your boy, then you are not sending them. 01:10:55.230 --> 01:11:00.240 align:middle line:90% 01:11:00.240 --> 01:11:04.070 align:middle line:84% So there are two other things that I 01:11:04.070 --> 01:11:06.680 align:middle line:84% wanted to do that I didn't have time to do. 01:11:06.680 --> 01:11:11.570 align:middle line:84% One of them I'll do later when we talk about the family. 01:11:11.570 --> 01:11:12.080 align:middle line:90% That's fine. 01:11:12.080 --> 01:11:13.430 align:middle line:90% We can take that. 01:11:13.430 --> 01:11:16.400 align:middle line:84% That's the Rebekah [INAUDIBLE] paper about the fact 01:11:16.400 --> 01:11:19.850 align:middle line:84% that parents have no idea what their kids are up to. 01:11:19.850 --> 01:11:23.270 align:middle line:84% But this one I want to briefly mention. 01:11:23.270 --> 01:11:26.360 align:middle line:84% It's a paper by Seema Jayachandran and Adriana 01:11:26.360 --> 01:11:31.820 align:middle line:84% Lleras-Muney who is looking at another way of affecting 01:11:31.820 --> 01:11:33.350 align:middle line:84% the return to education or the way 01:11:33.350 --> 01:11:36.590 align:middle line:84% returns are affecting is simply the fact that how long 01:11:36.590 --> 01:11:38.720 align:middle line:90% are you going to live? 01:11:38.720 --> 01:11:41.240 align:middle line:84% So if you expect to live for a very short period of time, 01:11:41.240 --> 01:11:42.970 align:middle line:84% then your education is not very worth it. 01:11:42.970 --> 01:11:45.880 align:middle line:84% Or if you expect your girl to live for a very short time 01:11:45.880 --> 01:11:49.030 align:middle line:84% because she's going to die in childbirth anyway, 01:11:49.030 --> 01:11:52.330 align:middle line:84% then she's not going to experience the return 01:11:52.330 --> 01:11:53.270 align:middle line:90% for a very short time. 01:11:53.270 --> 01:11:55.480 align:middle line:84% So that's a very economist way to think. 01:11:55.480 --> 01:12:00.590 align:middle line:84% But it is quite interesting that it works here. 01:12:00.590 --> 01:12:03.740 align:middle line:84% So what they are looking for, to know whether people-- 01:12:03.740 --> 01:12:06.400 align:middle line:84% so in principle, the implication of this model 01:12:06.400 --> 01:12:09.170 align:middle line:84% is that if people expect to live longer 01:12:09.170 --> 01:12:12.800 align:middle line:84% they should-- or expect their daughters to live longer, 01:12:12.800 --> 01:12:14.680 align:middle line:84% they should educate them more because they 01:12:14.680 --> 01:12:17.350 align:middle line:84% are going to enjoy the return of this education over a longer 01:12:17.350 --> 01:12:19.340 align:middle line:90% time period. 01:12:19.340 --> 01:12:22.300 align:middle line:84% So you need an instrument that affect 01:12:22.300 --> 01:12:25.210 align:middle line:84% the number of years someone can expect 01:12:25.210 --> 01:12:27.670 align:middle line:84% to live without affecting anything else, like the skill 01:12:27.670 --> 01:12:31.660 align:middle line:84% premium or the low skill wage, which 01:12:31.660 --> 01:12:34.080 align:middle line:90% seems like a very hard problem. 01:12:34.080 --> 01:12:39.660 align:middle line:84% And what they find is in Sri Lanka a program that introduces 01:12:39.660 --> 01:12:43.410 align:middle line:90% ambulance in all the villages. 01:12:43.410 --> 01:12:50.495 align:middle line:84% And when they did that, infant mortality collapsed. 01:12:50.495 --> 01:12:53.420 align:middle line:84% They did that, infant mortality collapsed. 01:12:53.420 --> 01:12:56.480 align:middle line:84% And in particular, the drop in infant mortality 01:12:56.480 --> 01:13:00.150 align:middle line:84% was much larger in places where infant mortality was very high. 01:13:00.150 --> 01:13:03.380 align:middle line:84% So basically, they're putting the ambulance, flattened 01:13:03.380 --> 01:13:06.110 align:middle line:84% out the infant mortality curve, leading 01:13:06.110 --> 01:13:07.970 align:middle line:84% to huge decline in infant mortality 01:13:07.970 --> 01:13:10.910 align:middle line:84% in places that were really far from the hospital 01:13:10.910 --> 01:13:12.870 align:middle line:84% and people couldn't get to the hospital. 01:13:12.870 --> 01:13:16.130 align:middle line:90% So the ambulance eliminate that. 01:13:16.130 --> 01:13:17.980 align:middle line:84% So now they use that in saying, well, 01:13:17.980 --> 01:13:24.730 align:middle line:84% that affects the life expectancy of the women but not the men. 01:13:24.730 --> 01:13:27.100 align:middle line:84% So now I'm going to use basically 01:13:27.100 --> 01:13:30.790 align:middle line:84% the pre-ambulance maternal mortality 01:13:30.790 --> 01:13:36.550 align:middle line:84% rate as an instrument for the decline in maternal mortality 01:13:36.550 --> 01:13:39.520 align:middle line:84% rate over the period to see whether people 01:13:39.520 --> 01:13:45.160 align:middle line:84% seem to respond by educating their girls more 01:13:45.160 --> 01:13:46.540 align:middle line:90% relative to the boy. 01:13:46.540 --> 01:13:49.990 align:middle line:84% So now what we have here is a triple difference 01:13:49.990 --> 01:13:51.760 align:middle line:90% specification. 01:13:51.760 --> 01:13:53.080 align:middle line:90% I just write down-- 01:13:53.080 --> 01:13:55.120 align:middle line:90% put the specification down-- 01:13:55.120 --> 01:13:58.870 align:middle line:84% where we can have a fixed effect-- 01:13:58.870 --> 01:14:03.400 align:middle line:84% we can have a two-way fixed effect whichever way, district, 01:14:03.400 --> 01:14:09.550 align:middle line:84% year, district time year, and gender time district, 01:14:09.550 --> 01:14:10.850 align:middle line:90% and gender time year. 01:14:10.850 --> 01:14:13.330 align:middle line:84% So all of the double interaction can be there. 01:14:13.330 --> 01:14:18.100 align:middle line:84% Because what we are going to use is maternal mortality 01:14:18.100 --> 01:14:20.650 align:middle line:84% in a particular year, like the time 01:14:20.650 --> 01:14:24.220 align:middle line:84% when the parents could observe it, 01:14:24.220 --> 01:14:27.170 align:middle line:90% times the female [INAUDIBLE]. 01:14:27.170 --> 01:14:34.652 align:middle line:84% And they find that more maternal mortality, less education. 01:14:34.652 --> 01:14:42.040 align:middle line:90% 01:14:42.040 --> 01:14:46.830 align:middle line:84% So using just that, they find that more maternal mortality 01:14:46.830 --> 01:14:48.040 align:middle line:90% leads to less education. 01:14:48.040 --> 01:14:49.860 align:middle line:84% So that's kind of a nice paper because I 01:14:49.860 --> 01:14:53.070 align:middle line:84% think many people were kind of intuiting that something 01:14:53.070 --> 01:14:55.420 align:middle line:90% like that might be going on. 01:14:55.420 --> 01:14:58.470 align:middle line:84% But since everything is usually correlated with everything 01:14:58.470 --> 01:14:59.880 align:middle line:90% it's very hard to test. 01:14:59.880 --> 01:15:04.740 align:middle line:84% And this ambulance program that had such a massive effect 01:15:04.740 --> 01:15:07.830 align:middle line:84% on maternal mortality reduction gave them 01:15:07.830 --> 01:15:11.270 align:middle line:90% a chance to test the idea. 01:15:11.270 --> 01:15:11.770 align:middle line:90% All right. 01:15:11.770 --> 01:15:14.680 align:middle line:84% So sorry-- so we're done for today. 01:15:14.680 --> 01:15:18.520 align:middle line:90% Sorry again for the slow start. 01:15:18.520 --> 01:15:22.310 align:middle line:84% And on Monday, we'll continue talking about the Indonesia 01:15:22.310 --> 01:15:22.810 align:middle line:90% paper. 01:15:22.810 --> 01:15:25.930 align:middle line:84% But we'll talk about this the estimating the return 01:15:25.930 --> 01:15:26.620 align:middle line:90% to education. 01:15:26.620 --> 01:15:28.790 align:middle line:84% We also talk about the Ghana paper, 01:15:28.790 --> 01:15:31.660 align:middle line:84% which I'm asking you to read as you go along, 01:15:31.660 --> 01:15:36.030 align:middle line:84% which is another take at estimating returns to education 01:15:36.030 --> 01:15:38.570 align:middle line:90% or impact of education. 01:15:38.570 --> 01:15:50.000 align:middle line:90%