WEBVTT 00:00:00.000 --> 00:00:06.474 align:middle line:84% [SQUEAKING] [RUSTLING] [CLICKING] 00:00:06.474 --> 00:00:18.460 align:middle line:90% 00:00:18.460 --> 00:00:19.960 align:middle line:84% BEN OLKEN: OK, so just to remind you 00:00:19.960 --> 00:00:21.380 align:middle line:90% guys where we were last time-- 00:00:21.380 --> 00:00:23.440 align:middle line:90% So we're talking about-- 00:00:23.440 --> 00:00:24.940 align:middle line:84% in the middle of the stuff on how do 00:00:24.940 --> 00:00:29.650 align:middle line:84% we figure out who's poor for redistribution programs. 00:00:29.650 --> 00:00:33.010 align:middle line:84% And what I was telling you about was this first experiment 00:00:33.010 --> 00:00:36.850 align:middle line:84% that we did in Indonesia looking at community-based approaches 00:00:36.850 --> 00:00:38.530 align:middle line:90% to identifying who is poor. 00:00:38.530 --> 00:00:41.140 align:middle line:84% So this idea is they're going to rank people or whatever. 00:00:41.140 --> 00:00:46.330 align:middle line:84% So what I want to talk about is a little bit on this. 00:00:46.330 --> 00:00:50.390 align:middle line:84% And then I want to talk about the self-targeting paper. 00:00:50.390 --> 00:00:56.627 align:middle line:84% So one thing you can do is we can just-- so how do we 00:00:56.627 --> 00:00:57.210 align:middle line:90% evaluate this? 00:00:57.210 --> 00:00:58.127 align:middle line:90% What are the outcomes? 00:00:58.127 --> 00:01:02.070 align:middle line:84% So just to remind you guys where we were like Wednesday 00:01:02.070 --> 00:01:04.230 align:middle line:90% when we talked about this-- 00:01:04.230 --> 00:01:11.700 align:middle line:84% Wednesday, yeah-- was we had a baseline survey 00:01:11.700 --> 00:01:14.550 align:middle line:90% to measure the truth. 00:01:14.550 --> 00:01:17.550 align:middle line:84% We then randomize villages into different ways 00:01:17.550 --> 00:01:19.350 align:middle line:90% of identifying poor households. 00:01:19.350 --> 00:01:22.090 align:middle line:84% And then we want to evaluate kind of which did better. 00:01:22.090 --> 00:01:23.820 align:middle line:84% So one simple thing you can do is 00:01:23.820 --> 00:01:27.090 align:middle line:84% you can say we can define mistargeting just 00:01:27.090 --> 00:01:29.910 align:middle line:84% to be a dummy variable if we got it wrong. 00:01:29.910 --> 00:01:33.240 align:middle line:84% So if you're poor and we gave you the program, 00:01:33.240 --> 00:01:35.280 align:middle line:90% that's an error. 00:01:35.280 --> 00:01:36.858 align:middle line:90% If we're rich and you-- 00:01:36.858 --> 00:01:39.400 align:middle line:84% sorry, poor and you did not get the program, that's an error. 00:01:39.400 --> 00:01:41.150 align:middle line:84% Rich and you got that program is an error. 00:01:41.150 --> 00:01:42.760 align:middle line:90% That's kind of discrete. 00:01:42.760 --> 00:01:44.510 align:middle line:84% But we may want to do something different. 00:01:44.510 --> 00:01:45.760 align:middle line:84% And in particular, one of the things 00:01:45.760 --> 00:01:47.677 align:middle line:84% we were interested in this paper was whether-- 00:01:47.677 --> 00:01:54.360 align:middle line:90% 00:01:54.360 --> 00:01:57.670 align:middle line:84% trying to evaluate these things on different welfare metrics. 00:01:57.670 --> 00:02:00.750 align:middle line:84% So in particular, we wanted to identify 00:02:00.750 --> 00:02:03.330 align:middle line:84% what not only did they do better on consumption, 00:02:03.330 --> 00:02:06.120 align:middle line:84% but did they do better on other alternative measures 00:02:06.120 --> 00:02:07.050 align:middle line:90% of well-being. 00:02:07.050 --> 00:02:09.900 align:middle line:84% So we started off by saying consumption, 00:02:09.900 --> 00:02:12.650 align:middle line:84% just again, that the per capita consumption. 00:02:12.650 --> 00:02:14.840 align:middle line:84% We then wanted to say, well, are the communities, 00:02:14.840 --> 00:02:16.850 align:middle line:84% when they're ranking people, doing a better job 00:02:16.850 --> 00:02:19.100 align:middle line:84% kind of in that village of matching how people 00:02:19.100 --> 00:02:21.680 align:middle line:84% kind of privately rank each other in the privacy 00:02:21.680 --> 00:02:22.430 align:middle line:90% of their own home. 00:02:22.430 --> 00:02:24.407 align:middle line:84% Like is there some generally accepted notion 00:02:24.407 --> 00:02:26.240 align:middle line:84% in this village of who's poor and who's not? 00:02:26.240 --> 00:02:28.130 align:middle line:84% And maybe that's what they're matching. 00:02:28.130 --> 00:02:30.410 align:middle line:84% Maybe they're matching how the elites in this village 00:02:30.410 --> 00:02:31.677 align:middle line:90% are matching people. 00:02:31.677 --> 00:02:33.260 align:middle line:84% And the other thing that they're doing 00:02:33.260 --> 00:02:34.490 align:middle line:84% is they're matching-- maybe they're 00:02:34.490 --> 00:02:36.532 align:middle line:84% matching people's self-assessment, their own kind 00:02:36.532 --> 00:02:38.810 align:middle line:84% of internal conceptions of poverty. 00:02:38.810 --> 00:02:41.750 align:middle line:84% And so we measured all four of those things at base-- all four 00:02:41.750 --> 00:02:43.292 align:middle line:84% of these things actually at baseline. 00:02:43.292 --> 00:02:45.770 align:middle line:84% And so we wanted to compare which of these things 00:02:45.770 --> 00:02:48.500 align:middle line:84% does better at matching these four different welfare metrics. 00:02:48.500 --> 00:02:52.820 align:middle line:84% And the thing I wanted to point out 00:02:52.820 --> 00:02:56.040 align:middle line:84% is one problem is these are on very different scales. 00:02:56.040 --> 00:03:00.387 align:middle line:84% So we wanted to find kind of a way of evaluating-- 00:03:00.387 --> 00:03:02.220 align:middle line:84% so we need to put all these things onto kind 00:03:02.220 --> 00:03:04.220 align:middle line:84% of a comparable metric to make sure we were kind 00:03:04.220 --> 00:03:07.270 align:middle line:90% of comparing comparable things. 00:03:07.270 --> 00:03:12.360 align:middle line:84% And so what we did is we came up with a ranked correlation. 00:03:12.360 --> 00:03:18.090 align:middle line:84% So in particular, we took each of those four metrics, 00:03:18.090 --> 00:03:19.980 align:middle line:84% and those pretty much are ordering 00:03:19.980 --> 00:03:23.470 align:middle line:84% of people in the village, maybe with ties. 00:03:23.470 --> 00:03:26.890 align:middle line:84% So like Ben is the richest based on consumption, 00:03:26.890 --> 00:03:30.010 align:middle line:84% or Ben is the richest based on-- or the happiest based on self 00:03:30.010 --> 00:03:31.910 align:middle line:84% assessed kind of utility or whatever. 00:03:31.910 --> 00:03:34.570 align:middle line:84% So all those things generate rankings. 00:03:34.570 --> 00:03:36.790 align:middle line:90% And then we can compute-- 00:03:36.790 --> 00:03:40.810 align:middle line:84% and each targeting method also generates a complete ranking. 00:03:40.810 --> 00:03:43.006 align:middle line:90% That was the ranking from the-- 00:03:43.006 --> 00:03:45.400 align:middle line:84% in this one, it's the ranking on the wall 00:03:45.400 --> 00:03:48.370 align:middle line:90% or on the clothes line. 00:03:48.370 --> 00:03:52.330 align:middle line:84% In the proxy means test, it's your predicted income score, 00:03:52.330 --> 00:03:53.530 align:middle line:90% it's also a ranking. 00:03:53.530 --> 00:03:56.180 align:middle line:84% And then for each of the 600 villages in our experiment, 00:03:56.180 --> 00:03:58.870 align:middle line:84% we can compute we can compute a ranked correlation, which 00:03:58.870 --> 00:04:01.930 align:middle line:84% is the correlation of ranks between the targeting 00:04:01.930 --> 00:04:04.150 align:middle line:84% outcome and each of those four different kind 00:04:04.150 --> 00:04:05.660 align:middle line:90% of welfare metrics. 00:04:05.660 --> 00:04:09.250 align:middle line:84% And so I just thought that was-- it's kind of a little bit 00:04:09.250 --> 00:04:11.440 align:middle line:84% unusual because we had to think about 00:04:11.440 --> 00:04:15.910 align:middle line:84% how do you take these things and put them on a common scale? 00:04:15.910 --> 00:04:21.365 align:middle line:84% So we're going to end up doing is creating essentially 600 00:04:21.365 --> 00:04:23.490 align:middle line:84% times 4 different rank correlations because there's 00:04:23.490 --> 00:04:25.365 align:middle line:84% 4 different welfare metrics and 600 villages. 00:04:25.365 --> 00:04:27.793 align:middle line:84% So there'll be 2,400 different rank correlations. 00:04:27.793 --> 00:04:30.210 align:middle line:84% And then we're going to regress for each different welfare 00:04:30.210 --> 00:04:33.060 align:middle line:84% metric this rank correlation itself 00:04:33.060 --> 00:04:35.800 align:middle line:90% on the targeting method used. 00:04:35.800 --> 00:04:38.190 align:middle line:84% So we're trying to predict which-- 00:04:38.190 --> 00:04:40.480 align:middle line:84% did this method generate a higher rank correlation? 00:04:40.480 --> 00:04:42.550 align:middle line:84% So it's a little unusual, which I just wanted 00:04:42.550 --> 00:04:45.510 align:middle line:90% to describe that in detail. 00:04:45.510 --> 00:04:47.100 align:middle line:90% And so what do you find? 00:04:47.100 --> 00:04:50.220 align:middle line:84% Well, on the one hand, you do find that the community method 00:04:50.220 --> 00:04:52.180 align:middle line:90% made more errors. 00:04:52.180 --> 00:04:55.500 align:middle line:84% So this is regressing kind of like this dummy 00:04:55.500 --> 00:04:59.633 align:middle line:84% of mistargeting by basing assumption on the community 00:04:59.633 --> 00:05:00.300 align:middle line:90% and the hybrid-- 00:05:00.300 --> 00:05:02.790 align:middle line:84% I'm not going to talk much about the hybrid-- 00:05:02.790 --> 00:05:05.220 align:middle line:84% and predominantly kind of by people 00:05:05.220 --> 00:05:07.950 align:middle line:84% near the middle of the distribution. 00:05:07.950 --> 00:05:11.340 align:middle line:84% But on the other hand, if you look at the 4 different rank-- 00:05:11.340 --> 00:05:12.750 align:middle line:84% if you switch to this table where 00:05:12.750 --> 00:05:14.670 align:middle line:84% I'm regressing the 4 rank correlations 00:05:14.670 --> 00:05:16.110 align:middle line:84% on the different treatments, what 00:05:16.110 --> 00:05:18.265 align:middle line:84% you can find is that actually that indeed 00:05:18.265 --> 00:05:19.890 align:middle line:84% consistent with what I just showed you, 00:05:19.890 --> 00:05:22.320 align:middle line:84% the rank correlation between what 00:05:22.320 --> 00:05:24.690 align:middle line:84% is produced by targeting and baseline welfare 00:05:24.690 --> 00:05:28.665 align:middle line:84% is lower if we use consumption as baseline welfare 00:05:28.665 --> 00:05:29.790 align:middle line:90% in the community treatment. 00:05:29.790 --> 00:05:32.990 align:middle line:84% But it's higher on basically everything else. 00:05:32.990 --> 00:05:34.930 align:middle line:84% So it's not just that these communities 00:05:34.930 --> 00:05:38.558 align:middle line:84% are like systematically making mistakes or being totally 00:05:38.558 --> 00:05:39.850 align:middle line:90% captured by elites or whatever. 00:05:39.850 --> 00:05:41.830 align:middle line:84% No, what's actually is happening is, 00:05:41.830 --> 00:05:44.230 align:middle line:84% I think, to my mind, the best assessment, 00:05:44.230 --> 00:05:46.300 align:middle line:84% best thing that's happening is there 00:05:46.300 --> 00:05:48.370 align:middle line:84% are some local notion of what poverty 00:05:48.370 --> 00:05:51.820 align:middle line:84% is in this village, which is highly correlated with but not 00:05:51.820 --> 00:05:53.780 align:middle line:84% perfectly correlated with income. 00:05:53.780 --> 00:05:57.710 align:middle line:84% And when you tell communities to tell you who's poor, 00:05:57.710 --> 00:05:59.630 align:middle line:90% that's what you get. 00:05:59.630 --> 00:06:01.670 align:middle line:84% And in particular, you strongly get 00:06:01.670 --> 00:06:06.780 align:middle line:84% that it's very correlated with-- what we have in communities is 00:06:06.780 --> 00:06:09.852 align:middle line:84% very correlated with kind of what you would get if you ask 00:06:09.852 --> 00:06:11.310 align:middle line:84% people about who is poor and who is 00:06:11.310 --> 00:06:12.852 align:middle line:84% rich in the privacy of your own home. 00:06:12.852 --> 00:06:14.420 align:middle line:90% And it's also correlated with-- 00:06:14.420 --> 00:06:16.170 align:middle line:84% same thing if you ask the subvillage head, 00:06:16.170 --> 00:06:18.010 align:middle line:84% the neighborhood head, to do the same thing. 00:06:18.010 --> 00:06:18.570 align:middle line:90% It's very correlated with that. 00:06:18.570 --> 00:06:20.220 align:middle line:84% It's also pretty strongly correlated 00:06:20.220 --> 00:06:21.933 align:middle line:84% with just your own self-assessed poverty. 00:06:21.933 --> 00:06:23.100 align:middle line:90% This is just a ladder scale. 00:06:23.100 --> 00:06:25.500 align:middle line:84% On a scale of 1 to 6, where 6 is the richest people in society 00:06:25.500 --> 00:06:27.510 align:middle line:84% in 1 is the poorest, where you put yourself, 00:06:27.510 --> 00:06:29.080 align:middle line:90% that thing generates ranks. 00:06:29.080 --> 00:06:31.038 align:middle line:84% And what you get out of the community treatment 00:06:31.038 --> 00:06:34.500 align:middle line:84% is like 10 percentage points more correlated with that thing 00:06:34.500 --> 00:06:35.350 align:middle line:90% in the community. 00:06:35.350 --> 00:06:44.228 align:middle line:84% So it seems like you're picking up 00:06:44.228 --> 00:06:45.770 align:middle line:84% a kind of a different welfare metric. 00:06:45.770 --> 00:06:47.120 align:middle line:84% So I think that's kind interesting that maybe 00:06:47.120 --> 00:06:48.703 align:middle line:84% these different local welfare metrics, 00:06:48.703 --> 00:06:50.328 align:middle line:84% and that's what you're picking up here. 00:06:50.328 --> 00:06:51.870 align:middle line:84% And the other thing I'll just mention 00:06:51.870 --> 00:06:55.410 align:middle line:84% is that the communities all like this community-based approach 00:06:55.410 --> 00:06:57.010 align:middle line:90% much better. 00:06:57.010 --> 00:06:59.370 align:middle line:90% So another question is this-- 00:06:59.370 --> 00:07:01.900 align:middle line:84% and we measured this in two ways-- 00:07:01.900 --> 00:07:03.690 align:middle line:84% one is just some general satisfaction. 00:07:03.690 --> 00:07:05.790 align:middle line:84% Like in an online survey, are you satisfied with the way 00:07:05.790 --> 00:07:07.248 align:middle line:84% targeting was done in this village? 00:07:07.248 --> 00:07:10.830 align:middle line:84% Way more satisfied in the community treatment? 00:07:10.830 --> 00:07:12.570 align:middle line:84% The other thing is we show them the list, 00:07:12.570 --> 00:07:13.890 align:middle line:90% and say, here's the list. 00:07:13.890 --> 00:07:15.510 align:middle line:90% Did we miss anybody? 00:07:15.510 --> 00:07:16.510 align:middle line:90% Anybody should be added? 00:07:16.510 --> 00:07:19.000 align:middle line:84% And they're making many few fewer edits to the list. 00:07:19.000 --> 00:07:20.625 align:middle line:84% So it seems like it's really picking up 00:07:20.625 --> 00:07:22.710 align:middle line:84% kind of that local welfare metric, 00:07:22.710 --> 00:07:25.717 align:middle line:84% and they're much happier with it. 00:07:25.717 --> 00:07:28.300 align:middle line:84% So in general, kind of the point of that paper I think is that 00:07:28.300 --> 00:07:31.060 align:middle line:84% the community has a somewhat-- we started off by actually 00:07:31.060 --> 00:07:32.920 align:middle line:84% designing this whole project as like-- 00:07:32.920 --> 00:07:34.753 align:middle line:84% we thought it was a trade off between a leak 00:07:34.753 --> 00:07:39.483 align:middle line:90% capture and better information. 00:07:39.483 --> 00:07:40.400 align:middle line:90% That was the decision. 00:07:40.400 --> 00:07:41.942 align:middle line:84% But actually, the third thing kind of 00:07:41.942 --> 00:07:44.180 align:middle line:84% emerged that we weren't really expecting, 00:07:44.180 --> 00:07:46.390 align:middle line:84% which it was really about this different well metric. 00:07:46.390 --> 00:07:47.350 align:middle line:84% And I think that's actually what I think 00:07:47.350 --> 00:07:48.530 align:middle line:90% is the most important thing. 00:07:48.530 --> 00:07:51.910 align:middle line:84% So we find kind of basically no elite capture. 00:07:51.910 --> 00:07:54.288 align:middle line:84% We went to all this effort to measure elites 00:07:54.288 --> 00:07:56.080 align:middle line:84% by like who's connected to the village head 00:07:56.080 --> 00:07:57.413 align:middle line:90% and who's family members of who. 00:07:57.413 --> 00:08:00.730 align:middle line:84% And so none of that seems to matter. 00:08:00.730 --> 00:08:04.000 align:middle line:84% We do actually show that there is local information 00:08:04.000 --> 00:08:04.852 align:middle line:90% about consumption. 00:08:04.852 --> 00:08:06.310 align:middle line:84% They do have additional information 00:08:06.310 --> 00:08:10.060 align:middle line:84% that the government doesn't have. 00:08:10.060 --> 00:08:12.400 align:middle line:84% But they are tending to do something a little bit 00:08:12.400 --> 00:08:13.960 align:middle line:90% different. 00:08:13.960 --> 00:08:15.710 align:middle line:84% OK, so any questions on this before I move 00:08:15.710 --> 00:08:16.585 align:middle line:90% on to the next paper? 00:08:16.585 --> 00:08:19.500 align:middle line:90% 00:08:19.500 --> 00:08:22.790 align:middle line:84% OK, so I think the conclusion is, 00:08:22.790 --> 00:08:24.815 align:middle line:84% if you want to use poverty headcount, 00:08:24.815 --> 00:08:26.440 align:middle line:84% minimize poverty headcount, maybe you'd 00:08:26.440 --> 00:08:27.970 align:middle line:84% want to use the database approach. 00:08:27.970 --> 00:08:29.860 align:middle line:84% But if your goal is to maximize utility 00:08:29.860 --> 00:08:31.937 align:middle line:84% and you believe that the social utility is 00:08:31.937 --> 00:08:33.520 align:middle line:84% a function of the individual utilities 00:08:33.520 --> 00:08:35.353 align:middle line:84% and you think that's what this is capturing, 00:08:35.353 --> 00:08:38.110 align:middle line:84% then this approach might work better. 00:08:38.110 --> 00:08:40.360 align:middle line:84% So the paper that you guys actually read for last time 00:08:40.360 --> 00:08:41.890 align:middle line:84% that I want to spend a fair amount of time-- that's also 00:08:41.890 --> 00:08:43.723 align:middle line:84% going to be talked about in the problem site 00:08:43.723 --> 00:08:46.807 align:middle line:90% is this paper on self targeting. 00:08:46.807 --> 00:08:47.640 align:middle line:90% So what is the idea? 00:08:47.640 --> 00:08:49.450 align:middle line:84% So the idea goes back to this paper 00:08:49.450 --> 00:08:52.940 align:middle line:84% by Nichols and Zeckhauser, which is that-- what they say, 00:08:52.940 --> 00:08:55.900 align:middle line:84% they say or deals can be used to target the poor. 00:08:55.900 --> 00:08:57.850 align:middle line:90% And what do they mean by that? 00:08:57.850 --> 00:08:58.850 align:middle line:90% Well, here's an example. 00:08:58.850 --> 00:09:00.490 align:middle line:84% Suppose you need to wait in a really long line 00:09:00.490 --> 00:09:01.812 align:middle line:90% to get unemployment benefits. 00:09:01.812 --> 00:09:03.520 align:middle line:84% And maybe suppose the unemployment office 00:09:03.520 --> 00:09:08.608 align:middle line:84% is only open from 9:00 to 3:00 in the middle of the workday. 00:09:08.608 --> 00:09:09.650 align:middle line:90% What is that going to do? 00:09:09.650 --> 00:09:12.033 align:middle line:84% That's going to be not very costly-- 00:09:12.033 --> 00:09:14.200 align:middle line:84% I mean it's costly for everyone because no one wants 00:09:14.200 --> 00:09:16.117 align:middle line:84% to stand in line at the unemployment benefits. 00:09:16.117 --> 00:09:19.000 align:middle line:84% But it's differentially costly for unemployed versus employed 00:09:19.000 --> 00:09:21.730 align:middle line:84% people because unemployed people don't really 00:09:21.730 --> 00:09:24.520 align:middle line:84% have that much to do between 9:00 and 3:00 on a workday. 00:09:24.520 --> 00:09:26.020 align:middle line:84% I mean they have other things they'd 00:09:26.020 --> 00:09:27.910 align:middle line:84% rather do, but don't have to go to a job 00:09:27.910 --> 00:09:28.780 align:middle line:90% because they're unemployed. 00:09:28.780 --> 00:09:30.580 align:middle line:84% Whereas employed people would have to miss a day of work 00:09:30.580 --> 00:09:30.940 align:middle line:90% to do that. 00:09:30.940 --> 00:09:32.357 align:middle line:84% So it's going to be differential-- 00:09:32.357 --> 00:09:34.390 align:middle line:84% so the idea is this idea of having you have 00:09:34.390 --> 00:09:37.180 align:middle line:84% to show up every once a week to screen your unemployment 00:09:37.180 --> 00:09:40.010 align:middle line:84% benefits to recertify your unemployment 00:09:40.010 --> 00:09:42.070 align:middle line:84% is going to differentially screen 00:09:42.070 --> 00:09:43.750 align:middle line:84% between people who are really unemployed 00:09:43.750 --> 00:09:45.800 align:middle line:90% and people who are not. 00:09:45.800 --> 00:09:47.770 align:middle line:90% And the-- one second-- 00:09:47.770 --> 00:09:50.560 align:middle line:84% and the downside of that approach 00:09:50.560 --> 00:09:54.700 align:middle line:84% is that it's imposing a cost on the unemployed people, which 00:09:54.700 --> 00:09:57.610 align:middle line:84% is, in principle, you think that cost is just totally 00:09:57.610 --> 00:09:59.230 align:middle line:90% kind of deadweight loss. 00:09:59.230 --> 00:10:01.390 align:middle line:84% Like we don't want to have to have 00:10:01.390 --> 00:10:03.040 align:middle line:84% the unemployed people have to do this. 00:10:03.040 --> 00:10:05.240 align:middle line:84% The only benefit is the screening cost. 00:10:05.240 --> 00:10:07.820 align:middle line:84% So the way to think about this is, on the other hand, 00:10:07.820 --> 00:10:11.920 align:middle line:84% if we can screen out enough non-needy people, 00:10:11.920 --> 00:10:14.410 align:middle line:84% we could take all that savings and give the truly 00:10:14.410 --> 00:10:15.430 align:middle line:90% unemployed kind of more. 00:10:15.430 --> 00:10:18.730 align:middle line:84% So it could be actually beneficial on net 00:10:18.730 --> 00:10:20.637 align:middle line:84% if you can think about it through the fact 00:10:20.637 --> 00:10:22.720 align:middle line:84% that you can kind of increase the transfers you're 00:10:22.720 --> 00:10:23.980 align:middle line:90% giving to the really needy. 00:10:23.980 --> 00:10:25.960 align:middle line:90% But it does impose costs. 00:10:25.960 --> 00:10:26.860 align:middle line:90% Yeah, [INAUDIBLE]. 00:10:26.860 --> 00:10:28.280 align:middle line:84% AUDIENCE: Well, I was saying this isn't quite 00:10:28.280 --> 00:10:29.660 align:middle line:90% targeting the poor, right? 00:10:29.660 --> 00:10:32.120 align:middle line:84% This is only targeting people with low [INAUDIBLE] the cost 00:10:32.120 --> 00:10:34.100 align:middle line:90% of time, which if-- 00:10:34.100 --> 00:10:34.980 align:middle line:90% BEN OLKEN: Correct. 00:10:34.980 --> 00:10:36.860 align:middle line:90% AUDIENCE: So if you were-- 00:10:36.860 --> 00:10:41.850 align:middle line:84% your income level was the same as somebody else's income level 00:10:41.850 --> 00:10:44.402 align:middle line:84% but you didn't have as much wealth as they did, 00:10:44.402 --> 00:10:45.960 align:middle line:90% it could be-- 00:10:45.960 --> 00:10:47.730 align:middle line:84% BEN OLKEN: Correct, yes, yes, yes, yes. 00:10:47.730 --> 00:10:50.700 align:middle line:90% So right. 00:10:50.700 --> 00:10:53.050 align:middle line:90% Hold that thought. 00:10:53.050 --> 00:10:54.937 align:middle line:84% Well, this is-- yes, so this one is 00:10:54.937 --> 00:10:56.270 align:middle line:90% to target unemployment benefits. 00:10:56.270 --> 00:10:57.900 align:middle line:84% So this is explicitly trying to target people who are not. 00:10:57.900 --> 00:11:00.000 align:middle line:84% This one, this example is trying to target 00:11:00.000 --> 00:11:01.480 align:middle line:90% people who are unemployed. 00:11:01.480 --> 00:11:03.370 align:middle line:84% So I think that example works pretty well. 00:11:03.370 --> 00:11:06.068 align:middle line:84% But I think you are absolutely right that the general question 00:11:06.068 --> 00:11:07.860 align:middle line:84% whenever you have these kinds of mechanisms 00:11:07.860 --> 00:11:10.293 align:middle line:84% is, is it targeting the thing you want or not? 00:11:10.293 --> 00:11:12.210 align:middle line:84% And I'm going to talk about a bunch of reasons 00:11:12.210 --> 00:11:14.418 align:middle line:84% why that may or may not work out kind of in practice. 00:11:14.418 --> 00:11:16.620 align:middle line:90% 00:11:16.620 --> 00:11:19.760 align:middle line:90% Other questions? 00:11:19.760 --> 00:11:24.950 align:middle line:84% OK, so here's a basic version of the model. 00:11:24.950 --> 00:11:28.030 align:middle line:84% So imagine that-- now, in this example, 00:11:28.030 --> 00:11:30.940 align:middle line:90% this is now from our paper. 00:11:30.940 --> 00:11:35.410 align:middle line:84% But this is the basic case where you have-- 00:11:35.410 --> 00:11:37.750 align:middle line:90% we have to walk some-- 00:11:37.750 --> 00:11:41.680 align:middle line:84% go some-- travel some distance to apply for benefits. 00:11:41.680 --> 00:11:43.480 align:middle line:84% And the thing we're trying to screen on 00:11:43.480 --> 00:11:45.800 align:middle line:84% is your opportunity cost of time. 00:11:45.800 --> 00:11:50.320 align:middle line:84% So imagine that the cost is going to be kind of-- 00:11:50.320 --> 00:11:51.820 align:middle line:84% so basically people who are richer 00:11:51.820 --> 00:11:53.650 align:middle line:84% imagine they have a higher opportunity cost of time 00:11:53.650 --> 00:11:54.858 align:middle line:90% because their wage is higher. 00:11:54.858 --> 00:11:57.040 align:middle line:84% So imagine this is a single dimension, where 00:11:57.040 --> 00:11:58.180 align:middle line:90% it's your wage. 00:11:58.180 --> 00:12:02.440 align:middle line:84% Right so in that case, if you had a close-- 00:12:02.440 --> 00:12:05.140 align:middle line:84% you don't have to walk very far, the cost 00:12:05.140 --> 00:12:08.650 align:middle line:84% will be downward sloping in consumption. 00:12:08.650 --> 00:12:11.440 align:middle line:84% But they will be more downward sloping in consumption 00:12:11.440 --> 00:12:14.890 align:middle line:84% if I increase the distance because the amount of time 00:12:14.890 --> 00:12:16.690 align:middle line:84% you have to spend walking there or whatever 00:12:16.690 --> 00:12:19.480 align:middle line:84% because that's going to be proportional to your wage. 00:12:19.480 --> 00:12:22.120 align:middle line:84% And so that's imposing a cost on everyone. 00:12:22.120 --> 00:12:23.620 align:middle line:84% So here's the whole thing now but it 00:12:23.620 --> 00:12:28.120 align:middle line:84% should sit differentially for the poor or the rich. 00:12:28.120 --> 00:12:29.620 align:middle line:84% And therefore who's going to show up 00:12:29.620 --> 00:12:31.653 align:middle line:90% is going to shift to the left. 00:12:31.653 --> 00:12:33.070 align:middle line:84% In this example, everyone-- people 00:12:33.070 --> 00:12:36.300 align:middle line:84% will show up if they're above this line. 00:12:36.300 --> 00:12:38.540 align:middle line:84% So increasing the cost in this way 00:12:38.540 --> 00:12:41.060 align:middle line:84% is going to get you better targeting. 00:12:41.060 --> 00:12:47.172 align:middle line:84% Now, in fact, it's more complicated than that 00:12:47.172 --> 00:12:48.130 align:middle line:90% for a bunch of reasons. 00:12:48.130 --> 00:12:49.540 align:middle line:84% I think you mentioned kind of one 00:12:49.540 --> 00:12:52.960 align:middle line:84% of them, which is that maybe the thing you're targeting is-- 00:12:52.960 --> 00:12:55.180 align:middle line:84% what you want to target, say, is maybe wealth. 00:12:55.180 --> 00:12:57.460 align:middle line:84% And what you're actually targeting is kind of wage. 00:12:57.460 --> 00:12:59.630 align:middle line:84% And those may not be perfectly correlated. 00:12:59.630 --> 00:13:00.772 align:middle line:90% So that's one example. 00:13:00.772 --> 00:13:02.230 align:middle line:84% Anyone remember some other examples 00:13:02.230 --> 00:13:05.350 align:middle line:84% we talk about of other reasons these things may or not 00:13:05.350 --> 00:13:07.580 align:middle line:90% be the same? 00:13:07.580 --> 00:13:08.447 align:middle line:90% Yeah. 00:13:08.447 --> 00:13:10.280 align:middle line:84% AUDIENCE: Different costs and getting there, 00:13:10.280 --> 00:13:12.315 align:middle line:90% like having a car. 00:13:12.315 --> 00:13:14.690 align:middle line:84% BEN OLKEN: Right, so it may be that this ordeal was setup 00:13:14.690 --> 00:13:16.373 align:middle line:84% for imagining everyone's walking, 00:13:16.373 --> 00:13:18.290 align:middle line:84% but actually we have different ways of dealing 00:13:18.290 --> 00:13:18.960 align:middle line:90% with the ordeal. 00:13:18.960 --> 00:13:19.878 align:middle line:90% So maybe we can drive. 00:13:19.878 --> 00:13:21.920 align:middle line:84% Maybe if you're really rich, you can hire someone 00:13:21.920 --> 00:13:23.966 align:middle line:90% to wait in line for you. 00:13:23.966 --> 00:13:26.340 align:middle line:84% You can pay an agent to do this thing for you. 00:13:26.340 --> 00:13:28.070 align:middle line:84% And so like actually maybe the rich 00:13:28.070 --> 00:13:31.093 align:middle line:84% have ways of undoing this ordeal. 00:13:31.093 --> 00:13:31.635 align:middle line:90% Other things? 00:13:31.635 --> 00:13:37.690 align:middle line:90% 00:13:37.690 --> 00:13:42.590 align:middle line:84% So the so the one we were really worried about 00:13:42.590 --> 00:13:48.770 align:middle line:84% was this one, actually, which is what 00:13:48.770 --> 00:13:53.720 align:middle line:84% happens if you have really concave utility in a day's 00:13:53.720 --> 00:13:55.130 align:middle line:90% wage, for example. 00:13:55.130 --> 00:13:59.000 align:middle line:84% And this actually-- when we were pilot testing this, 00:13:59.000 --> 00:14:02.150 align:middle line:84% it was not ex ante obvious this was going to work. 00:14:02.150 --> 00:14:03.900 align:middle line:84% It was certainly we thought it might work, 00:14:03.900 --> 00:14:05.420 align:middle line:90% so that's why we did it. 00:14:05.420 --> 00:14:07.320 align:middle line:84% But there was reasons to think it might not. 00:14:07.320 --> 00:14:09.815 align:middle line:84% And the one thing we heard from some qualitative reports 00:14:09.815 --> 00:14:13.935 align:middle line:84% is that some people said, look, I can't do this, 00:14:13.935 --> 00:14:16.060 align:middle line:84% I can't show up for a day, even though like there's 00:14:16.060 --> 00:14:20.050 align:middle line:84% like $1,000 of benefits on the line here, in NPV terms 00:14:20.050 --> 00:14:22.370 align:middle line:84% from my showing up in applying for this program. 00:14:22.370 --> 00:14:24.910 align:middle line:84% I'm going to miss a day's work if I do this. 00:14:24.910 --> 00:14:27.430 align:middle line:90% And that's going to be-- 00:14:27.430 --> 00:14:29.770 align:middle line:84% basically, what I earn that day is what I eat that day. 00:14:29.770 --> 00:14:30.970 align:middle line:84% So like I'm going to basically have 00:14:30.970 --> 00:14:32.178 align:middle line:90% to go without food for a day. 00:14:32.178 --> 00:14:34.477 align:middle line:84% So it's like super costly for me to apply for this. 00:14:34.477 --> 00:14:36.310 align:middle line:84% I mean I could do it, but it's super costly. 00:14:36.310 --> 00:14:40.840 align:middle line:84% Whereas, say, for richer people, they didn't have that. 00:14:40.840 --> 00:14:42.740 align:middle line:84% They could maybe smooth that, for example. 00:14:42.740 --> 00:14:47.140 align:middle line:84% So it may be that even though this previous one was 00:14:47.140 --> 00:14:49.460 align:middle line:90% in terms of income, net income. 00:14:49.460 --> 00:14:52.720 align:middle line:84% But if I start putting concave utility, 00:14:52.720 --> 00:14:55.240 align:middle line:84% if the thing is super duper concave, 00:14:55.240 --> 00:14:58.645 align:middle line:84% then actually the fact that this is shifted down 00:14:58.645 --> 00:15:00.770 align:middle line:84% is actually first order, could be really important, 00:15:00.770 --> 00:15:03.020 align:middle line:84% because actually these people could actually be losing 00:15:03.020 --> 00:15:04.310 align:middle line:90% a lot of utility over here. 00:15:04.310 --> 00:15:06.310 align:middle line:84% And so if I put concave utility, maybe the thing 00:15:06.310 --> 00:15:10.270 align:middle line:84% looks like this, for example, because here we 00:15:10.270 --> 00:15:12.250 align:middle line:84% have-- this side, we have the standard effect 00:15:12.250 --> 00:15:13.510 align:middle line:90% we have had before. 00:15:13.510 --> 00:15:16.060 align:middle line:84% Over here we have the effect that utility is 00:15:16.060 --> 00:15:17.570 align:middle line:90% super duper concave over here. 00:15:17.570 --> 00:15:19.720 align:middle line:84% And these people are just experiencing-- 00:15:19.720 --> 00:15:21.970 align:middle line:84% even though maybe it's a smaller income utility loss, 00:15:21.970 --> 00:15:23.463 align:middle line:84% it's a larger utility income loss. 00:15:23.463 --> 00:15:25.630 align:middle line:84% And so in this example, actually, maybe what happens 00:15:25.630 --> 00:15:32.010 align:middle line:84% is you have the people that apply shift in or whatever. 00:15:32.010 --> 00:15:34.930 align:middle line:90% So then it's not obvious. 00:15:34.930 --> 00:15:37.023 align:middle line:84% And then also, as you guys talked about, 00:15:37.023 --> 00:15:39.190 align:middle line:84% there was example of maybe you actually switch, kind 00:15:39.190 --> 00:15:40.148 align:middle line:90% of switch technologies. 00:15:40.148 --> 00:15:44.470 align:middle line:84% So maybe for example, the rich have different technologies. 00:15:44.470 --> 00:15:50.042 align:middle line:84% And as the cost goes up, they switch the way they get there. 00:15:50.042 --> 00:15:52.000 align:middle line:84% So the point of that was to say that actually-- 00:15:52.000 --> 00:15:53.950 align:middle line:84% the first point I think that we want to say is that, look, 00:15:53.950 --> 00:15:56.480 align:middle line:84% this Nichols Zeckhauser idea is a nice idea in theory, 00:15:56.480 --> 00:15:59.368 align:middle line:84% but it's not ex-ante totally obvious. 00:15:59.368 --> 00:16:01.660 align:middle line:84% And those are things that we wanted to test empirically 00:16:01.660 --> 00:16:02.160 align:middle line:90% in the data. 00:16:02.160 --> 00:16:05.330 align:middle line:90% 00:16:05.330 --> 00:16:07.025 align:middle line:90% The second thing was-- 00:16:07.025 --> 00:16:09.710 align:middle line:90% 00:16:09.710 --> 00:16:15.190 align:middle line:84% and again, this is also part of our learning process 00:16:15.190 --> 00:16:17.740 align:middle line:84% in writing, in doing this project. 00:16:17.740 --> 00:16:20.860 align:middle line:84% When we set out to do this paper, 00:16:20.860 --> 00:16:22.900 align:middle line:84% what we were worried about was this stuff. 00:16:22.900 --> 00:16:25.060 align:middle line:84% Like this is what we had in mind as the reasons 00:16:25.060 --> 00:16:26.680 align:middle line:84% this might not work or the things 00:16:26.680 --> 00:16:28.580 align:middle line:90% that might be really important. 00:16:28.580 --> 00:16:30.337 align:middle line:84% What we realized ex post after writing 00:16:30.337 --> 00:16:31.920 align:middle line:84% the paper is there was another factor, 00:16:31.920 --> 00:16:34.212 align:middle line:84% which was really important, which was the paper ends up 00:16:34.212 --> 00:16:39.557 align:middle line:84% being about, which is that in the context 00:16:39.557 --> 00:16:41.640 align:middle line:84% we're going to study, what you guys read about was 00:16:41.640 --> 00:16:43.415 align:middle line:84% it's not just that you do this thing 00:16:43.415 --> 00:16:44.790 align:middle line:84% and you get the benefit for sure. 00:16:44.790 --> 00:16:46.207 align:middle line:84% What happens is you do this thing. 00:16:46.207 --> 00:16:48.690 align:middle line:84% And then you get some stochastic like-- you get the benefit 00:16:48.690 --> 00:16:50.880 align:middle line:90% with some probability. 00:16:50.880 --> 00:16:52.740 align:middle line:84% And the probability is the proxy means test 00:16:52.740 --> 00:16:55.700 align:middle line:90% that we talked about last time. 00:16:55.700 --> 00:17:00.260 align:middle line:84% And so in the example here, after showing up 00:17:00.260 --> 00:17:04.619 align:middle line:84% and applying for benefits, they still proxy means test you. 00:17:04.619 --> 00:17:06.099 align:middle line:90% And you may or may not get it. 00:17:06.099 --> 00:17:09.518 align:middle line:84% And so in addition to a cost and benefits of applying, 00:17:09.518 --> 00:17:12.060 align:middle line:84% you also have to forecast, like am I likely to get this thing 00:17:12.060 --> 00:17:12.560 align:middle line:90% or not? 00:17:12.560 --> 00:17:15.349 align:middle line:90% 00:17:15.349 --> 00:17:17.970 align:middle line:84% And you're going to take that into account. 00:17:17.970 --> 00:17:20.599 align:middle line:84% And that's going to add another complication wrinkle 00:17:20.599 --> 00:17:23.569 align:middle line:84% to the model, which is that sophisticated households are 00:17:23.569 --> 00:17:24.217 align:middle line:90% going to-- 00:17:24.217 --> 00:17:26.300 align:middle line:84% and there's two different ways have to model this. 00:17:26.300 --> 00:17:29.330 align:middle line:84% Number one is it may be that I fully 00:17:29.330 --> 00:17:31.920 align:middle line:84% understand what the government's going to do, 00:17:31.920 --> 00:17:34.890 align:middle line:84% in which case, actually, there's not a lot of uncertainty. 00:17:34.890 --> 00:17:38.680 align:middle line:84% There may be still a little bit of uncertainty, but not a lot. 00:17:38.680 --> 00:17:40.590 align:middle line:84% And then the other case is, well, maybe I 00:17:40.590 --> 00:17:42.970 align:middle line:84% don't understand what the government's going to do. 00:17:42.970 --> 00:17:46.480 align:middle line:84% And in that case, it's kind of interesting to say, look, 00:17:46.480 --> 00:17:48.340 align:middle line:90% if I'm-- 00:17:48.340 --> 00:17:51.100 align:middle line:84% and it's actually pretty reasonable, 00:17:51.100 --> 00:17:53.850 align:middle line:84% which is you say, look, I know my income. 00:17:53.850 --> 00:17:56.940 align:middle line:84% I have some rough sense of what other people with my income 00:17:56.940 --> 00:17:58.900 align:middle line:84% level is likely to happen to them. 00:17:58.900 --> 00:18:00.660 align:middle line:84% But I don't really understand the details 00:18:00.660 --> 00:18:02.730 align:middle line:84% of what the government is doing with that proxy means test. 00:18:02.730 --> 00:18:04.170 align:middle line:90% And this is pretty plausible. 00:18:04.170 --> 00:18:07.135 align:middle line:84% There's like at least one of you in your comments 00:18:07.135 --> 00:18:08.760 align:middle line:84% was like, well, maybe they should know. 00:18:08.760 --> 00:18:10.740 align:middle line:84% But I think it's at least pretty plausible 00:18:10.740 --> 00:18:13.920 align:middle line:84% that they don't necessarily know exactly what's happening. 00:18:13.920 --> 00:18:16.830 align:middle line:90% And that means that the-- 00:18:16.830 --> 00:18:18.100 align:middle line:90% so if this one's going on. 00:18:18.100 --> 00:18:18.960 align:middle line:84% If this sophisticated one is going on, 00:18:18.960 --> 00:18:19.970 align:middle line:84% that's still helpful for the government 00:18:19.970 --> 00:18:22.470 align:middle line:84% because they have to bother to screen people who 00:18:22.470 --> 00:18:23.850 align:middle line:90% aren't going to get it anyway. 00:18:23.850 --> 00:18:26.070 align:middle line:84% But this one is now really valuable 00:18:26.070 --> 00:18:28.830 align:middle line:84% because I'm making my decision as to whether 00:18:28.830 --> 00:18:33.330 align:middle line:84% or not to apply based on my actual y, my actual income. 00:18:33.330 --> 00:18:36.935 align:middle line:84% But the government-- and that includes the part 00:18:36.935 --> 00:18:38.310 align:middle line:84% my income the government observes 00:18:38.310 --> 00:18:41.070 align:middle line:84% and the part of the income the government doesn't observe. 00:18:41.070 --> 00:18:43.410 align:middle line:84% So I'm making the decision including 00:18:43.410 --> 00:18:46.197 align:middle line:84% my unobservable information, and in so doing, 00:18:46.197 --> 00:18:48.030 align:middle line:84% I'm kind of revealing that information of it 00:18:48.030 --> 00:18:48.947 align:middle line:90% to the government. 00:18:48.947 --> 00:18:51.030 align:middle line:84% And so the government is getting more information, 00:18:51.030 --> 00:18:53.610 align:middle line:84% in some sense, through that decision. 00:18:53.610 --> 00:18:56.113 align:middle line:84% And that's kind of what we think is helpful here. 00:18:56.113 --> 00:18:58.030 align:middle line:84% And so just to show you what this looks like-- 00:18:58.030 --> 00:19:00.730 align:middle line:84% so this is if we plot in the data-- 00:19:00.730 --> 00:19:06.360 align:middle line:84% so in the paper, we know your baseline consumption score. 00:19:06.360 --> 00:19:08.160 align:middle line:84% And in our baseline survey, we also 00:19:08.160 --> 00:19:10.440 align:middle line:84% collected all the variables that are 00:19:10.440 --> 00:19:14.040 align:middle line:84% used by the government for the proxy means test screening. 00:19:14.040 --> 00:19:20.310 align:middle line:84% And so we can replicate the proxy means test screening. 00:19:20.310 --> 00:19:22.360 align:middle line:84% We can compute your proxy means test score 00:19:22.360 --> 00:19:23.610 align:middle line:90% as you would have observed it. 00:19:23.610 --> 00:19:25.318 align:middle line:84% Suppose you knew the government's formula 00:19:25.318 --> 00:19:27.408 align:middle line:90% and knew the truth about you. 00:19:27.408 --> 00:19:29.478 align:middle line:84% We can say, what's your proxy means test score? 00:19:29.478 --> 00:19:31.770 align:middle line:84% And then we can say for those who actually did show up, 00:19:31.770 --> 00:19:33.478 align:middle line:84% did they actually get the benefit or not? 00:19:33.478 --> 00:19:37.303 align:middle line:84% This should be a step function, right? 00:19:37.303 --> 00:19:37.970 align:middle line:90% You would think. 00:19:37.970 --> 00:19:39.605 align:middle line:84% Actually, the thresholds were a little 00:19:39.605 --> 00:19:40.730 align:middle line:90% different in each province. 00:19:40.730 --> 00:19:42.272 align:middle line:84% That's why there are different lines. 00:19:42.272 --> 00:19:44.840 align:middle line:84% So it should be like six step functions there were six-- 00:19:44.840 --> 00:19:46.340 align:middle line:84% sorry, each district, six districts. 00:19:46.340 --> 00:19:48.470 align:middle line:84% There should be six step functions in this graph. 00:19:48.470 --> 00:19:50.600 align:middle line:84% Why are there not six step functions? 00:19:50.600 --> 00:19:52.370 align:middle line:84% Because when the government goes in 00:19:52.370 --> 00:19:54.650 align:middle line:84% measures your four different variables, or maybe 00:19:54.650 --> 00:19:55.520 align:middle line:84% a little bit of measurement error 00:19:55.520 --> 00:19:57.895 align:middle line:84% compared to when we measure the four different variables. 00:19:57.895 --> 00:19:59.330 align:middle line:84% So even this thing is not perfect. 00:19:59.330 --> 00:20:03.320 align:middle line:90% But it's pretty steep. 00:20:03.320 --> 00:20:08.090 align:middle line:84% One second-- on the other hand, if we plot the same thing 00:20:08.090 --> 00:20:12.110 align:middle line:84% against your total income or your total consumption, 00:20:12.110 --> 00:20:15.380 align:middle line:84% it's still downward sloping, but it's a lot flatter. 00:20:15.380 --> 00:20:17.960 align:middle line:84% And that just captures the fact that I don't fully 00:20:17.960 --> 00:20:20.060 align:middle line:90% know, I still-- 00:20:20.060 --> 00:20:21.750 align:middle line:84% if I know-- so this is just saying, 00:20:21.750 --> 00:20:25.370 align:middle line:84% what do I infer about my probability of getting benefits 00:20:25.370 --> 00:20:28.535 align:middle line:84% if I'm, say, down here versus kind of over here or whatever. 00:20:28.535 --> 00:20:30.410 align:middle line:84% And it's still downward sloping, but it's not 00:20:30.410 --> 00:20:33.230 align:middle line:84% quite as flat because the government-- it's based 00:20:33.230 --> 00:20:34.910 align:middle line:90% on this unobservable component. 00:20:34.910 --> 00:20:37.620 align:middle line:84% And observable component, I don't fully forecast that. 00:20:37.620 --> 00:20:38.120 align:middle line:90% I'm sorry. 00:20:38.120 --> 00:20:39.203 align:middle line:90% Someone had their hand up. 00:20:39.203 --> 00:20:40.340 align:middle line:90% Yeah, [INAUDIBLE]. 00:20:40.340 --> 00:20:43.980 align:middle line:84% AUDIENCE: [INAUDIBLE] designing an experiment like this, 00:20:43.980 --> 00:20:46.350 align:middle line:84% how much are you thinking about theory paper 00:20:46.350 --> 00:20:48.270 align:middle line:84% that you referenced at the beginning? 00:20:48.270 --> 00:20:49.260 align:middle line:84% BEN OLKEN: The Nichols and Zeckhauser one? 00:20:49.260 --> 00:20:50.430 align:middle line:84% AUDIENCE: Yeah, how much are you just thinking 00:20:50.430 --> 00:20:51.900 align:middle line:90% about the local context? 00:20:51.900 --> 00:20:52.906 align:middle line:84% BEN OLKEN: We were totally thinking of Nichols 00:20:52.906 --> 00:20:54.210 align:middle line:90% and Zeckhauser paper. 00:20:54.210 --> 00:20:55.650 align:middle line:90% So and a local context-- 00:20:55.650 --> 00:20:57.000 align:middle line:90% I mean, both. 00:20:57.000 --> 00:21:02.970 align:middle line:84% I think that-- we were thinking about the Nichols 00:21:02.970 --> 00:21:04.610 align:middle line:90% and Zeckhauser paper. 00:21:04.610 --> 00:21:09.190 align:middle line:84% And that's why we intentionally not only-- 00:21:09.190 --> 00:21:11.140 align:middle line:90% So stepping way back-- 00:21:11.140 --> 00:21:14.300 align:middle line:90% 00:21:14.300 --> 00:21:15.350 align:middle line:90% so what were we doing? 00:21:15.350 --> 00:21:18.580 align:middle line:90% So stepping way, way back-- 00:21:18.580 --> 00:21:20.820 align:middle line:90% we did the first paper. 00:21:20.820 --> 00:21:23.032 align:middle line:84% And we presented the results to the government. 00:21:23.032 --> 00:21:24.740 align:middle line:84% And they said, that's pretty interesting. 00:21:24.740 --> 00:21:26.800 align:middle line:84% It looks like this improving targeting thing 00:21:26.800 --> 00:21:28.330 align:middle line:90% has some potential. 00:21:28.330 --> 00:21:31.210 align:middle line:84% But you did that one before in kind of a low stakes 00:21:31.210 --> 00:21:32.140 align:middle line:90% environment. 00:21:32.140 --> 00:21:34.600 align:middle line:84% We'd like you to try some stuff in a real environment 00:21:34.600 --> 00:21:36.400 align:middle line:90% now and see what actually works. 00:21:36.400 --> 00:21:38.745 align:middle line:84% So that whole-- so first we did little pilots. 00:21:38.745 --> 00:21:40.870 align:middle line:84% Then we did it in kind of a low stakes environment. 00:21:40.870 --> 00:21:42.700 align:middle line:84% Then they were sufficiently intrigued. 00:21:42.700 --> 00:21:44.440 align:middle line:84% They said let's do this in a high stakes environment. 00:21:44.440 --> 00:21:45.815 align:middle line:84% And particularly, they said we're 00:21:45.815 --> 00:21:47.470 align:middle line:84% expanding the conditional cash transfer 00:21:47.470 --> 00:21:50.110 align:middle line:84% program nationwide, or not nationwide, 00:21:50.110 --> 00:21:51.460 align:middle line:90% to a bunch of new areas. 00:21:51.460 --> 00:21:54.727 align:middle line:84% They said there's 600 villages which haven't 00:21:54.727 --> 00:21:55.810 align:middle line:90% gotten the program before. 00:21:55.810 --> 00:21:58.120 align:middle line:84% And we'd like you to test in those 600 villages 00:21:58.120 --> 00:22:00.070 align:middle line:84% three different approaches to doing the targeting to see 00:22:00.070 --> 00:22:02.170 align:middle line:84% which one is going to work best to inform our future decision 00:22:02.170 --> 00:22:02.810 align:middle line:90% making. 00:22:02.810 --> 00:22:05.830 align:middle line:84% So that's why we were able to do this for a high stakes-- 00:22:05.830 --> 00:22:08.380 align:middle line:84% like for real programs because the government 00:22:08.380 --> 00:22:10.563 align:middle line:84% wanted to understand what's going to happen. 00:22:10.563 --> 00:22:11.980 align:middle line:84% So actually, there's a third piece 00:22:11.980 --> 00:22:13.360 align:middle line:84% of this, which I don't think we analyzed here, 00:22:13.360 --> 00:22:15.280 align:middle line:84% but we wrote up in a report, which we actually 00:22:15.280 --> 00:22:17.728 align:middle line:84% replicate the version of the community treatment in 200 00:22:17.728 --> 00:22:19.270 align:middle line:84% other villages and show that it looks 00:22:19.270 --> 00:22:21.280 align:middle line:84% pretty similar, not exactly the same, but pretty similar. 00:22:21.280 --> 00:22:24.082 align:middle line:84% That's not on this paper because we didn't-- that wasn't how we 00:22:24.082 --> 00:22:24.790 align:middle line:90% wrote this paper. 00:22:24.790 --> 00:22:26.380 align:middle line:90% But like we did-- 00:22:26.380 --> 00:22:27.910 align:middle line:84% actually, there were 600 villages. 00:22:27.910 --> 00:22:30.413 align:middle line:84% 200 of them go to replicating the community thing. 00:22:30.413 --> 00:22:32.580 align:middle line:84% The other thing we said to the government was, look, 00:22:32.580 --> 00:22:35.040 align:middle line:84% there's this idea that maybe on-demand application-- 00:22:35.040 --> 00:22:37.415 align:middle line:84% that's kind of what they call this in the policy sphere-- 00:22:37.415 --> 00:22:43.440 align:middle line:84% can help can help you both reduce inclusion error 00:22:43.440 --> 00:22:44.400 align:middle line:90% and exclusion error. 00:22:44.400 --> 00:22:48.030 align:middle line:84% Well, I can it reduce inclusion error, 00:22:48.030 --> 00:22:49.260 align:middle line:90% falsely including the rich. 00:22:49.260 --> 00:22:51.090 align:middle line:84% It's exactly for the self-screening and 00:22:51.090 --> 00:22:52.120 align:middle line:90% self-selection argument. 00:22:52.120 --> 00:22:54.780 align:middle line:90% So that we had in mind. 00:22:54.780 --> 00:22:57.400 align:middle line:84% Why could it potentially reduce exclusion error? 00:22:57.400 --> 00:23:00.150 align:middle line:84% That's because, for a totally-- that's a local context 00:23:00.150 --> 00:23:02.190 align:middle line:84% reason, which is that basically-- but I think 00:23:02.190 --> 00:23:04.650 align:middle line:84% it's actually generically actually quite true, which is 00:23:04.650 --> 00:23:06.750 align:middle line:84% you try to go door to door and find everybody, 00:23:06.750 --> 00:23:08.040 align:middle line:90% but you miss some people. 00:23:08.040 --> 00:23:11.340 align:middle line:84% And in particular, poor people, because they 00:23:11.340 --> 00:23:14.070 align:middle line:84% live at the margin-- often live at the margins of society 00:23:14.070 --> 00:23:16.070 align:middle line:84% and the fringes of the village, or maybe they're 00:23:16.070 --> 00:23:18.610 align:middle line:84% not on everyone's radar screen, you may miss them. 00:23:18.610 --> 00:23:23.940 align:middle line:84% And so opening up an on-demand application process 00:23:23.940 --> 00:23:25.980 align:middle line:84% allows people to make themselves known. 00:23:25.980 --> 00:23:28.710 align:middle line:84% So that piece was kind of a real world piece. 00:23:28.710 --> 00:23:30.570 align:middle line:84% So we had both a theory in mind and the kind 00:23:30.570 --> 00:23:31.840 align:middle line:90% of real world piece in mind. 00:23:31.840 --> 00:23:33.960 align:middle line:84% But we also said, look, there are 00:23:33.960 --> 00:23:35.790 align:middle line:84% reasons to test this before you scale it up 00:23:35.790 --> 00:23:38.220 align:middle line:84% nationally because, for the reasons I was talking about, 00:23:38.220 --> 00:23:40.380 align:middle line:84% maybe the poor aren't going to be able to do this. 00:23:40.380 --> 00:23:41.713 align:middle line:90% Maybe they're going be too busy. 00:23:41.713 --> 00:23:44.360 align:middle line:84% So that's why we want to test it and see if it's going to work. 00:23:44.360 --> 00:23:47.270 align:middle line:84% In terms of the theory, I think that we 00:23:47.270 --> 00:23:48.920 align:middle line:84% had this close versus far treatment, 00:23:48.920 --> 00:23:52.250 align:middle line:84% where we explicitly varied the extent, the distance you 00:23:52.250 --> 00:23:53.300 align:middle line:90% had to travel. 00:23:53.300 --> 00:23:55.880 align:middle line:84% And that was explicitly based on this theoretical idea 00:23:55.880 --> 00:23:57.950 align:middle line:90% that longer-- 00:23:57.950 --> 00:23:59.982 align:middle line:84% that making the ordeal marginally more difficult 00:23:59.982 --> 00:24:01.190 align:middle line:90% should improve the screening. 00:24:01.190 --> 00:24:02.607 align:middle line:84% And that's what we wanted to test. 00:24:02.607 --> 00:24:05.180 align:middle line:84% So that close versus far-- so on-demand application 00:24:05.180 --> 00:24:09.020 align:middle line:84% was kind of a thing in general that they were interested in. 00:24:09.020 --> 00:24:10.610 align:middle line:84% But this close versus far treatment 00:24:10.610 --> 00:24:12.620 align:middle line:84% was explicitly designed from the theory. 00:24:12.620 --> 00:24:18.720 align:middle line:84% Now, we did not have the full model written down here. 00:24:18.720 --> 00:24:20.280 align:middle line:90% And that came after. 00:24:20.280 --> 00:24:23.220 align:middle line:84% So I mean part of it came in parallel. 00:24:23.220 --> 00:24:32.210 align:middle line:84% But-- and it was-- let me actually defer 00:24:32.210 --> 00:24:34.710 align:middle line:84% a little bit the model part to when I get to the model part. 00:24:34.710 --> 00:24:36.168 align:middle line:84% I want to talk about that in a sec. 00:24:36.168 --> 00:24:38.600 align:middle line:84% But does that give a little bit of background? 00:24:38.600 --> 00:24:40.870 align:middle line:84% And but I do think, to the extent, 00:24:40.870 --> 00:24:42.820 align:middle line:84% you can have a theory in mind as you're 00:24:42.820 --> 00:24:44.950 align:middle line:84% designing your experiments, all the better 00:24:44.950 --> 00:24:47.830 align:middle line:84% because there are things you will do that 00:24:47.830 --> 00:24:56.070 align:middle line:90% are going to be more precise. 00:24:56.070 --> 00:24:58.480 align:middle line:84% You do stuff that's more precise. 00:24:58.480 --> 00:25:02.200 align:middle line:84% And in particular, actually one of the empirical challenges-- 00:25:02.200 --> 00:25:04.450 align:middle line:84% it's actually came up related to one of your comments, 00:25:04.450 --> 00:25:05.242 align:middle line:90% I forget which one. 00:25:05.242 --> 00:25:10.950 align:middle line:84% But one of the things that actually we found, 00:25:10.950 --> 00:25:13.430 align:middle line:84% where we had a theoretical idea in mind, which is, 00:25:13.430 --> 00:25:15.180 align:middle line:84% look, we want to actually have these tests 00:25:15.180 --> 00:25:17.870 align:middle line:84% whether these ordeals are improving screening. 00:25:17.870 --> 00:25:19.700 align:middle line:84% It turns out that actually imposing 00:25:19.700 --> 00:25:22.190 align:middle line:84% ordeals on people for no good reason 00:25:22.190 --> 00:25:24.440 align:middle line:84% other than screening is something that I think just 00:25:24.440 --> 00:25:26.640 align:middle line:90% is contrary to human nature. 00:25:26.640 --> 00:25:29.780 align:middle line:90% And so we had this problem-- 00:25:29.780 --> 00:25:31.370 align:middle line:84% we had this situation which happened 00:25:31.370 --> 00:25:34.730 align:middle line:84% in the real world of like all kinds of spontaneous organizing 00:25:34.730 --> 00:25:37.350 align:middle line:84% in the field to make people's ordeal easier. 00:25:37.350 --> 00:25:40.020 align:middle line:84% So for example, people would be like, oh, you're in line. 00:25:40.020 --> 00:25:41.250 align:middle line:90% You're number 74 in line. 00:25:41.250 --> 00:25:42.667 align:middle line:84% Here's a little chip that's at 74. 00:25:42.667 --> 00:25:47.930 align:middle line:84% You can just come back later when your turn is in line. 00:25:47.930 --> 00:25:50.700 align:middle line:84% Or oh, there's so many people in line. 00:25:50.700 --> 00:25:52.152 align:middle line:84% The line is so long, why don't we 00:25:52.152 --> 00:25:54.110 align:middle line:84% like divide this village into four sub villages 00:25:54.110 --> 00:25:55.550 align:middle line:84% and have neighborhood one come in this day, neighborhood 00:25:55.550 --> 00:25:57.592 align:middle line:84% two come in this day, number three come this day, 00:25:57.592 --> 00:25:59.578 align:middle line:84% and neighborhood four come this day, to make 00:25:59.578 --> 00:26:00.870 align:middle line:90% it like less costly for people. 00:26:00.870 --> 00:26:02.630 align:middle line:84% So I think that's actually-- it's 00:26:02.630 --> 00:26:05.630 align:middle line:84% good in the sense of it's a great impulse 00:26:05.630 --> 00:26:09.210 align:middle line:84% that people are trying to make everyone's lives easier. 00:26:09.210 --> 00:26:11.420 align:middle line:84% It does make it harder to test some 00:26:11.420 --> 00:26:12.990 align:middle line:90% of these theoretical ideas. 00:26:12.990 --> 00:26:14.640 align:middle line:84% AUDIENCE: So you have that in mind. 00:26:14.640 --> 00:26:15.080 align:middle line:90% BEN OLKEN: Exactly. 00:26:15.080 --> 00:26:16.788 align:middle line:84% So the reason is-- the reason I mentioned 00:26:16.788 --> 00:26:19.070 align:middle line:84% that is because this was the thing we were trying 00:26:19.070 --> 00:26:22.190 align:middle line:84% to understand, it was important to monitor these issues 00:26:22.190 --> 00:26:24.170 align:middle line:90% and keep track of them. 00:26:24.170 --> 00:26:29.330 align:middle line:84% And as I said, the point I want to underscore here 00:26:29.330 --> 00:26:32.930 align:middle line:84% is that the goal here is that even though you're 00:26:32.930 --> 00:26:37.780 align:middle line:84% imposing costs on people, the goal is 00:26:37.780 --> 00:26:42.970 align:middle line:84% if you can differentially screen out enough of the non-poor, 00:26:42.970 --> 00:26:46.780 align:middle line:84% you can gain sufficiently large amounts in transfers 00:26:46.780 --> 00:26:49.540 align:middle line:84% that you can actually deliver much more benefits 00:26:49.540 --> 00:26:51.280 align:middle line:90% ex-post to the poor. 00:26:51.280 --> 00:26:54.190 align:middle line:84% So even though you're imposing these costs at the screening 00:26:54.190 --> 00:26:57.010 align:middle line:84% side, it can be wildly beneficial 00:26:57.010 --> 00:27:00.460 align:middle line:84% for the poor potentially because the costs 00:27:00.460 --> 00:27:02.650 align:middle line:90% relative to the program are-- 00:27:02.650 --> 00:27:06.337 align:middle line:84% the costs are, say, I don't know, 00:27:06.337 --> 00:27:07.920 align:middle line:84% half a day's wage just to stand there. 00:27:07.920 --> 00:27:09.520 align:middle line:84% And you can get $1,000 in the program. 00:27:09.520 --> 00:27:10.710 align:middle line:90% So if we can-- 00:27:10.710 --> 00:27:13.060 align:middle line:90% over $150 a year for six years. 00:27:13.060 --> 00:27:16.980 align:middle line:84% So if you can screen out a bunch of rich people and save all 00:27:16.980 --> 00:27:20.700 align:middle line:84% that money, there's a huge-- you could easily imagine this could 00:27:20.700 --> 00:27:21.510 align:middle line:90% be way-- 00:27:21.510 --> 00:27:23.760 align:middle line:84% and use that money to increase the benefit size. 00:27:23.760 --> 00:27:26.940 align:middle line:84% You could make this a substantially better program 00:27:26.940 --> 00:27:29.140 align:middle line:84% for the poor, even with these costs. 00:27:29.140 --> 00:27:31.050 align:middle line:84% So it's important to keep in mind. 00:27:31.050 --> 00:27:34.153 align:middle line:84% That's the rationale here, is imposing these costs 00:27:34.153 --> 00:27:36.570 align:middle line:84% to improve the screening, which can actually make programs 00:27:36.570 --> 00:27:40.120 align:middle line:90% kind of better for everybody. 00:27:40.120 --> 00:27:43.010 align:middle line:90% Other questions? 00:27:43.010 --> 00:27:48.545 align:middle line:90% OK, so how does the model work? 00:27:48.545 --> 00:27:52.910 align:middle line:90% 00:27:52.910 --> 00:27:58.640 align:middle line:84% So let me say-- yeah let me say one other thing actually 00:27:58.640 --> 00:28:00.890 align:middle line:84% before I get into the details of the model. 00:28:00.890 --> 00:28:02.800 align:middle line:84% This is an example where I think we 00:28:02.800 --> 00:28:05.060 align:middle line:90% learned a lot from the model. 00:28:05.060 --> 00:28:09.877 align:middle line:84% And in particular, it's kind of could be a fun exercise. 00:28:09.877 --> 00:28:11.210 align:middle line:90% I don't know if I still have it. 00:28:11.210 --> 00:28:15.400 align:middle line:84% If the very first draft of we wrote a wrote of this paper 00:28:15.400 --> 00:28:19.240 align:middle line:84% was not like this, did not highlight this whole 00:28:19.240 --> 00:28:22.840 align:middle line:84% like forecasting my lambda thing. 00:28:22.840 --> 00:28:26.280 align:middle line:84% We thought this paper was much more about the Nichols 00:28:26.280 --> 00:28:28.660 align:middle line:84% and Zeckhauser basic kind of screening and opportunity 00:28:28.660 --> 00:28:30.100 align:middle line:90% costs. 00:28:30.100 --> 00:28:34.302 align:middle line:84% And we looked at the fact, they said, oh, look, poor versus-- 00:28:34.302 --> 00:28:36.010 align:middle line:84% the close versus the far, seems like it's 00:28:36.010 --> 00:28:37.150 align:middle line:84% doing a little bit of screening, seems 00:28:37.150 --> 00:28:38.360 align:middle line:90% like everything's going on. 00:28:38.360 --> 00:28:41.260 align:middle line:84% And we actually thought this paper was about that. 00:28:41.260 --> 00:28:43.210 align:middle line:84% And it wasn't until we actually wrote down 00:28:43.210 --> 00:28:44.870 align:middle line:84% and tried to calibrate a model of this 00:28:44.870 --> 00:28:46.870 align:middle line:84% that we realized that those things, those forces 00:28:46.870 --> 00:28:50.560 align:middle line:84% were just quantitatively too small to be explaining 00:28:50.560 --> 00:28:53.240 align:middle line:84% the kind of magnitudes of things that we were getting. 00:28:53.240 --> 00:28:55.300 align:middle line:84% And this other force seemed to be really 00:28:55.300 --> 00:28:56.480 align:middle line:90% kind of driving things. 00:28:56.480 --> 00:29:00.708 align:middle line:84% So if you compare kind of our very first draft of slides 00:29:00.708 --> 00:29:03.250 align:middle line:84% on this paper, even once we saw the results itself, targeting 00:29:03.250 --> 00:29:09.070 align:middle line:84% kind of improved targeting by a lot, we had a simple theory. 00:29:09.070 --> 00:29:11.530 align:middle line:90% We explained why it could be. 00:29:11.530 --> 00:29:14.157 align:middle line:84% And it was based on this Nichols and Zeckhauser idea. 00:29:14.157 --> 00:29:16.240 align:middle line:84% We had results saying, look, self targeting works. 00:29:16.240 --> 00:29:18.245 align:middle line:90% And it's doing this thing. 00:29:18.245 --> 00:29:20.620 align:middle line:84% But until we actually wrote down and estimated the model, 00:29:20.620 --> 00:29:22.810 align:middle line:84% we didn't realize that those effects that we were talking 00:29:22.810 --> 00:29:24.040 align:middle line:84% about, where they were qualitatively 00:29:24.040 --> 00:29:25.990 align:middle line:84% in the right direction, weren't quantitatively large enough. 00:29:25.990 --> 00:29:27.910 align:middle line:84% So it was an example where I think we actually 00:29:27.910 --> 00:29:31.060 align:middle line:84% learned a lot and changed kind of our views 00:29:31.060 --> 00:29:34.722 align:middle line:84% and what the paper really meant by going through this exercise 00:29:34.722 --> 00:29:36.430 align:middle line:84% and trying to quantify some of the stuff. 00:29:36.430 --> 00:29:40.390 align:middle line:84% So I just wanted to mention that as well. 00:29:40.390 --> 00:29:41.720 align:middle line:90% So what's the model? 00:29:41.720 --> 00:29:45.280 align:middle line:84% So in each period, households get a utility 00:29:45.280 --> 00:29:46.537 align:middle line:90% from consumption x. 00:29:46.537 --> 00:29:48.745 align:middle line:84% Preferences are additively separable in their present 00:29:48.745 --> 00:29:50.110 align:middle line:90% and future utility-- 00:29:50.110 --> 00:29:52.360 align:middle line:90% discount factor delta. 00:29:52.360 --> 00:29:54.400 align:middle line:84% Their flow income is y in each period. 00:29:54.400 --> 00:29:57.070 align:middle line:90% There's no savings. 00:29:57.070 --> 00:29:59.600 align:middle line:84% y0 is the share observed by the government. 00:29:59.600 --> 00:30:01.153 align:middle line:90% So we're going to calculate-- 00:30:01.153 --> 00:30:03.070 align:middle line:84% y0, we can calculate that as before because we 00:30:03.070 --> 00:30:04.370 align:middle line:90% know the proxy means test. 00:30:04.370 --> 00:30:07.600 align:middle line:84% So y is your consumption. y0 is a proxy means test 00:30:07.600 --> 00:30:08.800 align:middle line:90% predicted consumption. 00:30:08.800 --> 00:30:13.873 align:middle line:84% And yu, the unobservable part, is y minus y0. 00:30:13.873 --> 00:30:15.790 align:middle line:84% So households decide whether to sign up or not 00:30:15.790 --> 00:30:18.310 align:middle line:84% by balancing costs versus expected discount of benefits. 00:30:18.310 --> 00:30:21.760 align:middle line:84% So the cost of signing up is c of l y, where 00:30:21.760 --> 00:30:23.980 align:middle line:84% l is the distance to the place you have to sign up 00:30:23.980 --> 00:30:26.015 align:middle line:90% and y is your income. 00:30:26.015 --> 00:30:30.310 align:middle line:84% But we're going to model that as proportional to your wage 00:30:30.310 --> 00:30:32.880 align:middle line:90% and maybe how you get there. 00:30:32.880 --> 00:30:34.800 align:middle line:84% For sophisticated households, you're 00:30:34.800 --> 00:30:39.570 align:middle line:84% going to say I get benefit B with probability mu of y0, 00:30:39.570 --> 00:30:41.780 align:middle line:90% where this is mu. 00:30:41.780 --> 00:30:44.130 align:middle line:84% So we're going to give them correct beliefs. 00:30:44.130 --> 00:30:46.018 align:middle line:84% And for unsophisticated households, 00:30:46.018 --> 00:30:48.310 align:middle line:84% when they up, they get it with probability lambda of y, 00:30:48.310 --> 00:30:50.580 align:middle line:90% which is this thing over here. 00:30:50.580 --> 00:30:52.775 align:middle line:84% So the mu people-- the sophisticated people 00:30:52.775 --> 00:30:53.650 align:middle line:90% know the steep curve. 00:30:53.650 --> 00:30:57.620 align:middle line:84% And the mu ones only know-- the [INAUDIBLE] the other one. 00:30:57.620 --> 00:31:03.870 align:middle line:84% And there are some utility shock measure, which is f of epsilon. 00:31:03.870 --> 00:31:06.530 align:middle line:84% So then what's my expected benefit from showing up? 00:31:06.530 --> 00:31:09.740 align:middle line:84% Well, for the sophisticated ones, 00:31:09.740 --> 00:31:12.350 align:middle line:84% it's just the cost of showing up plus the expected 00:31:12.350 --> 00:31:15.560 align:middle line:84% benefits of getting the program using kind 00:31:15.560 --> 00:31:19.247 align:middle line:90% of the mu one, the true one. 00:31:19.247 --> 00:31:21.080 align:middle line:84% And for the naive ones, it's the same thing, 00:31:21.080 --> 00:31:25.180 align:middle line:84% but using the lambda thing there. 00:31:25.180 --> 00:31:27.250 align:middle line:84% And we'll imagine that share alpha of households 00:31:27.250 --> 00:31:28.000 align:middle line:90% are sophisticated. 00:31:28.000 --> 00:31:29.860 align:middle line:84% 1 minus alpha are unsophisticated. 00:31:29.860 --> 00:31:31.480 align:middle line:84% And the final thing, which is just 00:31:31.480 --> 00:31:34.930 align:middle line:84% a little technical detail but in the paper is this lambda-- 00:31:34.930 --> 00:31:40.630 align:middle line:84% this mu thing is just a raw function of the underlying 00:31:40.630 --> 00:31:44.530 align:middle line:90% government's technology. 00:31:44.530 --> 00:31:48.430 align:middle line:84% Lambda is actually an equilibrium phenomenon 00:31:48.430 --> 00:31:51.710 align:middle line:84% because it depends on who's actually showing up in person. 00:31:51.710 --> 00:31:56.440 align:middle line:84% So there's a condition-- there's some kind of I don't integral-- 00:31:56.440 --> 00:31:57.670 align:middle line:90% I think it's equation 8-- 00:31:57.670 --> 00:32:00.220 align:middle line:84% there's some integral in the paper which basically states 00:32:00.220 --> 00:32:02.878 align:middle line:84% that thousands of rational beliefs in equilibrium just 00:32:02.878 --> 00:32:03.670 align:middle line:90% to close the model. 00:32:03.670 --> 00:32:08.150 align:middle line:90% 00:32:08.150 --> 00:32:08.760 align:middle line:90% So what was-- 00:32:08.760 --> 00:32:09.815 align:middle line:90% OK, so that's the model. 00:32:09.815 --> 00:32:10.940 align:middle line:90% So what was the experiment? 00:32:10.940 --> 00:32:13.200 align:middle line:84% So we had 400 villages who did this. 00:32:13.200 --> 00:32:15.310 align:middle line:84% This program was just-- this is a constraint 00:32:15.310 --> 00:32:16.310 align:middle line:90% that we're working with. 00:32:16.310 --> 00:32:18.740 align:middle line:84% This is a program targeted to really the very, very poor. 00:32:18.740 --> 00:32:21.310 align:middle line:84% So this is the government is aiming for the bottom 10% 00:32:21.310 --> 00:32:24.028 align:middle line:90% of the population in terms of-- 00:32:24.028 --> 00:32:26.320 align:middle line:84% And so in the villages were randomized into proxy means 00:32:26.320 --> 00:32:27.670 align:middle line:90% test are self targeting. 00:32:27.670 --> 00:32:29.800 align:middle line:84% And the self targeting is you had to go to the self target-- 00:32:29.800 --> 00:32:32.425 align:middle line:84% the way self targeting works is you went to the central meeting 00:32:32.425 --> 00:32:34.120 align:middle line:84% place and you said I want to apply. 00:32:34.120 --> 00:32:35.230 align:middle line:90% You to wait in some line. 00:32:35.230 --> 00:32:38.800 align:middle line:84% And then the government numerators kind of 00:32:38.800 --> 00:32:41.780 align:middle line:84% took all your information, put it into a computer. 00:32:41.780 --> 00:32:48.570 align:middle line:84% And then if-- the way it worked was if your answers say 00:32:48.570 --> 00:32:49.980 align:middle line:90% you're out, then you're out. 00:32:49.980 --> 00:32:53.340 align:middle line:84% If your answers say you're in and the government has data 00:32:53.340 --> 00:32:56.957 align:middle line:84% before which also says you're-- from its previous targeted 00:32:56.957 --> 00:32:59.040 align:middle line:84% survey that also says you're in and not even close 00:32:59.040 --> 00:33:01.320 align:middle line:84% to the threshold, then you're in. 00:33:01.320 --> 00:33:04.583 align:middle line:84% If you're at all close or if the government doesn't data on you, 00:33:04.583 --> 00:33:06.750 align:middle line:84% they actually do a field visit to verify that you're 00:33:06.750 --> 00:33:10.035 align:middle line:90% just not making stuff up. 00:33:10.035 --> 00:33:12.410 align:middle line:84% And I mentioned we varied the distance to the application 00:33:12.410 --> 00:33:12.910 align:middle line:90% site. 00:33:12.910 --> 00:33:15.363 align:middle line:90% 00:33:15.363 --> 00:33:17.780 align:middle line:84% There was also a treatment where we varied the opportunity 00:33:17.780 --> 00:33:19.520 align:middle line:84% cost of applying by actually requiring 00:33:19.520 --> 00:33:27.530 align:middle line:84% that both spouses show up at the same time as a way of varying 00:33:27.530 --> 00:33:29.060 align:middle line:90% the opportunity costs. 00:33:29.060 --> 00:33:32.750 align:middle line:84% That one, if I remember correctly, 00:33:32.750 --> 00:33:36.262 align:middle line:84% we had to put in exceptions in case, because reasonably, 00:33:36.262 --> 00:33:38.720 align:middle line:84% like what would happen if your spouse was kind of outside-- 00:33:38.720 --> 00:33:40.520 align:middle line:84% this is actually what we learned in the pilot process. 00:33:40.520 --> 00:33:42.660 align:middle line:84% People say, well, what happens if my spouse is out 00:33:42.660 --> 00:33:43.160 align:middle line:90% of the city? 00:33:43.160 --> 00:33:45.290 align:middle line:84% Like I shouldn't be denied benefits of the program. 00:33:45.290 --> 00:33:47.270 align:middle line:90% And of course that's fair. 00:33:47.270 --> 00:33:49.310 align:middle line:84% So there had to be an exception process 00:33:49.310 --> 00:33:53.698 align:middle line:84% by saying that the village head or whatever, the neighborhood 00:33:53.698 --> 00:33:55.490 align:middle line:84% head could sign a letter saying your spouse 00:33:55.490 --> 00:33:58.670 align:middle line:84% is out working somewhere else and can't come. 00:33:58.670 --> 00:34:02.270 align:middle line:84% And I think that one actually didn't do very much in practice 00:34:02.270 --> 00:34:04.595 align:middle line:84% because basically anyone whose spouse couldn't come 00:34:04.595 --> 00:34:05.720 align:middle line:90% could just get this letter. 00:34:05.720 --> 00:34:06.680 align:middle line:84% So I don't think it actually made 00:34:06.680 --> 00:34:08.570 align:middle line:84% that much of a difference in terms of the actual opportunity 00:34:08.570 --> 00:34:09.179 align:middle line:90% cost. 00:34:09.179 --> 00:34:11.750 align:middle line:84% So that's why we don't analyze that one as much in the paper. 00:34:11.750 --> 00:34:14.748 align:middle line:90% 00:34:14.748 --> 00:34:16.540 align:middle line:84% So we're going to investigate who signs up, 00:34:16.540 --> 00:34:18.760 align:middle line:84% compare them to the experimentally the PMT, 00:34:18.760 --> 00:34:20.860 align:middle line:84% and estimate the model structurally to tease out 00:34:20.860 --> 00:34:22.580 align:middle line:90% these mechanisms. 00:34:22.580 --> 00:34:26.710 align:middle line:84% So this is the probability of showing up 00:34:26.710 --> 00:34:30.739 align:middle line:84% as a function of your per capita consumption. 00:34:30.739 --> 00:34:33.530 align:middle line:84% So as an aside, in the paper, we talk about Fan regressions 00:34:33.530 --> 00:34:34.780 align:middle line:90% or non-parametric regressions. 00:34:34.780 --> 00:34:37.800 align:middle line:90% Have anyone seen these before? 00:34:37.800 --> 00:34:38.920 align:middle line:90% Anyone have any ideas? 00:34:38.920 --> 00:34:41.037 align:middle line:84% The first point is to clear up a confusion 00:34:41.037 --> 00:34:42.120 align:middle line:90% from when I was a student. 00:34:42.120 --> 00:34:45.429 align:middle line:84% These are not fans like this way, like a fan like you wave. 00:34:45.429 --> 00:34:47.820 align:middle line:90% This is like Mr. Fan. 00:34:47.820 --> 00:34:54.659 align:middle line:84% So just clear that up, clear up years of confusion on my part. 00:34:54.659 --> 00:34:56.530 align:middle line:90% Second of all, what is this? 00:34:56.530 --> 00:35:01.300 align:middle line:84% So these are basically local regressions. 00:35:01.300 --> 00:35:04.800 align:middle line:84% So what you basically do is suppose 00:35:04.800 --> 00:35:06.870 align:middle line:84% you have some x variable over here 00:35:06.870 --> 00:35:09.255 align:middle line:84% and you have some like dot cloud of outcomes. 00:35:09.255 --> 00:35:14.103 align:middle line:90% 00:35:14.103 --> 00:35:15.770 align:middle line:84% And if you put a line through this thing 00:35:15.770 --> 00:35:16.770 align:middle line:90% it's going to look like that. 00:35:16.770 --> 00:35:17.940 align:middle line:84% And that's not really what you want to capture. 00:35:17.940 --> 00:35:19.580 align:middle line:84% So you want to summarize this thing 00:35:19.580 --> 00:35:21.420 align:middle line:90% in some non-parametric way. 00:35:21.420 --> 00:35:26.120 align:middle line:84% So what these regressions do is they say for every point, 00:35:26.120 --> 00:35:29.930 align:middle line:84% let's divide our x-axis into say 100 equal bins, 00:35:29.930 --> 00:35:32.105 align:middle line:84% let's just say, whatever, some number of bins. 00:35:32.105 --> 00:35:35.820 align:middle line:90% 00:35:35.820 --> 00:35:39.220 align:middle line:84% And for each bin, we're going to start over here. 00:35:39.220 --> 00:35:40.810 align:middle line:84% And we're going to run a regression 00:35:40.810 --> 00:35:43.720 align:middle line:84% where each point is weighted based 00:35:43.720 --> 00:35:45.550 align:middle line:90% on the distance from this bin. 00:35:45.550 --> 00:35:47.170 align:middle line:90% And there are different weights. 00:35:47.170 --> 00:35:49.598 align:middle line:84% The weight you use are called kernels. 00:35:49.598 --> 00:35:50.890 align:middle line:90% So there are different kernels. 00:35:50.890 --> 00:35:57.150 align:middle line:84% So a simple one is a triangular kernel, which 00:35:57.150 --> 00:36:00.100 align:middle line:90% for example looks like that. 00:36:00.100 --> 00:36:02.670 align:middle line:84% So what we'll do over here is we'll go over here 00:36:02.670 --> 00:36:05.010 align:middle line:84% and we'll run a local regression where this point gets 00:36:05.010 --> 00:36:06.520 align:middle line:84% weighted a lot, and this gets weighed a little less, 00:36:06.520 --> 00:36:07.920 align:middle line:84% and this gets weighed a little less, and so on. 00:36:07.920 --> 00:36:10.090 align:middle line:84% And we'll get a slope it looks like this. 00:36:10.090 --> 00:36:17.940 align:middle line:84% And then we're going to do that for each of these 100 points, 00:36:17.940 --> 00:36:19.342 align:middle line:90% like that. 00:36:19.342 --> 00:36:20.675 align:middle line:90% And then we'll connect the dots. 00:36:20.675 --> 00:36:23.500 align:middle line:90% 00:36:23.500 --> 00:36:25.180 align:middle line:90% OK, so that's what that is. 00:36:25.180 --> 00:36:27.280 align:middle line:90% So in theta, l poly does this. 00:36:27.280 --> 00:36:29.980 align:middle line:84% I'm sure there's an r version that does that as well. 00:36:29.980 --> 00:36:32.438 align:middle line:84% But I'm dating myself because I know this data command, not 00:36:32.438 --> 00:36:33.460 align:middle line:90% the r command. 00:36:33.460 --> 00:36:36.130 align:middle line:84% And I just I wanted to mention this because I think 00:36:36.130 --> 00:36:38.237 align:middle line:90% these are really nice. 00:36:38.237 --> 00:36:39.820 align:middle line:84% And then you can bootstrap, by the way 00:36:39.820 --> 00:36:41.720 align:middle line:90% to get confidence intervals. 00:36:41.720 --> 00:36:43.720 align:middle line:84% So have we talked about bootstraps? 00:36:43.720 --> 00:36:46.120 align:middle line:90% Yes? 00:36:46.120 --> 00:36:48.640 align:middle line:90% All right so that's-- 00:36:48.640 --> 00:36:53.320 align:middle line:84% I find these very nice because they summarize the data 00:36:53.320 --> 00:36:56.150 align:middle line:84% in ways that are a little hard to see with dot clouds, 00:36:56.150 --> 00:36:59.938 align:middle line:84% especially because if this is a binary variable, if you try 00:36:59.938 --> 00:37:01.480 align:middle line:84% to just plot this thing as a scatter, 00:37:01.480 --> 00:37:02.740 align:middle line:90% it doesn't look like this. 00:37:02.740 --> 00:37:04.580 align:middle line:84% Right if this is a binary-- if you plot this 00:37:04.580 --> 00:37:06.970 align:middle line:84% to a scatter, what it looks is like a bunch 00:37:06.970 --> 00:37:15.130 align:middle line:84% of x's and some y's and it's not very easily visualizable 00:37:15.130 --> 00:37:17.000 align:middle line:84% as a scatter since it's a binary variable. 00:37:17.000 --> 00:37:22.732 align:middle line:84% So I find these kinds of non-parametric, locally 00:37:22.732 --> 00:37:24.190 align:middle line:84% weighted regressions really helpful 00:37:24.190 --> 00:37:28.280 align:middle line:84% as a way of summarizing the data. 00:37:28.280 --> 00:37:29.540 align:middle line:90% End of aside. 00:37:29.540 --> 00:37:31.220 align:middle line:90% Questions? 00:37:31.220 --> 00:37:32.030 align:middle line:90% And then you can-- 00:37:32.030 --> 00:37:33.738 align:middle line:84% by the way, the other thing about this is 00:37:33.738 --> 00:37:34.910 align:middle line:90% can play with the bandwidth. 00:37:34.910 --> 00:37:38.630 align:middle line:84% So the wider you make this thing, 00:37:38.630 --> 00:37:41.230 align:middle line:84% the more you're going to be imposing smoothing on it. 00:37:41.230 --> 00:37:42.980 align:middle line:84% If you make it super narrow, then you just 00:37:42.980 --> 00:37:44.950 align:middle line:90% get the dot cloud back. 00:37:44.950 --> 00:37:48.320 align:middle line:84% Eventually, it ends up looking like a binned scatter. 00:37:48.320 --> 00:37:50.817 align:middle line:90% So you have to play with the-- 00:37:50.817 --> 00:37:52.400 align:middle line:84% there may be-- I don't know if there's 00:37:52.400 --> 00:37:53.967 align:middle line:90% an optimal bandwidth for these. 00:37:53.967 --> 00:37:55.550 align:middle line:84% I don't think there is that I know of. 00:37:55.550 --> 00:37:56.360 align:middle line:90% But there may be. 00:37:56.360 --> 00:37:57.690 align:middle line:84% But you can certainly play with the bandwidth 00:37:57.690 --> 00:37:59.315 align:middle line:84% to get a reasonable degree of smoothing 00:37:59.315 --> 00:38:00.665 align:middle line:90% when you make these graphs. 00:38:00.665 --> 00:38:08.870 align:middle line:90% 00:38:08.870 --> 00:38:11.420 align:middle line:84% And the standard errors tend to blow up with the tails 00:38:11.420 --> 00:38:13.878 align:middle line:84% because you don't have that much data over here to estimate 00:38:13.878 --> 00:38:14.430 align:middle line:90% them. 00:38:14.430 --> 00:38:15.090 align:middle line:90% So what does that say? 00:38:15.090 --> 00:38:17.150 align:middle line:84% So this is like downward sloping income, great. 00:38:17.150 --> 00:38:19.010 align:middle line:84% But the more interesting point is 00:38:19.010 --> 00:38:21.620 align:middle line:84% that we can then decompose this based 00:38:21.620 --> 00:38:25.220 align:middle line:84% on your observables and unobservable consumption. 00:38:25.220 --> 00:38:28.070 align:middle line:84% And you get it downsloping in observable consumption. 00:38:28.070 --> 00:38:31.340 align:middle line:84% And then this says, look, maybe these households kind of 00:38:31.340 --> 00:38:34.040 align:middle line:84% forecast they're not going to get it and they don't apply. 00:38:34.040 --> 00:38:37.747 align:middle line:84% The kind of I think really exciting part from a targeting 00:38:37.747 --> 00:38:39.830 align:middle line:84% perspective is that it's actually downward sloping 00:38:39.830 --> 00:38:43.080 align:middle line:84% in the unobservable component of component of consumption, 00:38:43.080 --> 00:38:44.480 align:middle line:90% which is what this is. 00:38:44.480 --> 00:38:48.140 align:middle line:84% So this is y minus the observable-- y minus y hat 00:38:48.140 --> 00:38:49.280 align:middle line:90% on the x-axis. 00:38:49.280 --> 00:38:51.570 align:middle line:84% And whether you show up and asked 00:38:51.570 --> 00:38:55.043 align:middle line:90% to be screen on the y-axis. 00:38:55.043 --> 00:38:56.460 align:middle line:84% And this is also downward sloping. 00:38:56.460 --> 00:38:58.918 align:middle line:84% And that says the government is kind of potentially getting 00:38:58.918 --> 00:39:01.070 align:middle line:90% some information. 00:39:01.070 --> 00:39:04.280 align:middle line:84% And so and this just shows you this 00:39:04.280 --> 00:39:07.777 align:middle line:84% in regression form, which is that your probability 00:39:07.777 --> 00:39:09.860 align:middle line:84% of showing up depends both on the observable piece 00:39:09.860 --> 00:39:15.340 align:middle line:84% and the unobservable piece of your consumption. 00:39:15.340 --> 00:39:20.840 align:middle line:84% OK, so then we can look at the results. 00:39:20.840 --> 00:39:26.780 align:middle line:90% So this is a CDF of the results. 00:39:26.780 --> 00:39:29.930 align:middle line:84% So another point is I really like showing CDFs 00:39:29.930 --> 00:39:31.270 align:middle line:90% of results for two reasons. 00:39:31.270 --> 00:39:33.010 align:middle line:84% I don't know if you guys have talked about this nugget. 00:39:33.010 --> 00:39:34.400 align:middle line:90% So there's two advantages. 00:39:34.400 --> 00:39:36.275 align:middle line:84% First of all, someone want to interpret this? 00:39:36.275 --> 00:39:39.430 align:middle line:90% 00:39:39.430 --> 00:39:40.740 align:middle line:90% What do we take away from this? 00:39:40.740 --> 00:39:49.462 align:middle line:90% 00:39:49.462 --> 00:39:50.420 align:middle line:90% Wesley had his hand up. 00:39:50.420 --> 00:39:53.040 align:middle line:90% Go ahead, Wesley. 00:39:53.040 --> 00:39:54.530 align:middle line:84% AUDIENCE: So under self-targeting, 00:39:54.530 --> 00:40:01.120 align:middle line:84% the log per capita consumption, so you'll get to 100% of the-- 00:40:01.120 --> 00:40:08.180 align:middle line:84% you'll get to the full range of individuals applying earlier 00:40:08.180 --> 00:40:09.671 align:middle line:84% in the long per capita consumption 00:40:09.671 --> 00:40:12.830 align:middle line:84% under self-targeting [INAUDIBLE].. 00:40:12.830 --> 00:40:15.343 align:middle line:84% BEN OLKEN: Yeah, [INAUDIBLE],, what were you going to say? 00:40:15.343 --> 00:40:17.510 align:middle line:84% AUDIENCE: I guess the one that's further to the left 00:40:17.510 --> 00:40:19.390 align:middle line:90% is doing better. 00:40:19.390 --> 00:40:20.377 align:middle line:90% BEN OLKEN: In terms of? 00:40:20.377 --> 00:40:21.710 align:middle line:90% AUDIENCE: In terms of targeting. 00:40:21.710 --> 00:40:23.627 align:middle line:84% BEN OLKEN: Yeah, identifying poor households-- 00:40:23.627 --> 00:40:24.640 align:middle line:90% Right. 00:40:24.640 --> 00:40:27.068 align:middle line:84% And just to be clear, this line first order 00:40:27.068 --> 00:40:29.110 align:middle line:84% is this line first order stochastically dominates 00:40:29.110 --> 00:40:29.860 align:middle line:90% this line. 00:40:29.860 --> 00:40:32.260 align:middle line:84% So it's like it's to the left kind of everywhere 00:40:32.260 --> 00:40:34.500 align:middle line:90% on the distribution. 00:40:34.500 --> 00:40:38.330 align:middle line:84% So why do I like plotting CDFs as opposed to PDFs? 00:40:38.330 --> 00:40:40.140 align:middle line:90% There's two reasons. 00:40:40.140 --> 00:40:44.330 align:middle line:84% Number one, if you want to show your data non-parametrically, 00:40:44.330 --> 00:40:47.810 align:middle line:84% there's no statistics in creating this graph. 00:40:47.810 --> 00:40:51.800 align:middle line:84% There's no-- this is literally just the data, because why? 00:40:51.800 --> 00:40:53.120 align:middle line:90% How do you make-- what's a CDF? 00:40:53.120 --> 00:40:55.580 align:middle line:84% Well a CDF is for every point, you just 00:40:55.580 --> 00:40:57.860 align:middle line:90% calculate what is like-- 00:40:57.860 --> 00:41:00.950 align:middle line:84% for each x, we're just calculating 00:41:00.950 --> 00:41:12.330 align:middle line:84% what is the probability that say income is less than some x, 00:41:12.330 --> 00:41:15.407 align:middle line:84% conditional on treatment equals 1 or 0. 00:41:15.407 --> 00:41:17.490 align:middle line:84% So we're going to do this for each of those things 00:41:17.490 --> 00:41:19.710 align:middle line:84% and just go over the whole range of different values. 00:41:19.710 --> 00:41:22.560 align:middle line:90% That's what a CDF is. 00:41:22.560 --> 00:41:28.230 align:middle line:84% So there's no-- this is just like tabbing your data. 00:41:28.230 --> 00:41:30.000 align:middle line:84% There's nothing new to estimate here. 00:41:30.000 --> 00:41:34.950 align:middle line:84% When you want to start estimating PDFs, you have to-- 00:41:34.950 --> 00:41:37.890 align:middle line:84% a PDF is, as I'm sure you know the derivative of PDF. 00:41:37.890 --> 00:41:41.680 align:middle line:84% But you also need to smooth those things in practice. 00:41:41.680 --> 00:41:44.040 align:middle line:84% And again, that can depend on your smoothing parameters. 00:41:44.040 --> 00:41:46.770 align:middle line:90% This is literally just the data. 00:41:46.770 --> 00:41:49.770 align:middle line:84% OK, so that's nice because it's just kind 00:41:49.770 --> 00:41:51.750 align:middle line:90% of a way of showing your data. 00:41:51.750 --> 00:41:55.460 align:middle line:84% The other thing is this shows you a lot of information 00:41:55.460 --> 00:41:58.400 align:middle line:84% because this also shows you the difference in your treatments 00:41:58.400 --> 00:42:00.110 align:middle line:90% at every quantile. 00:42:00.110 --> 00:42:02.870 align:middle line:84% And so to read that, you just read this off like, 00:42:02.870 --> 00:42:05.002 align:middle line:84% so what is the difference in between treatment 00:42:05.002 --> 00:42:07.460 align:middle line:84% and control evaluated at the 50th percentile of the outcome 00:42:07.460 --> 00:42:08.002 align:middle line:90% distribution? 00:42:08.002 --> 00:42:09.890 align:middle line:90% Well, I just go to-- 00:42:09.890 --> 00:42:13.760 align:middle line:84% I go right here and I can read it the crossover here. 00:42:13.760 --> 00:42:18.040 align:middle line:84% And this gap here is like the 50th quantile. 00:42:18.040 --> 00:42:19.475 align:middle line:84% And so I can say, for example, is 00:42:19.475 --> 00:42:21.850 align:middle line:84% this having effects kind of lower or higher in the income 00:42:21.850 --> 00:42:23.380 align:middle line:90% distribution or whatever. 00:42:23.380 --> 00:42:30.160 align:middle line:84% So that's not necessarily the most important in this context. 00:42:30.160 --> 00:42:32.650 align:middle line:84% But in a lot of other contexts, that kind 00:42:32.650 --> 00:42:36.110 align:middle line:84% of additional information can be really valuable. 00:42:36.110 --> 00:42:38.390 align:middle line:90% So anyway so that's the CDF. 00:42:38.390 --> 00:42:41.560 align:middle line:84% So that tells you the difference between-- 00:42:41.560 --> 00:42:43.900 align:middle line:84% so you can see that the people who are selected 00:42:43.900 --> 00:42:48.790 align:middle line:84% are kind of uniformly left, they're poorer. 00:42:48.790 --> 00:42:52.390 align:middle line:84% This is showing you the probability 00:42:52.390 --> 00:42:54.470 align:middle line:90% of obtaining benefits. 00:42:54.470 --> 00:42:56.440 align:middle line:84% So this is graphing-- this is now 00:42:56.440 --> 00:42:59.710 align:middle line:84% estimating non-parametrically, what 00:42:59.710 --> 00:43:02.410 align:middle line:84% is the probability that you obtained benefits 00:43:02.410 --> 00:43:04.330 align:middle line:90% at each kind of level. 00:43:04.330 --> 00:43:07.790 align:middle line:84% And what you see is you see two things. 00:43:07.790 --> 00:43:11.270 align:middle line:84% So the desired targeting range is over here. 00:43:11.270 --> 00:43:12.510 align:middle line:90% You see two things going on. 00:43:12.510 --> 00:43:16.260 align:middle line:84% Number one, this black line is basically much, 00:43:16.260 --> 00:43:18.060 align:middle line:90% much below the orange line. 00:43:18.060 --> 00:43:18.930 align:middle line:90% Why is that? 00:43:18.930 --> 00:43:22.700 align:middle line:84% That's because the rich people are not bothering to-- 00:43:22.700 --> 00:43:26.600 align:middle line:84% the rich people who might get through the proxy means test 00:43:26.600 --> 00:43:30.120 align:middle line:84% like by mistake are not bothering to show up. 00:43:30.120 --> 00:43:33.280 align:middle line:90% And that's kind of the key-- 00:43:33.280 --> 00:43:36.130 align:middle line:84% I think the key thing that's going on in a lot of the paper 00:43:36.130 --> 00:43:37.870 align:middle line:84% is that these people are just not bothering to show up. 00:43:37.870 --> 00:43:39.953 align:middle line:84% And therefore even though they might have made it, 00:43:39.953 --> 00:43:41.765 align:middle line:84% the probability ex ante that they 00:43:41.765 --> 00:43:44.140 align:middle line:84% would have made it really small that they kind of decided 00:43:44.140 --> 00:43:45.685 align:middle line:90% not to do it. 00:43:45.685 --> 00:43:47.310 align:middle line:84% The other thing which is interesting is 00:43:47.310 --> 00:43:49.530 align:middle line:84% the other point, which we did not expect, 00:43:49.530 --> 00:43:52.710 align:middle line:84% actually, is that the black line is 00:43:52.710 --> 00:43:53.970 align:middle line:90% greater than the orange line. 00:43:53.970 --> 00:43:56.248 align:middle line:84% And that's for this other reason at the very far left. 00:43:56.248 --> 00:43:57.790 align:middle line:84% That's this other reason I mentioned, 00:43:57.790 --> 00:44:00.207 align:middle line:84% which is that you try to figure out who should be screened 00:44:00.207 --> 00:44:01.500 align:middle line:90% in a door-to-door approach. 00:44:01.500 --> 00:44:04.795 align:middle line:84% But maybe you don't get everyone necessarily. 00:44:04.795 --> 00:44:06.170 align:middle line:84% Whereas in an on-demand approach, 00:44:06.170 --> 00:44:10.730 align:middle line:84% people can make themselves be known and say, look, I'm poor. 00:44:10.730 --> 00:44:12.480 align:middle line:84% Please include me in this program. 00:44:12.480 --> 00:44:15.770 align:middle line:84% And so it turns out that actually this approach has both 00:44:15.770 --> 00:44:20.780 align:middle line:84% of these benefits of excluding the rich who you didn't mean 00:44:20.780 --> 00:44:25.610 align:middle line:84% to be-- who you didn't intend to target, and getting the-- 00:44:25.610 --> 00:44:27.590 align:middle line:84% allowing the poor to include themselves. 00:44:27.590 --> 00:44:29.090 align:middle line:84% And the other thing I'll say is that 00:44:29.090 --> 00:44:31.610 align:middle line:84% relative to other kinds of approaches that use kind 00:44:31.610 --> 00:44:33.608 align:middle line:90% of costly ordeals as targeting. 00:44:33.608 --> 00:44:35.400 align:middle line:84% Some approaches are like workfare programs. 00:44:35.400 --> 00:44:37.970 align:middle line:84% So for example, probably the largest program 00:44:37.970 --> 00:44:40.460 align:middle line:84% in the developing world, anti-poverty program 00:44:40.460 --> 00:44:42.777 align:middle line:84% is the MNREGA program in India right now. 00:44:42.777 --> 00:44:45.110 align:middle line:84% And it's the largest-- it's one of the largest programs, 00:44:45.110 --> 00:44:49.400 align:middle line:84% very large program, 140 million people, I think. 00:44:49.400 --> 00:44:51.830 align:middle line:84% Certainly in the order of 100 million people or more 00:44:51.830 --> 00:44:52.700 align:middle line:90% in this program. 00:44:52.700 --> 00:44:54.283 align:middle line:84% And what do they have to do in MNREGA? 00:44:54.283 --> 00:44:55.680 align:middle line:90% They have to go basically work. 00:44:55.680 --> 00:44:58.490 align:middle line:84% If you want to get paid, you have 00:44:58.490 --> 00:45:03.452 align:middle line:84% to go work through manual labor for a project for however 00:45:03.452 --> 00:45:04.410 align:middle line:90% many hours or whatever. 00:45:04.410 --> 00:45:06.190 align:middle line:90% And you get paid minimum wage. 00:45:06.190 --> 00:45:08.040 align:middle line:84% That's a costly ordeal because they 00:45:08.040 --> 00:45:10.540 align:middle line:84% could have other things they could be doing with their time. 00:45:10.540 --> 00:45:13.000 align:middle line:84% Or they might not like manual labor or other things 00:45:13.000 --> 00:45:14.450 align:middle line:90% it's a very costly thing to do. 00:45:14.450 --> 00:45:16.150 align:middle line:84% It takes all of your time for 100 days. 00:45:16.150 --> 00:45:19.420 align:middle line:84% This, by comparison, is a super small ordeal. 00:45:19.420 --> 00:45:21.577 align:middle line:84% It's like half a day of your time. 00:45:21.577 --> 00:45:23.410 align:middle line:84% And so I think what's really attractive here 00:45:23.410 --> 00:45:25.870 align:middle line:84% is to say, look, the Nichols Zeckhauser 00:45:25.870 --> 00:45:28.660 align:middle line:84% idea says that we have these really large ordeals we're 00:45:28.660 --> 00:45:29.330 align:middle line:90% going to do it. 00:45:29.330 --> 00:45:32.260 align:middle line:84% But here by going back to the pre-period 00:45:32.260 --> 00:45:34.180 align:middle line:84% and leveraging this uncertainty that people 00:45:34.180 --> 00:45:36.490 align:middle line:84% are going to have about the application process, 00:45:36.490 --> 00:45:38.350 align:middle line:84% maybe we can actually get a lot of those screening benefits 00:45:38.350 --> 00:45:39.433 align:middle line:90% with pretty small ordeals. 00:45:39.433 --> 00:45:43.430 align:middle line:84% And that's kind of what this shows. 00:45:43.430 --> 00:45:46.220 align:middle line:84% AUDIENCE: One thing that I would be curious with this paper 00:45:46.220 --> 00:45:49.010 align:middle line:84% is even at the peak of the [INAUDIBLE],, 00:45:49.010 --> 00:45:50.445 align:middle line:90% you have less than 20%. 00:45:50.445 --> 00:45:52.820 align:middle line:84% BEN OLKEN: Yes, a couple of you guys mentioned this, yes. 00:45:52.820 --> 00:45:54.368 align:middle line:84% AUDIENCE: So maybe qualitatively, 00:45:54.368 --> 00:45:55.910 align:middle line:84% do you have a sense of whether that's 00:45:55.910 --> 00:45:58.160 align:middle line:84% due to this issue you were talking before where still 00:45:58.160 --> 00:46:00.740 align:middle line:84% there's people who aren't announcing themselves, even 00:46:00.740 --> 00:46:03.582 align:middle line:84% with this self-targeting treatment [INAUDIBLE].. 00:46:03.582 --> 00:46:05.540 align:middle line:84% BEN OLKEN: So I think there's two things-- so I 00:46:05.540 --> 00:46:06.665 align:middle line:90% think you're totally right. 00:46:06.665 --> 00:46:07.910 align:middle line:90% Why is this only 20%? 00:46:07.910 --> 00:46:08.960 align:middle line:90% That's not great. 00:46:08.960 --> 00:46:11.250 align:middle line:84% So I think there's a couple of things going on here. 00:46:11.250 --> 00:46:18.290 align:middle line:84% The first is the show up rate at the bottom is about 60%. 00:46:18.290 --> 00:46:21.940 align:middle line:84% So part of it is 40% of the people here are not applying. 00:46:21.940 --> 00:46:23.260 align:middle line:90% That's not all. 00:46:23.260 --> 00:46:24.062 align:middle line:90% That's part of it. 00:46:24.062 --> 00:46:26.020 align:middle line:84% And I don't know if this is necessarily stigma. 00:46:26.020 --> 00:46:27.790 align:middle line:84% It could be some of the people actually do 00:46:27.790 --> 00:46:31.390 align:middle line:90% have challenges showing up. 00:46:31.390 --> 00:46:33.660 align:middle line:90% I think that's real. 00:46:33.660 --> 00:46:36.160 align:middle line:84% I'm not sure stigma was a huge problem here in this context, 00:46:36.160 --> 00:46:37.840 align:middle line:90% actually. 00:46:37.840 --> 00:46:38.590 align:middle line:90% But that could be. 00:46:38.590 --> 00:46:40.173 align:middle line:84% But I don't think it's a real problem. 00:46:40.173 --> 00:46:41.840 align:middle line:84% I think it's more just people are busy. 00:46:41.840 --> 00:46:43.048 align:middle line:90% They have other things to do. 00:46:43.048 --> 00:46:45.256 align:middle line:84% There was a lot of effort to get the information out. 00:46:45.256 --> 00:46:46.340 align:middle line:90% Maybe not everyone got it. 00:46:46.340 --> 00:46:47.180 align:middle line:90% So I think that's right. 00:46:47.180 --> 00:46:49.270 align:middle line:84% And I think that maybe doing this kind of multiple times 00:46:49.270 --> 00:46:50.978 align:middle line:84% maybe it might be really important to get 00:46:50.978 --> 00:46:52.093 align:middle line:90% this number higher. 00:46:52.093 --> 00:46:54.010 align:middle line:84% But the second point is that proxy means tests 00:46:54.010 --> 00:46:54.670 align:middle line:90% are imperfect. 00:46:54.670 --> 00:46:59.380 align:middle line:84% And I think that's the other piece of it. 00:46:59.380 --> 00:47:03.127 align:middle line:84% What's the probability you get benefits? 00:47:03.127 --> 00:47:04.960 align:middle line:84% I didn't bring it in as a function of total. 00:47:04.960 --> 00:47:07.370 align:middle line:84% Oh, yeah, here, what's the probability you get benefits 00:47:07.370 --> 00:47:08.620 align:middle line:90% as a function of your total y? 00:47:08.620 --> 00:47:10.300 align:middle line:90% Well, it's only like 0.3. 00:47:10.300 --> 00:47:13.030 align:middle line:84% And that's because it's imperfect. 00:47:13.030 --> 00:47:14.470 align:middle line:84% And it's also imperfect precisely 00:47:14.470 --> 00:47:17.620 align:middle line:84% because they're trying to target a really low fraction 00:47:17.620 --> 00:47:18.670 align:middle line:90% of people. 00:47:18.670 --> 00:47:21.370 align:middle line:84% They're setting this really tight. 00:47:21.370 --> 00:47:23.230 align:middle line:84% And that's kind of a policy decision. 00:47:23.230 --> 00:47:25.870 align:middle line:84% There's a broader question of this policy trade of, 00:47:25.870 --> 00:47:28.060 align:middle line:84% like do you want to target really tight? 00:47:28.060 --> 00:47:30.550 align:middle line:84% And you're going to make fewer inclusion errors, but also 00:47:30.550 --> 00:47:31.787 align:middle line:90% a lot more exclusion errors. 00:47:31.787 --> 00:47:33.370 align:middle line:84% Or do you want to target more broadly? 00:47:33.370 --> 00:47:35.320 align:middle line:84% And you're going to make fewer exclusion areas and more 00:47:35.320 --> 00:47:35.830 align:middle line:90% inclusion errors. 00:47:35.830 --> 00:47:37.997 align:middle line:84% And the particular program is targeted really tight. 00:47:37.997 --> 00:47:42.960 align:middle line:84% So I think it's the combination of this fact and this fact. 00:47:42.960 --> 00:47:45.430 align:middle line:84% So I think it's more about the imperfections 00:47:45.430 --> 00:47:47.980 align:middle line:84% in the proxy means test fundamentally 00:47:47.980 --> 00:47:51.490 align:middle line:84% at targeting people who are this poor is I think more. 00:47:51.490 --> 00:47:54.970 align:middle line:84% And I think understanding how to fix that problem 00:47:54.970 --> 00:47:57.428 align:middle line:84% and reduce exclusion is a first order issue that I don't 00:47:57.428 --> 00:47:58.720 align:middle line:90% think we have a good handle on. 00:47:58.720 --> 00:48:01.150 align:middle line:84% And actually, to me, to my mind, actually, I 00:48:01.150 --> 00:48:03.650 align:middle line:84% think one of the biggest issues that I don't think we really 00:48:03.650 --> 00:48:06.710 align:middle line:90% understand is how do you-- 00:48:06.710 --> 00:48:08.940 align:middle line:84% how do you reduce exclusion error in these contexts? 00:48:08.940 --> 00:48:11.600 align:middle line:84% And I think that this approach says actually, 00:48:11.600 --> 00:48:14.090 align:middle line:84% despite the fact that only 60% of people or 60% or 70% 00:48:14.090 --> 00:48:16.430 align:middle line:84% of people applying here, on net, this approach 00:48:16.430 --> 00:48:18.463 align:middle line:84% actually reduces the exclusion error because it 00:48:18.463 --> 00:48:19.880 align:middle line:84% has this other benefit of at least 00:48:19.880 --> 00:48:22.293 align:middle line:84% we allow everyone the option to apply. 00:48:22.293 --> 00:48:24.210 align:middle line:84% So on net, this is better than the status quo. 00:48:24.210 --> 00:48:25.952 align:middle line:90% But I completely agree with you. 00:48:25.952 --> 00:48:27.410 align:middle line:84% There's a ton of exclusionary here. 00:48:27.410 --> 00:48:30.260 align:middle line:84% And I think that how do you reduce 00:48:30.260 --> 00:48:31.350 align:middle line:90% that is really important. 00:48:31.350 --> 00:48:34.120 align:middle line:84% So I completely agree with that question. 00:48:34.120 --> 00:48:35.040 align:middle line:90% Other questions? 00:48:35.040 --> 00:48:40.730 align:middle line:90% 00:48:40.730 --> 00:48:41.920 align:middle line:90% So the final point-- 00:48:41.920 --> 00:48:46.230 align:middle line:84% so then we want to understand the different theoretical 00:48:46.230 --> 00:48:46.730 align:middle line:90% mechanisms. 00:48:46.730 --> 00:48:48.360 align:middle line:84% So we're going to estimate the model. 00:48:48.360 --> 00:48:51.230 align:middle line:90% So how do we do that? 00:48:51.230 --> 00:48:52.410 align:middle line:90% So what do we have to do? 00:48:52.410 --> 00:48:55.350 align:middle line:84% Well, the first thing is we have to parameterize some stuff. 00:48:55.350 --> 00:49:00.300 align:middle line:84% OK, we have to take those vague things like c of l and y 00:49:00.300 --> 00:49:04.230 align:middle line:84% and turn them into actual numbers that we can estimate, 00:49:04.230 --> 00:49:05.910 align:middle line:84% plug into a model in the computer. 00:49:05.910 --> 00:49:07.540 align:middle line:90% So parameterize some things. 00:49:07.540 --> 00:49:10.910 align:middle line:84% So we're going to parameterize the cost as follows. 00:49:10.910 --> 00:49:12.210 align:middle line:90% We're going to say your wage-- 00:49:12.210 --> 00:49:15.420 align:middle line:84% the cost is equal to your wage times the travel 00:49:15.420 --> 00:49:17.190 align:middle line:84% time plus the average amount of time 00:49:17.190 --> 00:49:19.620 align:middle line:90% that you have to wait there. 00:49:19.620 --> 00:49:21.397 align:middle line:90% This is your expected cost. 00:49:21.397 --> 00:49:22.980 align:middle line:84% It's not your realization of the cost. 00:49:22.980 --> 00:49:23.890 align:middle line:90% The expected cost. 00:49:23.890 --> 00:49:24.598 align:middle line:90% So it should be-- 00:49:24.598 --> 00:49:26.430 align:middle line:84% it's how much you forecast it's going 00:49:26.430 --> 00:49:33.330 align:middle line:84% to cost you to apply for this thing, plus any monetary cost 00:49:33.330 --> 00:49:34.410 align:middle line:90% of traveling there. 00:49:34.410 --> 00:49:37.640 align:middle line:90% 00:49:37.640 --> 00:49:40.760 align:middle line:84% And by the way, we were thinking about this in advance, 00:49:40.760 --> 00:49:43.610 align:middle line:84% so we collected the variables we needed to get this, 00:49:43.610 --> 00:49:44.900 align:middle line:90% estimate this. 00:49:44.900 --> 00:49:46.760 align:middle line:90% We just knew the distance. 00:49:46.760 --> 00:49:49.368 align:middle line:84% And we knew how people would typically get to these places 00:49:49.368 --> 00:49:49.910 align:middle line:90% and whatever. 00:49:49.910 --> 00:49:51.680 align:middle line:84% And we collected that in the baseline 00:49:51.680 --> 00:49:53.550 align:middle line:84% surveys because we had an idea we were going 00:49:53.550 --> 00:49:54.590 align:middle line:90% to interested in this question. 00:49:54.590 --> 00:49:56.630 align:middle line:84% That's another, going back to the theory point, 00:49:56.630 --> 00:49:58.670 align:middle line:84% we thought this was going to be this costing thing was 00:49:58.670 --> 00:49:59.760 align:middle line:90% different, cost was important. 00:49:59.760 --> 00:50:01.010 align:middle line:84% So we tried to collect some data that was going 00:50:01.010 --> 00:50:02.690 align:middle line:90% to be useful to estimate it. 00:50:02.690 --> 00:50:05.690 align:middle line:90% 00:50:05.690 --> 00:50:08.540 align:middle line:84% One thing, which by the way, is first order here, 00:50:08.540 --> 00:50:12.185 align:middle line:84% is consumption is probably measured with error. 00:50:12.185 --> 00:50:13.560 align:middle line:84% And so we have to deal with that. 00:50:13.560 --> 00:50:16.060 align:middle line:84% So we were going to parameterize the measurement error here. 00:50:16.060 --> 00:50:17.600 align:middle line:90% That's fine. 00:50:17.600 --> 00:50:19.702 align:middle line:84% We have a different data set with a short panel. 00:50:19.702 --> 00:50:22.160 align:middle line:84% And so we're going to use that where we measure consumption 00:50:22.160 --> 00:50:23.577 align:middle line:84% twice in a row for the same people 00:50:23.577 --> 00:50:25.940 align:middle line:84% and use that to parameterize measurement error. 00:50:25.940 --> 00:50:28.190 align:middle line:84% And the reason for that is that we observe consumption 00:50:28.190 --> 00:50:28.730 align:middle line:90% measured with error. 00:50:28.730 --> 00:50:29.960 align:middle line:84% But we assume people make their decisions 00:50:29.960 --> 00:50:31.043 align:middle line:90% based on true consumption. 00:50:31.043 --> 00:50:33.885 align:middle line:90% So we want to parameterize that. 00:50:33.885 --> 00:50:36.260 align:middle line:84% Otherwise we're going to load too much stuff on the error 00:50:36.260 --> 00:50:39.650 align:middle line:84% term, basically, if you don't incorporate that. 00:50:39.650 --> 00:50:42.783 align:middle line:90% We need this lambda term. 00:50:42.783 --> 00:50:43.700 align:middle line:90% We'll just make that-- 00:50:43.700 --> 00:50:45.840 align:middle line:90% we'll assume that's a probit. 00:50:45.840 --> 00:50:46.725 align:middle line:90% It seems sensible. 00:50:46.725 --> 00:50:49.620 align:middle line:90% 00:50:49.620 --> 00:50:55.990 align:middle line:84% And so the unknown parameters, we 00:50:55.990 --> 00:50:58.220 align:middle line:84% have to estimate our alpha, which is the share who 00:50:58.220 --> 00:50:58.970 align:middle line:90% are sophisticated. 00:50:58.970 --> 00:51:00.550 align:middle line:90% We have no idea. 00:51:00.550 --> 00:51:03.130 align:middle line:84% The mean and the variance of the utility shocks epsilon 00:51:03.130 --> 00:51:09.945 align:middle line:84% and the parameters of the probe of the lambda distribution, 00:51:09.945 --> 00:51:12.320 align:middle line:84% which is people's-- which is what are the unsophisticated 00:51:12.320 --> 00:51:14.060 align:middle line:90% people going to think? 00:51:14.060 --> 00:51:18.170 align:middle line:90% And so we're going to estimate-- 00:51:18.170 --> 00:51:20.680 align:middle line:84% so what we get out of the model is that for any given 00:51:20.680 --> 00:51:26.500 align:middle line:84% individual, i, your probability of applying for benefits 00:51:26.500 --> 00:51:28.390 align:middle line:84% is, well, if you're sophisticated, 00:51:28.390 --> 00:51:29.570 align:middle line:90% it's given by this-- 00:51:29.570 --> 00:51:31.360 align:middle line:84% what's the probability that your gain 00:51:31.360 --> 00:51:34.120 align:middle line:84% from applying as a function of the observables 00:51:34.120 --> 00:51:35.670 align:middle line:90% is greater than the-- 00:51:35.670 --> 00:51:37.795 align:middle line:84% less than the-- whatever, greater than the epsilon, 00:51:37.795 --> 00:51:42.620 align:middle line:84% greater than the epsilon cost, and maybe off by minus error. 00:51:42.620 --> 00:51:47.400 align:middle line:84% And the same thing for the unobservables. 00:51:47.400 --> 00:51:50.410 align:middle line:84% So that's what the model gives us 00:51:50.410 --> 00:51:53.770 align:middle line:84% as a function of the unknown parameters. 00:51:53.770 --> 00:51:55.870 align:middle line:84% And so that's what we try to estimate. 00:51:55.870 --> 00:51:59.550 align:middle line:84% And so you went over a GMM last week. 00:51:59.550 --> 00:52:02.020 align:middle line:84% So this should all be familiar for you. 00:52:02.020 --> 00:52:05.070 align:middle line:84% But I just want to say a word about GMM, 00:52:05.070 --> 00:52:07.800 align:middle line:84% which is you hear a lot, a lot, a lot about structural models 00:52:07.800 --> 00:52:10.530 align:middle line:84% in like many classes and many papers. 00:52:10.530 --> 00:52:13.080 align:middle line:90% And I think they are over-- 00:52:13.080 --> 00:52:15.170 align:middle line:84% like they have this whole hype around them. 00:52:15.170 --> 00:52:16.920 align:middle line:84% And they're not actually that complicated. 00:52:16.920 --> 00:52:19.530 align:middle line:84% And so I just wanted to spend one slide doing my best 00:52:19.530 --> 00:52:21.930 align:middle line:84% to demystify them because it took me a long time 00:52:21.930 --> 00:52:22.840 align:middle line:90% to figure this out. 00:52:22.840 --> 00:52:28.210 align:middle line:84% So I want to save you guys that time. 00:52:28.210 --> 00:52:32.100 align:middle line:84% So what are we doing when we estimate a structural model? 00:52:32.100 --> 00:52:33.650 align:middle line:84% So we write down a model, which is 00:52:33.650 --> 00:52:35.240 align:middle line:84% like a theoretical model, which has 00:52:35.240 --> 00:52:37.260 align:middle line:90% some unknown parameters in it. 00:52:37.260 --> 00:52:40.470 align:middle line:84% OK, so in our case, I just wrote down 00:52:40.470 --> 00:52:44.160 align:middle line:84% a model, which has unknown parameters, which 00:52:44.160 --> 00:52:46.520 align:middle line:84% generates a probability that I show up 00:52:46.520 --> 00:52:48.710 align:middle line:84% as a function of individualized characteristics 00:52:48.710 --> 00:52:50.090 align:middle line:90% and some unknown parameters. 00:52:50.090 --> 00:52:52.310 align:middle line:84% In our case, it was like alpha and the utility shock 00:52:52.310 --> 00:52:54.350 align:middle line:84% distribution and the data distribution or lambda 00:52:54.350 --> 00:52:55.321 align:middle line:90% distribution. 00:52:55.321 --> 00:52:58.090 align:middle line:90% 00:52:58.090 --> 00:53:00.010 align:middle line:84% What the structural estimation says 00:53:00.010 --> 00:53:05.120 align:middle line:84% is I would like to know these unknown parameters. 00:53:05.120 --> 00:53:08.700 align:middle line:84% So what values of those unknown parameters 00:53:08.700 --> 00:53:11.730 align:middle line:84% get the models predicted show up probabilities 00:53:11.730 --> 00:53:13.860 align:middle line:84% to match the actual show up probabilities 00:53:13.860 --> 00:53:15.885 align:middle line:90% in the data for similar people. 00:53:15.885 --> 00:53:19.130 align:middle line:90% 00:53:19.130 --> 00:53:27.280 align:middle line:84% So the model is like probability of showing up for individual i 00:53:27.280 --> 00:53:30.870 align:middle line:84% is a function of these various things. 00:53:30.870 --> 00:53:33.350 align:middle line:84% And I just want to choose the values 00:53:33.350 --> 00:53:36.260 align:middle line:84% of those unknown parameters so that the models predicted 00:53:36.260 --> 00:53:40.590 align:middle line:84% show up and the actual show up in some data are the same. 00:53:40.590 --> 00:53:44.545 align:middle line:84% OK, so to do this, you define moments. 00:53:44.545 --> 00:53:47.340 align:middle line:90% 00:53:47.340 --> 00:53:49.382 align:middle line:84% Moments are statistics of the data 00:53:49.382 --> 00:53:51.090 align:middle line:84% that you can also calculate in the model. 00:53:51.090 --> 00:53:54.860 align:middle line:90% 00:53:54.860 --> 00:53:56.840 align:middle line:84% And you need to have at least as many moments 00:53:56.840 --> 00:53:59.570 align:middle line:90% as unknown parameters. 00:53:59.570 --> 00:54:00.960 align:middle line:90% Or you can have more. 00:54:00.960 --> 00:54:04.470 align:middle line:84% More is fine, but you can't have fewer. 00:54:04.470 --> 00:54:10.190 align:middle line:84% OK, and you want to say you then search for the particular thing 00:54:10.190 --> 00:54:14.990 align:middle line:84% you do is you just do a search over these unknown parameter 00:54:14.990 --> 00:54:18.020 align:middle line:90% vector, in this case five-- 00:54:18.020 --> 00:54:20.160 align:middle line:84% five-dimensional parameter vector, 00:54:20.160 --> 00:54:22.100 align:middle line:84% so that the moments from the model 00:54:22.100 --> 00:54:24.950 align:middle line:84% match as close as possible the moments in the data. 00:54:24.950 --> 00:54:27.990 align:middle line:90% 00:54:27.990 --> 00:54:31.530 align:middle line:84% And that's why it's called the method of moments. 00:54:31.530 --> 00:54:34.710 align:middle line:90% So one sec. 00:54:34.710 --> 00:54:37.480 align:middle line:90% And that's it. 00:54:37.480 --> 00:54:40.230 align:middle line:90% That's the basic idea. 00:54:40.230 --> 00:54:42.660 align:middle line:84% Set up a model, the function of unknown parameters, 00:54:42.660 --> 00:54:45.330 align:middle line:84% calculate some moments in that unknown parameters. 00:54:45.330 --> 00:54:47.760 align:middle line:84% And search-- and keep mucking around 00:54:47.760 --> 00:54:49.140 align:middle line:84% with the values of the parameters 00:54:49.140 --> 00:54:50.760 align:middle line:84% until the values of your five moments 00:54:50.760 --> 00:54:53.310 align:middle line:84% hit the values of the similar five moments at the target 00:54:53.310 --> 00:54:55.780 align:middle line:90% five ones from the data. 00:54:55.780 --> 00:54:58.240 align:middle line:90% So that's the main idea. 00:54:58.240 --> 00:55:00.360 align:middle line:90% So I said that's it. 00:55:00.360 --> 00:55:02.040 align:middle line:90% All the rest is commentary. 00:55:02.040 --> 00:55:03.923 align:middle line:84% So one important piece of commentary 00:55:03.923 --> 00:55:05.340 align:middle line:84% is, well, what happens if you have 00:55:05.340 --> 00:55:07.033 align:middle line:90% more parameters than moments. 00:55:07.033 --> 00:55:09.700 align:middle line:84% If you have more parameters than moments, you can't do anything. 00:55:09.700 --> 00:55:12.070 align:middle line:84% If you have more moments than parameters, 00:55:12.070 --> 00:55:14.040 align:middle line:84% now I won't be able to generically hit 00:55:14.040 --> 00:55:17.940 align:middle line:84% all the moments because I don't have enough degrees of freedom. 00:55:17.940 --> 00:55:21.000 align:middle line:84% So one of the many handy things that GMM does 00:55:21.000 --> 00:55:22.830 align:middle line:84% is how should I optimally kind of weight 00:55:22.830 --> 00:55:26.040 align:middle line:84% the different moments to try to match the parameters as best 00:55:26.040 --> 00:55:27.620 align:middle line:90% I can. 00:55:27.620 --> 00:55:29.140 align:middle line:90% OK, sorry. 00:55:29.140 --> 00:55:30.320 align:middle line:90% AUDIENCE: Oh, yeah. 00:55:30.320 --> 00:55:31.820 align:middle line:90% I was wondering how-- 00:55:31.820 --> 00:55:34.400 align:middle line:84% because I guess the thing we're matching here is probability. 00:55:34.400 --> 00:55:36.442 align:middle line:84% We all just learned [INAUDIBLE] models 00:55:36.442 --> 00:55:38.720 align:middle line:90% or probit models and metrics. 00:55:38.720 --> 00:55:40.760 align:middle line:84% I guess we're doing a very similar thing. 00:55:40.760 --> 00:55:42.530 align:middle line:84% I'm wondering like just conceptually, 00:55:42.530 --> 00:55:44.510 align:middle line:84% maybe like how does this compare to when 00:55:44.510 --> 00:55:47.037 align:middle line:90% I run a probit model in Stata. 00:55:47.037 --> 00:55:49.120 align:middle line:84% BEN OLKEN: So those are maximum likelihood models. 00:55:49.120 --> 00:55:51.300 align:middle line:84% So there's two ways you can estimate a model. 00:55:51.300 --> 00:55:52.983 align:middle line:90% Sorry, this is the GMM version. 00:55:52.983 --> 00:55:54.900 align:middle line:84% The other version is maximum likelihood, where 00:55:54.900 --> 00:55:56.817 align:middle line:84% I specify the full kind of likelihood function 00:55:56.817 --> 00:56:00.420 align:middle line:84% from the data, and say, what was my full data generating 00:56:00.420 --> 00:56:06.270 align:middle line:84% process, and search over the-- and you're basically 00:56:06.270 --> 00:56:11.830 align:middle line:84% saying in maximum likelihood, you're 00:56:11.830 --> 00:56:20.590 align:middle line:84% essentially saying, what is the underlying set of parameters 00:56:20.590 --> 00:56:23.530 align:middle line:84% such that to maximize the likelihood 00:56:23.530 --> 00:56:27.550 align:middle line:84% that the data I observe actually came from that distribution. 00:56:27.550 --> 00:56:31.450 align:middle line:84% AUDIENCE: But I can also fit my probit model to GMM. 00:56:31.450 --> 00:56:36.370 align:middle line:84% BEN OLKEN: You can fit a binary variable model with GMM, yes. 00:56:36.370 --> 00:56:38.530 align:middle line:84% But the basic model you're learning in metrics 00:56:38.530 --> 00:56:39.747 align:middle line:90% is an [INAUDIBLE]. 00:56:39.747 --> 00:56:41.830 align:middle line:84% AUDIENCE: I guess maybe I misphrased the question. 00:56:41.830 --> 00:56:44.862 align:middle line:90% So it makes sense. 00:56:44.862 --> 00:56:46.570 align:middle line:84% BEN OLKEN: And, Ed, by the way correct me 00:56:46.570 --> 00:56:48.670 align:middle line:90% if I'm saying it wrong. 00:56:48.670 --> 00:56:50.350 align:middle line:84% AUDIENCE: That, what you just described 00:56:50.350 --> 00:56:52.360 align:middle line:84% is something that's a one liner in Stata, right? 00:56:52.360 --> 00:56:53.068 align:middle line:90% BEN OLKEN: Which? 00:56:53.068 --> 00:56:55.900 align:middle line:84% AUDIENCE: Like running the maximum likelihood of y on x. 00:56:55.900 --> 00:56:57.950 align:middle line:84% Obviously, this is not a one liner in Stata. 00:56:57.950 --> 00:57:01.030 align:middle line:84% BEN OLKEN: No, it's not that many lines-- 00:57:01.030 --> 00:57:02.808 align:middle line:90% maybe, depending on the p set. 00:57:02.808 --> 00:57:05.350 align:middle line:84% AUDIENCE: You don't necessarily have the likelihood function. 00:57:05.350 --> 00:57:07.725 align:middle line:84% So you have more information when you have the likelihood 00:57:07.725 --> 00:57:08.708 align:middle line:90% function. 00:57:08.708 --> 00:57:09.500 align:middle line:90% BEN OLKEN: Correct. 00:57:09.500 --> 00:57:10.658 align:middle line:90% AUDIENCE: OK. 00:57:10.658 --> 00:57:12.200 align:middle line:84% AUDIENCE: So you wouldn't necessarily 00:57:12.200 --> 00:57:13.880 align:middle line:84% want to put a probit model in the GMM 00:57:13.880 --> 00:57:17.090 align:middle line:84% because if you're already assuming I know everything 00:57:17.090 --> 00:57:18.860 align:middle line:84% about the data generating process, 00:57:18.860 --> 00:57:21.275 align:middle line:84% then you can use all the information of the likelihood. 00:57:21.275 --> 00:57:22.370 align:middle line:90% AUDIENCE: I see, I see. 00:57:22.370 --> 00:57:26.390 align:middle line:84% Whereas here we don't know how the parameters map [INAUDIBLE].. 00:57:26.390 --> 00:57:29.120 align:middle line:84% BEN OLKEN: We don't know the full data generating process. 00:57:29.120 --> 00:57:31.550 align:middle line:84% AUDIENCE: But you do a couple of points [INAUDIBLE] 00:57:31.550 --> 00:57:32.408 align:middle line:90% it has to satisfy. 00:57:32.408 --> 00:57:33.200 align:middle line:90% BEN OLKEN: Correct. 00:57:33.200 --> 00:57:36.012 align:middle line:84% AUDIENCE: You're assuming [INAUDIBLE].. 00:57:36.012 --> 00:57:37.220 align:middle line:90% BEN OLKEN: I agree with that. 00:57:37.220 --> 00:57:39.840 align:middle line:90% It's a less restrictive-- 00:57:39.840 --> 00:57:40.727 align:middle line:90% you know less. 00:57:40.727 --> 00:57:42.665 align:middle line:90% 00:57:42.665 --> 00:57:44.540 align:middle line:84% AUDIENCE: One point I was confused about is-- 00:57:44.540 --> 00:57:46.688 align:middle line:90% 00:57:46.688 --> 00:57:49.230 align:middle line:84% BEN OLKEN: Oh, sorry, let me say one thing before a question. 00:57:49.230 --> 00:57:51.720 align:middle line:84% But often writing down the whole likelihood function 00:57:51.720 --> 00:57:53.580 align:middle line:84% is very difficult. Whereas this thing, 00:57:53.580 --> 00:57:57.360 align:middle line:84% you can kind of usually generically do. 00:57:57.360 --> 00:57:58.860 align:middle line:84% Not always, you have to-- there are 00:57:58.860 --> 00:58:01.235 align:middle line:84% conditions for making sure that your model is identified, 00:58:01.235 --> 00:58:05.670 align:middle line:84% which is this idea that you have that you in fact can recover, 00:58:05.670 --> 00:58:08.373 align:middle line:84% that it's not the case that multiple different combinations 00:58:08.373 --> 00:58:10.290 align:middle line:84% of parameters would give you the same moments. 00:58:10.290 --> 00:58:12.340 align:middle line:84% That's an identification problem. 00:58:12.340 --> 00:58:14.100 align:middle line:84% But other than that, there's lots 00:58:14.100 --> 00:58:15.892 align:middle line:84% of cases in which you can actually do this. 00:58:15.892 --> 00:58:16.710 align:middle line:90% Sorry, go ahead. 00:58:16.710 --> 00:58:19.076 align:middle line:84% AUDIENCE: Yeah just trying to understand, 00:58:19.076 --> 00:58:21.040 align:middle line:84% what you think about when you need 00:58:21.040 --> 00:58:23.850 align:middle line:90% experimental or [INAUDIBLE]? 00:58:23.850 --> 00:58:25.350 align:middle line:84% BEN OLKEN: That is a great question. 00:58:25.350 --> 00:58:28.250 align:middle line:84% So where do the moments come from? 00:58:28.250 --> 00:58:30.000 align:middle line:90% And are the moments-- 00:58:30.000 --> 00:58:31.760 align:middle line:90% so I think that-- 00:58:31.760 --> 00:58:34.145 align:middle line:84% so the way to think about that I think is that your-- 00:58:34.145 --> 00:58:45.718 align:middle line:90% 00:58:45.718 --> 00:58:48.135 align:middle line:84% Let me see if I can craft a good answer for this question. 00:58:48.135 --> 00:58:57.360 align:middle line:90% 00:58:57.360 --> 00:59:02.442 align:middle line:90% In general, if you have a-- 00:59:02.442 --> 00:59:04.400 align:middle line:84% let me see if I can come up with a good answer, 00:59:04.400 --> 00:59:05.358 align:middle line:90% sorry, hold one second. 00:59:05.358 --> 00:59:15.960 align:middle line:90% 00:59:15.960 --> 00:59:19.590 align:middle line:84% As I'm changing things in the moment distribution-- 00:59:19.590 --> 00:59:22.470 align:middle line:84% so imagine I'm computing to different moments 00:59:22.470 --> 00:59:25.680 align:middle line:84% for tall people and short people for example. 00:59:25.680 --> 00:59:29.430 align:middle line:84% In the model, I'm going to assert 00:59:29.430 --> 00:59:31.380 align:middle line:84% that the only differences between these people 00:59:31.380 --> 00:59:33.710 align:middle line:90% are short versus tall. 00:59:33.710 --> 00:59:36.860 align:middle line:84% And in the data, I'm going to just 00:59:36.860 --> 00:59:42.920 align:middle line:84% observe short people and tall people, short and tall, right? 00:59:42.920 --> 00:59:46.250 align:middle line:84% If that variation between short and tall 00:59:46.250 --> 00:59:49.400 align:middle line:90% is essentially randomly-- 00:59:49.400 --> 00:59:50.960 align:middle line:84% imagine that was randomized, then 00:59:50.960 --> 00:59:53.143 align:middle line:84% in fact, those differences I'm going to observe, 00:59:53.143 --> 00:59:54.810 align:middle line:84% kind of those real moments, are in fact, 00:59:54.810 --> 00:59:57.018 align:middle line:84% going to be the true things that I should be plugging 00:59:57.018 --> 00:59:59.500 align:middle line:84% into the model because it's going to be the true, 00:59:59.500 --> 01:00:02.680 align:middle line:84% the causal difference between moving from short to tall. 01:00:02.680 --> 01:00:04.180 align:middle line:84% Whereas in the real world, you might 01:00:04.180 --> 01:00:05.780 align:middle line:84% imagine that short is correlated with other things. 01:00:05.780 --> 01:00:07.790 align:middle line:84% And tall is correlated with other things. 01:00:07.790 --> 01:00:11.080 align:middle line:84% And so those differences between the short and the tall 01:00:11.080 --> 01:00:13.180 align:middle line:84% may not just reflect kind of the short differences 01:00:13.180 --> 01:00:15.180 align:middle line:84% and the tall differences in the modeling respect 01:00:15.180 --> 01:00:17.510 align:middle line:84% kind of other unobserved characteristics. 01:00:17.510 --> 01:00:22.750 align:middle line:84% So in the same way that analyzing kind of a reduced 01:00:22.750 --> 01:00:25.360 align:middle line:84% form thing, you might be worried about omitted variable bias, 01:00:25.360 --> 01:00:27.110 align:middle line:84% you could be worried about the same things 01:00:27.110 --> 01:00:28.480 align:middle line:90% in structural estimation. 01:00:28.480 --> 01:00:33.820 align:middle line:84% In practice, people don't always worry 01:00:33.820 --> 01:00:36.220 align:middle line:90% about that in the same way. 01:00:36.220 --> 01:00:42.670 align:middle line:84% But I think in general, it's the same kind of concerns. 01:00:42.670 --> 01:00:50.410 align:middle line:84% And so as you can-- as you actually see in this paper, 01:00:50.410 --> 01:00:51.910 align:middle line:84% a lot of what we're trying to do are 01:00:51.910 --> 01:00:54.610 align:middle line:84% things identify the model on things that look 01:00:54.610 --> 01:00:56.668 align:middle line:90% kind of experimental movements. 01:00:56.668 --> 01:00:58.710 align:middle line:84% AUDIENCE: So kind of when you're writing a model, 01:00:58.710 --> 01:01:00.830 align:middle line:84% you're solving theoretical [INAUDIBLE] should 01:01:00.830 --> 01:01:04.228 align:middle line:84% be thinking about what assumptions [INAUDIBLE].. 01:01:04.228 --> 01:01:06.520 align:middle line:84% BEN OLKEN: No, sorry, I don't understand the last part. 01:01:06.520 --> 01:01:09.170 align:middle line:84% AUDIENCE: Like as in the time in which you're 01:01:09.170 --> 01:01:14.087 align:middle line:84% thinking about where you'd need experimental variation, that's 01:01:14.087 --> 01:01:16.142 align:middle line:84% when you're solving the model, you're 01:01:16.142 --> 01:01:17.725 align:middle line:84% [? thinking ?] what assumptions you're 01:01:17.725 --> 01:01:21.120 align:middle line:84% making, like in the kind of functional forms 01:01:21.120 --> 01:01:22.162 align:middle line:90% you're choosing. 01:01:22.162 --> 01:01:24.120 align:middle line:84% BEN OLKEN: Yeah, or another way of saying it is 01:01:24.120 --> 01:01:25.540 align:middle line:84% is like you want to say like what-- 01:01:25.540 --> 01:01:27.290 align:middle line:84% I mean another way of thinking about it is 01:01:27.290 --> 01:01:28.780 align:middle line:90% like from a design perspective. 01:01:28.780 --> 01:01:29.550 align:middle line:90% So here's a model. 01:01:29.550 --> 01:01:31.650 align:middle line:84% Here are the key unknown parameters 01:01:31.650 --> 01:01:32.700 align:middle line:90% I'd like to identify. 01:01:32.700 --> 01:01:35.940 align:middle line:84% What you want are you would like to have experimental variation 01:01:35.940 --> 01:01:40.320 align:middle line:84% in the real world that if I ran the same experiment in my model 01:01:40.320 --> 01:01:41.970 align:middle line:84% would give me really different answers 01:01:41.970 --> 01:01:45.820 align:middle line:84% depending on the parameter vector. 01:01:45.820 --> 01:01:49.380 align:middle line:84% So if I really want to identify-- 01:01:49.380 --> 01:01:51.670 align:middle line:84% that's I think the best way to think about it. 01:01:51.670 --> 01:01:54.900 align:middle line:84% So if I really want to identify some parameter, 01:01:54.900 --> 01:01:56.770 align:middle line:90% I would take my model. 01:01:56.770 --> 01:01:59.760 align:middle line:84% And I would say, well, what if I change this in the world would 01:01:59.760 --> 01:02:03.120 align:middle line:84% I get really different answers depending on different levels 01:02:03.120 --> 01:02:04.560 align:middle line:90% of that parameter vector? 01:02:04.560 --> 01:02:06.690 align:middle line:84% And that's going to say that this experiment is 01:02:06.690 --> 01:02:11.490 align:middle line:84% going to intuitively identify that parameter. 01:02:11.490 --> 01:02:15.405 align:middle line:84% And so I think technically what this is if you-- it's the-- 01:02:15.405 --> 01:02:18.830 align:middle line:90% 01:02:18.830 --> 01:02:19.830 align:middle line:90% I can never remember it. 01:02:19.830 --> 01:02:26.840 align:middle line:84% I think it's-- what you want is like d theta hat d moment, 01:02:26.840 --> 01:02:28.370 align:middle line:90% moment i. 01:02:28.370 --> 01:02:32.615 align:middle line:84% So you want to choose experimental moments, 01:02:32.615 --> 01:02:33.990 align:middle line:84% things are going to actually vary 01:02:33.990 --> 01:02:37.350 align:middle line:84% in the real world that are going to give you-- 01:02:37.350 --> 01:02:43.407 align:middle line:84% that are going to identify the parameter vector theta 01:02:43.407 --> 01:02:44.490 align:middle line:90% that you're interested in. 01:02:44.490 --> 01:02:46.507 align:middle line:90% 01:02:46.507 --> 01:02:49.090 align:middle line:84% And by the way, Ed, if you want to add any commentary on this, 01:02:49.090 --> 01:02:51.007 align:middle line:84% since you just did a whole recitation on this, 01:02:51.007 --> 01:02:54.077 align:middle line:90% I'm open to suggestions. 01:02:54.077 --> 01:02:56.160 align:middle line:84% AUDIENCE: Just to give you an objective point that 01:02:56.160 --> 01:02:58.463 align:middle line:84% is not that different ways of thinking about it. 01:02:58.463 --> 01:02:59.130 align:middle line:90% BEN OLKEN: Yeah. 01:02:59.130 --> 01:03:01.890 align:middle line:90% 01:03:01.890 --> 01:03:03.510 align:middle line:90% OK, other questions? 01:03:03.510 --> 01:03:04.260 align:middle line:84% AUDIENCE: I would say in the [? Berquist ?] 01:03:04.260 --> 01:03:05.780 align:middle line:84% and [? Bennerstein ?] papers [INAUDIBLE].. 01:03:05.780 --> 01:03:05.960 align:middle line:90% BEN OLKEN: What? 01:03:05.960 --> 01:03:06.320 align:middle line:90% Sorry. 01:03:06.320 --> 01:03:08.445 align:middle line:84% AUDIENCE: In the recitation, we did [? Bergquist ?] 01:03:08.445 --> 01:03:12.320 align:middle line:84% and [? Bennerstein. ?] And there were exactly two specific 01:03:12.320 --> 01:03:15.220 align:middle line:84% objects that you needed to estimate and design 01:03:15.220 --> 01:03:15.910 align:middle line:90% the experiment. 01:03:15.910 --> 01:03:17.590 align:middle line:84% BEN OLKEN: And the experiment estimate them-- 01:03:17.590 --> 01:03:18.130 align:middle line:90% yeah. 01:03:18.130 --> 01:03:21.730 align:middle line:84% Another paper was also kind of around the same time 01:03:21.730 --> 01:03:25.375 align:middle line:84% is Gabriel Kreindler's job market paper from when 01:03:25.375 --> 01:03:28.120 align:middle line:84% he was a student here, which is a similar feature of he 01:03:28.120 --> 01:03:30.185 align:middle line:84% wrote down-- this is a transportation paper. 01:03:30.185 --> 01:03:31.810 align:middle line:84% So we're not teaching it in this class. 01:03:31.810 --> 01:03:36.607 align:middle line:84% But had a couple-- the paper had a couple-- 01:03:36.607 --> 01:03:38.690 align:middle line:84% he had a model and a couple of unknown parameters. 01:03:38.690 --> 01:03:41.360 align:middle line:84% And he said, look, I really want to go 01:03:41.360 --> 01:03:43.675 align:middle line:84% design an experiment that's going to really give me 01:03:43.675 --> 01:03:45.050 align:middle line:84% variation that I need to estimate 01:03:45.050 --> 01:03:46.160 align:middle line:90% these unknown parameters. 01:03:46.160 --> 01:03:48.882 align:middle line:84% And so it was kind of a case where he wrote down the model, 01:03:48.882 --> 01:03:50.840 align:middle line:84% figured out what kind of the unknown parameters 01:03:50.840 --> 01:03:52.400 align:middle line:84% were, and then designed an experiment that was explicitly 01:03:52.400 --> 01:03:53.900 align:middle line:84% going to get that kind of variation. 01:03:53.900 --> 01:03:59.250 align:middle line:84% So that's another nice one that is 01:03:59.250 --> 01:04:02.850 align:middle line:84% taking this relationship between model structural estimation 01:04:02.850 --> 01:04:04.830 align:middle line:84% to the ex ante phase and designing experiments 01:04:04.830 --> 01:04:06.390 align:middle line:90% that's going to estimate-- 01:04:06.390 --> 01:04:08.490 align:middle line:84% that's going to generate the variation you need 01:04:08.490 --> 01:04:09.835 align:middle line:90% to estimate the model clearly. 01:04:09.835 --> 01:04:11.898 align:middle line:90% 01:04:11.898 --> 01:04:14.190 align:middle line:84% AUDIENCE: I don't know if this falls in the explanation 01:04:14.190 --> 01:04:16.770 align:middle line:90% part of the article part. 01:04:16.770 --> 01:04:19.200 align:middle line:84% You mentioned that, when you [? illustrate ?] a model, 01:04:19.200 --> 01:04:22.050 align:middle line:84% you found that ordeal mechanism size wasn't the big deal 01:04:22.050 --> 01:04:24.250 align:middle line:84% and it was actually the probability. 01:04:24.250 --> 01:04:26.250 align:middle line:84% BEN OLKEN: Where does that come from this slide. 01:04:26.250 --> 01:04:26.950 align:middle line:90% Not on this slide. 01:04:26.950 --> 01:04:27.760 align:middle line:90% It's going to come next. 01:04:27.760 --> 01:04:28.800 align:middle line:90% AUDIENCE: OK, all right. 01:04:28.800 --> 01:04:30.330 align:middle line:84% But it's like could you have written down 01:04:30.330 --> 01:04:31.530 align:middle line:84% the model in a way that didn't allow 01:04:31.530 --> 01:04:32.738 align:middle line:90% you to distinguish those two. 01:04:32.738 --> 01:04:35.368 align:middle line:84% How did you know to write down the model in the way 01:04:35.368 --> 01:04:38.644 align:middle line:84% that you were able to [INAUDIBLE]?? 01:04:38.644 --> 01:04:42.090 align:middle line:90% 01:04:42.090 --> 01:04:44.850 align:middle line:84% BEN OLKEN: So what do you do once you have the model? 01:04:44.850 --> 01:04:46.590 align:middle line:84% I haven't forgotten your question. 01:04:46.590 --> 01:04:48.430 align:middle line:84% What do you do once you have the-- 01:04:48.430 --> 01:04:50.430 align:middle line:84% this is more about the calibration of the model. 01:04:50.430 --> 01:04:52.170 align:middle line:90% I'm going to actually-- 01:04:52.170 --> 01:04:54.075 align:middle line:90% this is all how do we-- 01:04:54.075 --> 01:04:58.350 align:middle line:90% 01:04:58.350 --> 01:05:01.410 align:middle line:84% oh, no, sorry, this is actually relevant. 01:05:01.410 --> 01:05:02.440 align:middle line:90% So what do we do? 01:05:02.440 --> 01:05:04.590 align:middle line:90% So now we have the model. 01:05:04.590 --> 01:05:06.678 align:middle line:84% Now we can do different counterfactuals. 01:05:06.678 --> 01:05:08.220 align:middle line:84% So what does that mean is we can take 01:05:08.220 --> 01:05:10.710 align:middle line:84% the model with the parameters and start 01:05:10.710 --> 01:05:13.560 align:middle line:84% changing different things about the model and see 01:05:13.560 --> 01:05:17.280 align:middle line:84% how would the world behave in different counterfactual 01:05:17.280 --> 01:05:18.420 align:middle line:90% worlds. 01:05:18.420 --> 01:05:22.020 align:middle line:84% So the first thing we did is we were like, OK, 01:05:22.020 --> 01:05:24.010 align:middle line:84% we think it's this whole different technology-- 01:05:24.010 --> 01:05:25.050 align:middle line:90% so the first-- 01:05:25.050 --> 01:05:26.010 align:middle line:90% backing up. 01:05:26.010 --> 01:05:27.570 align:middle line:90% We found selection. 01:05:27.570 --> 01:05:29.340 align:middle line:84% We didn't find very much between the close 01:05:29.340 --> 01:05:30.360 align:middle line:90% and the far treatment. 01:05:30.360 --> 01:05:31.110 align:middle line:90% That was weird. 01:05:31.110 --> 01:05:32.610 align:middle line:84% We were like expecting to find that. 01:05:32.610 --> 01:05:33.600 align:middle line:90% We didn't find that. 01:05:33.600 --> 01:05:36.280 align:middle line:84% So then we were like, what's going on here? 01:05:36.280 --> 01:05:37.980 align:middle line:84% So our first hypothesis was this whole 01:05:37.980 --> 01:05:40.830 align:middle line:84% like different technologies for dealing with the ordeal. 01:05:40.830 --> 01:05:42.080 align:middle line:90% The rich take the bus. 01:05:42.080 --> 01:05:44.700 align:middle line:84% The rich the rich take the bus or take the motorcycle 01:05:44.700 --> 01:05:45.600 align:middle line:90% and the poor walk. 01:05:45.600 --> 01:05:47.760 align:middle line:90% That was kind of our idea. 01:05:47.760 --> 01:05:52.263 align:middle line:84% So we're like OK, let's play with that in the model. 01:05:52.263 --> 01:05:53.930 align:middle line:84% We didn't actually have random variation 01:05:53.930 --> 01:05:56.150 align:middle line:84% in like forcing them to take the same technology. 01:05:56.150 --> 01:05:57.650 align:middle line:84% We probably couldn't have done that. 01:05:57.650 --> 01:05:59.330 align:middle line:90% But we said, well, can this be-- 01:05:59.330 --> 01:06:02.600 align:middle line:84% clearly and this was kind of qualitatively in there, 01:06:02.600 --> 01:06:04.850 align:middle line:84% in terms of how did you actually get to the place, 01:06:04.850 --> 01:06:06.308 align:middle line:84% or how would you get to this place. 01:06:06.308 --> 01:06:07.920 align:middle line:84% And they were doing different things. 01:06:07.920 --> 01:06:10.740 align:middle line:84% But is this large enough to actually make a difference? 01:06:10.740 --> 01:06:12.560 align:middle line:90% So what do we do? 01:06:12.560 --> 01:06:16.610 align:middle line:84% We said, OK, let's parameterize this travel cost thing. 01:06:16.610 --> 01:06:20.953 align:middle line:84% And let's rerun the model taking this thing out. 01:06:20.953 --> 01:06:22.370 align:middle line:84% So we're going to rerun the model, 01:06:22.370 --> 01:06:24.953 align:middle line:84% but we're going to give everyone the exact same transportation 01:06:24.953 --> 01:06:25.850 align:middle line:90% technology. 01:06:25.850 --> 01:06:29.152 align:middle line:84% And the answer was it made no difference, essentially. 01:06:29.152 --> 01:06:31.610 align:middle line:84% And that's how we concluded that this thing, even though it 01:06:31.610 --> 01:06:33.950 align:middle line:84% was qualitatively in the direction we were thinking 01:06:33.950 --> 01:06:35.930 align:middle line:84% about, was not quantitatively large enough 01:06:35.930 --> 01:06:41.120 align:middle line:84% to explain anything, at least in the context of our model. 01:06:41.120 --> 01:06:44.450 align:middle line:84% On the other hand, we took the exact same model 01:06:44.450 --> 01:06:49.853 align:middle line:84% and we took out lambda and put in lambda bar. 01:06:49.853 --> 01:06:52.020 align:middle line:84% So we say let's assume everyone has the same beliefs 01:06:52.020 --> 01:06:53.520 align:middle line:90% about passing the thing. 01:06:53.520 --> 01:06:56.780 align:middle line:84% And that basically explains everything. 01:06:56.780 --> 01:07:00.200 align:middle line:84% So this was quantitatively-- so both of these 01:07:00.200 --> 01:07:02.690 align:middle line:84% were like qualitatively in the right direction. 01:07:02.690 --> 01:07:05.300 align:middle line:84% But we needed some way of quantifying kind of which 01:07:05.300 --> 01:07:07.890 align:middle line:84% of these different things were large or small. 01:07:07.890 --> 01:07:09.890 align:middle line:84% And so by taking the estimated model 01:07:09.890 --> 01:07:12.523 align:middle line:84% and rerunning it under different kind of counterfactuals, 01:07:12.523 --> 01:07:14.690 align:middle line:84% we could say, well, which of these different factors 01:07:14.690 --> 01:07:15.650 align:middle line:90% are really important? 01:07:15.650 --> 01:07:16.610 align:middle line:84% Because they're all kind of in there. 01:07:16.610 --> 01:07:18.140 align:middle line:84% And you take them out kind of one by one in the model. 01:07:18.140 --> 01:07:20.020 align:middle line:84% And that's how we figured it out. 01:07:20.020 --> 01:07:25.310 align:middle line:84% And this is what I was saying of like we didn't know this, 01:07:25.310 --> 01:07:26.770 align:middle line:90% but really we didn't know. 01:07:26.770 --> 01:07:30.400 align:middle line:84% We really wrote a paper like a PowerPoint presentation 01:07:30.400 --> 01:07:32.537 align:middle line:84% or a Beamer presentation somewhere on my computer, 01:07:32.537 --> 01:07:34.120 align:middle line:84% which is all about this one because it 01:07:34.120 --> 01:07:35.260 align:middle line:90% was like qualitatively there. 01:07:35.260 --> 01:07:36.340 align:middle line:84% And we actually had evidence showing 01:07:36.340 --> 01:07:37.660 align:middle line:84% they were taking different technologies. 01:07:37.660 --> 01:07:38.810 align:middle line:90% It was really great. 01:07:38.810 --> 01:07:40.510 align:middle line:84% It just turned out that once we wrote down the model, 01:07:40.510 --> 01:07:43.093 align:middle line:84% we're like, oh, this is like two orders of magnitude too small 01:07:43.093 --> 01:07:44.940 align:middle line:84% or something to explain anything. 01:07:44.940 --> 01:07:49.590 align:middle line:90% So that was the point. 01:07:49.590 --> 01:07:51.685 align:middle line:90% Other comments? 01:07:51.685 --> 01:07:55.970 align:middle line:90% 01:07:55.970 --> 01:07:58.640 align:middle line:84% And so this shows you a different counterfactuals 01:07:58.640 --> 01:07:59.370 align:middle line:90% or whatever. 01:07:59.370 --> 01:08:02.310 align:middle line:84% So for example, this is the coefficient on log consumption. 01:08:02.310 --> 01:08:06.110 align:middle line:84% This is the true kind of experimental coefficient, 01:08:06.110 --> 01:08:09.230 align:middle line:84% whatever consumption negatively predicts whether you apply. 01:08:09.230 --> 01:08:10.868 align:middle line:90% This is the estimated model. 01:08:10.868 --> 01:08:13.160 align:middle line:84% It's also kind of similarly negative, not so surprising 01:08:13.160 --> 01:08:14.570 align:middle line:84% because we fit the model to the data. 01:08:14.570 --> 01:08:15.740 align:middle line:84% Right it's not so surprising there. 01:08:15.740 --> 01:08:18.407 align:middle line:84% But then we can see what happens we turned different things off. 01:08:18.407 --> 01:08:21.245 align:middle line:84% And like for example, if you turn off the-- 01:08:21.245 --> 01:08:23.120 align:middle line:84% if you give everyone the same kind of beliefs 01:08:23.120 --> 01:08:24.495 align:middle line:84% about the probability of passing, 01:08:24.495 --> 01:08:26.060 align:middle line:84% this thing just totally goes away. 01:08:26.060 --> 01:08:27.518 align:middle line:84% But you give everyone the same kind 01:08:27.518 --> 01:08:29.689 align:middle line:84% of technology of traveling that [INAUDIBLE] 01:08:29.689 --> 01:08:32.370 align:middle line:90% makes much of a difference. 01:08:32.370 --> 01:08:35.040 align:middle line:90% OK, any other-- 01:08:35.040 --> 01:08:36.930 align:middle line:84% I'm going a bit slower than I had hoped-- 01:08:36.930 --> 01:08:40.520 align:middle line:84% but any other-- guys had lots of great comments. 01:08:40.520 --> 01:08:42.840 align:middle line:84% Oh, so let me hit a couple of the questions 01:08:42.840 --> 01:08:45.330 align:middle line:84% that you guys raised from your response on this. 01:08:45.330 --> 01:08:47.700 align:middle line:84% So one was, I think we talked about how do you 01:08:47.700 --> 01:08:48.569 align:middle line:90% pick the moments. 01:08:48.569 --> 01:08:50.979 align:middle line:84% You want to pick moments where this is going to be true, 01:08:50.979 --> 01:08:52.920 align:middle line:84% and the moments you think are kind of well identified, 01:08:52.920 --> 01:08:54.712 align:middle line:84% where there's the difference in the moments 01:08:54.712 --> 01:08:56.880 align:middle line:84% is not correlated kind of [INAUDIBLE] ideally. 01:08:56.880 --> 01:09:01.670 align:middle line:90% 01:09:01.670 --> 01:09:04.064 align:middle line:84% Are there any other questions that you guys-- 01:09:04.064 --> 01:09:05.689 align:middle line:84% we talked about the exclusionary point. 01:09:05.689 --> 01:09:06.800 align:middle line:84% Are there any other questions that you guys 01:09:06.800 --> 01:09:08.510 align:middle line:84% wanted to raise from your reading about this? 01:09:08.510 --> 01:09:09.649 align:middle line:90% I know it was like Wednesday. 01:09:09.649 --> 01:09:10.149 align:middle line:90% Sorry. 01:09:10.149 --> 01:09:13.160 align:middle line:90% 01:09:13.160 --> 01:09:16.192 align:middle line:84% Any other comments on this paper or questions? 01:09:16.192 --> 01:09:17.859 align:middle line:84% So now you're going to actually practice 01:09:17.859 --> 01:09:19.120 align:middle line:90% doing this on the problem set. 01:09:19.120 --> 01:09:23.140 align:middle line:90% 01:09:23.140 --> 01:09:26.590 align:middle line:84% All right, so what I also want to say about redistribution, 01:09:26.590 --> 01:09:27.200 align:middle line:90% a few things. 01:09:27.200 --> 01:09:35.620 align:middle line:90% 01:09:35.620 --> 01:09:39.492 align:middle line:84% So one is now that we know something about targeting, 01:09:39.492 --> 01:09:41.200 align:middle line:84% how do we think about welfare under this? 01:09:41.200 --> 01:09:45.859 align:middle line:84% How do we actually make some welfare judgments on this? 01:09:45.859 --> 01:09:50.470 align:middle line:84% So in order to do that, you have to take a stand 01:09:50.470 --> 01:09:53.200 align:middle line:84% on how do you think about exclusion error and inclusion 01:09:53.200 --> 01:09:54.160 align:middle line:90% error. 01:09:54.160 --> 01:09:56.358 align:middle line:84% And you have to be specific about writing 01:09:56.358 --> 01:09:58.150 align:middle line:84% on a model of how you think about those two 01:09:58.150 --> 01:09:59.317 align:middle line:90% different kinds of mistakes. 01:09:59.317 --> 01:10:01.780 align:middle line:84% So here's one example of how we did it. 01:10:01.780 --> 01:10:04.390 align:middle line:84% But you may choose different welfare functions. 01:10:04.390 --> 01:10:06.550 align:middle line:84% But the basic point is you have to write down 01:10:06.550 --> 01:10:07.570 align:middle line:90% a welfare function. 01:10:07.570 --> 01:10:12.370 align:middle line:84% And you also have to specify how your programs change. 01:10:12.370 --> 01:10:14.410 align:middle line:84% So we made the following assumptions. 01:10:14.410 --> 01:10:17.557 align:middle line:84% We said let's suppose everyone has CRRA utility where 01:10:17.557 --> 01:10:20.140 align:middle line:84% this is your baseline income and this is the benefits you get. 01:10:20.140 --> 01:10:22.090 align:middle line:84% And we'll assume some parameter value. 01:10:22.090 --> 01:10:24.130 align:middle line:84% I think we do three I think in the paper. 01:10:24.130 --> 01:10:24.820 align:middle line:90% Maybe we do two. 01:10:24.820 --> 01:10:27.220 align:middle line:84% I actually don't remember, two or three. 01:10:27.220 --> 01:10:30.143 align:middle line:84% And we're also going to assume a fixed budget b. 01:10:30.143 --> 01:10:31.560 align:middle line:84% So there's a fixed amount of money 01:10:31.560 --> 01:10:33.477 align:middle line:84% the government wants to spend on this program. 01:10:33.477 --> 01:10:36.930 align:middle line:84% And we're going to say as we make the program more narrowly 01:10:36.930 --> 01:10:38.742 align:middle line:84% targeted, fewer people are going to get it. 01:10:38.742 --> 01:10:40.200 align:middle line:84% But the amount they're going to get 01:10:40.200 --> 01:10:41.775 align:middle line:84% is going to go up proportionately. 01:10:41.775 --> 01:10:44.640 align:middle line:90% 01:10:44.640 --> 01:10:50.250 align:middle line:84% And we're also going to use the actual kind of proxy means 01:10:50.250 --> 01:10:56.280 align:middle line:84% tests to think about exclusion inclusion error. 01:10:56.280 --> 01:10:58.905 align:middle line:84% So what I mean by that is we go way back to-- 01:10:58.905 --> 01:11:03.100 align:middle line:90% 01:11:03.100 --> 01:11:13.700 align:middle line:84% way back, way back, further back, here. 01:11:13.700 --> 01:11:21.140 align:middle line:84% So as I-- basically we're going to say, look, 01:11:21.140 --> 01:11:23.970 align:middle line:84% this is a fundamental technology that we're dealing with here, 01:11:23.970 --> 01:11:25.880 align:middle line:84% which says, here's your predicted consumption 01:11:25.880 --> 01:11:27.350 align:middle line:84% as a function of your actual consumption. 01:11:27.350 --> 01:11:29.360 align:middle line:84% And what different programs look like is we're 01:11:29.360 --> 01:11:33.203 align:middle line:84% going to draw the line in different places. 01:11:33.203 --> 01:11:35.870 align:middle line:84% So we can have a really narrowly targeted program drawing a line 01:11:35.870 --> 01:11:40.730 align:middle line:84% here, like a universal program draws the line over here. 01:11:40.730 --> 01:11:42.560 align:middle line:84% The government can't necessarily-- 01:11:42.560 --> 01:11:45.440 align:middle line:84% we're going to hold this cloud as fixed and we're 01:11:45.440 --> 01:11:48.800 align:middle line:84% going to see what happens as we kind of move the line to make 01:11:48.800 --> 01:11:51.200 align:middle line:84% it more or less narrowly targeted, 01:11:51.200 --> 01:11:52.720 align:middle line:84% holding this technology as fixed. 01:11:52.720 --> 01:11:56.040 align:middle line:90% 01:11:56.040 --> 01:11:57.415 align:middle line:84% Now I have to zoom forward again. 01:11:57.415 --> 01:12:02.040 align:middle line:90% 01:12:02.040 --> 01:12:02.700 align:middle line:90% Almost there. 01:12:02.700 --> 01:12:07.530 align:middle line:90% 01:12:07.530 --> 01:12:11.160 align:middle line:90% OK, so that's what we're going-- 01:12:11.160 --> 01:12:12.794 align:middle line:90% we kind of run a counterfactual. 01:12:12.794 --> 01:12:16.042 align:middle line:90% 01:12:16.042 --> 01:12:19.660 align:middle line:84% What we then do is say, well, what is the total kind of-- 01:12:19.660 --> 01:12:22.120 align:middle line:84% under this welfare function, which is-- 01:12:22.120 --> 01:12:25.870 align:middle line:84% you can make a different welfare function-- 01:12:25.870 --> 01:12:28.540 align:middle line:84% what is the total social utility taking these 01:12:28.540 --> 01:12:29.710 align:middle line:90% into consideration. 01:12:29.710 --> 01:12:31.840 align:middle line:84% And in this case-- so, sorry, we're 01:12:31.840 --> 01:12:35.150 align:middle line:84% going to plot this as a function of inclusion error. 01:12:35.150 --> 01:12:37.660 align:middle line:84% So how much-- this is the most inclusive program 01:12:37.660 --> 01:12:39.462 align:middle line:84% this is the least inclusive program. 01:12:39.462 --> 01:12:41.170 align:middle line:84% Maybe we should have chosen a different-- 01:12:41.170 --> 01:12:43.660 align:middle line:84% we chose this to make it look like an ROC curve 01:12:43.660 --> 01:12:44.800 align:middle line:90% in statistics. 01:12:44.800 --> 01:12:47.050 align:middle line:84% But maybe we could have chosen different axes. 01:12:47.050 --> 01:12:49.270 align:middle line:84% The point I want to make is in this example, 01:12:49.270 --> 01:12:52.390 align:middle line:84% it turns out kind of relatively narrowly targeted programs 01:12:52.390 --> 01:12:55.750 align:middle line:84% look kind of welfare optimal because you're targeting-- 01:12:55.750 --> 01:12:57.520 align:middle line:84% even though you have more exclusion error, 01:12:57.520 --> 01:13:02.428 align:middle line:84% you're targeting so much more to the poor in this simulation. 01:13:02.428 --> 01:13:03.970 align:middle line:84% But you could do it in different way. 01:13:03.970 --> 01:13:05.512 align:middle line:84% The more general point I want to make 01:13:05.512 --> 01:13:08.287 align:middle line:84% is that you do have to-- if you're making some trade offs, 01:13:08.287 --> 01:13:10.120 align:middle line:84% you have to take a stand on how you evaluate 01:13:10.120 --> 01:13:11.162 align:middle line:90% the different trade offs. 01:13:11.162 --> 01:13:13.510 align:middle line:84% And then you can evaluate your decision 01:13:13.510 --> 01:13:15.310 align:middle line:84% based on that welfare assumption. 01:13:15.310 --> 01:13:17.230 align:middle line:84% And you can choose, as I said, like if you're 01:13:17.230 --> 01:13:22.180 align:middle line:84% going to choose as we increase row here and put more and more 01:13:22.180 --> 01:13:23.680 align:middle line:84% and more weight on the very poorest, 01:13:23.680 --> 01:13:25.653 align:middle line:84% that trade off may end up looking different. 01:13:25.653 --> 01:13:27.070 align:middle line:84% So you can do different-- you have 01:13:27.070 --> 01:13:28.695 align:middle line:84% to take a stand on the welfare function 01:13:28.695 --> 01:13:31.243 align:middle line:84% and then you can say some things. 01:13:31.243 --> 01:13:32.660 align:middle line:84% The other thing I'll note, though, 01:13:32.660 --> 01:13:35.440 align:middle line:84% is that you have the other problem 01:13:35.440 --> 01:13:37.690 align:middle line:84% here, which is not captured in this utility function, 01:13:37.690 --> 01:13:40.030 align:middle line:90% is horizontal inequity. 01:13:40.030 --> 01:13:43.417 align:middle line:84% So this is the fact that different like people, inequity 01:13:43.417 --> 01:13:45.250 align:middle line:84% is the probability that people who are alike 01:13:45.250 --> 01:13:46.850 align:middle line:90% are being treated differently. 01:13:46.850 --> 01:13:47.902 align:middle line:90% So we compute that. 01:13:47.902 --> 01:13:49.360 align:middle line:84% What is the probability that people 01:13:49.360 --> 01:13:52.840 align:middle line:90% who are treated kind of-- 01:13:52.840 --> 01:13:54.933 align:middle line:90% who are about the same-- 01:13:54.933 --> 01:13:57.100 align:middle line:84% sorry-- about the same in terms of their consumption 01:13:57.100 --> 01:13:58.090 align:middle line:90% get different outcomes. 01:13:58.090 --> 01:14:01.210 align:middle line:84% And actually these narrowly targeted programs, they 01:14:01.210 --> 01:14:02.763 align:middle line:84% have a lot of horizontal inequity 01:14:02.763 --> 01:14:04.930 align:middle line:84% precisely because you're targeting in the thick part 01:14:04.930 --> 01:14:07.420 align:middle line:90% of that cloud of the PMT. 01:14:07.420 --> 01:14:09.020 align:middle line:84% And so you're making a lot of errors. 01:14:09.020 --> 01:14:11.110 align:middle line:84% And so I think in general, you want 01:14:11.110 --> 01:14:12.905 align:middle line:84% to think about, from just maximizing 01:14:12.905 --> 01:14:15.280 align:middle line:84% kind of utilitarian social welfare metric, these narrowly 01:14:15.280 --> 01:14:16.360 align:middle line:90% targeted programs are great. 01:14:16.360 --> 01:14:18.580 align:middle line:84% But they have this real problem of horizontal inequity. 01:14:18.580 --> 01:14:19.870 align:middle line:84% And I think one of the advantages 01:14:19.870 --> 01:14:21.010 align:middle line:84% of the community-based approach is 01:14:21.010 --> 01:14:22.540 align:middle line:84% it can help reduce some of the challenges 01:14:22.540 --> 01:14:24.748 align:middle line:84% with horizontal inequity because at least people have 01:14:24.748 --> 01:14:27.820 align:middle line:84% some local buy in as to who's getting the programs 01:14:27.820 --> 01:14:29.870 align:middle line:90% and who is not. 01:14:29.870 --> 01:14:33.650 align:middle line:84% OK, so that's what I wanted to say about targeting. 01:14:33.650 --> 01:14:36.910 align:middle line:90% Any last questions on that? 01:14:36.910 --> 01:14:39.880 align:middle line:84% So I wanted to say a little bit about the form of transfers. 01:14:39.880 --> 01:14:42.890 align:middle line:84% And then I guess I we'll talk about tax starting next time. 01:14:42.890 --> 01:14:45.070 align:middle line:84% So there's now like a whole big literature 01:14:45.070 --> 01:14:50.680 align:middle line:84% about how do you think about designing transfer programs 01:14:50.680 --> 01:14:54.130 align:middle line:90% in developing country contexts? 01:14:54.130 --> 01:14:56.510 align:middle line:84% And how do you think about those issues? 01:14:56.510 --> 01:15:00.520 align:middle line:84% And so I guess I wanted to highlight I think two issues. 01:15:00.520 --> 01:15:02.410 align:middle line:84% And then a third one is this whole question 01:15:02.410 --> 01:15:03.730 align:middle line:84% of poverty traps, which Esther talked 01:15:03.730 --> 01:15:04.980 align:middle line:90% about earlier in the semester. 01:15:04.980 --> 01:15:07.880 align:middle line:84% So I'm not going to talk about that. 01:15:07.880 --> 01:15:10.690 align:middle line:84% But there's a bunch of questions like, should you give cash? 01:15:10.690 --> 01:15:12.070 align:middle line:90% Should you give in-kind stuff? 01:15:12.070 --> 01:15:13.240 align:middle line:84% Should you give productive assets? 01:15:13.240 --> 01:15:14.615 align:middle line:84% Should you give-- should you make 01:15:14.615 --> 01:15:17.325 align:middle line:84% transfers conditional on activities or not? 01:15:17.325 --> 01:15:19.075 align:middle line:84% Should you give a large, one-time transfer 01:15:19.075 --> 01:15:20.200 align:middle line:90% or lots of small transfers? 01:15:20.200 --> 01:15:23.233 align:middle line:84% Should you give people workfare programs like the MNREGA 01:15:23.233 --> 01:15:24.400 align:middle line:90% program I was talking about? 01:15:24.400 --> 01:15:25.658 align:middle line:90% Should you give them cash? 01:15:25.658 --> 01:15:26.950 align:middle line:90% How should you run the program? 01:15:26.950 --> 01:15:29.210 align:middle line:84% Should you give cash versus electronic payments? 01:15:29.210 --> 01:15:32.230 align:middle line:84% How do you identify people for these programs? 01:15:32.230 --> 01:15:33.510 align:middle line:90% Should you use smart cards? 01:15:33.510 --> 01:15:35.470 align:middle line:84% So do you use biometric authentication, 01:15:35.470 --> 01:15:37.210 align:middle line:90% and so on and so forth. 01:15:37.210 --> 01:15:40.150 align:middle line:84% I think all of this is actually a pretty active research area. 01:15:40.150 --> 01:15:43.780 align:middle line:84% I think this was kind of has been an active research area 01:15:43.780 --> 01:15:45.190 align:middle line:90% for the past 10 years or so. 01:15:45.190 --> 01:15:46.780 align:middle line:84% I think this has become a really active research area 01:15:46.780 --> 01:15:47.890 align:middle line:90% in the past couple of years. 01:15:47.890 --> 01:15:49.390 align:middle line:84% But I think that both of these areas 01:15:49.390 --> 01:15:52.690 align:middle line:84% are areas where people are thinking a lot about how do we 01:15:52.690 --> 01:15:55.995 align:middle line:84% effectively design and deliver these programs in a developing 01:15:55.995 --> 01:15:58.120 align:middle line:84% country context, where we have a lot of information 01:15:58.120 --> 01:15:59.920 align:middle line:84% challenges of the sort we saw in targeting, 01:15:59.920 --> 01:16:01.870 align:middle line:84% where we have a lot of governance 01:16:01.870 --> 01:16:03.790 align:middle line:84% challenges of the type we talk about in the political economy 01:16:03.790 --> 01:16:04.930 align:middle line:90% classes, and so on and so forth. 01:16:04.930 --> 01:16:06.980 align:middle line:84% Like how do you actually do this effectively? 01:16:06.980 --> 01:16:09.880 align:middle line:84% So I just want to highlight I think two examples or three 01:16:09.880 --> 01:16:12.090 align:middle line:90% here. 01:16:12.090 --> 01:16:16.192 align:middle line:84% So one is about cash versus in-kind. 01:16:16.192 --> 01:16:17.150 align:middle line:90% So what does this mean? 01:16:17.150 --> 01:16:19.340 align:middle line:84% Cash is they literally just give you cash. 01:16:19.340 --> 01:16:22.770 align:middle line:90% And in-kind is I give you stuff. 01:16:22.770 --> 01:16:23.497 align:middle line:90% So how do you-- 01:16:23.497 --> 01:16:25.830 align:middle line:84% before I get into it, how you think about the difference 01:16:25.830 --> 01:16:27.450 align:middle line:90% between cash and in-kind? 01:16:27.450 --> 01:16:30.787 align:middle line:84% Like put on your economist hats, which I should always 01:16:30.787 --> 01:16:31.620 align:middle line:90% be on in this class. 01:16:31.620 --> 01:16:33.240 align:middle line:84% Like how do you think about the difference between cash 01:16:33.240 --> 01:16:33.972 align:middle line:90% and in-kind? 01:16:33.972 --> 01:16:36.180 align:middle line:84% What are some of the kinds of things you think about? 01:16:36.180 --> 01:16:38.910 align:middle line:90% 01:16:38.910 --> 01:16:40.665 align:middle line:84% AUDIENCE: If people are rational, 01:16:40.665 --> 01:16:44.090 align:middle line:84% we should be giving them cash because you 01:16:44.090 --> 01:16:48.730 align:middle line:84% can't prescribe the highest utility [INAUDIBLE].. 01:16:48.730 --> 01:16:50.230 align:middle line:84% BEN OLKEN: Right, so economists love 01:16:50.230 --> 01:16:52.340 align:middle line:84% cash for the reason [? Rega ?] just said, 01:16:52.340 --> 01:16:54.310 align:middle line:84% which is that we should be like trusting people 01:16:54.310 --> 01:16:55.540 align:middle line:90% to make their own decisions. 01:16:55.540 --> 01:16:57.475 align:middle line:84% And I don't know what they want, but they 01:16:57.475 --> 01:16:58.600 align:middle line:90% should know what they want. 01:16:58.600 --> 01:17:00.100 align:middle line:84% And with cash, they can just buy it. 01:17:00.100 --> 01:17:02.530 align:middle line:90% 01:17:02.530 --> 01:17:05.230 align:middle line:84% AUDIENCE: Public support for a program that is in-kind 01:17:05.230 --> 01:17:06.100 align:middle line:90% might be higher. 01:17:06.100 --> 01:17:08.320 align:middle line:90% So if you need to [INAUDIBLE]. 01:17:08.320 --> 01:17:10.085 align:middle line:90% BEN OLKEN: Why? 01:17:10.085 --> 01:17:12.460 align:middle line:84% AUDIENCE: If people find out-- if people don't agree with 01:17:12.460 --> 01:17:14.627 align:middle line:84% the types of things that people are choosing to buy, 01:17:14.627 --> 01:17:16.690 align:middle line:84% even if it is maximizing for them, 01:17:16.690 --> 01:17:20.600 align:middle line:84% society might feel like they should have programs, 01:17:20.600 --> 01:17:21.760 align:middle line:90% so to the extent that-- 01:17:21.760 --> 01:17:23.140 align:middle line:90% especially in the United States. 01:17:23.140 --> 01:17:26.170 align:middle line:84% That's why we would restrict people's food stamps 01:17:26.170 --> 01:17:30.050 align:middle line:84% because even if buying alcohol might make people feel better 01:17:30.050 --> 01:17:31.410 align:middle line:90% or enjoy it more. 01:17:31.410 --> 01:17:32.410 align:middle line:90% BEN OLKEN: Yes, exactly. 01:17:32.410 --> 01:17:34.700 align:middle line:84% People may have paternalistic pressure-- 01:17:34.700 --> 01:17:37.250 align:middle line:84% political-- [INAUDIBLE] reason people 01:17:37.250 --> 01:17:40.370 align:middle line:84% want to-- only want them to use the money on certain things. 01:17:40.370 --> 01:17:41.050 align:middle line:90% Patrick. 01:17:41.050 --> 01:17:43.550 align:middle line:84% AUDIENCE: You might be able to address some of the targeting 01:17:43.550 --> 01:17:45.020 align:middle line:90% issues with [INAUDIBLE] cards. 01:17:45.020 --> 01:17:45.640 align:middle line:90% BEN OLKEN: What, sorry? 01:17:45.640 --> 01:17:48.223 align:middle line:84% AUDIENCE: You might be able to address some of the targeting-- 01:17:48.223 --> 01:17:49.070 align:middle line:90% BEN OLKEN: How? 01:17:49.070 --> 01:17:51.703 align:middle line:84% AUDIENCE: If you target inferior products. 01:17:51.703 --> 01:17:52.370 align:middle line:90% BEN OLKEN: Yeah. 01:17:52.370 --> 01:17:53.548 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:17:53.548 --> 01:17:54.590 align:middle line:90% BEN OLKEN: Yeah, exactly. 01:17:54.590 --> 01:17:57.800 align:middle line:84% So if I give out inferior goods, then basically rich people 01:17:57.800 --> 01:17:58.878 align:middle line:90% may not want them. 01:17:58.878 --> 01:18:00.920 align:middle line:84% That actually may have good targeting properties. 01:18:00.920 --> 01:18:02.210 align:middle line:90% Absolutely. 01:18:02.210 --> 01:18:04.420 align:middle line:84% Other things you might want to think about? 01:18:04.420 --> 01:18:06.100 align:middle line:90% Yeah. 01:18:06.100 --> 01:18:09.520 align:middle line:84% AUDIENCE: If we do an in-kind transfer, 01:18:09.520 --> 01:18:12.460 align:middle line:84% it could also help the suppliers. 01:18:12.460 --> 01:18:16.030 align:middle line:84% Like if we really like to want to help some rice company 01:18:16.030 --> 01:18:17.900 align:middle line:84% and give a lot of poor people rice, 01:18:17.900 --> 01:18:20.048 align:middle line:84% the government would be helping the rice industry. 01:18:20.048 --> 01:18:22.090 align:middle line:84% BEN OLKEN: What are the conditions for which that 01:18:22.090 --> 01:18:24.280 align:middle line:90% would have to be true? 01:18:24.280 --> 01:18:28.110 align:middle line:84% AUDIENCE: It would have to produce the supply shock that 01:18:28.110 --> 01:18:30.210 align:middle line:90% wouldn't otherwise happen. 01:18:30.210 --> 01:18:31.090 align:middle line:90% BEN OLKEN: Right. 01:18:31.090 --> 01:18:31.590 align:middle line:90% So yes. 01:18:31.590 --> 01:18:33.100 align:middle line:84% So you have to have a supply shock. 01:18:33.100 --> 01:18:34.767 align:middle line:84% So when is it going have a supply shock? 01:18:34.767 --> 01:18:38.348 align:middle line:90% 01:18:38.348 --> 01:18:39.140 align:middle line:90% AUDIENCE: Like if-- 01:18:39.140 --> 01:18:40.160 align:middle line:84% BEN OLKEN: So, [INAUDIBLE] back up. 01:18:40.160 --> 01:18:42.800 align:middle line:84% Are there some conditions where an in-kind transfer would have 01:18:42.800 --> 01:18:43.967 align:middle line:90% not generate a supply shock? 01:18:43.967 --> 01:18:47.556 align:middle line:90% 01:18:47.556 --> 01:18:49.998 align:middle line:84% AUDIENCE: Like if they were going to buy it with the cash. 01:18:49.998 --> 01:18:52.290 align:middle line:84% BEN OLKEN: Yeah, they're going to buy with cash anyway. 01:18:52.290 --> 01:18:52.950 align:middle line:84% Is that what you're going to say? 01:18:52.950 --> 01:18:53.730 align:middle line:90% Yeah. 01:18:53.730 --> 01:18:55.740 align:middle line:84% So imagine that I'm going to buy-- 01:18:55.740 --> 01:18:59.400 align:middle line:84% like in Indonesia, for example, everyone eats a lot of rice. 01:18:59.400 --> 01:19:01.440 align:middle line:84% But suppose everyone needs 30 kilos of rice. 01:19:01.440 --> 01:19:03.490 align:middle line:84% And I hand out everyone 5 kilos of rice. 01:19:03.490 --> 01:19:06.150 align:middle line:84% That should be totally irrelevant in terms of supply 01:19:06.150 --> 01:19:09.730 align:middle line:84% because people are going to buy that rice anyway. 01:19:09.730 --> 01:19:11.890 align:middle line:84% That's true as long as the market-- 01:19:11.890 --> 01:19:16.580 align:middle line:84% as local markets have elastic supply. 01:19:16.580 --> 01:19:23.730 align:middle line:84% If the markets are themselves have not inelastic supply, 01:19:23.730 --> 01:19:26.760 align:middle line:84% even if people are going to be consuming all that rice, 01:19:26.760 --> 01:19:28.375 align:middle line:84% you might be like moving physical rice 01:19:28.375 --> 01:19:30.750 align:middle line:84% and that actually may change through the aggregate supply 01:19:30.750 --> 01:19:31.140 align:middle line:90% in an area. 01:19:31.140 --> 01:19:32.890 align:middle line:84% So it's actually a little bit complicated. 01:19:32.890 --> 01:19:35.213 align:middle line:84% So number one is, on the demand side, it's not-- 01:19:35.213 --> 01:19:36.630 align:middle line:84% on the demand side, if people were 01:19:36.630 --> 01:19:38.220 align:middle line:84% going to consume all that anyway, 01:19:38.220 --> 01:19:39.600 align:middle line:84% it's not automatically going to be supply shock. 01:19:39.600 --> 01:19:40.860 align:middle line:84% But weather there is kind of depends 01:19:40.860 --> 01:19:43.030 align:middle line:84% on what the underlying market structure looks like. 01:19:43.030 --> 01:19:45.690 align:middle line:84% So there may or may not be a supply shock. 01:19:45.690 --> 01:19:48.480 align:middle line:84% Any other issues you might think about? 01:19:48.480 --> 01:19:52.260 align:middle line:84% AUDIENCE: Insurance issues, so you're saying that safer food, 01:19:52.260 --> 01:19:53.940 align:middle line:84% like to give it in-kind, then people 01:19:53.940 --> 01:19:55.860 align:middle line:84% don't have to worry about fluctuating 01:19:55.860 --> 01:19:57.877 align:middle line:90% food prices over seasons. 01:19:57.877 --> 01:19:59.460 align:middle line:84% I guess it's like a form of insurance. 01:19:59.460 --> 01:20:00.252 align:middle line:90% BEN OLKEN: Exactly. 01:20:00.252 --> 01:20:02.910 align:middle line:84% And there's a brand new paper on exactly this point called 01:20:02.910 --> 01:20:08.340 align:middle line:84% in-kind transfers as a form of insurance, which came out 01:20:08.340 --> 01:20:11.970 align:middle line:84% like in the last year or so, Was actually presented at the NBR 01:20:11.970 --> 01:20:13.800 align:middle line:90% meetings just a few weeks ago. 01:20:13.800 --> 01:20:16.050 align:middle line:84% And which makes exactly this point that typically when 01:20:16.050 --> 01:20:18.720 align:middle line:84% we fix transfer amounts in-kind, we 01:20:18.720 --> 01:20:21.870 align:middle line:84% say we're giving you a certain quantity of stuff, 01:20:21.870 --> 01:20:25.230 align:middle line:84% 10 kilos of rice, an apartment, whatever. 01:20:25.230 --> 01:20:27.820 align:middle line:84% Whereas when we fix things in cash, we typically say 01:20:27.820 --> 01:20:29.862 align:middle line:84% we're going to give you a certain amount of cash. 01:20:29.862 --> 01:20:32.490 align:middle line:84% Now, in principle, you could adjust it either way. 01:20:32.490 --> 01:20:36.420 align:middle line:84% I could say I'm going to give you the equivalent of 10 01:20:36.420 --> 01:20:39.330 align:middle line:90% kgs worth of rice at some-- 01:20:39.330 --> 01:20:40.080 align:middle line:90% or whatever. 01:20:40.080 --> 01:20:42.330 align:middle line:84% Or I'm going to give you the equivalent amount of rice 01:20:42.330 --> 01:20:45.678 align:middle line:84% that I can buy with 100,000 rupiah at market prices. 01:20:45.678 --> 01:20:47.220 align:middle line:84% And I can move the quantities around. 01:20:47.220 --> 01:20:52.470 align:middle line:84% Or vice versa, I can index the transfer to inflation, 01:20:52.470 --> 01:20:54.450 align:middle line:84% for example, or some particular types of goods. 01:20:54.450 --> 01:20:57.835 align:middle line:84% But in practice, that's imperfect. 01:20:57.835 --> 01:20:59.460 align:middle line:84% And therefore, I think they do actually 01:20:59.460 --> 01:21:01.740 align:middle line:84% provide that in and of that kind of in-kind assurance. 01:21:01.740 --> 01:21:02.740 align:middle line:90% I think I'm out of time. 01:21:02.740 --> 01:21:04.025 align:middle line:90% Let me stop here. 01:21:04.025 --> 01:21:05.400 align:middle line:84% I'm going to talk about one issue 01:21:05.400 --> 01:21:07.627 align:middle line:84% next time, which is the supply shock issue. 01:21:07.627 --> 01:21:09.210 align:middle line:84% But all of these are kind of relevant. 01:21:09.210 --> 01:21:10.770 align:middle line:90% And then I'll talk about-- 01:21:10.770 --> 01:21:12.720 align:middle line:84% the last two papers I'm going to talk about in this section 01:21:12.720 --> 01:21:14.137 align:middle line:84% before I talk about tax, I'm going 01:21:14.137 --> 01:21:16.440 align:middle line:84% to talk about this one, the Cunha et al. 01:21:16.440 --> 01:21:20.090 align:middle line:84% paper on supply shocks and in-kind transfers. 01:21:20.090 --> 01:21:23.340 align:middle line:84% I'm going to talk about the cash versus condition paper 01:21:23.340 --> 01:21:25.240 align:middle line:90% buy [? Bear ?] et al. 01:21:25.240 --> 01:21:26.490 align:middle line:90% on conditional cash transfers. 01:21:26.490 --> 01:21:29.670 align:middle line:84% And then I promise we'll switch to tax next time. 01:21:29.670 --> 01:21:30.990 align:middle line:90% And so you shouldn't be-- 01:21:30.990 --> 01:21:33.073 align:middle line:84% I think the reading for today was the tax reading. 01:21:33.073 --> 01:21:35.206 align:middle line:84% So we'll talk about that one next time. 01:21:35.206 --> 01:21:37.200 align:middle line:90% OK, Thanks. 01:21:37.200 --> 01:21:49.000 align:middle line:90%