WEBVTT 00:00:00.000 --> 00:00:01.952 align:middle line:90% [SQUEAKING] 00:00:01.952 --> 00:00:03.904 align:middle line:90% [RUSTLING] 00:00:03.904 --> 00:00:05.856 align:middle line:90% [CLICKING] 00:00:05.856 --> 00:00:18.544 align:middle line:90% 00:00:18.544 --> 00:00:19.235 align:middle line:90% BEN OLKEN: OK. 00:00:19.235 --> 00:00:21.890 align:middle line:84% So just to remind you guys where we were-- so last time 00:00:21.890 --> 00:00:25.048 align:middle line:84% we talked about the basic setup of models 00:00:25.048 --> 00:00:26.090 align:middle line:90% of thinking about credit. 00:00:26.090 --> 00:00:27.840 align:middle line:84% So we had adverse selection, moral hazard. 00:00:27.840 --> 00:00:30.132 align:middle line:84% We had monitoring models, and so on and so forth. 00:00:30.132 --> 00:00:32.299 align:middle line:84% And today, what I want to talk about is microcredit, 00:00:32.299 --> 00:00:33.770 align:middle line:90% microfinance. 00:00:33.770 --> 00:00:35.215 align:middle line:90% And in particular, just-- 00:00:35.215 --> 00:00:37.340 align:middle line:84% I'll give you a couple of words of background, some 00:00:37.340 --> 00:00:39.285 align:middle line:90% of the history of microfinance. 00:00:39.285 --> 00:00:41.660 align:middle line:84% We'll talk a little bit about the impact of microfinance, 00:00:41.660 --> 00:00:43.910 align:middle line:84% both overall its heterogeneous impacts, 00:00:43.910 --> 00:00:47.330 align:middle line:84% and then I'm going to talk about the role 00:00:47.330 --> 00:00:49.730 align:middle line:84% of particular microfinance institutions. 00:00:49.730 --> 00:00:51.320 align:middle line:90% And what I want to-- 00:00:51.320 --> 00:00:57.790 align:middle line:84% I guess I want to emphasize on this piece is that, as you'll 00:00:57.790 --> 00:00:59.800 align:middle line:84% see, microfinance is kind of a bundle 00:00:59.800 --> 00:01:01.130 align:middle line:90% of many different things. 00:01:01.130 --> 00:01:05.080 align:middle line:84% And there's been a nice series of research papers 00:01:05.080 --> 00:01:08.680 align:middle line:84% over a series of years that tried to help us unpack what's 00:01:08.680 --> 00:01:09.920 align:middle line:90% in that bundle. 00:01:09.920 --> 00:01:13.740 align:middle line:84% And I think that you'll see some nice elements of that. 00:01:13.740 --> 00:01:14.290 align:middle line:90% OK. 00:01:14.290 --> 00:01:15.457 align:middle line:90% So just a bit of background. 00:01:15.457 --> 00:01:21.590 align:middle line:84% So microfinance, the modern microfinance model, I guess, 00:01:21.590 --> 00:01:25.790 align:middle line:84% came through the Grameen Bank with Mohammed Yunus, 00:01:25.790 --> 00:01:31.820 align:middle line:84% who won the Nobel Prize a few years ago, helped create. 00:01:31.820 --> 00:01:35.510 align:middle line:84% And that made small loans to women in Bangladesh, 00:01:35.510 --> 00:01:37.138 align:middle line:90% to poor women in Bangladesh. 00:01:37.138 --> 00:01:39.680 align:middle line:84% I think, actually, these numbers are actually probably small. 00:01:39.680 --> 00:01:41.263 align:middle line:84% I think it's way more than $25 billion 00:01:41.263 --> 00:01:44.750 align:middle line:84% today, probably-- these probably are outdated. 00:01:44.750 --> 00:01:50.240 align:middle line:84% Hundreds of millions of clients, typically higher payment rates. 00:01:50.240 --> 00:01:52.700 align:middle line:84% Some microfinance institutions are profitable. 00:01:52.700 --> 00:01:54.680 align:middle line:90% Some are not. 00:01:54.680 --> 00:01:56.990 align:middle line:84% And these microfinance institutions are-- 00:01:56.990 --> 00:02:01.280 align:middle line:84% they are lending to a large number of small borrowers 00:02:01.280 --> 00:02:07.460 align:middle line:84% and providing them small loans, $50, $100, $500, in that-- 00:02:07.460 --> 00:02:08.449 align:middle line:90% maybe up to $1,000. 00:02:08.449 --> 00:02:10.032 align:middle line:84% Maybe that would be a really big loan. 00:02:10.032 --> 00:02:11.900 align:middle line:90% Small loans. 00:02:11.900 --> 00:02:13.430 align:middle line:84% And there's an idea that these can 00:02:13.430 --> 00:02:15.140 align:middle line:84% be really powerful for helping break 00:02:15.140 --> 00:02:17.723 align:middle line:84% some of the credit constraints that we talked about last time. 00:02:17.723 --> 00:02:19.457 align:middle line:84% So some of the stuff at the end of this-- 00:02:19.457 --> 00:02:21.290 align:middle line:84% some of the models we talked about last time 00:02:21.290 --> 00:02:23.390 align:middle line:84% would predict that people who are really poor 00:02:23.390 --> 00:02:26.310 align:middle line:84% would not be able to borrow anything or very little. 00:02:26.310 --> 00:02:28.730 align:middle line:84% And if there's a way to break those credit constraints 00:02:28.730 --> 00:02:30.543 align:middle line:84% and figure out a way to solve that, 00:02:30.543 --> 00:02:32.210 align:middle line:84% that could potentially be very powerful. 00:02:32.210 --> 00:02:34.810 align:middle line:90% 00:02:34.810 --> 00:02:39.890 align:middle line:90% So I would say-- 00:02:39.890 --> 00:02:49.240 align:middle line:90% 00:02:49.240 --> 00:02:53.120 align:middle line:84% suppose we wanted to start by understanding-- 00:02:53.120 --> 00:02:55.550 align:middle line:84% and I say there were lots of grand claims associated 00:02:55.550 --> 00:02:56.550 align:middle line:90% with microcredit, right? 00:02:56.550 --> 00:02:59.473 align:middle line:84% So if you think of when Yunus wins the Nobel Prize, 00:02:59.473 --> 00:03:00.890 align:middle line:84% the idea is this microcredit thing 00:03:00.890 --> 00:03:03.410 align:middle line:84% has been totally transformative to lots of people 00:03:03.410 --> 00:03:08.760 align:middle line:84% and changed people's lives in lots of ways. 00:03:08.760 --> 00:03:13.730 align:middle line:84% So if you were trying to think about trying to understand 00:03:13.730 --> 00:03:15.650 align:middle line:84% and evaluate some of these claims, 00:03:15.650 --> 00:03:17.810 align:middle line:84% what are some questions you'd want to answer? 00:03:17.810 --> 00:03:23.773 align:middle line:84% And how would you think about evaluating them? 00:03:23.773 --> 00:03:25.190 align:middle line:84% What are some questions you'd want 00:03:25.190 --> 00:03:28.520 align:middle line:90% to know about microfinance? 00:03:28.520 --> 00:03:30.440 align:middle line:84% This is now a question I'm posing to you. 00:03:30.440 --> 00:03:31.385 align:middle line:90% Yeah, [? Becca? ?] 00:03:31.385 --> 00:03:33.427 align:middle line:84% AUDIENCE: I suppose you want to know if they just 00:03:33.427 --> 00:03:37.312 align:middle line:84% substituted [INAUDIBLE] from some other form of lending? 00:03:37.312 --> 00:03:38.020 align:middle line:90% BEN OLKEN: Right. 00:03:38.020 --> 00:03:40.270 align:middle line:84% So you want to know is, is the change in net access to credit, 00:03:40.270 --> 00:03:41.200 align:middle line:90% not just gross? 00:03:41.200 --> 00:03:42.340 align:middle line:84% So just the fact that people are borrowing 00:03:42.340 --> 00:03:43.480 align:middle line:84% from microfinance institutions doesn't mean 00:03:43.480 --> 00:03:44.650 align:middle line:90% they're getting more credit. 00:03:44.650 --> 00:03:45.900 align:middle line:90% What else do you want to know? 00:03:45.900 --> 00:03:49.730 align:middle line:90% 00:03:49.730 --> 00:03:51.730 align:middle line:84% AUDIENCE: Maybe that individual that [INAUDIBLE] 00:03:51.730 --> 00:03:54.415 align:middle line:90% has access [INAUDIBLE]? 00:03:54.415 --> 00:03:56.540 align:middle line:84% BEN OLKEN: OK, so, right, so what kind of outcomes? 00:03:56.540 --> 00:03:57.870 align:middle line:90% AUDIENCE: Business profits. 00:03:57.870 --> 00:03:58.828 align:middle line:90% BEN OLKEN: What's that? 00:03:58.828 --> 00:04:00.560 align:middle line:84% AUDIENCE: Business profits, consumption. 00:04:00.560 --> 00:04:01.350 align:middle line:90% BEN OLKEN: Right. 00:04:01.350 --> 00:04:01.850 align:middle line:90% Right. 00:04:01.850 --> 00:04:03.892 align:middle line:84% So it seems like one thing you would want to know 00:04:03.892 --> 00:04:06.710 align:middle line:90% is want to know is it about-- 00:04:06.710 --> 00:04:10.130 align:middle line:84% is it actually-- if you believe that people are credit 00:04:10.130 --> 00:04:13.580 align:middle line:84% constrained, that they aren't-- if we believe that 00:04:13.580 --> 00:04:15.770 align:middle line:84% we're in a world where basically people can-- 00:04:15.770 --> 00:04:18.249 align:middle line:84% if we're in a world where people are unconstrained, 00:04:18.249 --> 00:04:23.690 align:middle line:84% that in the unconstrained world, where 00:04:23.690 --> 00:04:26.030 align:middle line:84% you have f prime equals r, you'd expect 00:04:26.030 --> 00:04:28.610 align:middle line:84% the marginal return of capital is 00:04:28.610 --> 00:04:29.927 align:middle line:90% equal to-- is not super high. 00:04:29.927 --> 00:04:31.260 align:middle line:90% It's equal to the interest rate. 00:04:31.260 --> 00:04:38.040 align:middle line:84% But if you're in a world where k is constrained, 00:04:38.040 --> 00:04:40.730 align:middle line:84% then you might think that f prime is much greater than r 00:04:40.730 --> 00:04:43.530 align:middle line:84% because people aren't able to get all the capital they need. 00:04:43.530 --> 00:04:45.290 align:middle line:84% So one thing you'd want to say is 00:04:45.290 --> 00:04:53.430 align:middle line:84% what is the impact of additional capital on people's profits, 00:04:53.430 --> 00:04:53.930 align:middle line:90% essentially? 00:04:53.930 --> 00:04:56.138 align:middle line:84% And like if you believe they were credit constrained, 00:04:56.138 --> 00:04:58.840 align:middle line:84% you might expect that thing to be really high. 00:04:58.840 --> 00:05:00.590 align:middle line:84% So that's one thing you'd want to look at. 00:05:00.590 --> 00:05:02.390 align:middle line:84% You might also-- see, you may also 00:05:02.390 --> 00:05:05.190 align:middle line:84% then want to look at downstream consumption outcomes. 00:05:05.190 --> 00:05:07.107 align:middle line:84% Yeah, [? Becca, ?] what were you going to say? 00:05:07.107 --> 00:05:09.560 align:middle line:84% AUDIENCE: You might also be concerned about information 00:05:09.560 --> 00:05:12.055 align:middle line:84% or whether people are taking on loans that are actually 00:05:12.055 --> 00:05:16.133 align:middle line:84% in their interest or whether it's predatory in some form? 00:05:16.133 --> 00:05:16.800 align:middle line:90% BEN OLKEN: Yeah. 00:05:16.800 --> 00:05:19.170 align:middle line:84% So what do you mean by predatory? 00:05:19.170 --> 00:05:21.660 align:middle line:90% AUDIENCE: So if you are-- 00:05:21.660 --> 00:05:24.600 align:middle line:84% if people don't actually have good information 00:05:24.600 --> 00:05:26.520 align:middle line:84% about their odds of being able to pay back 00:05:26.520 --> 00:05:29.700 align:middle line:84% or the form of credit that they're receiving, 00:05:29.700 --> 00:05:32.010 align:middle line:84% that you would not want them to be 00:05:32.010 --> 00:05:34.395 align:middle line:84% needing to pay off some interest in the future because-- 00:05:34.395 --> 00:05:37.383 align:middle line:90% 00:05:37.383 --> 00:05:38.050 align:middle line:90% BEN OLKEN: Yeah. 00:05:38.050 --> 00:05:40.240 align:middle line:84% So I think that's also related to some of the consumption 00:05:40.240 --> 00:05:42.310 align:middle line:84% questions, but I think you're hitting another point, which 00:05:42.310 --> 00:05:44.290 align:middle line:84% is that we may not just be interested in averages. 00:05:44.290 --> 00:05:46.748 align:middle line:84% We may be interested in the distribution of effects, right? 00:05:46.748 --> 00:05:47.657 align:middle line:90% So it may be-- 00:05:47.657 --> 00:05:49.240 align:middle line:84% and we're interested in distributions, 00:05:49.240 --> 00:05:51.110 align:middle line:90% I think, in two senses. 00:05:51.110 --> 00:05:53.800 align:middle line:84% The first sense is we're interested in just, is there-- 00:05:53.800 --> 00:05:55.780 align:middle line:84% what is the distribution of outcomes? 00:05:55.780 --> 00:05:58.060 align:middle line:84% So imagine that basically we had-- 00:05:58.060 --> 00:06:07.810 align:middle line:90% 00:06:07.810 --> 00:06:09.030 align:middle line:90% this is like outcomes. 00:06:09.030 --> 00:06:10.990 align:middle line:84% Imagine we had quantile treatment effects. 00:06:10.990 --> 00:06:12.700 align:middle line:84% There's a big difference in a world where basically we 00:06:12.700 --> 00:06:13.780 align:middle line:84% have a bunch of people with nothing 00:06:13.780 --> 00:06:15.340 align:middle line:84% and a bunch of both positive income effects. 00:06:15.340 --> 00:06:16.480 align:middle line:84% That's a world where actually we're 00:06:16.480 --> 00:06:18.063 align:middle line:84% pretty happy with this thing, and it's 00:06:18.063 --> 00:06:21.190 align:middle line:84% like nothing for everyone and good for some people. 00:06:21.190 --> 00:06:22.648 align:middle line:90% Or if we had a world-- this is-- 00:06:22.648 --> 00:06:24.190 align:middle line:84% these are quantile treatment effects. 00:06:24.190 --> 00:06:26.775 align:middle line:84% Or if we had a world where it looks like this, where 00:06:26.775 --> 00:06:28.900 align:middle line:84% some people are a lot worse off and some people are 00:06:28.900 --> 00:06:30.970 align:middle line:84% a lot better off, even if this went 00:06:30.970 --> 00:06:33.320 align:middle line:84% really high so the mean was positive, 00:06:33.320 --> 00:06:35.320 align:middle line:84% you would feel very differently about this world 00:06:35.320 --> 00:06:36.730 align:middle line:90% versus this world, right? 00:06:36.730 --> 00:06:38.982 align:middle line:84% Because here, like nobody's worse off. 00:06:38.982 --> 00:06:41.440 align:middle line:84% Here, people-- some people are actually substantially worse 00:06:41.440 --> 00:06:41.590 align:middle line:90% off. 00:06:41.590 --> 00:06:43.590 align:middle line:84% And that goes to the idea that maybe some people 00:06:43.590 --> 00:06:45.370 align:middle line:90% can't pay back. 00:06:45.370 --> 00:06:48.250 align:middle line:84% And if, by the way, you think of capital as-- 00:06:48.250 --> 00:06:50.825 align:middle line:84% access to capital as funding investments which may be risky, 00:06:50.825 --> 00:06:53.200 align:middle line:84% it's not crazy to think that you're in a world like this. 00:06:53.200 --> 00:06:55.330 align:middle line:84% Like, if these are positive expected return 00:06:55.330 --> 00:06:57.792 align:middle line:84% but risky investments, on average, people 00:06:57.792 --> 00:06:59.750 align:middle line:84% will be better off, but some will be worse off. 00:06:59.750 --> 00:07:01.135 align:middle line:84% And so understanding whether the world looks like this 00:07:01.135 --> 00:07:02.560 align:middle line:90% or like this is different. 00:07:02.560 --> 00:07:04.660 align:middle line:84% And then in addition, I think, [? Becca, ?] you 00:07:04.660 --> 00:07:07.000 align:middle line:84% asked another point, which is, it's not 00:07:07.000 --> 00:07:09.318 align:middle line:84% just that we're interested in the overall heterogeneity 00:07:09.318 --> 00:07:10.360 align:middle line:90% in the treatment effects. 00:07:10.360 --> 00:07:12.410 align:middle line:84% We're asking whether they're predictable or not. 00:07:12.410 --> 00:07:16.150 align:middle line:84% And so this is just saying, what are the quantile treatment 00:07:16.150 --> 00:07:17.740 align:middle line:90% effects, just in general. 00:07:17.740 --> 00:07:19.150 align:middle line:84% There's another question which say, do people-- 00:07:19.150 --> 00:07:20.590 align:middle line:90% is that predictable or not? 00:07:20.590 --> 00:07:22.545 align:middle line:84% And are people able to make good-- is it 00:07:22.545 --> 00:07:24.170 align:middle line:84% sort of that there's risk in the world? 00:07:24.170 --> 00:07:26.270 align:middle line:84% If we think of the model I talked about last time, 00:07:26.270 --> 00:07:29.050 align:middle line:84% this is a model of positive expected return activities, 00:07:29.050 --> 00:07:30.520 align:middle line:90% but risky, risky. 00:07:30.520 --> 00:07:32.020 align:middle line:84% There's another world, though, where 00:07:32.020 --> 00:07:32.870 align:middle line:84% you start to think about some of the selection 00:07:32.870 --> 00:07:34.060 align:middle line:84% in the world, where people are different 00:07:34.060 --> 00:07:35.140 align:middle line:90% in their characteristics. 00:07:35.140 --> 00:07:36.652 align:middle line:84% Some people have really good things. 00:07:36.652 --> 00:07:38.110 align:middle line:84% Some people have really bad things. 00:07:38.110 --> 00:07:39.100 align:middle line:90% Can they predict them? 00:07:39.100 --> 00:07:40.600 align:middle line:90% Can the banks predict them? 00:07:40.600 --> 00:07:42.280 align:middle line:84% Can we explain that heterogeneity 00:07:42.280 --> 00:07:43.450 align:middle line:90% in a reasonable way? 00:07:43.450 --> 00:07:44.440 align:middle line:90% [INAUDIBLE] 00:07:44.440 --> 00:07:46.353 align:middle line:84% AUDIENCE: [INAUDIBLE] take-up rates? 00:07:46.353 --> 00:07:47.020 align:middle line:90% BEN OLKEN: Yeah. 00:07:47.020 --> 00:07:48.460 align:middle line:90% Say more. 00:07:48.460 --> 00:07:51.790 align:middle line:84% AUDIENCE: So maybe a formal microcredit institution 00:07:51.790 --> 00:07:54.910 align:middle line:84% might set more restrictions on what you can use microcredit 00:07:54.910 --> 00:07:57.950 align:middle line:84% for, and perhaps that lowers the take-up rate relative 00:07:57.950 --> 00:08:02.380 align:middle line:84% to formal loaners who might not put such restrictions, 00:08:02.380 --> 00:08:04.383 align:middle line:84% maybe in deadlines or [INAUDIBLE].. 00:08:04.383 --> 00:08:05.050 align:middle line:90% BEN OLKEN: Yeah. 00:08:05.050 --> 00:08:05.350 align:middle line:90% Sorry. 00:08:05.350 --> 00:08:05.740 align:middle line:90% So, yes. 00:08:05.740 --> 00:08:06.615 align:middle line:90% Take-up-- absolutely. 00:08:06.615 --> 00:08:09.090 align:middle line:84% So are these restrictions reducing take up, 00:08:09.090 --> 00:08:10.090 align:middle line:90% and is that good or bad? 00:08:10.090 --> 00:08:11.620 align:middle line:90% You want to think about that? 00:08:11.620 --> 00:08:13.120 align:middle line:84% [? Ahmed, ?] do you have a question? 00:08:13.120 --> 00:08:14.020 align:middle line:90% AUDIENCE: Yeah. 00:08:14.020 --> 00:08:16.250 align:middle line:90% Conceptual question. 00:08:16.250 --> 00:08:18.712 align:middle line:84% Should I think similarly of the microcredit 00:08:18.712 --> 00:08:20.420 align:middle line:84% in developing countries and payday loans, 00:08:20.420 --> 00:08:23.780 align:middle line:90% say, in the United States? 00:08:23.780 --> 00:08:26.030 align:middle line:90% BEN OLKEN: Yes and no. 00:08:26.030 --> 00:08:33.409 align:middle line:84% They have-- I think they have different characteristics. 00:08:33.409 --> 00:08:36.520 align:middle line:84% So they're both targeting low-income individuals. 00:08:36.520 --> 00:08:40.000 align:middle line:84% The payday loans are collateralized basically 00:08:40.000 --> 00:08:41.650 align:middle line:84% against your future paychecks, I think, 00:08:41.650 --> 00:08:44.710 align:middle line:84% whereas these are uncollateralized, typically. 00:08:44.710 --> 00:08:46.820 align:middle line:84% And we'll talk about how they ensure repayment, 00:08:46.820 --> 00:08:47.770 align:middle line:84% so they're different in that sense. 00:08:47.770 --> 00:08:48.560 align:middle line:90% They're similar. 00:08:48.560 --> 00:08:50.477 align:middle line:84% They both have reasonably high interest rates. 00:08:50.477 --> 00:08:53.627 align:middle line:84% I think one question is whether the payday loans are mostly 00:08:53.627 --> 00:08:55.960 align:middle line:84% used for consumption smoothing, whereas these are mostly 00:08:55.960 --> 00:08:56.650 align:middle line:90% used for investment. 00:08:56.650 --> 00:08:58.525 align:middle line:84% That could be a really important distinction. 00:08:58.525 --> 00:09:01.030 align:middle line:84% So if these are screening clients on, 00:09:01.030 --> 00:09:03.550 align:middle line:84% do you have a small business that you're going to invest in, 00:09:03.550 --> 00:09:06.050 align:middle line:84% you might think those would have very different implications 00:09:06.050 --> 00:09:08.140 align:middle line:84% than if it's just available people who have really 00:09:08.140 --> 00:09:10.410 align:middle line:90% urgent consumption needs. 00:09:10.410 --> 00:09:11.910 align:middle line:84% AUDIENCE: Follow up on [INAUDIBLE].. 00:09:11.910 --> 00:09:17.160 align:middle line:84% You mentioned that [INAUDIBLE] credit agencies are actually 00:09:17.160 --> 00:09:19.490 align:middle line:84% extremely profitable for companies. 00:09:19.490 --> 00:09:20.030 align:middle line:90% I want-- 00:09:20.030 --> 00:09:20.495 align:middle line:90% BEN OLKEN: Some are. 00:09:20.495 --> 00:09:21.320 align:middle line:90% Some aren't, yeah. 00:09:21.320 --> 00:09:22.490 align:middle line:90% AUDIENCE: I wonder why. 00:09:22.490 --> 00:09:27.410 align:middle line:84% What was the economic friction that prevented microcredit 00:09:27.410 --> 00:09:29.767 align:middle line:90% [INAUDIBLE] 20 years ago? 00:09:29.767 --> 00:09:30.350 align:middle line:90% BEN OLKEN: Oh. 00:09:30.350 --> 00:09:32.988 align:middle line:90% So, it-- one-- so, yeah. 00:09:32.988 --> 00:09:34.280 align:middle line:90% So this is-- so you're asking-- 00:09:34.280 --> 00:09:36.613 align:middle line:84% I would call-- this is the difference between a question 00:09:36.613 --> 00:09:38.960 align:middle line:84% that an economist in this room would class and-- 00:09:38.960 --> 00:09:41.300 align:middle line:84% ask and someone in a business school class over at Sloan 00:09:41.300 --> 00:09:42.643 align:middle line:90% would ask. 00:09:42.643 --> 00:09:44.060 align:middle line:84% You ask the question I always ask, 00:09:44.060 --> 00:09:48.733 align:middle line:84% which is, if this was new and profitable, 00:09:48.733 --> 00:09:50.150 align:middle line:84% why didn't it existed in the past? 00:09:50.150 --> 00:09:52.525 align:middle line:84% Like, it can't be-- it must not have been in equilibrium. 00:09:52.525 --> 00:09:53.630 align:middle line:90% That's the economics view. 00:09:53.630 --> 00:09:55.970 align:middle line:84% The business school view is like, somebody had a good idea 00:09:55.970 --> 00:09:56.803 align:middle line:90% and made some money. 00:09:56.803 --> 00:09:58.610 align:middle line:90% [LAUGHTER] 00:09:58.610 --> 00:10:04.940 align:middle line:84% And so I say that actually not actually totally facetiously. 00:10:04.940 --> 00:10:07.945 align:middle line:84% I think that sometimes people just innovate. 00:10:07.945 --> 00:10:10.070 align:middle line:84% And come up with new ideas and make new businesses. 00:10:10.070 --> 00:10:11.240 align:middle line:90% And sometimes those things-- 00:10:11.240 --> 00:10:13.430 align:middle line:84% sometimes they were fundamental frictions, 00:10:13.430 --> 00:10:15.380 align:middle line:84% like you might have said, oh, it's technology, or whatever. 00:10:15.380 --> 00:10:17.588 align:middle line:84% But actually, I wouldn't be shocked if, in this case, 00:10:17.588 --> 00:10:19.428 align:middle line:90% it was just innovation. 00:10:19.428 --> 00:10:20.970 align:middle line:84% Like, someone came up with some ideas 00:10:20.970 --> 00:10:24.867 align:middle line:84% and said, oh, we think we can basically big bang-- 00:10:24.867 --> 00:10:26.700 align:middle line:84% you'll see one of the things is that there's 00:10:26.700 --> 00:10:27.650 align:middle line:90% lots of monitoring costs. 00:10:27.650 --> 00:10:28.980 align:middle line:84% It's a question of-- big banks may 00:10:28.980 --> 00:10:30.840 align:middle line:84% have said it's not profitable to make these little loans 00:10:30.840 --> 00:10:32.290 align:middle line:90% because it's too expensive. 00:10:32.290 --> 00:10:35.100 align:middle line:84% And I think that some of these organizations 00:10:35.100 --> 00:10:36.632 align:middle line:90% figured out a way to do it. 00:10:36.632 --> 00:10:39.090 align:middle line:84% So I think it may have just been organizational innovation. 00:10:39.090 --> 00:10:40.740 align:middle line:84% I do think one thing that's changing, 00:10:40.740 --> 00:10:42.270 align:middle line:84% and one thing, I'm not going to actually talk a lot now-- 00:10:42.270 --> 00:10:45.300 align:middle line:84% about now, but I think is an interesting kind of new wave, 00:10:45.300 --> 00:10:47.560 align:middle line:84% is digital technology and digital lending. 00:10:47.560 --> 00:10:49.620 align:middle line:84% And I think that actually fintech, 00:10:49.620 --> 00:10:53.280 align:middle line:84% and how do you think about using new technologies to solve 00:10:53.280 --> 00:10:55.050 align:middle line:90% some of these problems-- 00:10:55.050 --> 00:10:57.810 align:middle line:84% that may be something that's actually new 00:10:57.810 --> 00:10:59.435 align:middle line:84% that wasn't available in the past, 00:10:59.435 --> 00:11:01.060 align:middle line:84% and that may open up new opportunities. 00:11:01.060 --> 00:11:03.393 align:middle line:84% I'm not going to talk about fintech today, but that is-- 00:11:03.393 --> 00:11:05.940 align:middle line:84% that's-- so this stuff is organizational. 00:11:05.940 --> 00:11:08.110 align:middle line:90% I'm not sure any of it-- 00:11:08.110 --> 00:11:10.140 align:middle line:84% why was it done in 1976 and not 1956? 00:11:10.140 --> 00:11:13.200 align:middle line:84% I don't necessarily know that there's a hard and fast reason 00:11:13.200 --> 00:11:14.700 align:middle line:90% for that, a fundamental reason. 00:11:14.700 --> 00:11:18.330 align:middle line:84% The fintech stuff where we're using your cell phone records 00:11:18.330 --> 00:11:20.685 align:middle line:84% as a way of predicting your income and repayment 00:11:20.685 --> 00:11:23.310 align:middle line:84% status-- that actually does rely on a technological innovation, 00:11:23.310 --> 00:11:23.852 align:middle line:90% which is new. 00:11:23.852 --> 00:11:27.760 align:middle line:90% 00:11:27.760 --> 00:11:30.100 align:middle line:84% So why is evaluating this challenging? 00:11:30.100 --> 00:11:34.150 align:middle line:90% 00:11:34.150 --> 00:11:35.900 align:middle line:84% Why is evaluating microcredit challenging? 00:11:35.900 --> 00:11:37.940 align:middle line:84% It took a long time before we had good impact 00:11:37.940 --> 00:11:40.640 align:middle line:90% evaluations of microcredit. 00:11:40.640 --> 00:11:42.360 align:middle line:84% Wesley, you want to say anything? 00:11:42.360 --> 00:11:45.660 align:middle line:84% AUDIENCE: Well, like, logistically difficult or like 00:11:45.660 --> 00:11:47.160 align:middle line:90% in the real world? 00:11:47.160 --> 00:11:50.190 align:middle line:84% For example, microcredit institutions, 00:11:50.190 --> 00:11:52.350 align:middle line:84% if they believe that this is a silver bullet, 00:11:52.350 --> 00:11:53.910 align:middle line:90% may not want you-- 00:11:53.910 --> 00:11:56.922 align:middle line:84% they have no incentive to have you evaluate them. 00:11:56.922 --> 00:11:57.630 align:middle line:90% BEN OLKEN: Right. 00:11:57.630 --> 00:11:59.160 align:middle line:90% So that's point one, by the way. 00:11:59.160 --> 00:12:02.220 align:middle line:84% So vis-a-vis-- as long as they're profitable, or at least 00:12:02.220 --> 00:12:03.810 align:middle line:84% breaking even, they're good, right? 00:12:03.810 --> 00:12:04.720 align:middle line:90% They may think they're good. 00:12:04.720 --> 00:12:06.610 align:middle line:84% And so why would they want someone to evaluate them? 00:12:06.610 --> 00:12:07.810 align:middle line:90% So it's kind of risky for them. 00:12:07.810 --> 00:12:09.450 align:middle line:84% I actually think that's actually a fun-- 00:12:09.450 --> 00:12:10.242 align:middle line:90% an important point. 00:12:10.242 --> 00:12:14.163 align:middle line:84% And actually, I think convincing people to be evaluated is-- 00:12:14.163 --> 00:12:16.330 align:middle line:84% to evaluate themselves is actually really difficult. 00:12:16.330 --> 00:12:18.940 align:middle line:84% So that's part of it, especially if we're interested in not 00:12:18.940 --> 00:12:19.440 align:middle line:90% just-- 00:12:19.440 --> 00:12:21.390 align:middle line:84% like, they want to know what the impacts on profits 00:12:21.390 --> 00:12:22.710 align:middle line:84% are, but they don't need a rigorous impact evaluation 00:12:22.710 --> 00:12:24.450 align:middle line:90% to figure that out, right? 00:12:24.450 --> 00:12:26.003 align:middle line:84% And they may not necessarily want 00:12:26.003 --> 00:12:28.170 align:middle line:84% to know what the world looks like this, for example. 00:12:28.170 --> 00:12:29.503 align:middle line:90% That might be bad news for them. 00:12:29.503 --> 00:12:30.420 align:middle line:90% So I agree. 00:12:30.420 --> 00:12:31.363 align:middle line:90% That's part of it. 00:12:31.363 --> 00:12:33.780 align:middle line:84% But then what else about-- what about designing the impact 00:12:33.780 --> 00:12:34.445 align:middle line:90% evaluations? 00:12:34.445 --> 00:12:36.195 align:middle line:84% What are some things that might be tricky? 00:12:36.195 --> 00:12:40.162 align:middle line:90% 00:12:40.162 --> 00:12:40.870 align:middle line:90% Yeah, [? Sean? ?] 00:12:40.870 --> 00:12:42.727 align:middle line:84% AUDIENCE: I think there's 50 different dimensions of impact 00:12:42.727 --> 00:12:45.185 align:middle line:84% so you'd have to know what to measure and on what timeline. 00:12:45.185 --> 00:12:45.852 align:middle line:90% BEN OLKEN: Yeah. 00:12:45.852 --> 00:12:48.220 align:middle line:84% So you have lots of different outcomes, long timelines. 00:12:48.220 --> 00:12:51.570 align:middle line:90% 00:12:51.570 --> 00:12:53.850 align:middle line:84% So one of you, in your comments about-- that you 00:12:53.850 --> 00:12:56.400 align:middle line:84% sent in for today about the [INAUDIBLE] paper 00:12:56.400 --> 00:12:58.950 align:middle line:90% talked about GE issues. 00:12:58.950 --> 00:13:00.720 align:middle line:84% What did-- I don't remember who said that. 00:13:00.720 --> 00:13:02.083 align:middle line:90% I can look. 00:13:02.083 --> 00:13:03.750 align:middle line:84% Do you remember-- do any of you remember 00:13:03.750 --> 00:13:06.090 align:middle line:84% mentioning general equilibrium issues in evaluating 00:13:06.090 --> 00:13:07.140 align:middle line:90% these things? 00:13:07.140 --> 00:13:07.890 align:middle line:90% You're saying yes. 00:13:07.890 --> 00:13:10.420 align:middle line:84% Was it you, [? Sean? ?] Maybe it was you, but-- no? 00:13:10.420 --> 00:13:11.490 align:middle line:90% Was it [? Becca ?] or-- 00:13:11.490 --> 00:13:13.140 align:middle line:84% AUDIENCE: I can take a stab at it. 00:13:13.140 --> 00:13:13.560 align:middle line:90% BEN OLKEN: Yeah, what? 00:13:13.560 --> 00:13:14.602 align:middle line:90% AUDIENCE: Take a stab it. 00:13:14.602 --> 00:13:15.490 align:middle line:90% BEN OLKEN: Go for it. 00:13:15.490 --> 00:13:19.860 align:middle line:84% AUDIENCE: So if you're not satisfying [INAUDIBLE] in that 00:13:19.860 --> 00:13:21.938 align:middle line:90% if everybody in your village-- 00:13:21.938 --> 00:13:23.730 align:middle line:84% like, if some of the people in your village 00:13:23.730 --> 00:13:25.680 align:middle line:84% have access to microcredit and some people 00:13:25.680 --> 00:13:28.980 align:middle line:84% in the neighboring village don't have access, but as a function 00:13:28.980 --> 00:13:31.350 align:middle line:84% of a neighboring village having access 00:13:31.350 --> 00:13:35.130 align:middle line:84% they're being able to fund different investment projects, 00:13:35.130 --> 00:13:40.080 align:middle line:84% can boost up the availability of investment, 00:13:40.080 --> 00:13:41.790 align:middle line:84% profitable investments, available to you. 00:13:41.790 --> 00:13:42.663 align:middle line:90% And also-- 00:13:42.663 --> 00:13:43.830 align:middle line:90% BEN OLKEN: Yes, so say more. 00:13:43.830 --> 00:13:46.170 align:middle line:84% So you're saying something-- so I don't fully 00:13:46.170 --> 00:13:47.760 align:middle line:90% follow, but-- so say more. 00:13:47.760 --> 00:13:50.280 align:middle line:84% AUDIENCE: You're trying to evaluate this by, for instance, 00:13:50.280 --> 00:13:52.020 align:middle line:84% allowing some villages to have access 00:13:52.020 --> 00:13:55.200 align:middle line:84% to microcredit and other villagers not. 00:13:55.200 --> 00:13:57.330 align:middle line:84% As long as there's some form of interaction 00:13:57.330 --> 00:14:03.245 align:middle line:90% between those economies, the-- 00:14:03.245 --> 00:14:05.283 align:middle line:84% you have network effects just by virtue of-- 00:14:05.283 --> 00:14:05.950 align:middle line:90% BEN OLKEN: Yeah. 00:14:05.950 --> 00:14:07.570 align:middle line:90% So the main issue-- 00:14:07.570 --> 00:14:08.980 align:middle line:90% actually, even easier-- yeah. 00:14:08.980 --> 00:14:11.770 align:middle line:84% The main issue is you're doing lending, right? 00:14:11.770 --> 00:14:14.770 align:middle line:84% But people are interacting with each other in the underlying 00:14:14.770 --> 00:14:15.700 align:middle line:90% markets. 00:14:15.700 --> 00:14:18.040 align:middle line:84% So actually, I think the village one is actually better. 00:14:18.040 --> 00:14:20.080 align:middle line:84% Imagine-- what would be the problem if you do this-- 00:14:20.080 --> 00:14:22.247 align:middle line:84% imagine you randomize people at the individual level 00:14:22.247 --> 00:14:23.410 align:middle line:90% and access to credit. 00:14:23.410 --> 00:14:25.450 align:middle line:84% What would be the problem with that? 00:14:25.450 --> 00:14:25.950 align:middle line:90% Yeah. 00:14:25.950 --> 00:14:27.190 align:middle line:84% AUDIENCE: You usually have a surge 00:14:27.190 --> 00:14:28.720 align:middle line:84% in aggregate demand within the village 00:14:28.720 --> 00:14:30.220 align:middle line:84% and people would immediately start-- 00:14:30.220 --> 00:14:33.450 align:middle line:90% 00:14:33.450 --> 00:14:37.073 align:middle line:84% or I guess either on the supply side or the demand side. 00:14:37.073 --> 00:14:37.740 align:middle line:90% BEN OLKEN: Yeah. 00:14:37.740 --> 00:14:38.640 align:middle line:90% So you're thinking-- so, right. 00:14:38.640 --> 00:14:40.650 align:middle line:84% So your view is you would understate it because-- 00:14:40.650 --> 00:14:42.330 align:middle line:84% so your view is you would understate the impacts 00:14:42.330 --> 00:14:43.460 align:middle line:84% because there would be demand effects, 00:14:43.460 --> 00:14:44.910 align:middle line:84% and that would spill over to the control group. 00:14:44.910 --> 00:14:47.040 align:middle line:84% Anyone think you would overstate impacts? 00:14:47.040 --> 00:14:48.090 align:middle line:90% [? Sean? ?] 00:14:48.090 --> 00:14:50.850 align:middle line:84% AUDIENCE: Yeah, because there's diminishing returns [INAUDIBLE] 00:14:50.850 --> 00:14:51.781 align:middle line:90% that you-- 00:14:51.781 --> 00:14:53.906 align:middle line:84% like suppose there's limited demand in the village. 00:14:53.906 --> 00:14:54.370 align:middle line:90% BEN OLKEN: What? 00:14:54.370 --> 00:14:55.630 align:middle line:84% AUDIENCE: Suppose there's limited demand 00:14:55.630 --> 00:14:56.755 align:middle line:90% in a village for something. 00:14:56.755 --> 00:14:59.530 align:middle line:84% And all the loans are for local, I don't know, retail shops, 00:14:59.530 --> 00:15:01.793 align:middle line:84% then if you get loans to one person, they can-- 00:15:01.793 --> 00:15:04.210 align:middle line:84% they'll grow a lot, but if you gave the loans to everyone, 00:15:04.210 --> 00:15:05.590 align:middle line:90% you're keeping the same-- 00:15:05.590 --> 00:15:05.740 align:middle line:90% BEN OLKEN: Yeah. 00:15:05.740 --> 00:15:06.100 align:middle line:90% Exactly. 00:15:06.100 --> 00:15:06.940 align:middle line:84% So I think you also have to think 00:15:06.940 --> 00:15:08.470 align:middle line:84% about what these people are doing with their money. 00:15:08.470 --> 00:15:09.730 align:middle line:90% So I think that you're right. 00:15:09.730 --> 00:15:11.200 align:middle line:90% There could be these demand-- 00:15:11.200 --> 00:15:12.430 align:middle line:90% Keynesian demand effects. 00:15:12.430 --> 00:15:14.055 align:middle line:84% But [? Sean ?] is also right that there 00:15:14.055 --> 00:15:15.715 align:middle line:84% could be potential spillovers in terms 00:15:15.715 --> 00:15:16.840 align:middle line:90% of-- in the product market. 00:15:16.840 --> 00:15:19.090 align:middle line:90% So imagine we're all making-- 00:15:19.090 --> 00:15:20.605 align:middle line:90% we're all selling the same-- 00:15:20.605 --> 00:15:23.650 align:middle line:90% 00:15:23.650 --> 00:15:26.590 align:middle line:84% imagine, we're-- what is an example? 00:15:26.590 --> 00:15:30.220 align:middle line:84% We're all fried rice vendors, use my Indonesian examples. 00:15:30.220 --> 00:15:31.910 align:middle line:84% We're all fried-- it's very common. 00:15:31.910 --> 00:15:34.680 align:middle line:84% It's super common, actually, example, is that people have-- 00:15:34.680 --> 00:15:36.400 align:middle line:84% they have little fried rice carts. 00:15:36.400 --> 00:15:39.200 align:middle line:84% And there's only so much demand for fried rice going around, 00:15:39.200 --> 00:15:40.975 align:middle line:84% so if I have a better fried rice cart 00:15:40.975 --> 00:15:43.600 align:middle line:84% and I can move around-- maybe I get a motorbike fried rice cart 00:15:43.600 --> 00:15:45.670 align:middle line:84% so I can zip around faster or something, 00:15:45.670 --> 00:15:48.700 align:middle line:84% I might just be stealing business from other people. 00:15:48.700 --> 00:15:50.710 align:middle line:84% If the demand for fried rice is reasonably 00:15:50.710 --> 00:15:53.700 align:middle line:84% inelastic in my village, it'll look like I'm doing great, 00:15:53.700 --> 00:15:56.200 align:middle line:84% but actually, I'm doing great at the expense of other people 00:15:56.200 --> 00:15:58.450 align:middle line:90% who didn't get the loan. 00:15:58.450 --> 00:16:01.525 align:middle line:90% So I think that the-- 00:16:01.525 --> 00:16:04.030 align:middle line:84% you also have to think about these issues. 00:16:04.030 --> 00:16:06.280 align:middle line:84% And actually, I think actually that suggests 00:16:06.280 --> 00:16:08.530 align:middle line:84% that larger level randomizations, 00:16:08.530 --> 00:16:10.427 align:middle line:84% a village-level randomization or even 00:16:10.427 --> 00:16:12.760 align:middle line:84% subdistrict-level randomization is going to work better, 00:16:12.760 --> 00:16:15.790 align:middle line:84% where you-- better than an individual-level thing, where 00:16:15.790 --> 00:16:18.280 align:middle line:84% you have to worry about both these positive and negative 00:16:18.280 --> 00:16:19.565 align:middle line:90% spillover effects. 00:16:19.565 --> 00:16:21.940 align:middle line:84% So I just wanted you to think about some of these issues. 00:16:21.940 --> 00:16:23.920 align:middle line:84% And there's another thing, also, which 00:16:23.920 --> 00:16:26.897 align:middle line:84% is if we think the heterogeneity is important, 00:16:26.897 --> 00:16:28.980 align:middle line:84% we need to think about measuring the heterogeneity 00:16:28.980 --> 00:16:30.780 align:middle line:84% and make sure we can measure the relevant dimension 00:16:30.780 --> 00:16:31.410 align:middle line:90% of the heterogeneity. 00:16:31.410 --> 00:16:32.790 align:middle line:90% And the paper you guys read-- 00:16:32.790 --> 00:16:34.560 align:middle line:84% I guess it was for last time, but we're going to talk-- 00:16:34.560 --> 00:16:36.630 align:middle line:84% we're going to see today was really about saying, 00:16:36.630 --> 00:16:38.460 align:middle line:84% is there real heterogeneity in returns? 00:16:38.460 --> 00:16:39.480 align:middle line:84% And as you saw from that paper, there's 00:16:39.480 --> 00:16:40.860 align:middle line:84% a lot of heterogeneity in returns. 00:16:40.860 --> 00:16:42.860 align:middle line:84% And thinking about can you measure that properly 00:16:42.860 --> 00:16:44.780 align:middle line:90% is really important. 00:16:44.780 --> 00:16:46.570 align:middle line:90% OK. 00:16:46.570 --> 00:16:51.117 align:middle line:84% So I said, for a while, as I think someone pointed out, 00:16:51.117 --> 00:16:53.420 align:middle line:84% I don't remember who-- was it you, 00:16:53.420 --> 00:16:56.780 align:middle line:84% [? Wesley? ?] Basically, look, the reason was as follows. 00:16:56.780 --> 00:16:57.530 align:middle line:90% We're profitable. 00:16:57.530 --> 00:16:58.313 align:middle line:90% We're doing good. 00:16:58.313 --> 00:17:00.980 align:middle line:84% Why does anyone need to-- why do we need to evaluate our impact? 00:17:00.980 --> 00:17:03.355 align:middle line:84% And that kind of reasoning was very powerful for a while. 00:17:03.355 --> 00:17:07.555 align:middle line:90% 00:17:07.555 --> 00:17:08.930 align:middle line:84% And there are problems with this. 00:17:08.930 --> 00:17:12.262 align:middle line:84% Of course, number one is maybe they're not-- 00:17:12.262 --> 00:17:14.720 align:middle line:84% they may not be doing quite as well as they think they are. 00:17:14.720 --> 00:17:16.891 align:middle line:84% But number two is that as I think 00:17:16.891 --> 00:17:18.349 align:middle line:84% that Rick, as you pointed out, just 00:17:18.349 --> 00:17:19.970 align:middle line:84% because the fact that people are borrowing 00:17:19.970 --> 00:17:21.020 align:middle line:84% from microfinance organizations is not-- 00:17:21.020 --> 00:17:22.220 align:middle line:90% is actually good for them. 00:17:22.220 --> 00:17:25.143 align:middle line:84% And in particular, we have these negative people that could 00:17:25.143 --> 00:17:26.310 align:middle line:90% lead to debt trap and so on. 00:17:26.310 --> 00:17:30.840 align:middle line:84% So actually measuring these outcomes is really important. 00:17:30.840 --> 00:17:33.963 align:middle line:84% And by the way, this is coming to a huge-- 00:17:33.963 --> 00:17:34.880 align:middle line:90% this has really been-- 00:17:34.880 --> 00:17:37.100 align:middle line:84% this question of, is this good or bad, 00:17:37.100 --> 00:17:39.207 align:middle line:90% has led to major policy debates. 00:17:39.207 --> 00:17:40.790 align:middle line:84% For example, there was a confrontation 00:17:40.790 --> 00:17:43.250 align:middle line:84% in Andhra Pradesh in India between the state of AP 00:17:43.250 --> 00:17:44.960 align:middle line:84% and the microfinance institutions, 00:17:44.960 --> 00:17:47.780 align:middle line:84% where basically, at some point, the state government said, 00:17:47.780 --> 00:17:49.850 align:middle line:84% this is a terrible idea, and basically, no one 00:17:49.850 --> 00:17:52.340 align:middle line:84% has to repay all their microfinance loans. 00:17:52.340 --> 00:17:55.760 align:middle line:84% And that whole sector went belly up in that state overnight. 00:17:55.760 --> 00:17:59.735 align:middle line:90% And actually, I think-- 00:17:59.735 --> 00:18:01.110 align:middle line:84% I believe Emily Breza and Cynthia 00:18:01.110 --> 00:18:03.050 align:middle line:84% Kinnan have a paper studying that incident, 00:18:03.050 --> 00:18:07.060 align:middle line:90% and what happened from there on. 00:18:07.060 --> 00:18:07.560 align:middle line:90% OK. 00:18:07.560 --> 00:18:14.030 align:middle line:84% So there has been, however, a bunch of RCTs 00:18:14.030 --> 00:18:17.550 align:middle line:84% that then, after a while, tried to provide evidence on this. 00:18:17.550 --> 00:18:20.270 align:middle line:84% And there was actually a special issue in the journal AEJ 00:18:20.270 --> 00:18:26.522 align:middle line:84% Applied which put together six of these studies from India, 00:18:26.522 --> 00:18:27.980 align:middle line:84% Ethiopia, Mexico, Mongolia, Bosnia, 00:18:27.980 --> 00:18:29.870 align:middle line:84% and Morocco, all in one place and tried 00:18:29.870 --> 00:18:36.340 align:middle line:84% to harmonize them so people could see what they meant. 00:18:36.340 --> 00:18:41.460 align:middle line:84% And they basically had two different designs. 00:18:41.460 --> 00:18:43.840 align:middle line:84% So one was this place-based randomization. 00:18:43.840 --> 00:18:45.340 align:middle line:84% And that-- the advantage of that one 00:18:45.340 --> 00:18:47.340 align:middle line:84% is to try to deal with some of these spillovers. 00:18:47.340 --> 00:18:51.300 align:middle line:84% So the idea is we're going to say, look, 00:18:51.300 --> 00:18:53.000 align:middle line:84% the microfinance organization says 00:18:53.000 --> 00:18:56.870 align:middle line:84% we're going to think about expanding to a bunch of places. 00:18:56.870 --> 00:18:59.060 align:middle line:84% We're going to pick more places, and we're actually 00:18:59.060 --> 00:19:00.560 align:middle line:84% going expand to, and we'll randomize them. 00:19:00.560 --> 00:19:02.660 align:middle line:84% Maybe we'll randomize the order of rollout, for example. 00:19:02.660 --> 00:19:04.160 align:middle line:84% That's actually a pretty common one, 00:19:04.160 --> 00:19:06.650 align:middle line:84% to say, look, you pick the places you're going to go. 00:19:06.650 --> 00:19:08.273 align:middle line:84% You can't open them all up at once. 00:19:08.273 --> 00:19:10.190 align:middle line:84% And we'll just randomize the order of rollout. 00:19:10.190 --> 00:19:11.607 align:middle line:84% That's actually a very nice design 00:19:11.607 --> 00:19:17.120 align:middle line:84% that gives them some flexibility to control where they're going, 00:19:17.120 --> 00:19:19.430 align:middle line:84% works with their business plans, and operates on 00:19:19.430 --> 00:19:21.222 align:middle line:84% this dimension they're indifferent between, 00:19:21.222 --> 00:19:22.490 align:middle line:90% which is the order. 00:19:22.490 --> 00:19:27.410 align:middle line:84% Another one is this idea that's randomization on the bubble. 00:19:27.410 --> 00:19:30.615 align:middle line:84% So the idea here is that basically, the MFI scores 00:19:30.615 --> 00:19:32.240 align:middle line:84% applicants in terms of how excited they 00:19:32.240 --> 00:19:33.320 align:middle line:90% are to lend to them. 00:19:33.320 --> 00:19:36.658 align:middle line:84% And then rather than-- you could just have an RD on some cutoff, 00:19:36.658 --> 00:19:38.450 align:middle line:84% or another thing you could do is say, look, 00:19:38.450 --> 00:19:40.460 align:middle line:84% the ones that are really awesome you lend to. 00:19:40.460 --> 00:19:42.140 align:middle line:84% The ones that are not really awesome-- that are really 00:19:42.140 --> 00:19:43.040 align:middle line:90% terrible, you don't lend to. 00:19:43.040 --> 00:19:45.260 align:middle line:84% The ones in the middle, then maybe we'll randomize. 00:19:45.260 --> 00:19:47.718 align:middle line:84% And they also can be-- because they're roughly indifferent, 00:19:47.718 --> 00:19:50.330 align:middle line:90% they can be open to that, too. 00:19:50.330 --> 00:19:52.220 align:middle line:84% I think both of these are useful, by the way, 00:19:52.220 --> 00:19:53.600 align:middle line:84% as you're thinking about experimental design 00:19:53.600 --> 00:19:54.815 align:middle line:90% because both of these are-- 00:19:54.815 --> 00:19:57.800 align:middle line:90% 00:19:57.800 --> 00:20:00.170 align:middle line:84% one kind of problem you have when you're often designing 00:20:00.170 --> 00:20:03.260 align:middle line:84% experiments is you would like to randomize over 00:20:03.260 --> 00:20:04.380 align:middle line:90% an entire population. 00:20:04.380 --> 00:20:05.270 align:middle line:84% The person you're dealing with is 00:20:05.270 --> 00:20:08.030 align:middle line:84% like, no way am I going to do that because that would involve 00:20:08.030 --> 00:20:10.220 align:middle line:84% making a bunch of really stupid loans 00:20:10.220 --> 00:20:13.280 align:middle line:84% or forgoing a bunch of really great loans. 00:20:13.280 --> 00:20:16.250 align:middle line:90% Both of these have this-- 00:20:16.250 --> 00:20:20.270 align:middle line:84% both of these designs have the feature 00:20:20.270 --> 00:20:22.808 align:middle line:84% that you're exploiting the fact that-- 00:20:22.808 --> 00:20:25.100 align:middle line:84% you're figuring out what they're roughly indifferent on 00:20:25.100 --> 00:20:26.340 align:middle line:90% and doing that. 00:20:26.340 --> 00:20:28.788 align:middle line:84% And you should just be aware of that-- 00:20:28.788 --> 00:20:30.830 align:middle line:84% in general, if you're ever in the world of trying 00:20:30.830 --> 00:20:34.410 align:middle line:84% to negotiate what a possible randomization is, 00:20:34.410 --> 00:20:35.785 align:middle line:84% is just because you wanted to run 00:20:35.785 --> 00:20:38.368 align:middle line:84% a randomized experiment does not mean your partner has to give 00:20:38.368 --> 00:20:39.620 align:middle line:90% up all control whatsoever. 00:20:39.620 --> 00:20:41.337 align:middle line:84% They can say, yes, we want to do it here. 00:20:41.337 --> 00:20:42.920 align:middle line:84% No, we don't want to do it over there. 00:20:42.920 --> 00:20:45.650 align:middle line:84% Where are you-- the key is where are you 00:20:45.650 --> 00:20:48.500 align:middle line:84% roughly indifferent in terms of either place or time 00:20:48.500 --> 00:20:49.250 align:middle line:90% or something. 00:20:49.250 --> 00:20:51.290 align:middle line:90% And maybe we can randomize that. 00:20:51.290 --> 00:20:53.390 align:middle line:84% And that's certainly an approach that I've 00:20:53.390 --> 00:20:57.900 align:middle line:84% found some success with as well in very different contexts. 00:20:57.900 --> 00:20:59.790 align:middle line:84% Another thing that's really important 00:20:59.790 --> 00:21:03.120 align:middle line:84% is takeup rates are not really enormous. 00:21:03.120 --> 00:21:08.850 align:middle line:84% So that's not surprising when you think about it. 00:21:08.850 --> 00:21:12.960 align:middle line:84% Like, if I came to your town, wherever 00:21:12.960 --> 00:21:17.250 align:middle line:84% you guys from and said, hey, I'm opening a new bank. 00:21:17.250 --> 00:21:18.930 align:middle line:84% I'm opening loans to start a business. 00:21:18.930 --> 00:21:20.730 align:middle line:84% Like, what fraction of people in your town 00:21:20.730 --> 00:21:21.630 align:middle line:84% do you think would be like, oh, I'm 00:21:21.630 --> 00:21:23.160 align:middle line:84% really excited to take up this loan. 00:21:23.160 --> 00:21:26.613 align:middle line:84% Some people would, but not-- but it's not going to be everybody. 00:21:26.613 --> 00:21:28.530 align:middle line:84% It's going to-- and in fact, the take-up rates 00:21:28.530 --> 00:21:30.363 align:middle line:84% look more like 10%, which actually, I think, 00:21:30.363 --> 00:21:33.060 align:middle line:84% is pretty big when you think about it, 00:21:33.060 --> 00:21:36.100 align:middle line:84% but small as a fraction of the overall population. 00:21:36.100 --> 00:21:39.630 align:middle line:84% So why is that an issue for these designs? 00:21:39.630 --> 00:21:47.870 align:middle line:90% 00:21:47.870 --> 00:21:49.160 align:middle line:90% [? Aaron ?] [INAUDIBLE]? 00:21:49.160 --> 00:21:51.710 align:middle line:84% AUDIENCE: Well, if one of the key things we want to look at 00:21:51.710 --> 00:21:53.621 align:middle line:90% is heterogeneity, then just-- 00:21:53.621 --> 00:21:55.288 align:middle line:84% it's going to be very hard to have power 00:21:55.288 --> 00:21:56.990 align:middle line:84% to do that, you know, to take out this loan? 00:21:56.990 --> 00:21:57.940 align:middle line:90% BEN OLKEN: So, yes, that's true. 00:21:57.940 --> 00:21:59.607 align:middle line:84% Not a lot of power of heterogeneity, but 00:21:59.607 --> 00:22:01.300 align:middle line:84% how about just for the overall impact? 00:22:01.300 --> 00:22:03.842 align:middle line:84% Why is this a problem for just estimating the overall impact? 00:22:03.842 --> 00:22:06.120 align:middle line:90% 00:22:06.120 --> 00:22:06.990 align:middle line:90% Patrick? 00:22:06.990 --> 00:22:09.550 align:middle line:84% AUDIENCE: Might self-select if they believe it will have 00:22:09.550 --> 00:22:10.520 align:middle line:90% a bigger impact-- 00:22:10.520 --> 00:22:11.800 align:middle line:84% BEN OLKEN: So you're going to get self-selection. 00:22:11.800 --> 00:22:12.280 align:middle line:90% That's true. 00:22:12.280 --> 00:22:12.850 align:middle line:90% But that's OK. 00:22:12.850 --> 00:22:13.750 align:middle line:84% You're going to get the relevant-- you're 00:22:13.750 --> 00:22:15.100 align:middle line:84% going to get the impact on the people who are 00:22:15.100 --> 00:22:16.267 align:middle line:90% interested in being treated. 00:22:16.267 --> 00:22:19.560 align:middle line:84% It's a relevant [? lead. ?] [? Becca, ?] was that what you 00:22:19.560 --> 00:22:21.530 align:middle line:84% were going to say, or something else? 00:22:21.530 --> 00:22:23.670 align:middle line:90% AUDIENCE: Oh. 00:22:23.670 --> 00:22:24.230 align:middle line:90% Oh, yeah. 00:22:24.230 --> 00:22:25.696 align:middle line:84% I guess it just sort of [INAUDIBLE].. 00:22:25.696 --> 00:22:26.740 align:middle line:90% BEN OLKEN: So related to what? 00:22:26.740 --> 00:22:27.407 align:middle line:90% AUDIENCE: Power. 00:22:27.407 --> 00:22:29.030 align:middle line:90% BEN OLKEN: So say more about it. 00:22:29.030 --> 00:22:31.640 align:middle line:84% AUDIENCE: So if you have really low take-up, 00:22:31.640 --> 00:22:37.380 align:middle line:84% even if you are-- the treatment was being induced or given 00:22:37.380 --> 00:22:39.090 align:middle line:84% the option to use it, then you need 00:22:39.090 --> 00:22:41.328 align:middle line:90% to do some sort of [INAUDIBLE]. 00:22:41.328 --> 00:22:42.627 align:middle line:90% BEN OLKEN: Yeah. 00:22:42.627 --> 00:22:43.960 align:middle line:90% AUDIENCE: --measure that effect. 00:22:43.960 --> 00:22:44.627 align:middle line:90% BEN OLKEN: Yeah. 00:22:44.627 --> 00:22:46.220 align:middle line:84% AUDIENCE: And that's going to be-- 00:22:46.220 --> 00:22:48.515 align:middle line:84% you're going to lose a lot of power. 00:22:48.515 --> 00:22:49.140 align:middle line:90% BEN OLKEN: Yes. 00:22:49.140 --> 00:22:49.830 align:middle line:90% Exactly. 00:22:49.830 --> 00:22:51.615 align:middle line:90% So that's exactly-- yeah. 00:22:51.615 --> 00:22:52.990 align:middle line:84% AUDIENCE: Why is power concerning 00:22:52.990 --> 00:22:54.558 align:middle line:90% if the costs are dependent on-- 00:22:54.558 --> 00:22:55.350 align:middle line:90% BEN OLKEN: If what? 00:22:55.350 --> 00:22:56.730 align:middle line:84% AUDIENCE: Why is power concerning 00:22:56.730 --> 00:22:59.490 align:middle line:84% if our costs of operations are dependent on how many people 00:22:59.490 --> 00:23:00.140 align:middle line:90% take it up? 00:23:00.140 --> 00:23:00.640 align:middle line:90% So-- 00:23:00.640 --> 00:23:00.900 align:middle line:90% BEN OLKEN: OK. 00:23:00.900 --> 00:23:01.690 align:middle line:90% So it's a great question. 00:23:01.690 --> 00:23:02.550 align:middle line:90% So why is this a problem? 00:23:02.550 --> 00:23:03.570 align:middle line:90% Why is power concern? 00:23:03.570 --> 00:23:03.810 align:middle line:90% OK. 00:23:03.810 --> 00:23:04.710 align:middle line:90% Now I have to do the-- 00:23:04.710 --> 00:23:05.940 align:middle line:84% which class did I explain this to? 00:23:05.940 --> 00:23:07.770 align:middle line:84% I did not explain this in this class yet, right? 00:23:07.770 --> 00:23:08.910 align:middle line:84% That was in the other class, right? 00:23:08.910 --> 00:23:09.120 align:middle line:90% OK. 00:23:09.120 --> 00:23:09.660 align:middle line:90% Good. 00:23:09.660 --> 00:23:10.050 align:middle line:90% All right. 00:23:10.050 --> 00:23:10.770 align:middle line:84% AUDIENCE: You did it in this class. 00:23:10.770 --> 00:23:11.490 align:middle line:84% BEN OLKEN: I did it in this class. 00:23:11.490 --> 00:23:11.940 align:middle line:90% Right, OK, good. 00:23:11.940 --> 00:23:12.790 align:middle line:90% Then you should know the answer. 00:23:12.790 --> 00:23:13.877 align:middle line:90% So why is it a problem? 00:23:13.877 --> 00:23:15.960 align:middle line:84% AUDIENCE: No, I'm asking why power [INAUDIBLE]---- 00:23:15.960 --> 00:23:18.460 align:middle line:84% BEN OLKEN: I can never remember which class I teach this in, 00:23:18.460 --> 00:23:19.180 align:middle line:90% same basic ideas. 00:23:19.180 --> 00:23:20.830 align:middle line:84% AUDIENCE: If take-up is only 10%-- 00:23:20.830 --> 00:23:23.070 align:middle line:84% because it seems like your costs are also not-- 00:23:23.070 --> 00:23:25.080 align:middle line:84% are only being applied to 10%, so you could just 00:23:25.080 --> 00:23:27.090 align:middle line:84% expand the program if your costs are 00:23:27.090 --> 00:23:28.510 align:middle line:90% lower because of this take up. 00:23:28.510 --> 00:23:29.218 align:middle line:90% BEN OLKEN: Right. 00:23:29.218 --> 00:23:30.520 align:middle line:90% So you're right. 00:23:30.520 --> 00:23:31.790 align:middle line:90% So that's the question. 00:23:31.790 --> 00:23:33.430 align:middle line:90% Why would I-- so, OK. 00:23:33.430 --> 00:23:34.930 align:middle line:84% So if I already went through this, 00:23:34.930 --> 00:23:36.290 align:middle line:90% we can ask the question again. 00:23:36.290 --> 00:23:44.020 align:middle line:84% Why would I rather have 100% take up on a sample of size 100 00:23:44.020 --> 00:23:45.580 align:middle line:90% or a 10% sample-- 00:23:45.580 --> 00:23:47.530 align:middle line:84% take up on a sample of size 1,000? 00:23:47.530 --> 00:23:49.222 align:middle line:90% Like, which is better? 00:23:49.222 --> 00:23:50.180 align:middle line:90% AUDIENCE: I mean, yeah. 00:23:50.180 --> 00:23:55.580 align:middle line:84% I do get that the smaller sample [INAUDIBLE] would be better. 00:23:55.580 --> 00:23:58.390 align:middle line:84% But you would partially address that 00:23:58.390 --> 00:24:00.420 align:middle line:90% with a bigger sample, right? 00:24:00.420 --> 00:24:01.230 align:middle line:90% BEN OLKEN: But no. 00:24:01.230 --> 00:24:02.250 align:middle line:90% That's the whole point. 00:24:02.250 --> 00:24:04.330 align:middle line:84% So you would have to have a much bigger problem. 00:24:04.330 --> 00:24:07.050 align:middle line:84% So that's the whole point of the calculation I went through 00:24:07.050 --> 00:24:13.560 align:middle line:84% before is that in order to get a take-up of 100% on size-- 00:24:13.560 --> 00:24:21.840 align:middle line:84% 100% on size 100 is equal to, I think, 10% on size 10,000. 00:24:21.840 --> 00:24:22.812 align:middle line:90% Not on size 1,000. 00:24:22.812 --> 00:24:23.520 align:middle line:90% That's the point. 00:24:23.520 --> 00:24:24.850 align:middle line:84% Because the square root of n issue. 00:24:24.850 --> 00:24:25.590 align:middle line:90% So you have to go-- 00:24:25.590 --> 00:24:26.820 align:middle line:84% in order to deal with low take up, 00:24:26.820 --> 00:24:28.737 align:middle line:84% you'd have to have a really, really big sample 00:24:28.737 --> 00:24:31.988 align:middle line:84% size because of this point about the square root of n. 00:24:31.988 --> 00:24:33.780 align:middle line:84% Which I did explain in this class, yes, OK. 00:24:33.780 --> 00:24:34.020 align:middle line:90% Right? 00:24:34.020 --> 00:24:34.960 align:middle line:90% So that's the point. 00:24:34.960 --> 00:24:36.750 align:middle line:84% So the fact that the first stage-- 00:24:36.750 --> 00:24:39.160 align:middle line:90% is really low means that the-- 00:24:39.160 --> 00:24:42.380 align:middle line:90% 00:24:42.380 --> 00:24:44.040 align:middle line:84% it can be hard to [INAUDIBLE] effect. 00:24:44.040 --> 00:24:45.915 align:middle line:84% So what's the right solution to this problem? 00:24:45.915 --> 00:24:50.137 align:middle line:90% 00:24:50.137 --> 00:24:51.720 align:middle line:84% So you want to do-- by the way, so you 00:24:51.720 --> 00:24:54.350 align:middle line:84% want to do a cluster randomized design for the reasons we 00:24:54.350 --> 00:24:56.100 align:middle line:84% talked about, so you have to do spillover. 00:24:56.100 --> 00:24:57.558 align:middle line:84% So how do you deal with this issue. 00:24:57.558 --> 00:24:59.250 align:middle line:90% Yeah, [? Paulo? ?] You-- 00:24:59.250 --> 00:25:01.878 align:middle line:84% AUDIENCE: You-- in different villages or even 00:25:01.878 --> 00:25:03.420 align:middle line:84% within the same village, first figure 00:25:03.420 --> 00:25:05.337 align:middle line:84% out who would be interested in getting a loan, 00:25:05.337 --> 00:25:07.490 align:middle line:84% and then randomize who you give the loan to? 00:25:07.490 --> 00:25:08.330 align:middle line:84% BEN OLKEN: Well, you don't want to randomize 00:25:08.330 --> 00:25:10.010 align:middle line:84% who you give the loan to because that would have 00:25:10.010 --> 00:25:12.635 align:middle line:84% the general equilibrium problems we were talking about earlier. 00:25:12.635 --> 00:25:15.090 align:middle line:90% 00:25:15.090 --> 00:25:15.590 align:middle line:90% Sorry. 00:25:15.590 --> 00:25:18.570 align:middle line:90% Say something else? 00:25:18.570 --> 00:25:21.725 align:middle line:84% AUDIENCE: Randomize-- do exactly what [? Paulo ?] said, and then 00:25:21.725 --> 00:25:22.850 align:middle line:90% just randomize the village. 00:25:22.850 --> 00:25:23.330 align:middle line:90% BEN OLKEN: Exactly. 00:25:23.330 --> 00:25:23.830 align:middle line:90% So, right. 00:25:23.830 --> 00:25:25.910 align:middle line:84% So therefore, what you have to do 00:25:25.910 --> 00:25:27.980 align:middle line:84% is you have to get baseline data for everybody, 00:25:27.980 --> 00:25:30.170 align:middle line:84% figure out their characteristics that 00:25:30.170 --> 00:25:31.880 align:middle line:90% are going to predict take up. 00:25:31.880 --> 00:25:33.505 align:middle line:84% Then you randomize at the village level 00:25:33.505 --> 00:25:35.713 align:middle line:84% in order to deal with the general equilibrium issues. 00:25:35.713 --> 00:25:37.850 align:middle line:84% But you're going to look at the impact for people 00:25:37.850 --> 00:25:39.380 align:middle line:90% who are likely to take up. 00:25:39.380 --> 00:25:41.930 align:middle line:84% So you can predict take up from your baseline survey. 00:25:41.930 --> 00:25:45.730 align:middle line:84% And while 10%-- maybe only 10% in general, 00:25:45.730 --> 00:25:48.230 align:middle line:84% maybe if you ask a question of, if someone were to offer you 00:25:48.230 --> 00:25:49.313 align:middle line:90% a loan, would you take it? 00:25:49.313 --> 00:25:53.713 align:middle line:84% Maybe you can get 50% or 80% take up on that subsample. 00:25:53.713 --> 00:25:55.130 align:middle line:84% And that's the strategy on how you 00:25:55.130 --> 00:25:56.338 align:middle line:90% have to deal with that issue. 00:25:56.338 --> 00:25:57.870 align:middle line:90% Is that clear? 00:25:57.870 --> 00:25:59.730 align:middle line:84% AUDIENCE: I don't think I understand. 00:25:59.730 --> 00:26:03.380 align:middle line:84% BEN OLKEN: So the key point here is that it's 10% on average, 00:26:03.380 --> 00:26:07.100 align:middle line:84% but what I want to do is make an ex-ante restriction 00:26:07.100 --> 00:26:09.948 align:middle line:84% on my sample such that the take-up rate, which 00:26:09.948 --> 00:26:11.990 align:middle line:84% I can do equally in the treatment and the control 00:26:11.990 --> 00:26:14.990 align:middle line:84% areas, such on that restricted sample the take-up rate is 00:26:14.990 --> 00:26:15.620 align:middle line:90% really high. 00:26:15.620 --> 00:26:19.250 align:middle line:84% So imagine I do a baseline survey and I ask you-- 00:26:19.250 --> 00:26:20.720 align:middle line:84% suppose a microfinance institution 00:26:20.720 --> 00:26:22.040 align:middle line:90% was to come into this town. 00:26:22.040 --> 00:26:24.260 align:middle line:90% Would you take out a loan? 00:26:24.260 --> 00:26:27.605 align:middle line:84% So people who say yes to that question, the probability 00:26:27.605 --> 00:26:29.730 align:middle line:84% of take up is going to be-- conditional on offering 00:26:29.730 --> 00:26:31.530 align:middle line:84% it is going to be like, I don't know, 80% or something. 00:26:31.530 --> 00:26:32.910 align:middle line:90% Not 100%, but 80% or something. 00:26:32.910 --> 00:26:33.910 align:middle line:90% That high number. 00:26:33.910 --> 00:26:35.640 align:middle line:84% So then what you can do is you can-- you 00:26:35.640 --> 00:26:38.740 align:middle line:84% run that survey at baseline in treatment and control areas. 00:26:38.740 --> 00:26:40.560 align:middle line:84% You restrict your analysis, then, 00:26:40.560 --> 00:26:44.088 align:middle line:84% to people who are in that high predicted take up category 00:26:44.088 --> 00:26:45.630 align:middle line:84% and then put it in the control group. 00:26:45.630 --> 00:26:47.550 align:middle line:84% So you're doing it on a baseline question, 00:26:47.550 --> 00:26:49.070 align:middle line:84% so you're making the same restriction on treatment 00:26:49.070 --> 00:26:50.790 align:middle line:84% and control areas, but you're going 00:26:50.790 --> 00:26:53.280 align:middle line:84% to have high take up on that subsample. 00:26:53.280 --> 00:26:55.530 align:middle line:84% AUDIENCE: So you're still just randomizing it, though. 00:26:55.530 --> 00:26:56.790 align:middle line:84% BEN OLKEN: You're still randomized at the village 00:26:56.790 --> 00:26:58.592 align:middle line:84% level, but you're able to run your analysis 00:26:58.592 --> 00:27:00.300 align:middle line:84% on the people for whom you expect predict 00:27:00.300 --> 00:27:03.020 align:middle line:90% to take up to be really high. 00:27:03.020 --> 00:27:04.308 align:middle line:90% Is that clear? 00:27:04.308 --> 00:27:05.600 align:middle line:90% So that's kind of what they do. 00:27:05.600 --> 00:27:09.630 align:middle line:90% 00:27:09.630 --> 00:27:12.200 align:middle line:84% And so what they find is they find basically-- 00:27:12.200 --> 00:27:15.200 align:middle line:84% they find-- in general, summing up across all of these things, 00:27:15.200 --> 00:27:18.320 align:middle line:84% I would say they find, on average, not 00:27:18.320 --> 00:27:22.550 align:middle line:84% much on consumption, but they do find 00:27:22.550 --> 00:27:23.852 align:middle line:90% some heterogeneity of impacts. 00:27:23.852 --> 00:27:25.310 align:middle line:84% And those are focused on people who 00:27:25.310 --> 00:27:26.595 align:middle line:90% have the existing businesses. 00:27:26.595 --> 00:27:28.220 align:middle line:84% And I think at some level, that's not-- 00:27:28.220 --> 00:27:29.470 align:middle line:90% that should not be surprising. 00:27:29.470 --> 00:27:31.160 align:middle line:84% Like, what is this-- this was actually 00:27:31.160 --> 00:27:32.570 align:middle line:84% kind of surprising at the time because everyone 00:27:32.570 --> 00:27:34.778 align:middle line:84% thought this thing would radically transform poverty. 00:27:34.778 --> 00:27:35.840 align:middle line:90% It doesn't do that. 00:27:35.840 --> 00:27:38.750 align:middle line:84% What it does is it gives capital to people who have 00:27:38.750 --> 00:27:40.130 align:middle line:90% high marginal returns capital. 00:27:40.130 --> 00:27:42.680 align:middle line:84% And those particular individuals can grow their business. 00:27:42.680 --> 00:27:45.170 align:middle line:84% And that's-- people who have existing businesses may be 00:27:45.170 --> 00:27:49.980 align:middle line:84% the ones for whom the capital is going to be most useful. 00:27:49.980 --> 00:27:53.370 align:middle line:84% And so that's, I think, a summary of some of the impacts. 00:27:53.370 --> 00:27:57.260 align:middle line:84% So one thing in particular I'll note 00:27:57.260 --> 00:28:01.790 align:middle line:84% is there was a meta analysis by Meager, 00:28:01.790 --> 00:28:03.650 align:middle line:84% who's also a student here, which came out 00:28:03.650 --> 00:28:05.150 align:middle line:84% a few years later in AEJ Applied. 00:28:05.150 --> 00:28:06.650 align:middle line:84% And she basically said, look, there 00:28:06.650 --> 00:28:07.817 align:middle line:90% are seven different studies. 00:28:07.817 --> 00:28:10.670 align:middle line:84% How can we say something about how similar or different 00:28:10.670 --> 00:28:11.690 align:middle line:90% they are? 00:28:11.690 --> 00:28:14.240 align:middle line:84% And the thing she wants to grapple with 00:28:14.240 --> 00:28:17.090 align:middle line:84% is that basically, there are differences 00:28:17.090 --> 00:28:18.320 align:middle line:90% you observe across sites. 00:28:18.320 --> 00:28:20.450 align:middle line:90% Are those due to a-- 00:28:20.450 --> 00:28:22.310 align:middle line:90% I see some differences. 00:28:22.310 --> 00:28:25.160 align:middle line:84% Those are due to a combination of just sampling noise. 00:28:25.160 --> 00:28:27.625 align:middle line:84% If I have a sample, that's regular standard errors, 00:28:27.625 --> 00:28:29.000 align:middle line:84% or it could be that they're drawn 00:28:29.000 --> 00:28:30.208 align:middle line:90% from different distributions. 00:28:30.208 --> 00:28:32.570 align:middle line:84% Maybe the thing really works great in Indonesia 00:28:32.570 --> 00:28:34.820 align:middle line:84% and it works really terribly in India, or vice versa, 00:28:34.820 --> 00:28:36.950 align:middle line:90% or whatever. 00:28:36.950 --> 00:28:40.380 align:middle line:84% Let me actually-- I think in the interest of time, 00:28:40.380 --> 00:28:45.010 align:middle line:84% I think I'm not going to go through this 00:28:45.010 --> 00:28:47.900 align:middle line:90% in a lot of detail. 00:28:47.900 --> 00:28:49.330 align:middle line:84% But let me just basically say, she 00:28:49.330 --> 00:28:52.270 align:middle line:84% has a hierarchical Bayesian framework 00:28:52.270 --> 00:28:53.930 align:middle line:90% for thinking about this. 00:28:53.930 --> 00:28:54.430 align:middle line:90% Yeah. 00:28:54.430 --> 00:28:55.720 align:middle line:84% Let me actually not go through this in detail. 00:28:55.720 --> 00:28:56.710 align:middle line:84% But let me just note that if you're 00:28:56.710 --> 00:28:58.210 align:middle line:84% interested in these issues of how 00:28:58.210 --> 00:29:01.030 align:middle line:84% do you combine across places, you 00:29:01.030 --> 00:29:03.280 align:middle line:84% should go-- you should have a look at her paper, which 00:29:03.280 --> 00:29:07.450 align:middle line:84% helps you think through how do we think about this stuff. 00:29:07.450 --> 00:29:15.340 align:middle line:84% So let me not go through that in a lot of detail. 00:29:15.340 --> 00:29:18.340 align:middle line:84% But basically, they show not much-- 00:29:18.340 --> 00:29:21.440 align:middle line:90% 00:29:21.440 --> 00:29:32.600 align:middle line:84% not massive average impacts, but there are, I think, 00:29:32.600 --> 00:29:35.470 align:middle line:84% bigger impacts in general across these various studies 00:29:35.470 --> 00:29:37.220 align:middle line:84% for people who had prior businesses, which 00:29:37.220 --> 00:29:38.150 align:middle line:90% is what's shown here. 00:29:38.150 --> 00:29:40.490 align:middle line:84% This blue graph is the ones where people who don't have 00:29:40.490 --> 00:29:43.430 align:middle line:84% prior businesses, and in general, 00:29:43.430 --> 00:29:44.895 align:middle line:84% the distribution of outcomes look 00:29:44.895 --> 00:29:47.270 align:middle line:84% shifted better for people who had prior businesses, which 00:29:47.270 --> 00:29:49.040 align:middle line:84% is consistent with what we thought before, that basically 00:29:49.040 --> 00:29:52.070 align:middle line:84% this is providing more impact to people who have businesses. 00:29:52.070 --> 00:29:52.790 align:middle line:90% Yeah. 00:29:52.790 --> 00:29:54.740 align:middle line:84% AUDIENCE: Just a quick question for framing. 00:29:54.740 --> 00:29:59.558 align:middle line:84% What is the size of a microfinance loan in general. 00:29:59.558 --> 00:30:01.350 align:middle line:84% BEN OLKEN: I don't know the actual numbers, 00:30:01.350 --> 00:30:04.880 align:middle line:84% but examples I would give-- anywhere from $50, $100, $500, 00:30:04.880 --> 00:30:06.710 align:middle line:90% to up to $1,000, I would say. 00:30:06.710 --> 00:30:08.090 align:middle line:90% In that ballpark. 00:30:08.090 --> 00:30:10.520 align:middle line:84% They may go higher than that, but those are typical. 00:30:10.520 --> 00:30:12.470 align:middle line:84% I think the example you guys read in the Hussam paper was, 00:30:12.470 --> 00:30:12.980 align:middle line:90% like, $100. 00:30:12.980 --> 00:30:14.240 align:middle line:90% Those were grants, not loans. 00:30:14.240 --> 00:30:16.500 align:middle line:90% Those were, like, $100, right? 00:30:16.500 --> 00:30:17.990 align:middle line:90% So I think that-- 00:30:17.990 --> 00:30:21.860 align:middle line:84% and those are not atypical amounts. 00:30:21.860 --> 00:30:22.360 align:middle line:90% Yeah. 00:30:22.360 --> 00:30:25.680 align:middle line:90% 00:30:25.680 --> 00:30:26.180 align:middle line:90% OK. 00:30:26.180 --> 00:30:30.380 align:middle line:84% So in general, I would say what all those impact evaluations 00:30:30.380 --> 00:30:32.810 align:middle line:84% showed was that for the average borrower in the sample, 00:30:32.810 --> 00:30:34.190 align:middle line:90% it didn't do very much. 00:30:34.190 --> 00:30:36.600 align:middle line:84% But there were two important caveats. 00:30:36.600 --> 00:30:39.488 align:middle line:84% Number one, those were-- the sample-- 00:30:39.488 --> 00:30:41.030 align:middle line:84% there may be a sample selection here. 00:30:41.030 --> 00:30:42.140 align:middle line:84% Like, if you think about the designs 00:30:42.140 --> 00:30:44.180 align:middle line:84% I talked about, most of them were selecting 00:30:44.180 --> 00:30:47.630 align:middle line:84% on more marginal cases where the bank was indifferent, 00:30:47.630 --> 00:30:49.580 align:middle line:90% either people or places. 00:30:49.580 --> 00:30:52.200 align:middle line:84% And number two, there was this substantial heterogeneity. 00:30:52.200 --> 00:30:54.870 align:middle line:84% So that leads us to the paper that you guys read, 00:30:54.870 --> 00:30:56.730 align:middle line:84% which is if heterogeneity is important, 00:30:56.730 --> 00:30:58.063 align:middle line:90% what else is predicting returns? 00:30:58.063 --> 00:31:02.540 align:middle line:84% And people know what's going on here. 00:31:02.540 --> 00:31:06.130 align:middle line:84% So that leads us to this paper by Hussam, Rigol, and Roth 00:31:06.130 --> 00:31:07.657 align:middle line:90% that you guys read for today. 00:31:07.657 --> 00:31:10.240 align:middle line:84% So just to make sure everyone-- just remind everyone what this 00:31:10.240 --> 00:31:12.918 align:middle line:90% paper was, this was-- 00:31:12.918 --> 00:31:15.460 align:middle line:84% these-- actually, these were all graduate students, actually, 00:31:15.460 --> 00:31:17.492 align:middle line:84% here, when they were working on this project, 00:31:17.492 --> 00:31:19.450 align:middle line:84% as some of you-- at least some of you picked up 00:31:19.450 --> 00:31:23.020 align:middle line:90% in your reading. 00:31:23.020 --> 00:31:28.290 align:middle line:84% So what [INAUDIBLE] [? Natalia ?] and Ben did is 00:31:28.290 --> 00:31:28.790 align:middle line:90% they-- 00:31:28.790 --> 00:31:31.155 align:middle line:90% 00:31:31.155 --> 00:31:32.780 align:middle line:84% and I should say, by the way, one thing 00:31:32.780 --> 00:31:37.705 align:middle line:84% I think was kind of interesting is just as-- 00:31:37.705 --> 00:31:40.050 align:middle line:84% how this paper came about, in some sense, 00:31:40.050 --> 00:31:43.430 align:middle line:84% this paper, I think, builds on the targeting 00:31:43.430 --> 00:31:46.940 align:middle line:84% paper that I talked about three or four lectures ago 00:31:46.940 --> 00:31:49.470 align:middle line:84% and that we said, look-- and they said, look, 00:31:49.470 --> 00:31:52.520 align:middle line:84% there's local-- there's clearly local information for poverty 00:31:52.520 --> 00:31:53.450 align:middle line:90% targeting. 00:31:53.450 --> 00:31:58.103 align:middle line:84% That's about-- but that's about the level of people's income. 00:31:58.103 --> 00:31:59.520 align:middle line:84% But maybe if people know something 00:31:59.520 --> 00:32:00.870 align:middle line:84% about the level of people's income, 00:32:00.870 --> 00:32:03.120 align:middle line:84% they also know something about their marginal returns. 00:32:03.120 --> 00:32:04.933 align:middle line:84% And that's-- it's a harder problem. 00:32:04.933 --> 00:32:06.600 align:middle line:84% It's much easier to know about the level 00:32:06.600 --> 00:32:09.180 align:middle line:84% than about the returns because the level I can just look at. 00:32:09.180 --> 00:32:11.100 align:middle line:84% The returns is like a counterfactual. 00:32:11.100 --> 00:32:14.673 align:middle line:84% But in some sense, I think part of their idea is saying, 00:32:14.673 --> 00:32:16.590 align:middle line:84% well, look, we're interested in micro finance, 00:32:16.590 --> 00:32:18.960 align:middle line:84% but can we take these ideas from this other setting 00:32:18.960 --> 00:32:19.960 align:middle line:90% and adapt them here? 00:32:19.960 --> 00:32:22.350 align:middle line:84% And so it's also, I think, an interesting example 00:32:22.350 --> 00:32:27.152 align:middle line:84% of how there can be intellectual arbitrage from one 00:32:27.152 --> 00:32:28.110 align:middle line:90% field to another field. 00:32:28.110 --> 00:32:30.750 align:middle line:84% I think-- I do, having been here through this process, 00:32:30.750 --> 00:32:35.500 align:middle line:84% I think that was partially how that came about, 00:32:35.500 --> 00:32:37.050 align:middle line:84% which I think is really interesting, 00:32:37.050 --> 00:32:41.070 align:middle line:84% and I think can maybe be relevant guidance for all 00:32:41.070 --> 00:32:41.890 align:middle line:90% of you. 00:32:41.890 --> 00:32:42.390 align:middle line:90% So, OK. 00:32:42.390 --> 00:32:43.200 align:middle line:90% So what do they do? 00:32:43.200 --> 00:32:49.400 align:middle line:84% So they asked entrepreneurs in peri-urban Maharashtra 00:32:49.400 --> 00:32:52.820 align:middle line:84% to rank their peers, groups of four to six people, 00:32:52.820 --> 00:32:55.130 align:middle line:84% on metrics of business profitability and growth 00:32:55.130 --> 00:32:57.890 align:middle line:90% potential. 00:32:57.890 --> 00:32:59.270 align:middle line:84% And then they randomly distribute 00:32:59.270 --> 00:33:01.790 align:middle line:84% cash grants of $100 to a third of these entrepreneurs 00:33:01.790 --> 00:33:05.510 align:middle line:84% to measure their actual productivity. 00:33:05.510 --> 00:33:08.660 align:middle line:84% So why is it important to randomly assign this here? 00:33:08.660 --> 00:33:11.320 align:middle line:90% 00:33:11.320 --> 00:33:14.148 align:middle line:84% So why do they need this step of randomly assigning 00:33:14.148 --> 00:33:14.815 align:middle line:90% the cash grants? 00:33:14.815 --> 00:33:20.118 align:middle line:90% 00:33:20.118 --> 00:33:21.660 align:middle line:84% Like, why aren't they just doing this 00:33:21.660 --> 00:33:24.973 align:middle line:84% and looking at whether you can predict business profitability? 00:33:24.973 --> 00:33:30.900 align:middle line:90% 00:33:30.900 --> 00:33:31.700 align:middle line:90% Yeah. 00:33:31.700 --> 00:33:33.320 align:middle line:84% AUDIENCE: You're trying to predict returns, not levels? 00:33:33.320 --> 00:33:33.890 align:middle line:90% BEN OLKEN: Yeah, exactly. 00:33:33.890 --> 00:33:35.807 align:middle line:84% They're trying to predict returns, not levels. 00:33:35.807 --> 00:33:38.570 align:middle line:84% They're trying to predict what is the return to giving you 00:33:38.570 --> 00:33:41.360 align:middle line:84% an extra $100, not what is your average, right? 00:33:41.360 --> 00:33:43.730 align:middle line:84% So just to be clear, they're trying-- 00:33:43.730 --> 00:33:46.840 align:middle line:84% and this is an example where thinking precisely what 00:33:46.840 --> 00:33:47.840 align:middle line:90% the theory is important. 00:33:47.840 --> 00:33:52.970 align:middle line:84% Their goal is to predict f prime of k, not f of k. 00:33:52.970 --> 00:33:56.810 align:middle line:84% So if they were interested in just predicting the level, 00:33:56.810 --> 00:33:57.680 align:middle line:90% that may be f of k. 00:33:57.680 --> 00:34:00.240 align:middle line:84% But they want to say, well, what's the what's the return? 00:34:00.240 --> 00:34:02.930 align:middle line:84% So to do that, they need to predict this thing at baseline, 00:34:02.930 --> 00:34:04.820 align:middle line:84% get this information at baseline, 00:34:04.820 --> 00:34:08.670 align:middle line:84% randomly shock some people with $100, and then say, 00:34:08.670 --> 00:34:12.679 align:middle line:84% what is the treatment effect of $100 heterogeneous-- 00:34:12.679 --> 00:34:14.540 align:middle line:84% and how does that vary heterogeneously 00:34:14.540 --> 00:34:17.210 align:middle line:84% based on this information at baseline? 00:34:17.210 --> 00:34:17.840 align:middle line:90% Question. 00:34:17.840 --> 00:34:19.894 align:middle line:84% Do you think it's better to do a loan or a grant? 00:34:19.894 --> 00:34:25.760 align:middle line:90% 00:34:25.760 --> 00:34:26.570 align:middle line:90% They do a grant. 00:34:26.570 --> 00:34:33.576 align:middle line:90% 00:34:33.576 --> 00:34:35.409 align:middle line:84% Why do you think they may have done a grant? 00:34:35.409 --> 00:34:39.522 align:middle line:90% 00:34:39.522 --> 00:34:41.690 align:middle line:84% AUDIENCE: Because it's easier [INAUDIBLE].. 00:34:41.690 --> 00:34:42.560 align:middle line:90% BEN OLKEN: Yes. 00:34:42.560 --> 00:34:43.489 align:middle line:90% That's exactly why. 00:34:43.489 --> 00:34:45.770 align:middle line:90% So they wanted to do a loan. 00:34:45.770 --> 00:34:48.969 align:middle line:84% So this is not a case where it was ex ante 00:34:48.969 --> 00:34:49.969 align:middle line:90% the optimal thing to do. 00:34:49.969 --> 00:34:53.120 align:middle line:84% I can also, having, I guess, been 00:34:53.120 --> 00:34:55.760 align:middle line:84% there through this process, their first idea was, 00:34:55.760 --> 00:34:58.190 align:middle line:84% we should do loans and see what's the return to a loan. 00:34:58.190 --> 00:35:00.860 align:middle line:84% And they spent a long time negotiating 00:35:00.860 --> 00:35:02.360 align:middle line:84% with multiple microfinance lenders 00:35:02.360 --> 00:35:05.240 align:middle line:84% over a period of a long time, and in the end of the day, 00:35:05.240 --> 00:35:07.213 align:middle line:84% couldn't find a microfinance lender who 00:35:07.213 --> 00:35:09.380 align:middle line:84% was willing to randomly assign loans in this context 00:35:09.380 --> 00:35:10.260 align:middle line:90% they were wanting to do it. 00:35:10.260 --> 00:35:12.410 align:middle line:84% But they said, oh, actually, maybe the second best thing is 00:35:12.410 --> 00:35:13.040 align:middle line:90% we can do grants. 00:35:13.040 --> 00:35:13.880 align:middle line:84% And that's actually still pretty good 00:35:13.880 --> 00:35:16.020 align:middle line:84% because we can still get the thing we want, 00:35:16.020 --> 00:35:19.670 align:middle line:84% which is f prime of K. So we can get f prime of k 00:35:19.670 --> 00:35:23.587 align:middle line:84% directly rather than f prime of k minus k. 00:35:23.587 --> 00:35:25.670 align:middle line:84% They were still kind of relevant, both interesting 00:35:25.670 --> 00:35:26.480 align:middle line:90% parameters. 00:35:26.480 --> 00:35:28.012 align:middle line:84% And this one we can just-- if we can 00:35:28.012 --> 00:35:29.720 align:middle line:84% raise enough money to give out $100, then 00:35:29.720 --> 00:35:30.990 align:middle line:90% we can just do this ourselves. 00:35:30.990 --> 00:35:33.590 align:middle line:84% And so I mentioned that also because I 00:35:33.590 --> 00:35:35.780 align:middle line:90% think that was also a nice-- 00:35:35.780 --> 00:35:39.497 align:middle line:84% they had this idea, and they wanted to do it. 00:35:39.497 --> 00:35:41.330 align:middle line:84% And the first idea was they should do loans, 00:35:41.330 --> 00:35:42.852 align:middle line:90% and they tried. 00:35:42.852 --> 00:35:45.060 align:middle line:84% And they worked with a bunch of microfinance lenders. 00:35:45.060 --> 00:35:46.670 align:middle line:90% There was a whole process. 00:35:46.670 --> 00:35:49.250 align:middle line:84% But then they were adaptable enough 00:35:49.250 --> 00:35:51.230 align:middle line:84% to say, well, that thing isn't going 00:35:51.230 --> 00:35:52.820 align:middle line:90% to actually work in reality. 00:35:52.820 --> 00:35:55.563 align:middle line:84% Can we find a way to answer the same question that 00:35:55.563 --> 00:35:57.980 align:middle line:84% doesn't rely on us convincing some partner to do something 00:35:57.980 --> 00:35:58.610 align:middle line:90% they wouldn't want to do? 00:35:58.610 --> 00:35:59.660 align:middle line:84% And that's how they ended up on grant. 00:35:59.660 --> 00:36:01.700 align:middle line:84% So actually, the grants are totally fine 00:36:01.700 --> 00:36:07.370 align:middle line:84% and was a good example of adaptability getting there. 00:36:07.370 --> 00:36:08.408 align:middle line:90% Yeah, [INAUDIBLE]? 00:36:08.408 --> 00:36:10.760 align:middle line:84% AUDIENCE: [INAUDIBLE] for any reason we would expect 00:36:10.760 --> 00:36:12.343 align:middle line:84% the responses to those [INAUDIBLE]---- 00:36:12.343 --> 00:36:14.270 align:middle line:90% BEN OLKEN: What do you think? 00:36:14.270 --> 00:36:16.618 align:middle line:90% AUDIENCE: Well, I ask-- 00:36:16.618 --> 00:36:17.660 align:middle line:90% why I asked the question. 00:36:17.660 --> 00:36:20.710 align:middle line:90% [LAUGHTER] 00:36:20.710 --> 00:36:21.210 align:middle line:90% 00:36:21.210 --> 00:36:21.950 align:middle line:84% BEN OLKEN: What does someone else think? 00:36:21.950 --> 00:36:23.920 align:middle line:84% Do we think loans and grants may be different? 00:36:23.920 --> 00:36:25.670 align:middle line:84% AUDIENCE: I would think maybe the loan has 00:36:25.670 --> 00:36:27.350 align:middle line:90% more of an impact on capital? 00:36:27.350 --> 00:36:27.995 align:middle line:90% BEN OLKEN: Huh? 00:36:27.995 --> 00:36:29.370 align:middle line:84% AUDIENCE: Maybe the loan has more 00:36:29.370 --> 00:36:32.952 align:middle line:84% of an impact on the capital [INAUDIBLE] with it? 00:36:32.952 --> 00:36:33.910 align:middle line:90% BEN OLKEN: Because why? 00:36:33.910 --> 00:36:36.180 align:middle line:84% AUDIENCE: You have an incentive to return the money? 00:36:36.180 --> 00:36:36.847 align:middle line:90% BEN OLKEN: Yeah. 00:36:36.847 --> 00:36:38.150 align:middle line:90% So I think that-- 00:36:38.150 --> 00:36:39.580 align:middle line:90% because you have to pay it back. 00:36:39.580 --> 00:36:40.080 align:middle line:90% Yeah. 00:36:40.080 --> 00:36:42.443 align:middle line:90% So you might think-- yeah. 00:36:42.443 --> 00:36:44.610 align:middle line:84% So I think you-- sorry, what were you trying to say? 00:36:44.610 --> 00:36:46.150 align:middle line:90% AUDIENCE: That was [INAUDIBLE]. 00:36:46.150 --> 00:36:46.410 align:middle line:90% BEN OLKEN: Yeah. 00:36:46.410 --> 00:36:48.160 align:middle line:84% So you might think they're going to be different. 00:36:48.160 --> 00:36:48.660 align:middle line:90% Let's see. 00:36:48.660 --> 00:36:55.240 align:middle line:84% I think that-- so going back to the theory we talked about last 00:36:55.240 --> 00:36:56.350 align:middle line:90% time, would there be-- 00:36:56.350 --> 00:36:57.850 align:middle line:84% what would the differences there be? 00:36:57.850 --> 00:36:59.800 align:middle line:84% So if they're-- yeah, [? Becca? ?] 00:36:59.800 --> 00:37:02.230 align:middle line:84% AUDIENCE: I guess also, the stream of returns that you get 00:37:02.230 --> 00:37:04.535 align:middle line:84% and the timing of those returns [INAUDIBLE]?? 00:37:04.535 --> 00:37:05.160 align:middle line:90% BEN OLKEN: Yes. 00:37:05.160 --> 00:37:07.610 align:middle line:84% So the timing of the returns are going to be important, 00:37:07.610 --> 00:37:09.902 align:middle line:84% and we're going to talk about that in one of the papers 00:37:09.902 --> 00:37:12.960 align:middle line:84% that hopefully I'll get to later today, which is the-- 00:37:12.960 --> 00:37:16.208 align:middle line:84% you have to make sure that if it's a loan 00:37:16.208 --> 00:37:17.750 align:middle line:84% and you have to pay it back, you have 00:37:17.750 --> 00:37:20.420 align:middle line:84% to make sure it returns the money in time 00:37:20.420 --> 00:37:21.720 align:middle line:90% for you to pay back the loan. 00:37:21.720 --> 00:37:23.595 align:middle line:84% And so that actually-- that question 00:37:23.595 --> 00:37:25.220 align:middle line:84% of the timing of repayment can actually 00:37:25.220 --> 00:37:27.428 align:middle line:84% distort your investment choices, so that's important. 00:37:27.428 --> 00:37:28.730 align:middle line:90% Yeah. 00:37:28.730 --> 00:37:29.623 align:middle line:90% What else? 00:37:29.623 --> 00:37:31.040 align:middle line:84% AUDIENCE: I don't see exactly how. 00:37:31.040 --> 00:37:32.870 align:middle line:84% I feel like it would change your risk, 00:37:32.870 --> 00:37:35.723 align:middle line:84% your choice of the riskiness of what you do with it? 00:37:35.723 --> 00:37:37.640 align:middle line:84% BEN OLKEN: Why don't we then hold this thought 00:37:37.640 --> 00:37:40.400 align:middle line:84% till we get to the paper about grace periods, which 00:37:40.400 --> 00:37:44.160 align:middle line:84% is-- actually test this idea experimentally. 00:37:44.160 --> 00:37:44.660 align:middle line:90% Yeah. 00:37:44.660 --> 00:37:46.577 align:middle line:84% AUDIENCE: But also, not everyone would take up 00:37:46.577 --> 00:37:49.060 align:middle line:84% the loan, whereas everyone would take a cash grant, right? 00:37:49.060 --> 00:37:51.370 align:middle line:84% So if it was selection into the loan, 00:37:51.370 --> 00:37:54.650 align:middle line:84% then you might see higher effects. 00:37:54.650 --> 00:37:55.370 align:middle line:90% BEN OLKEN: Yep. 00:37:55.370 --> 00:37:55.730 align:middle line:90% Yeah. 00:37:55.730 --> 00:37:56.270 align:middle line:90% So that's right. 00:37:56.270 --> 00:37:57.650 align:middle line:84% So you might find-- so everyone's 00:37:57.650 --> 00:38:00.140 align:middle line:84% going to want the $100, and you're right. 00:38:00.140 --> 00:38:03.690 align:middle line:84% Basically, that may-- and that might change the result. 00:38:03.690 --> 00:38:06.830 align:middle line:84% So it might be that actually, what we're picking up here is 00:38:06.830 --> 00:38:08.632 align:middle line:84% who actually has a business and is-- 00:38:08.632 --> 00:38:11.090 align:middle line:84% has a profitable opportunity, whereas everyone would take-- 00:38:11.090 --> 00:38:11.630 align:middle line:90% whereas-- 00:38:11.630 --> 00:38:13.340 align:middle line:84% so they may not have heterogeneity 00:38:13.340 --> 00:38:15.650 align:middle line:84% in returns of a loan because that selection of the loan 00:38:15.650 --> 00:38:18.180 align:middle line:84% might actually get you some the selection of picking up. 00:38:18.180 --> 00:38:19.490 align:middle line:90% Yeah. 00:38:19.490 --> 00:38:20.850 align:middle line:84% Sorry, [? Aaron, ?] you had something you wanted to say? 00:38:20.850 --> 00:38:21.120 align:middle line:90% AUDIENCE: No. 00:38:21.120 --> 00:38:21.710 align:middle line:84% BEN OLKEN: [? Becca, ?] you still 00:38:21.710 --> 00:38:22.580 align:middle line:90% had something you want to say? 00:38:22.580 --> 00:38:23.800 align:middle line:90% AUDIENCE: I think it was-- 00:38:23.800 --> 00:38:26.720 align:middle line:90% BEN OLKEN: That point. 00:38:26.720 --> 00:38:30.140 align:middle line:84% I think it also depends on the degree to which the loan has 00:38:30.140 --> 00:38:31.980 align:middle line:90% limited liability or not. 00:38:31.980 --> 00:38:35.270 align:middle line:84% So if you think about the model from before, in the model 00:38:35.270 --> 00:38:45.185 align:middle line:90% from before, if you're fully-- 00:38:45.185 --> 00:38:46.570 align:middle line:90% what you going to say, sorry? 00:38:46.570 --> 00:38:47.588 align:middle line:90% AUDIENCE: Oh, sorry. 00:38:47.588 --> 00:38:48.630 align:middle line:90% BEN OLKEN: No, that's OK. 00:38:48.630 --> 00:38:51.240 align:middle line:84% AUDIENCE: Is it right to say that if you're able to properly 00:38:51.240 --> 00:38:58.520 align:middle line:84% estimate the hazard rate or the probability of somebody not 00:38:58.520 --> 00:39:00.740 align:middle line:84% paying back and their beliefs about not paying back, 00:39:00.740 --> 00:39:04.070 align:middle line:84% that you should be able to just give them a grant that should 00:39:04.070 --> 00:39:06.070 align:middle line:84% [? induce ?] the same-- because the difference-- 00:39:06.070 --> 00:39:06.940 align:middle line:90% BEN OLKEN: No, I think-- 00:39:06.940 --> 00:39:08.857 align:middle line:84% I actually think the point you made, actually, 00:39:08.857 --> 00:39:10.840 align:middle line:90% about the timing is relevant. 00:39:10.840 --> 00:39:12.670 align:middle line:84% AUDIENCE: Holding, holding that timing. 00:39:12.670 --> 00:39:15.170 align:middle line:84% BEN OLKEN: No, because they may do different things with it. 00:39:15.170 --> 00:39:16.120 align:middle line:90% That's the point. 00:39:16.120 --> 00:39:18.700 align:middle line:84% Imagine I basically-- for example, 00:39:18.700 --> 00:39:22.150 align:middle line:84% we know that people often borrow to go to school, right? 00:39:22.150 --> 00:39:25.150 align:middle line:84% So that could be a perfectly reasonable thing 00:39:25.150 --> 00:39:26.260 align:middle line:90% to do with a loan. 00:39:26.260 --> 00:39:30.500 align:middle line:84% But if you have a 10-year repayment horizon, 00:39:30.500 --> 00:39:32.590 align:middle line:84% but if you have a one-year repayment horizon, 00:39:32.590 --> 00:39:34.900 align:middle line:84% that would be a terrible thing to invest your invest 00:39:34.900 --> 00:39:36.670 align:middle line:84% your thing on because you're not going to have any returns 00:39:36.670 --> 00:39:37.540 align:middle line:90% you're still going be in school. 00:39:37.540 --> 00:39:38.165 align:middle line:90% AUDIENCE: Sure. 00:39:38.165 --> 00:39:39.335 align:middle line:90% [INAUDIBLE] 00:39:39.335 --> 00:39:39.960 align:middle line:90% BEN OLKEN: Yes. 00:39:39.960 --> 00:39:41.520 align:middle line:84% AUDIENCE: So assume that the payment horizon 00:39:41.520 --> 00:39:43.050 align:middle line:84% is 10 years and all of the things 00:39:43.050 --> 00:39:45.342 align:middle line:84% that people are investing in-- like, they would presume 00:39:45.342 --> 00:39:48.090 align:middle line:84% that they would probably make back the money before they 00:39:48.090 --> 00:39:49.980 align:middle line:90% would need to repay. 00:39:49.980 --> 00:39:52.110 align:middle line:84% If, on the limited liability point, 00:39:52.110 --> 00:39:55.140 align:middle line:84% couldn't you equilibrate to give somebody a grant 00:39:55.140 --> 00:39:59.265 align:middle line:84% that would be valued to them the same as-- 00:39:59.265 --> 00:40:03.250 align:middle line:90% 00:40:03.250 --> 00:40:05.770 align:middle line:84% based on their personal belief about the project 00:40:05.770 --> 00:40:06.760 align:middle line:90% working out and their-- 00:40:06.760 --> 00:40:08.860 align:middle line:84% BEN OLKEN: I'm not sure I see how. 00:40:08.860 --> 00:40:12.688 align:middle line:84% Maybe I'm missing something, but I don't immediately see that. 00:40:12.688 --> 00:40:14.230 align:middle line:84% How about I talk about it afterwards? 00:40:14.230 --> 00:40:16.480 align:middle line:90% But I don't quite see it. 00:40:16.480 --> 00:40:16.980 align:middle line:90% But, OK. 00:40:16.980 --> 00:40:18.730 align:middle line:84% So I think the answer is they're not quite 00:40:18.730 --> 00:40:20.067 align:middle line:90% the same for some reasons. 00:40:20.067 --> 00:40:21.900 align:middle line:84% People may do different things with the loan 00:40:21.900 --> 00:40:22.800 align:middle line:84% if they think they have to pay it back 00:40:22.800 --> 00:40:24.330 align:middle line:90% for these timing reasons. 00:40:24.330 --> 00:40:27.910 align:middle line:84% They may be more-- if there's risk-aversion reasons, imagine 00:40:27.910 --> 00:40:30.990 align:middle line:84% that if you have concave utility, 00:40:30.990 --> 00:40:33.200 align:middle line:84% you're worried about the bad state, so you may-- 00:40:33.200 --> 00:40:35.848 align:middle line:84% if you're risk neutral, maybe it doesn't make a difference. 00:40:35.848 --> 00:40:38.140 align:middle line:84% But if you're risk averse, then I might think of, well, 00:40:38.140 --> 00:40:39.330 align:middle line:84% if I have to pay this thing back in the bad state, 00:40:39.330 --> 00:40:40.203 align:middle line:90% I'm going to be-- 00:40:40.203 --> 00:40:42.120 align:middle line:84% take something that's more conservative, where 00:40:42.120 --> 00:40:43.110 align:middle line:84% if I don't have to pay it back, then 00:40:43.110 --> 00:40:44.380 align:middle line:84% I'm not so worried about the bad state. 00:40:44.380 --> 00:40:45.120 align:middle line:90% So there are reasons. 00:40:45.120 --> 00:40:45.540 align:middle line:90% They are different. 00:40:45.540 --> 00:40:46.710 align:middle line:84% And the selection point, I think, Hazel, 00:40:46.710 --> 00:40:48.252 align:middle line:84% you also mentioned was totally right. 00:40:48.252 --> 00:40:52.050 align:middle line:84% There are reasons these things can be different. 00:40:52.050 --> 00:40:53.550 align:middle line:84% And that's probably why they started 00:40:53.550 --> 00:40:55.830 align:middle line:84% with trying to do a loan if the whole point was 00:40:55.830 --> 00:40:56.760 align:middle line:84% to just think about microfinance, 00:40:56.760 --> 00:40:57.810 align:middle line:90% that was the first best. 00:40:57.810 --> 00:41:00.225 align:middle line:84% But I still think there's a lot they can learn from this, 00:41:00.225 --> 00:41:01.350 align:middle line:90% which is why they did this. 00:41:01.350 --> 00:41:12.870 align:middle line:90% 00:41:12.870 --> 00:41:16.380 align:middle line:84% So then the key regression that they're going to run 00:41:16.380 --> 00:41:17.970 align:middle line:90% is essentially this one. 00:41:17.970 --> 00:41:20.700 align:middle line:84% So cash drop gets you marginal returns. 00:41:20.700 --> 00:41:23.640 align:middle line:84% Predicting is just how your characteristics 00:41:23.640 --> 00:41:24.690 align:middle line:90% predict your outcomes. 00:41:24.690 --> 00:41:27.630 align:middle line:84% And then the key thing is this interaction of predicted times 00:41:27.630 --> 00:41:29.740 align:middle line:90% cash drop, right? 00:41:29.740 --> 00:41:32.710 align:middle line:90% 00:41:32.710 --> 00:41:39.070 align:middle line:84% They also want to say, is there additional information, 00:41:39.070 --> 00:41:44.840 align:middle line:84% or is this better than the machine learning approach? 00:41:44.840 --> 00:41:46.690 align:middle line:84% So they also say, let's predict your returns 00:41:46.690 --> 00:41:48.357 align:middle line:84% based on some machine learning approach. 00:41:48.357 --> 00:41:51.230 align:middle line:84% They take an auxiliary sample, some-- 00:41:51.230 --> 00:41:51.730 align:middle line:90% whatever. 00:41:51.730 --> 00:41:51.940 align:middle line:90% I don't know. 00:41:51.940 --> 00:41:54.040 align:middle line:84% Some training sample, like 20% or whatever. 00:41:54.040 --> 00:41:56.950 align:middle line:84% They predict your returns, your heterogeneous returns, 00:41:56.950 --> 00:41:58.600 align:middle line:90% based on your x characteristics. 00:41:58.600 --> 00:42:01.300 align:middle line:84% They then can take your machine learning prediction and horse 00:42:01.300 --> 00:42:05.120 align:middle line:84% race that against the cash drop prediction. 00:42:05.120 --> 00:42:10.240 align:middle line:84% So there's two ways of running that regression. 00:42:10.240 --> 00:42:18.800 align:middle line:84% I think the way I would run that regression is to basically-- 00:42:18.800 --> 00:42:20.250 align:middle line:90% sorry this got cut off. 00:42:20.250 --> 00:42:24.480 align:middle line:84% I would put them in the same sample, in the same regression, 00:42:24.480 --> 00:42:30.560 align:middle line:90% and see whether or not-- 00:42:30.560 --> 00:42:33.290 align:middle line:84% that's the ML plus ML times catch drop times ML. 00:42:33.290 --> 00:42:35.730 align:middle line:84% So I would say, let's control for this and say, 00:42:35.730 --> 00:42:37.960 align:middle line:84% does this predicted thing have additional-- 00:42:37.960 --> 00:42:41.030 align:middle line:84% this local predictions have additional predictive power 00:42:41.030 --> 00:42:42.200 align:middle line:90% over this? 00:42:42.200 --> 00:42:43.950 align:middle line:84% They, I think, if I remember it correctly, 00:42:43.950 --> 00:42:44.992 align:middle line:90% do it two different ways. 00:42:44.992 --> 00:42:47.820 align:middle line:90% 00:42:47.820 --> 00:42:48.900 align:middle line:90% They do a different way. 00:42:48.900 --> 00:42:53.790 align:middle line:84% They predict the heterogeneity, either just 00:42:53.790 --> 00:42:56.640 align:middle line:84% with the x characteristics or with the x characteristics 00:42:56.640 --> 00:43:00.900 align:middle line:84% and the local information and see which 00:43:00.900 --> 00:43:02.290 align:middle line:90% does better in this regression. 00:43:02.290 --> 00:43:04.650 align:middle line:84% So they run this regression, where predicted is either 00:43:04.650 --> 00:43:07.470 align:middle line:84% predicted based on just the baseline characteristics 00:43:07.470 --> 00:43:10.260 align:middle line:84% or based on characteristics plus these additional x variables. 00:43:10.260 --> 00:43:13.255 align:middle line:90% 00:43:13.255 --> 00:43:14.130 align:middle line:90% I don't know if you-- 00:43:14.130 --> 00:43:16.280 align:middle line:84% I don't know if I have opinions of those two 00:43:16.280 --> 00:43:17.420 align:middle line:90% different approaches. 00:43:17.420 --> 00:43:17.922 align:middle line:90% Yeah, Aaron? 00:43:17.922 --> 00:43:19.880 align:middle line:84% AUDIENCE: There's one question I had with this, 00:43:19.880 --> 00:43:24.120 align:middle line:84% in the paper, is a typical microfinance 00:43:24.120 --> 00:43:26.580 align:middle line:90% institution in this setting-- 00:43:26.580 --> 00:43:29.400 align:middle line:84% are their agents also embedded in the community? 00:43:29.400 --> 00:43:32.880 align:middle line:84% I guess my thought was maybe observables 00:43:32.880 --> 00:43:34.920 align:middle line:84% and this [? ML ?] exercise is not necessarily 00:43:34.920 --> 00:43:37.470 align:middle line:84% the relevant benchmark, but rather, what is-- 00:43:37.470 --> 00:43:39.983 align:middle line:84% what are institutions doing in status quo? 00:43:39.983 --> 00:43:40.650 align:middle line:90% BEN OLKEN: Yeah. 00:43:40.650 --> 00:43:41.560 align:middle line:90% I think that's a great point. 00:43:41.560 --> 00:43:42.977 align:middle line:84% So I think it's certainly the case 00:43:42.977 --> 00:43:47.700 align:middle line:84% that microfinance-- banks in general have loan officers, 00:43:47.700 --> 00:43:51.272 align:middle line:84% and their job is to find out stuff about you. 00:43:51.272 --> 00:43:52.230 align:middle line:90% And they interview you. 00:43:52.230 --> 00:43:54.647 align:middle line:84% And they say, do you look like you have-- is this business 00:43:54.647 --> 00:43:56.640 align:middle line:90% plan real, or is it not real? 00:43:56.640 --> 00:43:58.297 align:middle line:84% And maybe that's better than their ML. 00:43:58.297 --> 00:43:59.130 align:middle line:90% So I agree with you. 00:43:59.130 --> 00:44:00.900 align:middle line:84% That would be an interesting thing to do, 00:44:00.900 --> 00:44:02.880 align:middle line:84% would be to get the banks to do that. 00:44:02.880 --> 00:44:04.297 align:middle line:84% You have to think about, how would 00:44:04.297 --> 00:44:05.888 align:middle line:84% you incentivize the banks to do that 00:44:05.888 --> 00:44:08.430 align:middle line:84% in an incentive-compatible way and actually really put effort 00:44:08.430 --> 00:44:08.640 align:middle line:90% into it. 00:44:08.640 --> 00:44:09.120 align:middle line:90% But I agree with you. 00:44:09.120 --> 00:44:11.245 align:middle line:84% That would be another interesting benchmark to run. 00:44:11.245 --> 00:44:15.100 align:middle line:90% 00:44:15.100 --> 00:44:16.210 align:middle line:90% Yeah, [? Becca? ?] 00:44:16.210 --> 00:44:18.730 align:middle line:84% AUDIENCE: So on [? Aaron's ?] point, I'm a little bit 00:44:18.730 --> 00:44:19.270 align:middle line:90% confused. 00:44:19.270 --> 00:44:22.510 align:middle line:84% When you say, can the community identify for the entrepreneurs, 00:44:22.510 --> 00:44:24.340 align:middle line:90% are you asking-- 00:44:24.340 --> 00:44:26.200 align:middle line:84% it seems almost like the comparison 00:44:26.200 --> 00:44:28.510 align:middle line:84% that you're doing with the ML is saying, can-- 00:44:28.510 --> 00:44:32.980 align:middle line:84% is the community good at marginally 00:44:32.980 --> 00:44:38.830 align:middle line:84% identifying information beyond some baseline set of things 00:44:38.830 --> 00:44:41.140 align:middle line:84% that you would think would would predict entrepreneurs? 00:44:41.140 --> 00:44:42.040 align:middle line:84% BEN OLKEN: That's what this approach. 00:44:42.040 --> 00:44:42.980 align:middle line:90% AUDIENCE: Yes. 00:44:42.980 --> 00:44:43.840 align:middle line:84% BEN OLKEN: That's what this equation says. 00:44:43.840 --> 00:44:46.540 align:middle line:84% AUDIENCE: That's not necessarily measuring whether the community 00:44:46.540 --> 00:44:48.630 align:middle line:90% is good at-- 00:44:48.630 --> 00:44:49.973 align:middle line:90% is identifying [INAUDIBLE]. 00:44:49.973 --> 00:44:50.640 align:middle line:90% BEN OLKEN: Yeah. 00:44:50.640 --> 00:44:51.390 align:middle line:90% So that's right. 00:44:51.390 --> 00:44:53.160 align:middle line:90% So this is actually-- 00:44:53.160 --> 00:44:56.620 align:middle line:84% so when I was writing these slides, I-- 00:44:56.620 --> 00:45:02.080 align:middle line:84% this is the challenge with having your advisor teach 00:45:02.080 --> 00:45:02.900 align:middle line:90% your paper later. 00:45:02.900 --> 00:45:04.070 align:middle line:84% So I emailed them, and I was like, 00:45:04.070 --> 00:45:05.300 align:middle line:90% why are you doing it this way? 00:45:05.300 --> 00:45:06.758 align:middle line:84% Why didn't you do it the other way? 00:45:06.758 --> 00:45:08.620 align:middle line:90% 00:45:08.620 --> 00:45:11.080 align:middle line:84% And the answer that I got back from them 00:45:11.080 --> 00:45:14.960 align:middle line:84% was actually on exactly at this point, that this equation, 00:45:14.960 --> 00:45:18.070 align:middle line:90% which I wrote down over here-- 00:45:18.070 --> 00:45:22.420 align:middle line:84% my approach is this equation basically says, 00:45:22.420 --> 00:45:25.100 align:middle line:84% do they have residual information? 00:45:25.100 --> 00:45:27.220 align:middle line:84% So the test-- if I do it this way, 00:45:27.220 --> 00:45:30.670 align:middle line:84% then when I run this regression, what I get from beta 3 is, 00:45:30.670 --> 00:45:32.950 align:middle line:84% is there residual information that the community 00:45:32.950 --> 00:45:35.117 align:middle line:84% has that the machine learning approach doesn't know? 00:45:35.117 --> 00:45:37.838 align:middle line:84% But it doesn't really tell you quantitatively how much it is. 00:45:37.838 --> 00:45:39.880 align:middle line:84% Their approach that they're going to do this way, 00:45:39.880 --> 00:45:41.797 align:middle line:84% I think, tells you actually-- it does actually 00:45:41.797 --> 00:45:44.000 align:middle line:84% answer that quantitative question and says, 00:45:44.000 --> 00:45:46.360 align:middle line:84% look how much better is the overall targeting 00:45:46.360 --> 00:45:48.280 align:middle line:84% with this kind of combined approach of using 00:45:48.280 --> 00:45:50.770 align:middle line:84% all this information than if we just use the ML alone. 00:45:50.770 --> 00:45:52.420 align:middle line:84% So they're answering slightly different questions. 00:45:52.420 --> 00:45:54.420 align:middle line:84% I think that's why they went with this approach. 00:45:54.420 --> 00:46:01.150 align:middle line:90% 00:46:01.150 --> 00:46:02.957 align:middle line:90% Does that answer your question? 00:46:02.957 --> 00:46:03.540 align:middle line:90% AUDIENCE: Yes. 00:46:03.540 --> 00:46:04.720 align:middle line:90% BEN OLKEN: OK. 00:46:04.720 --> 00:46:08.333 align:middle line:84% Other-- so it's a slightly different question, 00:46:08.333 --> 00:46:09.250 align:middle line:90% and that's the answer. 00:46:09.250 --> 00:46:12.470 align:middle line:84% And each regression-- they both have-- they're both related. 00:46:12.470 --> 00:46:14.980 align:middle line:84% And I think if you get one, you're likely to get the other, 00:46:14.980 --> 00:46:16.240 align:middle line:90% but not totally. 00:46:16.240 --> 00:46:18.460 align:middle line:84% But I think that's why they did it. 00:46:18.460 --> 00:46:19.840 align:middle line:90% [? Ahmed, ?] you had a question? 00:46:19.840 --> 00:46:20.382 align:middle line:90% AUDIENCE: No. 00:46:20.382 --> 00:46:21.780 align:middle line:90% BEN OLKEN: OK. 00:46:21.780 --> 00:46:23.550 align:middle line:90% All right. 00:46:23.550 --> 00:46:25.770 align:middle line:90% So what do they find? 00:46:25.770 --> 00:46:31.530 align:middle line:84% So the first thing is they ask whether or not 00:46:31.530 --> 00:46:37.055 align:middle line:84% people are able to predict things about-- 00:46:37.055 --> 00:46:38.430 align:middle line:84% whether they're predicting things 00:46:38.430 --> 00:46:40.222 align:middle line:84% based about other people's characteristics. 00:46:40.222 --> 00:46:45.030 align:middle line:84% So like, do they know their income, profit, assets? 00:46:45.030 --> 00:46:50.418 align:middle line:84% So digit span is a test of how many digit you can remember. 00:46:50.418 --> 00:46:51.960 align:middle line:84% If I give you a random set of digits, 00:46:51.960 --> 00:46:53.043 align:middle line:90% how many can you remember? 00:46:53.043 --> 00:46:57.180 align:middle line:84% And that seems to be a test of numeracy and things like that. 00:46:57.180 --> 00:46:59.580 align:middle line:84% So they seem to have a lot of information about people. 00:46:59.580 --> 00:47:01.620 align:middle line:90% That's what this says. 00:47:01.620 --> 00:47:04.350 align:middle line:84% I think in some sense, this is the key graph. 00:47:04.350 --> 00:47:08.400 align:middle line:84% It's a little hard to read, but this 00:47:08.400 --> 00:47:14.280 align:middle line:84% is the predicted marginal returns from the people 00:47:14.280 --> 00:47:16.140 align:middle line:90% in the community. 00:47:16.140 --> 00:47:19.740 align:middle line:84% The black line is the outcome for those who lose. 00:47:19.740 --> 00:47:24.103 align:middle line:84% So in general, it's the case that those who lose have more-- 00:47:24.103 --> 00:47:26.520 align:middle line:84% those who are predicted to have higher returns have higher 00:47:26.520 --> 00:47:29.490 align:middle line:90% profits than those who don't. 00:47:29.490 --> 00:47:31.950 align:middle line:84% So your predicted marginal return 00:47:31.950 --> 00:47:35.610 align:middle line:84% is correlated with f of k in addition to f prime of k. 00:47:35.610 --> 00:47:39.690 align:middle line:84% The gap between the red line and the black line 00:47:39.690 --> 00:47:43.860 align:middle line:84% is the heterogeneity-- is the marginal return of the grant. 00:47:43.860 --> 00:47:47.370 align:middle line:84% So everybody gets a return from the grant, but I mean-- 00:47:47.370 --> 00:47:48.900 align:middle line:90% whatever the error bar is here. 00:47:48.900 --> 00:47:50.580 align:middle line:84% But in general, it looks like everybody's getting a return 00:47:50.580 --> 00:47:51.330 align:middle line:90% from the grant. 00:47:51.330 --> 00:47:55.410 align:middle line:84% But the gap is bigger based on these high percentiles 00:47:55.410 --> 00:47:58.230 align:middle line:84% and the low percentiles, which says that people are predicting 00:47:58.230 --> 00:48:01.800 align:middle line:84% not just the level but also predicting the marginal return. 00:48:01.800 --> 00:48:08.070 align:middle line:90% 00:48:08.070 --> 00:48:11.470 align:middle line:84% And that is what we see over here, 00:48:11.470 --> 00:48:14.430 align:middle line:84% which is that basically, this is winning 00:48:14.430 --> 00:48:16.950 align:middle line:90% the grant times the rank. 00:48:16.950 --> 00:48:20.880 align:middle line:84% And they basically find that those predictable 00:48:20.880 --> 00:48:22.320 align:middle line:84% heterogeneity, that people who are 00:48:22.320 --> 00:48:24.660 align:middle line:84% the-- have higher percentile ranks basically 00:48:24.660 --> 00:48:27.400 align:middle line:90% are doing better. 00:48:27.400 --> 00:48:30.060 align:middle line:90% And they can look at it nicely. 00:48:30.060 --> 00:48:32.287 align:middle line:84% In some ways, it's easier to interpret it 00:48:32.287 --> 00:48:34.620 align:middle line:84% just that they break into three groups, kind of dummies, 00:48:34.620 --> 00:48:37.748 align:middle line:84% like top third, middle third, bottom third. 00:48:37.748 --> 00:48:40.290 align:middle line:84% And you can basically see that the highest returns are coming 00:48:40.290 --> 00:48:41.665 align:middle line:84% from the people who are predicted 00:48:41.665 --> 00:48:47.540 align:middle line:84% to be in the top third by their peers. 00:48:47.540 --> 00:48:51.230 align:middle line:90% And what do they do? 00:48:51.230 --> 00:48:52.710 align:middle line:90% They buy more assets. 00:48:52.710 --> 00:48:55.520 align:middle line:84% They also work they work harder in response. 00:48:55.520 --> 00:48:56.780 align:middle line:90% And they hire more labor. 00:48:56.780 --> 00:49:00.153 align:middle line:84% So they both-- they increase both their asset-- 00:49:00.153 --> 00:49:01.820 align:middle line:84% they both increase their k more and they 00:49:01.820 --> 00:49:06.683 align:middle line:84% increase their l and other people's l in their businesses. 00:49:06.683 --> 00:49:08.600 align:middle line:84% And this is the thing I was just pointing out, 00:49:08.600 --> 00:49:13.430 align:middle line:84% which is this is the approach where I do-- where they 00:49:13.430 --> 00:49:16.280 align:middle line:84% take your prediction based on their observable 00:49:16.280 --> 00:49:18.580 align:middle line:84% characteristics and say, what's the heterogeneity 00:49:18.580 --> 00:49:21.050 align:middle line:84% and the marginal returns for the top and the bottom? 00:49:21.050 --> 00:49:23.780 align:middle line:84% And these is the ones that also include the rank. 00:49:23.780 --> 00:49:25.400 align:middle line:84% And the idea is that basically, these 00:49:25.400 --> 00:49:28.070 align:middle line:84% have some additional predictive power compared 00:49:28.070 --> 00:49:29.160 align:middle line:90% to the other ones. 00:49:29.160 --> 00:49:32.300 align:middle line:84% And you can-- where this is a lot better in predicting income 00:49:32.300 --> 00:49:34.670 align:middle line:84% and profits, they both have some-- 00:49:34.670 --> 00:49:37.430 align:middle line:90% 00:49:37.430 --> 00:49:38.660 align:middle line:90% sorry. 00:49:38.660 --> 00:49:41.930 align:middle line:84% For income and log income, the ones 00:49:41.930 --> 00:49:43.430 align:middle line:84% with the additional percentile ranks 00:49:43.430 --> 00:49:45.263 align:middle line:84% is doing a lot better than the ones without. 00:49:45.263 --> 00:49:47.630 align:middle line:84% For profits, maybe there's some evidence 00:49:47.630 --> 00:49:49.970 align:middle line:84% that the control has some predictive power, 00:49:49.970 --> 00:49:55.120 align:middle line:84% although it's not as clear in all specifications. 00:49:55.120 --> 00:49:58.430 align:middle line:90% So what does this mean? 00:49:58.430 --> 00:50:03.040 align:middle line:84% So I think it means that there is predictable heterogeneity, 00:50:03.040 --> 00:50:06.700 align:middle line:84% and local people in their community know it, 00:50:06.700 --> 00:50:11.718 align:middle line:84% maybe even above and beyond the observable x's. 00:50:11.718 --> 00:50:13.510 align:middle line:84% And then going back to [? Aaron's ?] point, 00:50:13.510 --> 00:50:15.220 align:middle line:84% then there's a question of what do you do about it? 00:50:15.220 --> 00:50:17.110 align:middle line:84% Do you try to elicit it from the community? 00:50:17.110 --> 00:50:20.237 align:middle line:84% Do you try to have bank agents in and figure it out? 00:50:20.237 --> 00:50:22.570 align:middle line:84% It's not clear, but I think that the whole point is that 00:50:22.570 --> 00:50:25.510 align:middle line:84% basically there is a lot-- this heterogeneity in returns is 00:50:25.510 --> 00:50:28.060 align:middle line:84% really something that is predictable, 00:50:28.060 --> 00:50:31.350 align:middle line:84% and understanding how do you get the right people in and select 00:50:31.350 --> 00:50:33.170 align:middle line:90% that is a really important-- 00:50:33.170 --> 00:50:34.690 align:middle line:84% it's something that banks obviously spend a lot of time 00:50:34.690 --> 00:50:35.190 align:middle line:90% on. 00:50:35.190 --> 00:50:38.730 align:middle line:84% And it's something important moving forward. 00:50:38.730 --> 00:50:39.985 align:middle line:90% Yeah, [? Kylash? ?] 00:50:39.985 --> 00:50:42.360 align:middle line:84% AUDIENCE: [INAUDIBLE] when you think about it [INAUDIBLE] 00:50:42.360 --> 00:50:44.360 align:middle line:84% do we know anything about what that actually is? 00:50:44.360 --> 00:50:49.384 align:middle line:84% Like, basically, what is the sum of people [INAUDIBLE] 00:50:49.384 --> 00:50:54.110 align:middle line:84% drives the differences in returns? 00:50:54.110 --> 00:50:56.660 align:middle line:84% BEN OLKEN: So I don't know that we necessarily know that 00:50:56.660 --> 00:50:59.280 align:middle line:84% because in some sense, the whole point of it is that it's-- 00:50:59.280 --> 00:51:01.820 align:middle line:84% there's predictable differences even above 00:51:01.820 --> 00:51:05.580 align:middle line:84% and beyond everything they can observe in the data. 00:51:05.580 --> 00:51:09.630 align:middle line:84% That's the point in the large part, is that we don't-- 00:51:09.630 --> 00:51:11.640 align:middle line:84% almost by definition-- the point of the exercise 00:51:11.640 --> 00:51:13.675 align:middle line:84% is there's an additional piece of it that 00:51:13.675 --> 00:51:16.050 align:middle line:84% is beyond what we can observe in the data, which makes it 00:51:16.050 --> 00:51:17.490 align:middle line:90% hard to answer that question. 00:51:17.490 --> 00:51:17.990 align:middle line:90% Yeah, Aaron? 00:51:17.990 --> 00:51:20.030 align:middle line:84% AUDIENCE: I think maybe [? Kylash's ?] question 00:51:20.030 --> 00:51:21.572 align:middle line:84% is something I was wondering, too, is 00:51:21.572 --> 00:51:23.250 align:middle line:84% when people are ranking their peers, 00:51:23.250 --> 00:51:26.260 align:middle line:90% what are they thinking about? 00:51:26.260 --> 00:51:26.860 align:middle line:90% I don't know. 00:51:26.860 --> 00:51:29.350 align:middle line:90% It just seems like it's-- 00:51:29.350 --> 00:51:33.020 align:middle line:84% what is it that communities know that [INAUDIBLE]?? 00:51:33.020 --> 00:51:35.440 align:middle line:84% I don't know if they did any kind of politic work 00:51:35.440 --> 00:51:36.100 align:middle line:90% to try to-- 00:51:36.100 --> 00:51:37.110 align:middle line:84% BEN OLKEN: So I don't know the answer to that question. 00:51:37.110 --> 00:51:40.060 align:middle line:84% As I said, it's a little hard to know because the whole point is 00:51:40.060 --> 00:51:41.768 align:middle line:84% it's something beyond what's in the data, 00:51:41.768 --> 00:51:43.150 align:middle line:90% so it's a little hard to know. 00:51:43.150 --> 00:51:43.900 align:middle line:90% I don't have a strong-- 00:51:43.900 --> 00:51:44.800 align:middle line:84% I don't have a great answer to that. 00:51:44.800 --> 00:51:45.300 align:middle line:90% Yeah. 00:51:45.300 --> 00:51:48.520 align:middle line:84% AUDIENCE: Well, it's-- by not visible in the data, it's not-- 00:51:48.520 --> 00:51:51.480 align:middle line:84% if things aren't visible in a one-time baseline survey, 00:51:51.480 --> 00:51:53.436 align:middle line:84% correct, when I go to the shop [INAUDIBLE].. 00:51:53.436 --> 00:51:56.370 align:middle line:84% So things that look like surface quality stuff, people-- 00:51:56.370 --> 00:51:57.830 align:middle line:90% BEN OLKEN: It could be. 00:51:57.830 --> 00:52:00.170 align:middle line:84% It could be entrepreneurial ability, 00:52:00.170 --> 00:52:01.380 align:middle line:90% which is hard to measure. 00:52:01.380 --> 00:52:02.870 align:middle line:84% I bet if you thought about your-- 00:52:02.870 --> 00:52:05.425 align:middle line:84% think about your friends from college, for example. 00:52:05.425 --> 00:52:06.800 align:middle line:84% If you were to rank them in terms 00:52:06.800 --> 00:52:07.910 align:middle line:84% of their entrepreneurial ability, 00:52:07.910 --> 00:52:09.200 align:middle line:84% do you think there's heterogeneity there? 00:52:09.200 --> 00:52:10.850 align:middle line:84% And do you think that's above and beyond what you 00:52:10.850 --> 00:52:11.840 align:middle line:90% predict based on their grades? 00:52:11.840 --> 00:52:12.340 align:middle line:90% Like-- 00:52:12.340 --> 00:52:14.090 align:middle line:90% [LAUGHTER] 00:52:14.090 --> 00:52:15.400 align:middle line:90% 00:52:15.400 --> 00:52:16.300 align:middle line:90% I mean, I don't know. 00:52:16.300 --> 00:52:17.020 align:middle line:90% Do that introspection. 00:52:17.020 --> 00:52:18.145 align:middle line:90% I'm curious what you think. 00:52:18.145 --> 00:52:20.560 align:middle line:84% Like, my guess is yes, but I don't know. 00:52:20.560 --> 00:52:22.775 align:middle line:84% And there is-- there's a lot of stuff that 00:52:22.775 --> 00:52:25.150 align:middle line:84% goes into people's personality and their business ability 00:52:25.150 --> 00:52:27.460 align:middle line:84% and whatever that may be really hard to capture 00:52:27.460 --> 00:52:30.655 align:middle line:84% in a simple survey or even a complex survey. 00:52:30.655 --> 00:52:31.655 align:middle line:90% Yeah, [? Becca? ?] What? 00:52:31.655 --> 00:52:33.080 align:middle line:90% AUDIENCE: LinkedIn does now. 00:52:33.080 --> 00:52:33.620 align:middle line:90% BEN OLKEN: I'm sorry, what? 00:52:33.620 --> 00:52:34.880 align:middle line:84% AUDIENCE: That's what LinkedIn does now. 00:52:34.880 --> 00:52:36.890 align:middle line:84% They ask people in your network to rate 00:52:36.890 --> 00:52:38.868 align:middle line:84% the relative skill of a particular thing 00:52:38.868 --> 00:52:40.160 align:middle line:90% of other people on the network. 00:52:40.160 --> 00:52:40.940 align:middle line:90% BEN OLKEN: Do they seriously? 00:52:40.940 --> 00:52:41.390 align:middle line:90% AUDIENCE: Yes. 00:52:41.390 --> 00:52:42.098 align:middle line:90% AUDIENCE: Mm-hmm. 00:52:42.098 --> 00:52:42.740 align:middle line:90% AUDIENCE: Yeah. 00:52:42.740 --> 00:52:43.410 align:middle line:84% BEN OLKEN: That's super interesting. 00:52:43.410 --> 00:52:44.210 align:middle line:90% AUDIENCE: Yes. 00:52:44.210 --> 00:52:45.668 align:middle line:84% BEN OLKEN: Has anyone studied that? 00:52:45.668 --> 00:52:46.910 align:middle line:90% [LAUGHTER] 00:52:46.910 --> 00:52:49.010 align:middle line:84% AUDIENCE: [INAUDIBLE] implemented by an economist 00:52:49.010 --> 00:52:49.900 align:middle line:90% who-- 00:52:49.900 --> 00:52:50.660 align:middle line:90% yeah. 00:52:50.660 --> 00:52:51.120 align:middle line:90% BEN OLKEN: [INAUDIBLE] 00:52:51.120 --> 00:52:51.745 align:middle line:90% AUDIENCE: Yeah. 00:52:51.745 --> 00:52:54.030 align:middle line:90% 00:52:54.030 --> 00:52:55.920 align:middle line:84% BEN OLKEN: What do they use that for? 00:52:55.920 --> 00:52:57.750 align:middle line:84% AUDIENCE: So I think it is-- like, you'll 00:52:57.750 --> 00:53:00.455 align:middle line:84% ask a question that's, like, who would you go to for questions 00:53:00.455 --> 00:53:01.080 align:middle line:90% on [INAUDIBLE]? 00:53:01.080 --> 00:53:03.160 align:middle line:84% And then it'll give you three of your friends, 00:53:03.160 --> 00:53:05.405 align:middle line:84% and then you select one of them just 00:53:05.405 --> 00:53:06.655 align:middle line:90% because it's fun or something. 00:53:06.655 --> 00:53:06.940 align:middle line:90% BEN OLKEN: Uh-huh. 00:53:06.940 --> 00:53:07.590 align:middle line:90% AUDIENCE: And then-- 00:53:07.590 --> 00:53:08.140 align:middle line:84% BEN OLKEN: And then what do they-- 00:53:08.140 --> 00:53:08.800 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:53:08.800 --> 00:53:10.180 align:middle line:84% BEN OLKEN: And then they take that person and give-- 00:53:10.180 --> 00:53:12.413 align:middle line:84% AUDIENCE: This person is more skillful, and then-- 00:53:12.413 --> 00:53:13.080 align:middle line:90% BEN OLKEN: Whoa. 00:53:13.080 --> 00:53:16.070 align:middle line:84% AUDIENCE: --I presume they sell this data to recruiters. 00:53:16.070 --> 00:53:16.866 align:middle line:90% BEN OLKEN: Wow. 00:53:16.866 --> 00:53:17.570 align:middle line:90% Huh. 00:53:17.570 --> 00:53:19.143 align:middle line:84% I'm not on LinkedIn, as you can see. 00:53:19.143 --> 00:53:21.060 align:middle line:84% AUDIENCE: Perhaps it goes into search results, 00:53:21.060 --> 00:53:23.890 align:middle line:84% and that's a freebie that you get. 00:53:23.890 --> 00:53:25.140 align:middle line:90% BEN OLKEN: That's fascinating. 00:53:25.140 --> 00:53:27.223 align:middle line:84% And I would love to see someone study that if they 00:53:27.223 --> 00:53:28.560 align:middle line:90% haven't done it already. 00:53:28.560 --> 00:53:30.112 align:middle line:90% That's super interesting. 00:53:30.112 --> 00:53:33.210 align:middle line:90% 00:53:33.210 --> 00:53:33.890 align:middle line:90% OK. 00:53:33.890 --> 00:53:35.090 align:middle line:84% But it's exactly the same idea, that people 00:53:35.090 --> 00:53:36.230 align:middle line:84% seem to know stuff about their peers, 00:53:36.230 --> 00:53:38.300 align:middle line:84% and maybe we can-- maybe LinkedIn has figured out 00:53:38.300 --> 00:53:40.010 align:middle line:90% a way to monetize that. 00:53:40.010 --> 00:53:41.498 align:middle line:84% But I think it's super interesting. 00:53:41.498 --> 00:53:43.790 align:middle line:84% I'd love to-- that seems like a great topic for someone 00:53:43.790 --> 00:53:46.600 align:middle line:84% to study if they haven't already. 00:53:46.600 --> 00:53:47.100 align:middle line:90% OK. 00:53:47.100 --> 00:53:48.330 align:middle line:84% So what I want to talk about-- is there 00:53:48.330 --> 00:53:50.747 align:middle line:84% any other comments you guys wanted to raise from the paper 00:53:50.747 --> 00:53:53.620 align:middle line:84% that you guys read, from the paper? 00:53:53.620 --> 00:53:54.120 align:middle line:90% OK. 00:53:54.120 --> 00:53:56.120 align:middle line:84% So what I want to talk about the last 20 minutes 00:53:56.120 --> 00:54:00.620 align:middle line:84% is the unbundling of microfinance. 00:54:00.620 --> 00:54:03.590 align:middle line:84% So here are some stylized facts about microfinance. 00:54:03.590 --> 00:54:05.390 align:middle line:84% In general, default rates are really low. 00:54:05.390 --> 00:54:08.840 align:middle line:84% Not always, but in general, they've been pretty low. 00:54:08.840 --> 00:54:13.010 align:middle line:84% And the original Grameen-Bank-style microfinance 00:54:13.010 --> 00:54:17.270 align:middle line:90% model had many elements. 00:54:17.270 --> 00:54:20.010 align:middle line:90% They lent only to women. 00:54:20.010 --> 00:54:22.380 align:middle line:84% They had a weekly repayment schedule 00:54:22.380 --> 00:54:24.930 align:middle line:84% where you start repaying immediately. 00:54:24.930 --> 00:54:28.037 align:middle line:84% As an aside, I think this is a little weird, right? 00:54:28.037 --> 00:54:28.620 align:middle line:90% Here's a loan. 00:54:28.620 --> 00:54:29.310 align:middle line:90% Go make an investment. 00:54:29.310 --> 00:54:30.935 align:middle line:84% And I want the first payment next week. 00:54:30.935 --> 00:54:33.048 align:middle line:90% Like, it may not be optimal. 00:54:33.048 --> 00:54:34.590 align:middle line:84% There may be reasons this is optimal. 00:54:34.590 --> 00:54:35.550 align:middle line:84% The reason I think they do it is they 00:54:35.550 --> 00:54:37.425 align:middle line:84% want people to get in the habit of repayment. 00:54:37.425 --> 00:54:39.720 align:middle line:84% I think that's the theory behind it. 00:54:39.720 --> 00:54:42.068 align:middle line:84% But as an investment decision, it's-- 00:54:42.068 --> 00:54:43.860 align:middle line:84% on the other hand, you can say, well, fine. 00:54:43.860 --> 00:54:47.460 align:middle line:84% I'm going to-- you need to buy a $1,100-- $100 machine. 00:54:47.460 --> 00:54:48.630 align:middle line:90% I'll give you $150. 00:54:48.630 --> 00:54:50.040 align:middle line:84% The first $50 you're just going to pay back to me 00:54:50.040 --> 00:54:51.030 align:middle line:84% in the first five weeks, then you're 00:54:51.030 --> 00:54:51.900 align:middle line:90% going to start getting a return. 00:54:51.900 --> 00:54:52.590 align:middle line:90% Maybe that's what they do. 00:54:52.590 --> 00:54:53.132 align:middle line:90% I don't know. 00:54:53.132 --> 00:54:54.510 align:middle line:90% Just saying, it's a thing. 00:54:54.510 --> 00:54:56.130 align:middle line:90% They have group lending. 00:54:56.130 --> 00:54:58.560 align:middle line:84% So basically, you come into this microfinance organization 00:54:58.560 --> 00:55:00.727 align:middle line:84% in a group, and initially, in a lot of these models, 00:55:00.727 --> 00:55:01.810 align:middle line:90% you have group liability. 00:55:01.810 --> 00:55:05.700 align:middle line:84% So that means that Hazel and Whitney and I are in a group, 00:55:05.700 --> 00:55:08.847 align:middle line:84% and if Hazel defaults, I'm on the hook for it. 00:55:08.847 --> 00:55:10.680 align:middle line:84% So don't default. That was the idea, though. 00:55:10.680 --> 00:55:13.263 align:middle line:84% So therefore, I'm not going to want to be in a group with them 00:55:13.263 --> 00:55:15.090 align:middle line:84% if I think they're kind of dodgy, 00:55:15.090 --> 00:55:16.698 align:middle line:84% so we solve the selection problem. 00:55:16.698 --> 00:55:18.240 align:middle line:84% And we solve the moral hazard problem 00:55:18.240 --> 00:55:19.620 align:middle line:84% because if she starts to default, 00:55:19.620 --> 00:55:22.770 align:middle line:84% I can go-- like, we're friends, and I can go bang on her door 00:55:22.770 --> 00:55:24.895 align:middle line:84% and be like, no, no, no, you have to pay this back. 00:55:24.895 --> 00:55:26.680 align:middle line:84% Otherwise, I'm going to hook for it. 00:55:26.680 --> 00:55:28.410 align:middle line:90% So that was the thing. 00:55:28.410 --> 00:55:30.240 align:middle line:84% They had regular meetings where they 00:55:30.240 --> 00:55:32.820 align:middle line:90% met with people all the time. 00:55:32.820 --> 00:55:34.230 align:middle line:90% They had dynamic incentives. 00:55:34.230 --> 00:55:36.972 align:middle line:84% So basically, you had-- you started off 00:55:36.972 --> 00:55:38.430 align:middle line:84% with really small loans, and if you 00:55:38.430 --> 00:55:40.347 align:middle line:84% pay them back you get bigger and bigger loans. 00:55:40.347 --> 00:55:42.810 align:middle line:84% So the return-- the incentive for paying back 00:55:42.810 --> 00:55:45.180 align:middle line:90% was bigger and bigger loans. 00:55:45.180 --> 00:55:47.520 align:middle line:84% They have credit officers, who basically 00:55:47.520 --> 00:55:49.710 align:middle line:84% are spending lots of time in these group meetings 00:55:49.710 --> 00:55:51.240 align:middle line:84% and talking to people and monitoring them and doing 00:55:51.240 --> 00:55:52.470 align:middle line:90% all kinds of stuff. 00:55:52.470 --> 00:55:57.010 align:middle line:84% And they have high interest rates, at least 20% a year, 00:55:57.010 --> 00:55:59.510 align:middle line:90% but often way higher. 00:55:59.510 --> 00:56:03.940 align:middle line:90% So this is a bundle, right? 00:56:03.940 --> 00:56:09.200 align:middle line:84% And there's a bunch of fear-- we can tell stories 00:56:09.200 --> 00:56:12.260 align:middle line:84% as to why all of these things might matter and how 00:56:12.260 --> 00:56:13.320 align:middle line:90% and whatever. 00:56:13.320 --> 00:56:17.390 align:middle line:84% And I think there's been a series of papers that 00:56:17.390 --> 00:56:23.180 align:middle line:84% have tried to decompose them and say, well, we're 00:56:23.180 --> 00:56:26.690 align:middle line:84% just going to study this, or we're just going to study this, 00:56:26.690 --> 00:56:30.350 align:middle line:84% or so on and so forth, to try to understand, how do we 00:56:30.350 --> 00:56:34.395 align:middle line:84% take this giant bundle of stuff and figure out what's important 00:56:34.395 --> 00:56:36.770 align:middle line:84% and what's not and what's actually helpful and what's not 00:56:36.770 --> 00:56:37.290 align:middle line:90% or whatever. 00:56:37.290 --> 00:56:41.960 align:middle line:84% And so I think it's also useful to see how this research 00:56:41.960 --> 00:56:46.730 align:middle line:84% program as a field has gone on because you can see people 00:56:46.730 --> 00:56:50.336 align:middle line:90% have been hacking away at-- 00:56:50.336 --> 00:56:53.150 align:middle line:84% chipping away at this big bundle to sort of like debundle it 00:56:53.150 --> 00:56:54.660 align:middle line:84% and understand what's going on here. 00:56:54.660 --> 00:56:55.160 align:middle line:90% Yeah? 00:56:55.160 --> 00:56:58.880 align:middle line:84% AUDIENCE: [INAUDIBLE] incentives the same for profits than-- 00:56:58.880 --> 00:57:00.890 align:middle line:90% for profits and nonprofits? 00:57:00.890 --> 00:57:03.057 align:middle line:84% BEN OLKEN: I don't know the answer to that question. 00:57:03.057 --> 00:57:04.653 align:middle line:90% 00:57:04.653 --> 00:57:06.070 align:middle line:84% In general, I think in all of them 00:57:06.070 --> 00:57:07.445 align:middle line:84% they want them to not lose money, 00:57:07.445 --> 00:57:09.460 align:middle line:84% but beyond that, I don't know the answer. 00:57:09.460 --> 00:57:10.280 align:middle line:90% Yeah, Wesley? 00:57:10.280 --> 00:57:11.360 align:middle line:84% AUDIENCE: Just thinking about our conversation 00:57:11.360 --> 00:57:12.050 align:middle line:90% from last time. 00:57:12.050 --> 00:57:14.840 align:middle line:84% Is this last point of high interest rates something 00:57:14.840 --> 00:57:17.130 align:middle line:84% that an organization like Grameen Bank 00:57:17.130 --> 00:57:19.723 align:middle line:84% would evangelize as, like, this is what microcredit should 00:57:19.723 --> 00:57:21.140 align:middle line:84% look like, or is it just something 00:57:21.140 --> 00:57:23.840 align:middle line:90% that comes to be by the-- 00:57:23.840 --> 00:57:26.690 align:middle line:84% BEN OLKEN: Well, I don't think they're proactively saying, 00:57:26.690 --> 00:57:28.460 align:middle line:84% yeah, it's really awesome our interest rates are really high. 00:57:28.460 --> 00:57:29.900 align:middle line:90% That's not how they spin things. 00:57:29.900 --> 00:57:30.567 align:middle line:90% AUDIENCE: Right. 00:57:30.567 --> 00:57:34.490 align:middle line:84% But-- if only because it seems like from our conversation from 00:57:34.490 --> 00:57:38.360 align:middle line:84% last class that this is a component of microcredit that 00:57:38.360 --> 00:57:41.950 align:middle line:84% would incentivize riskier investments than you would 00:57:41.950 --> 00:57:42.450 align:middle line:90% otherwise-- 00:57:42.450 --> 00:57:43.740 align:middle line:84% BEN OLKEN: On the other hand, they can't lose money, 00:57:43.740 --> 00:57:45.160 align:middle line:90% so that's the other problem. 00:57:45.160 --> 00:57:47.970 align:middle line:90% Maybe all this stuff cost money. 00:57:47.970 --> 00:57:49.470 align:middle line:84% Like, going back to the whole-- that 00:57:49.470 --> 00:57:51.060 align:middle line:84% was the point of the monitoring model. 00:57:51.060 --> 00:57:52.890 align:middle line:84% All this stuff costs money, and then 00:57:52.890 --> 00:57:54.660 align:middle line:84% if we have all this real costs and we 00:57:54.660 --> 00:57:57.608 align:middle line:84% have a really small loan, that gives you the high interest 00:57:57.608 --> 00:57:59.900 align:middle line:84% rate because you got to cover those costs with a really 00:57:59.900 --> 00:58:00.400 align:middle line:90% small base. 00:58:00.400 --> 00:58:02.780 align:middle line:84% So that's the-- that was the point of that-- sometimes 00:58:02.780 --> 00:58:04.820 align:middle line:84% that was the point of that monitoring model was 00:58:04.820 --> 00:58:07.910 align:middle line:84% to say, look, we have all these expensive activities. 00:58:07.910 --> 00:58:10.430 align:middle line:90% 00:58:10.430 --> 00:58:12.422 align:middle line:84% And they're basically fixed costs. 00:58:12.422 --> 00:58:14.630 align:middle line:84% And if we we have a fixed cost divided by small loan, 00:58:14.630 --> 00:58:15.710 align:middle line:84% that's going to look like a high interest rate. 00:58:15.710 --> 00:58:17.960 align:middle line:84% AUDIENCE: But aren't a lot of these subsidized anyway? 00:58:17.960 --> 00:58:19.868 align:middle line:90% BEN OLKEN: Not necessarily. 00:58:19.868 --> 00:58:21.160 align:middle line:90% A lot of them may be nonprofit. 00:58:21.160 --> 00:58:23.500 align:middle line:84% Well, it's not clear, but a lot of them are-- 00:58:23.500 --> 00:58:25.180 align:middle line:84% some are for-profit, and they just haven't made money. 00:58:25.180 --> 00:58:26.350 align:middle line:84% Some of them are not-for-profit, but they still 00:58:26.350 --> 00:58:27.320 align:middle line:90% got to break even. 00:58:27.320 --> 00:58:32.290 align:middle line:90% And so I think it's not, yeah. 00:58:32.290 --> 00:58:33.040 align:middle line:90% OK. 00:58:33.040 --> 00:58:36.055 align:middle line:84% So let me-- in the interest-- let 00:58:36.055 --> 00:58:40.828 align:middle line:84% me just pick a couple of these and see if they matter. 00:58:40.828 --> 00:58:41.870 align:middle line:90% Let me tell you a couple. 00:58:41.870 --> 00:58:44.710 align:middle line:84% So one, because I'm not going to have 00:58:44.710 --> 00:58:48.370 align:middle line:90% time to go through all of them. 00:58:48.370 --> 00:58:50.890 align:middle line:84% One was about lending to women and said, look, 00:58:50.890 --> 00:58:53.080 align:middle line:90% is that a relevant thing? 00:58:53.080 --> 00:58:55.840 align:middle line:84% So this paper by De Mel, McKenzie, and Woodruff 00:58:55.840 --> 00:58:58.180 align:middle line:90% basically said-- 00:58:58.180 --> 00:59:05.880 align:middle line:84% it was actually one of the original cash grant papers. 00:59:05.880 --> 00:59:07.620 align:middle line:90% And they said, let's-- 00:59:07.620 --> 00:59:10.295 align:middle line:90% 00:59:10.295 --> 00:59:11.580 align:middle line:90% they worked in Sri Lanka. 00:59:11.580 --> 00:59:13.410 align:middle line:84% They basically identified 400 households 00:59:13.410 --> 00:59:15.590 align:middle line:84% who had a small business, not a lot of capital. 00:59:15.590 --> 00:59:18.090 align:middle line:84% And they randomized them to get a small grant of either cash 00:59:18.090 --> 00:59:22.980 align:middle line:84% or an asset of about $100 or $200, 00:59:22.980 --> 00:59:25.200 align:middle line:90% and they followed them up-- 00:59:25.200 --> 00:59:26.910 align:middle line:84% I think six months later and then maybe 00:59:26.910 --> 00:59:28.330 align:middle line:90% even five years later. 00:59:28.330 --> 00:59:32.010 align:middle line:84% And the main finding was they had very large returns 00:59:32.010 --> 00:59:32.640 align:middle line:90% on capital. 00:59:32.640 --> 00:59:35.547 align:middle line:84% That's consistent with this idea that they 00:59:35.547 --> 00:59:37.380 align:middle line:84% were credit constrained, so that the returns 00:59:37.380 --> 00:59:41.920 align:middle line:84% looked like 60% returns per year, so really high returns. 00:59:41.920 --> 00:59:44.277 align:middle line:84% But they found in their paper-- there 00:59:44.277 --> 00:59:45.360 align:middle line:90% were no effects for women. 00:59:45.360 --> 00:59:47.160 align:middle line:84% They did some heterogeneous stuff, 00:59:47.160 --> 00:59:50.668 align:middle line:84% and they found no effects for female-owned businesses. 00:59:50.668 --> 00:59:51.710 align:middle line:90% So there's a main effect. 00:59:51.710 --> 00:59:53.353 align:middle line:84% Main effect times women is nothing. 00:59:53.353 --> 00:59:55.020 align:middle line:84% So that actually says, well, maybe there 00:59:55.020 --> 00:59:56.760 align:middle line:90% was not something going on. 00:59:56.760 --> 00:59:58.800 align:middle line:84% Maybe there was something different with women's 00:59:58.800 --> 00:59:59.300 align:middle line:90% businesses. 00:59:59.300 --> 01:00:02.190 align:middle line:84% And there were other papers that had similar findings. 01:00:02.190 --> 01:00:06.690 align:middle line:84% And I'll just note that one interesting recent paper 01:00:06.690 --> 01:00:08.550 align:middle line:84% on this is a paper by Bernhardt et al. 01:00:08.550 --> 01:00:10.558 align:middle line:90% Which goes back to this paper-- 01:00:10.558 --> 01:00:12.100 align:middle line:84% by this Del Mel et al paper and says, 01:00:12.100 --> 01:00:13.920 align:middle line:90% well, what's going on here? 01:00:13.920 --> 01:00:16.870 align:middle line:84% And they have an interesting hypothesis, 01:00:16.870 --> 01:00:19.830 align:middle line:84% which is their point is we shouldn't be looking 01:00:19.830 --> 01:00:23.430 align:middle line:90% at the business in isolation. 01:00:23.430 --> 01:00:25.280 align:middle line:84% We need to look at the whole household. 01:00:25.280 --> 01:00:27.860 align:middle line:84% And maybe what's going on is not that there are differences 01:00:27.860 --> 01:00:30.290 align:middle line:84% in men versus women as entrepreneurs, 01:00:30.290 --> 01:00:32.030 align:middle line:90% but what's going on-- 01:00:32.030 --> 01:00:33.980 align:middle line:84% maybe their whole intrahousehold dynamics 01:00:33.980 --> 01:00:36.162 align:middle line:84% are really important, and thinking about the whole-- 01:00:36.162 --> 01:00:38.120 align:middle line:84% actually we talked about intrahousehold issues. 01:00:38.120 --> 01:00:40.580 align:middle line:84% Maybe we should be thinking about them 01:00:40.580 --> 01:00:43.820 align:middle line:90% in the context of a household. 01:00:43.820 --> 01:00:50.690 align:middle line:84% And in particular, what they show is that basically-- 01:00:50.690 --> 01:00:55.010 align:middle line:84% they show that-- they replicate the results in Sri Lanka 01:00:55.010 --> 01:00:59.700 align:middle line:84% that there's no impact on the female-owned enterprise. 01:00:59.700 --> 01:01:04.440 align:middle line:84% But they note that when the woman gets a grant, 01:01:04.440 --> 01:01:08.020 align:middle line:84% their overall household income is actually going up. 01:01:08.020 --> 01:01:12.090 align:middle line:84% And so what actually they argue is that basically there are-- 01:01:12.090 --> 01:01:17.184 align:middle line:90% 01:01:17.184 --> 01:01:18.830 align:middle line:84% you have to be careful about looking 01:01:18.830 --> 01:01:19.940 align:middle line:90% at these female-owned business. 01:01:19.940 --> 01:01:22.148 align:middle line:84% And the other thing, I think, that they also look at, 01:01:22.148 --> 01:01:24.465 align:middle line:84% although I didn't bring that table, 01:01:24.465 --> 01:01:26.090 align:middle line:84% is they also show that actually, if you 01:01:26.090 --> 01:01:29.630 align:middle line:84% look at female-owned businesses where the woman is not 01:01:29.630 --> 01:01:32.330 align:middle line:84% part of a household with a man, where she's just 01:01:32.330 --> 01:01:36.050 align:middle line:84% acting on her own, that basically then her returns look 01:01:36.050 --> 01:01:37.500 align:middle line:90% just like the men's returns. 01:01:37.500 --> 01:01:39.830 align:middle line:84% So essentially what they argue is that what's happening 01:01:39.830 --> 01:01:42.140 align:middle line:84% is not that her business has lower returns, 01:01:42.140 --> 01:01:43.730 align:middle line:84% but in that household bargaining, 01:01:43.730 --> 01:01:45.770 align:middle line:84% if she gets the grant, maybe she channels into her husband's 01:01:45.770 --> 01:01:46.670 align:middle line:84% business, and her husband's business 01:01:46.670 --> 01:01:48.500 align:middle line:84% is the one that grows so in general their household 01:01:48.500 --> 01:01:49.480 align:middle line:90% returns are not so different. 01:01:49.480 --> 01:01:50.250 align:middle line:90% Yeah, [? Becca. ?] 01:01:50.250 --> 01:01:52.022 align:middle line:84% AUDIENCE: Are there restrictions that 01:01:52.022 --> 01:01:53.480 align:middle line:84% say that if you receive this grant, 01:01:53.480 --> 01:01:57.630 align:middle line:84% you need to demonstrably invest in your own business, 01:01:57.630 --> 01:01:58.130 align:middle line:90% or is this-- 01:01:58.130 --> 01:02:00.130 align:middle line:84% BEN OLKEN: I don't think for the cash grant one. 01:02:00.130 --> 01:02:00.333 align:middle line:90% 01:02:00.333 --> 01:02:01.080 align:middle line:90% AUDIENCE: A hand off to-- 01:02:01.080 --> 01:02:02.220 align:middle line:84% BEN OLKEN: I think they're able to hand it off. 01:02:02.220 --> 01:02:02.930 align:middle line:90% AUDIENCE: Yeah 01:02:02.930 --> 01:02:03.660 align:middle line:90% BEN OLKEN: Yeah. 01:02:03.660 --> 01:02:05.424 align:middle line:90% AUDIENCE: Is there a-- 01:02:05.424 --> 01:02:08.910 align:middle line:84% I don't-- [INAUDIBLE] if I give-- 01:02:08.910 --> 01:02:13.500 align:middle line:84% say the male entrepreneur [INAUDIBLE]---- 01:02:13.500 --> 01:02:16.110 align:middle line:84% the male household has his business. 01:02:16.110 --> 01:02:17.670 align:middle line:90% The woman has her business. 01:02:17.670 --> 01:02:20.280 align:middle line:84% If I give $1 to the man's business, 01:02:20.280 --> 01:02:22.980 align:middle line:84% what's the effect on the man's business, say, 01:02:22.980 --> 01:02:25.590 align:middle line:84% investment versus if I get the $1 to the women's business, 01:02:25.590 --> 01:02:29.525 align:middle line:84% to the woman, how much does the male's business increase its-- 01:02:29.525 --> 01:02:30.900 align:middle line:84% BEN OLKEN: Yeah, so I don't think 01:02:30.900 --> 01:02:32.800 align:middle line:84% they can totally look at that because I don't think they-- 01:02:32.800 --> 01:02:34.590 align:middle line:84% I don't think the data measured the man's business 01:02:34.590 --> 01:02:34.980 align:middle line:90% in that case. 01:02:34.980 --> 01:02:36.210 align:middle line:84% But they do measure household income, 01:02:36.210 --> 01:02:37.470 align:middle line:84% and that's where they can look at this. 01:02:37.470 --> 01:02:38.298 align:middle line:90% AUDIENCE: Is that-- 01:02:38.298 --> 01:02:40.860 align:middle line:84% BEN OLKEN: So they don't have the-- they don't quite 01:02:40.860 --> 01:02:44.280 align:middle line:84% have, I don't think, the data to answer exactly your question. 01:02:44.280 --> 01:02:46.320 align:middle line:84% But I think that this is suggestive of that, 01:02:46.320 --> 01:02:49.320 align:middle line:90% that's what's going on. 01:02:49.320 --> 01:02:49.860 align:middle line:90% OK. 01:02:49.860 --> 01:02:52.443 align:middle line:84% So I just wanted to mention that one briefly because I think-- 01:02:52.443 --> 01:02:53.330 align:middle line:90% I do think it-- 01:02:53.330 --> 01:02:54.380 align:middle line:84% because it illustrates this point 01:02:54.380 --> 01:02:55.160 align:middle line:84% that we have to be thinking-- when 01:02:55.160 --> 01:02:56.930 align:middle line:84% we think about these small household businesses, 01:02:56.930 --> 01:02:59.347 align:middle line:84% we should be thinking in the context of a household, which 01:02:59.347 --> 01:03:01.580 align:middle line:84% is making decisions across households in the same way 01:03:01.580 --> 01:03:03.530 align:middle line:84% we had plots and people were thinking about crop plots. 01:03:03.530 --> 01:03:05.988 align:middle line:84% You have to think about, what is the whole household doing? 01:03:05.988 --> 01:03:08.737 align:middle line:84% And actually, they point out that isn't this not thinking-- 01:03:08.737 --> 01:03:10.820 align:middle line:84% this whole paper points out that not thinking that 01:03:10.820 --> 01:03:13.640 align:middle line:84% through clearly actually led to very different kind of results. 01:03:13.640 --> 01:03:14.140 align:middle line:90% Yeah. 01:03:14.140 --> 01:03:15.430 align:middle line:90% AUDIENCE: Isn't there also a-- 01:03:15.430 --> 01:03:17.180 align:middle line:84% even if they're not like literally getting 01:03:17.180 --> 01:03:18.515 align:middle line:90% their [? grant ?] off, 01:03:18.515 --> 01:03:20.015 align:middle line:90% BEN OLKEN: This was-- 01:03:20.015 --> 01:03:20.720 align:middle line:90% yes, exactly. 01:03:20.720 --> 01:03:21.360 align:middle line:90% Exactly. 01:03:21.360 --> 01:03:21.860 align:middle line:90% Exactly. 01:03:21.860 --> 01:03:23.568 align:middle line:84% AUDIENCE: [INAUDIBLE] study [INAUDIBLE].. 01:03:23.568 --> 01:03:24.200 align:middle line:90% BEN OLKEN: Huh? 01:03:24.200 --> 01:03:26.242 align:middle line:84% AUDIENCE: Previous study is sort of like useless. 01:03:26.242 --> 01:03:27.908 align:middle line:84% BEN OLKEN: I don't think it was useless. 01:03:27.908 --> 01:03:28.940 align:middle line:90% [LAUGHTER] 01:03:28.940 --> 01:03:31.640 align:middle line:84% I think that-- well, first of all, the average effects 01:03:31.640 --> 01:03:32.770 align:middle line:90% are interesting. 01:03:32.770 --> 01:03:34.385 align:middle line:84% But yes, I think the point of this is 01:03:34.385 --> 01:03:35.760 align:middle line:84% not-- it's not that it's useless. 01:03:35.760 --> 01:03:37.105 align:middle line:84% I think the point of this study is 01:03:37.105 --> 01:03:38.660 align:middle line:84% to say, look, you want to understand these household 01:03:38.660 --> 01:03:39.350 align:middle line:90% issues, too. 01:03:39.350 --> 01:03:41.190 align:middle line:90% I don't think it was-- 01:03:41.190 --> 01:03:44.900 align:middle line:84% AUDIENCE: But you're really not measuring-- you're not even-- 01:03:44.900 --> 01:03:49.123 align:middle line:84% unless you're looking at women who are just 01:03:49.123 --> 01:03:50.540 align:middle line:84% the women in the household, you're 01:03:50.540 --> 01:03:53.778 align:middle line:84% really not even close to measuring the returns on these. 01:03:53.778 --> 01:03:55.570 align:middle line:84% BEN OLKEN: Yes, that is the point of this-- 01:03:55.570 --> 01:03:57.737 align:middle line:84% yes, that is the point of this Bernhardt paper, yes. 01:03:57.737 --> 01:03:58.870 align:middle line:90% But like-- so, right. 01:03:58.870 --> 01:03:59.830 align:middle line:90% So I'm just saying-- 01:03:59.830 --> 01:04:01.217 align:middle line:84% but people-- this-- people-- they 01:04:01.217 --> 01:04:02.800 align:middle line:84% didn't figure this out until recently. 01:04:02.800 --> 01:04:03.300 align:middle line:90% But, yes. 01:04:03.300 --> 01:04:04.060 align:middle line:90% AUDIENCE: Sure. 01:04:04.060 --> 01:04:06.673 align:middle line:84% BEN OLKEN: But I think that's why this paper came out and was 01:04:06.673 --> 01:04:07.840 align:middle line:90% published in a good journal. 01:04:07.840 --> 01:04:09.130 align:middle line:90% So I think that-- 01:04:09.130 --> 01:04:10.187 align:middle line:90% it was to say-- 01:04:10.187 --> 01:04:11.020 align:middle line:90% and the reason I'm-- 01:04:11.020 --> 01:04:12.400 align:middle line:84% I thought it was worth mentioning is exactly 01:04:12.400 --> 01:04:13.983 align:middle line:84% this point, is saying, look, if you're 01:04:13.983 --> 01:04:17.320 align:middle line:84% measuring these household businesses in a context where 01:04:17.320 --> 01:04:19.510 align:middle line:84% people are making decisions across businesses 01:04:19.510 --> 01:04:21.483 align:middle line:84% within a household, it is important to take 01:04:21.483 --> 01:04:22.150 align:middle line:90% the big picture. 01:04:22.150 --> 01:04:25.460 align:middle line:84% That's, I think, the main point here. 01:04:25.460 --> 01:04:26.000 align:middle line:90% Yeah? 01:04:26.000 --> 01:04:30.530 align:middle line:84% AUDIENCE: Is there a pulse take away from here in terms of-- 01:04:30.530 --> 01:04:34.100 align:middle line:84% if I want to help a family, which side of-- 01:04:34.100 --> 01:04:37.190 align:middle line:84% if I can help give grants to either the male-- 01:04:37.190 --> 01:04:40.120 align:middle line:84% the husband or the wife, is that a-- 01:04:40.120 --> 01:04:42.260 align:middle line:84% BEN OLKEN: That's a different question, which says, 01:04:42.260 --> 01:04:43.730 align:middle line:90% if we were to random-- 01:04:43.730 --> 01:04:45.900 align:middle line:84% if we had-- that's-- to answer that question, 01:04:45.900 --> 01:04:48.067 align:middle line:84% you have to have a different study, which is to say, 01:04:48.067 --> 01:04:49.100 align:middle line:90% I got to take-- 01:04:49.100 --> 01:04:51.680 align:middle line:84% I got to select a sample of people 01:04:51.680 --> 01:04:54.290 align:middle line:84% with both men and women and randomize who gets it 01:04:54.290 --> 01:04:57.190 align:middle line:90% and see what the outcomes are. 01:04:57.190 --> 01:04:57.690 align:middle line:90% So-- 01:04:57.690 --> 01:04:59.405 align:middle line:90% AUDIENCE: It's not [INAUDIBLE]. 01:04:59.405 --> 01:05:00.780 align:middle line:84% BEN OLKEN: I don't know of anyone 01:05:00.780 --> 01:05:02.280 align:middle line:84% who's run that study, although there 01:05:02.280 --> 01:05:04.580 align:middle line:84% are many studies in this area, and it is possible 01:05:04.580 --> 01:05:05.350 align:middle line:90% that someone has done it. 01:05:05.350 --> 01:05:06.240 align:middle line:90% And I don't know the answer. 01:05:06.240 --> 01:05:07.590 align:middle line:84% AUDIENCE: Can we talk about the conditional cash 01:05:07.590 --> 01:05:08.700 align:middle line:90% transfer that did that? 01:05:08.700 --> 01:05:11.283 align:middle line:84% BEN OLKEN: Cash transfers have done that for cash grants, yes. 01:05:11.283 --> 01:05:16.200 align:middle line:84% The GiveDirectly experiment does that in the context 01:05:16.200 --> 01:05:17.070 align:middle line:90% of cash transfers. 01:05:17.070 --> 01:05:18.528 align:middle line:84% That's one of their treatment arms. 01:05:18.528 --> 01:05:19.950 align:middle line:90% But that may be different for-- 01:05:19.950 --> 01:05:21.450 align:middle line:84% I mean, these were larger cash-- actually, 01:05:21.450 --> 01:05:22.410 align:middle line:84% that was pretty large, too, but they 01:05:22.410 --> 01:05:23.790 align:middle line:84% weren't restricted to entrepreneurs, 01:05:23.790 --> 01:05:24.690 align:middle line:84% people with entrepreneurial activities. 01:05:24.690 --> 01:05:26.430 align:middle line:84% So maybe for this entrepreneurial subset, 01:05:26.430 --> 01:05:27.617 align:middle line:90% it might be different. 01:05:27.617 --> 01:05:30.190 align:middle line:90% 01:05:30.190 --> 01:05:30.690 align:middle line:90% Yeah. 01:05:30.690 --> 01:05:31.830 align:middle line:90% Sorry, do you have a comment? 01:05:31.830 --> 01:05:32.372 align:middle line:90% AUDIENCE: Oh. 01:05:32.372 --> 01:05:35.105 align:middle line:84% Is there also some concern, potentially, in-- 01:05:35.105 --> 01:05:36.480 align:middle line:84% I don't know if they're adjusting 01:05:36.480 --> 01:05:38.880 align:middle line:84% for this, but the industries that [INAUDIBLE] 01:05:38.880 --> 01:05:40.146 align:middle line:90% businesses versus men? 01:05:40.146 --> 01:05:44.392 align:middle line:90% 01:05:44.392 --> 01:05:46.860 align:middle line:84% BEN OLKEN: I don't think they can do heterogeneity-- 01:05:46.860 --> 01:05:49.330 align:middle line:90% so what you'd want to do-- 01:05:49.330 --> 01:05:51.840 align:middle line:90% so you're absolutely right. 01:05:51.840 --> 01:05:53.340 align:middle line:84% So they're just running a regression 01:05:53.340 --> 01:06:02.870 align:middle line:84% of x equals treat plus woman plus treat times women, right? 01:06:02.870 --> 01:06:04.580 align:middle line:84% That's what that regression column 2 is. 01:06:04.580 --> 01:06:06.230 align:middle line:84% So you want to say, look, we also 01:06:06.230 --> 01:06:13.830 align:middle line:84% need to include plus industry plus treatment times industry, 01:06:13.830 --> 01:06:14.330 align:middle line:90% right? 01:06:14.330 --> 01:06:16.190 align:middle line:84% Not just industry dummies, but also treatment times industry 01:06:16.190 --> 01:06:18.648 align:middle line:84% because we may think there's heterogeneous returns if women 01:06:18.648 --> 01:06:19.550 align:middle line:90% are doing whatever. 01:06:19.550 --> 01:06:20.780 align:middle line:84% AUDIENCE: [INAUDIBLE] lower returns. 01:06:20.780 --> 01:06:21.230 align:middle line:90% BEN OLKEN: Yeah. 01:06:21.230 --> 01:06:22.520 align:middle line:84% Things that have low returns, capital 01:06:22.520 --> 01:06:23.390 align:middle line:90% is usually picking that up. 01:06:23.390 --> 01:06:23.750 align:middle line:90% Right. 01:06:23.750 --> 01:06:24.833 align:middle line:90% So you'd want to run that. 01:06:24.833 --> 01:06:27.510 align:middle line:84% Given that they only have 400 businesses or whatever, 01:06:27.510 --> 01:06:28.010 align:middle line:90% I'm sure-- 01:06:28.010 --> 01:06:29.060 align:middle line:84% I don't think they have enough power 01:06:29.060 --> 01:06:30.350 align:middle line:84% to run-- to include this specification. 01:06:30.350 --> 01:06:30.890 align:middle line:90% But you're absolutely right. 01:06:30.890 --> 01:06:32.450 align:middle line:84% That's another thing that you-- another hypothesis 01:06:32.450 --> 01:06:33.620 align:middle line:90% that you want to think about. 01:06:33.620 --> 01:06:34.120 align:middle line:90% Sure. 01:06:34.120 --> 01:06:40.350 align:middle line:90% 01:06:40.350 --> 01:06:42.930 align:middle line:90% OK. 01:06:42.930 --> 01:06:45.360 align:middle line:90% Let's skip this one. 01:06:45.360 --> 01:06:52.470 align:middle line:84% So another study that this team of coauthors has looked at 01:06:52.470 --> 01:06:56.410 align:middle line:84% is this question of this fact that you 01:06:56.410 --> 01:06:58.800 align:middle line:84% need to start repaying the loan as soon as you get it. 01:06:58.800 --> 01:07:00.190 align:middle line:90% So you get the loan. 01:07:00.190 --> 01:07:02.200 align:middle line:90% You start repaying next week. 01:07:02.200 --> 01:07:05.890 align:middle line:84% So is that-- as I mentioned, on the one hand, that 01:07:05.890 --> 01:07:09.160 align:middle line:90% seems really weird to me. 01:07:09.160 --> 01:07:10.780 align:middle line:84% Like, I'm going to make an investment, 01:07:10.780 --> 01:07:13.040 align:middle line:84% and it may take some time to get returns. 01:07:13.040 --> 01:07:14.890 align:middle line:84% On the other hand, some of the kinds 01:07:14.890 --> 01:07:17.650 align:middle line:84% of investments that people make look like I'm going to-- 01:07:17.650 --> 01:07:18.248 align:middle line:90% I have a shop. 01:07:18.248 --> 01:07:20.290 align:middle line:84% I'm going to buy some more inventory for my shop. 01:07:20.290 --> 01:07:22.450 align:middle line:84% It doesn't take really very long to actually start 01:07:22.450 --> 01:07:24.190 align:middle line:84% having returns on that, so maybe it's OK. 01:07:24.190 --> 01:07:27.970 align:middle line:84% Or, as I pointed out, you can actually replicate 01:07:27.970 --> 01:07:30.120 align:middle line:84% the non-grace period version of this by-- 01:07:30.120 --> 01:07:32.120 align:middle line:84% the non-weekly-repayment version by just saying, 01:07:32.120 --> 01:07:34.660 align:middle line:84% look, I'm going to take-- if I got $150 loan, 01:07:34.660 --> 01:07:36.760 align:middle line:84% I'm taking the first $50, put it aside, and use 01:07:36.760 --> 01:07:38.440 align:middle line:84% it to make my first payments, and then 01:07:38.440 --> 01:07:39.982 align:middle line:84% I'm taking and invest the other $100. 01:07:39.982 --> 01:07:42.770 align:middle line:84% So with a slightly larger loan, I can replicate this kind of 01:07:42.770 --> 01:07:43.270 align:middle line:90% myself. 01:07:43.270 --> 01:07:46.250 align:middle line:90% So does this matter? 01:07:46.250 --> 01:07:48.190 align:middle line:84% So what they do is they basically-- 01:07:48.190 --> 01:07:51.340 align:middle line:84% they run an experiment where they're going to nicely zero in 01:07:51.340 --> 01:07:53.760 align:middle line:84% on exactly this feature and say some people 01:07:53.760 --> 01:07:55.510 align:middle line:84% are going to get the normal contract where 01:07:55.510 --> 01:07:58.210 align:middle line:84% you start paying immediately, and some get a two-month grace 01:07:58.210 --> 01:07:58.710 align:middle line:90% period. 01:07:58.710 --> 01:08:02.420 align:middle line:84% Even this is not a very long period of time. 01:08:02.420 --> 01:08:05.490 align:middle line:84% But they get a two-month grace period to repay. 01:08:05.490 --> 01:08:07.780 align:middle line:84% And I think what happens is kind of interesting. 01:08:07.780 --> 01:08:09.720 align:middle line:84% So-- sorry, what do you expect to happen? 01:08:09.720 --> 01:08:14.040 align:middle line:90% 01:08:14.040 --> 01:08:15.000 align:middle line:90% Someone else. 01:08:15.000 --> 01:08:16.319 align:middle line:90% Yeah, [? Paulo. ?] 01:08:16.319 --> 01:08:18.194 align:middle line:84% AUDIENCE: With the grace period, will make it 01:08:18.194 --> 01:08:20.460 align:middle line:84% like higher returns back on average, but perhaps-- 01:08:20.460 --> 01:08:21.390 align:middle line:90% BEN OLKEN: What? 01:08:21.390 --> 01:08:23.790 align:middle line:84% AUDIENCE: You get higher returns with the grace period, 01:08:23.790 --> 01:08:25.972 align:middle line:84% but perhaps also more heterogeneity 01:08:25.972 --> 01:08:27.930 align:middle line:84% because people are taking riskier actions which 01:08:27.930 --> 01:08:31.643 align:middle line:84% raise the expected returns, but at the risk of losing it all? 01:08:31.643 --> 01:08:32.310 align:middle line:90% BEN OLKEN: Yeah. 01:08:32.310 --> 01:08:32.810 align:middle line:90% Right. 01:08:32.810 --> 01:08:33.390 align:middle line:90% So-- OK. 01:08:33.390 --> 01:08:34.109 align:middle line:90% Someone else? 01:08:34.109 --> 01:08:35.220 align:middle line:84% AUDIENCE: Might have a higher default 01:08:35.220 --> 01:08:36.510 align:middle line:90% rate in the grace period? 01:08:36.510 --> 01:08:37.470 align:middle line:90% BEN OLKEN: Why. 01:08:37.470 --> 01:08:39.660 align:middle line:84% AUDIENCE: Well, if the reason why you were doing it 01:08:39.660 --> 01:08:41.520 align:middle line:84% immediately [INAUDIBLE] you might 01:08:41.520 --> 01:08:43.937 align:middle line:84% have a reason to believe that a grace period [INAUDIBLE].. 01:08:43.937 --> 01:08:44.603 align:middle line:90% BEN OLKEN: Yeah. 01:08:44.603 --> 01:08:46.189 align:middle line:84% So if the whole point is this thing-- 01:08:46.189 --> 01:08:48.109 align:middle line:84% you might just have people get in the habit of default. 01:08:48.109 --> 01:08:49.460 align:middle line:84% It may also change your investment choices, 01:08:49.460 --> 01:08:51.020 align:middle line:84% and you may change different things. 01:08:51.020 --> 01:08:54.500 align:middle line:90% Other comments? 01:08:54.500 --> 01:08:56.500 align:middle line:90% AUDIENCE: [INAUDIBLE]. 01:08:56.500 --> 01:08:58.399 align:middle line:84% BEN OLKEN: Heterogeneous effects by what? 01:08:58.399 --> 01:09:00.080 align:middle line:90% AUDIENCE: So say some-- 01:09:00.080 --> 01:09:01.040 align:middle line:90% it will hurt some. 01:09:01.040 --> 01:09:03.035 align:middle line:84% Some people will not pay back, so you're 01:09:03.035 --> 01:09:04.490 align:middle line:90% adding behavioral biases. 01:09:04.490 --> 01:09:07.609 align:middle line:84% Some people really need to think about-- be disciplined 01:09:07.609 --> 01:09:10.470 align:middle line:84% about paying back in next week or so. 01:09:10.470 --> 01:09:13.069 align:middle line:84% But there's also a possibility that any investment opportunity 01:09:13.069 --> 01:09:16.340 align:middle line:84% that will take you more than a week to repay-- 01:09:16.340 --> 01:09:19.109 align:middle line:84% that you couldn't do before, now you will get do that. 01:09:19.109 --> 01:09:20.901 align:middle line:84% So the [? mean effect ?] can be zero, but-- 01:09:20.901 --> 01:09:22.901 align:middle line:84% BEN OLKEN: But there may be increasing variance. 01:09:22.901 --> 01:09:23.540 align:middle line:90% Yeah, could be. 01:09:23.540 --> 01:09:24.529 align:middle line:90% Absolutely. 01:09:24.529 --> 01:09:25.405 align:middle line:90% So what do they find? 01:09:25.405 --> 01:09:28.020 align:middle line:90% 01:09:28.020 --> 01:09:30.350 align:middle line:90% So they find people do seem to-- 01:09:30.350 --> 01:09:33.710 align:middle line:84% they use more of the thing on their business 01:09:33.710 --> 01:09:37.479 align:middle line:84% and less on household expenditures. 01:09:37.479 --> 01:09:38.979 align:middle line:84% This is not totally obvious to me, 01:09:38.979 --> 01:09:41.104 align:middle line:84% actually, that this would be the case, that there's 01:09:41.104 --> 01:09:45.880 align:middle line:84% this discrepancy between how using the money for loans 01:09:45.880 --> 01:09:47.085 align:middle line:90% versus business. 01:09:47.085 --> 01:09:49.210 align:middle line:84% It wasn't obvious to me that would be even a margin 01:09:49.210 --> 01:09:50.923 align:middle line:84% or why that margin should necessarily-- 01:09:50.923 --> 01:09:52.840 align:middle line:84% why should you move from consumption smoothing 01:09:52.840 --> 01:09:53.710 align:middle line:90% to business investment? 01:09:53.710 --> 01:09:55.015 align:middle line:90% Maybe the business investment-- 01:09:55.015 --> 01:09:57.057 align:middle line:84% I mean, the con smoothing never pays itself back, 01:09:57.057 --> 01:10:00.520 align:middle line:84% so that's not totally obvious to me that they would find that. 01:10:00.520 --> 01:10:02.463 align:middle line:84% But maybe in some broader sense, this is-- 01:10:02.463 --> 01:10:04.630 align:middle line:84% at least I understand what this is going to be like, 01:10:04.630 --> 01:10:07.083 align:middle line:84% and this is totally zero risk in some sense. 01:10:07.083 --> 01:10:08.500 align:middle line:84% Like, I know what it's going to be 01:10:08.500 --> 01:10:12.100 align:middle line:84% and I can plan for it, whereas this is more risky. 01:10:12.100 --> 01:10:17.690 align:middle line:84% They also find that there are higher impacts on default. 01:10:17.690 --> 01:10:23.440 align:middle line:84% So people do default more when you give them the grace period. 01:10:23.440 --> 01:10:26.030 align:middle line:84% So it is the case that there's this trade off. 01:10:26.030 --> 01:10:29.383 align:middle line:84% On the other hand, people's businesses actually do go up 01:10:29.383 --> 01:10:31.300 align:middle line:84% and their income goes up a lot more, actually, 01:10:31.300 --> 01:10:32.540 align:middle line:90% with the grace period. 01:10:32.540 --> 01:10:33.980 align:middle line:90% So I think that this-- 01:10:33.980 --> 01:10:36.460 align:middle line:84% this actually, I think, also illustrates 01:10:36.460 --> 01:10:38.350 align:middle line:84% that there may be a tension between what's 01:10:38.350 --> 01:10:41.360 align:middle line:84% good for the bank and what's good for the individuals, 01:10:41.360 --> 01:10:41.860 align:middle line:90% right? 01:10:41.860 --> 01:10:44.773 align:middle line:84% The bank does not care about your income. 01:10:44.773 --> 01:10:46.690 align:middle line:84% It only cares about it maybe if it cares about 01:10:46.690 --> 01:10:49.240 align:middle line:84% if it can charge higher interest rates to you. 01:10:49.240 --> 01:10:50.830 align:middle line:90% But the bank just wants-- 01:10:50.830 --> 01:10:52.210 align:middle line:84% conditional on making a the [? loan good ?] interest rate, 01:10:52.210 --> 01:10:53.440 align:middle line:90% it wants the loan paid back. 01:10:53.440 --> 01:10:54.940 align:middle line:84% So the bank is going to take actions 01:10:54.940 --> 01:10:57.607 align:middle line:84% to make sure the loan paid back, even if that is necessarily not 01:10:57.607 --> 01:10:58.840 align:middle line:90% necessarily optimal for you. 01:10:58.840 --> 01:11:00.632 align:middle line:84% And that's just-- there may be this tension 01:11:00.632 --> 01:11:01.990 align:middle line:90% between these things. 01:11:01.990 --> 01:11:02.740 align:middle line:90% Yeah? 01:11:02.740 --> 01:11:05.530 align:middle line:84% AUDIENCE: With this [? theory ?] behind the default instead 01:11:05.530 --> 01:11:08.470 align:middle line:84% of being about habit being about risky behavior? 01:11:08.470 --> 01:11:09.430 align:middle line:90% BEN OLKEN: Yeah, 100%. 01:11:09.430 --> 01:11:10.120 align:middle line:90% Absolutely. 01:11:10.120 --> 01:11:11.450 align:middle line:90% Yeah, absolutely. 01:11:11.450 --> 01:11:12.760 align:middle line:84% It totally could be about risk-taking behavior. 01:11:12.760 --> 01:11:13.630 align:middle line:90% I think it is, probably. 01:11:13.630 --> 01:11:15.255 align:middle line:84% I mean certainly-- we don't-- actually, 01:11:15.255 --> 01:11:17.491 align:middle line:84% I don't think we necessarily know. 01:11:17.491 --> 01:11:20.260 align:middle line:84% We know that people are investing more 01:11:20.260 --> 01:11:23.500 align:middle line:84% in their business, doing more stuff, earning higher incomes, 01:11:23.500 --> 01:11:25.287 align:middle line:90% and defaulting more. 01:11:25.287 --> 01:11:27.370 align:middle line:84% I don't think we necessarily know from this study, 01:11:27.370 --> 01:11:28.420 align:middle line:84% although I can't remember for sure-- 01:11:28.420 --> 01:11:29.920 align:middle line:84% I don't think they can differentiate 01:11:29.920 --> 01:11:32.273 align:middle line:84% between that the default is because it's increasing 01:11:32.273 --> 01:11:33.940 align:middle line:84% the variance of their outcome, sometimes 01:11:33.940 --> 01:11:37.160 align:middle line:84% it fails, versus this other direct effect of I 01:11:37.160 --> 01:11:39.160 align:middle line:84% get out of the habit of paying back or whatever. 01:11:39.160 --> 01:11:40.810 align:middle line:84% So I don't think we can differentiate between that. 01:11:40.810 --> 01:11:41.887 align:middle line:90% Yeah, [? Shannon? ?] 01:11:41.887 --> 01:11:44.470 align:middle line:84% AUDIENCE: Does the experimental design allows different people 01:11:44.470 --> 01:11:46.552 align:middle line:90% to select [INAUDIBLE]? 01:11:46.552 --> 01:11:47.510 align:middle line:90% BEN OLKEN: What, sorry? 01:11:47.510 --> 01:11:48.650 align:middle line:84% AUDIENCE: Does the experimental design 01:11:48.650 --> 01:11:51.077 align:middle line:84% allow for different groups of people to select into-- 01:11:51.077 --> 01:11:53.410 align:middle line:84% BEN OLKEN: I don't think so, but that's a good question. 01:11:53.410 --> 01:11:54.810 align:middle line:84% I don't remember it well enough to say, 01:11:54.810 --> 01:11:56.370 align:middle line:84% but I'm pretty sure they randomize it 01:11:56.370 --> 01:11:57.512 align:middle line:90% after people take up. 01:11:57.512 --> 01:11:59.220 align:middle line:84% I think it's a surprise, so I don't think 01:11:59.220 --> 01:12:00.303 align:middle line:90% it has a selection effect. 01:12:00.303 --> 01:12:01.650 align:middle line:90% But that would be interesting. 01:12:01.650 --> 01:12:04.360 align:middle line:84% You could imagine that it could be important. 01:12:04.360 --> 01:12:06.660 align:middle line:84% I don't think so, but I don't remember 100%. 01:12:06.660 --> 01:12:08.760 align:middle line:90% That's true. 01:12:08.760 --> 01:12:10.660 align:middle line:90% Other questions? 01:12:10.660 --> 01:12:11.470 align:middle line:90% Yeah, [? Ahmed? ?] 01:12:11.470 --> 01:12:13.678 align:middle line:84% AUDIENCE: On average, nonbusiness spending decreases, 01:12:13.678 --> 01:12:14.620 align:middle line:90% but I wonder-- 01:12:14.620 --> 01:12:19.210 align:middle line:84% if the increase in nonbusiness spending 01:12:19.210 --> 01:12:23.473 align:middle line:84% predicts default, the behavior [INAUDIBLE],, in a sense. 01:12:23.473 --> 01:12:24.140 align:middle line:90% BEN OLKEN: Yeah. 01:12:24.140 --> 01:12:25.595 align:middle line:84% I don't know how you'd identify that. 01:12:25.595 --> 01:12:26.820 align:middle line:84% These are some other characteristics 01:12:26.820 --> 01:12:28.570 align:middle line:84% that would predict heterogeneous response. 01:12:28.570 --> 01:12:31.190 align:middle line:84% AUDIENCE: Is it just like [INAUDIBLE] predict 01:12:31.190 --> 01:12:32.090 align:middle line:90% [INAUDIBLE]. 01:12:32.090 --> 01:12:33.170 align:middle line:90% It wouldn't be causal? 01:12:33.170 --> 01:12:34.940 align:middle line:84% BEN OLKEN: Yeah, I'm not even sure I 01:12:34.940 --> 01:12:36.530 align:middle line:84% would run that regression, but I think 01:12:36.530 --> 01:12:38.363 align:middle line:84% you would need some other kind of x variable 01:12:38.363 --> 01:12:39.620 align:middle line:90% that predicts that. 01:12:39.620 --> 01:12:41.480 align:middle line:84% I'm just about out of time, so let me just-- 01:12:41.480 --> 01:12:45.625 align:middle line:90% 01:12:45.625 --> 01:12:47.000 align:middle line:84% let me just sum them up by saying 01:12:47.000 --> 01:12:48.738 align:middle line:84% that people have gone through some 01:12:48.738 --> 01:12:50.530 align:middle line:84% of these other characteristics I mentioned. 01:12:50.530 --> 01:12:52.520 align:middle line:84% I'm not going to go through them in detail now. 01:12:52.520 --> 01:12:54.290 align:middle line:84% But you should look at them if you're interested. 01:12:54.290 --> 01:12:56.075 align:middle line:84% I think they went through group lending. 01:12:56.075 --> 01:12:59.017 align:middle line:90% 01:12:59.017 --> 01:13:00.350 align:middle line:90% They went through group lending. 01:13:00.350 --> 01:13:02.120 align:middle line:84% They went through some other things. 01:13:02.120 --> 01:13:04.430 align:middle line:84% People have systematically tried to decompose 01:13:04.430 --> 01:13:06.020 align:middle line:84% that list and understanding which of those things 01:13:06.020 --> 01:13:06.570 align:middle line:90% are important. 01:13:06.570 --> 01:13:08.237 align:middle line:84% And I think it turns out it's a lot more 01:13:08.237 --> 01:13:11.390 align:middle line:84% nuanced than the initial view from the Grameen Bank, which 01:13:11.390 --> 01:13:14.342 align:middle line:84% is you have to have this whole package or nothing. 01:13:14.342 --> 01:13:15.800 align:middle line:84% The final thing-- I'll just mention 01:13:15.800 --> 01:13:18.110 align:middle line:90% some other topics in credit. 01:13:18.110 --> 01:13:19.545 align:middle line:84% One thing we haven't talked about, 01:13:19.545 --> 01:13:21.920 align:middle line:84% which may come up next semester, but maybe not, depending 01:13:21.920 --> 01:13:27.460 align:middle line:84% on how much time they have, is local Indigenous institutions, 01:13:27.460 --> 01:13:30.800 align:middle line:84% so Rotating Savings and Credit Organizations. 01:13:30.800 --> 01:13:33.890 align:middle line:84% So one thing people sometimes do, 01:13:33.890 --> 01:13:35.530 align:middle line:84% if there's not a formal credit, is 01:13:35.530 --> 01:13:37.810 align:middle line:84% they'll get together and say, well, look, maybe 01:13:37.810 --> 01:13:38.920 align:middle line:90% we're all going to put-- 01:13:38.920 --> 01:13:39.820 align:middle line:90% every month, we'll get together. 01:13:39.820 --> 01:13:40.330 align:middle line:90% We'll have a meeting. 01:13:40.330 --> 01:13:42.160 align:middle line:84% We'll all put $10 in a pot, and we'll 01:13:42.160 --> 01:13:43.760 align:middle line:90% take turns who gets the $10. 01:13:43.760 --> 01:13:45.183 align:middle line:90% And you can think of that as-- 01:13:45.183 --> 01:13:47.350 align:middle line:84% that's a rotating organization, where-- so sometimes 01:13:47.350 --> 01:13:49.840 align:middle line:84% you can think of it as a credit organization. 01:13:49.840 --> 01:13:51.610 align:middle line:84% So the first person, in some sense, 01:13:51.610 --> 01:13:53.695 align:middle line:84% is getting a loan of $120-- if there 01:13:53.695 --> 01:13:56.770 align:middle line:84% are 12 of us in the, $120, which they're repaying over time 01:13:56.770 --> 01:13:58.390 align:middle line:84% by putting it into the pot later. 01:13:58.390 --> 01:14:00.580 align:middle line:84% And these informal institutions are 01:14:00.580 --> 01:14:02.530 align:middle line:84% very common throughout the developing world. 01:14:02.530 --> 01:14:04.193 align:middle line:84% And people have studied those to try 01:14:04.193 --> 01:14:05.360 align:middle line:90% to understand those as well. 01:14:05.360 --> 01:14:08.350 align:middle line:90% So I wanted to mention that. 01:14:08.350 --> 01:14:10.970 align:middle line:84% I think we're going to talk about demand for credit. 01:14:10.970 --> 01:14:13.810 align:middle line:84% This, we'll talk about it a little bit more, 01:14:13.810 --> 01:14:15.160 align:middle line:90% probably not in our station. 01:14:15.160 --> 01:14:17.698 align:middle line:90% Probably next semester. 01:14:17.698 --> 01:14:19.990 align:middle line:84% So I just want to mention understanding people's demand 01:14:19.990 --> 01:14:20.797 align:middle line:90% for credit. 01:14:20.797 --> 01:14:22.630 align:middle line:84% I guess we'll talk about that next semester. 01:14:22.630 --> 01:14:26.710 align:middle line:84% And we'll talk about some more stuff 01:14:26.710 --> 01:14:29.800 align:middle line:84% on banks as intermediaries and credit constraints 01:14:29.800 --> 01:14:32.500 align:middle line:90% for larger firms next semester. 01:14:32.500 --> 01:14:34.610 align:middle line:84% So I'm going to wrap up here Thanks very much. 01:14:34.610 --> 01:14:37.858 align:middle line:84% It's been great talking to you guys this semester. 01:14:37.858 --> 01:14:39.200 align:middle line:90% [APPLAUSE] 01:14:39.200 --> 01:14:40.360 align:middle line:90% Thank you. 01:14:40.360 --> 01:14:43.207 align:middle line:84% And I've left you five minutes or so to go fill out 01:14:43.207 --> 01:14:44.290 align:middle line:90% your teaching evaluations. 01:14:44.290 --> 01:14:46.310 align:middle line:90% So we very much appreciate that. 01:14:46.310 --> 01:14:47.110 align:middle line:90% OK. 01:14:47.110 --> 01:14:48.660 align:middle line:90% Thanks. 01:14:48.660 --> 01:15:01.000 align:middle line:90%