WEBVTT 00:00:00.000 --> 00:00:01.973 align:middle line:90% [SQUEAKING] 00:00:01.973 --> 00:00:03.446 align:middle line:90% [RUSTLING] 00:00:03.446 --> 00:00:05.901 align:middle line:90% [CLICKING] 00:00:05.901 --> 00:00:10.140 align:middle line:90% 00:00:10.140 --> 00:00:12.400 align:middle line:84% HAZHIR RAHMANDAD: OK, let's get started. 00:00:12.400 --> 00:00:15.040 align:middle line:84% Welcome to our last class, folks. 00:00:15.040 --> 00:00:17.760 align:middle line:90% It's bittersweet. 00:00:17.760 --> 00:00:22.980 align:middle line:84% So today, we'll continue talking about your personal projects. 00:00:22.980 --> 00:00:24.140 align:middle line:90% That will be the focus. 00:00:24.140 --> 00:00:28.520 align:middle line:84% And I suspect it can be a little bit more open-ended in the sense 00:00:28.520 --> 00:00:31.540 align:middle line:90% that this is the last session. 00:00:31.540 --> 00:00:36.320 align:middle line:84% What would be most useful for the specific project? 00:00:36.320 --> 00:00:40.040 align:middle line:84% Or even beyond that, if you're thinking about a modeling 00:00:40.040 --> 00:00:42.540 align:middle line:84% engagement and you want some feedback on it, 00:00:42.540 --> 00:00:45.500 align:middle line:84% we can open it up to that type of a conversation. 00:00:45.500 --> 00:00:47.640 align:middle line:84% We don't need to be focused on formulations 00:00:47.640 --> 00:00:50.860 align:middle line:84% as we have been in most of the semester. 00:00:50.860 --> 00:00:52.580 align:middle line:84% But if there are formulation issues, 00:00:52.580 --> 00:00:54.740 align:middle line:90% we can dig into those as well. 00:00:54.740 --> 00:00:59.000 align:middle line:90% That's perfectly fine. 00:00:59.000 --> 00:01:02.120 align:middle line:90% So that's one part of the class. 00:01:02.120 --> 00:01:04.560 align:middle line:84% And I think kind of not the second half, 00:01:04.560 --> 00:01:07.920 align:middle line:84% but last third, we can have a little bit more reflections, 00:01:07.920 --> 00:01:11.220 align:middle line:84% kind of what you have learned, what 00:01:11.220 --> 00:01:14.100 align:middle line:84% are kind of open questions for you in terms 00:01:14.100 --> 00:01:16.560 align:middle line:84% of the structure of class we could do differently. 00:01:16.560 --> 00:01:20.540 align:middle line:84% All of those are, I think, useful to have some conversation 00:01:20.540 --> 00:01:21.040 align:middle line:90% about. 00:01:21.040 --> 00:01:22.915 align:middle line:84% So those are the things I have in mind. 00:01:22.915 --> 00:01:24.540 align:middle line:84% But let's see if there is anything else 00:01:24.540 --> 00:01:28.060 align:middle line:84% that you folks think might be useful to discuss, 00:01:28.060 --> 00:01:31.200 align:middle line:90% cover as we go through it today. 00:01:31.200 --> 00:01:35.765 align:middle line:90% 00:01:35.765 --> 00:01:37.140 align:middle line:84% AUDIENCE: One thing I think would 00:01:37.140 --> 00:01:45.340 align:middle line:84% be useful is just how to continue our education in terms 00:01:45.340 --> 00:01:47.952 align:middle line:90% of getting better from here. 00:01:47.952 --> 00:01:49.160 align:middle line:90% HAZHIR RAHMANDAD: Good point. 00:01:49.160 --> 00:01:52.860 align:middle line:84% So we can definitely talk about that. 00:01:52.860 --> 00:01:55.640 align:middle line:84% And maybe we can even start with that. 00:01:55.640 --> 00:01:59.250 align:middle line:84% We don't need to prioritize that much. 00:01:59.250 --> 00:02:04.030 align:middle line:84% So there is a question of what you want to do with this, 00:02:04.030 --> 00:02:08.930 align:middle line:84% that is, how far you want to go on the learning curve. 00:02:08.930 --> 00:02:14.970 align:middle line:84% So I think you folks, the folks who have been in this class 00:02:14.970 --> 00:02:21.050 align:middle line:84% and have been doing the work now can build models on your own. 00:02:21.050 --> 00:02:23.290 align:middle line:84% But it is not easy, and you have experienced 00:02:23.290 --> 00:02:27.970 align:middle line:84% that, models that are useful for meaningful applications. 00:02:27.970 --> 00:02:33.410 align:middle line:84% So this is a point where shifting towards actually doing 00:02:33.410 --> 00:02:37.930 align:middle line:84% it for real problems that are close to your heart 00:02:37.930 --> 00:02:40.290 align:middle line:84% and you care about them makes a lot of sense 00:02:40.290 --> 00:02:43.970 align:middle line:84% because it will be hard, but it is not so frustrating that you 00:02:43.970 --> 00:02:47.897 align:middle line:84% would give up after a couple of attempts because you see, 00:02:47.897 --> 00:02:49.230 align:middle line:90% OK, there is no way I can do it. 00:02:49.230 --> 00:02:51.090 align:middle line:84% I think hopefully through this semester, 00:02:51.090 --> 00:02:54.850 align:middle line:90% you have reached that point. 00:02:54.850 --> 00:02:58.630 align:middle line:90% That's one general path. 00:02:58.630 --> 00:03:02.910 align:middle line:84% For more concentrated, formalized 00:03:02.910 --> 00:03:08.710 align:middle line:84% set of exercise type of things, we usually have our PhD 00:03:08.710 --> 00:03:11.797 align:middle line:84% students go through all of the challenges in business dynamics, 00:03:11.797 --> 00:03:13.630 align:middle line:84% at least the ones that are more challenging, 00:03:13.630 --> 00:03:16.310 align:middle line:84% for their preparation for their general exam. 00:03:16.310 --> 00:03:21.670 align:middle line:84% So that provides a set of good practice exercises. 00:03:21.670 --> 00:03:23.570 align:middle line:84% Many of them are also interesting problems, 00:03:23.570 --> 00:03:28.390 align:middle line:84% so you may find the topics are pretty interesting as well. 00:03:28.390 --> 00:03:31.230 align:middle line:84% We also have old general exams that our PhD 00:03:31.230 --> 00:03:32.370 align:middle line:90% students have gone through. 00:03:32.370 --> 00:03:35.950 align:middle line:84% That's another set of practice exercises. 00:03:35.950 --> 00:03:39.310 align:middle line:84% I would say the last couple of assignments 00:03:39.310 --> 00:03:42.005 align:middle line:84% that we had in the class, in terms of complexity, 00:03:42.005 --> 00:03:44.630 align:middle line:84% we're getting close to what you would have for the general exam 00:03:44.630 --> 00:03:47.990 align:middle line:84% questions, perhaps with the exception that general exam 00:03:47.990 --> 00:03:49.930 align:middle line:84% questions are a little bit more open-ended. 00:03:49.930 --> 00:03:54.450 align:middle line:84% So I kind of gave very clear set of constructs to use 00:03:54.450 --> 00:03:56.570 align:middle line:90% and mechanisms to capture. 00:03:56.570 --> 00:03:59.610 align:middle line:84% Those, you need to determine when you're doing a general exam 00:03:59.610 --> 00:04:00.650 align:middle line:90% question. 00:04:00.650 --> 00:04:03.030 align:middle line:84% So those would be a good next step as well. 00:04:03.030 --> 00:04:05.370 align:middle line:84% And if anybody is interested, we are 00:04:05.370 --> 00:04:09.710 align:middle line:84% happy to share those resources with anybody who's interested. 00:04:09.710 --> 00:04:12.410 align:middle line:90% 00:04:12.410 --> 00:04:15.130 align:middle line:84% There is kind of a whole different area that 00:04:15.130 --> 00:04:19.250 align:middle line:84% is connecting models with data, and we haven't really 00:04:19.250 --> 00:04:22.810 align:middle line:90% talked about that. 00:04:22.810 --> 00:04:26.170 align:middle line:84% There is a good chance the next version of the class that I'll 00:04:26.170 --> 00:04:32.410 align:middle line:84% be teaching in the fall, actually, will focus on that. 00:04:32.410 --> 00:04:35.870 align:middle line:84% In that space, I think for pragmatic applications, 00:04:35.870 --> 00:04:39.530 align:middle line:84% it is very important to bring data into models. 00:04:39.530 --> 00:04:42.610 align:middle line:84% It is both the calibration side, but also thinking 00:04:42.610 --> 00:04:46.130 align:middle line:84% about different ways data and models connect with each other, 00:04:46.130 --> 00:04:51.570 align:middle line:84% not just for fitting, but also for informing what structures 00:04:51.570 --> 00:04:53.710 align:middle line:90% to use and so on. 00:04:53.710 --> 00:04:59.490 align:middle line:84% We don't have good, clean text to teach all of that. 00:04:59.490 --> 00:05:04.910 align:middle line:84% It's a little bit more hands-on practice. 00:05:04.910 --> 00:05:07.470 align:middle line:84% Again, anybody interested, you're 00:05:07.470 --> 00:05:12.650 align:middle line:84% welcome to participate in the fall seminar. 00:05:12.650 --> 00:05:15.210 align:middle line:90% I'm happy to have visitors. 00:05:15.210 --> 00:05:18.230 align:middle line:90% 00:05:18.230 --> 00:05:21.170 align:middle line:84% Beyond that, it is really depending on your use cases. 00:05:21.170 --> 00:05:23.030 align:middle line:84% We are, I think, now advanced enough 00:05:23.030 --> 00:05:25.830 align:middle line:84% that there is no textbook that I would say, OK, 00:05:25.830 --> 00:05:27.950 align:middle line:84% go read that textbook that provides you 00:05:27.950 --> 00:05:31.470 align:middle line:90% with the next steps. 00:05:31.470 --> 00:05:35.750 align:middle line:84% Looking at good, old models, I mean 00:05:35.750 --> 00:05:38.910 align:middle line:84% other people's models and related topics is always 00:05:38.910 --> 00:05:40.770 align:middle line:90% very instructional. 00:05:40.770 --> 00:05:43.050 align:middle line:84% That's another good area to focus on. 00:05:43.050 --> 00:05:47.550 align:middle line:90% 00:05:47.550 --> 00:05:48.650 align:middle line:90% Other thoughts? 00:05:48.650 --> 00:06:00.140 align:middle line:90% 00:06:00.140 --> 00:06:05.820 align:middle line:90% OK, so let's open it up. 00:06:05.820 --> 00:06:08.640 align:middle line:84% Let me see if my Dropbox is happy. 00:06:08.640 --> 00:06:12.180 align:middle line:84% Well, I don't want to reboot, so let me get this thing out. 00:06:12.180 --> 00:06:14.940 align:middle line:90% 00:06:14.940 --> 00:06:21.820 align:middle line:84% And my Dropbox, finally, we have material 00:06:21.820 --> 00:06:22.960 align:middle line:90% in this session coming up. 00:06:22.960 --> 00:06:27.660 align:middle line:84% So right now we have four model submissions. 00:06:27.660 --> 00:06:30.240 align:middle line:84% If anybody else has theirs online, 00:06:30.240 --> 00:06:34.080 align:middle line:84% it might be that my Dropbox hasn't yet synced. 00:06:34.080 --> 00:06:36.560 align:middle line:90% So that's quite possible. 00:06:36.560 --> 00:06:40.420 align:middle line:90% 00:06:40.420 --> 00:06:46.300 align:middle line:84% Oh, my computer decided to restart. 00:06:46.300 --> 00:06:48.500 align:middle line:90% It didn't even ask me. 00:06:48.500 --> 00:06:52.600 align:middle line:84% So while this is restarting, anybody 00:06:52.600 --> 00:06:57.160 align:middle line:84% wants to bring up what they are thinking about, their project, 00:06:57.160 --> 00:07:00.020 align:middle line:84% and what they want to do with it next. 00:07:00.020 --> 00:07:02.130 align:middle line:84% And even if you haven't submitted something, 00:07:02.130 --> 00:07:03.380 align:middle line:90% we can have that conversation. 00:07:03.380 --> 00:07:04.838 align:middle line:84% If you haven't submitted something, 00:07:04.838 --> 00:07:07.060 align:middle line:84% it's easier to just talk about it. 00:07:07.060 --> 00:07:10.640 align:middle line:90% 00:07:10.640 --> 00:07:14.820 align:middle line:84% So I know a few are working on this and related research. 00:07:14.820 --> 00:07:17.320 align:middle line:84% So Maria, you are doing kind of research 00:07:17.320 --> 00:07:20.160 align:middle line:84% and using this as a little bit of a trying 00:07:20.160 --> 00:07:23.640 align:middle line:90% to get something in place. 00:07:23.640 --> 00:07:27.080 align:middle line:84% And you're also searching working on it 00:07:27.080 --> 00:07:31.280 align:middle line:84% as a research project in the longer term. 00:07:31.280 --> 00:07:36.040 align:middle line:84% So we had a look at your model last time. 00:07:36.040 --> 00:07:40.880 align:middle line:84% After that conversation, where do you see the project going? 00:07:40.880 --> 00:07:48.100 align:middle line:84% And what direction do you feel is most useful and viable? 00:07:48.100 --> 00:07:50.240 align:middle line:84% AUDIENCE: So this was really useful for me. 00:07:50.240 --> 00:07:53.100 align:middle line:84% And I learned how to identify the dominant feedback 00:07:53.100 --> 00:07:55.940 align:middle line:84% loops driving the behavior of the system. 00:07:55.940 --> 00:07:59.460 align:middle line:84% And then I tried to make some sensitivity analysis 00:07:59.460 --> 00:08:02.340 align:middle line:90% that I haven't completed yet. 00:08:02.340 --> 00:08:07.920 align:middle line:84% But my main problem is with data and how to calibrate it. 00:08:07.920 --> 00:08:12.180 align:middle line:90% So maybe I can get more into it. 00:08:12.180 --> 00:08:13.820 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 00:08:13.820 --> 00:08:18.360 align:middle line:84% So definitely, for serious empirical work, 00:08:18.360 --> 00:08:20.860 align:middle line:84% you definitely need the connection with data. 00:08:20.860 --> 00:08:22.360 align:middle line:84% These days, we have a lot more data. 00:08:22.360 --> 00:08:24.100 align:middle line:90% So that's the great news. 00:08:24.100 --> 00:08:26.740 align:middle line:84% The challenge still remains in terms 00:08:26.740 --> 00:08:30.420 align:middle line:84% of what are the best methods for estimating these models. 00:08:30.420 --> 00:08:31.600 align:middle line:90% It's a messy space. 00:08:31.600 --> 00:08:35.700 align:middle line:84% There is no easy template that just follow these steps 00:08:35.700 --> 00:08:36.480 align:middle line:90% and you are done. 00:08:36.480 --> 00:08:41.620 align:middle line:84% It really depends on complexity of the model, how much process 00:08:41.620 --> 00:08:45.100 align:middle line:84% noise we have, which, that is how much of the dynamics 00:08:45.100 --> 00:08:47.065 align:middle line:84% are determined by things inside the model 00:08:47.065 --> 00:08:49.440 align:middle line:84% and how much are determined by things that are completely 00:08:49.440 --> 00:08:53.280 align:middle line:84% outside of the boundary, and you can't observe them that well. 00:08:53.280 --> 00:08:58.240 align:middle line:84% So that makes it harder to estimate these models. 00:08:58.240 --> 00:09:03.200 align:middle line:84% But I think if I do the version that 00:09:03.200 --> 00:09:06.560 align:middle line:84% is on estimation in the fall, this might be useful for you 00:09:06.560 --> 00:09:09.520 align:middle line:90% as well, if you're interested. 00:09:09.520 --> 00:09:10.720 align:middle line:90% Good. 00:09:10.720 --> 00:09:17.600 align:middle line:84% Maria, what are your thoughts on what's next on this project? 00:09:17.600 --> 00:09:19.000 align:middle line:84% AUDIENCE: I think what's next is, 00:09:19.000 --> 00:09:24.640 align:middle line:84% I need to figure out the correct formulations for the existing, 00:09:24.640 --> 00:09:28.840 align:middle line:84% and then all the mechanisms I was thinking about. 00:09:28.840 --> 00:09:34.160 align:middle line:84% And I need to get the data, documentation. 00:09:34.160 --> 00:09:36.800 align:middle line:90% 00:09:36.800 --> 00:09:38.720 align:middle line:84% HAZHIR RAHMANDAD: So you feel a little bit 00:09:38.720 --> 00:09:44.403 align:middle line:84% overwhelmed with the complexity of the work that 00:09:44.403 --> 00:09:45.070 align:middle line:90% is ahead of you? 00:09:45.070 --> 00:09:48.790 align:middle line:90% AUDIENCE: No, not overwhelmed. 00:09:48.790 --> 00:09:51.190 align:middle line:84% It's just that there are different parts that 00:09:51.190 --> 00:09:52.970 align:middle line:90% have their own complexities. 00:09:52.970 --> 00:09:57.208 align:middle line:84% HAZHIR RAHMANDAD: Yes, absolutely. 00:09:57.208 --> 00:09:58.750 align:middle line:84% OK, on the modeling side, we have had 00:09:58.750 --> 00:10:00.270 align:middle line:90% a couple of back and forths. 00:10:00.270 --> 00:10:02.870 align:middle line:84% So you have a little bit of at least 00:10:02.870 --> 00:10:09.250 align:middle line:84% what the underlying stock and flow structure should be. 00:10:09.250 --> 00:10:15.350 align:middle line:84% One generally useful idea when connecting models to data is to, 00:10:15.350 --> 00:10:20.350 align:middle line:84% first of all, see if you can match 00:10:20.350 --> 00:10:22.510 align:middle line:84% the aggregation level of the data 00:10:22.510 --> 00:10:24.630 align:middle line:90% that you have in your model. 00:10:24.630 --> 00:10:27.410 align:middle line:84% Not always, but sometimes that is a good heuristic. 00:10:27.410 --> 00:10:32.590 align:middle line:84% So if you have data on five different age groups, 00:10:32.590 --> 00:10:35.950 align:middle line:84% it might be a good idea to break down your model 00:10:35.950 --> 00:10:37.910 align:middle line:90% into five different age groups. 00:10:37.910 --> 00:10:40.870 align:middle line:84% Now, if you have continuous age and continuous and 00:10:40.870 --> 00:10:42.830 align:middle line:84% individual-level data, for example, 00:10:42.830 --> 00:10:46.650 align:middle line:84% it's not worth disaggregating to that level. 00:10:46.650 --> 00:10:49.730 align:middle line:84% Sometimes it is, but most often it is not. 00:10:49.730 --> 00:10:54.390 align:middle line:84% But by getting the same level of aggregation in the model, 00:10:54.390 --> 00:10:57.190 align:middle line:84% in the kind of construct level to what you have in your data, 00:10:57.190 --> 00:11:00.690 align:middle line:84% you are minimizing data loss, basically, 00:11:00.690 --> 00:11:07.410 align:middle line:84% and you can capture the measurement process of the data 00:11:07.410 --> 00:11:08.710 align:middle line:90% as closely as possible. 00:11:08.710 --> 00:11:09.930 align:middle line:90% That's also key. 00:11:09.930 --> 00:11:13.250 align:middle line:84% The data generating process should be in your model. 00:11:13.250 --> 00:11:16.070 align:middle line:84% So if there are biases in the measurement process, 00:11:16.070 --> 00:11:18.530 align:middle line:84% if there are delays in the measurement process, 00:11:18.530 --> 00:11:20.970 align:middle line:84% you want to capture those in your model 00:11:20.970 --> 00:11:24.090 align:middle line:84% when you are creating variables that correspond 00:11:24.090 --> 00:11:25.470 align:middle line:90% to the real-world data. 00:11:25.470 --> 00:11:29.270 align:middle line:84% So always think about, there is a real-world process, 00:11:29.270 --> 00:11:31.330 align:middle line:84% and then there is a data-generating process 00:11:31.330 --> 00:11:33.850 align:middle line:84% that sits on top of that real-world process that 00:11:33.850 --> 00:11:35.570 align:middle line:90% generates this data. 00:11:35.570 --> 00:11:39.330 align:middle line:84% And then that data-generating process gives you 00:11:39.330 --> 00:11:42.830 align:middle line:84% the observables, gives you the data 00:11:42.830 --> 00:11:45.510 align:middle line:84% that you should compare against the simulated values 00:11:45.510 --> 00:11:46.950 align:middle line:90% for the same thing. 00:11:46.950 --> 00:11:50.290 align:middle line:84% So if you're doing epidemic modeling, 00:11:50.290 --> 00:11:53.750 align:middle line:84% there is a data-generating process. 00:11:53.750 --> 00:11:56.910 align:middle line:84% There is a process for the dynamics 00:11:56.910 --> 00:11:59.510 align:middle line:84% of the system-- people are getting sick and dying and going 00:11:59.510 --> 00:12:00.650 align:middle line:90% to hospitals and so on. 00:12:00.650 --> 00:12:02.830 align:middle line:84% And then there is a measurement process, that is, 00:12:02.830 --> 00:12:07.550 align:middle line:84% what's the delay for measuring when people got sick 00:12:07.550 --> 00:12:11.110 align:middle line:84% and what's the delay for reporting of deaths. 00:12:11.110 --> 00:12:13.730 align:middle line:84% So some countries, you look at their data, 00:12:13.730 --> 00:12:17.110 align:middle line:84% they had these oscillatory behaviors 00:12:17.110 --> 00:12:21.910 align:middle line:84% because they went with weekly cycles 00:12:21.910 --> 00:12:25.050 align:middle line:90% in their tracking of the cases. 00:12:25.050 --> 00:12:27.290 align:middle line:84% So if you wanted to match that data correctly, 00:12:27.290 --> 00:12:30.270 align:middle line:84% you needed to include these weekly cycles of measurement 00:12:30.270 --> 00:12:35.670 align:middle line:84% and, of course, the delays in it to do a decent job. 00:12:35.670 --> 00:12:38.730 align:middle line:84% Now, for some applications, that's more important, for some, 00:12:38.730 --> 00:12:43.490 align:middle line:84% less, but it generally is pretty important to have that. 00:12:43.490 --> 00:12:47.510 align:middle line:84% My wife works in a product management space, 00:12:47.510 --> 00:12:52.930 align:middle line:84% and she's dealing with data that is about registrations and sales 00:12:52.930 --> 00:12:56.670 align:middle line:90% of coffee making machines. 00:12:56.670 --> 00:12:59.650 align:middle line:90% So there are hundreds of models. 00:12:59.650 --> 00:13:01.710 align:middle line:90% There are these numbers. 00:13:01.710 --> 00:13:04.250 align:middle line:84% And when you think about the numbers 00:13:04.250 --> 00:13:07.490 align:middle line:84% and you're trying to come up with KPIs for how the company is 00:13:07.490 --> 00:13:09.470 align:middle line:84% doing, whether the sales is increasing or not, 00:13:09.470 --> 00:13:13.170 align:middle line:84% registrations are fraction of sales that gets registered going 00:13:13.170 --> 00:13:15.730 align:middle line:84% up or down, you need to actually account 00:13:15.730 --> 00:13:18.530 align:middle line:84% for the delays between the time somebody 00:13:18.530 --> 00:13:21.470 align:middle line:84% buys something until they register it, 00:13:21.470 --> 00:13:24.490 align:middle line:84% because that process takes actually 00:13:24.490 --> 00:13:27.090 align:middle line:90% more than a month on average. 00:13:27.090 --> 00:13:29.090 align:middle line:84% And you need to account for the fact 00:13:29.090 --> 00:13:31.050 align:middle line:84% that some people return their product 00:13:31.050 --> 00:13:33.650 align:middle line:84% and therefore sometimes you may have negative sales. 00:13:33.650 --> 00:13:38.880 align:middle line:84% If you don't do these, then the data doesn't make sense. 00:13:38.880 --> 00:13:42.580 align:middle line:84% And to do those, actually having a model can help. 00:13:42.580 --> 00:13:45.400 align:middle line:84% Having a simple SD model with the stocks and flows 00:13:45.400 --> 00:13:49.220 align:middle line:84% without much feedback can be pretty useful. 00:13:49.220 --> 00:13:53.213 align:middle line:84% So replicating the data-generating process. 00:13:53.213 --> 00:13:54.880 align:middle line:84% And that's one of the low-hanging fruits 00:13:54.880 --> 00:13:59.240 align:middle line:84% for using what you have learned in this course 00:13:59.240 --> 00:14:01.260 align:middle line:84% without building a very complex model, 00:14:01.260 --> 00:14:07.000 align:middle line:84% just capturing mechanically how the data is being generated 00:14:07.000 --> 00:14:09.640 align:middle line:84% that you're using in whatever data analysis you do, 00:14:09.640 --> 00:14:14.280 align:middle line:84% and then creating that filter for making sense and inferring 00:14:14.280 --> 00:14:18.500 align:middle line:84% the underlying values that you may care about even more. 00:14:18.500 --> 00:14:24.580 align:middle line:84% You really care about sales or customer happiness or whatnot, 00:14:24.580 --> 00:14:29.320 align:middle line:84% and that is better captured by back 00:14:29.320 --> 00:14:35.700 align:middle line:84% tracing from the reported data to the underlying mechanism. 00:14:35.700 --> 00:14:37.740 align:middle line:90% Good. 00:14:37.740 --> 00:14:39.700 align:middle line:90% AUDIENCE: So I have a question. 00:14:39.700 --> 00:14:44.620 align:middle line:84% What is your advice on how to capture them 00:14:44.620 --> 00:14:51.340 align:middle line:84% by mechanisms in terms of people moving between at-risk and not 00:14:51.340 --> 00:14:55.060 align:middle line:84% at-risk, and then the air conditioning effect? 00:14:55.060 --> 00:14:58.183 align:middle line:90% I'm not sure where to start. 00:14:58.183 --> 00:14:59.600 align:middle line:84% HAZHIR RAHMANDAD: Yes, good point. 00:14:59.600 --> 00:15:02.220 align:middle line:84% So basically in Maria's model, there 00:15:02.220 --> 00:15:06.420 align:middle line:84% are people who are at risk for heat incidents 00:15:06.420 --> 00:15:10.780 align:middle line:84% because of not having access to air conditioning and people who 00:15:10.780 --> 00:15:11.780 align:middle line:90% are at lower risk. 00:15:11.780 --> 00:15:13.960 align:middle line:84% Basically, this is largely based on age, 00:15:13.960 --> 00:15:16.380 align:middle line:84% if I understand correctly, and to some extent, based 00:15:16.380 --> 00:15:19.940 align:middle line:90% on other comorbidities. 00:15:19.940 --> 00:15:22.680 align:middle line:90% So that's the risk level. 00:15:22.680 --> 00:15:25.280 align:middle line:84% And then there is whether they have access to AC or not, 00:15:25.280 --> 00:15:28.900 align:middle line:84% basically, their house is cooled down or not. 00:15:28.900 --> 00:15:30.280 align:middle line:90% That's another dimension. 00:15:30.280 --> 00:15:32.940 align:middle line:84% So in the simplest case, you can think about two levels-- 00:15:32.940 --> 00:15:37.400 align:middle line:84% low risk and high risk and then AC access, no AC access. 00:15:37.400 --> 00:15:40.080 align:middle line:84% So you would have four stocks for people 00:15:40.080 --> 00:15:41.660 align:middle line:90% in each of these conditions. 00:15:41.660 --> 00:15:45.900 align:middle line:84% Then you would have transitions either going from left to right, 00:15:45.900 --> 00:15:49.240 align:middle line:84% from low risk to high risk, and from top 00:15:49.240 --> 00:15:53.640 align:middle line:84% to bottom, from no access to with access. 00:15:53.640 --> 00:15:59.460 align:middle line:84% So low risk to high risk, the most basic driver is aging. 00:15:59.460 --> 00:16:02.680 align:middle line:84% So if you have a population, you can even 00:16:02.680 --> 00:16:05.680 align:middle line:84% think of it as an aging chain with, OK, these 00:16:05.680 --> 00:16:10.920 align:middle line:84% are people under 65 and these are people over 65. 00:16:10.920 --> 00:16:13.820 align:middle line:84% And you're just modeling the transition between those two 00:16:13.820 --> 00:16:16.760 align:middle line:84% if that is your primary risk factor. 00:16:16.760 --> 00:16:18.600 align:middle line:84% Then, if you're thinking, no, actually 00:16:18.600 --> 00:16:20.760 align:middle line:84% there is another risk factor, you 00:16:20.760 --> 00:16:23.700 align:middle line:84% can capture it in two different ways. 00:16:23.700 --> 00:16:26.360 align:middle line:84% One is to think of these population groups, these two 00:16:26.360 --> 00:16:30.800 align:middle line:84% population groups, as combining age and the other risk factor. 00:16:30.800 --> 00:16:34.030 align:middle line:84% And then the transition would be, 00:16:34.030 --> 00:16:37.990 align:middle line:84% how much people are moving based on the age change, but also 00:16:37.990 --> 00:16:41.390 align:middle line:84% adjustment up or down based on the other risk factors 00:16:41.390 --> 00:16:43.830 align:middle line:90% that these people may have. 00:16:43.830 --> 00:16:47.193 align:middle line:84% AUDIENCE: Also the youngest, the youngest are also part of that. 00:16:47.193 --> 00:16:48.610 align:middle line:84% HAZHIR RAHMANDAD: Very good point. 00:16:48.610 --> 00:16:52.050 align:middle line:84% So then, you can do it in two different ways again. 00:16:52.050 --> 00:16:55.990 align:middle line:84% You could have it as three stocks for the youngest, 00:16:55.990 --> 00:16:59.170 align:middle line:90% low risk, and the elderly. 00:16:59.170 --> 00:17:03.810 align:middle line:84% That's kind of the first and third stock will be higher risk. 00:17:03.810 --> 00:17:06.369 align:middle line:84% And the middle one would be the lower risk. 00:17:06.369 --> 00:17:08.550 align:middle line:84% I would probably go with that rather than 00:17:08.550 --> 00:17:13.190 align:middle line:84% trying to combine the elderly and the very young 00:17:13.190 --> 00:17:15.170 align:middle line:84% into a single stock of high risk. 00:17:15.170 --> 00:17:19.109 align:middle line:84% The reason I feel a little bit hesitant about doing the latter 00:17:19.109 --> 00:17:22.510 align:middle line:84% is that capturing the flows between low and high risk 00:17:22.510 --> 00:17:24.430 align:middle line:90% becomes harder. 00:17:24.430 --> 00:17:28.670 align:middle line:84% So I would go with a clean age-based grouping. 00:17:28.670 --> 00:17:32.330 align:middle line:84% And you can have actually different risk levels 00:17:32.330 --> 00:17:33.870 align:middle line:90% for each age group. 00:17:33.870 --> 00:17:41.387 align:middle line:84% That is, 10% of the middle-aged folks may also be at risk, 00:17:41.387 --> 00:17:42.470 align:middle line:90% and that's perfectly fine. 00:17:42.470 --> 00:17:47.810 align:middle line:84% So you can calculate the total population at risk using that. 00:17:47.810 --> 00:17:52.170 align:middle line:84% Then, you use age to drive the risk profile. 00:17:52.170 --> 00:17:55.010 align:middle line:84% And then you will have basically changes 00:17:55.010 --> 00:18:08.270 align:middle line:84% in AC in housing as the driver of the other part. 00:18:08.270 --> 00:18:11.530 align:middle line:90% 00:18:11.530 --> 00:18:15.690 align:middle line:90% So does that help? 00:18:15.690 --> 00:18:16.570 align:middle line:90% AUDIENCE: Yeah. 00:18:16.570 --> 00:18:18.270 align:middle line:90% I need to think about it more. 00:18:18.270 --> 00:18:23.410 align:middle line:84% And that's on the one hand, in terms of the risk, 00:18:23.410 --> 00:18:27.250 align:middle line:84% I need to think also how to formulate. 00:18:27.250 --> 00:18:31.790 align:middle line:84% Because the idea in this model is to capture also the fixes 00:18:31.790 --> 00:18:33.350 align:middle line:90% that fail effect. 00:18:33.350 --> 00:18:41.310 align:middle line:84% So the AC adoption and then how that reduces the risk. 00:18:41.310 --> 00:18:45.110 align:middle line:84% And then the excess heat from AC. 00:18:45.110 --> 00:18:51.550 align:middle line:84% So on one hand, capturing excess heat in terms of climate change 00:18:51.550 --> 00:18:54.390 align:middle line:84% and then excess heat from using aircon. 00:18:54.390 --> 00:18:56.170 align:middle line:84% There's a lot of different parts. 00:18:56.170 --> 00:18:57.420 align:middle line:90% HAZHIR RAHMANDAD: Interesting. 00:18:57.420 --> 00:19:00.750 align:middle line:84% AUDIENCE: But I'm not sure if I want to disaggregate that 00:19:00.750 --> 00:19:02.853 align:middle line:90% much the stock slightly. 00:19:02.853 --> 00:19:04.270 align:middle line:84% HAZHIR RAHMANDAD: So it's actually 00:19:04.270 --> 00:19:07.270 align:middle line:84% an interesting general question many of you 00:19:07.270 --> 00:19:12.110 align:middle line:84% will deal with-- is desegregation helping or hurting 00:19:12.110 --> 00:19:13.170 align:middle line:90% in many ways? 00:19:13.170 --> 00:19:17.430 align:middle line:84% So for precision, it usually should help. 00:19:17.430 --> 00:19:21.750 align:middle line:84% But for building a model in terms of conceptual model 00:19:21.750 --> 00:19:25.230 align:middle line:84% building, disaggregation is not always clear. 00:19:25.230 --> 00:19:28.150 align:middle line:84% Early on, it might be actually easier to disaggregate. 00:19:28.150 --> 00:19:31.330 align:middle line:84% That is, coming up with good ways of aggregation 00:19:31.330 --> 00:19:34.730 align:middle line:90% is not obvious. 00:19:34.730 --> 00:19:38.650 align:middle line:84% You can make cleaner models by disaggregating and then 00:19:38.650 --> 00:19:45.970 align:middle line:84% following, for example, the age groups or the categories of AC 00:19:45.970 --> 00:19:48.110 align:middle line:90% availability access. 00:19:48.110 --> 00:19:51.970 align:middle line:84% If you think different categories of that matters, 00:19:51.970 --> 00:19:52.910 align:middle line:90% go for it. 00:19:52.910 --> 00:19:57.090 align:middle line:84% So don't be too afraid of disaggregating a little bit 00:19:57.090 --> 00:19:57.590 align:middle line:90% more. 00:19:57.590 --> 00:20:00.150 align:middle line:84% Because then the formulations come more naturally, 00:20:00.150 --> 00:20:02.530 align:middle line:84% because you're not combining multiple processes 00:20:02.530 --> 00:20:06.630 align:middle line:84% into a single transition from one stock to another. 00:20:06.630 --> 00:20:09.710 align:middle line:84% You're just capturing one very well-defined mechanism. 00:20:09.710 --> 00:20:11.150 align:middle line:84% So aging is very straightforward. 00:20:11.150 --> 00:20:12.430 align:middle line:90% You just have a time constant. 00:20:12.430 --> 00:20:16.210 align:middle line:84% You're just having first-order movement of people 00:20:16.210 --> 00:20:17.930 align:middle line:90% across different stocks. 00:20:17.930 --> 00:20:21.550 align:middle line:84% It's much easier to do it that way at least early on. 00:20:21.550 --> 00:20:24.150 align:middle line:84% Now, once you are a very experienced modeler, 00:20:24.150 --> 00:20:26.520 align:middle line:84% you may feel comfortable combining them 00:20:26.520 --> 00:20:29.320 align:middle line:84% and you say, OK, I think this captures 00:20:29.320 --> 00:20:33.100 align:middle line:84% the core idea with a lot less complexity, 00:20:33.100 --> 00:20:35.220 align:middle line:90% so it enhances communication. 00:20:35.220 --> 00:20:39.380 align:middle line:84% I can get early results faster for academic journals. 00:20:39.380 --> 00:20:42.040 align:middle line:84% That is much more attractive because the reviewers 00:20:42.040 --> 00:20:43.960 align:middle line:90% can follow what is happening. 00:20:43.960 --> 00:20:47.060 align:middle line:84% You can tell a story more easily with a simple model. 00:20:47.060 --> 00:20:49.280 align:middle line:84% So there are a lot of benefits from aggregating 00:20:49.280 --> 00:20:51.200 align:middle line:90% and having a simpler model. 00:20:51.200 --> 00:20:54.920 align:middle line:84% But early on, it is not easy to do that if you haven't 00:20:54.920 --> 00:20:56.540 align:middle line:90% had that much experience. 00:20:56.540 --> 00:21:00.956 align:middle line:90% 00:21:00.956 --> 00:21:03.480 align:middle line:90% Finally, the computer is back. 00:21:03.480 --> 00:21:05.500 align:middle line:90% So let's see. 00:21:05.500 --> 00:21:08.900 align:middle line:84% We can start with the models we have here. 00:21:08.900 --> 00:21:13.260 align:middle line:84% So Gerardo, we haven't, I think, looked at your model as much. 00:21:13.260 --> 00:21:15.400 align:middle line:90% So let's open that one. 00:21:15.400 --> 00:21:16.840 align:middle line:90% AUDIENCE: Yes. 00:21:16.840 --> 00:21:19.980 align:middle line:84% Actually, it's really a work in progress. 00:21:19.980 --> 00:21:26.220 align:middle line:84% I tried to start the process of designing the model, 00:21:26.220 --> 00:21:27.320 align:middle line:90% but it's not complete. 00:21:27.320 --> 00:21:30.110 align:middle line:84% So I haven't had so much time this week. 00:21:30.110 --> 00:21:31.360 align:middle line:90% HAZHIR RAHMANDAD: Sounds good. 00:21:31.360 --> 00:21:34.340 align:middle line:84% So actually, maybe you can walk us 00:21:34.340 --> 00:21:36.540 align:middle line:84% through what you're trying to do and how you're 00:21:36.540 --> 00:21:37.802 align:middle line:90% trying to do it in the model. 00:21:37.802 --> 00:21:39.260 align:middle line:84% So first, what you're trying to do, 00:21:39.260 --> 00:21:41.340 align:middle line:84% big picture, and then we can look. 00:21:41.340 --> 00:21:43.060 align:middle line:84% AUDIENCE: OK, big picture, I'm trying 00:21:43.060 --> 00:21:49.100 align:middle line:84% to capture the mechanism of how startups 00:21:49.100 --> 00:21:55.340 align:middle line:84% have an impact on environmental problem. 00:21:55.340 --> 00:22:03.020 align:middle line:84% And with the last problem definition, 00:22:03.020 --> 00:22:05.980 align:middle line:84% thanks to the feedback that I received, 00:22:05.980 --> 00:22:08.860 align:middle line:84% I modelled this with three sources. 00:22:08.860 --> 00:22:11.820 align:middle line:90% One is the startups. 00:22:11.820 --> 00:22:13.960 align:middle line:90% There are three types of stocks. 00:22:13.960 --> 00:22:20.540 align:middle line:84% One is startup that became firms with scale-up rate. 00:22:20.540 --> 00:22:25.960 align:middle line:84% The green startups now have an impact on the economic capital. 00:22:25.960 --> 00:22:33.280 align:middle line:84% So the startups have an average economic capital contribution. 00:22:33.280 --> 00:22:40.220 align:middle line:84% And when the startup scales up, this contribution grows a lot. 00:22:40.220 --> 00:22:44.320 align:middle line:84% And so we have contribution on the field with the other stuff. 00:22:44.320 --> 00:22:48.280 align:middle line:84% And at the bottom, we have the other stocks 00:22:48.280 --> 00:22:53.640 align:middle line:84% related to the energy consumption of the firm 00:22:53.640 --> 00:22:55.200 align:middle line:90% and of the startup. 00:22:55.200 --> 00:22:58.760 align:middle line:84% And the way that I am trying to think about this problem 00:22:58.760 --> 00:23:04.600 align:middle line:84% is really close to the coflow of the building efficiency 00:23:04.600 --> 00:23:09.600 align:middle line:84% that we designed in the other class. 00:23:09.600 --> 00:23:12.200 align:middle line:84% And so this is really helpful for me 00:23:12.200 --> 00:23:13.960 align:middle line:90% to really match these two. 00:23:13.960 --> 00:23:19.700 align:middle line:90% 00:23:19.700 --> 00:23:21.860 align:middle line:84% HAZHIR RAHMANDAD: So just as we are 00:23:21.860 --> 00:23:25.820 align:middle line:84% looking at this, for example, green startup energy consumption 00:23:25.820 --> 00:23:31.180 align:middle line:84% needs to have a coflow outflow that represents 00:23:31.180 --> 00:23:33.460 align:middle line:90% the green startup death rate. 00:23:33.460 --> 00:23:34.460 align:middle line:90% AUDIENCE: Yes, yes, yes. 00:23:34.460 --> 00:23:36.418 align:middle line:84% HAZHIR RAHMANDAD: Yeah, just keep that in mind. 00:23:36.418 --> 00:23:37.060 align:middle line:90% OK. 00:23:37.060 --> 00:23:39.692 align:middle line:90% AUDIENCE: Yes. 00:23:39.692 --> 00:23:41.400 align:middle line:84% HAZHIR RAHMANDAD: Basically, for coflows, 00:23:41.400 --> 00:23:43.420 align:middle line:90% this is generally the case. 00:23:43.420 --> 00:23:46.260 align:middle line:84% For every flow for the main stock, 00:23:46.260 --> 00:23:49.860 align:middle line:84% you need a flow for the coflow that represents that. 00:23:49.860 --> 00:23:55.620 align:middle line:84% And most of the time until you know how to do it differently, 00:23:55.620 --> 00:24:02.180 align:middle line:84% the amounts that you take out of those using for those outflows 00:24:02.180 --> 00:24:06.500 align:middle line:84% should be the average amount of the material that 00:24:06.500 --> 00:24:10.140 align:middle line:84% is available per unit of the stock. 00:24:10.140 --> 00:24:13.380 align:middle line:84% So that is this average energy intensity 00:24:13.380 --> 00:24:15.100 align:middle line:90% is what should be coming out. 00:24:15.100 --> 00:24:18.590 align:middle line:84% Those two are the basics for coflows. 00:24:18.590 --> 00:24:21.770 align:middle line:84% Now, the second one, you could violate under some conditions 00:24:21.770 --> 00:24:27.170 align:middle line:84% if you are willing to make some assumptions. 00:24:27.170 --> 00:24:30.417 align:middle line:90% But, good. 00:24:30.417 --> 00:24:32.250 align:middle line:84% AUDIENCE: What I'm trying to do is basically 00:24:32.250 --> 00:24:34.730 align:middle line:84% replicate the same mechanism for green startup 00:24:34.730 --> 00:24:36.970 align:middle line:90% and brown startup in firm. 00:24:36.970 --> 00:24:39.790 align:middle line:84% I can do this also with subscripts, I think. 00:24:39.790 --> 00:24:46.090 align:middle line:84% But at first, I try to draw these stocks just 00:24:46.090 --> 00:24:52.010 align:middle line:84% to better see what is happening in broader terms. 00:24:52.010 --> 00:24:55.770 align:middle line:84% And the concept here is that a green startup 00:24:55.770 --> 00:25:03.090 align:middle line:84% is a startup that is defined as a startup that is particularly 00:25:03.090 --> 00:25:08.850 align:middle line:84% efficient or is working on technology that is particularly 00:25:08.850 --> 00:25:10.450 align:middle line:90% good for the environment. 00:25:10.450 --> 00:25:13.990 align:middle line:84% And brown startups are all the other types of startups, 00:25:13.990 --> 00:25:18.070 align:middle line:84% for example, that are set up in generative AI or all 00:25:18.070 --> 00:25:22.870 align:middle line:84% the other sectors that are not a particular benefit 00:25:22.870 --> 00:25:26.230 align:middle line:90% for the environment. 00:25:26.230 --> 00:25:27.510 align:middle line:90% HAZHIR RAHMANDAD: Got it. 00:25:27.510 --> 00:25:30.150 align:middle line:84% AUDIENCE: And so the idea is to capture 00:25:30.150 --> 00:25:36.110 align:middle line:84% the total emission generated by the growth of the two 00:25:36.110 --> 00:25:37.910 align:middle line:90% type of startups. 00:25:37.910 --> 00:25:46.190 align:middle line:84% And my idea is to try to figure out in which condition 00:25:46.190 --> 00:25:51.590 align:middle line:84% the total emissions of the ecosystem would go down. 00:25:51.590 --> 00:25:53.390 align:middle line:90% HAZHIR RAHMANDAD: Yeah, OK. 00:25:53.390 --> 00:25:54.410 align:middle line:90% All of that makes sense. 00:25:54.410 --> 00:25:57.190 align:middle line:84% I think the basic structure in terms of stocks and flows 00:25:57.190 --> 00:26:00.470 align:middle line:90% is solid. 00:26:00.470 --> 00:26:05.270 align:middle line:84% Whether you need the economic capital or not 00:26:05.270 --> 00:26:07.610 align:middle line:90% is a little bit unclear to me. 00:26:07.610 --> 00:26:10.250 align:middle line:84% Just kind of looking at this stock and flow, 00:26:10.250 --> 00:26:15.610 align:middle line:84% it feels to me that right now you 00:26:15.610 --> 00:26:18.170 align:middle line:84% would have an average economic capital 00:26:18.170 --> 00:26:21.490 align:middle line:90% per firm versus per startup. 00:26:21.490 --> 00:26:23.570 align:middle line:84% These two will be different, but that's basically 00:26:23.570 --> 00:26:25.010 align:middle line:90% showing their size. 00:26:25.010 --> 00:26:28.970 align:middle line:84% And the change in that size, that 00:26:28.970 --> 00:26:33.810 align:middle line:84% is for some reason the green firms might be growing bigger 00:26:33.810 --> 00:26:37.730 align:middle line:84% over time, is not the focus of your analysis. 00:26:37.730 --> 00:26:40.030 align:middle line:84% So if that is not the focus of your analysis, 00:26:40.030 --> 00:26:42.210 align:middle line:84% you could even treat those as parameters 00:26:42.210 --> 00:26:46.570 align:middle line:84% and keep your model simpler until you need to change it, 00:26:46.570 --> 00:26:50.470 align:middle line:84% until you decide to incorporate mechanisms that are showing, 00:26:50.470 --> 00:26:54.110 align:middle line:84% for example, green firms are becoming bigger over time 00:26:54.110 --> 00:26:57.410 align:middle line:84% gradually, or something like that. 00:26:57.410 --> 00:27:00.170 align:middle line:90% But it's OK what you have here. 00:27:00.170 --> 00:27:03.970 align:middle line:84% It's just more complexity than you necessarily need. 00:27:03.970 --> 00:27:06.410 align:middle line:84% And then I think at the heart of your problem 00:27:06.410 --> 00:27:10.130 align:middle line:84% is that these green firms and perhaps green startups, 00:27:10.130 --> 00:27:15.060 align:middle line:84% some of them contribute to brown startups' energy consumption. 00:27:15.060 --> 00:27:18.440 align:middle line:84% Basically, some of them are in the business 00:27:18.440 --> 00:27:24.580 align:middle line:84% of enhancing the energy efficiency of other firms. 00:27:24.580 --> 00:27:27.920 align:middle line:84% So these could be consulting operations or other types 00:27:27.920 --> 00:27:30.720 align:middle line:84% of technology firms that are producing new technologies 00:27:30.720 --> 00:27:32.440 align:middle line:90% to reduce the emissions. 00:27:32.440 --> 00:27:36.420 align:middle line:84% So as such, that is where the two chains get linked together. 00:27:36.420 --> 00:27:39.060 align:middle line:84% And that's what a lot of the interesting dynamics come from. 00:27:39.060 --> 00:27:41.680 align:middle line:84% So you definitely need to highlight that. 00:27:41.680 --> 00:27:47.220 align:middle line:84% And then going with the subscripts, I like that as well. 00:27:47.220 --> 00:27:51.200 align:middle line:84% But what you have here is a little bit easier visually. 00:27:51.200 --> 00:27:53.860 align:middle line:84% The benefit of going with subscripts is twofold. 00:27:53.860 --> 00:27:57.300 align:middle line:84% One is that you can have, then, more than just two categories. 00:27:57.300 --> 00:28:02.080 align:middle line:84% You can have green firms that are not 00:28:02.080 --> 00:28:05.060 align:middle line:84% in the business of reducing energy efficiency of others 00:28:05.060 --> 00:28:10.120 align:middle line:84% but themselves are much greener at the beginning 00:28:10.120 --> 00:28:12.960 align:middle line:84% or they're in low-polluting industries. 00:28:12.960 --> 00:28:16.820 align:middle line:84% So you can distinguish based on industries rather than these two 00:28:16.820 --> 00:28:17.680 align:middle line:90% big categories. 00:28:17.680 --> 00:28:21.940 align:middle line:84% So you can easily extend the model that way. 00:28:21.940 --> 00:28:24.980 align:middle line:84% And there are reasons potentially to do that. 00:28:24.980 --> 00:28:29.780 align:middle line:84% And then there is the parallelism 00:28:29.780 --> 00:28:33.300 align:middle line:84% in formulations that gets enforced much more 00:28:33.300 --> 00:28:35.280 align:middle line:84% cleanly when you have subscripts. 00:28:35.280 --> 00:28:38.420 align:middle line:84% So that's a little bit more making 00:28:38.420 --> 00:28:41.480 align:middle line:84% sure your model is consistent across the two different sectors 00:28:41.480 --> 00:28:44.300 align:middle line:84% and always trying to come up with general formulations that 00:28:44.300 --> 00:28:45.820 align:middle line:90% allow for that. 00:28:45.820 --> 00:28:49.660 align:middle line:84% One other part of your story that you 00:28:49.660 --> 00:28:54.740 align:middle line:84% may need to think how exactly it will work 00:28:54.740 --> 00:29:03.940 align:middle line:84% is the energy consumption per unit of the startup 00:29:03.940 --> 00:29:09.640 align:middle line:84% or brown firm, so per unit of activity of each of these firms. 00:29:09.640 --> 00:29:13.040 align:middle line:84% That energy consumption per unit of activity 00:29:13.040 --> 00:29:15.840 align:middle line:84% itself is changing over time, partly 00:29:15.840 --> 00:29:20.980 align:middle line:84% because of the green startups creating new technologies. 00:29:20.980 --> 00:29:27.520 align:middle line:84% And so that becomes another endogenous dynamic 00:29:27.520 --> 00:29:30.280 align:middle line:84% that your model needs to capture. 00:29:30.280 --> 00:29:33.000 align:middle line:84% Just highlighting that because to capture 00:29:33.000 --> 00:29:35.800 align:middle line:84% that, you need some concept of what 00:29:35.800 --> 00:29:41.480 align:middle line:84% is the current state of energy efficiency for, for example, 00:29:41.480 --> 00:29:44.100 align:middle line:84% these brown industries, which it might be mining, 00:29:44.100 --> 00:29:46.940 align:middle line:84% it might be cement production, and so on. 00:29:46.940 --> 00:29:49.920 align:middle line:84% So they might have pretty high levels of emissions 00:29:49.920 --> 00:29:53.280 align:middle line:84% at the beginning, but that high level of emission 00:29:53.280 --> 00:29:56.280 align:middle line:84% has been going down over time because of new technologies 00:29:56.280 --> 00:30:00.140 align:middle line:84% and new focus on energy consumption. 00:30:00.140 --> 00:30:04.300 align:middle line:84% So that's another piece of the endogenous story. 00:30:04.300 --> 00:30:07.940 align:middle line:84% So I would say, simplifying the model 00:30:07.940 --> 00:30:12.300 align:middle line:84% by making your economic capital more of a parameter 00:30:12.300 --> 00:30:14.980 align:middle line:84% and instead making sure you have those 00:30:14.980 --> 00:30:20.660 align:middle line:84% two pieces of the endogeneity in the model 00:30:20.660 --> 00:30:24.300 align:middle line:84% would make a pretty nice, solid starting model, which 00:30:24.300 --> 00:30:28.460 align:middle line:84% is actually a conceptually more transparent model. 00:30:28.460 --> 00:30:30.580 align:middle line:84% And then depending on what you want 00:30:30.580 --> 00:30:33.220 align:middle line:84% to do with this, if you're moving towards combining 00:30:33.220 --> 00:30:35.720 align:middle line:84% with empirical data across different industries 00:30:35.720 --> 00:30:38.660 align:middle line:84% and so on, then you could disaggregate 00:30:38.660 --> 00:30:42.560 align:middle line:84% for different industries rather than just having two. 00:30:42.560 --> 00:30:45.600 align:middle line:84% But I wouldn't recommend that at this stage, 00:30:45.600 --> 00:30:48.160 align:middle line:84% because I think this is clean and well-defined enough 00:30:48.160 --> 00:30:50.540 align:middle line:84% that it's giving you a nice result. 00:30:50.540 --> 00:30:52.780 align:middle line:84% AUDIENCE: Yeah, this equation really 00:30:52.780 --> 00:30:55.680 align:middle line:84% I used at the bottom of this model. 00:30:55.680 --> 00:30:58.180 align:middle line:84% There is the other part related to how 00:30:58.180 --> 00:31:03.700 align:middle line:84% investment in green technology in a certain way 00:31:03.700 --> 00:31:07.010 align:middle line:84% related always to the other model, to green building, 00:31:07.010 --> 00:31:08.350 align:middle line:90% this is really close. 00:31:08.350 --> 00:31:14.230 align:middle line:84% And so here, I use the unit of kilowatt for dollars. 00:31:14.230 --> 00:31:15.850 align:middle line:90% And I'm not really sure. 00:31:15.850 --> 00:31:23.930 align:middle line:84% In this part, I use this energy intensity of capital, used this. 00:31:23.930 --> 00:31:25.850 align:middle line:90% I don't know if it's good. 00:31:25.850 --> 00:31:29.930 align:middle line:84% HAZHIR RAHMANDAD: So I like the minimum feasible energy 00:31:29.930 --> 00:31:32.670 align:middle line:84% intensity per capital as a stock. 00:31:32.670 --> 00:31:35.990 align:middle line:84% That's, I think, a conceptually useful variable to have, 00:31:35.990 --> 00:31:40.290 align:middle line:84% which can then drive the new energy 00:31:40.290 --> 00:31:42.990 align:middle line:84% intensity for the new firms that are being built, 00:31:42.990 --> 00:31:48.530 align:middle line:84% both brown startups and green startups. 00:31:48.530 --> 00:31:52.250 align:middle line:90% So as such, I like the idea. 00:31:52.250 --> 00:31:55.570 align:middle line:84% The unit, you have it as kind of kilowatt 00:31:55.570 --> 00:31:58.210 align:middle line:90% per dollar or something. 00:31:58.210 --> 00:32:00.770 align:middle line:90% It's a little bit tricky. 00:32:00.770 --> 00:32:04.910 align:middle line:84% I don't like dollar, but I know that most people measure capital 00:32:04.910 --> 00:32:08.870 align:middle line:84% in dollars and they are comfortable with it, 00:32:08.870 --> 00:32:12.230 align:middle line:84% just because when the exchange rates change 00:32:12.230 --> 00:32:17.010 align:middle line:84% the value of dollar, the unit of capital is not changing, 00:32:17.010 --> 00:32:18.810 align:middle line:84% energy intensity is not changing. 00:32:18.810 --> 00:32:21.350 align:middle line:84% Nothing is changing, but it's just that the dollar 00:32:21.350 --> 00:32:23.050 align:middle line:90% value has changed. 00:32:23.050 --> 00:32:24.130 align:middle line:90% But it is OK. 00:32:24.130 --> 00:32:28.550 align:middle line:84% I think I might have had something that 00:32:28.550 --> 00:32:30.590 align:middle line:90% was more dollar equivalent. 00:32:30.590 --> 00:32:34.070 align:middle line:90% 00:32:34.070 --> 00:32:36.130 align:middle line:84% Based on the prices of a given year, 00:32:36.130 --> 00:32:40.150 align:middle line:84% what was the value of that capital or something like that. 00:32:40.150 --> 00:32:41.180 align:middle line:90% AUDIENCE: OK, I see. 00:32:41.180 --> 00:32:42.930 align:middle line:84% HAZHIR RAHMANDAD: But it's not a big deal. 00:32:42.930 --> 00:32:47.910 align:middle line:84% I think you're fine with using dollar there as well. 00:32:47.910 --> 00:32:52.110 align:middle line:90% Yeah, this makes sense. 00:32:52.110 --> 00:32:55.550 align:middle line:84% AUDIENCE: And the other part is related to my question. 00:32:55.550 --> 00:33:00.470 align:middle line:84% I was thinking about how to really connect the probability 00:33:00.470 --> 00:33:03.930 align:middle line:84% definition or I called the limitation effect. 00:33:03.930 --> 00:33:06.410 align:middle line:84% This effect of green startup, this 00:33:06.410 --> 00:33:09.810 align:middle line:84% is particularly important when it 00:33:09.810 --> 00:33:13.610 align:middle line:84% comes with the brown firm in a certain way. 00:33:13.610 --> 00:33:17.010 align:middle line:84% And I was thinking, if it's possible to think 00:33:17.010 --> 00:33:20.790 align:middle line:84% about it like the retrofit effect in the green buildings, 00:33:20.790 --> 00:33:23.550 align:middle line:84% but it doesn't make a lot of sense for me. 00:33:23.550 --> 00:33:26.890 align:middle line:84% And so I would struggle just to figure out 00:33:26.890 --> 00:33:31.250 align:middle line:84% how to model this part in a good way. 00:33:31.250 --> 00:33:34.330 align:middle line:84% HAZHIR RAHMANDAD: So can you tell, what is it exactly 00:33:34.330 --> 00:33:37.090 align:middle line:90% that you're trying to model? 00:33:37.090 --> 00:33:38.250 align:middle line:90% AUDIENCE: Yeah. 00:33:38.250 --> 00:33:43.250 align:middle line:84% I'm trying to model two different types of mechanisms. 00:33:43.250 --> 00:33:49.250 align:middle line:84% One is mechanism, for example, the green startup developed 00:33:49.250 --> 00:33:55.250 align:middle line:84% a new technology that has less energy 00:33:55.250 --> 00:34:00.930 align:middle line:84% intensity to use and to sell this technology. 00:34:00.930 --> 00:34:03.310 align:middle line:84% In this particular case, a new firm 00:34:03.310 --> 00:34:11.429 align:middle line:84% can, for example, imitate the investment of this startup 00:34:11.429 --> 00:34:16.230 align:middle line:84% and start to invest in this technology 00:34:16.230 --> 00:34:21.770 align:middle line:84% and to use this technology in a particular business. 00:34:21.770 --> 00:34:26.630 align:middle line:84% And so in that case, we have a reduction 00:34:26.630 --> 00:34:29.969 align:middle line:84% in the consumption for the brown firm 00:34:29.969 --> 00:34:34.110 align:middle line:84% because it's starting to use this technology. 00:34:34.110 --> 00:34:37.750 align:middle line:84% That is caused by the startup, the green startup, 00:34:37.750 --> 00:34:43.590 align:middle line:84% in a certain way gain the work on that technology. 00:34:43.590 --> 00:34:47.790 align:middle line:84% The other one is related to the same mechanism. 00:34:47.790 --> 00:34:52.790 align:middle line:84% It's where the brown firm buy the technology from the startup. 00:34:52.790 --> 00:34:56.610 align:middle line:84% That is, from an economic point of view, 00:34:56.610 --> 00:34:58.110 align:middle line:84% completely different because there 00:34:58.110 --> 00:35:00.980 align:middle line:90% is a benefit from the startup. 00:35:00.980 --> 00:35:05.742 align:middle line:84% But from a consumption point of view, it is actually the same. 00:35:05.742 --> 00:35:06.700 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 00:35:06.700 --> 00:35:09.520 align:middle line:84% So I would say for that distinction, 00:35:09.520 --> 00:35:12.180 align:middle line:84% your model is too aggregate to worry too much 00:35:12.180 --> 00:35:13.660 align:middle line:90% about that distinction. 00:35:13.660 --> 00:35:17.340 align:middle line:84% That is in a way, you can think about it, OK, a fraction of it 00:35:17.340 --> 00:35:21.020 align:middle line:84% is going to be benefiting the startup, a fraction of it 00:35:21.020 --> 00:35:24.780 align:middle line:84% is just benefiting the brownfield company 00:35:24.780 --> 00:35:28.340 align:middle line:84% by reducing its energy consumption. 00:35:28.340 --> 00:35:31.860 align:middle line:84% And if you assume that fraction remains the same 00:35:31.860 --> 00:35:34.700 align:middle line:84% or is not dynamically changing, then you 00:35:34.700 --> 00:35:36.120 align:middle line:90% don't need to separate the two. 00:35:36.120 --> 00:35:39.140 align:middle line:84% You just use the average for both the benefit 00:35:39.140 --> 00:35:41.740 align:middle line:84% to startup, if you want to endogenize that, 00:35:41.740 --> 00:35:46.420 align:middle line:84% and for the reduction in energy consumption. 00:35:46.420 --> 00:35:48.638 align:middle line:90% So that would be my suggestion. 00:35:48.638 --> 00:35:50.180 align:middle line:84% And in that sense, it would be pretty 00:35:50.180 --> 00:35:52.660 align:middle line:84% close to what you saw in the retrofit example 00:35:52.660 --> 00:35:57.040 align:middle line:84% that we were working on in terms of formulations. 00:35:57.040 --> 00:35:59.960 align:middle line:84% But then you could have it feedback back 00:35:59.960 --> 00:36:04.080 align:middle line:84% into growth rate of startups, green startups, 00:36:04.080 --> 00:36:07.560 align:middle line:84% based on the rate of adoption of this thing 00:36:07.560 --> 00:36:10.320 align:middle line:84% among the brownfields, because that's 00:36:10.320 --> 00:36:13.580 align:middle line:84% what is driving the growth of some of these green startups. 00:36:13.580 --> 00:36:17.520 align:middle line:84% And you can capture that explicitly and that's fine. 00:36:17.520 --> 00:36:19.680 align:middle line:90% AUDIENCE: Yeah. 00:36:19.680 --> 00:36:20.720 align:middle line:90% HAZHIR RAHMANDAD: OK. 00:36:20.720 --> 00:36:21.460 align:middle line:90% AUDIENCE: Thank you, thank you. 00:36:21.460 --> 00:36:22.520 align:middle line:90% HAZHIR RAHMANDAD: Sure. 00:36:22.520 --> 00:36:23.180 align:middle line:90% This is good. 00:36:23.180 --> 00:36:25.540 align:middle line:84% I think you are moving in a good direction. 00:36:25.540 --> 00:36:28.500 align:middle line:84% I think the pieces are pretty sensible. 00:36:28.500 --> 00:36:32.060 align:middle line:84% This is going to be challenging still, but it is a challenge, 00:36:32.060 --> 00:36:34.320 align:middle line:90% I think, you can manage. 00:36:34.320 --> 00:36:35.288 align:middle line:90% AUDIENCE: Thank you. 00:36:35.288 --> 00:36:37.080 align:middle line:84% Yeah, the other thing that I'm trying to do 00:36:37.080 --> 00:36:42.080 align:middle line:84% is try to endogenize the death rate and the birth 00:36:42.080 --> 00:36:43.520 align:middle line:90% rate of startups. 00:36:43.520 --> 00:36:45.800 align:middle line:84% So in this, I think, for the moment 00:36:45.800 --> 00:36:47.300 align:middle line:90% it's really too much complex. 00:36:47.300 --> 00:36:50.680 align:middle line:84% But I think it's a good part because this 00:36:50.680 --> 00:36:54.320 align:middle line:84% opens to some entrepreneurial mechanism that 00:36:54.320 --> 00:36:56.152 align:middle line:90% are really important. 00:36:56.152 --> 00:36:57.360 align:middle line:90% HAZHIR RAHMANDAD: Absolutely. 00:36:57.360 --> 00:36:59.660 align:middle line:84% So I think this is very important. 00:36:59.660 --> 00:37:03.880 align:middle line:84% Have a hierarchy of requirements that you will add in future, 00:37:03.880 --> 00:37:05.960 align:middle line:84% but right now don't worry about them. 00:37:05.960 --> 00:37:13.020 align:middle line:84% So exogenous birth rate even death rate is fine for now. 00:37:13.020 --> 00:37:16.560 align:middle line:84% But later on, you can incorporate that. 00:37:16.560 --> 00:37:18.780 align:middle line:84% And we kind of went through a few of these mechanisms 00:37:18.780 --> 00:37:21.620 align:middle line:84% that you can add increment by increment. 00:37:21.620 --> 00:37:24.380 align:middle line:84% In a way, maybe your model is already 00:37:24.380 --> 00:37:28.300 align:middle line:84% a little bit bigger than what it should 00:37:28.300 --> 00:37:31.440 align:middle line:90% be for starting the simulation. 00:37:31.440 --> 00:37:34.020 align:middle line:84% Because you want to have a simulating model as soon 00:37:34.020 --> 00:37:37.460 align:middle line:84% as possible so that you can understand what you currently 00:37:37.460 --> 00:37:40.900 align:middle line:84% have with all the assumptions that go into it, and then 00:37:40.900 --> 00:37:43.740 align:middle line:90% add another layer. 00:37:43.740 --> 00:37:46.400 align:middle line:84% When you are not a very experienced modeler, 00:37:46.400 --> 00:37:48.220 align:middle line:84% it is risky to have a model that may 00:37:48.220 --> 00:37:52.820 align:middle line:84% have two, three pages of structures 00:37:52.820 --> 00:37:54.140 align:middle line:90% before you simulate it. 00:37:54.140 --> 00:37:55.740 align:middle line:84% Because then once you simulate it, 00:37:55.740 --> 00:38:00.520 align:middle line:84% it becomes a unwieldy mess that you don't know how 00:38:00.520 --> 00:38:02.960 align:middle line:90% to bring it back under control. 00:38:02.960 --> 00:38:06.080 align:middle line:84% So start with something that is under control. 00:38:06.080 --> 00:38:07.800 align:middle line:84% It could be as simple as, OK, I just 00:38:07.800 --> 00:38:12.480 align:middle line:84% have a green startup and green firm stock and flow structure 00:38:12.480 --> 00:38:13.260 align:middle line:90% and that's it. 00:38:13.260 --> 00:38:17.800 align:middle line:84% And then I add the energy consumption per unit. 00:38:17.800 --> 00:38:19.100 align:middle line:90% And then I add the other two. 00:38:19.100 --> 00:38:21.320 align:middle line:84% And they are disconnected from each other. 00:38:21.320 --> 00:38:24.800 align:middle line:84% So each iteration, I make sure it is working, 00:38:24.800 --> 00:38:27.200 align:middle line:90% the model simulates well. 00:38:27.200 --> 00:38:29.300 align:middle line:90% I understand what it does. 00:38:29.300 --> 00:38:30.560 align:middle line:90% Then add the next ones. 00:38:30.560 --> 00:38:33.100 align:middle line:84% So once you add the brownfield, but they are not connected, 00:38:33.100 --> 00:38:36.080 align:middle line:84% they are just really the same thing, that's easy. 00:38:36.080 --> 00:38:37.820 align:middle line:90% Then make the first connection. 00:38:37.820 --> 00:38:44.640 align:middle line:84% That's the impact of greenfield on the rate of improvement 00:38:44.640 --> 00:38:47.200 align:middle line:84% in the brownfield or something that you think 00:38:47.200 --> 00:38:48.880 align:middle line:90% might be the first priority. 00:38:48.880 --> 00:38:52.010 align:middle line:84% And then just add in these iterations. 00:38:52.010 --> 00:38:54.790 align:middle line:84% The iterative modeling process, and we highlighted that 00:38:54.790 --> 00:38:57.610 align:middle line:84% at the beginning of the semester, is really your friend. 00:38:57.610 --> 00:39:01.730 align:middle line:84% Without that, it very quickly becomes very frustrating. 00:39:01.730 --> 00:39:05.550 align:middle line:84% So if you have a big model that has a lot of moving parts 00:39:05.550 --> 00:39:09.550 align:middle line:84% that you don't understand, you can spend a whole week full-time 00:39:09.550 --> 00:39:12.610 align:middle line:84% trying to deal with it and make sense of it, 00:39:12.610 --> 00:39:16.030 align:middle line:84% and it just baffles you and defeats you 00:39:16.030 --> 00:39:18.330 align:middle line:90% and it becomes very frustrating. 00:39:18.330 --> 00:39:20.370 align:middle line:90% So I see a lot of nodding heads. 00:39:20.370 --> 00:39:25.430 align:middle line:84% So that shows you this course has done what it has to do. 00:39:25.430 --> 00:39:26.550 align:middle line:90% Yeah. 00:39:26.550 --> 00:39:28.030 align:middle line:90% AUDIENCE: Can ask a question? 00:39:28.030 --> 00:39:32.230 align:middle line:84% I think it can also open a discussion here related to 00:39:32.230 --> 00:39:39.590 align:middle line:84% in which case a model can really make a theoretical contribution 00:39:39.590 --> 00:39:42.030 align:middle line:90% to a field of research. 00:39:42.030 --> 00:39:44.290 align:middle line:84% HAZHIR RAHMANDAD: Yes, very good question. 00:39:44.290 --> 00:39:48.270 align:middle line:84% So in this case, first of all, you always go back 00:39:48.270 --> 00:39:51.850 align:middle line:84% to what is the current state of understanding in the field. 00:39:51.850 --> 00:39:56.610 align:middle line:84% So if the current state of understanding has no models 00:39:56.610 --> 00:39:58.730 align:middle line:84% and has no interaction of greenfield 00:39:58.730 --> 00:40:02.090 align:middle line:84% firms and brownfield firms, then as soon 00:40:02.090 --> 00:40:05.690 align:middle line:84% as you build these quantitatively 00:40:05.690 --> 00:40:07.690 align:middle line:84% and make a connection between the two 00:40:07.690 --> 00:40:10.250 align:middle line:84% and show that connection starts to matter 00:40:10.250 --> 00:40:12.850 align:middle line:84% and there is some interesting non-trivial dynamic, 00:40:12.850 --> 00:40:14.290 align:middle line:90% you are making a contribution. 00:40:14.290 --> 00:40:19.530 align:middle line:84% Now, depending on the audience, how big that contribution is, 00:40:19.530 --> 00:40:22.970 align:middle line:84% and how much you need to continue to make it empirically 00:40:22.970 --> 00:40:26.490 align:middle line:84% grounded or not, to add five different mechanisms 00:40:26.490 --> 00:40:29.330 align:middle line:84% and look at the boundary conditions for the impact 00:40:29.330 --> 00:40:33.010 align:middle line:84% of each or not, those are a little bit more field specific 00:40:33.010 --> 00:40:35.150 align:middle line:90% and also journal specific. 00:40:35.150 --> 00:40:38.570 align:middle line:84% So if you go with a very top journal in an economics 00:40:38.570 --> 00:40:42.490 align:middle line:84% type of audience, you need to do a lot more analysis 00:40:42.490 --> 00:40:46.650 align:middle line:84% than some of the engineering journals, for example. 00:40:46.650 --> 00:40:48.450 align:middle line:90% The scales are pretty different. 00:40:48.450 --> 00:40:51.030 align:middle line:84% It's almost an order of magnitude different amount 00:40:51.030 --> 00:40:54.670 align:middle line:84% of work that goes into a typical publication in one 00:40:54.670 --> 00:40:56.150 align:middle line:90% versus another. 00:40:56.150 --> 00:40:57.970 align:middle line:90% So that is not general. 00:40:57.970 --> 00:40:59.910 align:middle line:84% But it is always the starting point 00:40:59.910 --> 00:41:03.310 align:middle line:84% is the front line of what is already done. 00:41:03.310 --> 00:41:06.910 align:middle line:84% Now, because the types of tools you are bringing 00:41:06.910 --> 00:41:09.950 align:middle line:84% into the picture are usually a little bit different-- if you're 00:41:09.950 --> 00:41:13.270 align:middle line:84% building a system dynamics model and going into an audience that 00:41:13.270 --> 00:41:16.790 align:middle line:84% has not been using models in the past, 00:41:16.790 --> 00:41:20.770 align:middle line:84% there is another both positive and negative element to it. 00:41:20.770 --> 00:41:23.950 align:middle line:84% The positive side is that just bringing these tools 00:41:23.950 --> 00:41:27.510 align:middle line:84% may provide them with novelty, with something 00:41:27.510 --> 00:41:30.090 align:middle line:84% that they may appreciate as, oh, this is interesting. 00:41:30.090 --> 00:41:31.870 align:middle line:90% Let me understand it. 00:41:31.870 --> 00:41:33.670 align:middle line:90% So that's good. 00:41:33.670 --> 00:41:36.710 align:middle line:84% The negative side is, you actually 00:41:36.710 --> 00:41:38.010 align:middle line:90% need to do your homework. 00:41:38.010 --> 00:41:39.930 align:middle line:84% It's not negative, but it is more work. 00:41:39.930 --> 00:41:42.390 align:middle line:84% You need to do your homework to understand 00:41:42.390 --> 00:41:46.540 align:middle line:84% how your framework and your modeling work 00:41:46.540 --> 00:41:50.240 align:middle line:84% is already explored in the other language, 00:41:50.240 --> 00:41:53.100 align:middle line:84% in the other way of thinking, and the other way of doing 00:41:53.100 --> 00:41:54.880 align:middle line:90% research in that field. 00:41:54.880 --> 00:41:57.060 align:middle line:84% You really need to understand that 00:41:57.060 --> 00:42:00.900 align:middle line:84% and be able to speak about your model in that language. 00:42:00.900 --> 00:42:03.660 align:middle line:84% You shouldn't go and talk in system dynamics lingo 00:42:03.660 --> 00:42:05.420 align:middle line:84% and talk about feedback loop dominance 00:42:05.420 --> 00:42:08.420 align:middle line:84% and so on if you haven't yet understood 00:42:08.420 --> 00:42:10.860 align:middle line:84% how they think about this phenomenon using 00:42:10.860 --> 00:42:15.740 align:middle line:84% their own vocabulary, using their own tools and methods. 00:42:15.740 --> 00:42:18.140 align:middle line:84% That is really the most important part 00:42:18.140 --> 00:42:19.960 align:middle line:90% of making a contribution. 00:42:19.960 --> 00:42:24.660 align:middle line:84% Because ultimately, if you are the host in some other field's 00:42:24.660 --> 00:42:30.300 align:middle line:84% journal and in some other field's research, conversation-- 00:42:30.300 --> 00:42:33.760 align:middle line:84% if you're a guest, sorry, not the host. 00:42:33.760 --> 00:42:37.960 align:middle line:84% If you're a guest, you need to follow the host's language. 00:42:37.960 --> 00:42:40.360 align:middle line:84% You can't just come and say, oh, I have this model. 00:42:40.360 --> 00:42:41.700 align:middle line:90% I think it is great. 00:42:41.700 --> 00:42:43.620 align:middle line:84% I don't care to read your journals 00:42:43.620 --> 00:42:45.820 align:middle line:84% and understand how you think about this, 00:42:45.820 --> 00:42:47.640 align:middle line:84% but you should come and read my paper 00:42:47.640 --> 00:42:50.000 align:middle line:90% and tell me how great it is. 00:42:50.000 --> 00:42:52.320 align:middle line:90% That's not how the world works. 00:42:52.320 --> 00:42:54.900 align:middle line:84% You actually have to completely reverse it. 00:42:54.900 --> 00:42:58.400 align:middle line:84% You need to go read what they have done, how they think 00:42:58.400 --> 00:43:02.060 align:middle line:84% about the connection between the green firms and brown firms, 00:43:02.060 --> 00:43:07.100 align:middle line:84% then write your paper in the language of that field. 00:43:07.100 --> 00:43:10.755 align:middle line:84% This is what we know from all the other people who 00:43:10.755 --> 00:43:12.880 align:middle line:84% have worked on the relationship between green firms 00:43:12.880 --> 00:43:15.520 align:middle line:84% and brown firms, but one mechanism 00:43:15.520 --> 00:43:17.880 align:middle line:84% has been missing from this or three mechanisms 00:43:17.880 --> 00:43:20.200 align:middle line:84% have been missing from that conversation which 00:43:20.200 --> 00:43:21.500 align:middle line:90% could be important. 00:43:21.500 --> 00:43:24.360 align:middle line:84% Now, I'm going to show why this could be important 00:43:24.360 --> 00:43:26.740 align:middle line:84% and under what conditions it is important. 00:43:26.740 --> 00:43:29.000 align:middle line:84% And to show that, because it requires 00:43:29.000 --> 00:43:31.500 align:middle line:84% looking at the interactions between these things, 00:43:31.500 --> 00:43:33.320 align:middle line:84% we actually need tools that might not have 00:43:33.320 --> 00:43:35.560 align:middle line:90% been used in this field before. 00:43:35.560 --> 00:43:38.473 align:middle line:84% That's how you bring modeling into the picture. 00:43:38.473 --> 00:43:39.640 align:middle line:90% So that's one of the things. 00:43:39.640 --> 00:43:43.620 align:middle line:84% Like, you ask John Sterman, he would say, 00:43:43.620 --> 00:43:45.980 align:middle line:84% he gets pissed off whenever he sees 00:43:45.980 --> 00:43:50.390 align:middle line:84% a title of an article in our conference that says "X, Y, Z-- 00:43:50.390 --> 00:43:53.980 align:middle line:90% A System Dynamics Approach." 00:43:53.980 --> 00:43:57.580 align:middle line:84% Because that is usually trying to sell the work 00:43:57.580 --> 00:44:00.420 align:middle line:90% based on using system dynamics. 00:44:00.420 --> 00:44:04.020 align:middle line:84% Using system dynamics per se is not a good or bad thing. 00:44:04.020 --> 00:44:06.640 align:middle line:84% It is a question of, does it add value? 00:44:06.640 --> 00:44:09.740 align:middle line:84% And if it does add value for that audience 00:44:09.740 --> 00:44:11.740 align:middle line:84% compared to what already they are doing, 00:44:11.740 --> 00:44:14.820 align:middle line:90% you need to show that, not tell. 00:44:14.820 --> 00:44:17.080 align:middle line:84% Show, not tell is kind of the idea. 00:44:17.080 --> 00:44:19.200 align:middle line:90% It's not a sales pitch. 00:44:19.200 --> 00:44:22.140 align:middle line:84% It is something that people read and at the end 00:44:22.140 --> 00:44:23.680 align:middle line:84% say, oh, this is really interesting. 00:44:23.680 --> 00:44:27.040 align:middle line:84% I'm now understanding my own field a little bit better. 00:44:27.040 --> 00:44:29.800 align:middle line:84% I'm understanding my own phenomenon a little bit better. 00:44:29.800 --> 00:44:32.980 align:middle line:90% 00:44:32.980 --> 00:44:36.347 align:middle line:90% OK, let's switch to-- 00:44:36.347 --> 00:44:37.180 align:middle line:90% AUDIENCE: Thank you. 00:44:37.180 --> 00:44:38.138 align:middle line:90% HAZHIR RAHMANDAD: Sure. 00:44:38.138 --> 00:44:41.740 align:middle line:84% No, I think this is going in a good direction. 00:44:41.740 --> 00:44:42.540 align:middle line:90% Good work. 00:44:42.540 --> 00:44:45.400 align:middle line:90% 00:44:45.400 --> 00:44:48.400 align:middle line:84% So we have anybody else has submitted but is not 00:44:48.400 --> 00:44:50.400 align:middle line:90% seeing their thing here? 00:44:50.400 --> 00:44:52.600 align:middle line:90% Let me see. 00:44:52.600 --> 00:44:54.720 align:middle line:84% Oh, my Dropbox is finally up to date. 00:44:54.720 --> 00:44:55.800 align:middle line:90% OK. 00:44:55.800 --> 00:45:02.300 align:middle line:84% So let's this time start with Kathy's model. 00:45:02.300 --> 00:45:10.000 align:middle line:90% 00:45:10.000 --> 00:45:17.280 align:middle line:84% So you have a little bit of an overall of the model. 00:45:17.280 --> 00:45:19.960 align:middle line:90% Good. 00:45:19.960 --> 00:45:25.820 align:middle line:84% So you have the low frequency versus high frequency readers. 00:45:25.820 --> 00:45:31.400 align:middle line:84% And there is basically, right now an exogenous shift 00:45:31.400 --> 00:45:32.300 align:middle line:90% in these numbers. 00:45:32.300 --> 00:45:34.297 align:middle line:84% So this is not an endogenous story. 00:45:34.297 --> 00:45:34.880 align:middle line:90% AUDIENCE: Yes. 00:45:34.880 --> 00:45:35.400 align:middle line:90% HAZHIR RAHMANDAD: OK. 00:45:35.400 --> 00:45:36.025 align:middle line:90% AUDIENCE: Yeah. 00:45:36.025 --> 00:45:37.700 align:middle line:90% So the shift is not endogenous. 00:45:37.700 --> 00:45:41.490 align:middle line:84% The things that are endogenous are just the social influence 00:45:41.490 --> 00:45:44.870 align:middle line:84% impacting the desired reading rates and desired buying 00:45:44.870 --> 00:45:47.410 align:middle line:90% rates separately. 00:45:47.410 --> 00:45:49.290 align:middle line:84% And then there's an intrinsically-- 00:45:49.290 --> 00:45:53.710 align:middle line:84% so I separated out what drives book buying and book 00:45:53.710 --> 00:45:56.770 align:middle line:84% reading and social value and intrinsic value. 00:45:56.770 --> 00:45:57.995 align:middle line:90% So those are the two. 00:45:57.995 --> 00:45:58.870 align:middle line:90% HAZHIR RAHMANDAD: OK. 00:45:58.870 --> 00:46:00.330 align:middle line:90% So let's look at those. 00:46:00.330 --> 00:46:07.130 align:middle line:84% So there is book buying and then desired fraction of books. 00:46:07.130 --> 00:46:09.050 align:middle line:84% And it's a fraction of books that are read. 00:46:09.050 --> 00:46:11.630 align:middle line:90% That's good. 00:46:11.630 --> 00:46:13.130 align:middle line:90% OK. 00:46:13.130 --> 00:46:18.750 align:middle line:84% And then you have the social influence 00:46:18.750 --> 00:46:22.670 align:middle line:90% coming from reading level. 00:46:22.670 --> 00:46:29.390 align:middle line:84% AUDIENCE: So this is how many books out of max capacity people 00:46:29.390 --> 00:46:29.970 align:middle line:90% are reading. 00:46:29.970 --> 00:46:32.790 align:middle line:84% So if everyone's reading at their max capacity, 00:46:32.790 --> 00:46:33.970 align:middle line:90% then it should be one. 00:46:33.970 --> 00:46:37.990 align:middle line:84% And if everyone's reading at half of their max capacity, 00:46:37.990 --> 00:46:41.810 align:middle line:84% so the capacity is kind of normalized 00:46:41.810 --> 00:46:46.498 align:middle line:84% between the high frequency and low frequency folks right now. 00:46:46.498 --> 00:46:48.290 align:middle line:90% HAZHIR RAHMANDAD: Got it. 00:46:48.290 --> 00:46:51.550 align:middle line:84% So there is a capacity utilization. 00:46:51.550 --> 00:46:54.770 align:middle line:90% 00:46:54.770 --> 00:47:00.410 align:middle line:84% So let's see what this behavior-wise looks like. 00:47:00.410 --> 00:47:06.770 align:middle line:84% So if the reading level goes up right now to-- 00:47:06.770 --> 00:47:09.830 align:middle line:90% what is the unit here? 00:47:09.830 --> 00:47:10.690 align:middle line:90% Dimensionless? 00:47:10.690 --> 00:47:12.070 align:middle line:90% What is the average? 00:47:12.070 --> 00:47:13.932 align:middle line:84% AUDIENCE: It's the percent of capacity. 00:47:13.932 --> 00:47:15.890 align:middle line:84% HAZHIR RAHMANDAD: This is average reading level 00:47:15.890 --> 00:47:18.330 align:middle line:84% as a percentage of your capacity to read. 00:47:18.330 --> 00:47:20.490 align:middle line:84% And your capacity to read, where is it 00:47:20.490 --> 00:47:22.970 align:middle line:84% defined so that I can make sense of it? 00:47:22.970 --> 00:47:24.530 align:middle line:84% AUDIENCE: It's to the left and right. 00:47:24.530 --> 00:47:25.905 align:middle line:84% HAZHIR RAHMANDAD: Yeah, I got it. 00:47:25.905 --> 00:47:27.810 align:middle line:84% Max HFR reading capacity in this case 00:47:27.810 --> 00:47:31.830 align:middle line:84% is 10 books per month per person, for example. 00:47:31.830 --> 00:47:34.192 align:middle line:84% AUDIENCE: Yeah, and then two below. 00:47:34.192 --> 00:47:35.150 align:middle line:90% HAZHIR RAHMANDAD: Good. 00:47:35.150 --> 00:47:41.510 align:middle line:84% So then when average reading level goes up, 00:47:41.510 --> 00:47:46.490 align:middle line:84% the social influence goes up and then comes down. 00:47:46.490 --> 00:47:51.390 align:middle line:84% So what's the story here with the formulation? 00:47:51.390 --> 00:47:59.150 align:middle line:84% AUDIENCE: So it's formulated with this exponential growth 00:47:59.150 --> 00:48:01.750 align:middle line:84% based on the reading level increases it. 00:48:01.750 --> 00:48:04.110 align:middle line:84% The k parameter I just set it to 1. 00:48:04.110 --> 00:48:09.550 align:middle line:90% 00:48:09.550 --> 00:48:12.670 align:middle line:84% Or sorry, if capacity utilization is 1, 00:48:12.670 --> 00:48:14.410 align:middle line:90% then that balances it. 00:48:14.410 --> 00:48:17.310 align:middle line:90% Then, that's it. 00:48:17.310 --> 00:48:18.490 align:middle line:90% HAZHIR RAHMANDAD: I see. 00:48:18.490 --> 00:48:23.030 align:middle line:84% So there is indicated desired fraction of books. 00:48:23.030 --> 00:48:25.550 align:middle line:90% No, that's coming from it. 00:48:25.550 --> 00:48:28.950 align:middle line:84% AUDIENCE: I think, yeah, there's a bit of an overshoot as reading 00:48:28.950 --> 00:48:30.510 align:middle line:90% level spikes. 00:48:30.510 --> 00:48:35.490 align:middle line:90% 00:48:35.490 --> 00:48:38.190 align:middle line:84% HAZHIR RAHMANDAD: So reading level is going up. 00:48:38.190 --> 00:48:39.413 align:middle line:90% AUDIENCE: Yeah, very quickly. 00:48:39.413 --> 00:48:40.330 align:middle line:90% HAZHIR RAHMANDAD: Yes. 00:48:40.330 --> 00:48:44.410 align:middle line:84% I'm just trying to understand why social influence goes 00:48:44.410 --> 00:48:47.570 align:middle line:90% down here. 00:48:47.570 --> 00:48:52.070 align:middle line:84% It's a formulation question than something being wrong. 00:48:52.070 --> 00:48:56.210 align:middle line:90% AUDIENCE: It could be wrong. 00:48:56.210 --> 00:49:00.010 align:middle line:84% HAZHIR RAHMANDAD: So for me, this formulation 00:49:00.010 --> 00:49:01.610 align:middle line:90% is already too complex. 00:49:01.610 --> 00:49:04.190 align:middle line:84% That is, separate the components out of it. 00:49:04.190 --> 00:49:09.690 align:middle line:84% So if you have social influence strength on HFR, that's fine. 00:49:09.690 --> 00:49:13.690 align:middle line:84% 1 minus capacity utilization, if that 00:49:13.690 --> 00:49:18.370 align:middle line:84% is the effect of saturation, if that is the idea, 00:49:18.370 --> 00:49:20.490 align:middle line:84% then put an effect of saturation out. 00:49:20.490 --> 00:49:22.490 align:middle line:84% Because otherwise it's just hard. 00:49:22.490 --> 00:49:24.770 align:middle line:84% It leads to a little bit of confusion 00:49:24.770 --> 00:49:27.230 align:middle line:90% for people looking at it. 00:49:27.230 --> 00:49:31.130 align:middle line:84% And then the last piece, so this one also has 1 minus e 00:49:31.130 --> 00:49:32.400 align:middle line:90% to the power of this thing. 00:49:32.400 --> 00:49:33.680 align:middle line:90% It's OK. 00:49:33.680 --> 00:49:38.240 align:middle line:84% But that can be a separate effect 00:49:38.240 --> 00:49:40.540 align:middle line:84% if you want to have it as a separate effect. 00:49:40.540 --> 00:49:43.080 align:middle line:84% So then this thing is this effect and that effect 00:49:43.080 --> 00:49:45.198 align:middle line:90% multiplied by each other. 00:49:45.198 --> 00:49:46.740 align:middle line:84% You could keep one of them inside it, 00:49:46.740 --> 00:49:50.760 align:middle line:84% but don't keep both of them inside. 00:49:50.760 --> 00:49:59.880 align:middle line:84% So then that leads to more buying of books. 00:49:59.880 --> 00:50:02.540 align:middle line:84% So this social influence, conceptually, what is it? 00:50:02.540 --> 00:50:05.653 align:middle line:84% Is it about reading or is it about buying? 00:50:05.653 --> 00:50:07.320 align:middle line:84% AUDIENCE: The social influence right now 00:50:07.320 --> 00:50:09.828 align:middle line:90% is coming from reading. 00:50:09.828 --> 00:50:11.620 align:middle line:84% HAZHIR RAHMANDAD: It's coming from reading, 00:50:11.620 --> 00:50:15.120 align:middle line:84% but is it about pushing people towards what? 00:50:15.120 --> 00:50:17.405 align:middle line:84% AUDIENCE: Oh, pushing people towards buying. 00:50:17.405 --> 00:50:18.280 align:middle line:90% HAZHIR RAHMANDAD: OK. 00:50:18.280 --> 00:50:18.960 align:middle line:90% AUDIENCE: Yeah. 00:50:18.960 --> 00:50:21.797 align:middle line:84% HAZHIR RAHMANDAD: So you see others are reading books. 00:50:21.797 --> 00:50:23.880 align:middle line:84% AUDIENCE: Yeah, and then you want to buy the book. 00:50:23.880 --> 00:50:26.320 align:middle line:84% HAZHIR RAHMANDAD: And then you want to buy books. 00:50:26.320 --> 00:50:29.160 align:middle line:84% But are you seeing people read the books 00:50:29.160 --> 00:50:33.080 align:middle line:84% or are you seeing them talk about it on social media? 00:50:33.080 --> 00:50:35.035 align:middle line:84% That was kind of your starting point. 00:50:35.035 --> 00:50:35.660 align:middle line:90% AUDIENCE: Yeah. 00:50:35.660 --> 00:50:40.220 align:middle line:84% So I think I'm assuming that the influence is 00:50:40.220 --> 00:50:45.260 align:middle line:84% coming from people who are able to speak about the book. 00:50:45.260 --> 00:50:49.260 align:middle line:90% And thus, they've read the book. 00:50:49.260 --> 00:50:52.640 align:middle line:84% But that doesn't mean that the buyer will then read the book. 00:50:52.640 --> 00:50:56.100 align:middle line:90% 00:50:56.100 --> 00:50:57.420 align:middle line:90% HAZHIR RAHMANDAD: OK. 00:50:57.420 --> 00:51:02.140 align:middle line:84% So I think the process right now you have is that, 00:51:02.140 --> 00:51:05.100 align:middle line:84% actually, it seems like people go buy it 00:51:05.100 --> 00:51:08.860 align:middle line:84% and that average reading level is determining 00:51:08.860 --> 00:51:13.580 align:middle line:84% the social pressure, whereas the story you were telling us 00:51:13.580 --> 00:51:17.580 align:middle line:84% was more about, oh, there are a lot of people who kind don't 00:51:17.580 --> 00:51:18.760 align:middle line:90% really read the book. 00:51:18.760 --> 00:51:22.740 align:middle line:84% They just go and do it, look at a summary of it, 00:51:22.740 --> 00:51:25.980 align:middle line:84% and then bring it and use it in their social media 00:51:25.980 --> 00:51:29.480 align:middle line:84% to boast that this is a good book or a bad book 00:51:29.480 --> 00:51:31.015 align:middle line:84% and we know about it or something. 00:51:31.015 --> 00:51:31.640 align:middle line:90% AUDIENCE: Yeah. 00:51:31.640 --> 00:51:36.160 align:middle line:84% And then I started thinking about the influence 00:51:36.160 --> 00:51:40.640 align:middle line:84% isn't necessarily coming from the aspirational book buyers. 00:51:40.640 --> 00:51:44.720 align:middle line:90% 00:51:44.720 --> 00:51:46.180 align:middle line:84% It could be formulated another way. 00:51:46.180 --> 00:51:49.480 align:middle line:84% This is just the way that I've done it on this draft. 00:51:49.480 --> 00:51:51.220 align:middle line:90% HAZHIR RAHMANDAD: So it's OK. 00:51:51.220 --> 00:51:56.140 align:middle line:84% It's just kind of the story is now a little bit has shifted. 00:51:56.140 --> 00:51:58.080 align:middle line:84% So that's why I'm asking whether you wanted 00:51:58.080 --> 00:52:01.975 align:middle line:90% to make that shift consciously. 00:52:01.975 --> 00:52:02.600 align:middle line:90% AUDIENCE: Yeah. 00:52:02.600 --> 00:52:04.560 align:middle line:84% Because I was thinking about the people who 00:52:04.560 --> 00:52:08.600 align:middle line:84% are influencing probably have read it 00:52:08.600 --> 00:52:12.300 align:middle line:84% or at least can talk about it coherently enough. 00:52:12.300 --> 00:52:13.400 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 00:52:13.400 --> 00:52:16.840 align:middle line:84% So the thing is, the average reading level 00:52:16.840 --> 00:52:19.720 align:middle line:84% is fraction of your maximum that you're willing to read 00:52:19.720 --> 00:52:25.277 align:middle line:84% or you're actually reading for these high versus low. 00:52:25.277 --> 00:52:27.110 align:middle line:84% AUDIENCE: Yeah, and it's combined right now. 00:52:27.110 --> 00:52:29.510 align:middle line:84% HAZHIR RAHMANDAD: Oh, it is also combined. 00:52:29.510 --> 00:52:34.290 align:middle line:84% So that adds a little bit even more. 00:52:34.290 --> 00:52:36.910 align:middle line:90% 00:52:36.910 --> 00:52:37.570 align:middle line:90% Let's see. 00:52:37.570 --> 00:52:42.710 align:middle line:90% 00:52:42.710 --> 00:52:47.870 align:middle line:84% So this is high frequency readers 00:52:47.870 --> 00:52:51.230 align:middle line:84% multiplied by maximum and low frequency readers. 00:52:51.230 --> 00:52:54.070 align:middle line:84% So this should be divided by high frequency readers 00:52:54.070 --> 00:52:56.470 align:middle line:90% plus low frequency readers. 00:52:56.470 --> 00:53:00.190 align:middle line:84% Now, I suspect you have put them as fraction of population 00:53:00.190 --> 00:53:01.830 align:middle line:90% from the beginning. 00:53:01.830 --> 00:53:03.650 align:middle line:90% So that doesn't play a role. 00:53:03.650 --> 00:53:07.510 align:middle line:84% So high frequency readers plus low frequency readers 00:53:07.510 --> 00:53:09.870 align:middle line:90% add up to 1. 00:53:09.870 --> 00:53:11.555 align:middle line:84% AUDIENCE: Well, that's the denominator. 00:53:11.555 --> 00:53:13.430 align:middle line:84% HAZHIR RAHMANDAD: Oh, there is a denominator? 00:53:13.430 --> 00:53:17.970 align:middle line:84% AUDIENCE: Yes, so that on top is the numerator. 00:53:17.970 --> 00:53:18.962 align:middle line:90% And then it is divided. 00:53:18.962 --> 00:53:20.670 align:middle line:84% HAZHIR RAHMANDAD: Oh, yes, yes, yes, yes. 00:53:20.670 --> 00:53:21.503 align:middle line:90% You're right, sorry. 00:53:21.503 --> 00:53:23.670 align:middle line:84% I was just looking at this second line 00:53:23.670 --> 00:53:25.330 align:middle line:84% and thinking that's the formulation. 00:53:25.330 --> 00:53:27.590 align:middle line:90% You have this first numerator. 00:53:27.590 --> 00:53:28.170 align:middle line:90% OK, yeah. 00:53:28.170 --> 00:53:31.050 align:middle line:90% 00:53:31.050 --> 00:53:32.825 align:middle line:90% Yes, and these are the maximum. 00:53:32.825 --> 00:53:33.450 align:middle line:90% AUDIENCE: Yeah. 00:53:33.450 --> 00:53:36.690 align:middle line:90% 00:53:36.690 --> 00:53:40.250 align:middle line:84% HAZHIR RAHMANDAD: OK, that makes sense. 00:53:40.250 --> 00:53:42.730 align:middle line:84% So this is kind of a social norm, 00:53:42.730 --> 00:53:50.290 align:middle line:84% for what fraction of your total you want to be reading. 00:53:50.290 --> 00:53:53.890 align:middle line:84% If that is the social norm idea, then that 00:53:53.890 --> 00:53:58.690 align:middle line:84% becomes a very good anchor for this, what fraction 00:53:58.690 --> 00:54:00.810 align:middle line:90% you are going to be reading. 00:54:00.810 --> 00:54:02.970 align:middle line:84% So then when I go to the equation 00:54:02.970 --> 00:54:05.250 align:middle line:84% where it uses this, which is here, 00:54:05.250 --> 00:54:09.250 align:middle line:90% I want see it as an anchor. 00:54:09.250 --> 00:54:14.570 align:middle line:84% Whereas, right now, you have it in this complex formulation 00:54:14.570 --> 00:54:19.570 align:middle line:84% that is 1 minus e to the power of minus k multiplied 00:54:19.570 --> 00:54:21.450 align:middle line:90% by average reading level. 00:54:21.450 --> 00:54:26.230 align:middle line:84% So this is doing a nonlinear transformation 00:54:26.230 --> 00:54:28.230 align:middle line:84% on an anchor that, in my view, should 00:54:28.230 --> 00:54:31.990 align:middle line:90% have been just the main thing. 00:54:31.990 --> 00:54:36.510 align:middle line:84% I don't understand why I need this nonlinear transformation. 00:54:36.510 --> 00:54:39.290 align:middle line:84% Now, what was your thinking when you built that? 00:54:39.290 --> 00:54:48.130 align:middle line:90% 00:54:48.130 --> 00:54:50.710 align:middle line:84% AUDIENCE: I think I was just overcomplicating it. 00:54:50.710 --> 00:54:51.510 align:middle line:84% HAZHIR RAHMANDAD: Yeah, but there 00:54:51.510 --> 00:54:53.093 align:middle line:84% was something in the back of your mind 00:54:53.093 --> 00:54:56.570 align:middle line:84% that you decided to put this complicated equation. 00:54:56.570 --> 00:54:58.090 align:middle line:90% So what was that? 00:54:58.090 --> 00:55:04.390 align:middle line:84% AUDIENCE: I wanted to have the steepness 00:55:04.390 --> 00:55:06.790 align:middle line:84% of the social influence to attenuate 00:55:06.790 --> 00:55:09.290 align:middle line:90% how quickly we get somewhere. 00:55:09.290 --> 00:55:12.070 align:middle line:84% And then I was kind of just rooting around 00:55:12.070 --> 00:55:19.710 align:middle line:84% for formulations that have this non-linear shape, because I 00:55:19.710 --> 00:55:22.008 align:middle line:84% was thinking that it might approach it in that regard. 00:55:22.008 --> 00:55:23.050 align:middle line:90% HAZHIR RAHMANDAD: Got it. 00:55:23.050 --> 00:55:26.090 align:middle line:84% So I think really at the heart of the problem 00:55:26.090 --> 00:55:32.650 align:middle line:84% here is that you built an input that was really an anchor, 00:55:32.650 --> 00:55:35.570 align:middle line:84% but you were talking about it and thinking about it in terms 00:55:35.570 --> 00:55:37.730 align:middle line:84% of social influence, because you were framing 00:55:37.730 --> 00:55:39.650 align:middle line:90% this as social influence. 00:55:39.650 --> 00:55:41.890 align:middle line:84% So if you're doing social influence, 00:55:41.890 --> 00:55:46.430 align:middle line:84% you probably needed some other mechanism. 00:55:46.430 --> 00:55:48.810 align:middle line:84% Because social influence is about people 00:55:48.810 --> 00:55:52.170 align:middle line:84% basically talking about the fact that they are reading books. 00:55:52.170 --> 00:55:56.730 align:middle line:84% So you needed to be counting how many books are being read 00:55:56.730 --> 00:55:58.250 align:middle line:84% or some version of that, and then 00:55:58.250 --> 00:56:00.610 align:middle line:84% multiply that by what fraction of people 00:56:00.610 --> 00:56:02.590 align:middle line:84% are willing to put it on their social media, 00:56:02.590 --> 00:56:06.362 align:middle line:84% and each person who puts it on social media, how many people 00:56:06.362 --> 00:56:08.070 align:middle line:84% will read it, and then when they read it, 00:56:08.070 --> 00:56:10.250 align:middle line:84% what fraction of those people get 00:56:10.250 --> 00:56:12.825 align:middle line:84% interested in going and buying a book. 00:56:12.825 --> 00:56:13.450 align:middle line:90% AUDIENCE: Yeah. 00:56:13.450 --> 00:56:15.617 align:middle line:84% HAZHIR RAHMANDAD: So that becomes a social influence 00:56:15.617 --> 00:56:17.170 align:middle line:90% pathway. 00:56:17.170 --> 00:56:22.320 align:middle line:84% But you didn't build that infrastructure to go into it. 00:56:22.320 --> 00:56:25.140 align:middle line:84% What you built is a little bit of a different way that says, 00:56:25.140 --> 00:56:27.047 align:middle line:84% oh, people look at their friends and see 00:56:27.047 --> 00:56:28.880 align:middle line:84% how much they are reading books, and if they 00:56:28.880 --> 00:56:33.480 align:middle line:84% are reading a lot of books, they anchor to that level. 00:56:33.480 --> 00:56:35.180 align:middle line:84% And then therefore, based on that, 00:56:35.180 --> 00:56:38.240 align:middle line:84% they should decide how many books they want to buy. 00:56:38.240 --> 00:56:42.480 align:middle line:84% AUDIENCE: What's the conceptual difference between those, 00:56:42.480 --> 00:56:44.080 align:middle line:90% like psychologically? 00:56:44.080 --> 00:56:46.760 align:middle line:84% HAZHIR RAHMANDAD: Yes, so the cover stories 00:56:46.760 --> 00:56:48.000 align:middle line:90% are a bit different. 00:56:48.000 --> 00:56:50.160 align:middle line:84% So the cover story of the first one 00:56:50.160 --> 00:56:54.640 align:middle line:84% is, I see folks posting about books 00:56:54.640 --> 00:56:57.960 align:middle line:84% and that triggers me to go and buy books. 00:56:57.960 --> 00:57:02.120 align:middle line:84% The second one says, I see my friends are spending 00:57:02.120 --> 00:57:06.400 align:middle line:84% time reading books and they are spending two hours a day 00:57:06.400 --> 00:57:09.840 align:middle line:84% reading books, so maybe I should be spending two hours a day 00:57:09.840 --> 00:57:10.840 align:middle line:90% reading books. 00:57:10.840 --> 00:57:13.720 align:middle line:84% So one is about how I think about allocation 00:57:13.720 --> 00:57:15.800 align:middle line:90% of my time, the second one. 00:57:15.800 --> 00:57:19.500 align:middle line:84% The first one is, what triggers me to go and buy books? 00:57:19.500 --> 00:57:21.380 align:middle line:84% So if you go with the first one, I 00:57:21.380 --> 00:57:24.840 align:middle line:84% would be worried about buying too many books, 00:57:24.840 --> 00:57:26.800 align:middle line:84% because that might easily happen. 00:57:26.800 --> 00:57:29.980 align:middle line:90% It doesn't have a bound on that. 00:57:29.980 --> 00:57:33.740 align:middle line:84% The time-based one has boundaries of time 00:57:33.740 --> 00:57:40.660 align:middle line:84% that you can enforce but doesn't have the exponential growth 00:57:40.660 --> 00:57:41.520 align:middle line:90% built into it. 00:57:41.520 --> 00:57:44.060 align:middle line:84% It doesn't have the potential for exponential growth 00:57:44.060 --> 00:57:44.580 align:middle line:90% that easily. 00:57:44.580 --> 00:57:46.760 align:middle line:84% Because, yeah, the norms are spreading, 00:57:46.760 --> 00:57:49.780 align:middle line:84% but usually you're kind of in the middle, whatever 00:57:49.780 --> 00:57:50.400 align:middle line:90% is happening. 00:57:50.400 --> 00:57:54.020 align:middle line:84% So that's a source of inconsistency 00:57:54.020 --> 00:58:00.500 align:middle line:84% that then will make the rest of the model internally 00:58:00.500 --> 00:58:02.400 align:middle line:84% a little bit inconsistent on that. 00:58:02.400 --> 00:58:04.898 align:middle line:90% 00:58:04.898 --> 00:58:05.940 align:middle line:90% AUDIENCE: That's helpful. 00:58:05.940 --> 00:58:09.020 align:middle line:90% 00:58:09.020 --> 00:58:12.203 align:middle line:84% So I just need to decide whether it's about anchoring versus-- 00:58:12.203 --> 00:58:13.120 align:middle line:90% HAZHIR RAHMANDAD: Yes. 00:58:13.120 --> 00:58:15.780 align:middle line:84% And whichever you pick, you can potentially 00:58:15.780 --> 00:58:18.380 align:middle line:84% include a little bit of the other one as well. 00:58:18.380 --> 00:58:22.560 align:middle line:84% So if I'm anchoring to what fraction of the time 00:58:22.560 --> 00:58:26.960 align:middle line:84% my friends are spending on reading books, 00:58:26.960 --> 00:58:30.700 align:middle line:84% then I might get tempted to buy more books or less books, 00:58:30.700 --> 00:58:37.120 align:middle line:84% depending on how many times I see folks 00:58:37.120 --> 00:58:39.260 align:middle line:84% on social media talking about books. 00:58:39.260 --> 00:58:41.640 align:middle line:90% That's perfectly fine. 00:58:41.640 --> 00:58:42.660 align:middle line:90% I might do that. 00:58:42.660 --> 00:58:46.560 align:middle line:90% 00:58:46.560 --> 00:58:54.120 align:middle line:84% The other thing that is a little bit conceptually unclear to me 00:58:54.120 --> 00:58:57.160 align:middle line:84% here, you have these kind of low frequency versus high frequency 00:58:57.160 --> 00:58:58.480 align:middle line:90% readers. 00:58:58.480 --> 00:59:03.380 align:middle line:84% It feels like this is a built-in difference between two groups. 00:59:03.380 --> 00:59:07.120 align:middle line:84% It's like you have a group that is genetically even. 00:59:07.120 --> 00:59:08.360 align:middle line:90% It's very hard coded. 00:59:08.360 --> 00:59:10.020 align:middle line:84% I'm not saying you're saying that. 00:59:10.020 --> 00:59:12.440 align:middle line:90% But it's hard coded. 00:59:12.440 --> 00:59:15.920 align:middle line:84% And then when they decide how much of their time 00:59:15.920 --> 00:59:17.980 align:middle line:84% they are spending on reading books, 00:59:17.980 --> 00:59:21.540 align:middle line:84% they go and look at the people who are not high frequency 00:59:21.540 --> 00:59:25.360 align:middle line:84% readers, and they look at them and say, OK, 00:59:25.360 --> 00:59:31.080 align:middle line:84% this person is doing two hours per day, per week, 00:59:31.080 --> 00:59:33.180 align:middle line:90% or whatever is the number. 00:59:33.180 --> 00:59:37.180 align:middle line:84% And that two hours is 100% of what they 00:59:37.180 --> 00:59:42.780 align:middle line:84% would do given their internal interests or capacity 00:59:42.780 --> 00:59:43.700 align:middle line:90% or whatever. 00:59:43.700 --> 00:59:46.660 align:middle line:84% So because it is 100%, now, I should 00:59:46.660 --> 00:59:50.140 align:middle line:90% do 100% of my 10 hour per week. 00:59:50.140 --> 00:59:52.900 align:middle line:84% That's, again, I feel a little bit 00:59:52.900 --> 00:59:56.840 align:middle line:84% uncomfortable with making that connection, 00:59:56.840 --> 01:00:00.400 align:middle line:84% that strong assumption across the groups. 01:00:00.400 --> 01:00:02.480 align:middle line:84% AUDIENCE: So I should keep average reading level? 01:00:02.480 --> 01:00:04.580 align:middle line:84% HAZHIR RAHMANDAD: So it's a little bit tricky. 01:00:04.580 --> 01:00:07.340 align:middle line:84% How you would need to deal with it is not obvious to me. 01:00:07.340 --> 01:00:10.060 align:middle line:84% Because my first-order response is, 01:00:10.060 --> 01:00:11.980 align:middle line:84% even if people are inherently very 01:00:11.980 --> 01:00:15.770 align:middle line:84% different in their propensity to read books, 01:00:15.770 --> 01:00:17.350 align:middle line:84% when they look at the other person, 01:00:17.350 --> 01:00:21.290 align:middle line:84% they wouldn't know if you're from the high propensity 01:00:21.290 --> 01:00:23.450 align:middle line:90% or low propensity by default. 01:00:23.450 --> 01:00:25.870 align:middle line:84% And they wouldn't adjust for that even if they knew that, 01:00:25.870 --> 01:00:29.190 align:middle line:84% oh, yeah, this friend of mine reads a lot of books, 01:00:29.190 --> 01:00:31.490 align:middle line:84% and this other friend doesn't read a lot of books. 01:00:31.490 --> 01:00:34.330 align:middle line:84% They wouldn't adjust for it as, OK, this 01:00:34.330 --> 01:00:39.470 align:middle line:84% is an internal capability, so I should discount it that way. 01:00:39.470 --> 01:00:44.070 align:middle line:84% So I may just go with just look at the average of the hours, 01:00:44.070 --> 01:00:47.970 align:middle line:84% not fraction of time, and use that hour as the driver. 01:00:47.970 --> 01:00:50.330 align:middle line:84% Now, you may have some inherent interest 01:00:50.330 --> 01:00:52.790 align:middle line:84% in books versus those who are not interested. 01:00:52.790 --> 01:00:58.810 align:middle line:84% And that becomes another anchor that is my underlying anchor is, 01:00:58.810 --> 01:01:03.250 align:middle line:84% I enjoy reading books 10 hours per week and this other person 01:01:03.250 --> 01:01:06.030 align:middle line:84% enjoys reading books 30 minutes per week. 01:01:06.030 --> 01:01:08.270 align:middle line:90% And that is a separate anchor. 01:01:08.270 --> 01:01:11.310 align:middle line:84% And then you have some kind of averaging between these anchors 01:01:11.310 --> 01:01:14.790 align:middle line:90% perhaps, or something like that. 01:01:14.790 --> 01:01:17.470 align:middle line:84% So in both of these cases, this is really 01:01:17.470 --> 01:01:20.810 align:middle line:84% about thinking about these mechanisms. 01:01:20.810 --> 01:01:24.070 align:middle line:84% That is, you see how we are bringing out 01:01:24.070 --> 01:01:27.270 align:middle line:84% what the equation is telling about the underlying 01:01:27.270 --> 01:01:28.290 align:middle line:90% assumptions. 01:01:28.290 --> 01:01:30.190 align:middle line:84% You need to think about those assumptions 01:01:30.190 --> 01:01:33.250 align:middle line:84% and make sure you feel those are good assumptions, 01:01:33.250 --> 01:01:34.730 align:middle line:84% those are defensible assumptions. 01:01:34.730 --> 01:01:39.230 align:middle line:90% That is what is good modeling. 01:01:39.230 --> 01:01:40.070 align:middle line:90% Yeah. 01:01:40.070 --> 01:01:41.590 align:middle line:90% AUDIENCE: OK. 01:01:41.590 --> 01:01:44.030 align:middle line:90% HAZHIR RAHMANDAD: OK, very good. 01:01:44.030 --> 01:01:47.230 align:middle line:90% Any questions? 01:01:47.230 --> 01:01:49.710 align:middle line:84% AUDIENCE: Yeah, I think I wrote a little blurb 01:01:49.710 --> 01:01:52.530 align:middle line:84% on the upper left, proposed next steps. 01:01:52.530 --> 01:01:57.270 align:middle line:84% But I think I am open to feedback about 01:01:57.270 --> 01:01:58.510 align:middle line:90% whether I should-- 01:01:58.510 --> 01:01:59.468 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:01:59.468 --> 01:02:01.230 align:middle line:84% So it's really, I think this was more 01:02:01.230 --> 01:02:05.190 align:middle line:90% of a problem for this course. 01:02:05.190 --> 01:02:13.170 align:middle line:84% I don't think this is strong enough a problem for a research 01:02:13.170 --> 01:02:16.850 align:middle line:84% project, so I wouldn't be spending too much time on this 01:02:16.850 --> 01:02:17.690 align:middle line:90% at this point. 01:02:17.690 --> 01:02:19.970 align:middle line:84% This is good you got the learning experience 01:02:19.970 --> 01:02:23.210 align:middle line:84% you wanted out of it, but I would probably not 01:02:23.210 --> 01:02:25.510 align:middle line:90% spend much more time on that. 01:02:25.510 --> 01:02:28.290 align:middle line:84% It is a good idea to try to still improve it 01:02:28.290 --> 01:02:31.230 align:middle line:84% based on the feedback, so one more round is perfectly fine. 01:02:31.230 --> 01:02:35.570 align:middle line:84% But beyond that, I wouldn't spend time on this. 01:02:35.570 --> 01:02:37.410 align:middle line:90% OK. 01:02:37.410 --> 01:02:41.190 align:middle line:84% Let's see, let's look at Jason's. 01:02:41.190 --> 01:02:43.955 align:middle line:90% 01:02:43.955 --> 01:02:45.850 align:middle line:84% AUDIENCE: I'm back to one flight. 01:02:45.850 --> 01:02:47.850 align:middle line:90% HAZHIR RAHMANDAD: OK. 01:02:47.850 --> 01:02:49.290 align:middle line:90% AUDIENCE: Yeah. 01:02:49.290 --> 01:02:51.650 align:middle line:84% So we do have two separate categories 01:02:51.650 --> 01:02:54.810 align:middle line:84% of shoppers that are used in the industry 01:02:54.810 --> 01:02:59.030 align:middle line:84% to identify different booking patterns. 01:02:59.030 --> 01:03:02.010 align:middle line:84% Future shoppers are trying to book 01:03:02.010 --> 01:03:05.370 align:middle line:84% earlier before the flight business comes closer 01:03:05.370 --> 01:03:07.192 align:middle line:84% to departure and is willing to pay more. 01:03:07.192 --> 01:03:08.150 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:03:08.150 --> 01:03:12.790 align:middle line:84% AUDIENCE: And as a result, the revenue manager's job 01:03:12.790 --> 01:03:16.550 align:middle line:84% is to project some fraction of the seats 01:03:16.550 --> 01:03:19.310 align:middle line:84% on the aircraft for that demand that they're 01:03:19.310 --> 01:03:22.110 align:middle line:90% expecting to come later. 01:03:22.110 --> 01:03:32.310 align:middle line:84% So I built a model that kind of runs forward with that setup. 01:03:32.310 --> 01:03:35.870 align:middle line:84% There's no feedback yet on the forecast. 01:03:35.870 --> 01:03:38.310 align:middle line:84% That's the direction I want to take this. 01:03:38.310 --> 01:03:40.730 align:middle line:84% It's based on the realized demand. 01:03:40.730 --> 01:03:42.835 align:middle line:84% But we need to see the next step. 01:03:42.835 --> 01:03:43.710 align:middle line:90% HAZHIR RAHMANDAD: OK. 01:03:43.710 --> 01:03:46.050 align:middle line:90% So let's dig in. 01:03:46.050 --> 01:03:48.950 align:middle line:84% So you have the leisure shopping. 01:03:48.950 --> 01:03:53.690 align:middle line:84% You just have some exogenous arrival of people per day 01:03:53.690 --> 01:03:57.830 align:middle line:84% who are interested in buying that flight. 01:03:57.830 --> 01:04:02.270 align:middle line:84% And then you have the business shoppers 01:04:02.270 --> 01:04:11.080 align:middle line:84% going through another much closer to the flight date. 01:04:11.080 --> 01:04:15.840 align:middle line:84% Pretty sharp demand going up and coming down here. 01:04:15.840 --> 01:04:18.720 align:middle line:84% Perhaps sharper than it is in reality, but that's all right. 01:04:18.720 --> 01:04:21.620 align:middle line:90% You can change those easily. 01:04:21.620 --> 01:04:27.940 align:middle line:84% Then, this leads to total shopping. 01:04:27.940 --> 01:04:32.060 align:middle line:84% And then you have a fulfillment fraction. 01:04:32.060 --> 01:04:35.192 align:middle line:84% AUDIENCE: The idea is the business shopper is flexible, 01:04:35.192 --> 01:04:36.900 align:middle line:84% so they're willing to buy down if there's 01:04:36.900 --> 01:04:38.580 align:middle line:90% a cheaper pair available. 01:04:38.580 --> 01:04:44.020 align:middle line:84% So they'll purchase the cheapest thing on the shelf that's 01:04:44.020 --> 01:04:46.120 align:middle line:90% available when they go to book. 01:04:46.120 --> 01:04:50.700 align:middle line:90% 01:04:50.700 --> 01:04:53.500 align:middle line:84% So you can't distinguish between business and leisure 01:04:53.500 --> 01:04:54.658 align:middle line:90% when they materialize. 01:04:54.658 --> 01:04:57.200 align:middle line:84% HAZHIR RAHMANDAD: So for now, let's leave the left-hand side, 01:04:57.200 --> 01:04:58.440 align:middle line:90% the demand on one side. 01:04:58.440 --> 01:05:02.100 align:middle line:84% Let's go and see what's happening with the bookings. 01:05:02.100 --> 01:05:07.080 align:middle line:84% So you have low-fare and high-fare seats 01:05:07.080 --> 01:05:09.757 align:middle line:84% that are remaining and then those that are booked. 01:05:09.757 --> 01:05:10.340 align:middle line:90% AUDIENCE: Yes. 01:05:10.340 --> 01:05:17.360 align:middle line:90% 01:05:17.360 --> 01:05:20.640 align:middle line:84% HAZHIR RAHMANDAD: And then there could 01:05:20.640 --> 01:05:26.920 align:middle line:84% be movement between low fare versus high fare based 01:05:26.920 --> 01:05:33.943 align:middle line:84% on what the airline thinks is the demand. 01:05:33.943 --> 01:05:35.360 align:middle line:84% AUDIENCE: And they can reallocate. 01:05:35.360 --> 01:05:36.943 align:middle line:84% HAZHIR RAHMANDAD: They can reallocate. 01:05:36.943 --> 01:05:37.740 align:middle line:90% OK, good. 01:05:37.740 --> 01:05:41.880 align:middle line:84% So for now, let's leave out reallocation as well 01:05:41.880 --> 01:05:48.240 align:middle line:84% and let's focus on the booking rate for low fare. 01:05:48.240 --> 01:05:59.020 align:middle line:84% So booking rate for low fare comes from leisure and business. 01:05:59.020 --> 01:06:01.640 align:middle line:90% 01:06:01.640 --> 01:06:18.700 align:middle line:84% And low-fare bookings are some fulfillment fraction 01:06:18.700 --> 01:06:25.346 align:middle line:84% for low fare and low-fare fulfillment fraction is? 01:06:25.346 --> 01:06:27.320 align:middle line:84% AUDIENCE: Just shipping from an inventory. 01:06:27.320 --> 01:06:32.980 align:middle line:90% 01:06:32.980 --> 01:06:35.560 align:middle line:84% Since its capacity constrained things, 01:06:35.560 --> 01:06:40.900 align:middle line:84% there's 100 low-fare seats remaining, 01:06:40.900 --> 01:06:43.960 align:middle line:90% we compare that to the desired-- 01:06:43.960 --> 01:06:47.900 align:middle line:90% 01:06:47.900 --> 01:06:50.980 align:middle line:84% HAZHIR RAHMANDAD: So this is kind of a max low-fare booking 01:06:50.980 --> 01:06:57.860 align:middle line:90% rate that is driving it. 01:06:57.860 --> 01:07:06.450 align:middle line:84% So that is really the main driver 01:07:06.450 --> 01:07:11.330 align:middle line:84% unless you don't have enough shoppers. 01:07:11.330 --> 01:07:13.530 align:middle line:84% AUDIENCE: Yeah, because if you have seats left, 01:07:13.530 --> 01:07:15.495 align:middle line:84% everyone's going to drop in the-- 01:07:15.495 --> 01:07:17.370 align:middle line:84% HAZHIR RAHMANDAD: Yeah, so this is really not 01:07:17.370 --> 01:07:20.890 align:middle line:90% even incorporating price yet. 01:07:20.890 --> 01:07:24.190 align:middle line:84% So low fare is just a fixed price, whatever it is. 01:07:24.190 --> 01:07:24.890 align:middle line:90% AUDIENCE: Right. 01:07:24.890 --> 01:07:26.690 align:middle line:84% But we do have a separate category, 01:07:26.690 --> 01:07:28.390 align:middle line:90% shoppers willing to pay more. 01:07:28.390 --> 01:07:30.450 align:middle line:84% And if in fact low fare is sold out, 01:07:30.450 --> 01:07:32.513 align:middle line:84% then they'll drop in that high fare. 01:07:32.513 --> 01:07:33.930 align:middle line:84% HAZHIR RAHMANDAD: And this is kind 01:07:33.930 --> 01:07:38.810 align:middle line:84% of a minimum booking time that drives the max low-fare booking 01:07:38.810 --> 01:07:41.450 align:middle line:90% rate. 01:07:41.450 --> 01:07:43.630 align:middle line:84% And that gives the fulfillment fraction. 01:07:43.630 --> 01:07:47.730 align:middle line:84% Fulfillment fraction is applying to both 01:07:47.730 --> 01:07:49.850 align:middle line:90% of the low-fare and business. 01:07:49.850 --> 01:07:53.570 align:middle line:84% AUDIENCE: Proportionally allocated, yes. 01:07:53.570 --> 01:07:56.010 align:middle line:90% So I could redo this. 01:07:56.010 --> 01:07:59.270 align:middle line:84% HAZHIR RAHMANDAD: No, that part is fine. 01:07:59.270 --> 01:08:02.270 align:middle line:84% So now that gets the bookings and then 01:08:02.270 --> 01:08:07.070 align:middle line:90% the remainder of the high-fare? 01:08:07.070 --> 01:08:09.050 align:middle line:84% AUDIENCE: Goes to draw from the high-fare pool. 01:08:09.050 --> 01:08:12.190 align:middle line:90% 01:08:12.190 --> 01:08:15.250 align:middle line:84% If their demand wasn't satisfied from the low-fare stock, 01:08:15.250 --> 01:08:19.870 align:middle line:84% they'll look at the high-fare stock. 01:08:19.870 --> 01:08:22.990 align:middle line:84% HAZHIR RAHMANDAD: And then fulfillment ratio in this case 01:08:22.990 --> 01:08:26.819 align:middle line:84% goes boom, boom, from 0 to 1, 1 to 0. 01:08:26.819 --> 01:08:28.069 align:middle line:90% AUDIENCE: One fare's sold out. 01:08:28.069 --> 01:08:31.029 align:middle line:84% HAZHIR RAHMANDAD: It's pretty fast shifting here. 01:08:31.029 --> 01:08:31.910 align:middle line:90% AUDIENCE: For now. 01:08:31.910 --> 01:08:34.069 align:middle line:90% HAZHIR RAHMANDAD: For now. 01:08:34.069 --> 01:08:34.569 align:middle line:90% OK. 01:08:34.569 --> 01:08:36.810 align:middle line:90% So then let me go back. 01:08:36.810 --> 01:08:39.790 align:middle line:84% So you have the low-fare booking, residual business 01:08:39.790 --> 01:08:40.410 align:middle line:90% shopping. 01:08:40.410 --> 01:08:45.069 align:middle line:84% This is those who are not satisfied 01:08:45.069 --> 01:08:47.957 align:middle line:90% because they didn't get it. 01:08:47.957 --> 01:08:49.790 align:middle line:84% AUDIENCE: They would have bought a low fare, 01:08:49.790 --> 01:08:50.832 align:middle line:90% but there were none left. 01:08:50.832 --> 01:08:52.512 align:middle line:90% So now they have no choice. 01:08:52.512 --> 01:08:53.470 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:08:53.470 --> 01:08:57.930 align:middle line:84% So this is why you have this residual folks buying 01:08:57.930 --> 01:08:58.970 align:middle line:90% the high fare. 01:08:58.970 --> 01:09:05.450 align:middle line:84% That brings down the high fare and that takes care of the rest. 01:09:05.450 --> 01:09:12.450 align:middle line:84% OK, so I think the basic is making sense. 01:09:12.450 --> 01:09:16.490 align:middle line:90% 01:09:16.490 --> 01:09:20.490 align:middle line:84% So if you have only two price categories, 01:09:20.490 --> 01:09:23.970 align:middle line:90% this is the way to do it. 01:09:23.970 --> 01:09:30.189 align:middle line:84% If you have n price categories, then you have a cascade. 01:09:30.189 --> 01:09:31.250 align:middle line:90% AUDIENCE: Yes. 01:09:31.250 --> 01:09:33.590 align:middle line:84% HAZHIR RAHMANDAD: So basically the same idea, 01:09:33.590 --> 01:09:37.649 align:middle line:84% but you start from the top, you allocate it 01:09:37.649 --> 01:09:41.069 align:middle line:84% among anybody who is interested in that price range. 01:09:41.069 --> 01:09:45.170 align:middle line:84% So you may have multiple demand groups, basically. 01:09:45.170 --> 01:09:48.550 align:middle line:84% So you have multiple demand groups and multiple fare groups. 01:09:48.550 --> 01:09:52.656 align:middle line:90% 01:09:52.656 --> 01:09:56.870 align:middle line:84% All the demand groups are not interested in all the fares. 01:09:56.870 --> 01:10:03.750 align:middle line:84% So you start by allocating it, the first, the cheapest 01:10:03.750 --> 01:10:08.430 align:middle line:84% among all of them, then the next cheapest among those 01:10:08.430 --> 01:10:09.790 align:middle line:90% who are interested. 01:10:09.790 --> 01:10:14.110 align:middle line:84% And usually, you can make the two groups the same 01:10:14.110 --> 01:10:16.870 align:middle line:90% number just to keep things. 01:10:16.870 --> 01:10:19.070 align:middle line:84% AUDIENCE: Separate passenger groups as fare groups. 01:10:19.070 --> 01:10:20.710 align:middle line:84% HAZHIR RAHMANDAD: As fare groups, 01:10:20.710 --> 01:10:26.790 align:middle line:84% so that you basically take out folks in the second passenger 01:10:26.790 --> 01:10:29.830 align:middle line:90% group. 01:10:29.830 --> 01:10:31.390 align:middle line:84% The second passenger group is not 01:10:31.390 --> 01:10:38.930 align:middle line:84% going to buy anything other than the cheapest. 01:10:38.930 --> 01:10:40.443 align:middle line:90% That's the idea. 01:10:40.443 --> 01:10:41.110 align:middle line:90% AUDIENCE: Right. 01:10:41.110 --> 01:10:46.170 align:middle line:84% HAZHIR RAHMANDAD: And then the cheapest price group. 01:10:46.170 --> 01:10:47.385 align:middle line:90% AUDIENCE: Right. 01:10:47.385 --> 01:10:49.510 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:10:49.510 --> 01:10:53.190 align:middle line:84% And so on, so that would become-- 01:10:53.190 --> 01:11:01.300 align:middle line:84% and then if you do it that way and you do it that robustly, 01:11:01.300 --> 01:11:04.360 align:middle line:90% then this works well. 01:11:04.360 --> 01:11:07.740 align:middle line:84% But what you are missing is still is 01:11:07.740 --> 01:11:10.520 align:middle line:84% the fact that people don't completely go away. 01:11:10.520 --> 01:11:13.740 align:middle line:84% That is, that lowest passenger group may stick around, 01:11:13.740 --> 01:11:17.020 align:middle line:90% may come back occasionally. 01:11:17.020 --> 01:11:21.900 align:middle line:84% So you would have basically these two stocks 01:11:21.900 --> 01:11:27.740 align:middle line:84% of people who are still buying and people 01:11:27.740 --> 01:11:29.400 align:middle line:90% who are already satisfied. 01:11:29.400 --> 01:11:33.660 align:middle line:84% So the total demand goes into those who are buying 01:11:33.660 --> 01:11:38.560 align:middle line:84% or could at the beginning go directly into the demand. 01:11:38.560 --> 01:11:42.640 align:middle line:84% And then when they are not satisfied, 01:11:42.640 --> 01:11:44.560 align:middle line:84% they come back to this waiting stock. 01:11:44.560 --> 01:11:48.760 align:middle line:84% And with the time constant, they come back into trying to buy. 01:11:48.760 --> 01:11:52.060 align:middle line:90% So that would be the structure. 01:11:52.060 --> 01:11:59.280 align:middle line:84% And then the management of fares based on these protection rates 01:11:59.280 --> 01:12:00.140 align:middle line:90% is the rest. 01:12:00.140 --> 01:12:02.200 align:middle line:84% It's how many of the seats do you 01:12:02.200 --> 01:12:04.040 align:middle line:90% allocate to this versus that. 01:12:04.040 --> 01:12:07.320 align:middle line:84% And that question is, what is the revenue 01:12:07.320 --> 01:12:10.013 align:middle line:90% managers in the system? 01:12:10.013 --> 01:12:10.680 align:middle line:90% AUDIENCE: Right. 01:12:10.680 --> 01:12:15.900 align:middle line:84% And that decision is a dual process. 01:12:15.900 --> 01:12:20.160 align:middle line:84% There's a kind of output from a forecasting, 01:12:20.160 --> 01:12:25.420 align:middle line:84% from an optimization procedure, that says, 01:12:25.420 --> 01:12:28.280 align:middle line:84% OK, given the value of these fares 01:12:28.280 --> 01:12:32.660 align:middle line:84% and the expected demand that's coming, 01:12:32.660 --> 01:12:35.680 align:middle line:84% this is the right number to hold back such that we're 01:12:35.680 --> 01:12:38.480 align:middle line:84% indifferent from taking a low fare now to taking 01:12:38.480 --> 01:12:42.520 align:middle line:90% a high fare a week from now. 01:12:42.520 --> 01:12:44.560 align:middle line:84% But there's also behavioral element too, 01:12:44.560 --> 01:12:47.720 align:middle line:84% where an individual can adjust that up or down 01:12:47.720 --> 01:12:49.580 align:middle line:90% based on their feelings. 01:12:49.580 --> 01:12:52.060 align:middle line:84% And then those behavioral decisions 01:12:52.060 --> 01:12:54.160 align:middle line:90% get baked into future forecasts. 01:12:54.160 --> 01:12:55.900 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:12:55.900 --> 01:12:56.940 align:middle line:90% Yep. 01:12:56.940 --> 01:12:57.520 align:middle line:90% OK. 01:12:57.520 --> 01:13:01.820 align:middle line:84% So right now the adjustment. you have here is, 01:13:01.820 --> 01:13:04.240 align:middle line:90% you have a remaining high fare. 01:13:04.240 --> 01:13:06.860 align:middle line:90% 01:13:06.860 --> 01:13:10.300 align:middle line:84% AUDIENCE: Right now, it's just whatever my expected bookings 01:13:10.300 --> 01:13:10.800 align:middle line:90% are. 01:13:10.800 --> 01:13:15.100 align:middle line:84% And then I put in a multiplier in case I want to buffer it. 01:13:15.100 --> 01:13:18.300 align:middle line:84% If I feel really that a high fare is a lot more 01:13:18.300 --> 01:13:20.660 align:middle line:84% than the low fare, then it's probably 01:13:20.660 --> 01:13:25.020 align:middle line:84% worth my time to bump up and protect even more seats. 01:13:25.020 --> 01:13:26.760 align:middle line:90% This is a very heuristic rule. 01:13:26.760 --> 01:13:35.640 align:middle line:84% In reality, they do an expected marginal seat revenue thing. 01:13:35.640 --> 01:13:39.863 align:middle line:90% 01:13:39.863 --> 01:13:41.780 align:middle line:84% HAZHIR RAHMANDAD: So basically between the gap 01:13:41.780 --> 01:13:50.000 align:middle line:84% between high fare booked versus what you 01:13:50.000 --> 01:13:52.880 align:middle line:90% expected to book at this time? 01:13:52.880 --> 01:13:55.560 align:middle line:84% Or high fare remaining versus what you expected 01:13:55.560 --> 01:13:57.440 align:middle line:90% to be remaining at this time? 01:13:57.440 --> 01:14:00.940 align:middle line:84% AUDIENCE: Yeah, right now, the forecast is static. 01:14:00.940 --> 01:14:04.660 align:middle line:84% So I'm saying I expect 35 units of popular demand. 01:14:04.660 --> 01:14:06.840 align:middle line:84% HAZHIR RAHMANDAD: Well, what does 01:14:06.840 --> 01:14:11.820 align:middle line:84% I expect 35 units of high-fare demand mean? 01:14:11.820 --> 01:14:15.880 align:middle line:84% A total integral under the curve of that? 01:14:15.880 --> 01:14:17.260 align:middle line:90% AUDIENCE: Total for this flight. 01:14:17.260 --> 01:14:20.960 align:middle line:84% And then every time I get one of those bookings, 01:14:20.960 --> 01:14:24.840 align:middle line:84% I say my demand remaining is 35 minus however 01:14:24.840 --> 01:14:26.160 align:middle line:90% many I have right now. 01:14:26.160 --> 01:14:27.660 align:middle line:90% HAZHIR RAHMANDAD: OK. 01:14:27.660 --> 01:14:37.160 align:middle line:84% And how do I decide that is good or bad at any point in time? 01:14:37.160 --> 01:14:40.360 align:middle line:90% AUDIENCE: I need more. 01:14:40.360 --> 01:14:43.360 align:middle line:84% So in this single flight model, you 01:14:43.360 --> 01:14:46.462 align:middle line:84% can't unless you have a comparable booking curve. 01:14:46.462 --> 01:14:49.170 align:middle line:84% HAZHIR RAHMANDAD: Basically, you need a booking curve comparison. 01:14:49.170 --> 01:14:51.110 align:middle line:84% AUDIENCE: Yeah, so that's why last time I 01:14:51.110 --> 01:14:53.850 align:middle line:90% had multiple flights. 01:14:53.850 --> 01:14:55.190 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:14:55.190 --> 01:14:58.810 align:middle line:84% Well, you can have a booking curve comparison here as well. 01:14:58.810 --> 01:14:59.737 align:middle line:90% That's not a problem. 01:14:59.737 --> 01:15:01.070 align:middle line:90% AUDIENCE: It would be exogenous. 01:15:01.070 --> 01:15:02.987 align:middle line:84% HAZHIR RAHMANDAD: Yeah, it would be exogenous. 01:15:02.987 --> 01:15:08.150 align:middle line:84% But the thing is that right now the time doesn't play a role. 01:15:08.150 --> 01:15:12.510 align:middle line:84% And that is a little bit confusing. 01:15:12.510 --> 01:15:20.990 align:middle line:84% I expect if I have sold 5 high-fare seats out of the 35, 01:15:20.990 --> 01:15:25.470 align:middle line:84% by a day before the flight, I should be really worried. 01:15:25.470 --> 01:15:28.550 align:middle line:84% But I have sold 5 high-first seats a year 01:15:28.550 --> 01:15:31.030 align:middle line:84% before that, I would be very happy. 01:15:31.030 --> 01:15:34.110 align:middle line:84% AUDIENCE: So if I want an exogenous booking curve, 01:15:34.110 --> 01:15:39.310 align:middle line:90% how would I do that technically? 01:15:39.310 --> 01:15:42.230 align:middle line:84% HAZHIR RAHMANDAD: Any exogenous input 01:15:42.230 --> 01:15:45.110 align:middle line:84% can do that for you or any of the methods 01:15:45.110 --> 01:15:48.590 align:middle line:84% you have here for creating these exogenous, anything works. 01:15:48.590 --> 01:15:49.450 align:middle line:90% I don't know. 01:15:49.450 --> 01:15:52.030 align:middle line:90% Yeah, it is a cumulative curve. 01:15:52.030 --> 01:15:56.370 align:middle line:84% So basically it is cumulative number of high fares 01:15:56.370 --> 01:15:59.810 align:middle line:84% that you have sold in the past as a function of time 01:15:59.810 --> 01:16:01.470 align:middle line:90% before thing. 01:16:01.470 --> 01:16:04.350 align:middle line:90% So that is your reference curve. 01:16:04.350 --> 01:16:05.930 align:middle line:84% And you're always comparing how many 01:16:05.930 --> 01:16:08.630 align:middle line:84% I have booked so far versus that reference curve. 01:16:08.630 --> 01:16:15.230 align:middle line:84% If you are above that, you have booked more, 01:16:15.230 --> 01:16:18.470 align:middle line:84% you would shift more high-fare seats because demand is good. 01:16:18.470 --> 01:16:20.875 align:middle line:90% 01:16:20.875 --> 01:16:22.250 align:middle line:84% AUDIENCE: So I can use this model 01:16:22.250 --> 01:16:26.490 align:middle line:84% to generate reference curves and then feed those reference 01:16:26.490 --> 01:16:26.990 align:middle line:90% curves? 01:16:26.990 --> 01:16:28.948 align:middle line:84% HAZHIR RAHMANDAD: You could, but you could also 01:16:28.948 --> 01:16:32.650 align:middle line:84% think a little bit harder about it 01:16:32.650 --> 01:16:39.530 align:middle line:84% and come up with what is the distribution that you are using 01:16:39.530 --> 01:16:46.350 align:middle line:84% for your leisure shopping and for your business shopping that 01:16:46.350 --> 01:16:49.390 align:middle line:84% is your high-fare and low-fare demand. 01:16:49.390 --> 01:16:51.210 align:middle line:84% So think of them as distributions, 01:16:51.210 --> 01:16:55.630 align:middle line:84% think of their cumulative distribution formulation, 01:16:55.630 --> 01:16:56.630 align:middle line:90% and use that. 01:16:56.630 --> 01:17:01.250 align:middle line:84% And then create small deviations in the realized demand 01:17:01.250 --> 01:17:04.330 align:middle line:84% so that they are initially consistent. 01:17:04.330 --> 01:17:08.130 align:middle line:90% 01:17:08.130 --> 01:17:10.650 align:middle line:84% So it could even be the same distribution, 01:17:10.650 --> 01:17:13.550 align:middle line:84% but on the left-hand side, you can have it as stochastic. 01:17:13.550 --> 01:17:16.190 align:middle line:84% That is, the realized demand is coming 01:17:16.190 --> 01:17:18.210 align:middle line:84% from the same underlying distribution, 01:17:18.210 --> 01:17:19.410 align:middle line:90% but it is a stochastic. 01:17:19.410 --> 01:17:21.965 align:middle line:90% So you use a random number. 01:17:21.965 --> 01:17:23.090 align:middle line:90% AUDIENCE: That makes sense. 01:17:23.090 --> 01:17:26.058 align:middle line:84% I have the cumulative curves just coming out of the-- 01:17:26.058 --> 01:17:27.850 align:middle line:84% HAZHIR RAHMANDAD: Yeah, from the potential. 01:17:27.850 --> 01:17:31.270 align:middle line:84% AUDIENCE: And then I could build the stochastic. 01:17:31.270 --> 01:17:33.750 align:middle line:84% So the stochastic part would come off of the rate 01:17:33.750 --> 01:17:37.910 align:middle line:84% and then I just accumulate the stochastic rate. 01:17:37.910 --> 01:17:39.777 align:middle line:84% And then feed that as a reference curve. 01:17:39.777 --> 01:17:42.360 align:middle line:84% HAZHIR RAHMANDAD: You can't feed it back as a reference curve, 01:17:42.360 --> 01:17:45.400 align:middle line:84% because that thing is what is realized this time. 01:17:45.400 --> 01:17:48.240 align:middle line:84% You need the reference to be in place from beforehand. 01:17:48.240 --> 01:17:51.740 align:middle line:84% That's why I'm saying you need to solve that to have 01:17:51.740 --> 01:17:55.820 align:middle line:90% an analytical curve, basically. 01:17:55.820 --> 01:17:59.260 align:middle line:84% And a probability distribution is usually the best. 01:17:59.260 --> 01:18:02.140 align:middle line:84% One that has an analytical cumulative curve 01:18:02.140 --> 01:18:03.740 align:middle line:90% will give you what you need. 01:18:03.740 --> 01:18:05.140 align:middle line:90% AUDIENCE: OK. 01:18:05.140 --> 01:18:06.380 align:middle line:90% HAZHIR RAHMANDAD: Good. 01:18:06.380 --> 01:18:06.880 align:middle line:90% OK. 01:18:06.880 --> 01:18:08.640 align:middle line:84% So I think this is good progress. 01:18:08.640 --> 01:18:10.860 align:middle line:90% This is in the right direction. 01:18:10.860 --> 01:18:13.700 align:middle line:90% The concepts are working. 01:18:13.700 --> 01:18:16.020 align:middle line:84% The part that is really not yet there 01:18:16.020 --> 01:18:19.580 align:middle line:84% is this feedback that is changing the target PL, 01:18:19.580 --> 01:18:22.300 align:middle line:90% protected level. 01:18:22.300 --> 01:18:25.620 align:middle line:84% It is doing something right now, but what it is doing 01:18:25.620 --> 01:18:31.105 align:middle line:84% is not as consistent with the things changing over time. 01:18:31.105 --> 01:18:32.480 align:middle line:84% AUDIENCE: Yeah, it's not dynamic. 01:18:32.480 --> 01:18:37.420 align:middle line:90% 01:18:37.420 --> 01:18:41.040 align:middle line:84% HAZHIR RAHMANDAD: It's not consistent with how they would 01:18:41.040 --> 01:18:42.695 align:middle line:90% probably think in this context. 01:18:42.695 --> 01:18:43.320 align:middle line:90% AUDIENCE: Yeah. 01:18:43.320 --> 01:18:46.520 align:middle line:90% HAZHIR RAHMANDAD: OK, good. 01:18:46.520 --> 01:18:49.760 align:middle line:84% Anything else that you want to discuss on the model? 01:18:49.760 --> 01:18:53.320 align:middle line:84% AUDIENCE: I think before I put in more fare pockets, 01:18:53.320 --> 01:18:55.500 align:middle line:84% because that's more just technical. 01:18:55.500 --> 01:18:57.720 align:middle line:84% HAZHIR RAHMANDAD: Yeah, that's technical. 01:18:57.720 --> 01:19:01.080 align:middle line:84% AUDIENCE: Probably the PL feedback and think clearly 01:19:01.080 --> 01:19:06.400 align:middle line:84% about the process for adjusting that protection 01:19:06.400 --> 01:19:12.120 align:middle line:84% level by comparing the prior year booking. 01:19:12.120 --> 01:19:18.702 align:middle line:90% And then one day, if there's a-- 01:19:18.702 --> 01:19:20.160 align:middle line:84% HAZHIR RAHMANDAD: So you could also 01:19:20.160 --> 01:19:23.780 align:middle line:84% think of this as happening over multiple cycles. 01:19:23.780 --> 01:19:26.320 align:middle line:84% So the first time you start with the distribution 01:19:26.320 --> 01:19:31.080 align:middle line:84% that I talked about, the next time you update the distribution 01:19:31.080 --> 01:19:34.400 align:middle line:90% based on the realized sales. 01:19:34.400 --> 01:19:38.620 align:middle line:84% And this is based on realized bookings, not the demand really. 01:19:38.620 --> 01:19:41.800 align:middle line:84% And then the next time you update it further and further. 01:19:41.800 --> 01:19:44.020 align:middle line:90% Yeah, so you can do that. 01:19:44.020 --> 01:19:45.540 align:middle line:90% AUDIENCE: That's the direction. 01:19:45.540 --> 01:19:49.020 align:middle line:90% HAZHIR RAHMANDAD: Cool, OK. 01:19:49.020 --> 01:19:50.460 align:middle line:90% Great. 01:19:50.460 --> 01:19:55.920 align:middle line:84% So again you have research that is related to these topics. 01:19:55.920 --> 01:19:57.420 align:middle line:84% So I don't know how much this model 01:19:57.420 --> 01:20:00.540 align:middle line:84% will be useful for those research projects. 01:20:00.540 --> 01:20:03.500 align:middle line:84% Don't spend too much time unless it is really 01:20:03.500 --> 01:20:06.900 align:middle line:84% at the heart of a project you're already working on. 01:20:06.900 --> 01:20:13.400 align:middle line:84% Again, this is useful practice for modeling in Vensim setup, 01:20:13.400 --> 01:20:16.700 align:middle line:84% but this is not the easiest way of building models 01:20:16.700 --> 01:20:17.680 align:middle line:90% for this problem. 01:20:17.680 --> 01:20:20.100 align:middle line:84% That's why there is a whole kind of literature 01:20:20.100 --> 01:20:22.820 align:middle line:84% on revenue management that people build 01:20:22.820 --> 01:20:25.200 align:middle line:90% models of different sorts. 01:20:25.200 --> 01:20:29.860 align:middle line:90% 01:20:29.860 --> 01:20:31.580 align:middle line:84% Actually, maybe we should take a break. 01:20:31.580 --> 01:20:33.160 align:middle line:90% I think we're at 10:30. 01:20:33.160 --> 01:20:34.200 align:middle line:90% So let's take a break. 01:20:34.200 --> 01:20:38.240 align:middle line:84% When we come back, we'll look at the remaining couple of models 01:20:38.240 --> 01:20:43.080 align:middle line:84% and then have our general conversation. 01:20:43.080 --> 01:20:46.240 align:middle line:90% OK, let's get back. 01:20:46.240 --> 01:20:53.060 align:middle line:84% So we have Ross and Hassan's models we'll start looking at. 01:20:53.060 --> 01:21:03.680 align:middle line:90% 01:21:03.680 --> 01:21:06.440 align:middle line:90% I have model parameters. 01:21:06.440 --> 01:21:12.680 align:middle line:84% These are the analysis that we discussed last time. 01:21:12.680 --> 01:21:17.160 align:middle line:84% So just as a reminder for our thinking, 01:21:17.160 --> 01:21:25.720 align:middle line:84% we had basically two types of customers, 01:21:25.720 --> 01:21:29.880 align:middle line:84% one that were more price sensitive 01:21:29.880 --> 01:21:31.920 align:middle line:84% and one that was less price sensitive, 01:21:31.920 --> 01:21:36.010 align:middle line:84% and one that was kind of driven by exclusivity 01:21:36.010 --> 01:21:38.550 align:middle line:84% and the other that looked at prestige as, 01:21:38.550 --> 01:21:41.610 align:middle line:84% OK, if the exclusive group have it, maybe I should have it. 01:21:41.610 --> 01:21:43.830 align:middle line:90% That's the basic idea. 01:21:43.830 --> 01:21:51.550 align:middle line:84% And then the firm maximizing profitability was the key idea. 01:21:51.550 --> 01:22:04.710 align:middle line:84% And we started with quantity and price decisions for the firm. 01:22:04.710 --> 01:22:12.070 align:middle line:84% And with that analysis, the profit maximizing option here 01:22:12.070 --> 01:22:16.070 align:middle line:84% comes down at low levels of quantity 01:22:16.070 --> 01:22:18.470 align:middle line:90% and high levels of price. 01:22:18.470 --> 01:22:19.537 align:middle line:90% AUDIENCE: Yeah. 01:22:19.537 --> 01:22:21.370 align:middle line:84% HAZHIR RAHMANDAD: That's pretty interesting. 01:22:21.370 --> 01:22:25.230 align:middle line:84% And actually, the shape of the curvature of this 01:22:25.230 --> 01:22:28.190 align:middle line:90% is also not trivial. 01:22:28.190 --> 01:22:33.810 align:middle line:84% So you can increase quantity supplied 01:22:33.810 --> 01:22:37.010 align:middle line:90% and bring down relative price. 01:22:37.010 --> 01:22:40.850 align:middle line:84% So there is kind of a whole different range 01:22:40.850 --> 01:22:46.370 align:middle line:90% of profitable practices. 01:22:46.370 --> 01:22:50.730 align:middle line:84% Of course, high price and very high quantity 01:22:50.730 --> 01:22:52.970 align:middle line:90% is not good for you. 01:22:52.970 --> 01:22:57.610 align:middle line:84% And it's interesting that for all of these more quantities, 01:22:57.610 --> 01:22:59.390 align:middle line:84% not necessarily good, it of course, 01:22:59.390 --> 01:23:03.210 align:middle line:84% depends on that elasticity assumptions that 01:23:03.210 --> 01:23:04.930 align:middle line:90% go into your model. 01:23:04.930 --> 01:23:06.890 align:middle line:84% AUDIENCE: It's also about the accessibility. 01:23:06.890 --> 01:23:09.558 align:middle line:84% HAZHIR RAHMANDAD: Can you speak up? 01:23:09.558 --> 01:23:11.850 align:middle line:84% AUDIENCE: Oh, I think it's also about the accessibility 01:23:11.850 --> 01:23:16.030 align:middle line:84% of the good that is observed, like on the street, for example, 01:23:16.030 --> 01:23:17.630 align:middle line:90% that affects prestige. 01:23:17.630 --> 01:23:19.065 align:middle line:90% I have comparative slides. 01:23:19.065 --> 01:23:19.940 align:middle line:90% HAZHIR RAHMANDAD: OK. 01:23:19.940 --> 01:23:21.190 align:middle line:90% So let's look at it. 01:23:21.190 --> 01:23:26.670 align:middle line:84% So here you have the prestige as well. 01:23:26.670 --> 01:23:31.010 align:middle line:84% And the higher prestige happens, of course, 01:23:31.010 --> 01:23:39.570 align:middle line:84% with lower quantity supplied and with higher levels of price. 01:23:39.570 --> 01:23:43.430 align:middle line:84% AUDIENCE: And then the one picked is prestige at 0. 01:23:43.430 --> 01:23:45.197 align:middle line:90% It's just ordinary. 01:23:45.197 --> 01:23:47.030 align:middle line:84% HAZHIR RAHMANDAD: OK, so this is really just 01:23:47.030 --> 01:23:50.430 align:middle line:84% to take out the effect of prestige. 01:23:50.430 --> 01:23:55.490 align:middle line:84% Good analysis, so that's a nice way of just saying, OK, 01:23:55.490 --> 01:23:58.130 align:middle line:84% without prestige effects, what would happen? 01:23:58.130 --> 01:24:01.790 align:middle line:84% And the short answer is, either you 01:24:01.790 --> 01:24:05.070 align:middle line:84% pay at high prices with low quantities 01:24:05.070 --> 01:24:08.070 align:middle line:84% or high quantities and low prices. 01:24:08.070 --> 01:24:13.790 align:middle line:84% AUDIENCE: I was trying to look at the optimal strategy, which 01:24:13.790 --> 01:24:15.430 align:middle line:90% is the black dot. 01:24:15.430 --> 01:24:16.870 align:middle line:84% I think the contrast between them 01:24:16.870 --> 01:24:19.230 align:middle line:84% was the thing that I tried to see. 01:24:19.230 --> 01:24:20.730 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:24:20.730 --> 01:24:23.390 align:middle line:84% So you were saying the maximum profit 01:24:23.390 --> 01:24:25.270 align:middle line:90% happens here versus there. 01:24:25.270 --> 01:24:28.250 align:middle line:84% So yeah, I think that shift is also interesting. 01:24:28.250 --> 01:24:34.850 align:middle line:84% And in fact, you could get this as a function 01:24:34.850 --> 01:24:36.070 align:middle line:90% of different parameters. 01:24:36.070 --> 01:24:37.430 align:middle line:90% Is that what you're doing here? 01:24:37.430 --> 01:24:38.055 align:middle line:90% AUDIENCE: Yeah. 01:24:38.055 --> 01:24:40.890 align:middle line:84% So now I'm changing what percentage of the demand 01:24:40.890 --> 01:24:42.630 align:middle line:90% is price sensitive. 01:24:42.630 --> 01:24:45.330 align:middle line:84% So A is like everyone's price sensitive. 01:24:45.330 --> 01:24:48.510 align:middle line:84% And then it drops down to 90% being price sensitive. 01:24:48.510 --> 01:24:52.690 align:middle line:90% 01:24:52.690 --> 01:24:54.690 align:middle line:84% HAZHIR RAHMANDAD: So the first one is 1. 01:24:54.690 --> 01:24:57.530 align:middle line:90% Next one is 95%. 01:24:57.530 --> 01:25:03.680 align:middle line:84% And as you increase the fraction of price sensitive people-- 01:25:03.680 --> 01:25:07.230 align:middle line:90% oh, decrease it, you go up. 01:25:07.230 --> 01:25:09.590 align:middle line:90% But the shift is pretty rapid. 01:25:09.590 --> 01:25:10.770 align:middle line:90% AUDIENCE: Yeah. 01:25:10.770 --> 01:25:13.172 align:middle line:84% That's why I narrowed it down to 90%. 01:25:13.172 --> 01:25:14.130 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:25:14.130 --> 01:25:18.590 align:middle line:84% But do you have any intuition why that shift is so rapid? 01:25:18.590 --> 01:25:21.970 align:middle line:90% 01:25:21.970 --> 01:25:25.530 align:middle line:84% AUDIENCE: I'm not so sure about that. 01:25:25.530 --> 01:25:32.360 align:middle line:84% HAZHIR RAHMANDAD: So some of it is, of course, the flat payoff 01:25:32.360 --> 01:25:33.060 align:middle line:90% landscape. 01:25:33.060 --> 01:25:36.000 align:middle line:84% That is, you can have fairly similar profitability 01:25:36.000 --> 01:25:38.600 align:middle line:90% for a big band. 01:25:38.600 --> 01:25:40.620 align:middle line:90% So that is part of the story. 01:25:40.620 --> 01:25:44.080 align:middle line:84% In fact, here you can see it very clearly. 01:25:44.080 --> 01:25:50.680 align:middle line:84% And therefore, small shifts can shift the optimum quite a bit. 01:25:50.680 --> 01:25:54.840 align:middle line:84% The other part, the more interesting question 01:25:54.840 --> 01:26:04.440 align:middle line:84% is whether the price-sensitive customers, the existence 01:26:04.440 --> 01:26:08.920 align:middle line:84% of this small group of prestige focused 01:26:08.920 --> 01:26:11.180 align:middle line:84% can create a really tipping dynamic. 01:26:11.180 --> 01:26:17.680 align:middle line:84% That is, it activates the prestige dynamics. 01:26:17.680 --> 01:26:20.920 align:middle line:84% And you have a little bit of it here, maybe. 01:26:20.920 --> 01:26:22.680 align:middle line:84% It activates the prestige dynamics 01:26:22.680 --> 01:26:27.940 align:middle line:84% in ways that then brings price-sensitive customers 01:26:27.940 --> 01:26:33.220 align:middle line:84% and makes the optimal price much higher. 01:26:33.220 --> 01:26:34.980 align:middle line:90% AUDIENCE: Can I ask a question? 01:26:34.980 --> 01:26:38.820 align:middle line:84% The profitability, is it percentage 01:26:38.820 --> 01:26:42.020 align:middle line:90% or is it the amount of profit? 01:26:42.020 --> 01:26:44.400 align:middle line:84% AUDIENCE: It's the amount of accumulated profit. 01:26:44.400 --> 01:26:44.900 align:middle line:90% Yeah. 01:26:44.900 --> 01:26:47.917 align:middle line:84% AUDIENCE: So it is with size, too? 01:26:47.917 --> 01:26:49.500 align:middle line:84% AUDIENCE: Yeah, so it's not comparable 01:26:49.500 --> 01:26:52.460 align:middle line:90% across the different panels. 01:26:52.460 --> 01:26:54.920 align:middle line:84% AUDIENCE: Sure, but for one panel it is? 01:26:54.920 --> 01:26:55.660 align:middle line:90% AUDIENCE: Yeah. 01:26:55.660 --> 01:26:57.700 align:middle line:90% AUDIENCE: OK. 01:26:57.700 --> 01:26:59.020 align:middle line:90% HAZHIR RAHMANDAD: Good. 01:26:59.020 --> 01:27:02.740 align:middle line:84% By the way, you have generated these 01:27:02.740 --> 01:27:06.140 align:middle line:84% writing the code with the equations separately outside 01:27:06.140 --> 01:27:06.700 align:middle line:90% of Vensim? 01:27:06.700 --> 01:27:09.210 align:middle line:90% AUDIENCE: Yeah, I used Python. 01:27:09.210 --> 01:27:10.460 align:middle line:90% HAZHIR RAHMANDAD: That's fine. 01:27:10.460 --> 01:27:14.820 align:middle line:84% So just for technical alternatives 01:27:14.820 --> 01:27:17.240 align:middle line:84% that you have at your disposal, this is a simple model. 01:27:17.240 --> 01:27:19.760 align:middle line:84% You could just write the code easily online. 01:27:19.760 --> 01:27:23.020 align:middle line:84% There is a PySD package that takes Vensim models 01:27:23.020 --> 01:27:27.080 align:middle line:84% and turns them into Python code, so that takes the translation 01:27:27.080 --> 01:27:30.500 align:middle line:90% work out of your hair. 01:27:30.500 --> 01:27:35.122 align:middle line:84% Now, that doesn't work perfectly for every function in Vensim. 01:27:35.122 --> 01:27:36.580 align:middle line:84% Some of the more complex functions, 01:27:36.580 --> 01:27:39.880 align:middle line:84% it doesn't have the translation, but it works for most of them. 01:27:39.880 --> 01:27:43.160 align:middle line:84% And the code it generates is not optimized in any way. 01:27:43.160 --> 01:27:45.980 align:middle line:84% So if you want to do some code optimization, 01:27:45.980 --> 01:27:47.900 align:middle line:84% you may need to think about that. 01:27:47.900 --> 01:27:49.320 align:middle line:90% So that's number one. 01:27:49.320 --> 01:27:52.560 align:middle line:84% Number two, you have actually built-in Vensim functionality 01:27:52.560 --> 01:27:56.040 align:middle line:84% to do all these types of analysis, sensitivity analysis, 01:27:56.040 --> 01:28:00.560 align:middle line:84% and generate the data and export the data. 01:28:00.560 --> 01:28:04.360 align:middle line:84% You can automate the generation and exporting of the data 01:28:04.360 --> 01:28:08.080 align:middle line:84% by connecting Vensim with Python. 01:28:08.080 --> 01:28:12.960 align:middle line:84% So there are two different ways to connect Vensim and Python. 01:28:12.960 --> 01:28:16.820 align:middle line:84% One is through DLL, which is dynamic libraries. 01:28:16.820 --> 01:28:20.960 align:middle line:84% And that basically allows Python to have 01:28:20.960 --> 01:28:27.900 align:middle line:84% full control over Vensim, sends commands 01:28:27.900 --> 01:28:30.980 align:middle line:84% that Vensim runs and then gets the results out of it 01:28:30.980 --> 01:28:34.780 align:middle line:90% and processes the data. 01:28:34.780 --> 01:28:43.400 align:middle line:84% And the other is basically using Vensim script language. 01:28:43.400 --> 01:28:47.380 align:middle line:84% So Vensim has a scripting language for generating code 01:28:47.380 --> 01:28:49.700 align:middle line:84% to run Vensim and get data and so on. 01:28:49.700 --> 01:28:55.020 align:middle line:84% And you can basically tell Python to write that code 01:28:55.020 --> 01:28:59.220 align:middle line:84% and then send that code to run Vensim. 01:28:59.220 --> 01:29:01.200 align:middle line:90% Both of those work OK. 01:29:01.200 --> 01:29:03.780 align:middle line:84% If you have a more complex model, 01:29:03.780 --> 01:29:06.540 align:middle line:84% then translation of Vensim to another language 01:29:06.540 --> 01:29:09.780 align:middle line:84% or running the simulations in another language 01:29:09.780 --> 01:29:12.780 align:middle line:84% could become the bottleneck, then these solutions 01:29:12.780 --> 01:29:14.120 align:middle line:90% are more attractive. 01:29:14.120 --> 01:29:16.660 align:middle line:84% So just for what you have done, this is good. 01:29:16.660 --> 01:29:19.540 align:middle line:84% I think this is the right choice. 01:29:19.540 --> 01:29:22.690 align:middle line:84% You always want to do fancy graphing outside of Vensim 01:29:22.690 --> 01:29:25.610 align:middle line:90% anyway. 01:29:25.610 --> 01:29:28.810 align:middle line:84% AUDIENCE: The last one is try to see accessibility 01:29:28.810 --> 01:29:30.170 align:middle line:90% affecting prestige. 01:29:30.170 --> 01:29:33.667 align:middle line:84% So the first is when the sensitivity to accessibility 01:29:33.667 --> 01:29:35.750 align:middle line:84% is 0, so you don't really care how much you see it 01:29:35.750 --> 01:29:38.168 align:middle line:90% on the street, for example. 01:29:38.168 --> 01:29:40.210 align:middle line:84% HAZHIR RAHMANDAD: So sensitivity to accessibility 01:29:40.210 --> 01:29:43.830 align:middle line:84% is 0 means you don't care if others have it or not? 01:29:43.830 --> 01:29:47.890 align:middle line:84% AUDIENCE: Yeah, it doesn't affect the prestige. 01:29:47.890 --> 01:29:49.973 align:middle line:84% You have more wiggle room in terms of that. 01:29:49.973 --> 01:29:50.890 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:29:50.890 --> 01:29:55.150 align:middle line:84% So what that means then in practice, 01:29:55.150 --> 01:29:59.270 align:middle line:84% it brings the quantity a little bit further to the right 01:29:59.270 --> 01:30:05.170 align:middle line:84% because now it doesn't erode the performance. 01:30:05.170 --> 01:30:06.030 align:middle line:90% Makes sense. 01:30:06.030 --> 01:30:08.490 align:middle line:84% AUDIENCE: What effects prestige besides accessibility? 01:30:08.490 --> 01:30:09.210 align:middle line:90% I'm just curious. 01:30:09.210 --> 01:30:11.502 align:middle line:84% AUDIENCE: For this simulation, it's just accessibility, 01:30:11.502 --> 01:30:12.570 align:middle line:90% which is quantity. 01:30:12.570 --> 01:30:15.110 align:middle line:84% And then the price, relative price. 01:30:15.110 --> 01:30:16.990 align:middle line:84% AUDIENCE: So when is accessibility is 0, 01:30:16.990 --> 01:30:18.265 align:middle line:90% it's just price. 01:30:18.265 --> 01:30:18.890 align:middle line:90% AUDIENCE: Yeah. 01:30:18.890 --> 01:30:20.350 align:middle line:90% AUDIENCE: OK. 01:30:20.350 --> 01:30:22.870 align:middle line:90% So that's why it's parallel? 01:30:22.870 --> 01:30:23.992 align:middle line:90% AUDIENCE: Yeah. 01:30:23.992 --> 01:30:24.950 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:30:24.950 --> 01:30:29.190 align:middle line:84% So this time prestige only increases with price. 01:30:29.190 --> 01:30:30.830 align:middle line:84% Does the prestige increase with price 01:30:30.830 --> 01:30:34.470 align:middle line:84% or with fraction of the customers who are 01:30:34.470 --> 01:30:36.270 align:middle line:90% of one type versus the other? 01:30:36.270 --> 01:30:38.650 align:middle line:84% AUDIENCE: Oh, it's going to just increase the price. 01:30:38.650 --> 01:30:41.070 align:middle line:84% But the response to the prestige is going 01:30:41.070 --> 01:30:42.550 align:middle line:90% to have different impacts. 01:30:42.550 --> 01:30:45.922 align:middle line:90% HAZHIR RAHMANDAD: Got it. 01:30:45.922 --> 01:30:50.430 align:middle line:84% And then sensitivity to accessibility when you bring it 01:30:50.430 --> 01:30:54.790 align:middle line:90% up, you see that shift. 01:30:54.790 --> 01:31:00.350 align:middle line:84% It also tightens the range of profitable strategies, 01:31:00.350 --> 01:31:03.390 align:middle line:90% basically, because of lack. 01:31:03.390 --> 01:31:07.590 align:middle line:90% 01:31:07.590 --> 01:31:09.470 align:middle line:90% Very interesting. 01:31:09.470 --> 01:31:10.210 align:middle line:90% Very good. 01:31:10.210 --> 01:31:12.310 align:middle line:90% This is pretty nice analysis. 01:31:12.310 --> 01:31:13.050 align:middle line:90% Good job. 01:31:13.050 --> 01:31:13.950 align:middle line:90% It's kind of clear. 01:31:13.950 --> 01:31:16.970 align:middle line:84% And I think all of the mechanisms make sense, 01:31:16.970 --> 01:31:18.670 align:middle line:90% but they are not trivial. 01:31:18.670 --> 01:31:22.570 align:middle line:84% So you can see that the model is adding some value, 01:31:22.570 --> 01:31:24.910 align:middle line:84% but it is also the right level of complexity. 01:31:24.910 --> 01:31:28.610 align:middle line:90% So this is the right choice. 01:31:28.610 --> 01:31:32.690 align:middle line:84% I think, again, perhaps this is not 01:31:32.690 --> 01:31:34.870 align:middle line:84% something you want to spend too much time on it. 01:31:34.870 --> 01:31:38.490 align:middle line:84% But if you wanted to do more in this direction, 01:31:38.490 --> 01:31:40.770 align:middle line:84% some dynamic policy of what would 01:31:40.770 --> 01:31:45.730 align:middle line:84% be building a new brand, what would be the pathway to building 01:31:45.730 --> 01:31:53.650 align:middle line:84% something profitable and of decent size and impact 01:31:53.650 --> 01:31:56.643 align:middle line:90% is an interesting question. 01:31:56.643 --> 01:31:57.810 align:middle line:90% AUDIENCE: I have a question. 01:31:57.810 --> 01:32:00.410 align:middle line:84% The last time, you said you can't 01:32:00.410 --> 01:32:06.090 align:middle line:84% linear in state variable kind of price policy in Vensim. 01:32:06.090 --> 01:32:08.330 align:middle line:84% And I was wondering how you do that? 01:32:08.330 --> 01:32:11.650 align:middle line:84% HAZHIR RAHMANDAD: So the way for that would be, 01:32:11.650 --> 01:32:17.660 align:middle line:84% you can say, OK, my quantity that I'm going to produce 01:32:17.660 --> 01:32:20.480 align:middle line:84% and my price, either of them, you pick that. 01:32:20.480 --> 01:32:23.900 align:middle line:84% You say it is a function of the state of the system. 01:32:23.900 --> 01:32:28.200 align:middle line:84% The state of the system is time and the number 01:32:28.200 --> 01:32:31.780 align:middle line:84% of customers I have in each of these two groups. 01:32:31.780 --> 01:32:33.720 align:middle line:84% So those are the three variables. 01:32:33.720 --> 01:32:37.000 align:middle line:84% The simplest functional form would be a linear one. 01:32:37.000 --> 01:32:39.780 align:middle line:84% Not sure if it's, in this case, the best choice, 01:32:39.780 --> 01:32:41.980 align:middle line:90% but it's an OK starting point. 01:32:41.980 --> 01:32:44.320 align:middle line:84% I always start with that and then change that. 01:32:44.320 --> 01:32:47.520 align:middle line:84% So it would be a constant plus alpha multiplied 01:32:47.520 --> 01:32:50.240 align:middle line:84% by the number of customers in category 1 01:32:50.240 --> 01:32:54.480 align:middle line:84% plus beta customers in category 2 plus gamma time. 01:32:54.480 --> 01:33:00.120 align:middle line:84% So that becomes your quantity supplied, similarly for price. 01:33:00.120 --> 01:33:08.620 align:middle line:84% Then you do an optimization on those eight parameters 01:33:08.620 --> 01:33:12.200 align:middle line:84% that is maximizing cumulative profits 01:33:12.200 --> 01:33:17.500 align:middle line:84% over the time of the simulation by changing those eight 01:33:17.500 --> 01:33:18.940 align:middle line:90% parameters. 01:33:18.940 --> 01:33:22.980 align:middle line:84% And it will find the right coefficients that 01:33:22.980 --> 01:33:25.260 align:middle line:90% maximizes your profitability. 01:33:25.260 --> 01:33:29.300 align:middle line:84% And those coefficients will generate a dynamic response 01:33:29.300 --> 01:33:34.220 align:middle line:84% in terms of the quantity and price over time. 01:33:34.220 --> 01:33:39.200 align:middle line:84% Now those are linear response strategies. 01:33:39.200 --> 01:33:42.140 align:middle line:84% They are not necessarily great because if there 01:33:42.140 --> 01:33:44.780 align:middle line:84% is any non-linearity that it is better 01:33:44.780 --> 01:33:51.080 align:middle line:84% to get a lot of good reputation, but then at the very end, 01:33:51.080 --> 01:33:58.140 align:middle line:84% bring down my price or do something that expands my market 01:33:58.140 --> 01:34:01.980 align:middle line:84% or whatever, that is not going to be feasible because it 01:34:01.980 --> 01:34:03.840 align:middle line:90% doesn't change it easily. 01:34:03.840 --> 01:34:08.340 align:middle line:84% So then you can have a second-order factor come into it 01:34:08.340 --> 01:34:09.780 align:middle line:90% or something like that. 01:34:09.780 --> 01:34:12.400 align:middle line:90% That's one way to do it. 01:34:12.400 --> 01:34:18.520 align:middle line:84% Another way to do it is to say, oh, my price over time 01:34:18.520 --> 01:34:21.120 align:middle line:90% is going to be dynamic. 01:34:21.120 --> 01:34:26.600 align:middle line:84% And so I define a piecewise linear function 01:34:26.600 --> 01:34:30.800 align:middle line:90% that says my price is changing. 01:34:30.800 --> 01:34:35.640 align:middle line:84% I put 5, 6 points over time and let 01:34:35.640 --> 01:34:38.820 align:middle line:84% those determine my dynamics of price over time 01:34:38.820 --> 01:34:41.680 align:middle line:84% and similarly for quantity over time. 01:34:41.680 --> 01:34:45.440 align:middle line:84% And then I just optimize the coordinates, 01:34:45.440 --> 01:34:49.160 align:middle line:84% let's say the y values, of those 5, 01:34:49.160 --> 01:34:52.160 align:middle line:84% 6 points and then do a linear approximation 01:34:52.160 --> 01:34:57.200 align:middle line:84% between those 5, 6 points to generate my price values. 01:34:57.200 --> 01:34:58.260 align:middle line:90% That's another one. 01:34:58.260 --> 01:35:00.860 align:middle line:84% You can do that inside Vensim as well or elsewhere. 01:35:00.860 --> 01:35:02.980 align:middle line:90% That concept is similar. 01:35:02.980 --> 01:35:05.600 align:middle line:90% 01:35:05.600 --> 01:35:09.560 align:middle line:84% That is an open control policy, open in the sense 01:35:09.560 --> 01:35:14.180 align:middle line:84% that it is not getting feedback from the state of the system. 01:35:14.180 --> 01:35:18.240 align:middle line:84% But for a deterministic model like this, it is fine. 01:35:18.240 --> 01:35:21.520 align:middle line:84% You can go with an open response policy. 01:35:21.520 --> 01:35:25.280 align:middle line:84% It gives the same result as a closed response. 01:35:25.280 --> 01:35:30.380 align:middle line:84% That is, your policy is a function of the state variable. 01:35:30.380 --> 01:35:33.640 align:middle line:90% It gives similar insights. 01:35:33.640 --> 01:35:36.180 align:middle line:84% And in fact, it would be a more nuanced answer 01:35:36.180 --> 01:35:39.300 align:middle line:84% when you do it with this over time linear, piecewise linear, 01:35:39.300 --> 01:35:43.500 align:middle line:84% because it can be pretty nonlinear. 01:35:43.500 --> 01:35:48.060 align:middle line:84% In fact, you can make it even more flexible. 01:35:48.060 --> 01:35:50.260 align:middle line:84% So one way to do the piecewise linear 01:35:50.260 --> 01:35:55.400 align:middle line:84% is to set fixed points over time, so time 10, 20, 30, 40, 01:35:55.400 --> 01:35:59.700 align:middle line:84% whatever, and get the y variables for each of those 01:35:59.700 --> 01:36:02.320 align:middle line:84% and optimize over what is the y value 01:36:02.320 --> 01:36:04.440 align:middle line:84% and then do the piecewise linear. 01:36:04.440 --> 01:36:07.700 align:middle line:84% The other is to make the x also flexible. 01:36:07.700 --> 01:36:12.570 align:middle line:84% So then, if there is a sharp change, 01:36:12.570 --> 01:36:16.430 align:middle line:84% it will bring a few x's in that place and do the sharp change. 01:36:16.430 --> 01:36:19.330 align:middle line:84% And then in other parts, it will be more distant because there 01:36:19.330 --> 01:36:20.690 align:middle line:90% is not much changing. 01:36:20.690 --> 01:36:21.950 align:middle line:90% That's also doable. 01:36:21.950 --> 01:36:25.530 align:middle line:84% Again, these are things with a little bit of manipulation 01:36:25.530 --> 01:36:28.090 align:middle line:90% of subscripts you can do. 01:36:28.090 --> 01:36:33.010 align:middle line:84% Did I mention that I think my 2011 paper in System Dynamics 01:36:33.010 --> 01:36:38.210 align:middle line:84% Review on openness and pricing, optimizing openness and pricing, 01:36:38.210 --> 01:36:40.970 align:middle line:84% has kind of the infrastructure in Vensim 01:36:40.970 --> 01:36:45.930 align:middle line:84% to do that in a differential game setting, if it is worth it. 01:36:45.930 --> 01:36:48.530 align:middle line:84% This is all kind of manipulation of Vensim. 01:36:48.530 --> 01:36:50.830 align:middle line:84% There is nothing conceptually complex. 01:36:50.830 --> 01:36:57.130 align:middle line:84% But that's half an hour of time it can save for you, perhaps. 01:36:57.130 --> 01:37:01.430 align:middle line:84% So those are a few alternative ways to play with that. 01:37:01.430 --> 01:37:04.770 align:middle line:84% I would say all of that is good practice 01:37:04.770 --> 01:37:08.070 align:middle line:84% to get some skills and familiarity 01:37:08.070 --> 01:37:10.350 align:middle line:84% with a different space that in system dynamics 01:37:10.350 --> 01:37:11.890 align:middle line:84% we normally don't talk about a lot. 01:37:11.890 --> 01:37:13.330 align:middle line:90% But it's pretty important. 01:37:13.330 --> 01:37:19.870 align:middle line:84% People who care about designing policies and strategies 01:37:19.870 --> 01:37:23.090 align:middle line:84% are always also bringing these ideas into the picture. 01:37:23.090 --> 01:37:28.510 align:middle line:84% And it's a useful starting point into that space. 01:37:28.510 --> 01:37:29.530 align:middle line:90% Great. 01:37:29.530 --> 01:37:31.050 align:middle line:90% Any other thoughts on this? 01:37:31.050 --> 01:37:41.390 align:middle line:90% 01:37:41.390 --> 01:37:42.050 align:middle line:90% OK. 01:37:42.050 --> 01:37:47.070 align:middle line:90% So let's go to Hassan's model. 01:37:47.070 --> 01:37:48.882 align:middle line:90% So which one should I open? 01:37:48.882 --> 01:37:50.090 align:middle line:90% AUDIENCE: The second version. 01:37:50.090 --> 01:37:56.790 align:middle line:90% 01:37:56.790 --> 01:37:59.650 align:middle line:84% So maybe another view, the first one. 01:37:59.650 --> 01:38:03.407 align:middle line:90% 01:38:03.407 --> 01:38:04.282 align:middle line:90% HAZHIR RAHMANDAD: OK. 01:38:04.282 --> 01:38:10.350 align:middle line:84% So we have low smoking history, population with no nodules; 01:38:10.350 --> 01:38:13.590 align:middle line:84% high smoking history, population without nodules. 01:38:13.590 --> 01:38:17.370 align:middle line:84% So you have low and high smoke levels 01:38:17.370 --> 01:38:22.010 align:middle line:84% and you have the nodule progression from no nodules, 01:38:22.010 --> 01:38:24.770 align:middle line:84% non-detectable, detectable, symptomatic. 01:38:24.770 --> 01:38:29.330 align:middle line:84% AUDIENCE: And then the last four are treatments. 01:38:29.330 --> 01:38:33.290 align:middle line:84% HAZHIR RAHMANDAD: In symptomatic treatment. 01:38:33.290 --> 01:38:34.710 align:middle line:90% TX is treatment? 01:38:34.710 --> 01:38:40.370 align:middle line:90% 01:38:40.370 --> 01:38:43.750 align:middle line:84% So detectable would not be treated, correct? 01:38:43.750 --> 01:38:46.250 align:middle line:84% AUDIENCE: It will be detected if they're screened, but won't 01:38:46.250 --> 01:38:48.050 align:middle line:90% have symptoms. 01:38:48.050 --> 01:38:51.530 align:middle line:84% HAZHIR RAHMANDAD: Yes, but could they be treated or not? 01:38:51.530 --> 01:38:56.010 align:middle line:84% So then you would want detectable with treatment 01:38:56.010 --> 01:38:56.510 align:middle line:90% as well. 01:38:56.510 --> 01:38:59.930 align:middle line:84% AUDIENCE: Yeah, so these are who are screened sent to treatment. 01:38:59.930 --> 01:39:02.570 align:middle line:84% The two on the right are those who are 01:39:02.570 --> 01:39:04.130 align:middle line:90% symptomatic sent to treatment. 01:39:04.130 --> 01:39:05.350 align:middle line:84% The two on the bottom right are not screened. 01:39:05.350 --> 01:39:06.392 align:middle line:90% HAZHIR RAHMANDAD: Got it. 01:39:06.392 --> 01:39:07.010 align:middle line:90% OK, perfect. 01:39:07.010 --> 01:39:10.030 align:middle line:90% 01:39:10.030 --> 01:39:14.790 align:middle line:84% But why, so if you have low smoking history 01:39:14.790 --> 01:39:18.870 align:middle line:84% population with detectable nodules, they get a screened 01:39:18.870 --> 01:39:20.470 align:middle line:90% and they get treated? 01:39:20.470 --> 01:39:22.990 align:middle line:84% AUDIENCE: So they won't extreme because they're not 01:39:22.990 --> 01:39:23.990 align:middle line:90% eligible for screening. 01:39:23.990 --> 01:39:26.790 align:middle line:84% Only the highest, so the eligibility for screening 01:39:26.790 --> 01:39:28.550 align:middle line:90% comes from high smoking status. 01:39:28.550 --> 01:39:29.550 align:middle line:90% HAZHIR RAHMANDAD: I see. 01:39:29.550 --> 01:39:32.217 align:middle line:84% AUDIENCE: And I'm assuming that because there's a concept called 01:39:32.217 --> 01:39:34.670 align:middle line:84% accidental screening, which means if someone's in a car 01:39:34.670 --> 01:39:37.588 align:middle line:84% accident, they'll get screened, I'm assuming that's very small. 01:39:37.588 --> 01:39:38.630 align:middle line:90% HAZHIR RAHMANDAD: Got it. 01:39:38.630 --> 01:39:40.390 align:middle line:90% OK. 01:39:40.390 --> 01:39:42.690 align:middle line:84% With that explanation, it all makes sense. 01:39:42.690 --> 01:39:45.790 align:middle line:90% 01:39:45.790 --> 01:39:48.030 align:middle line:84% And basically, the rate of screening 01:39:48.030 --> 01:39:52.690 align:middle line:84% is determining how many people you catch in this stage. 01:39:52.690 --> 01:39:56.710 align:middle line:90% And then, that helps. 01:39:56.710 --> 01:40:02.820 align:middle line:84% And then the recovery rate here from treatment? 01:40:02.820 --> 01:40:05.180 align:middle line:84% AUDIENCE: So I'm not very comfortable with how 01:40:05.180 --> 01:40:06.200 align:middle line:90% I formulated that. 01:40:06.200 --> 01:40:12.100 align:middle line:84% So we have in data, we know how big a percentage of people 01:40:12.100 --> 01:40:16.420 align:middle line:84% in each one of these four types of treatment will survive 01:40:16.420 --> 01:40:19.380 align:middle line:90% or will recover, essentially. 01:40:19.380 --> 01:40:25.380 align:middle line:84% But the times are different for recovering or passing. 01:40:25.380 --> 01:40:26.860 align:middle line:90% I wasn't very comfortable. 01:40:26.860 --> 01:40:27.818 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:40:27.818 --> 01:40:30.060 align:middle line:84% So I think one thing that is probably 01:40:30.060 --> 01:40:34.140 align:middle line:84% important to have is you need death rates 01:40:34.140 --> 01:40:39.140 align:middle line:84% from symptomatic nodule in both of these stocks 01:40:39.140 --> 01:40:41.660 align:middle line:90% before they get treated as well. 01:40:41.660 --> 01:40:45.580 align:middle line:84% Not everybody who is symptomatic gets 01:40:45.580 --> 01:40:48.060 align:middle line:84% treated unless you're in a country 01:40:48.060 --> 01:40:50.273 align:middle line:84% and set up that you really have-- 01:40:50.273 --> 01:40:51.940 align:middle line:84% AUDIENCE: Yes, I'm assuming that there's 01:40:51.940 --> 01:40:55.320 align:middle line:84% infinite capacity for screening and treatment at this point. 01:40:55.320 --> 01:40:57.720 align:middle line:90% But you're right. 01:40:57.720 --> 01:41:01.320 align:middle line:84% And I'm assuming that there is no other cause of death 01:41:01.320 --> 01:41:06.202 align:middle line:84% but lung cancer in this case, which is a very big assumption. 01:41:06.202 --> 01:41:07.160 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:41:07.160 --> 01:41:08.522 align:middle line:90% AUDIENCE: But I see your point. 01:41:08.522 --> 01:41:10.480 align:middle line:84% I think another thing that I can definitely add 01:41:10.480 --> 01:41:15.557 align:middle line:84% is not just passing from other causes, but post 80 years, 01:41:15.557 --> 01:41:17.140 align:middle line:84% people are not eligible for screening, 01:41:17.140 --> 01:41:20.322 align:middle line:84% so they will exit the whole chain at some point in time. 01:41:20.322 --> 01:41:21.280 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:41:21.280 --> 01:41:23.580 align:middle line:84% So I think the exit rates need to be there. 01:41:23.580 --> 01:41:26.500 align:middle line:84% Otherwise, it creates these kind of weird dynamics. 01:41:26.500 --> 01:41:31.840 align:middle line:84% Right now, everything is moving towards right and bottom. 01:41:31.840 --> 01:41:38.520 align:middle line:84% And in the steady state, that doesn't make much sense. 01:41:38.520 --> 01:41:41.020 align:middle line:90% OK. 01:41:41.020 --> 01:41:41.520 align:middle line:90% Good. 01:41:41.520 --> 01:41:44.600 align:middle line:84% I think the basic story makes sense. 01:41:44.600 --> 01:41:47.840 align:middle line:84% If you want us to dig into any of these, we can. 01:41:47.840 --> 01:41:50.420 align:middle line:84% So you have a screening participation. 01:41:50.420 --> 01:41:52.530 align:middle line:90% This is as a fraction of people. 01:41:52.530 --> 01:41:54.280 align:middle line:84% AUDIENCE: This is exogenous at this point. 01:41:54.280 --> 01:41:57.520 align:middle line:84% I'm hoping that I can make that endogenous based 01:41:57.520 --> 01:42:00.540 align:middle line:84% on false negatives, false positives, perception 01:42:00.540 --> 01:42:01.540 align:middle line:90% of risk and benefits. 01:42:01.540 --> 01:42:02.823 align:middle line:90% That's the long-term plan. 01:42:02.823 --> 01:42:03.740 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:42:03.740 --> 01:42:07.900 align:middle line:90% But the unit is fraction? 01:42:07.900 --> 01:42:18.860 align:middle line:84% It's fraction of these folks who are in time to uptake screening, 01:42:18.860 --> 01:42:20.720 align:middle line:90% decide to be screened? 01:42:20.720 --> 01:42:26.020 align:middle line:90% 01:42:26.020 --> 01:42:27.300 align:middle line:90% OK. 01:42:27.300 --> 01:42:30.900 align:middle line:90% This is when they get screened. 01:42:30.900 --> 01:42:32.060 align:middle line:90% And then there is? 01:42:32.060 --> 01:42:37.060 align:middle line:90% 01:42:37.060 --> 01:42:39.500 align:middle line:84% AUDIENCE: That rate depends on the growth 01:42:39.500 --> 01:42:42.820 align:middle line:84% rate of nodules based on the comparison 01:42:42.820 --> 01:42:44.303 align:middle line:90% between the thresholds. 01:42:44.303 --> 01:42:45.220 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:42:45.220 --> 01:42:51.820 align:middle line:84% So this is the symptomatic nodule size threshold. 01:42:51.820 --> 01:42:55.160 align:middle line:84% So the terms here are a little bit confusing for me-- 01:42:55.160 --> 01:42:58.410 align:middle line:90% 01:42:58.410 --> 01:43:01.430 align:middle line:84% 1 minus is screening participation. 01:43:01.430 --> 01:43:05.050 align:middle line:90% 01:43:05.050 --> 01:43:09.130 align:middle line:90% So this is actually tricky. 01:43:09.130 --> 01:43:12.650 align:middle line:84% Why do you have 1 minus screening participation 01:43:12.650 --> 01:43:17.650 align:middle line:84% in the movement of people from medium 01:43:17.650 --> 01:43:20.110 align:middle line:84% size to large-sized nodules or whatever it is? 01:43:20.110 --> 01:43:21.690 align:middle line:84% AUDIENCE: I'm assuming that if they participate in screening, 01:43:21.690 --> 01:43:23.010 align:middle line:90% they'll enter treatment. 01:43:23.010 --> 01:43:26.753 align:middle line:84% So that will stop the growth of the nodule. 01:43:26.753 --> 01:43:28.670 align:middle line:84% And they will not be in that pipeline anymore. 01:43:28.670 --> 01:43:32.130 align:middle line:90% They'll move for treatments. 01:43:32.130 --> 01:43:35.130 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:43:35.130 --> 01:43:38.510 align:middle line:84% But if everybody is participating in the screening, 01:43:38.510 --> 01:43:40.210 align:middle line:84% so if your screening participation 01:43:40.210 --> 01:43:42.050 align:middle line:90% is 1, what does that mean? 01:43:42.050 --> 01:43:52.370 align:middle line:84% It means right now that the residence time here 01:43:52.370 --> 01:43:57.350 align:middle line:84% that is under the time to uptake screening, 01:43:57.350 --> 01:44:00.230 align:middle line:84% they will move down to the screened group. 01:44:00.230 --> 01:44:06.210 align:middle line:84% But during that time, their nodules don't stop growing. 01:44:06.210 --> 01:44:08.870 align:middle line:90% 01:44:08.870 --> 01:44:10.550 align:middle line:90% You see what I mean? 01:44:10.550 --> 01:44:14.110 align:middle line:84% So during that time, their nodules are actually growing. 01:44:14.110 --> 01:44:16.490 align:middle line:90% So the rate would not shut off. 01:44:16.490 --> 01:44:17.172 align:middle line:90% AUDIENCE: No. 01:44:17.172 --> 01:44:19.130 align:middle line:84% HAZHIR RAHMANDAD: This rate would not shut off. 01:44:19.130 --> 01:44:19.870 align:middle line:90% AUDIENCE: But now it does. 01:44:19.870 --> 01:44:20.953 align:middle line:90% HAZHIR RAHMANDAD: It does. 01:44:20.953 --> 01:44:24.310 align:middle line:84% So this is actually pretty fundamental, basic. 01:44:24.310 --> 01:44:28.830 align:middle line:84% It is a very common confusion that the mechanism 01:44:28.830 --> 01:44:33.610 align:middle line:84% for moving to the right here is about growth of nodules. 01:44:33.610 --> 01:44:37.190 align:middle line:84% It has nothing to do with the screening. 01:44:37.190 --> 01:44:40.110 align:middle line:84% So if you increase the screening or decrease the screening, 01:44:40.110 --> 01:44:42.670 align:middle line:90% that process does not shut off. 01:44:42.670 --> 01:44:47.950 align:middle line:84% What does change is that more of the people go down 01:44:47.950 --> 01:44:50.870 align:middle line:84% and therefore they are not exposed to this movement 01:44:50.870 --> 01:44:52.230 align:middle line:90% to the right. 01:44:52.230 --> 01:44:55.110 align:middle line:84% But those who are exposed to moving to the right, 01:44:55.110 --> 01:44:58.930 align:middle line:84% you should let them move to the right with their baseline rate. 01:44:58.930 --> 01:45:02.250 align:middle line:84% So there shouldn't be any impact of a screening on this flow 01:45:02.250 --> 01:45:03.130 align:middle line:90% variable. 01:45:03.130 --> 01:45:05.530 align:middle line:90% This is pretty important. 01:45:05.530 --> 01:45:07.530 align:middle line:90% AUDIENCE: That's a great point. 01:45:07.530 --> 01:45:08.910 align:middle line:90% Follow-up question on that. 01:45:08.910 --> 01:45:11.970 align:middle line:84% Will the survival rate or recovery rate of treatments 01:45:11.970 --> 01:45:14.490 align:middle line:90% work the same way? 01:45:14.490 --> 01:45:16.170 align:middle line:84% HAZHIR RAHMANDAD: Pretty much so. 01:45:16.170 --> 01:45:19.290 align:middle line:84% AUDIENCE: Because right now I'm doing that 1 minus 4 01:45:19.290 --> 01:45:20.665 align:middle line:90% for people who are passing. 01:45:20.665 --> 01:45:22.290 align:middle line:84% But I'm not very comfortable with that. 01:45:22.290 --> 01:45:23.790 align:middle line:84% HAZHIR RAHMANDAD: So which equation? 01:45:23.790 --> 01:45:26.813 align:middle line:84% AUDIENCE: Any of those treatment stocks have two outputs. 01:45:26.813 --> 01:45:27.730 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:45:27.730 --> 01:45:30.590 align:middle line:84% So you have the passing and you have the recovering. 01:45:30.590 --> 01:45:32.620 align:middle line:90% AUDIENCE: Yes. 01:45:32.620 --> 01:45:34.370 align:middle line:84% HAZHIR RAHMANDAD: So right now because you 01:45:34.370 --> 01:45:38.410 align:middle line:84% don't have aging or getting out of these stocks 01:45:38.410 --> 01:45:42.570 align:middle line:84% through other channels, it is a little bit confusing. 01:45:42.570 --> 01:45:44.490 align:middle line:84% So basically, you are letting people, 01:45:44.490 --> 01:45:49.550 align:middle line:84% you're saying once you are starting in this journey, 01:45:49.550 --> 01:45:52.030 align:middle line:84% you go all the way either to death 01:45:52.030 --> 01:45:56.270 align:middle line:84% or to recovery from treatment, and you have no other choices. 01:45:56.270 --> 01:46:00.750 align:middle line:84% That part makes it harder to say what exactly this outflow is. 01:46:00.750 --> 01:46:02.570 align:middle line:90% Is it aging out of the group? 01:46:02.570 --> 01:46:08.150 align:middle line:84% Is it fully recovering from cancer? 01:46:08.150 --> 01:46:16.550 align:middle line:84% So if you think of it as recovering from cancer, 01:46:16.550 --> 01:46:19.270 align:middle line:84% the way you are formulating it here is you say-- 01:46:19.270 --> 01:46:22.630 align:middle line:84% AUDIENCE: Let's say recovery proportion 01:46:22.630 --> 01:46:24.590 align:middle line:84% times the population divided by the time 01:46:24.590 --> 01:46:25.793 align:middle line:90% that it takes to recover. 01:46:25.793 --> 01:46:26.710 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:46:26.710 --> 01:46:30.630 align:middle line:84% So in this case, what I would do is not use two different time 01:46:30.630 --> 01:46:32.340 align:middle line:90% constants. 01:46:32.340 --> 01:46:34.590 align:middle line:84% AUDIENCE: What if you know that the time constants are 01:46:34.590 --> 01:46:36.630 align:middle line:84% different, actually takes different amounts of time 01:46:36.630 --> 01:46:38.838 align:middle line:84% for, let's say, if someone who goes through treatment 01:46:38.838 --> 01:46:40.273 align:middle line:90% or someone who passes? 01:46:40.273 --> 01:46:41.190 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:46:41.190 --> 01:46:43.870 align:middle line:84% So basically the underlying mechanism 01:46:43.870 --> 01:46:49.780 align:middle line:84% there is that people are going through different levels 01:46:49.780 --> 01:46:51.740 align:middle line:90% of complications. 01:46:51.740 --> 01:46:54.540 align:middle line:84% So you actually have multiple steps. 01:46:54.540 --> 01:46:58.040 align:middle line:84% And the further you go to more complicated cases, 01:46:58.040 --> 01:47:02.420 align:middle line:90% the death rate increases. 01:47:02.420 --> 01:47:11.020 align:middle line:84% So you can have recovery from early stages of complications 01:47:11.020 --> 01:47:15.600 align:middle line:84% that the drug helps, you survive, and you're not dying. 01:47:15.600 --> 01:47:18.300 align:middle line:90% 01:47:18.300 --> 01:47:21.060 align:middle line:84% So let's think about it without this model. 01:47:21.060 --> 01:47:24.320 align:middle line:84% You have three different levels of complications-- 01:47:24.320 --> 01:47:26.760 align:middle line:90% low, medium, high. 01:47:26.760 --> 01:47:32.140 align:middle line:84% Low complication, there is kind of a rate of outflow 01:47:32.140 --> 01:47:34.740 align:middle line:84% that you are completely recovering. 01:47:34.740 --> 01:47:36.640 align:middle line:84% You have that fixed rate coming down. 01:47:36.640 --> 01:47:39.260 align:middle line:90% 01:47:39.260 --> 01:47:41.720 align:middle line:84% And there is no death from low complication. 01:47:41.720 --> 01:47:44.040 align:middle line:84% You can only go to medium complications. 01:47:44.040 --> 01:47:50.240 align:middle line:84% Similarly, from medium, the last one you go to it, you die. 01:47:50.240 --> 01:47:52.140 align:middle line:84% You don't have recovery, for example. 01:47:52.140 --> 01:47:54.600 align:middle line:84% Or you could have a recovery, but it would be very small. 01:47:54.600 --> 01:47:57.580 align:middle line:84% So if you don't have recovery from the last one, 01:47:57.580 --> 01:48:01.680 align:middle line:84% you just have recovery from the first two, 01:48:01.680 --> 01:48:04.440 align:middle line:84% so that is the disaggregation that 01:48:04.440 --> 01:48:09.120 align:middle line:84% captures the idea you are trying to capture. 01:48:09.120 --> 01:48:15.000 align:middle line:84% Now, if you're aggregating all of this, 01:48:15.000 --> 01:48:17.320 align:middle line:84% you basically have a fraction that 01:48:17.320 --> 01:48:21.920 align:middle line:84% is coming out from this end or a fraction that is coming out 01:48:21.920 --> 01:48:23.320 align:middle line:90% from the other end. 01:48:23.320 --> 01:48:28.320 align:middle line:84% And all you need for that is two degrees of freedom, basically. 01:48:28.320 --> 01:48:31.920 align:middle line:84% The way you can implement these two degrees of freedom 01:48:31.920 --> 01:48:37.860 align:middle line:84% could be a fraction dying versus the rest that are not dying, 01:48:37.860 --> 01:48:40.480 align:middle line:84% which is what you have, and an average time 01:48:40.480 --> 01:48:43.360 align:middle line:84% constant for how long is the average residence 01:48:43.360 --> 01:48:46.660 align:middle line:84% time in this total three-stock thing. 01:48:46.660 --> 01:48:52.620 align:middle line:84% OK, so you use an average time constant and a fraction dying. 01:48:52.620 --> 01:48:55.640 align:middle line:84% Introducing a third parameter doesn't give you anything. 01:48:55.640 --> 01:48:59.620 align:middle line:84% It just adds one degree of freedom that is not needed 01:48:59.620 --> 01:49:03.340 align:middle line:84% and will confuse you in interpreting or estimating 01:49:03.340 --> 01:49:04.080 align:middle line:90% the model. 01:49:04.080 --> 01:49:07.060 align:middle line:84% So that's the way I think about it 01:49:07.060 --> 01:49:09.800 align:middle line:84% and why I said you don't need these two parameters, 01:49:09.800 --> 01:49:12.540 align:middle line:90% you just need one. 01:49:12.540 --> 01:49:16.660 align:middle line:84% It just doesn't give you anything, basically. 01:49:16.660 --> 01:49:18.220 align:middle line:90% Yeah. 01:49:18.220 --> 01:49:18.900 align:middle line:90% Good. 01:49:18.900 --> 01:49:24.000 align:middle line:90% So this all makes sense. 01:49:24.000 --> 01:49:26.200 align:middle line:84% Let's see what else you have here. 01:49:26.200 --> 01:49:27.760 align:middle line:90% Passing thresholds. 01:49:27.760 --> 01:49:30.980 align:middle line:84% AUDIENCE: So the idea is the decision maker 01:49:30.980 --> 01:49:33.120 align:middle line:84% can change that eligibility threshold, 01:49:33.120 --> 01:49:35.765 align:middle line:84% make it more inclusive or exclusive. 01:49:35.765 --> 01:49:36.640 align:middle line:90% HAZHIR RAHMANDAD: OK. 01:49:36.640 --> 01:49:38.700 align:middle line:90% So that will then change? 01:49:38.700 --> 01:49:42.420 align:middle line:84% AUDIENCE: Change the rates that people go from the top 01:49:42.420 --> 01:49:43.900 align:middle line:90% to the bottom. 01:49:43.900 --> 01:49:48.980 align:middle line:90% 01:49:48.980 --> 01:49:51.720 align:middle line:84% That's something I haven't done, but there's a nodule development 01:49:51.720 --> 01:49:53.640 align:middle line:90% rate and a nodule growth rate. 01:49:53.640 --> 01:49:57.290 align:middle line:84% That needs to depend on that threshold. 01:49:57.290 --> 01:49:59.540 align:middle line:84% HAZHIR RAHMANDAD: Why does it depend on the threshold? 01:49:59.540 --> 01:50:01.840 align:middle line:84% AUDIENCE: If I make the threshold more inclusive, 01:50:01.840 --> 01:50:05.280 align:middle line:84% then the average smoking history of the highest smoking history 01:50:05.280 --> 01:50:09.280 align:middle line:84% will be lower, and the average smoking history the lowest 01:50:09.280 --> 01:50:10.140 align:middle line:90% smoking history-- 01:50:10.140 --> 01:50:11.740 align:middle line:84% HAZHIR RAHMANDAD: Oh, that is tricky. 01:50:11.740 --> 01:50:14.820 align:middle line:90% 01:50:14.820 --> 01:50:16.820 align:middle line:84% AUDIENCE: Right now, it's not dependent on that. 01:50:16.820 --> 01:50:19.080 align:middle line:84% HAZHIR RAHMANDAD: Yeah, but you can't actually 01:50:19.080 --> 01:50:22.220 align:middle line:84% change the history completely here. 01:50:22.220 --> 01:50:25.240 align:middle line:90% 01:50:25.240 --> 01:50:29.480 align:middle line:84% Because once you're in the kind of a low smoking history, 01:50:29.480 --> 01:50:31.860 align:middle line:84% you can think of these as some threshold, 01:50:31.860 --> 01:50:34.280 align:middle line:90% but that's a fixed threshold. 01:50:34.280 --> 01:50:36.520 align:middle line:84% Changing that threshold for whether you 01:50:36.520 --> 01:50:40.280 align:middle line:84% are in the top group or the bottom group over time 01:50:40.280 --> 01:50:43.030 align:middle line:84% is not going to work here, because those 01:50:43.030 --> 01:50:46.990 align:middle line:84% are the definitions of people in these stocks. 01:50:46.990 --> 01:50:49.550 align:middle line:90% This is not a coflow. 01:50:49.550 --> 01:50:51.470 align:middle line:84% Once you are in the upper one, you 01:50:51.470 --> 01:50:56.150 align:middle line:84% are saying you have whatever, some cumulative threshold 01:50:56.150 --> 01:50:57.690 align:middle line:90% of smoking in the past. 01:50:57.690 --> 01:50:59.110 align:middle line:90% But that is the definition. 01:50:59.110 --> 01:51:02.910 align:middle line:84% You can't meet simulation, come and change that definition. 01:51:02.910 --> 01:51:06.790 align:middle line:90% So what are your choices? 01:51:06.790 --> 01:51:09.890 align:middle line:84% One choice is to have more categories 01:51:09.890 --> 01:51:17.350 align:middle line:84% so that you have 100, 200, 300, whatever it is, the unit of it. 01:51:17.350 --> 01:51:20.130 align:middle line:84% And then if you're changing the threshold, 01:51:20.130 --> 01:51:21.830 align:middle line:84% you're going to go from one of these 01:51:21.830 --> 01:51:25.460 align:middle line:84% to the upper one or lower one and allow them to be tested. 01:51:25.460 --> 01:51:27.210 align:middle line:84% AUDIENCE: Let's say you have three layers. 01:51:27.210 --> 01:51:28.990 align:middle line:84% And then the second layer can be either 01:51:28.990 --> 01:51:30.330 align:middle line:90% including screening or not? 01:51:30.330 --> 01:51:32.310 align:middle line:84% HAZHIR RAHMANDAD: Yes, so that's one way. 01:51:32.310 --> 01:51:36.230 align:middle line:84% Another way is to say, I want to keep just these two to keep 01:51:36.230 --> 01:51:37.050 align:middle line:90% things simple. 01:51:37.050 --> 01:51:40.550 align:middle line:84% But given the changing threshold, 01:51:40.550 --> 01:51:47.170 align:middle line:84% I would have a larger fraction of the top row or bottom 01:51:47.170 --> 01:51:51.342 align:middle line:84% row included into the eligible group. 01:51:51.342 --> 01:51:53.650 align:middle line:84% AUDIENCE: Right now we're doing 0%, 100%. 01:51:53.650 --> 01:51:55.510 align:middle line:90% HAZHIR RAHMANDAD: 0% and 100%. 01:51:55.510 --> 01:52:01.730 align:middle line:84% Now, making that not 0% and 100% becomes tricky again, 01:52:01.730 --> 01:52:05.290 align:middle line:84% because as soon as you make that flexible, now 01:52:05.290 --> 01:52:11.170 align:middle line:84% you have taken some people with more or less history 01:52:11.170 --> 01:52:13.010 align:middle line:84% into a different category, and you 01:52:13.010 --> 01:52:22.090 align:middle line:84% are changing the average history for that group. 01:52:22.090 --> 01:52:26.330 align:middle line:84% So it requires more complex formulations 01:52:26.330 --> 01:52:29.530 align:middle line:84% because now it is kind of a coflow, that 01:52:29.530 --> 01:52:36.130 align:middle line:84% is, you're changing the average of the thing in the stock. 01:52:36.130 --> 01:52:40.770 align:middle line:84% Short answer is, I don't recommend doing it that way. 01:52:40.770 --> 01:52:41.970 align:middle line:90% There are ways to do it. 01:52:41.970 --> 01:52:46.190 align:middle line:84% And so my model of COVID, if you had 01:52:46.190 --> 01:52:47.930 align:middle line:84% time to go into the details of it, 01:52:47.930 --> 01:52:52.110 align:middle line:84% you would find that it has an equity that 01:52:52.110 --> 01:52:56.630 align:middle line:84% is, think of it as number of symptoms that somebody may have. 01:52:56.630 --> 01:53:00.230 align:middle line:84% And that number is changing whether you go into hospital 01:53:00.230 --> 01:53:03.790 align:middle line:90% or not, whether you die or not. 01:53:03.790 --> 01:53:05.390 align:middle line:84% So people who have more equity are 01:53:05.390 --> 01:53:06.870 align:middle line:84% more likely to go to hospital, more 01:53:06.870 --> 01:53:10.390 align:middle line:84% likely to be detected, and therefore whether you 01:53:10.390 --> 01:53:14.110 align:middle line:84% are detected or not-- so all of these are a function of equity. 01:53:14.110 --> 01:53:17.310 align:middle line:84% So I need to actually move people around 01:53:17.310 --> 01:53:21.150 align:middle line:84% with their equity levels changing dynamically. 01:53:21.150 --> 01:53:22.730 align:middle line:90% So there are ways to do it. 01:53:22.730 --> 01:53:24.110 align:middle line:90% It is tough. 01:53:24.110 --> 01:53:26.250 align:middle line:90% It becomes complex. 01:53:26.250 --> 01:53:31.330 align:middle line:84% So it's a nice challenge to do it and try to do it. 01:53:31.330 --> 01:53:35.710 align:middle line:84% But also it is not easy to get it right. 01:53:35.710 --> 01:53:38.210 align:middle line:90% So those are the two options. 01:53:38.210 --> 01:53:40.230 align:middle line:84% AUDIENCE: My goal at this point is just 01:53:40.230 --> 01:53:43.730 align:middle line:84% to turn this into a reasonable model 01:53:43.730 --> 01:53:50.050 align:middle line:84% that I can test changes of inclusivity that 01:53:50.050 --> 01:53:52.618 align:middle line:84% can be discrete changes in three layers. 01:53:52.618 --> 01:53:53.910 align:middle line:90% HAZHIR RAHMANDAD: Yes, exactly. 01:53:53.910 --> 01:53:58.470 align:middle line:84% So with that in mind, I would say go with, first of all, 01:53:58.470 --> 01:54:03.370 align:middle line:84% subscripting things so that you can have multiple layers 01:54:03.370 --> 01:54:06.330 align:middle line:84% without worrying about, oh, I have to recreate 01:54:06.330 --> 01:54:09.650 align:middle line:90% this whole structure. 01:54:09.650 --> 01:54:11.730 align:middle line:84% AUDIENCE: Subscripted lower layers? 01:54:11.730 --> 01:54:15.790 align:middle line:84% HAZHIR RAHMANDAD: Yeah, at least the movement up to bottom. 01:54:15.790 --> 01:54:17.370 align:middle line:84% But I think nodules, you can also 01:54:17.370 --> 01:54:20.870 align:middle line:84% subscript as well if you want to. 01:54:20.870 --> 01:54:23.430 align:middle line:84% And then use those discrete thresholds, 01:54:23.430 --> 01:54:26.310 align:middle line:84% movement of thresholds based on those discrete categories. 01:54:26.310 --> 01:54:29.490 align:middle line:84% That would be cleaner for what your purposes are. 01:54:29.490 --> 01:54:33.810 align:middle line:84% And it's easier to explain to outsiders, much easier 01:54:33.810 --> 01:54:37.480 align:middle line:90% to explain as well. 01:54:37.480 --> 01:54:39.280 align:middle line:84% AUDIENCE: Can I ask a clarifying question? 01:54:39.280 --> 01:54:45.560 align:middle line:84% What's the proposed fix for the nodule growth still happening 01:54:45.560 --> 01:54:50.640 align:middle line:84% even after you moved into the screening for a solution? 01:54:50.640 --> 01:54:53.452 align:middle line:84% AUDIENCE: What are you talking about? 01:54:53.452 --> 01:54:55.097 align:middle line:84% AUDIENCE: What Hazhir pointed out. 01:54:55.097 --> 01:54:56.680 align:middle line:84% AUDIENCE: I think the proposed fix was 01:54:56.680 --> 01:54:58.860 align:middle line:90% the formulation for the growth. 01:54:58.860 --> 01:55:01.498 align:middle line:84% I'll take the 1 minus the screen participation out. 01:55:01.498 --> 01:55:03.040 align:middle line:84% Independent of whether or not they're 01:55:03.040 --> 01:55:06.960 align:middle line:84% screening, the participating in the screening, 01:55:06.960 --> 01:55:10.520 align:middle line:90% they will move to the right. 01:55:10.520 --> 01:55:13.080 align:middle line:84% AUDIENCE: But how can they move to the right 01:55:13.080 --> 01:55:15.200 align:middle line:84% if they've moved into the screening? 01:55:15.200 --> 01:55:18.560 align:middle line:84% AUDIENCE: Those of them who are moving down to treatment, 01:55:18.560 --> 01:55:19.060 align:middle line:90% they will. 01:55:19.060 --> 01:55:20.420 align:middle line:90% So whatever's left-- 01:55:20.420 --> 01:55:21.837 align:middle line:84% AUDIENCE: Oh, you'll have another. 01:55:21.837 --> 01:55:23.510 align:middle line:90% You'll have a flow coming-- 01:55:23.510 --> 01:55:25.260 align:middle line:84% HAZHIR RAHMANDAD: These people are coming. 01:55:25.260 --> 01:55:27.720 align:middle line:84% So for this group, you have people 01:55:27.720 --> 01:55:32.655 align:middle line:84% from high smoking history, detectable nodules to screened. 01:55:32.655 --> 01:55:33.280 align:middle line:90% AUDIENCE: Yeah. 01:55:33.280 --> 01:55:34.340 align:middle line:84% HAZHIR RAHMANDAD: Once they are screened, 01:55:34.340 --> 01:55:36.260 align:middle line:84% they are not moving to the right. 01:55:36.260 --> 01:55:38.420 align:middle line:90% They are not getting worse. 01:55:38.420 --> 01:55:40.360 align:middle line:84% Or at least that's the assumption right now. 01:55:40.360 --> 01:55:41.860 align:middle line:84% You could easily have the assumption 01:55:41.860 --> 01:55:43.700 align:middle line:84% that they do get worse, but much slower 01:55:43.700 --> 01:55:46.240 align:middle line:84% rate because they are under treatment. 01:55:46.240 --> 01:55:48.040 align:middle line:84% Some people would not respond to treatment, 01:55:48.040 --> 01:55:50.140 align:middle line:84% so I think it's pretty reasonable 01:55:50.140 --> 01:55:53.100 align:middle line:84% to include the possibility of moving to the right 01:55:53.100 --> 01:55:55.798 align:middle line:90% but not with the same rate. 01:55:55.798 --> 01:55:56.340 align:middle line:90% AUDIENCE: OK. 01:55:56.340 --> 01:55:57.367 align:middle line:90% I think I misunderstood. 01:55:57.367 --> 01:55:59.700 align:middle line:84% I thought you wanted to account for growth while they're 01:55:59.700 --> 01:56:01.940 align:middle line:90% waiting for screening. 01:56:01.940 --> 01:56:04.160 align:middle line:84% HAZHIR RAHMANDAD: Yes, it is being accounted for. 01:56:04.160 --> 01:56:07.340 align:middle line:84% But I was saying that this screening participation should 01:56:07.340 --> 01:56:08.940 align:middle line:90% not go into this rate. 01:56:08.940 --> 01:56:11.420 align:middle line:84% Because the movement to the right has nothing to 01:56:11.420 --> 01:56:13.540 align:middle line:90% do with screening. 01:56:13.540 --> 01:56:15.860 align:middle line:84% The movement to the right has only 01:56:15.860 --> 01:56:17.720 align:middle line:84% to do with the growth of the tumor. 01:56:17.720 --> 01:56:20.020 align:middle line:84% As long as you are in this group that you have not yet 01:56:20.020 --> 01:56:27.000 align:middle line:84% been detected, you are exposed to the risk of growth in tumor. 01:56:27.000 --> 01:56:30.140 align:middle line:84% It has nothing to do with screening at that point in time. 01:56:30.140 --> 01:56:33.960 align:middle line:84% The only conditions that we have a fraction moving 01:56:33.960 --> 01:56:36.560 align:middle line:84% this way versus that way, our conditions 01:56:36.560 --> 01:56:40.040 align:middle line:84% that you can imagine it to be at the end of your stay, 01:56:40.040 --> 01:56:42.820 align:middle line:90% you have two different pathways. 01:56:42.820 --> 01:56:44.380 align:middle line:90% You die or you survive. 01:56:44.380 --> 01:56:48.560 align:middle line:84% Or you go to the upper grade or you go and retake 01:56:48.560 --> 01:56:50.640 align:middle line:90% your class or whatever. 01:56:50.640 --> 01:56:55.400 align:middle line:84% This is, you have a bifurcation of pathways 01:56:55.400 --> 01:57:00.020 align:middle line:84% at the end of a decision choice possibility. 01:57:00.020 --> 01:57:04.080 align:middle line:84% Those are the places that we will have two parameters going 01:57:04.080 --> 01:57:05.760 align:middle line:84% into both of them, one for the time 01:57:05.760 --> 01:57:08.640 align:middle line:84% constant, how long you stay, the other for what 01:57:08.640 --> 01:57:10.520 align:middle line:90% fraction go right or left. 01:57:10.520 --> 01:57:11.860 align:middle line:90% This is not one of those. 01:57:11.860 --> 01:57:14.600 align:middle line:90% 01:57:14.600 --> 01:57:15.800 align:middle line:90% OK, good. 01:57:15.800 --> 01:57:20.600 align:middle line:84% So I think you have the pieces and I 01:57:20.600 --> 01:57:22.360 align:middle line:84% think you understand what you need 01:57:22.360 --> 01:57:26.960 align:middle line:84% to do to get it ready for now exposing it to data 01:57:26.960 --> 01:57:28.960 align:middle line:84% or whatever you want to do with it. 01:57:28.960 --> 01:57:33.230 align:middle line:84% In terms of data availability, which of these stocks 01:57:33.230 --> 01:57:37.833 align:middle line:84% are in the typical datasets you might be looking at? 01:57:37.833 --> 01:57:39.750 align:middle line:84% AUDIENCE: Data availability, so technically, I 01:57:39.750 --> 01:57:42.910 align:middle line:84% have access to anything NCI has because we're part of the NCI 01:57:42.910 --> 01:57:46.510 align:middle line:84% Modeling Consortium, but I haven't seen much time series 01:57:46.510 --> 01:57:47.010 align:middle line:90% data. 01:57:47.010 --> 01:57:50.150 align:middle line:84% They have a lot of these large-scale cross-sectional 01:57:50.150 --> 01:57:54.770 align:middle line:84% datasets, which I'm not sure how I can bring in here. 01:57:54.770 --> 01:57:57.050 align:middle line:84% But a lot of the parameters that I've left here, 01:57:57.050 --> 01:58:01.070 align:middle line:84% I have values for them at the national level or state level, 01:58:01.070 --> 01:58:02.530 align:middle line:90% stratified values for them. 01:58:02.530 --> 01:58:06.872 align:middle line:84% So a lot of them I don't necessarily need to calibrate. 01:58:06.872 --> 01:58:07.830 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:58:07.830 --> 01:58:13.510 align:middle line:84% So I suspect you would have some time series like death 01:58:13.510 --> 01:58:15.030 align:middle line:90% at the end of these. 01:58:15.030 --> 01:58:19.430 align:middle line:84% Those, you usually have time series. 01:58:19.430 --> 01:58:23.190 align:middle line:84% Screening and finding new cases, maybe you have. 01:58:23.190 --> 01:58:24.150 align:middle line:90% I don't know. 01:58:24.150 --> 01:58:25.412 align:middle line:90% AUDIENCE: I think we do. 01:58:25.412 --> 01:58:27.370 align:middle line:84% HAZHIR RAHMANDAD: Yeah, that would be my guess, 01:58:27.370 --> 01:58:29.547 align:middle line:84% at least nationally you may have. 01:58:29.547 --> 01:58:31.130 align:middle line:84% AUDIENCE: Capacity constraints on that 01:58:31.130 --> 01:58:33.530 align:middle line:84% we're not looking in here, so that may be different. 01:58:33.530 --> 01:58:34.530 align:middle line:90% HAZHIR RAHMANDAD: Yes. 01:58:34.530 --> 01:58:38.170 align:middle line:84% So the capacity can be built into the time 01:58:38.170 --> 01:58:40.090 align:middle line:90% to uptake screening. 01:58:40.090 --> 01:58:42.350 align:middle line:84% Basically, you are combining two things. 01:58:42.350 --> 01:58:44.930 align:middle line:84% One is the demand side, how often 01:58:44.930 --> 01:58:47.750 align:middle line:90% people want to go and screen. 01:58:47.750 --> 01:58:50.510 align:middle line:84% The other is how much availability of screening is. 01:58:50.510 --> 01:58:53.970 align:middle line:84% You just turn both of them into a time constant, 01:58:53.970 --> 01:58:59.130 align:middle line:84% and that time to uptake screening can be state specific, 01:58:59.130 --> 01:59:00.950 align:middle line:90% can change over time. 01:59:00.950 --> 01:59:06.230 align:middle line:84% So in a regression mindset, you can say there is a time effect 01:59:06.230 --> 01:59:08.490 align:middle line:84% and there is a location fixed effect. 01:59:08.490 --> 01:59:10.850 align:middle line:84% You combine those two to get a time 01:59:10.850 --> 01:59:13.690 align:middle line:84% to uptake screening that can vary 01:59:13.690 --> 01:59:15.470 align:middle line:90% over time and across the states. 01:59:15.470 --> 01:59:17.470 align:middle line:90% And that does a decent job. 01:59:17.470 --> 01:59:20.930 align:middle line:84% So in that sense, you would have a way 01:59:20.930 --> 01:59:25.650 align:middle line:84% of doing that nuanced analysis you may need on that. 01:59:25.650 --> 01:59:26.890 align:middle line:90% Good. 01:59:26.890 --> 01:59:29.470 align:middle line:84% AUDIENCE: I will need to endogenize the participation 01:59:29.470 --> 01:59:30.210 align:middle line:90% at some point. 01:59:30.210 --> 01:59:32.270 align:middle line:84% And if you don't mind, I'll reach back to you 01:59:32.270 --> 01:59:35.010 align:middle line:84% again with whatever formulation I come up with. 01:59:35.010 --> 01:59:40.990 align:middle line:84% I have the luxury of working with the clinical team who 01:59:40.990 --> 01:59:44.510 align:middle line:84% may know or have an insight of what's going on in there. 01:59:44.510 --> 01:59:45.550 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 01:59:45.550 --> 01:59:50.810 align:middle line:84% So participation rate right now, what is it reflecting? 01:59:50.810 --> 01:59:53.290 align:middle line:84% Is it the people deciding to get a screened? 01:59:53.290 --> 01:59:57.550 align:middle line:84% Or is it the authorities that set the level? 01:59:57.550 --> 01:59:58.950 align:middle line:84% AUDIENCE: No, it's the percentage 01:59:58.950 --> 02:00:02.273 align:middle line:84% of those who are eligible that show up to screen. 02:00:02.273 --> 02:00:03.190 align:middle line:90% HAZHIR RAHMANDAD: Yes. 02:00:03.190 --> 02:00:07.790 align:middle line:84% So that part is largely related to this time 02:00:07.790 --> 02:00:11.390 align:middle line:84% to uptake screening, the way I think about it. 02:00:11.390 --> 02:00:14.670 align:middle line:84% In fact, if you don't have this screening participation feeding 02:00:14.670 --> 02:00:18.910 align:middle line:84% back into growth of nodules, these two parameters 02:00:18.910 --> 02:00:21.843 align:middle line:90% become superfluous. 02:00:21.843 --> 02:00:23.010 align:middle line:90% You don't need both of them. 02:00:23.010 --> 02:00:26.170 align:middle line:84% You just need one degree of freedom for each of these. 02:00:26.170 --> 02:00:29.170 align:middle line:84% Now, you may have different parameters 02:00:29.170 --> 02:00:33.490 align:middle line:84% for those with larger symptomatic nodules 02:00:33.490 --> 02:00:36.250 align:middle line:90% versus detectable nodules. 02:00:36.250 --> 02:00:40.330 align:middle line:84% So it may or may not, but you could 02:00:40.330 --> 02:00:44.330 align:middle line:84% separate two different parameters for those. 02:00:44.330 --> 02:00:48.290 align:middle line:84% But it also depends on how much resolution 02:00:48.290 --> 02:00:51.690 align:middle line:84% you have on being able empirically 02:00:51.690 --> 02:00:54.970 align:middle line:84% to say what fraction of people are in the detectable 02:00:54.970 --> 02:00:58.570 align:middle line:90% versus symptomatic phase. 02:00:58.570 --> 02:01:00.690 align:middle line:84% If you have that, then it makes sense 02:01:00.690 --> 02:01:03.430 align:middle line:84% to separate them and have these two parameters. 02:01:03.430 --> 02:01:06.270 align:middle line:84% If not, you could even combine those two groups. 02:01:06.270 --> 02:01:10.210 align:middle line:84% AUDIENCE: Can I add that those are not the only two stocks who 02:01:10.210 --> 02:01:12.210 align:middle line:90% are going through screening? 02:01:12.210 --> 02:01:14.610 align:middle line:84% The other two stocks to their left 02:01:14.610 --> 02:01:16.090 align:middle line:90% will experience screening, too. 02:01:16.090 --> 02:01:18.590 align:middle line:84% It's just their screening will be negative, the results. 02:01:18.590 --> 02:01:20.330 align:middle line:90% So they won't move their stocks. 02:01:20.330 --> 02:01:22.785 align:middle line:90% So there will be a number of-- 02:01:22.785 --> 02:01:24.160 align:middle line:84% HAZHIR RAHMANDAD: Negative cases? 02:01:24.160 --> 02:01:26.243 align:middle line:84% AUDIENCE: --false positives included, essentially, 02:01:26.243 --> 02:01:27.520 align:middle line:90% or eligibility false positive. 02:01:27.520 --> 02:01:30.578 align:middle line:90% So if eligibility is 100%-- 02:01:30.578 --> 02:01:32.120 align:middle line:84% HAZHIR RAHMANDAD: Yes, yes, yes, yes. 02:01:32.120 --> 02:01:36.080 align:middle line:84% So from your total tests, those who are smoking but 02:01:36.080 --> 02:01:41.120 align:middle line:90% don't have detectable anything. 02:01:41.120 --> 02:01:45.780 align:middle line:84% So in terms of fraction of tests that get positive results, 02:01:45.780 --> 02:01:50.020 align:middle line:84% you need to capture that with whatever screening criteria is. 02:01:50.020 --> 02:01:51.640 align:middle line:90% So that's absolutely the case. 02:01:51.640 --> 02:01:53.320 align:middle line:84% And this kind of goes back to the issue 02:01:53.320 --> 02:01:55.200 align:middle line:84% that we talked about earlier today 02:01:55.200 --> 02:01:59.200 align:middle line:84% on capturing the data generation process. 02:01:59.200 --> 02:02:02.200 align:middle line:84% When you go into the field and you ask yourself, OK, 02:02:02.200 --> 02:02:04.500 align:middle line:90% what do we have data on? 02:02:04.500 --> 02:02:07.840 align:middle line:84% We may have time series data on how many screening 02:02:07.840 --> 02:02:10.560 align:middle line:84% has been done in the US over the last 40 years. 02:02:10.560 --> 02:02:12.580 align:middle line:84% So in your model, what does that represent? 02:02:12.580 --> 02:02:16.840 align:middle line:84% It represents all the people in all of these stocks 02:02:16.840 --> 02:02:18.340 align:middle line:90% who are getting screened. 02:02:18.340 --> 02:02:21.640 align:middle line:84% These are high smoking history people. 02:02:21.640 --> 02:02:25.360 align:middle line:84% They would get a screening, assuming some age requirement 02:02:25.360 --> 02:02:26.580 align:middle line:90% and so on. 02:02:26.580 --> 02:02:29.440 align:middle line:90% So how do I capture that? 02:02:29.440 --> 02:02:31.740 align:middle line:84% I definitely need to account for these guys 02:02:31.740 --> 02:02:35.760 align:middle line:84% as well, even though they may not get positive results. 02:02:35.760 --> 02:02:40.820 align:middle line:84% And then I may have something about the number 02:02:40.820 --> 02:02:48.720 align:middle line:84% of people with detected cancer at any point in time. 02:02:48.720 --> 02:02:54.740 align:middle line:84% That is the sum of people here and here and here. 02:02:54.740 --> 02:02:58.600 align:middle line:90% All of those need to be counted. 02:02:58.600 --> 02:03:01.060 align:middle line:84% And even here, those four, so that's 02:03:01.060 --> 02:03:03.160 align:middle line:84% the data generating process and so on. 02:03:03.160 --> 02:03:05.580 align:middle line:90% So that's the idea. 02:03:05.580 --> 02:03:06.100 align:middle line:90% Great. 02:03:06.100 --> 02:03:07.060 align:middle line:90% AUDIENCE: Thank you. 02:03:07.060 --> 02:03:08.620 align:middle line:90% HAZHIR RAHMANDAD: Sure. 02:03:08.620 --> 02:03:15.020 align:middle line:84% Anybody else on the models for today? 02:03:15.020 --> 02:03:17.360 align:middle line:84% I think we covered everything we had here. 02:03:17.360 --> 02:03:25.400 align:middle line:84% So maybe if not, we can go to our last conversations, just 02:03:25.400 --> 02:03:28.900 align:middle line:90% a little bit of reflections. 02:03:28.900 --> 02:03:46.320 align:middle line:90% 02:03:46.320 --> 02:03:51.960 align:middle line:84% So this is just open conversation. 02:03:51.960 --> 02:03:54.200 align:middle line:84% I want you to reflect on what you 02:03:54.200 --> 02:03:58.920 align:middle line:84% have learned in the semester, what might be tips 02:03:58.920 --> 02:04:00.300 align:middle line:90% that are useful for others. 02:04:00.300 --> 02:04:04.840 align:middle line:84% I think, hopefully, this course will be online soon 02:04:04.840 --> 02:04:09.160 align:middle line:84% and others can also benefit from your thoughts and reflections. 02:04:09.160 --> 02:04:13.160 align:middle line:84% Also, what we didn't cover that could be helpful 02:04:13.160 --> 02:04:18.440 align:middle line:84% as you are building your skills and capabilities, what you feel 02:04:18.440 --> 02:04:23.900 align:middle line:84% is still not there and you need to build that. 02:04:23.900 --> 02:04:26.500 align:middle line:84% And also any feedback on the class structure, how 02:04:26.500 --> 02:04:28.500 align:middle line:90% we could do things differently. 02:04:28.500 --> 02:04:31.020 align:middle line:84% I don't know when would be the next time we offer 02:04:31.020 --> 02:04:35.020 align:middle line:84% this version of the seminar, but hopefully every two or three 02:04:35.020 --> 02:04:39.520 align:middle line:84% years, we will have another offering related to this. 02:04:39.520 --> 02:04:42.360 align:middle line:84% So that would be useful for the next time. 02:04:42.360 --> 02:04:45.260 align:middle line:90% So this is open floor. 02:04:45.260 --> 02:04:46.840 align:middle line:90% Any thoughts are very welcome. 02:04:46.840 --> 02:04:49.813 align:middle line:90% 02:04:49.813 --> 02:04:51.980 align:middle line:84% AUDIENCE: Can I just mentioned that, for me, there's 02:04:51.980 --> 02:04:56.660 align:middle line:84% been what I've learned and what has changed in my habits 02:04:56.660 --> 02:04:57.515 align:middle line:90% as a result of this. 02:04:57.515 --> 02:04:58.640 align:middle line:90% HAZHIR RAHMANDAD: OK, good. 02:04:58.640 --> 02:05:01.000 align:middle line:84% AUDIENCE: There's a lot of stuff that we learned, theoretically, 02:05:01.000 --> 02:05:02.940 align:middle line:84% which I may come back to some of the notes 02:05:02.940 --> 02:05:05.440 align:middle line:84% and then revisit whenever I need them. 02:05:05.440 --> 02:05:10.460 align:middle line:84% But as habits that changed, I wasn't as careful 02:05:10.460 --> 02:05:12.980 align:middle line:90% as I am right now about units. 02:05:12.980 --> 02:05:14.660 align:middle line:84% I wasn't as careful as I am right now 02:05:14.660 --> 02:05:19.170 align:middle line:84% about different versions, smaller versions of the model. 02:05:19.170 --> 02:05:22.370 align:middle line:84% These two specifically stand out as now, 02:05:22.370 --> 02:05:24.710 align:middle line:84% whatever I'm doing with modeling, 02:05:24.710 --> 02:05:25.835 align:middle line:90% I will stick to these two. 02:05:25.835 --> 02:05:27.210 align:middle line:84% HAZHIR RAHMANDAD: That's actually 02:05:27.210 --> 02:05:29.170 align:middle line:84% a very good way of thinking about it, 02:05:29.170 --> 02:05:32.270 align:middle line:84% the habits that might have changed in how you tackle it. 02:05:32.270 --> 02:05:34.210 align:middle line:84% So I think that's a nice reflection maybe 02:05:34.210 --> 02:05:38.010 align:middle line:90% to add to the list of questions. 02:05:38.010 --> 02:05:38.550 align:middle line:90% Good. 02:05:38.550 --> 02:05:41.770 align:middle line:90% Other thoughts? 02:05:41.770 --> 02:05:45.130 align:middle line:84% Did you find others kind of similar changes 02:05:45.130 --> 02:05:49.410 align:middle line:84% in habits, other dimensions that were more salient for you? 02:05:49.410 --> 02:05:51.210 align:middle line:84% AUDIENCE: For me, it was the change 02:05:51.210 --> 02:05:55.010 align:middle line:84% of pace or maybe getting used to having 02:05:55.010 --> 02:05:58.050 align:middle line:90% to build the model quickly. 02:05:58.050 --> 02:06:00.610 align:middle line:90% I think that was reduced for me. 02:06:00.610 --> 02:06:04.890 align:middle line:84% Maybe, otherwise it would have taken me much longer than 02:06:04.890 --> 02:06:08.227 align:middle line:90% a week, let's say, to sit down. 02:06:08.227 --> 02:06:09.810 align:middle line:84% HAZHIR RAHMANDAD: I see, I see, I see. 02:06:09.810 --> 02:06:13.770 align:middle line:84% So a little bit of a push to actually get started and build 02:06:13.770 --> 02:06:17.670 align:middle line:84% models was sometimes helpful for some folks. 02:06:17.670 --> 02:06:21.550 align:middle line:84% I remember when I was learning modeling, 02:06:21.550 --> 02:06:25.630 align:middle line:84% I took System Dynamics now almost 30 years ago 02:06:25.630 --> 02:06:33.190 align:middle line:84% when I was an undergrad in Iran, our teacher, our professor, 02:06:33.190 --> 02:06:35.370 align:middle line:84% was a graduate of the program here, actually. 02:06:35.370 --> 02:06:38.910 align:middle line:84% And he pushed us to build maybe 20 models 02:06:38.910 --> 02:06:41.310 align:middle line:84% over two semesters of introductory course 02:06:41.310 --> 02:06:42.910 align:middle line:90% and a follow-up course. 02:06:42.910 --> 02:06:46.270 align:middle line:84% And I feel I learned a lot of the modeling 02:06:46.270 --> 02:06:48.710 align:middle line:84% that I learned because of that being under pressure 02:06:48.710 --> 02:06:50.910 align:middle line:90% and trying to build many models. 02:06:50.910 --> 02:06:53.550 align:middle line:84% And all of those headaches that you have experienced 02:06:53.550 --> 02:06:57.050 align:middle line:84% this semester, that was where I kind of saw, 02:06:57.050 --> 02:06:58.490 align:middle line:90% OK, I have to struggle. 02:06:58.490 --> 02:07:03.550 align:middle line:84% And it's hard to learn these skills without going 02:07:03.550 --> 02:07:04.970 align:middle line:90% through some of that struggles. 02:07:04.970 --> 02:07:07.850 align:middle line:84% Now, the less struggle, the better. 02:07:07.850 --> 02:07:10.790 align:middle line:84% But you just can't completely avoid those. 02:07:10.790 --> 02:07:12.550 align:middle line:84% You're not learning if you're not 02:07:12.550 --> 02:07:16.350 align:middle line:84% hitting some dead alleys and wrong passes. 02:07:16.350 --> 02:07:20.567 align:middle line:90% 02:07:20.567 --> 02:07:22.150 align:middle line:84% AUDIENCE: Especially at the beginning, 02:07:22.150 --> 02:07:25.490 align:middle line:84% we all were failing at the same core conceptual things. 02:07:25.490 --> 02:07:29.585 align:middle line:84% So taking the time to maybe do the first model together, 02:07:29.585 --> 02:07:31.210 align:middle line:84% like we could all come up with our own, 02:07:31.210 --> 02:07:34.810 align:middle line:84% and then together think about those subtle issues 02:07:34.810 --> 02:07:36.250 align:middle line:90% with modeling. 02:07:36.250 --> 02:07:38.970 align:middle line:84% Or at the very least, see an example 02:07:38.970 --> 02:07:40.670 align:middle line:84% of how it would be correctly done. 02:07:40.670 --> 02:07:42.650 align:middle line:84% It's like, OK, well, we all failed in doing it. 02:07:42.650 --> 02:07:45.130 align:middle line:84% Then, it's like, if none of us actually figured it out, 02:07:45.130 --> 02:07:46.440 align:middle line:90% then it's like more helpful. 02:07:46.440 --> 02:07:48.690 align:middle line:84% I don't know if anyone actually went back and sat down 02:07:48.690 --> 02:07:51.107 align:middle line:84% and thought about how to do it correctly, but assuming you 02:07:51.107 --> 02:07:53.950 align:middle line:90% don't, then you lose that. 02:07:53.950 --> 02:07:55.450 align:middle line:84% And then the other thing being, this 02:07:55.450 --> 02:07:57.610 align:middle line:84% might just be me, because I guess I probably 02:07:57.610 --> 02:07:59.830 align:middle line:84% haven't taken as many analytic courses as you guys, 02:07:59.830 --> 02:08:05.250 align:middle line:84% but I didn't really know how to represent formulations 02:08:05.250 --> 02:08:08.450 align:middle line:84% that were very, very basic, like adding or multiplication. 02:08:08.450 --> 02:08:10.850 align:middle line:84% And maybe covering a little bit more like, 02:08:10.850 --> 02:08:12.980 align:middle line:84% what do all those, smooth delays, 02:08:12.980 --> 02:08:15.920 align:middle line:84% when are the right times to use all of those functions. 02:08:15.920 --> 02:08:19.040 align:middle line:84% As someone who's never actually taken a class, 02:08:19.040 --> 02:08:20.400 align:middle line:90% I understand how to do that. 02:08:20.400 --> 02:08:22.700 align:middle line:84% It might have been helpful, so moving forward, 02:08:22.700 --> 02:08:26.400 align:middle line:84% I wouldn't be like trying to do some weird variable modeling 02:08:26.400 --> 02:08:28.938 align:middle line:84% when I could just use smooth variable. 02:08:28.938 --> 02:08:30.980 align:middle line:84% HAZHIR RAHMANDAD: Yeah, that's a very good point. 02:08:30.980 --> 02:08:33.880 align:middle line:84% And I think kind of a little bit of a challenge for a class 02:08:33.880 --> 02:08:35.680 align:middle line:84% like this is exactly not everybody 02:08:35.680 --> 02:08:38.600 align:middle line:84% comes with the same background of what 02:08:38.600 --> 02:08:41.300 align:middle line:84% is covered in their current level of knowledge. 02:08:41.300 --> 02:08:46.760 align:middle line:84% And in a way, I kind of assumed everybody 02:08:46.760 --> 02:08:50.888 align:middle line:84% knows the basic molecules, that they 02:08:50.888 --> 02:08:52.180 align:middle line:90% have a lot of these components. 02:08:52.180 --> 02:08:55.260 align:middle line:84% But we have never taught those even in our 873 class. 02:08:55.260 --> 02:08:57.960 align:middle line:84% So you're absolutely right, I think. 02:08:57.960 --> 02:09:02.480 align:middle line:84% The other point about going through a model that is well 02:09:02.480 --> 02:09:03.860 align:middle line:90% built is also a very good one. 02:09:03.860 --> 02:09:07.840 align:middle line:84% I tried for the first half of the class 02:09:07.840 --> 02:09:11.280 align:middle line:84% to create a version of a complete model, 02:09:11.280 --> 02:09:13.260 align:middle line:90% but after the class. 02:09:13.260 --> 02:09:16.440 align:middle line:84% And then the later, I haven't had the chance to do that. 02:09:16.440 --> 02:09:18.740 align:middle line:84% So maybe I should try to spend some time 02:09:18.740 --> 02:09:22.180 align:middle line:84% and build those and have that in the resources that come with it. 02:09:22.180 --> 02:09:23.680 align:middle line:84% AUDIENCE: I thought that was useful. 02:09:23.680 --> 02:09:25.140 align:middle line:84% I did go back through, particularly 02:09:25.140 --> 02:09:28.460 align:middle line:90% for some of the specifics. 02:09:28.460 --> 02:09:32.700 align:middle line:84% I remember we had a very long discussion when 02:09:32.700 --> 02:09:40.140 align:middle line:84% we did the bigger SIR model about the vaccinated, indicated 02:09:40.140 --> 02:09:42.080 align:middle line:90% protection situation. 02:09:42.080 --> 02:09:44.320 align:middle line:84% I definitely looked back at that at different times, 02:09:44.320 --> 02:09:49.260 align:middle line:84% because I think that was a key concept that we 02:09:49.260 --> 02:09:51.660 align:middle line:84% used for a lot of them, in terms of creating stocks 02:09:51.660 --> 02:09:53.020 align:middle line:90% at the right time. 02:09:53.020 --> 02:09:55.120 align:middle line:84% And I thought that was very useful. 02:09:55.120 --> 02:09:58.340 align:middle line:84% And it felt more tangible to be able to look at something 02:09:58.340 --> 02:10:02.700 align:middle line:84% like that than the molecules website 02:10:02.700 --> 02:10:04.580 align:middle line:84% we covered briefly in the beginning, 02:10:04.580 --> 02:10:07.940 align:middle line:84% not necessarily because it was like in a context we used. 02:10:07.940 --> 02:10:11.080 align:middle line:84% But the SD molecules website, I almost 02:10:11.080 --> 02:10:14.120 align:middle line:84% didn't know what to go look for sometimes. 02:10:14.120 --> 02:10:15.260 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 02:10:15.260 --> 02:10:19.320 align:middle line:84% And for that, there is a way to systematically learn 02:10:19.320 --> 02:10:20.100 align:middle line:90% the molecules. 02:10:20.100 --> 02:10:23.880 align:middle line:84% And that would be basically go for each molecule, 02:10:23.880 --> 02:10:28.920 align:middle line:84% go read the equations, then build it yourself, 02:10:28.920 --> 02:10:32.180 align:middle line:90% then make a few examples of it. 02:10:32.180 --> 02:10:35.580 align:middle line:84% What are the use cases where I can use this structure? 02:10:35.580 --> 02:10:37.880 align:middle line:90% But that's a big investment. 02:10:37.880 --> 02:10:40.740 align:middle line:84% If you do it for even the 10 bigger ones, 02:10:40.740 --> 02:10:44.080 align:middle line:84% because some of them are almost trivial, but there are about 10 02:10:44.080 --> 02:10:47.720 align:middle line:84% that are big enough that you need to actually go 02:10:47.720 --> 02:10:48.980 align:middle line:90% through that process. 02:10:48.980 --> 02:10:50.380 align:middle line:84% And then you need to simulate it. 02:10:50.380 --> 02:10:53.560 align:middle line:84% It's not just the examples, but understand. 02:10:53.560 --> 02:10:57.040 align:middle line:84% Each of those may take two or three hours at least. 02:10:57.040 --> 02:11:00.760 align:middle line:84% So it's a full week of work to just go through the molecule 02:11:00.760 --> 02:11:02.840 align:middle line:90% systematically. 02:11:02.840 --> 02:11:04.760 align:middle line:84% I would recommend that for somebody 02:11:04.760 --> 02:11:08.280 align:middle line:84% who wants to build models regularly, 02:11:08.280 --> 02:11:11.200 align:middle line:84% but maybe it's not the most efficient way for our class. 02:11:11.200 --> 02:11:14.500 align:middle line:90% So that's, I think, pretty fair. 02:11:14.500 --> 02:11:17.140 align:middle line:84% AUDIENCE: I'd like also, maybe to have 02:11:17.140 --> 02:11:19.540 align:middle line:84% a look at the dynamics a little bit more. 02:11:19.540 --> 02:11:21.820 align:middle line:84% Because I think we looked a lot at the structures. 02:11:21.820 --> 02:11:22.620 align:middle line:84% HAZHIR RAHMANDAD: The structures, but we 02:11:22.620 --> 02:11:23.560 align:middle line:90% didn't do the dynamics. 02:11:23.560 --> 02:11:24.560 align:middle line:90% You're absolutely right. 02:11:24.560 --> 02:11:26.460 align:middle line:84% I think that's probably one thing 02:11:26.460 --> 02:11:29.340 align:middle line:84% that in the overall balance, we should 02:11:29.340 --> 02:11:33.480 align:middle line:84% have moved a little bit more towards looking at the dynamics. 02:11:33.480 --> 02:11:38.860 align:middle line:84% Because I think formulations need to be there. 02:11:38.860 --> 02:11:41.240 align:middle line:84% But even once you have the formulations, 02:11:41.240 --> 02:11:46.060 align:middle line:84% there is a whole process of making 02:11:46.060 --> 02:11:48.700 align:middle line:84% sure the model is in the right parameter 02:11:48.700 --> 02:11:50.820 align:middle line:84% space, the part of the parameter space you're 02:11:50.820 --> 02:11:52.440 align:middle line:90% targeting with your baseline. 02:11:52.440 --> 02:11:55.380 align:middle line:84% If it is very off, even if all the formulations 02:11:55.380 --> 02:11:57.820 align:middle line:84% are correct in the sense that the final model will not 02:11:57.820 --> 02:12:01.460 align:middle line:84% change at all, still, you will have weird behaviors 02:12:01.460 --> 02:12:04.320 align:middle line:84% that you can't make sense of and you may get disappointed. 02:12:04.320 --> 02:12:08.850 align:middle line:84% And so just shifting the model parameters in the right range 02:12:08.850 --> 02:12:09.990 align:middle line:90% is a skill. 02:12:09.990 --> 02:12:13.330 align:middle line:84% But also during that process, you usually detect a lot 02:12:13.330 --> 02:12:17.650 align:middle line:84% of formulation issues that, OK, this formulation I taught makes 02:12:17.650 --> 02:12:19.650 align:middle line:90% sense, but it actually doesn't. 02:12:19.650 --> 02:12:22.550 align:middle line:84% And we didn't go through that source of feedback. 02:12:22.550 --> 02:12:25.310 align:middle line:84% So I think that's definitely important. 02:12:25.310 --> 02:12:28.390 align:middle line:84% And for our PhD students who are taking general exams, 02:12:28.390 --> 02:12:31.050 align:middle line:84% they have gone through that suffering more 02:12:31.050 --> 02:12:35.850 align:middle line:84% because going through your example general exams, 02:12:35.850 --> 02:12:37.650 align:middle line:84% you will learn that it is not just 02:12:37.650 --> 02:12:39.790 align:middle line:84% coming up with good formulations, 02:12:39.790 --> 02:12:42.850 align:middle line:84% it is also getting the model to behave in the space 02:12:42.850 --> 02:12:45.370 align:middle line:84% that you want it to behave and replicate 02:12:45.370 --> 02:12:47.915 align:middle line:84% the basic reference modes that you're trying to capture. 02:12:47.915 --> 02:12:49.290 align:middle line:84% AUDIENCE: The time that we did do 02:12:49.290 --> 02:12:51.810 align:middle line:84% that with the predator-prey model was 02:12:51.810 --> 02:12:54.010 align:middle line:84% very helpful in terms of figuring out 02:12:54.010 --> 02:12:55.010 align:middle line:90% how to expand the model. 02:12:55.010 --> 02:12:56.550 align:middle line:90% HAZHIR RAHMANDAD: Yeah, yeah. 02:12:56.550 --> 02:12:59.750 align:middle line:84% And that has actually two important components. 02:12:59.750 --> 02:13:02.990 align:middle line:84% And I think breaking that down into detecting 02:13:02.990 --> 02:13:05.270 align:middle line:84% what is generating the behavior, we 02:13:05.270 --> 02:13:07.270 align:middle line:84% have gone through that two or three times 02:13:07.270 --> 02:13:10.210 align:middle line:84% in the class, fewer times than we should have. 02:13:10.210 --> 02:13:12.910 align:middle line:84% But I think it gives you a feel for it 02:13:12.910 --> 02:13:16.030 align:middle line:84% with the causal tracing, come up with a hypothesis, 02:13:16.030 --> 02:13:17.950 align:middle line:90% testing that hypothesis. 02:13:17.950 --> 02:13:20.090 align:middle line:90% That's really needed. 02:13:20.090 --> 02:13:23.430 align:middle line:84% If you don't do that, you can't shift your model 02:13:23.430 --> 02:13:25.230 align:middle line:90% towards the right direction. 02:13:25.230 --> 02:13:26.650 align:middle line:90% This is very important. 02:13:26.650 --> 02:13:29.950 align:middle line:84% I think the most common failure mode, once you 02:13:29.950 --> 02:13:33.130 align:middle line:84% have even a decent model in terms of formulations, 02:13:33.130 --> 02:13:37.830 align:middle line:84% is that you want to shift it in the right space of behavior, 02:13:37.830 --> 02:13:40.510 align:middle line:84% but you don't know how to do it, so you end up just 02:13:40.510 --> 02:13:43.390 align:middle line:90% ad hoc parameter changes. 02:13:43.390 --> 02:13:45.750 align:middle line:90% And that is groping in the dark. 02:13:45.750 --> 02:13:48.830 align:middle line:84% So think about even the simplest models we have, 02:13:48.830 --> 02:13:51.270 align:middle line:90% they have a dozen parameters. 02:13:51.270 --> 02:13:56.230 align:middle line:84% So you're walking in a dozen dimensional space. 02:13:56.230 --> 02:13:59.390 align:middle line:84% If you do a random walk in dozen dimensional space, 02:13:59.390 --> 02:14:00.250 align:middle line:90% it is hopeless. 02:14:00.250 --> 02:14:04.950 align:middle line:84% There is nothing you can do that makes a meaningful change. 02:14:04.950 --> 02:14:07.970 align:middle line:84% It's a completely hopeless process, 02:14:07.970 --> 02:14:11.330 align:middle line:84% so you definitely need to narrow it down. 02:14:11.330 --> 02:14:14.530 align:middle line:84% What is generating the current behavior? 02:14:14.530 --> 02:14:16.610 align:middle line:84% Based on that understanding, what 02:14:16.610 --> 02:14:18.730 align:middle line:84% are one or two parameters that really 02:14:18.730 --> 02:14:21.650 align:middle line:84% matter because they are shifting the behavior away from what 02:14:21.650 --> 02:14:23.870 align:middle line:90% I wanted to reflect? 02:14:23.870 --> 02:14:28.510 align:middle line:84% And let me focus on those two parameters or two equations, 02:14:28.510 --> 02:14:31.130 align:middle line:90% shift those, see the outcome. 02:14:31.130 --> 02:14:37.250 align:middle line:84% You need to do this hypothesis generation and testing 02:14:37.250 --> 02:14:42.090 align:middle line:90% or it will be very, very hard. 02:14:42.090 --> 02:14:43.970 align:middle line:90% Good. 02:14:43.970 --> 02:14:46.730 align:middle line:90% Other thoughts? 02:14:46.730 --> 02:14:50.490 align:middle line:84% AUDIENCE: I took 873 and 879, so I 02:14:50.490 --> 02:14:53.370 align:middle line:84% think it would be more helpful if there was some more-- 02:14:53.370 --> 02:14:55.453 align:middle line:84% HAZHIR RAHMANDAD: Could you speak up a little bit? 02:14:55.453 --> 02:14:57.550 align:middle line:84% AUDIENCE: Oh, I took 873 and 879. 02:14:57.550 --> 02:14:59.690 align:middle line:84% And it'd be nice if there's another course, 02:14:59.690 --> 02:15:03.590 align:middle line:84% like an intermediate level, where they do all dry structure 02:15:03.590 --> 02:15:05.470 align:middle line:90% and like molecules thing. 02:15:05.470 --> 02:15:08.850 align:middle line:84% And then this would be more like once you have some of structure, 02:15:08.850 --> 02:15:11.232 align:middle line:84% how do you use it like downstream. 02:15:11.232 --> 02:15:12.190 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 02:15:12.190 --> 02:15:13.350 align:middle line:90% That's fair. 02:15:13.350 --> 02:15:15.950 align:middle line:84% So we used to have a course that was 02:15:15.950 --> 02:15:21.710 align:middle line:84% called 875, which was actually that exact intermediate course. 02:15:21.710 --> 02:15:23.650 align:middle line:90% In fact, we had 876 as well. 02:15:23.650 --> 02:15:28.150 align:middle line:84% It's just because the structure of courses here at Sloan 02:15:28.150 --> 02:15:33.030 align:middle line:84% have moved towards whatever more MBA students take them survive 02:15:33.030 --> 02:15:37.510 align:middle line:84% and whatever fewer MBA students take die, 02:15:37.510 --> 02:15:40.010 align:middle line:84% we haven't been running those versions. 02:15:40.010 --> 02:15:43.230 align:middle line:84% And basically 879 has become the course 02:15:43.230 --> 02:15:49.390 align:middle line:84% where we iterate through different variants of modeling. 02:15:49.390 --> 02:15:56.870 align:middle line:84% But we really don't have an easy bridge between the two 02:15:56.870 --> 02:15:58.240 align:middle line:90% that is kind of that. 02:15:58.240 --> 02:15:59.820 align:middle line:90% And you're right on that. 02:15:59.820 --> 02:16:03.780 align:middle line:84% Now, the versions that are kind of doing a little bit of this 02:16:03.780 --> 02:16:06.140 align:middle line:84% is basically project-based versions, 02:16:06.140 --> 02:16:08.260 align:middle line:84% that rather than doing 10 different models 02:16:08.260 --> 02:16:10.940 align:middle line:84% during the semester, you're doing one model, 02:16:10.940 --> 02:16:14.480 align:middle line:84% but you get a lot more feedback on that 02:16:14.480 --> 02:16:17.260 align:middle line:84% so that you build that model, or maybe two, 02:16:17.260 --> 02:16:20.700 align:middle line:90% over the course of the semester. 02:16:20.700 --> 02:16:25.140 align:middle line:84% But most of the students don't get the opportunity 02:16:25.140 --> 02:16:29.780 align:middle line:84% to take two of those, except for our PhD students in the C group. 02:16:29.780 --> 02:16:32.299 align:middle line:84% Most others don't have the time or interest 02:16:32.299 --> 02:16:38.580 align:middle line:84% in taking two different versions of it at the same time. 02:16:38.580 --> 02:16:45.260 align:middle line:84% So when I was a student 25 years ago, 875 and 876 02:16:45.260 --> 02:16:47.520 align:middle line:84% existed, and they were kind of distinct courses. 02:16:47.520 --> 02:16:50.920 align:middle line:84% And one was based on modeling with projects 02:16:50.920 --> 02:16:52.120 align:middle line:90% with real-world clients. 02:16:52.120 --> 02:16:54.820 align:middle line:84% And it was a kind of a lab-based type of a setting. 02:16:54.820 --> 02:16:57.639 align:middle line:84% You would work with the client to help solve a problem using 02:16:57.639 --> 02:16:59.440 align:middle line:90% system dynamics. 02:16:59.440 --> 02:17:01.879 align:middle line:84% And then you would have an analysis course 02:17:01.879 --> 02:17:03.400 align:middle line:84% that was more about understanding 02:17:03.400 --> 02:17:08.320 align:middle line:84% model behavior and basically all of the things 02:17:08.320 --> 02:17:11.160 align:middle line:84% that we discussed on how to understand. 02:17:11.160 --> 02:17:13.480 align:middle line:90% That was a whole course. 02:17:13.480 --> 02:17:16.280 align:middle line:84% There is enough complexity there that you 02:17:16.280 --> 02:17:19.020 align:middle line:90% can do a full course on it. 02:17:19.020 --> 02:17:20.520 align:middle line:84% AUDIENCE: I think this is a good way 02:17:20.520 --> 02:17:24.559 align:middle line:84% of getting a lot of those skills through a different program 02:17:24.559 --> 02:17:25.840 align:middle line:90% every week. 02:17:25.840 --> 02:17:29.120 align:middle line:84% The thing that I become uncertain of 02:17:29.120 --> 02:17:34.160 align:middle line:84% is just what don't I know that should be basic, you know? 02:17:34.160 --> 02:17:36.959 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 02:17:36.959 --> 02:17:38.059 align:middle line:90% It's a good question. 02:17:38.059 --> 02:17:42.719 align:middle line:84% So I think what you don't know that you should 02:17:42.719 --> 02:17:46.120 align:middle line:84% know is when you start modeling right now, what 02:17:46.120 --> 02:17:47.620 align:middle line:90% will be the bottleneck for you? 02:17:47.620 --> 02:17:52.320 align:middle line:84% And my guess is from looking at the projects, some of it 02:17:52.320 --> 02:17:55.820 align:middle line:84% is that we didn't spend that much time on conceptualization, 02:17:55.820 --> 02:17:58.260 align:middle line:84% that is, starting from a messy problem 02:17:58.260 --> 02:18:01.059 align:middle line:84% to a well-defined problem definition, which 02:18:01.059 --> 02:18:05.160 align:middle line:84% we tried to do with the personal problems. 02:18:05.160 --> 02:18:07.120 align:middle line:90% I think we did some of that. 02:18:07.120 --> 02:18:10.459 align:middle line:84% But each of you got one data point there 02:18:10.459 --> 02:18:11.879 align:middle line:90% for your own project. 02:18:11.879 --> 02:18:12.920 align:middle line:90% You got some feedback. 02:18:12.920 --> 02:18:15.340 align:middle line:90% You tried to clarify it. 02:18:15.340 --> 02:18:22.420 align:middle line:84% If you had received 10 of those, those steps of the process 02:18:22.420 --> 02:18:24.360 align:middle line:90% are, I think, pretty important. 02:18:24.360 --> 02:18:28.059 align:middle line:84% They are even sometimes more important and more complex 02:18:28.059 --> 02:18:29.760 align:middle line:90% than the formulation parts. 02:18:29.760 --> 02:18:34.820 align:middle line:84% But they are also harder to pin down into a algorithmic way 02:18:34.820 --> 02:18:36.580 align:middle line:90% of thinking about it. 02:18:36.580 --> 02:18:39.340 align:middle line:84% So for that, I think even looking 02:18:39.340 --> 02:18:42.000 align:middle line:84% at what we have done with other students, that is, 02:18:42.000 --> 02:18:45.040 align:middle line:84% look at the first version of the problem definition. 02:18:45.040 --> 02:18:46.200 align:middle line:90% What is your critique? 02:18:46.200 --> 02:18:47.920 align:middle line:84% What is the critique that I offered? 02:18:47.920 --> 02:18:50.660 align:middle line:84% What was the last thing that they got to? 02:18:50.660 --> 02:18:53.160 align:middle line:90% That is helpful. 02:18:53.160 --> 02:18:54.180 align:middle line:90% You need feedback. 02:18:54.180 --> 02:18:57.180 align:middle line:84% You need to put your stake in and say, 02:18:57.180 --> 02:19:00.400 align:middle line:84% this is what I think they should do, and then 02:19:00.400 --> 02:19:04.820 align:middle line:84% get an alternative view of it, how I critiqued it, 02:19:04.820 --> 02:19:06.740 align:middle line:90% how that finalized. 02:19:06.740 --> 02:19:10.040 align:middle line:84% That is one way to get that type of feedback. 02:19:10.040 --> 02:19:14.200 align:middle line:84% But we didn't have time in the class to do it. 02:19:14.200 --> 02:19:17.740 align:middle line:84% So that would be, I think, my first place. 02:19:17.740 --> 02:19:19.879 align:middle line:84% I think on the formulation stuff, 02:19:19.879 --> 02:19:22.100 align:middle line:84% if you have a well-defined problem, 02:19:22.100 --> 02:19:26.240 align:middle line:84% actually, I was pretty happy with, by week 9, what 02:19:26.240 --> 02:19:28.440 align:middle line:84% you folks were doing in terms of the modeling 02:19:28.440 --> 02:19:30.959 align:middle line:84% of well-defined problems, well-defined in the sense 02:19:30.959 --> 02:19:34.559 align:middle line:84% that the structures are relatively well known 02:19:34.559 --> 02:19:36.719 align:middle line:84% from the problem definition and the feedbacks 02:19:36.719 --> 02:19:40.040 align:middle line:90% are relatively clear. 02:19:40.040 --> 02:19:43.700 align:middle line:84% You started to build models that were close. 02:19:43.700 --> 02:19:47.160 align:middle line:84% Comparing SIR example, that we had 02:19:47.160 --> 02:19:50.550 align:middle line:84% in the first week and the growth of a professional field 02:19:50.550 --> 02:19:54.090 align:middle line:84% we had in week 9, the gap was big. 02:19:54.090 --> 02:19:55.410 align:middle line:90% it was a much simpler. 02:19:55.410 --> 02:20:01.210 align:middle line:84% SIR is much simpler model compared to the one we had. 02:20:01.210 --> 02:20:03.610 align:middle line:84% And you guys were doing much better with the second one. 02:20:03.610 --> 02:20:08.670 align:middle line:84% So I think I could see that learning was happening 02:20:08.670 --> 02:20:13.930 align:middle line:84% pretty well, maybe better than I expected, 02:20:13.930 --> 02:20:17.330 align:middle line:84% given where, on average, the classes started. 02:20:17.330 --> 02:20:22.430 align:middle line:84% And SIR, I think we had a few quite a few problems 02:20:22.430 --> 02:20:24.410 align:middle line:90% in everybody's submissions. 02:20:24.410 --> 02:20:28.370 align:middle line:84% But if you start with a new problem, 02:20:28.370 --> 02:20:31.190 align:middle line:84% the conceptualization to get it to a well-defined problem 02:20:31.190 --> 02:20:34.510 align:middle line:90% will remain still complex. 02:20:34.510 --> 02:20:36.350 align:middle line:84% We didn't do anything with data, and that 02:20:36.350 --> 02:20:43.350 align:middle line:84% is a whole big additional thing that one 02:20:43.350 --> 02:20:47.550 align:middle line:84% needs to bring into modeling for real-world applications. 02:20:47.550 --> 02:20:54.190 align:middle line:84% And we don't have time to do that in one iteration. 02:20:54.190 --> 02:20:56.460 align:middle line:90% 02:20:56.460 --> 02:20:58.710 align:middle line:84% AUDIENCE: One of the things I thought was helpful too, 02:20:58.710 --> 02:21:01.230 align:middle line:84% honestly, about the past few weeks, 02:21:01.230 --> 02:21:04.850 align:middle line:84% is that I feel like going through everybody's models 02:21:04.850 --> 02:21:07.250 align:middle line:84% with different problems is really helpful in terms 02:21:07.250 --> 02:21:10.290 align:middle line:84% of getting into the process of, oh, it actually 02:21:10.290 --> 02:21:13.370 align:middle line:84% is totally possible to look at an existing model 02:21:13.370 --> 02:21:15.610 align:middle line:84% that I do not understand at all yet, and come 02:21:15.610 --> 02:21:19.930 align:middle line:84% to understand both what the actual problem is 02:21:19.930 --> 02:21:22.670 align:middle line:84% and also the formulation surrounding it. 02:21:22.670 --> 02:21:24.230 align:middle line:90% HAZHIR RAHMANDAD: Yeah, good. 02:21:24.230 --> 02:21:28.770 align:middle line:84% I'm glad to hear that, because I don't know how easy to follow it 02:21:28.770 --> 02:21:32.030 align:middle line:84% is for you when you are seeing one of these new problems. 02:21:32.030 --> 02:21:36.590 align:middle line:84% So it is always a risk that we go too fast. 02:21:36.590 --> 02:21:38.178 align:middle line:84% And then half of the class is looking 02:21:38.178 --> 02:21:39.470 align:middle line:90% at this like, what's happening? 02:21:39.470 --> 02:21:43.810 align:middle line:84% I don't know what this model is about and why. 02:21:43.810 --> 02:21:48.950 align:middle line:84% And I have had this challenge of balancing how quickly we 02:21:48.950 --> 02:21:49.810 align:middle line:90% want to do things. 02:21:49.810 --> 02:21:53.990 align:middle line:84% So I kind go very fast into feedback, boom, boom, boom, 02:21:53.990 --> 02:21:58.150 align:middle line:84% boom, versus trying to get everybody in the class 02:21:58.150 --> 02:22:01.630 align:middle line:84% connect with that feedback and compare that 02:22:01.630 --> 02:22:02.890 align:middle line:90% with what they think about it. 02:22:02.890 --> 02:22:07.790 align:middle line:84% So it has been a challenge for me to balance that. 02:22:07.790 --> 02:22:11.470 align:middle line:84% And I'm actually curious what everybody's response 02:22:11.470 --> 02:22:14.150 align:middle line:90% to that, what we did. 02:22:14.150 --> 02:22:18.550 align:middle line:90% Was it on the too fast side? 02:22:18.550 --> 02:22:20.730 align:middle line:90% Or was it OK, speed-wise? 02:22:20.730 --> 02:22:22.410 align:middle line:90% I don't think it was too slow. 02:22:22.410 --> 02:22:28.590 align:middle line:90% 02:22:28.590 --> 02:22:32.790 align:middle line:84% And not only for this last three weeks, I think even before that, 02:22:32.790 --> 02:22:35.910 align:middle line:84% there was always this balancing act of trying to go through 02:22:35.910 --> 02:22:39.770 align:middle line:84% as many models as possible and give as many pieces of feedback, 02:22:39.770 --> 02:22:46.900 align:middle line:84% but not overwhelm folks or "I didn't figure out what was being 02:22:46.900 --> 02:22:49.080 align:middle line:90% said" type of response. 02:22:49.080 --> 02:22:56.460 align:middle line:90% 02:22:56.460 --> 02:22:58.040 align:middle line:84% AUDIENCE: Sometimes it was confusing, 02:22:58.040 --> 02:23:00.060 align:middle line:84% but I think usually the way you started out 02:23:00.060 --> 02:23:03.700 align:middle line:84% was you spent like 30 or 40 minutes on the first model, that 02:23:03.700 --> 02:23:07.280 align:middle line:84% was like slow enough to catch up. 02:23:07.280 --> 02:23:10.360 align:middle line:84% And then by the time you were on model 5 with the same problems 02:23:10.360 --> 02:23:15.565 align:middle line:84% and terms, it was not as bad as when it started out. 02:23:15.565 --> 02:23:16.440 align:middle line:90% HAZHIR RAHMANDAD: OK. 02:23:16.440 --> 02:23:19.140 align:middle line:84% That's also a good reminder for me 02:23:19.140 --> 02:23:23.260 align:middle line:84% to just use that first model to just settle 02:23:23.260 --> 02:23:25.360 align:middle line:90% into seeing the big picture. 02:23:25.360 --> 02:23:28.100 align:middle line:84% And I think by week three or four, 02:23:28.100 --> 02:23:31.880 align:middle line:84% we started looking at the structures of the models first, 02:23:31.880 --> 02:23:35.740 align:middle line:84% so that kind of big picture is relatively clear on how 02:23:35.740 --> 02:23:39.140 align:middle line:90% the stocks are cut. 02:23:39.140 --> 02:23:44.380 align:middle line:84% I think Jay Forrester always used to say, cutting the stocks, 02:23:44.380 --> 02:23:47.380 align:middle line:84% picking the right stocks is half of the modeling. 02:23:47.380 --> 02:23:52.060 align:middle line:84% It's really the hard part of the modeling. 02:23:52.060 --> 02:23:55.360 align:middle line:84% So if you do that well, the rest is much easier 02:23:55.360 --> 02:23:57.340 align:middle line:90% to get into the right place. 02:23:57.340 --> 02:24:02.540 align:middle line:84% And I think that is a very good observation generally. 02:24:02.540 --> 02:24:06.160 align:middle line:84% AUDIENCE: I'll ask question that is on my mind, which 02:24:06.160 --> 02:24:13.880 align:middle line:84% is the class was good for figuring out how to represent 02:24:13.880 --> 02:24:16.120 align:middle line:84% certain things, but what to represent 02:24:16.120 --> 02:24:17.560 align:middle line:84% is still kind of an open question. 02:24:17.560 --> 02:24:18.477 align:middle line:90% HAZHIR RAHMANDAD: Yes. 02:24:18.477 --> 02:24:21.680 align:middle line:90% 02:24:21.680 --> 02:24:25.240 align:middle line:84% AUDIENCE: And there was an obvious transition 02:24:25.240 --> 02:24:27.680 align:middle line:84% in the problem statements you gave 02:24:27.680 --> 02:24:32.720 align:middle line:84% us, where weeks one through four or five, 02:24:32.720 --> 02:24:35.100 align:middle line:84% it's pretty clear what we had to represent. 02:24:35.100 --> 02:24:41.400 align:middle line:84% And then we all practiced how to implement that and learn 02:24:41.400 --> 02:24:44.560 align:middle line:84% the power of the molecules and linking them up, 02:24:44.560 --> 02:24:48.220 align:middle line:84% and the expressiveness you can get from just connecting those 02:24:48.220 --> 02:24:49.540 align:middle line:90% correctly. 02:24:49.540 --> 02:24:56.140 align:middle line:84% But then the latter half was more open ended. 02:24:56.140 --> 02:25:01.340 align:middle line:84% And thinking through what is important to represent 02:25:01.340 --> 02:25:06.682 align:middle line:84% to capture a particular problems is the essence of that. 02:25:06.682 --> 02:25:07.640 align:middle line:90% HAZHIR RAHMANDAD: Yeah. 02:25:07.640 --> 02:25:08.280 align:middle line:90% No, absolutely. 02:25:08.280 --> 02:25:10.340 align:middle line:84% And I think that conceptualization part 02:25:10.340 --> 02:25:14.000 align:middle line:90% is what we didn't cover much. 02:25:14.000 --> 02:25:19.180 align:middle line:84% And I feel actually, this is the right way 02:25:19.180 --> 02:25:20.520 align:middle line:90% going through the experience. 02:25:20.520 --> 02:25:23.160 align:middle line:84% That is, that conceptualization is the harder problem. 02:25:23.160 --> 02:25:26.660 align:middle line:84% You need to have enough experience with the components 02:25:26.660 --> 02:25:30.380 align:middle line:84% that give you affordances for capturing different things 02:25:30.380 --> 02:25:33.920 align:middle line:84% to later come and say, OK, now I have a new problem. 02:25:33.920 --> 02:25:36.520 align:middle line:84% How do I cut it so that I can use those affordances, 02:25:36.520 --> 02:25:39.682 align:middle line:84% those capabilities to capture it? 02:25:39.682 --> 02:25:41.140 align:middle line:84% AUDIENCE: The risk you run, though, 02:25:41.140 --> 02:25:44.120 align:middle line:84% is you overindex on your avoidance capability 02:25:44.120 --> 02:25:49.560 align:middle line:84% and it shortcuts any clear thinking about what's 02:25:49.560 --> 02:25:50.880 align:middle line:90% actually going on here. 02:25:50.880 --> 02:25:53.240 align:middle line:84% HAZHIR RAHMANDAD: Yes, that's true. 02:25:53.240 --> 02:25:58.040 align:middle line:84% I think I hope Nelson's session gave you a little bit of a focus 02:25:58.040 --> 02:26:01.360 align:middle line:84% on the other way of, OK, you really 02:26:01.360 --> 02:26:03.160 align:middle line:84% need to ask the hard question of, 02:26:03.160 --> 02:26:05.460 align:middle line:84% why am I modeling in the first place? 02:26:05.460 --> 02:26:09.360 align:middle line:84% And if I'm modeling it in the first place, what is really 02:26:09.360 --> 02:26:12.500 align:middle line:90% critical and what else is out? 02:26:12.500 --> 02:26:14.340 align:middle line:84% And we didn't really discuss that, 02:26:14.340 --> 02:26:19.320 align:middle line:84% but that's an important part of that conceptualization question. 02:26:19.320 --> 02:26:21.520 align:middle line:84% AUDIENCE: The most helpful question to me 02:26:21.520 --> 02:26:26.240 align:middle line:84% that he really asked us was, what is the hypothesis you have 02:26:26.240 --> 02:26:31.240 align:middle line:84% that this is a dynamic problem, as well? 02:26:31.240 --> 02:26:34.760 align:middle line:84% Specifically, what feedbacks kind of happened. 02:26:34.760 --> 02:26:36.640 align:middle line:90% HAZHIR RAHMANDAD: Yes. 02:26:36.640 --> 02:26:39.910 align:middle line:84% And that question is very, very important 02:26:39.910 --> 02:26:44.030 align:middle line:84% in getting concrete contributions 02:26:44.030 --> 02:26:47.510 align:middle line:90% that academics may appreciate. 02:26:47.510 --> 02:26:50.270 align:middle line:84% It might be secondary importance for a lot 02:26:50.270 --> 02:26:55.790 align:middle line:90% of simple operational problems. 02:26:55.790 --> 02:27:02.007 align:middle line:84% Like if I'm thinking about these sales of coffee makers thing 02:27:02.007 --> 02:27:04.590 align:middle line:84% that I was talking about, there is really nothing that complex 02:27:04.590 --> 02:27:05.650 align:middle line:90% dynamically. 02:27:05.650 --> 02:27:08.710 align:middle line:84% I just need to make sense of some data that I have 02:27:08.710 --> 02:27:10.410 align:middle line:90% and see what are the trends. 02:27:10.410 --> 02:27:12.710 align:middle line:84% And to do that, I need actually a small chunk 02:27:12.710 --> 02:27:14.150 align:middle line:90% of a dynamic model. 02:27:14.150 --> 02:27:16.810 align:middle line:84% There is nothing really interesting dynamically there. 02:27:16.810 --> 02:27:19.710 align:middle line:84% But if I don't do the accounting correctly, 02:27:19.710 --> 02:27:21.570 align:middle line:84% I wouldn't make sense of my data. 02:27:21.570 --> 02:27:24.710 align:middle line:84% That's a different type of a use case. 02:27:24.710 --> 02:27:28.710 align:middle line:84% So for some of you, those use cases 02:27:28.710 --> 02:27:30.310 align:middle line:90% may be even more important. 02:27:30.310 --> 02:27:35.110 align:middle line:84% In real-world problems, 80%, 90% of it 02:27:35.110 --> 02:27:43.650 align:middle line:84% is these kind of a basic, low-grade, small dynamic issues 02:27:43.650 --> 02:27:47.150 align:middle line:84% that you actually need to handle one way or another. 02:27:47.150 --> 02:27:50.630 align:middle line:84% Now, sometimes just stock and flow thinking is all you need. 02:27:50.630 --> 02:27:53.810 align:middle line:84% And then you will write your code in your Excel spreadsheet 02:27:53.810 --> 02:27:57.930 align:middle line:84% or in Python to transform the data in a way that captures 02:27:57.930 --> 02:28:02.030 align:middle line:90% those small dynamic factors. 02:28:02.030 --> 02:28:04.090 align:middle line:90% And that's very legitimate. 02:28:04.090 --> 02:28:06.470 align:middle line:84% You wouldn't be publishing papers with those. 02:28:06.470 --> 02:28:10.410 align:middle line:84% But it is still something useful. 02:28:10.410 --> 02:28:13.210 align:middle line:84% And therefore, I didn't discount those ways 02:28:13.210 --> 02:28:15.570 align:middle line:84% of doing it in this class, partly because I 02:28:15.570 --> 02:28:19.730 align:middle line:84% think it's a useful skill in the real-world, practical 02:28:19.730 --> 02:28:22.290 align:middle line:90% applications. 02:28:22.290 --> 02:28:24.815 align:middle line:84% AUDIENCE: I find that to be my issue. 02:28:24.815 --> 02:28:26.690 align:middle line:84% And I think that that's what I was struggling 02:28:26.690 --> 02:28:29.250 align:middle line:84% with the final project is determining 02:28:29.250 --> 02:28:33.750 align:middle line:84% or, in general, actually, is all these problems that I think of, 02:28:33.750 --> 02:28:39.130 align:middle line:84% it's not lending to a really interesting dynamic model. 02:28:39.130 --> 02:28:43.630 align:middle line:84% And then when it does, maybe there's a skills mismatch 02:28:43.630 --> 02:28:48.310 align:middle line:84% or I don't know if the right data or something. 02:28:48.310 --> 02:28:57.630 align:middle line:84% I don't know for this course, but brainstorming 02:28:57.630 --> 02:29:02.310 align:middle line:84% or how to have a taste for what kinds of problems 02:29:02.310 --> 02:29:04.830 align:middle line:90% are really the good ones. 02:29:04.830 --> 02:29:07.490 align:middle line:84% HAZHIR RAHMANDAD: It's a very important, very hard question. 02:29:07.490 --> 02:29:11.750 align:middle line:84% I think I fully agree with you on its importance. 02:29:11.750 --> 02:29:17.150 align:middle line:84% And some of it really comes from, 02:29:17.150 --> 02:29:20.090 align:middle line:84% so I see two ways of getting closer to it. 02:29:20.090 --> 02:29:24.070 align:middle line:84% One is understanding the domain you are contributing to. 02:29:24.070 --> 02:29:28.330 align:middle line:84% You really need to know what is known, what is unknown, 02:29:28.330 --> 02:29:35.130 align:middle line:84% what is assumed, so what are the mechanisms that people account 02:29:35.130 --> 02:29:37.430 align:middle line:84% when they think about that problem. 02:29:37.430 --> 02:29:39.930 align:middle line:84% You have to have those mechanisms 02:29:39.930 --> 02:29:41.970 align:middle line:90% as your starting point. 02:29:41.970 --> 02:29:44.710 align:middle line:84% That's starting from where the current literature is. 02:29:44.710 --> 02:29:48.810 align:middle line:84% And then add an additional mechanism 02:29:48.810 --> 02:29:52.490 align:middle line:84% that you find relevant, important, interesting 02:29:52.490 --> 02:29:56.570 align:middle line:84% from usually qualitative data, from some insight that 02:29:56.570 --> 02:30:00.210 align:middle line:84% has not been yet incorporated into the literature. 02:30:00.210 --> 02:30:04.130 align:middle line:84% And then having enough-- so this is the second path-- 02:30:04.130 --> 02:30:07.610 align:middle line:84% enough intuition from building models that you would say, 02:30:07.610 --> 02:30:11.710 align:middle line:84% I think that mechanism is going to have a meaningful impact. 02:30:11.710 --> 02:30:14.570 align:middle line:84% It will change the dynamics in a way. 02:30:14.570 --> 02:30:16.290 align:middle line:90% That is my hunch. 02:30:16.290 --> 02:30:19.710 align:middle line:84% And those hunches are uncalibrated at the beginning, 02:30:19.710 --> 02:30:20.710 align:middle line:90% so you have no clue. 02:30:20.710 --> 02:30:26.050 align:middle line:84% And as you build models, those hunches become more reliable. 02:30:26.050 --> 02:30:28.490 align:middle line:84% So a very experienced person would 02:30:28.490 --> 02:30:32.420 align:middle line:84% come to a new field, look at the literature, would say, 02:30:32.420 --> 02:30:35.400 align:middle line:84% OK, these are the structures, I can see them as the stocks 02:30:35.400 --> 02:30:37.520 align:middle line:90% and flows, how they work. 02:30:37.520 --> 02:30:40.280 align:middle line:84% But then there is this mechanism that I 02:30:40.280 --> 02:30:44.280 align:middle line:84% think is important, nobody has seen, that makes a good paper. 02:30:44.280 --> 02:30:49.720 align:middle line:84% That's how you bring those two pieces together, typically. 02:30:49.720 --> 02:30:52.480 align:middle line:84% Some of it is the homework on the domain 02:30:52.480 --> 02:30:53.900 align:middle line:90% you are contributing to. 02:30:53.900 --> 02:30:57.200 align:middle line:84% Some of it is the homework in your system dynamics skill 02:30:57.200 --> 02:30:59.000 align:middle line:84% development so that you can detect 02:30:59.000 --> 02:31:01.720 align:middle line:90% those interesting mechanisms. 02:31:01.720 --> 02:31:04.260 align:middle line:84% That's as much as I can put a structure around that. 02:31:04.260 --> 02:31:10.120 align:middle line:84% I don't think there is much more you can say in general about it. 02:31:10.120 --> 02:31:11.400 align:middle line:90% Yeah. 02:31:11.400 --> 02:31:15.180 align:middle line:84% OK, I think it's a good time to stop. 02:31:15.180 --> 02:31:18.780 align:middle line:84% We have a speaker coming for a seminar, 02:31:18.780 --> 02:31:21.900 align:middle line:84% so feel free to stick around for our talk. 02:31:21.900 --> 02:31:22.780 align:middle line:90% And thanks a lot. 02:31:22.780 --> 02:31:24.800 align:middle line:84% It was a great semester with you. 02:31:24.800 --> 02:31:25.300 align:middle line:90% Thank you. 02:31:25.300 --> 02:31:28.650 align:middle line:90% [APPLAUSE] 02:31:28.650 --> 02:31:31.000 align:middle line:90%