1
00:00:00,070 --> 00:00:01,780
The following
content is provided
2
00:00:01,780 --> 00:00:04,019
under a Creative
Commons license.
3
00:00:04,019 --> 00:00:06,870
Your support will help MIT
OpenCourseWare continue
4
00:00:06,870 --> 00:00:10,730
to offer high quality
educational resources for free.
5
00:00:10,730 --> 00:00:13,330
To make a donation or
view additional materials
6
00:00:13,330 --> 00:00:15,780
from hundreds of
MIT courses, visit
7
00:00:15,780 --> 00:00:22,659
MIT OpenCourseWare
at ocw.mit.edu
8
00:00:22,659 --> 00:00:24,200
ROSS COLLINS: Welcome
to the tutorial
9
00:00:24,200 --> 00:00:26,640
on the method of
simulated moments.
10
00:00:26,640 --> 00:00:28,430
Collaborating together
on this tutorial
11
00:00:28,430 --> 00:00:32,320
is Armin Ashoury, Ross
Collins, and Ali Kamil.
12
00:00:32,320 --> 00:00:34,555
We will present the
tutorial in two parts.
13
00:00:34,555 --> 00:00:36,920
The first part, presented
by me, Ross Collins,
14
00:00:36,920 --> 00:00:40,717
is an overview of the MSM, why
it's useful, and how it works.
15
00:00:40,717 --> 00:00:42,550
The second part, presented
by Armin Ashoury,
16
00:00:42,550 --> 00:00:43,966
will go through
the details of how
17
00:00:43,966 --> 00:00:47,436
to implement the
MSM in Vensim DSS.
18
00:00:47,436 --> 00:00:48,810
Background
information on the MSM
19
00:00:48,810 --> 00:00:52,680
is included in the associated
book chapter with this video.
20
00:00:52,680 --> 00:00:55,500
The chapter goes into much more
mathematical detail in the MSM
21
00:00:55,500 --> 00:00:57,470
formulation, which the
interested viewer should
22
00:00:57,470 --> 00:00:59,540
consult if looking
for more information.
23
00:00:59,540 --> 00:01:02,791
In this video, we do not dissect
every individual formula,
24
00:01:02,791 --> 00:01:04,999
but instead try to give a
useful overview of the MSM,
25
00:01:04,999 --> 00:01:07,690
and then apply it
to a simple example.
26
00:01:07,690 --> 00:01:09,650
The step-by-step
replication the tutorial
27
00:01:09,650 --> 00:01:12,233
will likely require reading and
understanding the book chapter
28
00:01:12,233 --> 00:01:13,480
in more detail.
29
00:01:13,480 --> 00:01:15,210
Still, this video
should give a flavor
30
00:01:15,210 --> 00:01:17,960
of why MSM is useful, how it
works in the general level,
31
00:01:17,960 --> 00:01:21,140
and what sort of time and effort
is required to implement it
32
00:01:21,140 --> 00:01:22,370
in system dynamics models.
33
00:01:25,890 --> 00:01:28,070
So what does the MSM let you do?
34
00:01:28,070 --> 00:01:30,470
Well, according
to this quote, it
35
00:01:30,470 --> 00:01:32,430
allows one to "compare
time series data
36
00:01:32,430 --> 00:01:34,580
against the same
variables in a model,
37
00:01:34,580 --> 00:01:37,160
and minimize the weighted sum
of a function of the error term
38
00:01:37,160 --> 00:01:39,520
by changing the uncertain
parameters until best
39
00:01:39,520 --> 00:01:42,640
fitting estimates are found
through optimization,"
40
00:01:42,640 --> 00:01:45,420
This is important
for model validation.
41
00:01:45,420 --> 00:01:47,610
So specifically
what MSM does is it
42
00:01:47,610 --> 00:01:49,470
compares the
moments of real data
43
00:01:49,470 --> 00:01:51,710
against the moments
of simulated data
44
00:01:51,710 --> 00:01:54,740
or just the model output
from a simulation.
45
00:01:54,740 --> 00:01:58,830
So moments are things like
mean, variance, skewness.
46
00:01:58,830 --> 00:02:00,011
Kurtosis is another one.
47
00:02:00,011 --> 00:02:02,260
These are the first, second,
third, and fourth moments
48
00:02:02,260 --> 00:02:03,710
respectively.
49
00:02:03,710 --> 00:02:06,510
An important thing to realize
is that unknown parameters
50
00:02:06,510 --> 00:02:10,759
in a model will alter
the simulated moments.
51
00:02:10,759 --> 00:02:14,070
So if you have some unknown
parameters, x, and some moment,
52
00:02:14,070 --> 00:02:19,180
y, if you change x,
then y will change.
53
00:02:19,180 --> 00:02:24,100
So what the optimization method
does-- referred to in the quote
54
00:02:24,100 --> 00:02:27,940
up at the top-- is
it will find values
55
00:02:27,940 --> 00:02:30,310
for our unknown
parameters that minimize
56
00:02:30,310 --> 00:02:31,784
the weighted
difference-- and I'll
57
00:02:31,784 --> 00:02:34,450
get to why weighted is important
later-- the weighted difference
58
00:02:34,450 --> 00:02:36,490
between real and
simulated moments.
59
00:02:36,490 --> 00:02:41,060
And these weighted differences
are called the error terms.
60
00:02:41,060 --> 00:02:47,410
So we're going to apply this
MSM to a simple obesity model.
61
00:02:47,410 --> 00:02:52,110
We basically have
weight data in Excel--
62
00:02:52,110 --> 00:02:55,460
this Excel file is included
with this video-- we have
63
00:02:55,460 --> 00:02:58,580
weight data for a population
of 1,000 individuals at five
64
00:02:58,580 --> 00:03:00,240
different points in
time, specifically
65
00:03:00,240 --> 00:03:02,610
years 1, 5, 10, 15, and 20.
66
00:03:02,610 --> 00:03:06,460
The moment conditions we use
are mean and standard deviation.
67
00:03:06,460 --> 00:03:10,880
And given that we have five
years of data and two moments,
68
00:03:10,880 --> 00:03:13,701
we have 10 moments in total.
69
00:03:13,701 --> 00:03:15,200
So the unknown
parameters that we're
70
00:03:15,200 --> 00:03:18,580
trying to estimate
through the MSM technique
71
00:03:18,580 --> 00:03:20,940
are called Overfeeding,
Starvation,
72
00:03:20,940 --> 00:03:24,010
and Energy Intake Extra Trend.
73
00:03:24,010 --> 00:03:26,610
So I am going to show
this model really quick.
74
00:03:26,610 --> 00:03:28,830
It's a very simple
model that tracks
75
00:03:28,830 --> 00:03:30,900
the weight gain of
1,000 individuals.
76
00:03:30,900 --> 00:03:33,260
You can see you have the
stock structure here,
77
00:03:33,260 --> 00:03:36,310
an array of 1,000 people, so
we're utilizing subscripts
78
00:03:36,310 --> 00:03:38,700
to denote these
1,000 individuals.
79
00:03:38,700 --> 00:03:43,390
And the major feedback is
that as weight change accrues,
80
00:03:43,390 --> 00:03:48,960
you have some energy intake
balance that changes.
81
00:03:48,960 --> 00:03:52,150
This feeds back into
one's energy intake
82
00:03:52,150 --> 00:03:54,270
through the starvation
and overfeed parameters
83
00:03:54,270 --> 00:03:56,430
and then alters
the weight change.
84
00:03:56,430 --> 00:03:58,300
So the three uncertain
parameters are here.
85
00:03:58,300 --> 00:04:01,550
We have an underlying
trend in energy intake
86
00:04:01,550 --> 00:04:03,914
that impacts the weight
trajectory of each individual.
87
00:04:03,914 --> 00:04:06,205
And then we have the starvation
and overfeed parameters
88
00:04:06,205 --> 00:04:08,970
that impact this major feedback.
89
00:04:08,970 --> 00:04:11,930
If energy intake balance
is on the heavier side,
90
00:04:11,930 --> 00:04:15,300
starvation kicks in with
some parameter of force.
91
00:04:15,300 --> 00:04:17,709
From the lower side,
the overfeed parameter
92
00:04:17,709 --> 00:04:21,700
kicks in with some
amount of force.
93
00:04:21,700 --> 00:04:23,320
So let's see.
94
00:04:23,320 --> 00:04:25,580
If we click on the weight
here and show a graph,
95
00:04:25,580 --> 00:04:28,750
we're only going to get to
show 16 of the individuals,
96
00:04:28,750 --> 00:04:30,940
but you can see the
trajectory of them
97
00:04:30,940 --> 00:04:34,210
through time--
over the 20 years.
98
00:04:34,210 --> 00:04:36,920
The initial conditions
are captured here.
99
00:04:36,920 --> 00:04:41,045
The mean of individuals
is 80 kilograms--
100
00:04:41,045 --> 00:04:43,260
so you can see it sort
of circled around there--
101
00:04:43,260 --> 00:04:45,350
and the standard deviation
is 5 kilograms, which
102
00:04:45,350 --> 00:04:47,566
is why the initial
conditions differ.
103
00:04:47,566 --> 00:04:48,940
But you can see
the trajectories,
104
00:04:48,940 --> 00:04:51,630
and we have this for
1,000 individuals.
105
00:04:51,630 --> 00:04:54,570
So then the idea of the
MSM is you have these 1,000
106
00:04:54,570 --> 00:04:59,460
trajectories, and you
have the moments that
107
00:04:59,460 --> 00:05:01,170
define these trajectories.
108
00:05:01,170 --> 00:05:04,040
And we compare that
against the real data
109
00:05:04,040 --> 00:05:05,310
that we have in the exercise.
110
00:05:08,680 --> 00:05:09,964
So let's see.
111
00:05:09,964 --> 00:05:11,630
Final slide that I'm
going to talk about
112
00:05:11,630 --> 00:05:16,050
before I hand it off to
Armin to go into the Vensim
113
00:05:16,050 --> 00:05:17,310
implementation.
114
00:05:17,310 --> 00:05:20,970
So basically, there are
four overarching steps
115
00:05:20,970 --> 00:05:25,050
and four Vensim models
associated to each step.
116
00:05:25,050 --> 00:05:28,570
So in the first step, we are
basically making the first cut
117
00:05:28,570 --> 00:05:32,440
at estimating these
three unknown parameters.
118
00:05:32,440 --> 00:05:36,330
And to do that,
we need-- and I'm
119
00:05:36,330 --> 00:05:38,180
going to pull up the
book chapter here--
120
00:05:38,180 --> 00:05:42,067
we need to minimize the
weighted differences
121
00:05:42,067 --> 00:05:42,900
of these parameters.
122
00:05:42,900 --> 00:05:45,483
And that's what you can see here
in equation three in the book
123
00:05:45,483 --> 00:05:46,310
chapter.
124
00:05:46,310 --> 00:05:50,280
This theta hat is the set
of parameters-- in our case
125
00:05:50,280 --> 00:05:52,190
just three-- and
we want to minimize
126
00:05:52,190 --> 00:05:56,180
the difference between simulated
moments and the real moments
127
00:05:56,180 --> 00:05:58,050
to use for data.
128
00:05:58,050 --> 00:06:00,010
And we have here in
the middle this W,
129
00:06:00,010 --> 00:06:01,876
which is a weighting matrix.
130
00:06:01,876 --> 00:06:03,500
And the purpose of
the weighting matrix
131
00:06:03,500 --> 00:06:08,000
is to give less weight to really
noisy and uncertain moments,
132
00:06:08,000 --> 00:06:11,660
since those are going to sort
of hijacked the optimization
133
00:06:11,660 --> 00:06:16,040
process when trying
to choose parameters.
134
00:06:16,040 --> 00:06:19,750
So the first W that we
use in the first step
135
00:06:19,750 --> 00:06:25,050
is essentially just taking
the real data moments
136
00:06:25,050 --> 00:06:28,150
and squaring them and then
taking the reciprocal.
137
00:06:28,150 --> 00:06:31,720
And those are along the
diagonal elements of W. Armin
138
00:06:31,720 --> 00:06:33,220
will go into more
detail about that,
139
00:06:33,220 --> 00:06:36,580
but that's our initial
starting point for W.
140
00:06:36,580 --> 00:06:40,360
So the output of that are
an initial set of parameter
141
00:06:40,360 --> 00:06:43,230
estimates, not
necessarily optimized,
142
00:06:43,230 --> 00:06:46,420
but certainly informed by
this initial weighting matrix.
143
00:06:46,420 --> 00:06:50,060
The second step is to calculate
essentially a more efficiently
144
00:06:50,060 --> 00:06:56,030
weighted W, or what we call W*,
and it uses an algorithm that
145
00:06:56,030 --> 00:06:57,675
is discussed in the chapter.
146
00:06:57,675 --> 00:07:00,657
It's equation four down here.
147
00:07:00,657 --> 00:07:03,240
As you can see, it's reasonably
complicated and requires first
148
00:07:03,240 --> 00:07:05,150
calculating the
variance-covariance matrix
149
00:07:05,150 --> 00:07:08,360
of the simulated moments, which
again we can do in Vensim--
150
00:07:08,360 --> 00:07:10,060
even though it's
easier in MATLAB--
151
00:07:10,060 --> 00:07:15,340
and W* is simply the
inverse of this matrix.
152
00:07:15,340 --> 00:07:18,050
So once we've got W*, we
basically go into the second
153
00:07:18,050 --> 00:07:21,140
step, which is very
similar to the first step.
154
00:07:21,140 --> 00:07:24,380
It's an optimization
yet again, but now we
155
00:07:24,380 --> 00:07:26,840
have a more
efficiently weighted W.
156
00:07:26,840 --> 00:07:31,817
And that gives us a better
set of optimized parameters.
157
00:07:31,817 --> 00:07:33,400
The important thing
to note is that we
158
00:07:33,400 --> 00:07:37,550
can iterate if we
are not happy with W,
159
00:07:37,550 --> 00:07:40,710
but the final step involves
calculating competence
160
00:07:40,710 --> 00:07:42,910
intervals for each of
our uncertain parameters.
161
00:07:42,910 --> 00:07:45,630
Again, in our case,
we have three of them.
162
00:07:45,630 --> 00:07:47,540
So these confidence
intervals are just
163
00:07:47,540 --> 00:07:50,280
like other confidence intervals
and statistical exercises.
164
00:07:50,280 --> 00:07:52,970
We want those bounds to
be as tight as possible,
165
00:07:52,970 --> 00:07:56,610
and we don't want
them to include zero.
166
00:07:56,610 --> 00:08:00,460
So that is the
overview of the MSM.
167
00:08:00,460 --> 00:08:02,370
I'm going to hand
it off now to Armin,
168
00:08:02,370 --> 00:08:07,830
who is going to go through
the major steps actually
169
00:08:07,830 --> 00:08:10,020
implemented in Vensim.
170
00:08:10,020 --> 00:08:12,609
So thank you for listening.
171
00:08:12,609 --> 00:08:14,150
ARMIN ASHOURY: In
each model, we have
172
00:08:14,150 --> 00:08:18,480
a part that captured the moments
we need for the MSM process.
173
00:08:18,480 --> 00:08:21,250
So here, for example,
in the FirstStep model,
174
00:08:21,250 --> 00:08:25,590
we see that we have the
main model of obesity.
175
00:08:25,590 --> 00:08:27,330
And here we have
the part that tries
176
00:08:27,330 --> 00:08:31,120
to capture the different
moments that we have.
177
00:08:31,120 --> 00:08:34,450
For example, here we want to
catch the mean of body weights
178
00:08:34,450 --> 00:08:36,720
and also similar
deviation of body weights.
179
00:08:36,720 --> 00:08:42,330
Simply, this part tries to
create a matrix of moments.
180
00:08:46,040 --> 00:08:47,900
For each column, we
have one simulation.
181
00:08:47,900 --> 00:08:50,650
So here, for example, we
have like 10 simulations.
182
00:08:50,650 --> 00:08:54,890
And for each row, we have
one of our moments here.
183
00:08:54,890 --> 00:08:59,565
Since we have like five years of
data, which are year 1, 5, 10,
184
00:08:59,565 --> 00:09:04,040
15, and 20, we have to
use two moments, which
185
00:09:04,040 --> 00:09:06,670
are mean and standard deviation.
186
00:09:06,670 --> 00:09:09,940
We have like 10 rows that,
for the first five years,
187
00:09:09,940 --> 00:09:16,020
we have first five rows, we
have the mean of our years.
188
00:09:16,020 --> 00:09:19,845
And for the second five rows,
we have a standard deviation
189
00:09:19,845 --> 00:09:22,040
of our years.
190
00:09:22,040 --> 00:09:27,080
So we can create this matrix
in whatever way you want,
191
00:09:27,080 --> 00:09:29,230
but here we try to
capture these moments
192
00:09:29,230 --> 00:09:35,756
and create that matrix
using second flow system.
193
00:09:35,756 --> 00:09:37,710
In the FirstStep
of moment, we tried
194
00:09:37,710 --> 00:09:41,840
to find three unknown parameters
of obesity model, which
195
00:09:41,840 --> 00:09:44,866
are extra trends,
starvation, overfeed.
196
00:09:44,866 --> 00:09:48,160
We use formula five
in the book chapter
197
00:09:48,160 --> 00:09:50,000
and try to optimize
those parameters
198
00:09:50,000 --> 00:09:51,170
based on that formula.
199
00:09:51,170 --> 00:09:54,780
So here in formula five, you
see like different parameters
200
00:09:54,780 --> 00:09:55,620
we have.
201
00:09:55,620 --> 00:09:59,740
For example, M of s is the
average simulated vector
202
00:09:59,740 --> 00:10:00,580
of moments.
203
00:10:00,580 --> 00:10:03,390
So for example,
in our example, we
204
00:10:03,390 --> 00:10:07,710
have like 10 simulation for
each one of those moments,
205
00:10:07,710 --> 00:10:12,090
so we average the moments
over those simulation
206
00:10:12,090 --> 00:10:16,180
and create a vector,
which is M of s here.
207
00:10:16,180 --> 00:10:19,990
And M of d is the real
moments values in that vector
208
00:10:19,990 --> 00:10:22,720
that we capture from
the real data we have.
209
00:10:22,720 --> 00:10:26,180
W here is the initial
value for the weights.
210
00:10:26,180 --> 00:10:28,530
For the initial value,
we use the Formula One
211
00:10:28,530 --> 00:10:34,060
over moments, real
moments, to power of two.
212
00:10:34,060 --> 00:10:36,570
In the model, we
call the function--
213
00:10:36,570 --> 00:10:40,910
a function here
as the error term.
214
00:10:40,910 --> 00:10:45,070
So here in the view 2
of our FirstStep model,
215
00:10:45,070 --> 00:10:48,170
we can see that we have our
three unknown parameters here.
216
00:10:48,170 --> 00:10:51,610
We put them all together
in one vector here
217
00:10:51,610 --> 00:10:55,730
called parameter so that we
can deal with them easily.
218
00:10:55,730 --> 00:10:59,850
So from the obesity
model, we get the moments
219
00:10:59,850 --> 00:11:01,430
that we need here.
220
00:11:01,430 --> 00:11:04,450
So if you take a look at the
moments here-- the moments
221
00:11:04,450 --> 00:11:06,560
we have here-- in
the last year-- year
222
00:11:06,560 --> 00:11:10,680
20-- you'll see that
we have matrices that
223
00:11:10,680 --> 00:11:14,845
has 10 moments in it,
and for each moments
224
00:11:14,845 --> 00:11:16,310
we have like 10 simulations.
225
00:11:16,310 --> 00:11:21,600
So we have 10 in 10
matrix for each moments
226
00:11:21,600 --> 00:11:22,770
in each simulation.
227
00:11:22,770 --> 00:11:24,370
So we get those values.
228
00:11:24,370 --> 00:11:28,365
And here we can get an average
over those 10 simulations,
229
00:11:28,365 --> 00:11:34,830
and we create the M of s
that we talked about here.
230
00:11:34,830 --> 00:11:38,640
So using that and
using the real data
231
00:11:38,640 --> 00:11:42,760
that we have we calculate the
difference between the moment--
232
00:11:42,760 --> 00:11:45,060
the selected moments
and the real moments.
233
00:11:45,060 --> 00:11:49,170
And here, we see that
we calculate the W
234
00:11:49,170 --> 00:11:53,280
based on the formula we have
for the initial value, which
235
00:11:53,280 --> 00:11:59,740
is one over real moments
to the power of 2.
236
00:11:59,740 --> 00:12:05,030
And using these data, we
calculate the error terms here,
237
00:12:05,030 --> 00:12:10,830
which we just calculated
for the last time
238
00:12:10,830 --> 00:12:13,810
of the simulation,
which is the year 20.
239
00:12:13,810 --> 00:12:15,810
So as you can see
here, this formula
240
00:12:15,810 --> 00:12:21,325
here is a formula 5 from the
chapter books that we have.
241
00:12:21,325 --> 00:12:24,580
And finally, we have
to do some optimization
242
00:12:24,580 --> 00:12:26,330
on the model to get the value.
243
00:12:26,330 --> 00:12:30,570
So here, you'll see that we
want to optimize over the error
244
00:12:30,570 --> 00:12:32,800
terms, and we want
to minimize it.
245
00:12:32,800 --> 00:12:35,520
So we put error term
here on base minus one.
246
00:12:35,520 --> 00:12:39,200
You have to do to choose a
policy for this optimization.
247
00:12:39,200 --> 00:12:44,190
So we are what we want to
optimize over the parameters,
248
00:12:44,190 --> 00:12:46,130
so we have to choose
the parameters
249
00:12:46,130 --> 00:12:49,540
as the variable we
want to optimize.
250
00:12:49,540 --> 00:12:52,830
And then we do the
optimization and you
251
00:12:52,830 --> 00:12:57,410
will get the values we need.
252
00:12:57,410 --> 00:13:02,860
So we do optimization, and
it may take for awhile, maybe
253
00:13:02,860 --> 00:13:05,660
like 20 or 30 minutes.
254
00:13:05,660 --> 00:13:09,950
So we are fast
forwarding to the values
255
00:13:09,950 --> 00:13:14,580
we have at the end of that.
256
00:13:14,580 --> 00:13:17,100
So as you can see
here, we have optimized
257
00:13:17,100 --> 00:13:22,330
parameters-- optimized values
for our parameters, which
258
00:13:22,330 --> 00:13:25,140
are kind of close to the
real values that we have.
259
00:13:28,520 --> 00:13:31,410
So in the next model,
which is the CalcW model,
260
00:13:31,410 --> 00:13:33,960
we try to calculate
the weight we
261
00:13:33,960 --> 00:13:39,000
need for the MSM process
using other models.
262
00:13:39,000 --> 00:13:42,570
So here we import optimized,
unknown parameters
263
00:13:42,570 --> 00:13:45,530
from the previous model,
which is the FirstStep model,
264
00:13:45,530 --> 00:13:46,850
to CalcW model.
265
00:13:46,850 --> 00:13:50,650
Then we calculate
a variable called
266
00:13:50,650 --> 00:13:53,050
S using the formula
for in the book chapter
267
00:13:53,050 --> 00:13:55,390
that you can see it over here.
268
00:13:55,390 --> 00:13:57,700
And the inverse
of that S variable
269
00:13:57,700 --> 00:14:01,140
is the weight that we
need for our MSM process.
270
00:14:03,920 --> 00:14:06,290
In the first view of
CalcW, although everything
271
00:14:06,290 --> 00:14:09,930
is the same as this previous
model, in the second view,
272
00:14:09,930 --> 00:14:13,850
we try to calculate the
S based on the formula we
273
00:14:13,850 --> 00:14:16,100
have in the book chapter.
274
00:14:16,100 --> 00:14:19,810
So here we can see that
we try to calculate
275
00:14:19,810 --> 00:14:22,220
the average of
simulated moments.
276
00:14:22,220 --> 00:14:24,750
And this is exactly the
same as the previous model,
277
00:14:24,750 --> 00:14:26,980
and we try to
calculate the average
278
00:14:26,980 --> 00:14:29,400
of each moment over
10 simulations.
279
00:14:29,400 --> 00:14:33,550
And after that, you can
see that that average
280
00:14:33,550 --> 00:14:36,720
of simulated moments
here in the formula 4.
281
00:14:36,720 --> 00:14:41,440
And we have to subtract
the real value of moments
282
00:14:41,440 --> 00:14:45,740
from that average and then
multiply to its transpose
283
00:14:45,740 --> 00:14:48,790
and get another
average over that.
284
00:14:48,790 --> 00:14:53,560
So here in the moments
differences variable,
285
00:14:53,560 --> 00:14:56,820
we try to calculate
the formula we
286
00:14:56,820 --> 00:15:00,760
have in the book
chapter formula 4.
287
00:15:00,760 --> 00:15:03,610
And using that, we just
calculated the S hat
288
00:15:03,610 --> 00:15:07,260
for the last final
time of simulation.
289
00:15:11,200 --> 00:15:14,680
We only need to calculate the
value of S just for one time,
290
00:15:14,680 --> 00:15:18,880
so that by inversing the
value of the matrix S,
291
00:15:18,880 --> 00:15:24,010
we can get the value of matrix
W that we need the MSM process.
292
00:15:24,010 --> 00:15:28,630
So we just want the
models for one time.
293
00:15:28,630 --> 00:15:35,260
And here, after the
simulation ends,
294
00:15:35,260 --> 00:15:38,670
you can see the
value of S hat we
295
00:15:38,670 --> 00:15:41,020
need in the final time
of simulation here.
296
00:15:41,020 --> 00:15:48,560
So we have like 10 by 10
matrix that has like difference
297
00:15:48,560 --> 00:15:52,896
moments, and we have
to inverse this matrix
298
00:15:52,896 --> 00:15:57,070
to get the real weight
matrix that we need.
299
00:16:00,480 --> 00:16:02,890
In the next model, which
is the SecondStep model,
300
00:16:02,890 --> 00:16:07,190
we again want to find
all known parameters
301
00:16:07,190 --> 00:16:10,250
that we have using some
optimization this time
302
00:16:10,250 --> 00:16:14,210
using the calculated w from
the previous part in the CalcW
303
00:16:14,210 --> 00:16:14,920
part.
304
00:16:14,920 --> 00:16:19,990
So we imported S matrix
from the previous part.
305
00:16:19,990 --> 00:16:24,990
And by inversing that S matrix,
we get the W that we need.
306
00:16:24,990 --> 00:16:28,320
And also, we can import
optimized parameters
307
00:16:28,320 --> 00:16:32,840
from the FirstStep
model, or we can just
308
00:16:32,840 --> 00:16:35,220
use some random other
variables, values
309
00:16:35,220 --> 00:16:40,650
as the initial values
of the parameters.
310
00:16:40,650 --> 00:16:44,140
And so finally, we do
another optimization
311
00:16:44,140 --> 00:16:46,800
to find our three
unknown parameters.
312
00:16:46,800 --> 00:16:49,660
And we again use
the formula (5).
313
00:16:49,660 --> 00:16:52,320
Just this time, we
have the W calculated
314
00:16:52,320 --> 00:16:59,500
from the previous model,
which was the CalcW model.
315
00:16:59,500 --> 00:17:02,725
So here in the SecondStep
model, in view one,
316
00:17:02,725 --> 00:17:05,180
you can see everything's
the same as the two
317
00:17:05,180 --> 00:17:06,364
previous models.
318
00:17:06,364 --> 00:17:09,210
And in view two,
you can see the only
319
00:17:09,210 --> 00:17:12,230
[? serious ?] thing, that
difference from the FirstStep
320
00:17:12,230 --> 00:17:16,109
model, is that we
import the value of S
321
00:17:16,109 --> 00:17:19,609
and then inverse it to calculate
the W instead of calculating
322
00:17:19,609 --> 00:17:26,033
W based on the data
moments that we have.
323
00:17:26,033 --> 00:17:30,540
Here we have some kind
of [? bargain ?] Vensim.
324
00:17:30,540 --> 00:17:35,010
And if you try to import the
values of S hat from previous
325
00:17:35,010 --> 00:17:37,460
model using some
kind of function,
326
00:17:37,460 --> 00:17:41,410
like GET VDF CONSTANT or any
other function that tries
327
00:17:41,410 --> 00:17:44,150
to import the values
from the previous model,
328
00:17:44,150 --> 00:17:47,690
then my Vensim cannot calculate
the inverse of that metrics
329
00:17:47,690 --> 00:17:48,480
[INAUDIBLE].
330
00:17:48,480 --> 00:17:54,500
So here we just copy-pasted
the values of the calculated S
331
00:17:54,500 --> 00:17:58,160
matrix from previous
model and just put it here
332
00:17:58,160 --> 00:18:02,380
as the exogenous
variable, and here we
333
00:18:02,380 --> 00:18:05,820
calculate the
inverse of my matrix,
334
00:18:05,820 --> 00:18:09,510
instead of just using
the imported values
335
00:18:09,510 --> 00:18:12,380
from the previous model.
336
00:18:12,380 --> 00:18:13,860
For the SecondStep
model, again, we
337
00:18:13,860 --> 00:18:17,420
have to optimize the parameter.
338
00:18:17,420 --> 00:18:22,480
So again, we have like the same
system as the FirstStep model.
339
00:18:22,480 --> 00:18:25,000
We tried to minimize
the errors terms,
340
00:18:25,000 --> 00:18:28,010
and we are trying to
optimize the parameters.
341
00:18:28,010 --> 00:18:32,530
And after like the
simulation-- simulation may
342
00:18:32,530 --> 00:18:37,560
take like an hour or so-- so
after the simulation ends,
343
00:18:37,560 --> 00:18:45,732
we have optimized value
for our unknown parameters.
344
00:18:45,732 --> 00:18:47,190
After the optimization
is finished,
345
00:18:47,190 --> 00:18:51,600
we can see the optimized
parameters value here.
346
00:18:51,600 --> 00:18:55,870
So we get these values for
each one of these parameters.
347
00:18:55,870 --> 00:19:00,220
And we can see, for example,
0.002 is for [INAUDIBLE]
348
00:19:00,220 --> 00:19:04,600
random, 0.35 is for
starvation, and 0.13
349
00:19:04,600 --> 00:19:09,222
is for an overfeed
parameters here,
350
00:19:09,222 --> 00:19:11,974
The next model, which is the
Confidence Interval Model,
351
00:19:11,974 --> 00:19:13,390
we try to calculate
the confidence
352
00:19:13,390 --> 00:19:15,830
interval of our
unknown parameters.
353
00:19:15,830 --> 00:19:19,425
So here we need to import
the optimized parameters
354
00:19:19,425 --> 00:19:23,310
from the SecondStep model and
the W from the CalcW model.
355
00:19:23,310 --> 00:19:26,130
And then we calculate the
confidence intervals first
356
00:19:26,130 --> 00:19:28,410
by calculating a
variable called Q.
357
00:19:28,410 --> 00:19:32,860
We can see the formula for
calculating the Q here.
358
00:19:32,860 --> 00:19:36,740
And then using that
Q, we calculate
359
00:19:36,740 --> 00:19:40,240
the confidence interval using
this formula we can see here.
360
00:19:40,240 --> 00:19:45,720
We just have to add and increase
our values from each parameter.
361
00:19:45,720 --> 00:19:50,380
So here the confidence level
factor, for example for 95%,
362
00:19:50,380 --> 00:19:53,130
the confidence
interval-- assuming
363
00:19:53,130 --> 00:19:56,156
we have normal
distribution-- is 1.96.
364
00:19:56,156 --> 00:19:59,870
Then we have to multiply that
value, that confidence level
365
00:19:59,870 --> 00:20:03,380
factor to a square root of
q, and adding and subtracting
366
00:20:03,380 --> 00:20:04,230
from the parameters.
367
00:20:04,230 --> 00:20:06,769
So after that, we have
the confidence interval
368
00:20:06,769 --> 00:20:08,060
for each one of our parameters.
369
00:20:11,860 --> 00:20:14,130
So here in the Confidence
Interval Model,
370
00:20:14,130 --> 00:20:16,410
we can see that
again we calculate
371
00:20:16,410 --> 00:20:18,990
the average simulated
moment, and we
372
00:20:18,990 --> 00:20:21,180
get the S from
the previous part.
373
00:20:21,180 --> 00:20:25,470
Also, we have two import the
value of parameters meters
374
00:20:25,470 --> 00:20:29,150
from the previous model, which
was the SecondStep model.
375
00:20:29,150 --> 00:20:32,100
So we get the optimized
parameters here.
376
00:20:32,100 --> 00:20:34,940
We get the value of S here.
377
00:20:34,940 --> 00:20:40,770
And here we try to calculate
the confidence interval
378
00:20:40,770 --> 00:20:43,010
using the formula that we have.
379
00:20:43,010 --> 00:20:48,100
So here in the
formula 7, you can
380
00:20:48,100 --> 00:20:53,730
see that we have 1 plus 1/K. K
here is number of simulation.
381
00:20:53,730 --> 00:20:55,870
So for example, we
have 10 simulation.
382
00:20:55,870 --> 00:20:59,845
Then we have delta as
minus 1, which is the W.
383
00:20:59,845 --> 00:21:06,320
And again, delta--
delta here is the change
384
00:21:06,320 --> 00:21:08,890
in moments of a
parameter by changing
385
00:21:08,890 --> 00:21:14,600
the parameter for epsilon or
[? theta ?] [INAUDIBLE] value.
386
00:21:14,600 --> 00:21:19,620
So for example, if you add
0.001 to an optimized value
387
00:21:19,620 --> 00:21:25,140
of "Starvation", how much
it's average simulated mean
388
00:21:25,140 --> 00:21:28,910
may change or standard deviation
may change to two moments
389
00:21:28,910 --> 00:21:31,320
that we have.
390
00:21:31,320 --> 00:21:40,540
So we tried to calculate
that delta here using
391
00:21:40,540 --> 00:21:42,390
some [INAUDIBLE] variable.
392
00:21:42,390 --> 00:21:46,475
In [INAUDIBLE] variable,
we just add and decrease
393
00:21:46,475 --> 00:21:48,720
a value of epsilon
to each parameters,
394
00:21:48,720 --> 00:21:53,170
and then we get the delta here.
395
00:21:53,170 --> 00:21:55,090
Using this system
that you can see here,
396
00:21:55,090 --> 00:21:57,770
we calculate the confidence
interval we need.
397
00:22:00,670 --> 00:22:05,335
So again, we need to run
the model only for one time.
398
00:22:08,400 --> 00:22:10,920
Just by running the
model just for one time,
399
00:22:10,920 --> 00:22:13,370
we can calculate the confidence
interval that we need.
400
00:22:16,820 --> 00:22:18,620
So after running
the model, we can
401
00:22:18,620 --> 00:22:22,480
see this we have the
confidence intervals for,
402
00:22:22,480 --> 00:22:24,780
for example, upper
bound confidence
403
00:22:24,780 --> 00:22:27,820
interval for each one
of the parameters here,
404
00:22:27,820 --> 00:22:30,780
and the lower bound
of confidence interval
405
00:22:30,780 --> 00:22:33,035
for each one of the parameters
that we have in here.
406
00:22:38,300 --> 00:22:41,265
The base obesity
model and the moments,
407
00:22:41,265 --> 00:22:44,220
the capturing moments
structure that we
408
00:22:44,220 --> 00:22:48,450
have in all four models,
are kind of the same,
409
00:22:48,450 --> 00:22:50,970
except for the last model,
which is the confidence interval
410
00:22:50,970 --> 00:22:51,490
model.
411
00:22:51,490 --> 00:22:54,460
We have to add one
other set of scripts
412
00:22:54,460 --> 00:22:57,964
that is for
sensitivity analysis.
413
00:22:57,964 --> 00:22:59,380
For each one of
the parameters, we
414
00:22:59,380 --> 00:23:02,020
have one upper bound
and one lower bound,
415
00:23:02,020 --> 00:23:06,020
so we have to add six
subscripts to the model.
416
00:23:06,020 --> 00:23:07,470
And for each one
of the subscript,
417
00:23:07,470 --> 00:23:10,890
we have to run a
simulation to, for example,
418
00:23:10,890 --> 00:23:13,130
catch the upper bound
of one of the parameters
419
00:23:13,130 --> 00:23:15,070
or catch the lower
bound of the parameters.
420
00:23:15,070 --> 00:23:17,250
So you have to change
the model in a way
421
00:23:17,250 --> 00:23:22,530
that it runs the simulation for
each one of those subscription.
422
00:23:22,530 --> 00:23:30,020
But everything else is the
same in our four models
423
00:23:30,020 --> 00:23:32,850
for the base of
the obesity model
424
00:23:32,850 --> 00:23:35,685
and the moment capturing
model for a structure.
425
00:23:39,690 --> 00:23:42,750
Finally, if we put aside
the [INAUDIBLE] in Vensim
426
00:23:42,750 --> 00:23:45,750
and assume that we can
just import and export
427
00:23:45,750 --> 00:23:49,440
[INAUDIBLE] between models and
calculate the inverse of matrix
428
00:23:49,440 --> 00:23:54,590
without any problem, we can
create just one common script
429
00:23:54,590 --> 00:23:59,000
that runs our sequential models
and just import and export
430
00:23:59,000 --> 00:23:59,940
the data between them.
431
00:23:59,940 --> 00:24:05,310
And after running all
the models and processes,
432
00:24:05,310 --> 00:24:08,600
and it just gives us
the final solution
433
00:24:08,600 --> 00:24:13,200
of the optimized parameters
and their confidence intervals.
434
00:24:13,200 --> 00:24:17,110
So here for example, we
created a common script
435
00:24:17,110 --> 00:24:23,870
that tells Vensim to
optimize the first model,
436
00:24:23,870 --> 00:24:27,870
then import the value
from the first model
437
00:24:27,870 --> 00:24:32,280
and calculate the W while
you're running the second model.
438
00:24:32,280 --> 00:24:36,850
And after that, just import
the W from the CalcW model
439
00:24:36,850 --> 00:24:39,320
and just do another
optimization to get
440
00:24:39,320 --> 00:24:43,740
the second to do another
optimization on SecondStep
441
00:24:43,740 --> 00:24:45,310
model.
442
00:24:45,310 --> 00:24:49,780
Finally, export the values
to the confidence values
443
00:24:49,780 --> 00:24:52,570
of unknown parameters to
the confidence interval,
444
00:24:52,570 --> 00:24:55,760
and just calculate the
confidence interval.
445
00:24:55,760 --> 00:25:00,790
So using such as system, using
such a common script system,
446
00:25:00,790 --> 00:25:07,395
we can just do one-- just
gives Vensim one common,
447
00:25:07,395 --> 00:25:13,570
and it will just do the whole
process of our series models
448
00:25:13,570 --> 00:25:16,570
one after each other, and
do the whole thing together.
449
00:25:16,570 --> 00:25:21,780
And just gives us the
final parameter's value
450
00:25:21,780 --> 00:25:24,390
and their confidence interval.
451
00:25:24,390 --> 00:25:26,370
So that was it.
452
00:25:26,370 --> 00:25:28,091
Thank you so much for watching.