1
00:00:00 --> 00:00:00



2
00:00:00 --> 00:00:02
ANNOUNCER: Open content is
provided under a creative

3
00:00:02 --> 00:00:03
commons license.


4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:10
offer high-quality educational
resources for free.

6
00:00:10 --> 00:00:13
To make a donation, or view
additional materials from

7
00:00:13 --> 00:00:16
hundreds of MIT courses,
visit MIT OpenCourseWare

8
00:00:16 --> 00:00:17
at ocw.mit.edu.


9
00:00:17 --> 00:00:24
PROFESSOR JOHN GUTTAG:
Good morning.

10
00:00:24 --> 00:00:26
We should start with the
confession, for those of

11
00:00:26 --> 00:00:29
you looking at this on
OpenCourseWare, that I'm

12
00:00:29 --> 00:00:34
currently lecturing to
an empty auditorium.

13
00:00:34 --> 00:00:38
The fifth lecture for 600
this term, we ran into some

14
00:00:38 --> 00:00:43
technical difficulties, which
left us with a recording we

15
00:00:43 --> 00:00:45
weren't very satisfied with.


16
00:00:45 --> 00:00:51
So, I'm-- this is a redo, and
if you will hear no questions

17
00:00:51 --> 00:00:54
from the audience and that's
because there is no audience.

18
00:00:54 --> 00:00:57
Nevertheless I will do
my best to pretend.

19
00:00:57 --> 00:01:01
I've been told this is a little
bit like giving a speech before

20
00:01:01 --> 00:01:07
the US Congress when C-SPAN
is the only thing watching.

21
00:01:07 --> 00:01:09
OK.


22
00:01:09 --> 00:01:14
Computers are supposed to be
good for crunching numbers.

23
00:01:14 --> 00:01:18
And we've looked a little bit
at numbers this term, but I now

24
00:01:18 --> 00:01:21
want to get into looking at
them in more depth than

25
00:01:21 --> 00:01:23
we've been doing.


26
00:01:23 --> 00:01:32
Python has two different
kinds of numbers.

27
00:01:32 --> 00:01:35
So far, the only kind we've
really paid any attention

28
00:01:35 --> 00:01:40
to is type int.


29
00:01:40 --> 00:01:44
And those were intended to
mirror the integers, as we

30
00:01:44 --> 00:01:48
all learned about starting
in elementary school.

31
00:01:48 --> 00:01:52
And they're good for things
that you can count.

32
00:01:52 --> 00:01:55
Any place you'd use
whole numbers.

33
00:01:55 --> 00:02:00
Interestingly, Python, unlike
some languages, has what

34
00:02:00 --> 00:02:09
are called arbitrary
precision integers.

35
00:02:09 --> 00:02:12
By that, we mean, you
can make numbers as big

36
00:02:12 --> 00:02:14
as you want them to.


37
00:02:14 --> 00:02:17
Let's look at an example.


38
00:02:17 --> 00:02:20
We'll just take a for a
variable name, and we'll set

39
00:02:20 --> 00:02:27
a to be two raised to the
one-thousandth power.

40
00:02:27 --> 00:02:31
That, by the way, is
a really big number.

41
00:02:31 --> 00:02:34
And now what happens if
we try and display it?

42
00:02:34 --> 00:02:37
We get a lot of digits.


43
00:02:37 --> 00:02:39
You can see why I'm doing this
on the screen instead of

44
00:02:39 --> 00:02:43
writing it on the blackboard.


45
00:02:43 --> 00:02:45
I'm not going to ask you
whether you believe this is

46
00:02:45 --> 00:02:48
the right answer, trust
me, trust Python.

47
00:02:48 --> 00:02:51
I would like you to notice,
at the very end of

48
00:02:51 --> 00:02:55
this is the letter L.


49
00:02:55 --> 00:02:58
What does that mean?


50
00:02:58 --> 00:03:05
It means long.


51
00:03:05 --> 00:03:10
That's telling us that it's
representing these-- this

52
00:03:10 --> 00:03:13
particular integer in
what it calls it's

53
00:03:13 --> 00:03:18
internal long format.


54
00:03:18 --> 00:03:20
You needn't worry about that.


55
00:03:20 --> 00:03:22
The only thing to say about it
is, when you're dealing with

56
00:03:22 --> 00:03:26
long integers, it's a lot less
efficient than when you're

57
00:03:26 --> 00:03:28
dealing with smaller numbers.


58
00:03:28 --> 00:03:30
And that's all it's kind
of warning you, by

59
00:03:30 --> 00:03:35
printing this L.


60
00:03:35 --> 00:03:38
About two billion is
the magic number.

61
00:03:38 --> 00:03:41
When you get over two billion,
it's now going to deal with

62
00:03:41 --> 00:03:44
long integers, so if, for
example, you're trying to deal

63
00:03:44 --> 00:03:50
with the US budget deficit, you
will need integers of type L.

64
00:03:50 --> 00:03:52
OK.


65
00:03:52 --> 00:03:54
Let's look at another
interesting example.

66
00:03:54 --> 00:04:00
Suppose I said, b equal to two
raised to the nine hundred

67
00:04:00 --> 00:04:04
ninety-ninth power.


68
00:04:04 --> 00:04:10
I can display b, and it's a
different number, considerably

69
00:04:10 --> 00:04:14
smaller, but again,
ending in an L.

70
00:04:14 --> 00:04:21
And now, what you think I'll
get if we try a divided by b?

71
00:04:21 --> 00:04:25
And remember, we're now
doing integer division.

72
00:04:25 --> 00:04:27
Well, let's see.


73
00:04:27 --> 00:04:31
We get 2L.


74
00:04:31 --> 00:04:34
Well, you'd expect it to be
two, because if you think about

75
00:04:34 --> 00:04:38
the meaning of exponentiation,
indeed, the difference between

76
00:04:38 --> 00:04:42
raising something to the nine
hundred ninety-ninth power and

77
00:04:42 --> 00:04:45
to the one-thousandth power
should be, in this case, two,

78
00:04:45 --> 00:04:49
since that's what we're
raising to a power.

79
00:04:49 --> 00:04:52
Why does it say 2L, right?


80
00:04:52 --> 00:04:56
Two is considerably less than
two billion, and that's because

81
00:04:56 --> 00:05:00
once you get L, you stay L.


82
00:05:00 --> 00:05:04
Not particularly important,
but kind of worth knowing.

83
00:05:04 --> 00:05:08
Well, why am I bothering you
with this whole issue of how

84
00:05:08 --> 00:05:12
numbers are represented
in the computer?

85
00:05:12 --> 00:05:15
In an ideal world, you would
ignore this completely, and

86
00:05:15 --> 00:05:20
just say, numbers do what
numbers are supposed to do.

87
00:05:20 --> 00:05:23
But as we're about to see,
sometimes in Python, and in

88
00:05:23 --> 00:05:28
fact in every programming
language, things behave

89
00:05:28 --> 00:05:32
contrary to what your
intuition suggests.

90
00:05:32 --> 00:05:35
And I want to spend a little
time helping you understand

91
00:05:35 --> 00:05:38
why this happens.


92
00:05:38 --> 00:05:41
So let's look at a
different kind of number.

93
00:05:41 --> 00:05:48
And now we're going to look at
what Python, and almost every

94
00:05:48 --> 00:05:53
other programming language,
calls type float.

95
00:05:53 --> 00:06:03
Which is short for
floating point.

96
00:06:03 --> 00:06:07
And that's the way that
programming languages typically

97
00:06:07 --> 00:06:12
represent what we think
of as real numbers.

98
00:06:12 --> 00:06:15
So, let's look at an example.


99
00:06:15 --> 00:06:24
I'm going to set the variable
x to be 0.1, 1/10, and now

100
00:06:24 --> 00:06:28
we're going to display x.


101
00:06:28 --> 00:06:29
Huh?


102
00:06:29 --> 00:06:31
Take a look at this.


103
00:06:31 --> 00:06:34
Why isn't it .1?


104
00:06:34 --> 00:06:39
Why is it 0.1, a whole bunch of
zeros, and then this mysterious

105
00:06:39 --> 00:06:43
one appearing at the end?


106
00:06:43 --> 00:06:45
Is it because Python just
wants to be obnoxious

107
00:06:45 --> 00:06:47
and is making life hard?


108
00:06:47 --> 00:06:52
No, it has to do with the way
the numbers are represented

109
00:06:52 --> 00:06:54
inside the computer.


110
00:06:54 --> 00:06:59
Python, like almost every
modern programming language,

111
00:06:59 --> 00:07:05
represents numbers using the
i triple e floating point

112
00:07:05 --> 00:07:20
standard, and it's
i triple e 754.

113
00:07:20 --> 00:07:23
Never again will you have
to remember that it's 754.

114
00:07:23 --> 00:07:27
I promise not to ask you
that question on a quiz.

115
00:07:27 --> 00:07:29
But that's what they do.


116
00:07:29 --> 00:07:41
This is a variant of
scientific notation.

117
00:07:41 --> 00:07:46
Something you probably learned
about in high school, as a way

118
00:07:46 --> 00:07:49
to represent very
large numbers.

119
00:07:49 --> 00:07:54
Typically, the way we do that,
is we represent the numbers in

120
00:07:54 --> 00:08:07
the form of a mantissa
and an exponent.

121
00:08:07 --> 00:08:12
So we represent a floating
point number as a pair, of a

122
00:08:12 --> 00:08:20
mantissa and an exponent.


123
00:08:20 --> 00:08:26
And because computers work in
the binary system, it's unlike

124
00:08:26 --> 00:08:28
what you probably learned
in high school, where we

125
00:08:28 --> 00:08:31
raise ten to some power.


126
00:08:31 --> 00:08:35
Here we'll always be
raising two to some power.

127
00:08:35 --> 00:08:38
Maybe a little later in the
term, if we talk about computer

128
00:08:38 --> 00:08:41
architecture, we'll get around
to explaining why computers

129
00:08:41 --> 00:08:46
working binary, but for now,
just assume that they do and

130
00:08:46 --> 00:08:48
in fact they always have.


131
00:08:48 --> 00:08:50
All right.


132
00:08:50 --> 00:09:01
Purists manage to refer to the
mantissa as a significant, but

133
00:09:01 --> 00:09:05
I won't do that, because I'm an
old guy and it was a mantissa

134
00:09:05 --> 00:09:08
when I first learned about it
and I just can't break

135
00:09:08 --> 00:09:11
myself of the habit.


136
00:09:11 --> 00:09:11
All right.


137
00:09:11 --> 00:09:18
So how does this work?


138
00:09:18 --> 00:09:21
Well, when we recognize
so-- when we represent

139
00:09:21 --> 00:09:35
something, the mantissa
is between one and two.

140
00:09:35 --> 00:09:37
Whoops.


141
00:09:37 --> 00:09:42
Strictly less than two,
greater than or equal to one.

142
00:09:42 --> 00:10:07
The exponent, is in the
range, -1022 to +1023.

143
00:10:07 --> 00:10:13
So this lets us represent
numbers up to about 10 to the

144
00:10:13 --> 00:10:19
308th, plus or minus 10 to
the 308th, plus or minus.

145
00:10:19 --> 00:10:24
So, quite a large
range of numbers.

146
00:10:24 --> 00:10:26
Where did these magic
things come from?

147
00:10:26 --> 00:10:29
You know, what-- kind of a
strange numbers to see here.

148
00:10:29 --> 00:10:37
Well, it has to do with the
fact that computers typically

149
00:10:37 --> 00:10:41
have words in them, and the
words today in a modern

150
00:10:41 --> 00:10:46
computer are 64 bits.


151
00:10:46 --> 00:10:50
For many years they were 32
bits, before that they were 16

152
00:10:50 --> 00:10:54
bits, before that they were 8
bits, they've continually

153
00:10:54 --> 00:10:58
grown, but we've been at 64 for
a while and I think we'll

154
00:10:58 --> 00:11:02
be stuck at 64 for a while.


155
00:11:02 --> 00:11:07
So as we do this, what we do
is, we get one bit for the

156
00:11:07 --> 00:11:14
sign-- is it a positive or
negative number?-- 11 for the

157
00:11:14 --> 00:11:28
exponent, and that leaves
52 for the mantissa.

158
00:11:28 --> 00:11:32
And that basically tells us
how we're storing numbers.

159
00:11:32 --> 00:11:34
Hi, are you here for
the 600 lecture?

160
00:11:34 --> 00:11:40
There is none today, because
we have a quiz this evening.

161
00:11:40 --> 00:11:43
It's now the time that the
lecture would normally have

162
00:11:43 --> 00:11:46
started, and a couple of
students who forgot that we

163
00:11:46 --> 00:11:48
have a quiz this evening,
instead of a lecture,

164
00:11:48 --> 00:11:53
just strolled in, and
now strolled out.

165
00:11:53 --> 00:11:55
OK.


166
00:11:55 --> 00:12:00
You may never need to know
these constants again, but it's

167
00:12:00 --> 00:12:04
worth knowing that they exist,
and that basically, this gives

168
00:12:04 --> 00:12:08
us about the equivalent
of seventeen decimal

169
00:12:08 --> 00:12:13
digits of precision.


170
00:12:13 --> 00:12:16
So we can represent
numbers up to seventeen

171
00:12:16 --> 00:12:21
decimal digits long.


172
00:12:21 --> 00:12:26
This is an important concept to
understand, that unlike the

173
00:12:26 --> 00:12:30
long ints where they can grow
arbitrarily big, when we're

174
00:12:30 --> 00:12:33
dealing with floating points,
if we need something more than

175
00:12:33 --> 00:12:37
seventeen decimal digits, in
Python at least, we won't

176
00:12:37 --> 00:12:39
be able to get it.


177
00:12:39 --> 00:12:41
And that's true in
many languages.

178
00:12:41 --> 00:12:45
Now the good news is, this is
an enormous number, and it's

179
00:12:45 --> 00:12:48
highly unlikely that ever in
your life, you will need

180
00:12:48 --> 00:12:51
more precision than that.


181
00:12:51 --> 00:12:53
All right.


182
00:12:53 --> 00:13:00
Now, let's go back to the 0.1
mystery that we started at, and

183
00:13:00 --> 00:13:06
ask ourselves, why we have a
problem representing that

184
00:13:06 --> 00:13:11
number in the computer, hence,
we get something funny out from

185
00:13:11 --> 00:13:14
we try and print it back.


186
00:13:14 --> 00:13:18
Well, let's look at an
easier problem first.

187
00:13:18 --> 00:13:26
Let's look at representing
the fraction 1/8.

188
00:13:26 --> 00:13:29
That has a nice representation.


189
00:13:29 --> 00:13:36
That's equal in decimal to
0.125, and we can represent

190
00:13:36 --> 00:13:42
it conveniently in both
base 10 and base 2.

191
00:13:42 --> 00:13:53
So if you want to represent
it in base 10, what is it?

192
00:13:53 --> 00:13:56
What is that equal to?


193
00:13:56 --> 00:14:02
Well, we'll take a mantissa,
1.25, and now we need to

194
00:14:02 --> 00:14:09
multiply it by something that
we can represent nicely, and

195
00:14:09 --> 00:14:14
in fact that will be
times 10 to the -1.

196
00:14:14 --> 00:14:17
So the exponent would simply
be -1, and we have a

197
00:14:17 --> 00:14:20
nice representation.


198
00:14:20 --> 00:14:24
Suppose we want to
represent it in base 2?

199
00:14:24 --> 00:14:29
What would it be?


200
00:14:29 --> 00:14:44
1.0 times-- anybody?--
Well, 2 to the -3.

201
00:14:44 --> 00:14:53
So, in binary notation, that
would be written as 0.001.

202
00:14:53 --> 00:14:56
So you see, 1/8 is kind
of a nice number.

203
00:14:56 --> 00:15:01
We can represent it nicely in
either base 10 or base 2.

204
00:15:01 --> 00:15:14
But how about that
pesky fraction 1/10?

205
00:15:14 --> 00:15:23
Well, in base 10, we know how
to represent, it's 1 times

206
00:15:23 --> 00:15:30
10 to the-- 10 to the
what?-- 10 to the 1?

207
00:15:30 --> 00:15:32
No.


208
00:15:32 --> 00:15:41
But in base 2, it's a problem.


209
00:15:41 --> 00:15:47
There is no finite binary
number that exactly represents

210
00:15:47 --> 00:15:51
this decimal fraction.


211
00:15:51 --> 00:15:56
In fact, if we try and find the
binary number, what we find is,

212
00:15:56 --> 00:15:59
we get an infinitely
repeating series.

213
00:15:59 --> 00:16:06
Zero zero zero one one
zero zero one one zero

214
00:16:06 --> 00:16:09
zero, and et cetera.


215
00:16:09 --> 00:16:14
Stop at any finite number
of bits, and you get only

216
00:16:14 --> 00:16:20
an approximation to the
decimal fraction 1/10.

217
00:16:20 --> 00:16:28
So on most computers, if you
were to print the decimal value

218
00:16:28 --> 00:16:31
of the binary approximation--
and that's what we're printing

219
00:16:31 --> 00:16:33
here, on this screen, right?


220
00:16:33 --> 00:16:38
We think in decimal, so Python
quite nicely for us is printing

221
00:16:38 --> 00:16:43
things in decimal-- it would
have to display-- well I'm not

222
00:16:43 --> 00:16:49
going to write it, it's a very
long number, lots of digits--

223
00:16:49 --> 00:16:57
however, in Python, whenever we
display something, it uses the

224
00:16:57 --> 00:17:03
built-in function repr, short
for representation, that it

225
00:17:03 --> 00:17:07
converts the internal
representation in this case of

226
00:17:07 --> 00:17:12
a number, to a string, and then
displays that string in

227
00:17:12 --> 00:17:15
this case on the screen.


228
00:17:15 --> 00:17:24
For floats, it rounds it
to seventeen digits.

229
00:17:24 --> 00:17:31
There's that magic
number seventeen again.

230
00:17:31 --> 00:17:36
Hence, when it rounds it to
seventeen digits, we get

231
00:17:36 --> 00:17:44
exactly what you see in the
bottom of the screen up there.

232
00:17:44 --> 00:17:49
Answer to the mystery, why
does it display this?

233
00:17:49 --> 00:17:52
Now why should we care?


234
00:17:52 --> 00:17:56
Well, it's not so much that we
care about what gets displayed,

235
00:17:56 --> 00:17:59
but we have to think about
the implications, at least

236
00:17:59 --> 00:18:02
sometimes we have to think
about the implications, of what

237
00:18:02 --> 00:18:07
this inexact representation of
numbers means when we start

238
00:18:07 --> 00:18:13
doing more-or-less complex
computations on those numbers.

239
00:18:13 --> 00:18:17
So let's look at a
little example here.

240
00:18:17 --> 00:18:21
I'll start by starting
the variable s to 0.0 .

241
00:18:21 --> 00:18:24
Notice I'm being careful
to make it a float.

242
00:18:24 --> 00:18:33
And then for i in range, let's
see, let's take 10, we'll

243
00:18:33 --> 00:18:44
increase s by 0.1 .


244
00:18:44 --> 00:18:47
All right, we've done that,
and now, what happens

245
00:18:47 --> 00:18:52
when I print s?


246
00:18:52 --> 00:18:56
Well, again you don't
get what your intuition

247
00:18:56 --> 00:18:58
says you should get.


248
00:18:58 --> 00:19:05
Notice the last two digits,
which are eight and nine.

249
00:19:05 --> 00:19:09
Well, what's happening here?


250
00:19:09 --> 00:19:13
What's happened, is the
error has accumulated.

251
00:19:13 --> 00:19:17
I had a small error when I
started, but every time I added

252
00:19:17 --> 00:19:22
it, the error got bigger
and it accumulates.

253
00:19:22 --> 00:19:27
Sometimes you can get in
trouble in a computation

254
00:19:27 --> 00:19:30
because of that.


255
00:19:30 --> 00:19:32
Now what happens, by
the way, if I print s?

256
00:19:32 --> 00:19:36
That's kind of an
interesting question.

257
00:19:36 --> 00:19:40
Notice that it prints one.


258
00:19:40 --> 00:19:42
And why is that?


259
00:19:42 --> 00:19:46
It's because the print command
has done a rounding here.

260
00:19:46 --> 00:19:50
It automatically rounds.


261
00:19:50 --> 00:19:54
And that's kind of good, but
it's also kind of bad, because

262
00:19:54 --> 00:19:56
that means when you're
debugging your program, you

263
00:19:56 --> 00:19:59
can get very confused.


264
00:19:59 --> 00:20:02
You say, it says it's one, why
am I getting a different answer

265
00:20:02 --> 00:20:04
when I do the computation?


266
00:20:04 --> 00:20:08
And that's because it's
not really one inside.

267
00:20:08 --> 00:20:10
So you have to be careful.


268
00:20:10 --> 00:20:16
Now mostly, these round-off
errors balance each other out.

269
00:20:16 --> 00:20:19
Some floats are slightly higher
than they're supposed to be,

270
00:20:19 --> 00:20:23
some are slightly lower, and in
most computations it all comes

271
00:20:23 --> 00:20:25
out in the wash and you
get the right answer.

272
00:20:25 --> 00:20:30
Truth be told, most of the
time, you can avoid worrying

273
00:20:30 --> 00:20:32
about these things.


274
00:20:32 --> 00:20:37
But, as we say in Latin,
caveat computor.

275
00:20:37 --> 00:20:40
Sometimes you have to
worry a little bit.

276
00:20:40 --> 00:20:44
Now there is one thing about
floating points about which

277
00:20:44 --> 00:20:49
you should always worry.


278
00:20:49 --> 00:20:54
And that's really the point I
want to drive home, and that's

279
00:20:54 --> 00:21:08
about the meaning
of double equal.

280
00:21:08 --> 00:21:12
Let's look at an
example of this.

281
00:21:12 --> 00:21:17
So we've before seen the use of
import, so I'm going to import

282
00:21:17 --> 00:21:21
math, it gives me some useful
mathematical functions, then

283
00:21:21 --> 00:21:29
I'm going to set the variable
a to the square root of two.

284
00:21:29 --> 00:21:31
Whoops.


285
00:21:31 --> 00:21:33
Why didn't this work?


286
00:21:33 --> 00:21:42
Because what I should have said
is math dot square root of two.

287
00:21:42 --> 00:21:46
Explaining to the interpreter
that I want to get the function

288
00:21:46 --> 00:21:52
sqrt from the module math.


289
00:21:52 --> 00:21:59
So now I've got a here, and I
can look at what a is, yeah,

290
00:21:59 --> 00:22:02
some approximation to the
square root about of two.

291
00:22:02 --> 00:22:04
Now here's the
interesting question.

292
00:22:04 --> 00:22:13
Suppose I ask about the Boolean
a times a equals equals two.

293
00:22:13 --> 00:22:16
Now in my heart, I think, if
I've taken the square root of

294
00:22:16 --> 00:22:20
number and then I've multiplied
it by itself, I could get

295
00:22:20 --> 00:22:22
the original number back.


296
00:22:22 --> 00:22:25
After all, that's the
meaning of square root.

297
00:22:25 --> 00:22:28
But by now, you won't be
surprised if the answer of this

298
00:22:28 --> 00:22:33
is false, because we know what
we've stored is only an

299
00:22:33 --> 00:22:37
approximation to
the square root.

300
00:22:37 --> 00:22:39
And that's kind of interesting.


301
00:22:39 --> 00:22:44
So we can see that, by, if I
look at a times a, I'll get two

302
00:22:44 --> 00:22:48
point a whole bunch of zeros
and then a four at the end.

303
00:22:48 --> 00:22:53
So this means, if I've got a
test in my program, in some

304
00:22:53 --> 00:22:58
sense it will give me the
unexpected answer false.

305
00:22:58 --> 00:23:04
What this tells us, is that
it's very risky to ever use the

306
00:23:04 --> 00:23:08
built-in double--equals to
compare floating points, and in

307
00:23:08 --> 00:23:13
fact, you should never be
testing for equality, you

308
00:23:13 --> 00:23:18
should always be testing
for close enough.

309
00:23:18 --> 00:23:22
So typically, what you want to
do in your program, is ask the

310
00:23:22 --> 00:23:30
following question: is the
absolute value of a times a

311
00:23:30 --> 00:23:40
minus 2.0 less than epsilon?


312
00:23:40 --> 00:23:42
If we could easily type
Greek, we'd have written

313
00:23:42 --> 00:23:46
it that way, but we can't.


314
00:23:46 --> 00:23:50
So that's some small value
chosen to be appropriate

315
00:23:50 --> 00:23:52
for the application.


316
00:23:52 --> 00:23:55
Saying, if these two things
are within epsilon of each

317
00:23:55 --> 00:24:00
other, then I'm going
to treat them as equal.

318
00:24:00 --> 00:24:04
And so what I typically do when
I'm writing a Python code

319
00:24:04 --> 00:24:06
that's going to deal with
floating point numbers, and I

320
00:24:06 --> 00:24:12
do this from time to time, is I
introduce a function called

321
00:24:12 --> 00:24:17
almost equal, or near, or
pick your favorite word,

322
00:24:17 --> 00:24:20
that does this for me.


323
00:24:20 --> 00:24:24
And wherever I would normally
written double x equals y,

324
00:24:24 --> 00:24:31
instead I write, near x,y,
and it computes it for me.

325
00:24:31 --> 00:24:36
Not a big deal, but keep this
in mind, or as soon as you

326
00:24:36 --> 00:24:39
start dealing with numbers,
you will get very frustrated

327
00:24:39 --> 00:24:43
in trying to understand
what your program does.

328
00:24:43 --> 00:24:45
OK.


329
00:24:45 --> 00:24:48
Enough of numbers for a
while, I'm sure some of you

330
00:24:48 --> 00:24:52
will find this a relief.


331
00:24:52 --> 00:24:57
I now want to get away from
details of floating point, and

332
00:24:57 --> 00:25:01
talk about general methods
again, returning to the real

333
00:25:01 --> 00:25:06
theme of the course of solving
problems using computers.

334
00:25:06 --> 00:25:11
Last week, we looked at the
rather silly problem of

335
00:25:11 --> 00:25:15
finding the square root
of a perfect square.

336
00:25:15 --> 00:25:21
Well, that's not
usually what you need.

337
00:25:21 --> 00:25:25
Let's think about the more
useful problem of finding the

338
00:25:25 --> 00:25:27
square root of a real number.


339
00:25:27 --> 00:25:29
Well, you've just seen
how you do that.

340
00:25:29 --> 00:25:32
You import math and
you call sqrt.

341
00:25:32 --> 00:25:36
Let's pretend that we didn't
know that trick, or let's

342
00:25:36 --> 00:25:39
pretend it's your job to
introduce-- implement,

343
00:25:39 --> 00:25:41
rather-- math.


344
00:25:41 --> 00:25:46
And so, you need to figure out
how to implement square root.

345
00:25:46 --> 00:25:49
Why might this be a challenge?


346
00:25:49 --> 00:25:52
What are some of the issues?


347
00:25:52 --> 00:25:53
And there are several.


348
00:25:53 --> 00:26:09
One is, what we've just seen
might not be an exact answer.

349
00:26:09 --> 00:26:16
For example, the
square root of two.

350
00:26:16 --> 00:26:20
So we need to worry about that,
and clearly the way we're going

351
00:26:20 --> 00:26:24
to solve that, as we'll
see, is using a concept

352
00:26:24 --> 00:26:26
similar to epsilon.


353
00:26:26 --> 00:26:33
In fact, we'll even
call it epsilon.

354
00:26:33 --> 00:26:35
Another problem with the
method we looked at last

355
00:26:35 --> 00:26:40
time is, there we were doing
exhaustive enumeration.

356
00:26:40 --> 00:26:44
We were enumerating all the
possible answers, checking

357
00:26:44 --> 00:26:47
each one, and if it
was good, stopping.

358
00:26:47 --> 00:26:55
Well, the problem with reals,
as opposed to integers, is we

359
00:26:55 --> 00:27:06
can't enumerate all guesses.


360
00:27:06 --> 00:27:14
And that's because the
reals are uncountable.

361
00:27:14 --> 00:27:19
If I ask you to enumerate the
positive integers, you'll say

362
00:27:19 --> 00:27:22
one, two, three, four, five.


363
00:27:22 --> 00:27:28
If I ask you to enumerate the
reals, the positive reals,

364
00:27:28 --> 00:27:29
where do you start?


365
00:27:29 --> 00:27:32
One over a billion,
plus who knows?

366
00:27:32 --> 00:27:37
Now as we've just seen in fact,
since there's a limit to the

367
00:27:37 --> 00:27:42
precision floating point,
technically you can enumerate

368
00:27:42 --> 00:27:44
all the floating point numbers.


369
00:27:44 --> 00:27:48
And I say technically, because
if you tried to do that,

370
00:27:48 --> 00:27:53
your computation would not
terminate any time soon.

371
00:27:53 --> 00:27:57
So even though in some, in
principle you could enumerate

372
00:27:57 --> 00:27:59
them, in fact you really can't.


373
00:27:59 --> 00:28:02
And so we think of the floating
points, like the reals,

374
00:28:02 --> 00:28:04
as being innumerable.


375
00:28:04 --> 00:28:10
Or not innumerable, as to
say as being uncountable.

376
00:28:10 --> 00:28:14
So we can't do that.


377
00:28:14 --> 00:28:19
So we have to find something
clever, because we're now

378
00:28:19 --> 00:28:24
searching a very large
space of possible answers.

379
00:28:24 --> 00:28:29
What would, technically you
might call a large state space.

380
00:28:29 --> 00:28:33
So we're going to take our
previous method of guess and

381
00:28:33 --> 00:28:39
check, and replace it by
something called guess,

382
00:28:39 --> 00:28:45
check, and improve.


383
00:28:45 --> 00:28:51
Previously, we just generated
guesses in some systematic way,

384
00:28:51 --> 00:28:53
but without knowing that we
were getting closer

385
00:28:53 --> 00:28:55
to the answer.


386
00:28:55 --> 00:28:58
Think of the original barnyard
problem with the chickens and

387
00:28:58 --> 00:29:02
the heads and the legs, we just
enumerated possibilities, but

388
00:29:02 --> 00:29:04
we didn't know that one guess
was better than the

389
00:29:04 --> 00:29:07
previous guess.


390
00:29:07 --> 00:29:12
Now, we're going to find a way
to do the enumeration where we

391
00:29:12 --> 00:29:19
have good reason to believe, at
least with high probability,

392
00:29:19 --> 00:29:25
that each guess is better
than the previous guess.

393
00:29:25 --> 00:29:41
This is what's called
successive approximation.

394
00:29:41 --> 00:29:45
And that's a very
important concept.

395
00:29:45 --> 00:29:50
Many problems are solved
computationally using

396
00:29:50 --> 00:29:56
successive approximation.


397
00:29:56 --> 00:30:01
Every successive approximation
method has the same

398
00:30:01 --> 00:30:03
rough structure.


399
00:30:03 --> 00:30:09
You start with some guess,
which would be the initial

400
00:30:09 --> 00:30:20
guess, you then iterate-- and
in a minute I'll tell you why

401
00:30:20 --> 00:30:29
I'm doing it this particular
way, over some range.

402
00:30:29 --> 00:30:33
I've chosen one hundred, but
doesn't have to be one hundred,

403
00:30:33 --> 00:30:45
just some number there-- if f
of x, that is to say some

404
00:30:45 --> 00:30:47
some function of my--


405
00:30:47 --> 00:30:50
Whoops, I shouldn't
have said x.

406
00:30:50 --> 00:30:58
My notes say x, but it's the
wrong thing-- if f of x, f of

407
00:30:58 --> 00:31:05
the guess, is close enough, so
for example, if when I square

408
00:31:05 --> 00:31:09
guess, I get close enough to
the number who's root I'm--

409
00:31:09 --> 00:31:20
square root I'm looking for,
then I'll return the guess.

410
00:31:20 --> 00:31:39
If it's not close enough,
I'll get a better guess.

411
00:31:39 --> 00:31:45
If I do my, in this case, one
hundred iterations, and I've

412
00:31:45 --> 00:31:48
not get-- gotten a guess
that's good enough, I'm going

413
00:31:48 --> 00:31:54
to quit with some error.


414
00:31:54 --> 00:31:55
Saying, wow.


415
00:31:55 --> 00:31:59
I thought my method was good
enough that a hundred guesses

416
00:31:59 --> 00:32:00
should've gotten me there.


417
00:32:00 --> 00:32:03
If it didn't, I may be wrong.


418
00:32:03 --> 00:32:07
I always like to have some
limit, so that my program can't

419
00:32:07 --> 00:32:12
spin off into the ether,
guessing forever.

420
00:32:12 --> 00:32:12
OK.


421
00:32:12 --> 00:32:35
Let's look at an
example of that.

422
00:32:35 --> 00:32:40
So here's a successive
approximation to

423
00:32:40 --> 00:32:42
the square root.


424
00:32:42 --> 00:32:45
I've called it square root bi.


425
00:32:45 --> 00:32:49
The bi is not a reference to
the sexual preferences of the

426
00:32:49 --> 00:32:53
function, but a reference to
the fact that this is an

427
00:32:53 --> 00:33:12
example of what's called
a bi-section method.

428
00:33:12 --> 00:33:18
The basic idea behind any
bi-section method is the same,

429
00:33:18 --> 00:33:21
and we'll see lots of examples
of this semester, is that you

430
00:33:21 --> 00:33:31
have some linearly-arranged
space of possible answers.

431
00:33:31 --> 00:33:36
And it has the property that if
I take a guess somewhere, let's

432
00:33:36 --> 00:33:42
say there, I guess that's the
answer to the question, if it

433
00:33:42 --> 00:33:47
turns out that's not the
answer, I can easily determine

434
00:33:47 --> 00:33:53
whether the answer lies to the
left or the right of the guess.

435
00:33:53 --> 00:33:59
So if I guess that 89.12 is the
square root of a number, and it

436
00:33:59 --> 00:34:03
turns out not to be the square
root of the number, I have a

437
00:34:03 --> 00:34:08
way of saying, is 89.12
too big or too small.

438
00:34:08 --> 00:34:11
If it was too big, then
I know I'd better guess

439
00:34:11 --> 00:34:13
some number over here.


440
00:34:13 --> 00:34:16
It was too small, then
I'd better guess some

441
00:34:16 --> 00:34:20
number over here.


442
00:34:20 --> 00:34:23
Why do I call it bi-section?


443
00:34:23 --> 00:34:27
Because I'm dividing it in
half, and in general as we'll

444
00:34:27 --> 00:34:33
see, when I know what my space
of answers is, I always, as my

445
00:34:33 --> 00:34:37
next guess, choose something
half-way along that line.

446
00:34:37 --> 00:34:41
So I made a guess, and let's
say was too small, and I know

447
00:34:41 --> 00:34:46
the answer is between here and
here, this was too small, I now

448
00:34:46 --> 00:34:50
know that the answer is between
here and here, so my next

449
00:34:50 --> 00:34:53
guess will be in the middle.


450
00:34:53 --> 00:34:56
The beauty of always guessing
in the middle is, at each

451
00:34:56 --> 00:35:00
guess, if it's wrong, I get to
throw out half of

452
00:35:00 --> 00:35:02
the state space.


453
00:35:02 --> 00:35:05
So I know how long it's going
to take me to search the

454
00:35:05 --> 00:35:10
possibilities in some sense,
because I'm getting

455
00:35:10 --> 00:35:16
logarithmically progressed.


456
00:35:16 --> 00:35:21
This is exactly what we saw
when we looked at recursion in

457
00:35:21 --> 00:35:25
some sense, where we solved the
problem by, at each step,

458
00:35:25 --> 00:35:28
solving a smaller problem.


459
00:35:28 --> 00:35:33
The same problem, but on a
smaller solution space.

460
00:35:33 --> 00:35:35
Now as it happens, I'm not
using recursion in this

461
00:35:35 --> 00:35:38
implementation we have up on
the screen, I'm doing it

462
00:35:38 --> 00:35:42
iteratively but the
idea is the same.

463
00:35:42 --> 00:35:46
So we'll take a quick look at
it now, then we'll quit and

464
00:35:46 --> 00:35:48
we'll come back to in the next
lecture a little

465
00:35:48 --> 00:35:51
more thoroughly.


466
00:35:51 --> 00:35:53
I'm going to warn you right
now, that there's a bug in

467
00:35:53 --> 00:35:57
this code, and in the next
lecture, we'll see if we

468
00:35:57 --> 00:36:00
can discover what that is.


469
00:36:00 --> 00:36:04
So, it takes two arguments; x,
the number whose square root

470
00:36:04 --> 00:36:09
we're looking for, and epsilon,
how close we need to get.

471
00:36:09 --> 00:36:15
It assumes that x is
non-negative, and that epsilon

472
00:36:15 --> 00:36:18
is greater than zero.


473
00:36:18 --> 00:36:21
Why do we need to assume that's
epsilon is greater than zero?

474
00:36:21 --> 00:36:24
Well, if you made epsilon zero,
and then say, we're looking for

475
00:36:24 --> 00:36:29
the square root of two, we know
we'll never get an answer.

476
00:36:29 --> 00:36:35
So, we want it to be positive,
and then it returns y such that

477
00:36:35 --> 00:36:39
y times y is within
epsilon of x.

478
00:36:39 --> 00:36:44
It's near, to use the
terminology we used before.

479
00:36:44 --> 00:36:47
The next thing we see
in the program, is two

480
00:36:47 --> 00:36:49
assert statements.


481
00:36:49 --> 00:36:53
This is because I never
trust the people who

482
00:36:53 --> 00:36:57
call my functions to
do the right thing.

483
00:36:57 --> 00:37:00
Even though I said I'm going to
assume certain things about x

484
00:37:00 --> 00:37:04
and epsilon, I'm actually
going to test it.

485
00:37:04 --> 00:37:07
And so, I'm going to assert
that x is greater than or equal

486
00:37:07 --> 00:37:11
to zero, and that epsilon
is greater than zero.

487
00:37:11 --> 00:37:16
What assert does, is it tests
the predicate, say x greater

488
00:37:16 --> 00:37:21
than or equal to zero, if it's
true, it does nothing, just

489
00:37:21 --> 00:37:23
goes on to the next statement.


490
00:37:23 --> 00:37:28
But if it's false, it prints a
message, the string, which is

491
00:37:28 --> 00:37:33
my second argument here, and
then the program just stops.

492
00:37:33 --> 00:37:35
So rather than my function
going off and doing something

493
00:37:35 --> 00:37:39
bizarre, for example running
forever, it just stops with a

494
00:37:39 --> 00:37:42
message saying, you called me
with arguments you shouldn't

495
00:37:42 --> 00:37:45
have called me with.


496
00:37:45 --> 00:37:49
All right, so that's the
specification and then my

497
00:37:49 --> 00:37:52
check of the assumptions.


498
00:37:52 --> 00:37:59
The next thing it does, is it
looks for a range such that I

499
00:37:59 --> 00:38:04
believe I am assured that my
answer lies between the ran--

500
00:38:04 --> 00:38:08
these values, and I'm going to
say, well, my answer will be

501
00:38:08 --> 00:38:13
no smaller than zero,
and no bigger than x.

502
00:38:13 --> 00:38:16
Now, is this the tightest
possible range?

503
00:38:16 --> 00:38:19
Maybe not, but I'm not
too fussy about that.

504
00:38:19 --> 00:38:25
I'm just trying to make sure
that I cover the space.

505
00:38:25 --> 00:38:28
Then I'll start with a guess,
and again I'm not going to

506
00:38:28 --> 00:38:32
worry too much about the guess,
I'm going to take low plus high

507
00:38:32 --> 00:38:36
and divide by two, that is to
say, choose something in the

508
00:38:36 --> 00:38:41
middle of this space, and then
essentially do what

509
00:38:41 --> 00:38:44
we've got here.


510
00:38:44 --> 00:38:46
So it's a little bit more
involved here, I'm going to set

511
00:38:46 --> 00:38:52
my counter to one, just to keep
checking, then say, while the

512
00:38:52 --> 00:38:56
absolute value of the guess
squared minus x is greater than

513
00:38:56 --> 00:39:00
epsilon, that is to say, why my
guess is not yet good enough,

514
00:39:00 --> 00:39:06
and the counter is not greater
than a hundred, I'll

515
00:39:06 --> 00:39:09
get the next guess.


516
00:39:09 --> 00:39:11
Notice by the way, I have a
print statement here which I've

517
00:39:11 --> 00:39:15
commented out, but I sort of
figured that my program would

518
00:39:15 --> 00:39:20
not work correctly the first
time, and so, I, when I first

519
00:39:20 --> 00:39:24
typed and put in a print
statement, it would let me see

520
00:39:24 --> 00:39:27
what was happening each
iteration through this loop, so

521
00:39:27 --> 00:39:30
that if it didn't work, I could
get a sense of why not.

522
00:39:30 --> 00:39:33
In the next lecture, when we
look for the bug in this

523
00:39:33 --> 00:39:36
program, you will see me
uncomment out that print

524
00:39:36 --> 00:39:42
statement, but for now,
we go to the next thing.

525
00:39:42 --> 00:39:46
And we're here, we know the
guess wasn't good enough, so I

526
00:39:46 --> 00:39:52
now say, if the guess squared
was less than x, then I will

527
00:39:52 --> 00:39:58
change the low bound
to be the guess.

528
00:39:58 --> 00:40:03
Otherwise, I'll change the
high bound to be the guess.

529
00:40:03 --> 00:40:05
So I move either the low bound
or I move the high bound,

530
00:40:05 --> 00:40:10
either way I'm cutting the
search space in half each step.

531
00:40:10 --> 00:40:13
I'll get my new guess.


532
00:40:13 --> 00:40:17
I'll increment my
counter, and off I go.

533
00:40:17 --> 00:40:20
In the happy event that
eventually I get a good

534
00:40:20 --> 00:40:26
enough guess, you'll see
a-- I'll exit the loop.

535
00:40:26 --> 00:40:30
When I exit the loop, I
checked, did I exit it because

536
00:40:30 --> 00:40:32
I exceeded the counter,
I didn't have a

537
00:40:32 --> 00:40:34
good-enough guess.


538
00:40:34 --> 00:40:39
If so, I'll print the message
iteration count exceeded.

539
00:40:39 --> 00:40:46
Otherwise, I'll print the
result and return it.

540
00:40:46 --> 00:40:48
Now again, if I were writing a
square root function to be used

541
00:40:48 --> 00:40:52
in another program, I probably
wouldn't bother printing the

542
00:40:52 --> 00:40:55
result and the number of
iterations and all of that, but

543
00:40:55 --> 00:40:58
again, I'm doing that here
for, because we want to

544
00:40:58 --> 00:40:58
see what it's doing.


545
00:40:58 --> 00:41:00
All right.


546
00:41:00 --> 00:41:02
We'll run it a couple
times and then I'll let

547
00:41:02 --> 00:41:06
you out for the day.


548
00:41:06 --> 00:41:20
Let's go do this.


549
00:41:20 --> 00:41:21
All right.


550
00:41:21 --> 00:41:23
We're here.


551
00:41:23 --> 00:41:25
Well, notice when I run
it, nothing happens.

552
00:41:25 --> 00:41:26
Why did nothing happen?


553
00:41:26 --> 00:41:29
Well, nothing happens,
it was just a function.

554
00:41:29 --> 00:41:33
Functions don't do anything
until I call them.

555
00:41:33 --> 00:41:37
So let's call it.


556
00:41:37 --> 00:41:49
Let's call square root bi with
40.001 Took only one at--

557
00:41:49 --> 00:41:52
iteration, that was pretty
fast, estimated two as

558
00:41:52 --> 00:41:55
an answer, we're pretty
happy with that answer.

559
00:41:55 --> 00:41:59
Let's try another example.


560
00:41:59 --> 00:42:03
Let's look at nine.


561
00:42:03 --> 00:42:05
I always like to, by the
way, start with questions

562
00:42:05 --> 00:42:08
whose answer I know.


563
00:42:08 --> 00:42:11
We'll try and get a
little bit more precise.

564
00:42:11 --> 00:42:12
Well, all right.


565
00:42:12 --> 00:42:16
Here it took eighteen
iterations.

566
00:42:16 --> 00:42:19
Didn't actually give me the
answer three, which we know

567
00:42:19 --> 00:42:24
happens to be the answer, but
it gave me something that was

568
00:42:24 --> 00:42:28
within epsilon of three, so it
meets the specification, so

569
00:42:28 --> 00:42:32
I should be perfectly happy.


570
00:42:32 --> 00:42:38
Let's look at another example.


571
00:42:38 --> 00:42:46
Try a bigger number here.


572
00:42:46 --> 00:42:47
All right?


573
00:42:47 --> 00:42:50
So I've looked for the square
root of a thousand, here it

574
00:42:50 --> 00:42:53
took twenty-nine iterations,
we're kind of creeping up

575
00:42:53 --> 00:42:55
there, gave me an estimate.


576
00:42:55 --> 00:42:59
Ah, let's look at our
infamous example of two,

577
00:42:59 --> 00:43:12
see what it does here.


578
00:43:12 --> 00:43:14
Worked around.


579
00:43:14 --> 00:43:20
Now, we can see it's actually
working, and I'm getting

580
00:43:20 --> 00:43:25
answers that we believe are
good-enough answers, but we

581
00:43:25 --> 00:43:29
also see that the speed of what
we talk about as convergence--

582
00:43:29 --> 00:43:33
how many iterations it takes,
the number of iterations-- is

583
00:43:33 --> 00:43:37
variable, and it seems to be
related to at least two things,

584
00:43:37 --> 00:43:40
and we'll see more about
this in the next lecture.

585
00:43:40 --> 00:43:42
The size of the number whose
square root we're looking

586
00:43:42 --> 00:43:49
for, and the precision to
which I want the answer.

587
00:43:49 --> 00:43:55
Next lecture, we'll look at a,
what's wrong with this one, and

588
00:43:55 --> 00:43:58
I would ask you to between now
and the next lecture, think

589
00:43:58 --> 00:44:02
about it, see if you can find
the bug yourself, we'll look

590
00:44:02 --> 00:44:05
first for the bug, and then
after that, we'll look

591
00:44:05 --> 00:44:08
at a better method of
finding the answer.

592
00:44:08 --> 00:44:10
Thank you.


593
00:44:10 --> 00:44:12