WEBVTT 00:00:00.000 --> 00:00:02.420 align:middle line:90% [SQUEAKING] 00:00:02.420 --> 00:00:04.356 align:middle line:90% [RUSTLING] 00:00:04.356 --> 00:00:08.228 align:middle line:90% [CLICKING] 00:00:08.228 --> 00:00:12.573 align:middle line:90% 00:00:12.573 --> 00:00:14.240 align:middle line:84% JUSTIN REICH: All right, let's just talk 00:00:14.240 --> 00:00:16.170 align:middle line:90% about games a little bit. 00:00:16.170 --> 00:00:21.813 align:middle line:84% So you're learning journal assignment was, 00:00:21.813 --> 00:00:23.480 align:middle line:84% what are the learning games that you all 00:00:23.480 --> 00:00:27.120 align:middle line:84% played in your youth or your contemporary times? 00:00:27.120 --> 00:00:29.600 align:middle line:84% We heard Omar has talked with us a couple of times 00:00:29.600 --> 00:00:33.107 align:middle line:84% about his passion for typing games and the importance of them 00:00:33.107 --> 00:00:34.440 align:middle line:90% in his middle school experience. 00:00:34.440 --> 00:00:36.680 align:middle line:84% But what other kinds of games are 00:00:36.680 --> 00:00:39.740 align:middle line:84% memorable to folks in their growing up for learning 00:00:39.740 --> 00:00:40.620 align:middle line:90% or otherwise? 00:00:40.620 --> 00:00:41.900 align:middle line:90% Yeah, Emily, 00:00:41.900 --> 00:00:43.010 align:middle line:90% AUDIENCE: I did, like-- 00:00:43.010 --> 00:00:45.737 align:middle line:84% there was this game, like Game 24, or something. 00:00:45.737 --> 00:00:46.820 align:middle line:90% It was like, you'll have-- 00:00:46.820 --> 00:00:47.610 align:middle line:90% AUDIENCE: Math 24. 00:00:47.610 --> 00:00:48.310 align:middle line:90% AUDIENCE: Oh, Math 24. 00:00:48.310 --> 00:00:49.560 align:middle line:90% AUDIENCE: Yeah, I'm not a fan. 00:00:49.560 --> 00:00:51.870 align:middle line:84% AUDIENCE: Yeah, you have four numbers on a card, 00:00:51.870 --> 00:00:54.830 align:middle line:84% and then you're supposed to add, subtract, multiply, 00:00:54.830 --> 00:00:58.072 align:middle line:84% or divide however you want to get to the number 24. 00:00:58.072 --> 00:01:00.030 align:middle line:84% AUDIENCE: Did you do it online or in the cards? 00:01:00.030 --> 00:01:01.230 align:middle line:90% AUDIENCE: No, in the cards. 00:01:01.230 --> 00:01:03.200 align:middle line:84% AUDIENCE: Yeah, that was in elementary school. 00:01:03.200 --> 00:01:04.200 align:middle line:90% JUSTIN REICH: All right. 00:01:04.200 --> 00:01:05.090 align:middle line:90% Cool. 00:01:05.090 --> 00:01:06.120 align:middle line:90% This is it? 00:01:06.120 --> 00:01:07.110 align:middle line:90% No, we don't know. 00:01:07.110 --> 00:01:08.068 align:middle line:90% AUDIENCE: I don't know. 00:01:08.068 --> 00:01:10.213 align:middle line:84% I used the-- yeah, those cards-- card game version. 00:01:10.213 --> 00:01:12.630 align:middle line:84% JUSTIN REICH: Oh, all right, spinning these things around. 00:01:12.630 --> 00:01:14.330 align:middle line:90% Cool. 00:01:14.330 --> 00:01:16.470 align:middle line:84% And where did-- this showed up in elementary class? 00:01:16.470 --> 00:01:18.380 align:middle line:90% AUDIENCE: Yeah, it was like-- 00:01:18.380 --> 00:01:21.180 align:middle line:84% they put it in the indoor recess spot. 00:01:21.180 --> 00:01:22.820 align:middle line:90% [LAUGHTER] 00:01:22.820 --> 00:01:23.970 align:middle line:90% JUSTIN REICH: That's great. 00:01:23.970 --> 00:01:25.730 align:middle line:90% That's great. 00:01:25.730 --> 00:01:26.460 align:middle line:90% What else? 00:01:26.460 --> 00:01:28.210 align:middle line:84% What other kinds of games did people play? 00:01:28.210 --> 00:01:30.440 align:middle line:90% 00:01:30.440 --> 00:01:31.820 align:middle line:90% Sabrina, what was your-- 00:01:31.820 --> 00:01:35.790 align:middle line:84% AUDIENCE: None of these count as really, really educational, 00:01:35.790 --> 00:01:39.653 align:middle line:84% but I used to play a lot of Coolmath Games as a kid. 00:01:39.653 --> 00:01:41.990 align:middle line:90% I had to change [INAUDIBLE]. 00:01:41.990 --> 00:01:43.379 align:middle line:90% AUDIENCE: They kind of are. 00:01:43.379 --> 00:01:45.546 align:middle line:84% AUDIENCE: At first, they were, and then you just got 00:01:45.546 --> 00:01:46.170 align:middle line:90% to the straight-up flash games. 00:01:46.170 --> 00:01:48.350 align:middle line:84% AUDIENCE: They teach you time management 00:01:48.350 --> 00:01:49.998 align:middle line:90% with all my [INAUDIBLE]. 00:01:49.998 --> 00:01:51.774 align:middle line:90% [LAUGHTER] 00:01:51.774 --> 00:01:55.100 align:middle line:84% AUDIENCE: They teach you Alt-Tab on the computer 00:01:55.100 --> 00:01:57.020 align:middle line:84% when the teacher walks behind your monitor. 00:01:57.020 --> 00:01:59.255 align:middle line:84% JUSTIN REICH: Is Fruit Ninja a math game? 00:01:59.255 --> 00:02:00.450 align:middle line:90% AUDIENCE: Oh, my gosh. 00:02:00.450 --> 00:02:02.450 align:middle line:84% They also teach you, like, teamwork, with, like, 00:02:02.450 --> 00:02:03.420 align:middle line:90% Fireboy and Watergirl. 00:02:03.420 --> 00:02:04.962 align:middle line:84% AUDIENCE: Yes, Fireboy and Watergirl. 00:02:04.962 --> 00:02:07.130 align:middle line:90% [LAUGHTER] 00:02:07.130 --> 00:02:09.350 align:middle line:84% JUSTIN REICH: Yeah, there is a pretty funny 00:02:09.350 --> 00:02:13.920 align:middle line:84% just naming convention of let's call it Coolmath Games. 00:02:13.920 --> 00:02:16.385 align:middle line:84% And then maybe parents won't notice. 00:02:16.385 --> 00:02:18.260 align:middle line:84% I mean, there's definitely a first glance of, 00:02:18.260 --> 00:02:20.005 align:middle line:84% oh, you're playing Coolmath Games. 00:02:20.005 --> 00:02:20.880 align:middle line:90% That's probably fine. 00:02:20.880 --> 00:02:23.490 align:middle line:84% They're like, no, those are not math games. 00:02:23.490 --> 00:02:25.580 align:middle line:90% AUDIENCE: Categories at the top. 00:02:25.580 --> 00:02:26.420 align:middle line:90% JUSTIN REICH: Yeah. 00:02:26.420 --> 00:02:27.795 align:middle line:84% AUDIENCE: [INAUDIBLE] categories. 00:02:27.795 --> 00:02:29.000 align:middle line:90% AUDIENCE: Numbers, logic. 00:02:29.000 --> 00:02:32.370 align:middle line:84% JUSTIN REICH: Yeah, platformer, block removal. 00:02:32.370 --> 00:02:33.710 align:middle line:90% Yeah, I don't know. 00:02:33.710 --> 00:02:34.910 align:middle line:90% There's definitely some-- 00:02:34.910 --> 00:02:36.140 align:middle line:90% AUDIENCE: Applied math games. 00:02:36.140 --> 00:02:38.720 align:middle line:84% JUSTIN REICH: Darts, test your aim in an online multiplayer. 00:02:38.720 --> 00:02:39.330 align:middle line:90% OK. 00:02:39.330 --> 00:02:40.870 align:middle line:90% Idle Breakout, maybe. 00:02:40.870 --> 00:02:43.370 align:middle line:84% AUDIENCE: There used to be those multiplication penguin race 00:02:43.370 --> 00:02:43.870 align:middle line:90% games. 00:02:43.870 --> 00:02:46.010 align:middle line:84% And you have to answer the multiplication questions 00:02:46.010 --> 00:02:47.310 align:middle line:90% as fast as you could. 00:02:47.310 --> 00:02:49.680 align:middle line:90% JUSTIN REICH: Yep, definitely. 00:02:49.680 --> 00:02:50.630 align:middle line:90% AUDIENCE: Yeah. 00:02:50.630 --> 00:02:52.340 align:middle line:90% JUSTIN REICH: Definitely, a-- 00:02:52.340 --> 00:02:53.730 align:middle line:90% oh, Math Playground. 00:02:53.730 --> 00:02:56.540 align:middle line:84% Yeah, that is another one that I've seen. 00:02:56.540 --> 00:03:00.930 align:middle line:84% No, this, I think, is a direct descendant of Math Blaster. 00:03:00.930 --> 00:03:04.130 align:middle line:90% 00:03:04.130 --> 00:03:05.637 align:middle line:90% AUDIENCE: Play Now. 00:03:05.637 --> 00:03:06.595 align:middle line:90% JUSTIN REICH: Play Now. 00:03:06.595 --> 00:03:09.170 align:middle line:90% 00:03:09.170 --> 00:03:10.310 align:middle line:90% AUDIENCE: Playing against-- 00:03:10.310 --> 00:03:12.685 align:middle line:84% JUSTIN REICH: Luke and Shamar-- are Luke and Shamar real? 00:03:12.685 --> 00:03:13.320 align:middle line:90% We don't know. 00:03:13.320 --> 00:03:15.320 align:middle line:84% Let's just see how close this is to Math-- yeah, 00:03:15.320 --> 00:03:18.450 align:middle line:90% this is exactly Math Blaster. 00:03:18.450 --> 00:03:21.080 align:middle line:90% 00:03:21.080 --> 00:03:23.510 align:middle line:84% And we'll play a little Math Blaster later. 00:03:23.510 --> 00:03:26.120 align:middle line:90% Good. 00:03:26.120 --> 00:03:27.090 align:middle line:90% Good job, everyone. 00:03:27.090 --> 00:03:30.560 align:middle line:84% Nice, strong, strong single-digit multiplication 00:03:30.560 --> 00:03:32.280 align:middle line:90% skills in the MIT students. 00:03:32.280 --> 00:03:34.190 align:middle line:90% I'm very proud of you. 00:03:34.190 --> 00:03:36.410 align:middle line:90% Other things? 00:03:36.410 --> 00:03:38.930 align:middle line:84% AUDIENCE: I was saying, in my other class, 00:03:38.930 --> 00:03:40.430 align:middle line:84% my [INAUDIBLE] learning class, we're 00:03:40.430 --> 00:03:41.972 align:middle line:84% really pushing the boundary of what's 00:03:41.972 --> 00:03:43.490 align:middle line:90% considered a learning game. 00:03:43.490 --> 00:03:45.800 align:middle line:84% And one-- I grew up playing a lot of board games 00:03:45.800 --> 00:03:46.910 align:middle line:90% with my family. 00:03:46.910 --> 00:03:48.928 align:middle line:84% And one that I'm right now doing a project 00:03:48.928 --> 00:03:51.470 align:middle line:84% on that makes me really realize all the skills that I learned 00:03:51.470 --> 00:03:53.310 align:middle line:84% is a board game called The Resistance. 00:03:53.310 --> 00:03:54.920 align:middle line:90% Have you heard of that? 00:03:54.920 --> 00:03:56.675 align:middle line:84% And I was just thinking about the skills 00:03:56.675 --> 00:03:58.550 align:middle line:84% that I've learned from that, obviously, like, 00:03:58.550 --> 00:04:02.180 align:middle line:84% how to keep a poker face, how to formulate an argument to back up 00:04:02.180 --> 00:04:06.140 align:middle line:84% the claims that I've made, and how to just have 00:04:06.140 --> 00:04:08.960 align:middle line:90% constructive arguments about-- 00:04:08.960 --> 00:04:10.290 align:middle line:90% between people. 00:04:10.290 --> 00:04:13.137 align:middle line:84% So a lot of communication skills, and just life skills 00:04:13.137 --> 00:04:14.720 align:middle line:84% that I use to this day that were still 00:04:14.720 --> 00:04:18.440 align:middle line:84% just developed in some of those almost role-playing 00:04:18.440 --> 00:04:20.540 align:middle line:90% communication games. 00:04:20.540 --> 00:04:23.120 align:middle line:84% JUSTIN REICH: Have any of you ever played Mafia or Werewolf? 00:04:23.120 --> 00:04:25.340 align:middle line:84% This is a derivation of Mafia or Werewolf. 00:04:25.340 --> 00:04:27.980 align:middle line:84% The guy who invented Werewolf-- there's 00:04:27.980 --> 00:04:31.100 align:middle line:90% a guy who invented Mafia. 00:04:31.100 --> 00:04:34.320 align:middle line:90% It's not just cultural. 00:04:34.320 --> 00:04:36.420 align:middle line:84% It's not like a century old game. 00:04:36.420 --> 00:04:39.240 align:middle line:84% It's a guy who now, I think, lives in Newton or Wellesley. 00:04:39.240 --> 00:04:41.190 align:middle line:84% He applied for a job in my lab once. 00:04:41.190 --> 00:04:42.872 align:middle line:90% He's a Russian immigrant. 00:04:42.872 --> 00:04:44.580 align:middle line:84% He grew up in Russia, and then came here. 00:04:44.580 --> 00:04:47.138 align:middle line:84% But he was like, yeah, I invented Mafia. 00:04:47.138 --> 00:04:49.430 align:middle line:84% And I was like, I played that at camp when I was a kid. 00:04:49.430 --> 00:04:52.780 align:middle line:84% Nope, it came from a particular moment and things like that. 00:04:52.780 --> 00:04:53.530 align:middle line:90% It's kind of cool. 00:04:53.530 --> 00:04:56.480 align:middle line:90% 00:04:56.480 --> 00:04:59.870 align:middle line:84% That's definitely a topic I want us to talk about today 00:04:59.870 --> 00:05:03.110 align:middle line:84% and is featured in the book, this idea of transfer. 00:05:03.110 --> 00:05:06.110 align:middle line:84% What is it that you learned playing The Resistance that we 00:05:06.110 --> 00:05:09.560 align:middle line:84% might plausibly think you would actually be able to deploy 00:05:09.560 --> 00:05:11.430 align:middle line:90% in other non-similar contexts? 00:05:11.430 --> 00:05:14.930 align:middle line:84% And what does research tell us about that? 00:05:14.930 --> 00:05:18.740 align:middle line:84% Which ends up being a crucial question 00:05:18.740 --> 00:05:22.700 align:middle line:84% that the utility and value of learning games hinges around. 00:05:22.700 --> 00:05:26.520 align:middle line:84% If you can get better at lots of things by playing Minecraft, 00:05:26.520 --> 00:05:28.320 align:middle line:84% if you can get better at critical thinking, 00:05:28.320 --> 00:05:29.550 align:middle line:84% if you can get better at teamwork, 00:05:29.550 --> 00:05:31.175 align:middle line:84% if you can get better at socialization, 00:05:31.175 --> 00:05:33.030 align:middle line:84% then, oh, Minecraft sounds pretty cool. 00:05:33.030 --> 00:05:35.150 align:middle line:84% If actually playing a lot of Minecraft 00:05:35.150 --> 00:05:37.400 align:middle line:84% just makes you better at Minecraft, 00:05:37.400 --> 00:05:39.980 align:middle line:84% then the social utility of Minecraft, 00:05:39.980 --> 00:05:42.500 align:middle line:84% playing it in schools or for learning experiences, 00:05:42.500 --> 00:05:44.363 align:middle line:90% goes down quite a bit. 00:05:44.363 --> 00:05:46.530 align:middle line:84% What other games have people played or other things? 00:05:46.530 --> 00:05:48.740 align:middle line:90% What was yours, Khalil? 00:05:48.740 --> 00:05:50.390 align:middle line:84% AUDIENCE: So I could generalize myself, 00:05:50.390 --> 00:05:54.800 align:middle line:84% but there's a lot of games that rely on coordinate systems 00:05:54.800 --> 00:05:56.960 align:middle line:90% as an integral part of the game. 00:05:56.960 --> 00:06:01.790 align:middle line:84% And all of these, you could easily draw the parallels of it 00:06:01.790 --> 00:06:04.790 align:middle line:84% towards math education with regards to that. 00:06:04.790 --> 00:06:08.450 align:middle line:84% The game that is most closely resemble that, I remember, 00:06:08.450 --> 00:06:10.100 align:middle line:90% is Game Over Gopher. 00:06:10.100 --> 00:06:12.230 align:middle line:84% You're just like a carrot zombie, 00:06:12.230 --> 00:06:16.940 align:middle line:90% like a versus zombie-like game. 00:06:16.940 --> 00:06:20.390 align:middle line:90% But it relies on you placing-- 00:06:20.390 --> 00:06:23.030 align:middle line:84% it tells you, hey, place this at this coordinate, 00:06:23.030 --> 00:06:25.290 align:middle line:84% and you have to figure out that quickly. 00:06:25.290 --> 00:06:29.030 align:middle line:84% So you have to internalize the idea of the coordinate system 00:06:29.030 --> 00:06:32.630 align:middle line:84% beyond just like a slow search, because that's 00:06:32.630 --> 00:06:35.630 align:middle line:90% what just gets you so far. 00:06:35.630 --> 00:06:37.130 align:middle line:84% JUSTIN REICH: How quickly will we be 00:06:37.130 --> 00:06:38.500 align:middle line:90% able to see the basics of this? 00:06:38.500 --> 00:06:46.672 align:middle line:90% 00:06:46.672 --> 00:06:48.890 align:middle line:90% [MUMBLING] 00:06:48.890 --> 00:06:49.920 align:middle line:90% No plot! 00:06:49.920 --> 00:06:51.396 align:middle line:90% No plot! 00:06:51.396 --> 00:06:54.590 align:middle line:90% [LAUGHTER] 00:06:54.590 --> 00:06:55.200 align:middle line:90% Oh, I see. 00:06:55.200 --> 00:06:57.625 align:middle line:90% All right. 00:06:57.625 --> 00:06:59.250 align:middle line:84% AUDIENCE: Is this a tower defense game? 00:06:59.250 --> 00:07:02.720 align:middle line:90% AUDIENCE: Yeah, essentially. 00:07:02.720 --> 00:07:06.980 align:middle line:84% JUSTIN REICH: Yeah, stupid gopher. 00:07:06.980 --> 00:07:11.390 align:middle line:90% [LAUGHTER] 00:07:11.390 --> 00:07:15.000 align:middle line:90% [MUMBLING] 00:07:15.000 --> 00:07:24.560 align:middle line:90% 00:07:24.560 --> 00:07:26.330 align:middle line:90% Cool. 00:07:26.330 --> 00:07:29.280 align:middle line:84% The coordinate-- now I've turned the sound off on that. 00:07:29.280 --> 00:07:33.660 align:middle line:84% Let's see if this-- let's see if we can find this, Apple Trek. 00:07:33.660 --> 00:07:38.570 align:middle line:90% 00:07:38.570 --> 00:07:46.400 align:middle line:84% Apple Trek, originally released in 1971, also had-- 00:07:46.400 --> 00:07:49.160 align:middle line:90% let's get into it. 00:07:49.160 --> 00:07:50.660 align:middle line:84% This was the game that I played when 00:07:50.660 --> 00:07:54.020 align:middle line:84% I was a kid that had a sort of coordinate system 00:07:54.020 --> 00:07:57.920 align:middle line:84% in it, where you just had to shoot 00:07:57.920 --> 00:08:01.590 align:middle line:90% photon torpedoes at bad guys. 00:08:01.590 --> 00:08:02.825 align:middle line:90% AUDIENCE: You can press F. 00:08:02.825 --> 00:08:04.220 align:middle line:90% JUSTIN REICH: F will-- 00:08:04.220 --> 00:08:05.930 align:middle line:90% thank you. 00:08:05.930 --> 00:08:07.230 align:middle line:90% Yeah, good. 00:08:07.230 --> 00:08:08.415 align:middle line:84% So you have to sort of fly through-- you're just sort 00:08:08.415 --> 00:08:09.750 align:middle line:90% of flying through the universe. 00:08:09.750 --> 00:08:12.083 align:middle line:84% And eventually, if you want to shoot the photon torpedo, 00:08:12.083 --> 00:08:15.600 align:middle line:84% you have to figure out what the angle is. 00:08:15.600 --> 00:08:20.030 align:middle line:84% And fired at 90 degrees, or 70 degrees, 00:08:20.030 --> 00:08:22.620 align:middle line:84% or other kinds of things like that. 00:08:22.620 --> 00:08:25.280 align:middle line:84% It wasn't a very good gameplay video. 00:08:25.280 --> 00:08:27.530 align:middle line:84% But it's an example of a game with a coordinate system 00:08:27.530 --> 00:08:28.617 align:middle line:90% from 1971. 00:08:28.617 --> 00:08:30.200 align:middle line:84% So that certainly seems like a feature 00:08:30.200 --> 00:08:32.365 align:middle line:90% that shows up repeatedly. 00:08:32.365 --> 00:08:33.990 align:middle line:84% AUDIENCE: Could you look up Odell Fish? 00:08:33.990 --> 00:08:37.510 align:middle line:84% That was, I think, a descendant of that. 00:08:37.510 --> 00:08:38.760 align:middle line:90% JUSTIN REICH: Odell Fish game? 00:08:38.760 --> 00:08:40.880 align:middle line:90% AUDIENCE: Yeah. 00:08:40.880 --> 00:08:43.950 align:middle line:84% But yeah, it was-- no, there was an older version. 00:08:43.950 --> 00:08:47.420 align:middle line:84% There was one from 1986, that it pretty much was just a fish, 00:08:47.420 --> 00:08:49.610 align:middle line:84% and you had to not get eaten by the otter. 00:08:49.610 --> 00:08:51.830 align:middle line:84% So all you had to do was move up and down. 00:08:51.830 --> 00:08:54.195 align:middle line:84% And it was riveting for third graders in the mid '90s. 00:08:54.195 --> 00:08:56.830 align:middle line:90% 00:08:56.830 --> 00:08:58.673 align:middle line:90% JUSTIN REICH: Let's see. 00:08:58.673 --> 00:08:59.590 align:middle line:90% AUDIENCE: [INAUDIBLE]. 00:08:59.590 --> 00:09:01.390 align:middle line:90% AUDIENCE: Yes. 00:09:01.390 --> 00:09:04.140 align:middle line:90% Ah, the memories. 00:09:04.140 --> 00:09:08.060 align:middle line:90% [MUSIC PLAYING] 00:09:08.060 --> 00:09:11.608 align:middle line:90% 00:09:11.608 --> 00:09:13.150 align:middle line:84% JUSTIN REICH: I'm not quite getting-- 00:09:13.150 --> 00:09:14.300 align:middle line:84% AUDIENCE: [INAUDIBLE] down a little bit. 00:09:14.300 --> 00:09:16.690 align:middle line:84% JUSTIN REICH: It's not-- yeah, it's not letting me-- 00:09:16.690 --> 00:09:17.240 align:middle line:90% oh, there. 00:09:17.240 --> 00:09:19.460 align:middle line:84% AUDIENCE: This is the up-to-date graphics, too, guys. 00:09:19.460 --> 00:09:21.017 align:middle line:90% Think about that. 00:09:21.017 --> 00:09:22.100 align:middle line:90% JUSTIN REICH: That's good. 00:09:22.100 --> 00:09:23.890 align:middle line:90% All right. 00:09:23.890 --> 00:09:29.050 align:middle line:84% Are you all too old to-- how many of you had Kahoot! 00:09:29.050 --> 00:09:33.640 align:middle line:84% As a central part of your middle school or high school, or even-- 00:09:33.640 --> 00:09:34.630 align:middle line:90% we won't out. 00:09:34.630 --> 00:09:37.450 align:middle line:84% What's your-- what year did you graduate high school, Jared? 00:09:37.450 --> 00:09:38.230 align:middle line:90% AUDIENCE: 2019. 00:09:38.230 --> 00:09:39.022 align:middle line:90% JUSTIN REICH: 2019. 00:09:39.022 --> 00:09:39.890 align:middle line:90% That's cool. 00:09:39.890 --> 00:09:40.780 align:middle line:90% Yeah. 00:09:40.780 --> 00:09:43.910 align:middle line:84% AUDIENCE: It was-- we used it a lot in high school, though, 00:09:43.910 --> 00:09:46.270 align:middle line:90% like 2015. 00:09:46.270 --> 00:09:48.135 align:middle line:90% We used it a lot. 00:09:48.135 --> 00:09:49.510 align:middle line:84% JUSTIN REICH: Pretty substantial. 00:09:49.510 --> 00:09:55.600 align:middle line:84% How did it get integrated into classroom experience? 00:09:55.600 --> 00:09:58.540 align:middle line:84% AUDIENCE: I think eventually my school had a program of we just 00:09:58.540 --> 00:09:59.620 align:middle line:90% had Chromebooks. 00:09:59.620 --> 00:10:04.030 align:middle line:84% And so then that just naturally led to, OK, 00:10:04.030 --> 00:10:07.420 align:middle line:84% we already all have our Chromebooks. 00:10:07.420 --> 00:10:09.170 align:middle line:84% Do some sort of quiz or whatever. 00:10:09.170 --> 00:10:12.250 align:middle line:84% And it gets real intense in the classroom, [INAUDIBLE], 00:10:12.250 --> 00:10:12.890 align:middle line:90% obviously. 00:10:12.890 --> 00:10:14.785 align:middle line:90% 00:10:14.785 --> 00:10:16.160 align:middle line:84% JUSTIN REICH: People are nodding. 00:10:16.160 --> 00:10:19.030 align:middle line:84% You generally think of Kahoot! fondly. 00:10:19.030 --> 00:10:20.620 align:middle line:90% That was a good-- 00:10:20.620 --> 00:10:22.930 align:middle line:84% I mean, I built similar things in my class 00:10:22.930 --> 00:10:24.260 align:middle line:90% when I was a teacher in 2003. 00:10:24.260 --> 00:10:24.802 align:middle line:90% I don't know. 00:10:24.802 --> 00:10:29.410 align:middle line:84% Somehow I had a Jeopardy! setup that was with like a PowerPoint 00:10:29.410 --> 00:10:33.070 align:middle line:84% thing, where I just deleted little blocks to show 00:10:33.070 --> 00:10:34.705 align:middle line:84% the questions, and then people guessed 00:10:34.705 --> 00:10:38.390 align:middle line:84% the answers and stuff like that, so the quiz show format. 00:10:38.390 --> 00:10:40.670 align:middle line:90% I mean, it really is not-- 00:10:40.670 --> 00:10:43.865 align:middle line:84% that is a classic example of gamification or 00:10:43.865 --> 00:10:44.990 align:middle line:90% chocolate-covered broccoli. 00:10:44.990 --> 00:10:49.310 align:middle line:84% There's no intrinsic game mechanic that's in there. 00:10:49.310 --> 00:10:52.573 align:middle line:84% That's just like competition over review questions, 00:10:52.573 --> 00:10:53.990 align:middle line:84% or knowledge, or things like that. 00:10:53.990 --> 00:10:55.300 align:middle line:90% But it ends up being pretty-- 00:10:55.300 --> 00:10:58.885 align:middle line:84% I mean, it's widely enjoyed by learners 00:10:58.885 --> 00:11:00.760 align:middle line:84% from lots of different backgrounds and things 00:11:00.760 --> 00:11:01.260 align:middle line:90% like that. 00:11:01.260 --> 00:11:03.177 align:middle line:84% AUDIENCE: I feel like it stems from just being 00:11:03.177 --> 00:11:04.410 align:middle line:90% fed broccoli your whole life. 00:11:04.410 --> 00:11:06.340 align:middle line:84% So when you do get that chocolate-- 00:11:06.340 --> 00:11:06.940 align:middle line:84% JUSTIN REICH: Chocolate-covered broccoli, 00:11:06.940 --> 00:11:08.540 align:middle line:90% like, man, this is great! 00:11:08.540 --> 00:11:09.740 align:middle line:90% It's so-- yeah, yeah. 00:11:09.740 --> 00:11:10.400 align:middle line:90% No, I think that's right. 00:11:10.400 --> 00:11:13.067 align:middle line:84% It's like, man, this is so much better than just doing a review. 00:11:13.067 --> 00:11:16.000 align:middle line:84% This is so much better than whatever the typical routine 00:11:16.000 --> 00:11:18.490 align:middle line:90% of class is. 00:11:18.490 --> 00:11:20.187 align:middle line:84% AUDIENCE: [INAUDIBLE] how great-- 00:11:20.187 --> 00:11:22.645 align:middle line:84% it's like you could write your own custom name or whatever. 00:11:22.645 --> 00:11:24.400 align:middle line:90% JUSTIN REICH: Yeah, yeah. 00:11:24.400 --> 00:11:26.630 align:middle line:84% There's those identity components. 00:11:26.630 --> 00:11:28.780 align:middle line:90% There's the surprise components. 00:11:28.780 --> 00:11:32.740 align:middle line:84% AUDIENCE: Also, outside of Jeopardy! being a group thing, 00:11:32.740 --> 00:11:34.392 align:middle line:90% everyone gets to give an answer. 00:11:34.392 --> 00:11:36.350 align:middle line:84% JUSTIN REICH: Everybody gets to give an answer. 00:11:36.350 --> 00:11:38.230 align:middle line:84% Yeah, it does some good mechanics. 00:11:38.230 --> 00:11:41.890 align:middle line:84% Doesn't Kahoot! have things where on your own display, 00:11:41.890 --> 00:11:43.790 align:middle line:84% there's a class-wide leaderboard, 00:11:43.790 --> 00:11:46.190 align:middle line:84% but there's your own personal leaderboard? 00:11:46.190 --> 00:11:50.200 align:middle line:84% So even if you're getting demolished in class, 00:11:50.200 --> 00:11:52.467 align:middle line:84% it just shows you competing against the people who are 00:11:52.467 --> 00:11:53.800 align:middle line:90% nearest to you so that there's-- 00:11:53.800 --> 00:11:55.150 align:middle line:84% AUDIENCE: It tells you how far away you are. 00:11:55.150 --> 00:11:57.610 align:middle line:84% AUDIENCE: Yeah, like 5 points behind this other player. 00:11:57.610 --> 00:11:58.570 align:middle line:90% AUDIENCE: Yeah, yeah. 00:11:58.570 --> 00:11:59.612 align:middle line:90% JUSTIN REICH: Yeah, yeah. 00:11:59.612 --> 00:12:00.880 align:middle line:90% So it's giving you relative-- 00:12:00.880 --> 00:12:04.295 align:middle line:84% some-- I mean, like, Peloton and other kinds of things 00:12:04.295 --> 00:12:04.920 align:middle line:90% do that, right? 00:12:04.920 --> 00:12:05.510 align:middle line:90% I don't know. 00:12:05.510 --> 00:12:07.450 align:middle line:84% I don't have a Peloton, but I think 00:12:07.450 --> 00:12:11.142 align:middle line:84% as these leaderboard systems across distributed computing 00:12:11.142 --> 00:12:12.850 align:middle line:84% environments are developed, they're like, 00:12:12.850 --> 00:12:14.767 align:middle line:84% oh, you don't have to compete against-- if you 00:12:14.767 --> 00:12:16.430 align:middle line:84% have people compete against everyone, 00:12:16.430 --> 00:12:18.110 align:middle line:90% then only one person can win. 00:12:18.110 --> 00:12:21.100 align:middle line:84% If you have everybody competing against a small subset, which 00:12:21.100 --> 00:12:23.750 align:middle line:84% is not necessarily overlapping with anybody else's subset, 00:12:23.750 --> 00:12:29.105 align:middle line:84% then everybody but one can beat somebody in some kind of way. 00:12:29.105 --> 00:12:30.730 align:middle line:84% And then you make up some other people, 00:12:30.730 --> 00:12:33.080 align:middle line:84% and they can beat the rest of them. 00:12:33.080 --> 00:12:33.580 align:middle line:90% Good. 00:12:33.580 --> 00:12:36.661 align:middle line:84% Other games that people played that were-- 00:12:36.661 --> 00:12:39.160 align:middle line:84% AUDIENCE: I guess related to expanding 00:12:39.160 --> 00:12:45.400 align:middle line:84% what a learning game is, most of my youth, I guess, 00:12:45.400 --> 00:12:46.910 align:middle line:90% I was in Iran. 00:12:46.910 --> 00:12:50.890 align:middle line:84% So you didn't really have the internet bandwidth 00:12:50.890 --> 00:12:54.560 align:middle line:84% for online learning games, or anything. 00:12:54.560 --> 00:12:57.760 align:middle line:84% And learning games in general were not a thing 00:12:57.760 --> 00:13:01.340 align:middle line:84% that I was really aware of until I got to the US. 00:13:01.340 --> 00:13:03.850 align:middle line:84% But I guess some stuff that I thought 00:13:03.850 --> 00:13:10.840 align:middle line:84% about was one specific one is like-- my parents got 00:13:10.840 --> 00:13:14.890 align:middle line:84% me a toy sewing machine that was like functional. 00:13:14.890 --> 00:13:17.030 align:middle line:84% And I learned how to sew with that. 00:13:17.030 --> 00:13:21.100 align:middle line:84% And then, generally, just like it was gamification of arts 00:13:21.100 --> 00:13:22.870 align:middle line:90% and crafts a lot of times. 00:13:22.870 --> 00:13:25.250 align:middle line:84% My mom would give me a bunch of stuff-- 00:13:25.250 --> 00:13:26.840 align:middle line:84% be like, go make something with this. 00:13:26.840 --> 00:13:31.100 align:middle line:84% And my sister and I are very much perfectionists. 00:13:31.100 --> 00:13:34.803 align:middle line:84% So we would always be like, oh, I want it to be like this. 00:13:34.803 --> 00:13:36.470 align:middle line:84% And then my mom would be like, OK, well, 00:13:36.470 --> 00:13:40.632 align:middle line:84% if you want that, here's some explanation of how 00:13:40.632 --> 00:13:41.840 align:middle line:90% you might be able to do that. 00:13:41.840 --> 00:13:45.670 align:middle line:84% And I think a lot of that was, one, obviously, 00:13:45.670 --> 00:13:48.040 align:middle line:84% like the skill itself of like I learned 00:13:48.040 --> 00:13:50.420 align:middle line:84% how to sew on a sewing machine and stuff like that. 00:13:50.420 --> 00:13:53.775 align:middle line:84% But also, a lot of practice of like-- 00:13:53.775 --> 00:13:56.637 align:middle line:90% 00:13:56.637 --> 00:13:57.470 align:middle line:90% what do you call it? 00:13:57.470 --> 00:14:01.570 align:middle line:84% Like, spatial understanding, and how do you 00:14:01.570 --> 00:14:05.320 align:middle line:84% write letters on a poster so that they're not all 00:14:05.320 --> 00:14:06.650 align:middle line:90% bunched up at one end? 00:14:06.650 --> 00:14:11.950 align:middle line:84% And just how to plan a certain space effectively, 00:14:11.950 --> 00:14:12.950 align:middle line:90% and things like that. 00:14:12.950 --> 00:14:13.390 align:middle line:90% JUSTIN REICH: Yeah. 00:14:13.390 --> 00:14:15.515 align:middle line:84% And what makes that-- what makes that a game to you 00:14:15.515 --> 00:14:17.350 align:middle line:84% as opposed to a playful apprenticeship? 00:14:17.350 --> 00:14:20.560 align:middle line:90% 00:14:20.560 --> 00:14:24.520 align:middle line:84% AUDIENCE: I guess some of it might be just the stakes. 00:14:24.520 --> 00:14:26.265 align:middle line:90% And then I guess it is-- 00:14:26.265 --> 00:14:29.980 align:middle line:90% 00:14:29.980 --> 00:14:33.310 align:middle line:84% I mean, maybe it was also just a little bit influenced by how 00:14:33.310 --> 00:14:36.000 align:middle line:90% it was presented, I guess, of-- 00:14:36.000 --> 00:14:38.311 align:middle line:90% if it was just like-- 00:14:38.311 --> 00:14:42.010 align:middle line:84% like if it had some pretend aspect, 00:14:42.010 --> 00:14:46.070 align:middle line:84% or if it's a competition with my sister, or things like that, 00:14:46.070 --> 00:14:46.570 align:middle line:90% I guess. 00:14:46.570 --> 00:14:47.080 align:middle line:90% JUSTIN REICH: Yeah. 00:14:47.080 --> 00:14:48.730 align:middle line:84% Famously, there are philosophers who 00:14:48.730 --> 00:14:51.265 align:middle line:84% have tried to pin down the word game, and some of them 00:14:51.265 --> 00:14:52.640 align:middle line:84% have concluded it's not possible. 00:14:52.640 --> 00:14:56.560 align:middle line:84% I think Wittgenstein was one of the ones who said that there 00:14:56.560 --> 00:15:01.490 align:middle line:90% really is no way to accept-- 00:15:01.490 --> 00:15:04.225 align:middle line:84% to create a definition of game that correctly includes 00:15:04.225 --> 00:15:06.350 align:middle line:84% all the things that people think should be included 00:15:06.350 --> 00:15:09.058 align:middle line:84% and excludes all the things that people think should be excluded. 00:15:09.058 --> 00:15:12.340 align:middle line:84% It's quite difficult to figure out what gameness is 00:15:12.340 --> 00:15:15.940 align:middle line:84% and how it relates to sports, or play, or other-- 00:15:15.940 --> 00:15:18.400 align:middle line:84% real life, other kinds of activities. 00:15:18.400 --> 00:15:24.940 align:middle line:84% But that's great that those things felt gameful to you. 00:15:24.940 --> 00:15:27.880 align:middle line:84% They certainly have a bunch of-- what you describe immediately 00:15:27.880 --> 00:15:32.323 align:middle line:84% evokes situated learning kinds of things of, oh, my mom 00:15:32.323 --> 00:15:32.990 align:middle line:90% does this thing. 00:15:32.990 --> 00:15:34.700 align:middle line:84% And so she gave me a little version of this thing. 00:15:34.700 --> 00:15:35.780 align:middle line:84% And then we did some of that stuff. 00:15:35.780 --> 00:15:36.905 align:middle line:90% And she gave us some tasks. 00:15:36.905 --> 00:15:38.230 align:middle line:90% And we played around with them. 00:15:38.230 --> 00:15:43.870 align:middle line:84% And it is funny how across cultures, across history, 00:15:43.870 --> 00:15:47.510 align:middle line:84% young people really like playing at what adults do. 00:15:47.510 --> 00:15:50.800 align:middle line:84% I have a great photograph from Maasailand 00:15:50.800 --> 00:15:53.410 align:middle line:84% with this little three-year-old Maasai 00:15:53.410 --> 00:15:57.230 align:middle line:84% dude who's got a long stick and a set of rocks in the sand. 00:15:57.230 --> 00:16:00.220 align:middle line:84% And he's whacking the rocks in the sand 00:16:00.220 --> 00:16:02.530 align:middle line:84% because the Maasai are all herders. 00:16:02.530 --> 00:16:05.510 align:middle line:84% And so he just watched his older brothers grow up herding goats, 00:16:05.510 --> 00:16:08.640 align:middle line:84% and he knew that he was going to end up herding goats. 00:16:08.640 --> 00:16:10.640 align:middle line:84% And there's just a lot of goat herding going on. 00:16:10.640 --> 00:16:13.140 align:middle line:84% But a thing that's fun for him is to grab a couple of rocks, 00:16:13.140 --> 00:16:16.160 align:middle line:84% pretend they're goats, and start herding them around in the sand. 00:16:16.160 --> 00:16:19.090 align:middle line:84% And I think it's funny, because it's relatable. 00:16:19.090 --> 00:16:23.230 align:middle line:84% Whatever culture you're in, playing at being older 00:16:23.230 --> 00:16:26.090 align:middle line:90% is a big part of youth. 00:16:26.090 --> 00:16:26.590 align:middle line:90% Good. 00:16:26.590 --> 00:16:28.160 align:middle line:90% Other games that people played? 00:16:28.160 --> 00:16:28.957 align:middle line:90% Yeah, Omar. 00:16:28.957 --> 00:16:31.040 align:middle line:84% AUDIENCE: This one might unlock a couple memories, 00:16:31.040 --> 00:16:33.532 align:middle line:84% but Starfall was a big part of my childhood. 00:16:33.532 --> 00:16:34.240 align:middle line:90% JUSTIN REICH: No. 00:16:34.240 --> 00:16:36.160 align:middle line:90% What's that? 00:16:36.160 --> 00:16:38.950 align:middle line:84% AUDIENCE: I primarily played it during kindergarten 00:16:38.950 --> 00:16:41.080 align:middle line:90% and preschool and first grade. 00:16:41.080 --> 00:16:43.208 align:middle line:90% Yeah, it's that first link. 00:16:43.208 --> 00:16:44.500 align:middle line:90% They added a whole new section. 00:16:44.500 --> 00:16:45.470 align:middle line:90% I don't even remember. 00:16:45.470 --> 00:16:47.150 align:middle line:90% I don't ever recall seeing that. 00:16:47.150 --> 00:16:49.340 align:middle line:84% So they've significantly updated the website. 00:16:49.340 --> 00:16:55.600 align:middle line:84% But that was where I really did a lot of reading, intro 00:16:55.600 --> 00:16:57.610 align:middle line:90% to reading. 00:16:57.610 --> 00:17:00.770 align:middle line:84% And I mean, for me, it got pretty old, pretty fast, 00:17:00.770 --> 00:17:02.750 align:middle line:84% but it's very nostalgic nowadays. 00:17:02.750 --> 00:17:05.480 align:middle line:90% So that was one of them. 00:17:05.480 --> 00:17:06.993 align:middle line:84% And then also, like, Leapfrog toys. 00:17:06.993 --> 00:17:07.660 align:middle line:90% [VIDEO PLAYBACK] 00:17:07.660 --> 00:17:10.168 align:middle line:90% - (SINGING) Ah, ah, ah, ah 00:17:10.168 --> 00:17:11.650 align:middle line:90% Short A makes this kind 00:17:11.650 --> 00:17:13.300 align:middle line:90% AUDIENCE: We had a computer lab. 00:17:13.300 --> 00:17:14.217 align:middle line:90% And we'd all go there. 00:17:14.217 --> 00:17:14.800 align:middle line:90% [END PLAYBACK] 00:17:14.800 --> 00:17:15.770 align:middle line:90% And we'd watch these. 00:17:15.770 --> 00:17:19.450 align:middle line:84% And it was really cool, especially because we 00:17:19.450 --> 00:17:23.619 align:middle line:84% didn't have a lot of computing technology 00:17:23.619 --> 00:17:25.640 align:middle line:90% when I grew up at that time. 00:17:25.640 --> 00:17:27.670 align:middle line:84% So it was really cool to see this. 00:17:27.670 --> 00:17:31.300 align:middle line:84% JUSTIN REICH: Kind of a special highlight for the day, 00:17:31.300 --> 00:17:33.250 align:middle line:84% to be able to do these kinds of things. 00:17:33.250 --> 00:17:36.367 align:middle line:84% And I hope that as you think about these kinds of things, 00:17:36.367 --> 00:17:38.950 align:middle line:84% think about Kahoot! and stuff like that, that your mind drifts 00:17:38.950 --> 00:17:42.340 align:middle line:84% back to what you read in Larry Cuban and his description 00:17:42.340 --> 00:17:45.250 align:middle line:84% of the material concerns of teachers, the schedule 00:17:45.250 --> 00:17:47.980 align:middle line:84% concerns of teachers, and things like, oh, it's 00:17:47.980 --> 00:17:51.310 align:middle line:84% pretty handy to be able to have a bunch of digital books 00:17:51.310 --> 00:17:53.830 align:middle line:84% that people find extremely compelling to shove 00:17:53.830 --> 00:17:56.240 align:middle line:84% six-year-olds in front of for an hour during the day 00:17:56.240 --> 00:17:58.850 align:middle line:84% so you can finish your lunch, or do some grading, 00:17:58.850 --> 00:18:03.010 align:middle line:84% or do a little planning and prep for other times of the day. 00:18:03.010 --> 00:18:04.760 align:middle line:84% I don't know if this is well-studied, 00:18:04.760 --> 00:18:09.737 align:middle line:84% but I bet Larry Cuban's finding that television and film 00:18:09.737 --> 00:18:12.070 align:middle line:84% strips were more likely to be used at the end of the day 00:18:12.070 --> 00:18:14.500 align:middle line:84% in elementary schools probably parallels findings 00:18:14.500 --> 00:18:16.210 align:middle line:84% that you might have today about when 00:18:16.210 --> 00:18:18.040 align:middle line:84% Kahoot! games or other things like 00:18:18.040 --> 00:18:22.840 align:middle line:84% this used in the rhythm of the day, and so forth. 00:18:22.840 --> 00:18:25.720 align:middle line:84% So yeah, different technologies, but we 00:18:25.720 --> 00:18:30.040 align:middle line:84% can apply some of the similar sorts of analytic tools 00:18:30.040 --> 00:18:31.280 align:middle line:90% and frameworks to them. 00:18:31.280 --> 00:18:33.520 align:middle line:90% Good. 00:18:33.520 --> 00:18:36.085 align:middle line:84% What are some things that I wanted to-- 00:18:36.085 --> 00:18:37.460 align:middle line:84% let me talk for you a little bit, 00:18:37.460 --> 00:18:39.418 align:middle line:84% and then I'll give you a little bit of a chance 00:18:39.418 --> 00:18:41.360 align:middle line:90% to play some games. 00:18:41.360 --> 00:18:44.020 align:middle line:84% And then we can talk a little bit about transfer. 00:18:44.020 --> 00:18:46.870 align:middle line:84% One thing that I just wanted to draw your attention to that's 00:18:46.870 --> 00:18:51.640 align:middle line:84% in the syllabus is for Failure to Disrupt in the Rabbit Hole 00:18:51.640 --> 00:18:53.330 align:middle line:84% readings that you're not required to do. 00:18:53.330 --> 00:18:57.370 align:middle line:84% One of those links for every chapter is a podcast episode. 00:18:57.370 --> 00:19:00.040 align:middle line:84% And I was thinking about them, because some people who 00:19:00.040 --> 00:19:02.390 align:middle line:84% are missing class this week because of Sloan's SIP Week, 00:19:02.390 --> 00:19:03.925 align:middle line:84% I was going to recommend those to them as sort 00:19:03.925 --> 00:19:05.258 align:middle line:90% of supplementary kind of things. 00:19:05.258 --> 00:19:08.900 align:middle line:84% But for this chapter in particular, 00:19:08.900 --> 00:19:10.812 align:middle line:84% Scot Osterweil, who developed the Logical 00:19:10.812 --> 00:19:12.770 align:middle line:84% Journey of the Zoombinis, is one of the guests, 00:19:12.770 --> 00:19:14.200 align:middle line:84% and then Constance Steinkuehler, who actually 00:19:14.200 --> 00:19:16.533 align:middle line:84% had a job for a while in the US Department of Education, 00:19:16.533 --> 00:19:19.492 align:middle line:84% thinking about the role of games in education systems is there. 00:19:19.492 --> 00:19:20.950 align:middle line:84% Probably one of the most fun things 00:19:20.950 --> 00:19:24.130 align:middle line:84% about the podcast episodes is that I really 00:19:24.130 --> 00:19:27.760 align:middle line:84% encourage the people who came to disagree with me. 00:19:27.760 --> 00:19:33.400 align:middle line:84% And so in most episodes, there are 00:19:33.400 --> 00:19:35.740 align:middle line:84% claims made about where the book is 00:19:35.740 --> 00:19:39.260 align:middle line:84% wrong by very well-informed people on those topics. 00:19:39.260 --> 00:19:41.170 align:middle line:84% So Constance Steinkuehler does not 00:19:41.170 --> 00:19:43.930 align:middle line:84% agree with me in this episode about the relationship 00:19:43.930 --> 00:19:46.130 align:middle line:84% between games and learning and transfer. 00:19:46.130 --> 00:19:48.550 align:middle line:90% And it's fun to listen to that. 00:19:48.550 --> 00:19:53.530 align:middle line:84% I think Mitch and Natalie had some disagreements with me 00:19:53.530 --> 00:19:58.610 align:middle line:84% about what the learning goals of scratch are in practice. 00:19:58.610 --> 00:19:59.777 align:middle line:90% So those can be kind of fun. 00:19:59.777 --> 00:20:00.443 align:middle line:90% So I don't know. 00:20:00.443 --> 00:20:02.630 align:middle line:84% If you're going for a run or something like that, 00:20:02.630 --> 00:20:04.510 align:middle line:84% you need some light podcast listening to. 00:20:04.510 --> 00:20:06.430 align:middle line:84% These can be fun episodes, if you're not 00:20:06.430 --> 00:20:08.560 align:middle line:84% getting enough education technology in your 3 hours 00:20:08.560 --> 00:20:09.820 align:middle line:90% a week with me. 00:20:09.820 --> 00:20:12.640 align:middle line:90% There's more to be had. 00:20:12.640 --> 00:20:16.330 align:middle line:84% So the way that this chapter is structured 00:20:16.330 --> 00:20:19.930 align:middle line:84% is I call it testing the theory of the genres of learning 00:20:19.930 --> 00:20:25.150 align:middle line:84% at scale, which is what you should be-- 00:20:25.150 --> 00:20:31.570 align:middle line:84% a big reason for organizing the idea of three genres of learning 00:20:31.570 --> 00:20:34.720 align:middle line:84% at scale is I want people to be able to take new technologies 00:20:34.720 --> 00:20:37.487 align:middle line:84% and say, oh, that is kind of like this. 00:20:37.487 --> 00:20:39.320 align:middle line:84% And if we say, oh, that's kind of like this, 00:20:39.320 --> 00:20:40.870 align:middle line:84% then we can say, all right, so we probably know 00:20:40.870 --> 00:20:42.440 align:middle line:90% something about its technology. 00:20:42.440 --> 00:20:45.490 align:middle line:84% We probably know something about the pedagogical underpinnings. 00:20:45.490 --> 00:20:47.270 align:middle line:84% We can make some guess about research. 00:20:47.270 --> 00:20:49.240 align:middle line:84% Like a big thing that I would want someone 00:20:49.240 --> 00:20:52.090 align:middle line:84% who reads Failure to Disrupt to be able to do 00:20:52.090 --> 00:20:55.030 align:middle line:84% is to look at new AI tools and be like, 00:20:55.030 --> 00:20:57.760 align:middle line:84% we did not have generative pre-trained transformers 00:20:57.760 --> 00:21:00.380 align:middle line:84% that were pretty useful until November of 2022. 00:21:00.380 --> 00:21:02.942 align:middle line:84% But the things that we're trying to do with them 00:21:02.942 --> 00:21:04.900 align:middle line:84% are going to be similar to other kinds of stuff 00:21:04.900 --> 00:21:05.990 align:middle line:90% that we've done in the past. 00:21:05.990 --> 00:21:07.420 align:middle line:84% And we should be able to slot the things 00:21:07.420 --> 00:21:09.545 align:middle line:84% that we're going to do to them into different kinds 00:21:09.545 --> 00:21:10.720 align:middle line:90% of categories. 00:21:10.720 --> 00:21:15.220 align:middle line:84% Games struck me as a good testing place for this, 00:21:15.220 --> 00:21:18.790 align:middle line:84% because it's not the case that I think games fit neatly 00:21:18.790 --> 00:21:20.410 align:middle line:84% into one of these three categories 00:21:20.410 --> 00:21:22.090 align:middle line:84% of instructor-guided learning at scale, 00:21:22.090 --> 00:21:23.600 align:middle line:84% or algorithm-guided learning at scale, 00:21:23.600 --> 00:21:24.975 align:middle line:84% or peer-guided learning at scale. 00:21:24.975 --> 00:21:28.030 align:middle line:84% But I do think that any given game probably 00:21:28.030 --> 00:21:30.740 align:middle line:84% fits into one of those categories pretty nicely, 00:21:30.740 --> 00:21:34.240 align:middle line:84% or has some of recognizable hybridity 00:21:34.240 --> 00:21:36.850 align:middle line:90% across those categories. 00:21:36.850 --> 00:21:42.970 align:middle line:84% So something like Math Blaster is 00:21:42.970 --> 00:21:46.450 align:middle line:84% kind of an example of you could decide whether or not 00:21:46.450 --> 00:21:48.520 align:middle line:84% it's instructor-guided learning at scale, 00:21:48.520 --> 00:21:51.103 align:middle line:84% or algorithm-guided learning at scale, or something like that. 00:21:51.103 --> 00:21:54.340 align:middle line:84% It doesn't really have a lot of item response theory to it. 00:21:54.340 --> 00:21:57.790 align:middle line:84% But it's basically just an autograder. 00:21:57.790 --> 00:21:58.338 align:middle line:90% You go in. 00:21:58.338 --> 00:21:59.630 align:middle line:90% You're given a type of problem. 00:21:59.630 --> 00:22:01.633 align:middle line:84% You practice that problem over and over again. 00:22:01.633 --> 00:22:03.550 align:middle line:84% Maybe the difficulty goes up, and you practice 00:22:03.550 --> 00:22:06.250 align:middle line:90% another type of problem. 00:22:06.250 --> 00:22:08.270 align:middle line:84% Had any of you ever heard of Vanished before? 00:22:08.270 --> 00:22:10.420 align:middle line:84% Maybe some of you in the Games for Learning class, 00:22:10.420 --> 00:22:12.663 align:middle line:90% did you read about Vanished? 00:22:12.663 --> 00:22:15.080 align:middle line:84% So Vanished, which I write about a little bit in the book, 00:22:15.080 --> 00:22:19.120 align:middle line:84% was this really cool project that the MIT Education 00:22:19.120 --> 00:22:21.350 align:middle line:84% Arcade did with the Smithsonian Museum. 00:22:21.350 --> 00:22:25.000 align:middle line:84% It was sort of-- did anybody watch the television show Lost? 00:22:25.000 --> 00:22:26.420 align:middle line:90% OK, so a couple nods there. 00:22:26.420 --> 00:22:28.490 align:middle line:84% I don't-- have any of you ever participated-- 00:22:28.490 --> 00:22:30.935 align:middle line:90% I mean, actually, I don't know. 00:22:30.935 --> 00:22:33.310 align:middle line:84% Are you all aware of what is going on with Kate Middleton 00:22:33.310 --> 00:22:35.590 align:middle line:84% right now, where people are trying to figure out 00:22:35.590 --> 00:22:38.380 align:middle line:84% why she's not showing up in public and all these conspiracy 00:22:38.380 --> 00:22:39.500 align:middle line:90% theories show up about it? 00:22:39.500 --> 00:22:40.730 align:middle line:84% And people do these detective work. 00:22:40.730 --> 00:22:42.438 align:middle line:84% There's a bunch of things like that, or-- 00:22:42.438 --> 00:22:46.462 align:middle line:84% I don't know-- when the Malaysian airline flight crashed 00:22:46.462 --> 00:22:47.420 align:middle line:90% and nobody really knew. 00:22:47.420 --> 00:22:47.962 align:middle line:90% I don't know. 00:22:47.962 --> 00:22:50.750 align:middle line:84% There's something that human beings have that's like, 00:22:50.750 --> 00:22:53.690 align:middle line:84% let's all solve the mystery together. 00:22:53.690 --> 00:22:56.960 align:middle line:84% And it gets kind of woven into conspiratorial thinking. 00:22:56.960 --> 00:23:00.340 align:middle line:84% It overlaps nicely with fiction, with things like Lost, 00:23:00.340 --> 00:23:03.200 align:middle line:84% or other television shows that do that. 00:23:03.200 --> 00:23:07.210 align:middle line:84% But Vanished was basically like an alternate reality game 00:23:07.210 --> 00:23:12.950 align:middle line:84% that was played out on the web, where there were codes to break. 00:23:12.950 --> 00:23:15.470 align:middle line:84% There were secret messages spread across websites. 00:23:15.470 --> 00:23:17.643 align:middle line:84% There were scientific claims that were happening-- 00:23:17.643 --> 00:23:18.560 align:middle line:90% other sorts of things. 00:23:18.560 --> 00:23:20.950 align:middle line:84% There were messages from the future being beamed back 00:23:20.950 --> 00:23:22.360 align:middle line:90% into the past. 00:23:22.360 --> 00:23:27.460 align:middle line:84% And students signed up either as classes or individuals 00:23:27.460 --> 00:23:30.587 align:middle line:84% to get the kind of messages that went along. 00:23:30.587 --> 00:23:32.920 align:middle line:84% I think the game ran for a couple of months or something 00:23:32.920 --> 00:23:37.630 align:middle line:84% like that, but very much a peer-guided learning at scale 00:23:37.630 --> 00:23:39.130 align:middle line:90% kind of example-- 00:23:39.130 --> 00:23:41.380 align:middle line:84% just a series of puzzles put out there 00:23:41.380 --> 00:23:43.490 align:middle line:84% that you collaboratively tried to solve. 00:23:43.490 --> 00:23:48.040 align:middle line:84% You had to interact with other people to find. 00:23:48.040 --> 00:23:54.943 align:middle line:84% A big-- these two kinds of games illustrate 00:23:54.943 --> 00:23:56.860 align:middle line:84% some of the basic principles that we were just 00:23:56.860 --> 00:23:58.660 align:middle line:90% talking about a little bit. 00:23:58.660 --> 00:24:01.270 align:middle line:84% Math Blaster, so what you did in Math Blaster 00:24:01.270 --> 00:24:03.340 align:middle line:84% is you took that little character, 00:24:03.340 --> 00:24:06.490 align:middle line:84% and you ran it back and forth to get the correct answer. 00:24:06.490 --> 00:24:09.080 align:middle line:84% Actually, what was the penguin game that we just played? 00:24:09.080 --> 00:24:10.030 align:middle line:90% What was that called? 00:24:10.030 --> 00:24:11.990 align:middle line:90% Penguin Race-Off or something? 00:24:11.990 --> 00:24:13.720 align:middle line:90% AUDIENCE: Yes, I think so. 00:24:13.720 --> 00:24:16.060 align:middle line:84% JUSTIN REICH: So Penguin Race-Off 00:24:16.060 --> 00:24:21.670 align:middle line:84% is an example of what sometimes gets called gamification, which 00:24:21.670 --> 00:24:23.512 align:middle line:84% is, how do we take a mathematic-- 00:24:23.512 --> 00:24:25.720 align:middle line:84% how do we take an intellectual exercise that we think 00:24:25.720 --> 00:24:29.170 align:middle line:84% is useful for people and put game elements on it to inspire 00:24:29.170 --> 00:24:32.350 align:middle line:90% people to do that activity? 00:24:32.350 --> 00:24:34.450 align:middle line:84% What other kinds of software have you 00:24:34.450 --> 00:24:39.040 align:middle line:84% played with that takes basically a worksheet activity 00:24:39.040 --> 00:24:41.805 align:middle line:84% and adds game elements on top of it? 00:24:41.805 --> 00:24:52.000 align:middle line:90% 00:24:52.000 --> 00:24:54.580 align:middle line:84% AUDIENCE: Just kind of thinking about the slide 00:24:54.580 --> 00:24:56.878 align:middle line:84% you presented earlier, like Duolingo, kind of. 00:24:56.878 --> 00:24:58.420 align:middle line:84% JUSTIN REICH: Yep, Duolingo has a ton 00:24:58.420 --> 00:25:00.290 align:middle line:90% of gamification built into it. 00:25:00.290 --> 00:25:01.155 align:middle line:90% I don't know. 00:25:01.155 --> 00:25:02.530 align:middle line:84% How many of you-- how many of you 00:25:02.530 --> 00:25:04.970 align:middle line:90% use Duolingo, or have used it? 00:25:04.970 --> 00:25:05.885 align:middle line:90% Oh, just a couple. 00:25:05.885 --> 00:25:08.260 align:middle line:84% So maybe we'll have some more people play around with it. 00:25:08.260 --> 00:25:11.170 align:middle line:84% But they have millions of people who are doing little language 00:25:11.170 --> 00:25:11.810 align:middle line:90% activities. 00:25:11.810 --> 00:25:13.477 align:middle line:84% But a big part of why they're doing it-- 00:25:13.477 --> 00:25:16.320 align:middle line:84% like, little language activities are not that inherently fun. 00:25:16.320 --> 00:25:19.380 align:middle line:84% But what are the features in Duolingo that try to get people 00:25:19.380 --> 00:25:24.000 align:middle line:84% hooked to do it and keep doing it? 00:25:24.000 --> 00:25:24.840 align:middle line:90% AUDIENCE: Streaks. 00:25:24.840 --> 00:25:26.430 align:middle line:84% JUSTIN REICH: Streaks are a big one, 00:25:26.430 --> 00:25:29.980 align:middle line:84% which is just a number that you try to increment every day. 00:25:29.980 --> 00:25:32.520 align:middle line:90% That's kind of a game. 00:25:32.520 --> 00:25:34.110 align:middle line:90% AUDIENCE: Visual design. 00:25:34.110 --> 00:25:36.030 align:middle line:84% JUSTIN REICH: What kind of elements? 00:25:36.030 --> 00:25:38.710 align:middle line:84% AUDIENCE: Colorful design, interactive design, 00:25:38.710 --> 00:25:42.342 align:middle line:84% all of these bubbly feelings to everything in it. 00:25:42.342 --> 00:25:43.300 align:middle line:90% JUSTIN REICH: Yep, yep. 00:25:43.300 --> 00:25:46.270 align:middle line:84% So yeah, like creating it and turning it into a playful space. 00:25:46.270 --> 00:25:48.020 align:middle line:84% Some of the most important visual elements 00:25:48.020 --> 00:25:49.440 align:middle line:84% are when you get something right. 00:25:49.440 --> 00:25:53.610 align:middle line:84% There's usually a little flash for sound, a little ding. 00:25:53.610 --> 00:25:55.560 align:middle line:84% edX had these big green check marks 00:25:55.560 --> 00:25:57.550 align:middle line:84% for a while when you got answers correct. 00:25:57.550 --> 00:26:00.217 align:middle line:84% AUDIENCE: I think another thing that's been a recent development 00:26:00.217 --> 00:26:01.860 align:middle line:90% has been their PR. 00:26:01.860 --> 00:26:07.290 align:middle line:84% Recently, at least on TikTok, Duolingo has gone unhinged. 00:26:07.290 --> 00:26:08.520 align:middle line:90% And the bird's everywhere. 00:26:08.520 --> 00:26:10.530 align:middle line:90% It comments on weird things. 00:26:10.530 --> 00:26:13.290 align:middle line:84% And every time it comments somewhere, 00:26:13.290 --> 00:26:17.580 align:middle line:90% there's a million replies to it. 00:26:17.580 --> 00:26:20.950 align:middle line:84% And I think, wanting to be like a part of that community or say, 00:26:20.950 --> 00:26:23.650 align:middle line:84% I've done that, is also like a big thing in society today. 00:26:23.650 --> 00:26:24.692 align:middle line:90% JUSTIN REICH: Yeah, good. 00:26:24.692 --> 00:26:26.880 align:middle line:90% 00:26:26.880 --> 00:26:32.860 align:middle line:84% Mari, who's not here, wrote in-- like, to me, Duolingo, 00:26:32.860 --> 00:26:35.250 align:middle line:84% when I think about it, is a classic example 00:26:35.250 --> 00:26:41.300 align:middle line:84% of gamification, very instructionist kind of inspired, 00:26:41.300 --> 00:26:43.300 align:middle line:84% like let's just put some game elements on stuff. 00:26:43.300 --> 00:26:43.882 align:middle line:90% And Mari wrote an essay. 00:26:43.882 --> 00:26:45.424 align:middle line:84% It was like, no, actually, this thing 00:26:45.424 --> 00:26:49.200 align:middle line:84% is super motivated by situated learning, the way 00:26:49.200 --> 00:26:50.550 align:middle line:90% that it makes you-- 00:26:50.550 --> 00:26:53.610 align:middle line:84% things like that are like, oh, the Duo bird is out there, 00:26:53.610 --> 00:26:56.320 align:middle line:84% and I can have a parasocial relationship with the Duo bird. 00:26:56.320 --> 00:26:59.100 align:middle line:84% And it will comment on my TikTok feeds, and I can respond to it, 00:26:59.100 --> 00:27:01.625 align:middle line:84% and things like that got me thinking a little bit 00:27:01.625 --> 00:27:02.500 align:middle line:90% differently about it. 00:27:02.500 --> 00:27:04.440 align:middle line:84% Maybe we keep talking about that. 00:27:04.440 --> 00:27:08.550 align:middle line:84% But yeah, the term of art for this kind of approach 00:27:08.550 --> 00:27:11.640 align:middle line:84% is chocolate-covered broccoli, that, basically, this 00:27:11.640 --> 00:27:13.830 align:middle line:84% is super boring math to do, but we're just 00:27:13.830 --> 00:27:17.483 align:middle line:84% going to pour chocolate on it to get you to try to do more of it. 00:27:17.483 --> 00:27:19.900 align:middle line:84% And of course, like the joke of chocolate-covered broccoli 00:27:19.900 --> 00:27:22.590 align:middle line:84% is that chocolate-covered broccoli does actually not 00:27:22.590 --> 00:27:23.880 align:middle line:90% taste that good. 00:27:23.880 --> 00:27:25.797 align:middle line:84% And Math Blaster is actually not that fun. 00:27:25.797 --> 00:27:27.880 align:middle line:84% It's fun for a little while, and then you're like, 00:27:27.880 --> 00:27:30.720 align:middle line:90% oh, I'm just doing division. 00:27:30.720 --> 00:27:34.770 align:middle line:84% Whereas Vanished and other kinds of games like it, 00:27:34.770 --> 00:27:36.550 align:middle line:84% there's not an as-well-accepted name, 00:27:36.550 --> 00:27:38.940 align:middle line:84% but my colleagues at the Education Arcade 00:27:38.940 --> 00:27:41.670 align:middle line:84% refer to these games as resonant games. 00:27:41.670 --> 00:27:43.680 align:middle line:84% And a key feature of resonant games 00:27:43.680 --> 00:27:47.910 align:middle line:84% is that instead of trying to add game elements on top 00:27:47.910 --> 00:27:50.380 align:middle line:84% of some kind of intellectual exercise, 00:27:50.380 --> 00:27:52.530 align:middle line:84% you're trying to find the playfulness 00:27:52.530 --> 00:27:56.110 align:middle line:90% in the intellectual exercise. 00:27:56.110 --> 00:28:00.670 align:middle line:84% It's not, how do we add game mechanics to math? 00:28:00.670 --> 00:28:04.960 align:middle line:84% It's, what is already fun and playful and interesting in math? 00:28:04.960 --> 00:28:08.850 align:middle line:84% And then how do we have that fun and playfulness kind of surface 00:28:08.850 --> 00:28:11.340 align:middle line:84% and emerge so that we get learners 00:28:11.340 --> 00:28:13.710 align:middle line:90% to engage with that first? 00:28:13.710 --> 00:28:17.230 align:middle line:84% In Vanished, how do we teach scientific thinking, 00:28:17.230 --> 00:28:20.310 align:middle line:84% scientific practices by having those practices emerge 00:28:20.310 --> 00:28:23.580 align:middle line:84% kind of naturally in this conspiratorial, 00:28:23.580 --> 00:28:27.300 align:middle line:84% mystery kind of story that happens? 00:28:27.300 --> 00:28:27.960 align:middle line:90% Oh, good. 00:28:27.960 --> 00:28:30.570 align:middle line:84% This is a great example of what Khalil was just 00:28:30.570 --> 00:28:37.710 align:middle line:84% talking about of the very playful environment. 00:28:37.710 --> 00:28:39.638 align:middle line:84% We'll give you some experience points. 00:28:39.638 --> 00:28:40.930 align:middle line:90% We'll tell you about your time. 00:28:40.930 --> 00:28:42.018 align:middle line:90% We'll call you amazing. 00:28:42.018 --> 00:28:43.810 align:middle line:84% We'll give you these funny little stickers. 00:28:43.810 --> 00:28:45.240 align:middle line:90% I mean, this is-- 00:28:45.240 --> 00:28:47.440 align:middle line:90% I don't know if any of you do-- 00:28:47.440 --> 00:28:49.633 align:middle line:84% did your elementary teachers put stickers 00:28:49.633 --> 00:28:51.550 align:middle line:84% on successful assignments and stuff like that? 00:28:51.550 --> 00:28:53.230 align:middle line:84% Can you think of gold stars and stuff like that? 00:28:53.230 --> 00:28:55.150 align:middle line:84% Yeah, this is just like a digital version of that. 00:28:55.150 --> 00:28:56.358 align:middle line:90% It is the same kind of thing. 00:28:56.358 --> 00:28:59.280 align:middle line:84% Let's give you some kind of visual reward reminder 00:28:59.280 --> 00:29:04.500 align:middle line:84% to get you to keep playing and doing these things. 00:29:04.500 --> 00:29:07.163 align:middle line:84% This is one that I talk about in the book, Lumosity, 00:29:07.163 --> 00:29:09.330 align:middle line:84% which maybe we'll talk about a little bit later when 00:29:09.330 --> 00:29:12.810 align:middle line:84% we talk about transfer and things like that. 00:29:12.810 --> 00:29:14.370 align:middle line:84% All right, so I would like to invite 00:29:14.370 --> 00:29:17.410 align:middle line:90% you to play two of these games. 00:29:17.410 --> 00:29:19.403 align:middle line:84% And you can find these links in the syllabus. 00:29:19.403 --> 00:29:20.820 align:middle line:84% Why don't we take 15 minutes or so 00:29:20.820 --> 00:29:22.920 align:middle line:84% and let you pull out some computers? 00:29:22.920 --> 00:29:24.030 align:middle line:90% I'm going to give you-- 00:29:24.030 --> 00:29:24.750 align:middle line:90% I'm going to give you. 00:29:24.750 --> 00:29:26.010 align:middle line:84% Maybe before you pull them out, let me just 00:29:26.010 --> 00:29:27.060 align:middle line:84% explain these four things, and then you 00:29:27.060 --> 00:29:28.518 align:middle line:84% can do whatever you want with them. 00:29:28.518 --> 00:29:30.420 align:middle line:90% The first one is Math Blaster. 00:29:30.420 --> 00:29:34.510 align:middle line:84% I don't think I successfully found the 1983 version. 00:29:34.510 --> 00:29:36.900 align:middle line:90% I think I found a 1996 one. 00:29:36.900 --> 00:29:39.240 align:middle line:84% But you should play a few rounds of Math Blaster 00:29:39.240 --> 00:29:41.490 align:middle line:84% and see what all the hype is about. 00:29:41.490 --> 00:29:43.830 align:middle line:84% The second game is The Logical Journey 00:29:43.830 --> 00:29:48.100 align:middle line:84% of the Zoombinis, which was a key part of Erin's childhood. 00:29:48.100 --> 00:29:50.050 align:middle line:84% And so now she'll get to go back to that. 00:29:50.050 --> 00:29:52.687 align:middle line:84% It has a really long kind of intro story and stuff like that. 00:29:52.687 --> 00:29:54.520 align:middle line:84% If that's what you're into, you can do that. 00:29:54.520 --> 00:29:55.420 align:middle line:84% But if you want to play the game, 00:29:55.420 --> 00:29:57.600 align:middle line:84% you'll have to skip a bunch of introductory stuff 00:29:57.600 --> 00:30:02.128 align:middle line:84% to get into some interesting combinatorics kinds of things. 00:30:02.128 --> 00:30:04.170 align:middle line:84% One thing, which is not a game, but if any of you 00:30:04.170 --> 00:30:07.860 align:middle line:84% are big Minecraft fans, I gave you 00:30:07.860 --> 00:30:10.440 align:middle line:84% a link to what is essentially the Minecraft 00:30:10.440 --> 00:30:12.390 align:middle line:90% website for teachers. 00:30:12.390 --> 00:30:13.890 align:middle line:84% You won't be able to-- unless you've 00:30:13.890 --> 00:30:15.200 align:middle line:84% got Minecraft on your computer, you 00:30:15.200 --> 00:30:17.367 align:middle line:84% won't be able to jump in there and play with things. 00:30:17.367 --> 00:30:22.830 align:middle line:84% But a thing that you can do is go to their Lesson Plan Finder. 00:30:22.830 --> 00:30:25.473 align:middle line:84% And that's probably the most useful-- 00:30:25.473 --> 00:30:27.390 align:middle line:84% probably the thing you'd find most interesting 00:30:27.390 --> 00:30:29.490 align:middle line:84% is go to Explore Lessons, and pick 00:30:29.490 --> 00:30:32.760 align:middle line:84% a topic you're interested in, and see whether or not 00:30:32.760 --> 00:30:34.200 align:middle line:90% you think the-- 00:30:34.200 --> 00:30:35.950 align:middle line:84% good, going back to the coordinate system, 00:30:35.950 --> 00:30:40.290 align:middle line:84% like how would how would you use Minecraft 00:30:40.290 --> 00:30:41.710 align:middle line:90% to teach the coordinate system? 00:30:41.710 --> 00:30:44.730 align:middle line:84% And they've got all kinds of guiding activities and student 00:30:44.730 --> 00:30:45.460 align:middle line:90% activities. 00:30:45.460 --> 00:30:46.890 align:middle line:84% And you won't be able to see the world that's 00:30:46.890 --> 00:30:49.015 align:middle line:84% associated with them, but each one of these lessons 00:30:49.015 --> 00:30:51.193 align:middle line:84% has a kind of Minecraft space that's been built. 00:30:51.193 --> 00:30:53.110 align:middle line:84% You can at least get some screenshots from it. 00:30:53.110 --> 00:30:56.785 align:middle line:84% And then if anybody hasn't played Duolingo, it's not-- 00:30:56.785 --> 00:30:58.410 align:middle line:84% you could probably pretty quickly start 00:30:58.410 --> 00:31:01.358 align:middle line:84% creating an account and doing some things like that. 00:31:01.358 --> 00:31:03.400 align:middle line:84% If you want to do it with the person next to you, 00:31:03.400 --> 00:31:05.310 align:middle line:90% you should feel free to do that. 00:31:05.310 --> 00:31:08.230 align:middle line:84% The easiest place to find all of these links-- 00:31:08.230 --> 00:31:11.340 align:middle line:84% you could just Google them, but they are on the syllabus, 00:31:11.340 --> 00:31:13.870 align:middle line:90% bit.ly/lmtsyllabus. 00:31:13.870 --> 00:31:19.680 align:middle line:84% And then on the day for learning games, 00:31:19.680 --> 00:31:21.900 align:middle line:84% I've put four of these links there. 00:31:21.900 --> 00:31:23.770 align:middle line:90% So it's 1:35. 00:31:23.770 --> 00:31:26.238 align:middle line:84% Why don't we take until about 1:50 and just let you-- 00:31:26.238 --> 00:31:28.780 align:middle line:84% and feel free to do it with a partner or something like that. 00:31:28.780 --> 00:31:30.570 align:middle line:84% And if they've got sound, you can turn on the sound quietly 00:31:30.570 --> 00:31:31.403 align:middle line:90% and stuff like that. 00:31:31.403 --> 00:31:33.736 align:middle line:84% So you can hear it without bothering people next to you. 00:31:33.736 --> 00:31:35.820 align:middle line:84% But they're pretty fun things to mess around with. 00:31:35.820 --> 00:31:36.340 align:middle line:90% Sound good? 00:31:36.340 --> 00:31:37.235 align:middle line:90% Ready to go? 00:31:37.235 --> 00:31:41.970 align:middle line:90% 00:31:41.970 --> 00:31:44.940 align:middle line:84% So having-- who played around with Zoombinis? 00:31:44.940 --> 00:31:47.140 align:middle line:84% It looked like we had a lot of Zoombinis players. 00:31:47.140 --> 00:31:51.120 align:middle line:84% What are your reactions having spent some time with Zoombinis? 00:31:51.120 --> 00:31:52.890 align:middle line:90% AUDIENCE: It's cute. 00:31:52.890 --> 00:31:55.500 align:middle line:90% AUDIENCE: It takes so long. 00:31:55.500 --> 00:31:57.730 align:middle line:90% I want it to go. 00:31:57.730 --> 00:31:58.325 align:middle line:90% Hurry up. 00:31:58.325 --> 00:31:58.950 align:middle line:90% AUDIENCE: Yeah. 00:31:58.950 --> 00:32:01.540 align:middle line:84% Then you get to see them walk up and bounce up. 00:32:01.540 --> 00:32:03.010 align:middle line:90% JUSTIN REICH: Yeah, yeah. 00:32:03.010 --> 00:32:05.525 align:middle line:84% So the more actually, most of you 00:32:05.525 --> 00:32:06.900 align:middle line:84% are playing with sound off, which 00:32:06.900 --> 00:32:09.690 align:middle line:84% might be a mistake, because I think part 00:32:09.690 --> 00:32:12.060 align:middle line:84% of the stakes of Zoombinis are set-- like, 00:32:12.060 --> 00:32:14.040 align:middle line:84% there's a narrator in the back who tells you 00:32:14.040 --> 00:32:16.360 align:middle line:84% about how the Zoombinis were oppressed, 00:32:16.360 --> 00:32:18.640 align:middle line:84% and you're saving them, and things like that. 00:32:18.640 --> 00:32:24.330 align:middle line:84% And having you develop a kind of affinity for them 00:32:24.330 --> 00:32:27.290 align:middle line:84% is in a storytelling world as part of the game. 00:32:27.290 --> 00:32:29.040 align:middle line:84% Having them be cute, I think, is important 00:32:29.040 --> 00:32:30.990 align:middle line:84% that you develop some care for them 00:32:30.990 --> 00:32:34.240 align:middle line:84% and want to engage with them and things like that. 00:32:34.240 --> 00:32:38.370 align:middle line:84% In fact, the more you peel away those elements, 00:32:38.370 --> 00:32:40.900 align:middle line:84% the kind of narrative elements, the social elements, 00:32:40.900 --> 00:32:43.750 align:middle line:84% the artistic elements, and just get down to the math, 00:32:43.750 --> 00:32:48.640 align:middle line:84% the more that could risk some of the fun of the engagement. 00:32:48.640 --> 00:32:50.070 align:middle line:84% AUDIENCE: I'd rather play Sudoku. 00:32:50.070 --> 00:32:51.445 align:middle line:84% JUSTIN REICH: Rather play Sudoku. 00:32:51.445 --> 00:32:54.280 align:middle line:84% OK, good, which is just like a straight immersion into-- 00:32:54.280 --> 00:32:55.560 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:32:55.560 --> 00:32:58.590 align:middle line:84% JUSTIN REICH: And those kinds of personal preferences 00:32:58.590 --> 00:33:02.010 align:middle line:84% raise some real challenges for teachers, right? 00:33:02.010 --> 00:33:04.830 align:middle line:84% You can imagine having-- if you were to take time in a math 00:33:04.830 --> 00:33:06.600 align:middle line:84% class to have people play The Logical 00:33:06.600 --> 00:33:08.850 align:middle line:84% Journey of the Zoombinis, there would be some kids who 00:33:08.850 --> 00:33:10.810 align:middle line:84% are like, this is the best day of class ever, 00:33:10.810 --> 00:33:12.720 align:middle line:84% and there would be other kids who are like, 00:33:12.720 --> 00:33:16.050 align:middle line:84% can we just go back to doing whatever we were doing before? 00:33:16.050 --> 00:33:19.200 align:middle line:84% And it's like an expensive investment of time 00:33:19.200 --> 00:33:22.230 align:middle line:84% and other kinds of things to have a third of your class 00:33:22.230 --> 00:33:25.350 align:middle line:84% be like, ah this game is not that fun. 00:33:25.350 --> 00:33:27.768 align:middle line:84% Emily did it sell you eventually? 00:33:27.768 --> 00:33:29.060 align:middle line:90% AUDIENCE: No, it seemed boring. 00:33:29.060 --> 00:33:30.285 align:middle line:90% JUSTIN REICH: Yeah. 00:33:30.285 --> 00:33:31.410 align:middle line:90% AUDIENCE: I switched games. 00:33:31.410 --> 00:33:34.830 align:middle line:84% AUDIENCE: I would say I kind of like it, honestly. 00:33:34.830 --> 00:33:35.730 align:middle line:90% JUSTIN REICH: Good. 00:33:35.730 --> 00:33:38.475 align:middle line:90% AUDIENCE: I think it's just-- 00:33:38.475 --> 00:33:39.340 align:middle line:90% I don't know. 00:33:39.340 --> 00:33:43.090 align:middle line:84% It could be like that tendency, like you said, of create-- 00:33:43.090 --> 00:33:44.910 align:middle line:84% like having an affinity for them, 00:33:44.910 --> 00:33:48.120 align:middle line:84% or it's like these cute little creatures, going 00:33:48.120 --> 00:33:52.500 align:middle line:84% across the screen, and how much you enjoy that, 00:33:52.500 --> 00:33:54.100 align:middle line:84% or how much you might care for that. 00:33:54.100 --> 00:33:59.760 align:middle line:84% And I'm very much down for a fun little game with little cute-- 00:33:59.760 --> 00:34:01.590 align:middle line:84% JUSTIN REICH: Characters, and identities, 00:34:01.590 --> 00:34:02.592 align:middle line:90% and things like that. 00:34:02.592 --> 00:34:04.050 align:middle line:84% AUDIENCE: I feel like it might just 00:34:04.050 --> 00:34:08.460 align:middle line:84% be because now I'm so used to things having to happen very 00:34:08.460 --> 00:34:10.650 align:middle line:84% quickly, because even lectures, I 00:34:10.650 --> 00:34:14.050 align:middle line:84% tend to watch them in two times speed instead of the normal way, 00:34:14.050 --> 00:34:15.429 align:middle line:90% like the normal time. 00:34:15.429 --> 00:34:17.710 align:middle line:84% So then it's like watching-- playing these games, 00:34:17.710 --> 00:34:21.159 align:middle line:84% they're so slow that I'm just like, you've lost my attention. 00:34:21.159 --> 00:34:22.492 align:middle line:90% I don't want to play it anymore. 00:34:22.492 --> 00:34:23.284 align:middle line:90% JUSTIN REICH: Good. 00:34:23.284 --> 00:34:25.380 align:middle line:84% Yeah, there's a little kind of gap in there. 00:34:25.380 --> 00:34:26.547 align:middle line:90% AUDIENCE: I have a question. 00:34:26.547 --> 00:34:28.679 align:middle line:84% Did you guys skip the transitions, 00:34:28.679 --> 00:34:30.054 align:middle line:84% or were you sitting through them? 00:34:30.054 --> 00:34:31.012 align:middle line:90% AUDIENCE: Skipped them. 00:34:31.012 --> 00:34:32.370 align:middle line:90% AUDIENCE: I clicked out of them. 00:34:32.370 --> 00:34:33.578 align:middle line:90% AUDIENCE: Skipped them, yeah. 00:34:33.578 --> 00:34:34.530 align:middle line:90% AUDIENCE: OK. 00:34:34.530 --> 00:34:36.900 align:middle line:90% Because the transitions are-- 00:34:36.900 --> 00:34:39.429 align:middle line:84% I see what you mean, but the transitions are like, 00:34:39.429 --> 00:34:43.560 align:middle line:84% OK, you're slowly highlighting the next place I'm going, 00:34:43.560 --> 00:34:45.540 align:middle line:90% and have for a minute. 00:34:45.540 --> 00:34:47.550 align:middle line:84% And it's like, OK, I've seen enough. 00:34:47.550 --> 00:34:51.659 align:middle line:84% AUDIENCE: It was fun to figure out what the traits were 00:34:51.659 --> 00:34:52.900 align:middle line:90% that they were selecting for. 00:34:52.900 --> 00:34:54.195 align:middle line:90% But then you kind of-- 00:34:54.195 --> 00:34:56.025 align:middle line:84% I felt like I instantly found the traits, 00:34:56.025 --> 00:34:57.400 align:middle line:84% and then I was just like, oh, OK. 00:34:57.400 --> 00:35:02.100 align:middle line:84% I'll just-- I didn't really have to completely do 00:35:02.100 --> 00:35:05.310 align:middle line:84% a crazy search for the traits or whatever. 00:35:05.310 --> 00:35:07.975 align:middle line:84% I was just like, oh, OK, well, maybe this hair is good. 00:35:07.975 --> 00:35:09.850 align:middle line:84% And then all that hair turned out to be fine. 00:35:09.850 --> 00:35:12.000 align:middle line:90% I'm like, oh, OK, great. 00:35:12.000 --> 00:35:14.730 align:middle line:84% JUSTIN REICH: I mean, it might have been harder if you were 10. 00:35:14.730 --> 00:35:16.938 align:middle line:84% AUDIENCE: Actually, you can build those two together. 00:35:16.938 --> 00:35:18.780 align:middle line:84% I'm like-- I don't remember when I 00:35:18.780 --> 00:35:21.130 align:middle line:84% was going through this game as a young person 00:35:21.130 --> 00:35:23.760 align:middle line:84% if I recognized or was even looking for the traits. 00:35:23.760 --> 00:35:26.880 align:middle line:84% I think maybe I might have been more pulled 00:35:26.880 --> 00:35:29.530 align:middle line:84% to the affinity to the characters, and like, oh, no, 00:35:29.530 --> 00:35:30.453 align:middle line:90% they got bounced back. 00:35:30.453 --> 00:35:31.120 align:middle line:90% That's not good. 00:35:31.120 --> 00:35:33.078 align:middle line:84% And just doing-- I would do it with my sisters, 00:35:33.078 --> 00:35:34.920 align:middle line:84% and we'd just drop them all over the place 00:35:34.920 --> 00:35:35.920 align:middle line:90% and see if they'd go in. 00:35:35.920 --> 00:35:38.730 align:middle line:84% But I cannot remember if I noticed that there were 00:35:38.730 --> 00:35:39.372 align:middle line:90% conditions. 00:35:39.372 --> 00:35:41.580 align:middle line:84% And I just-- we cared-- we were OK with the slowness, 00:35:41.580 --> 00:35:43.690 align:middle line:84% because we cared more about just getting them through. 00:35:43.690 --> 00:35:44.470 align:middle line:90% It didn't matter. 00:35:44.470 --> 00:35:45.820 align:middle line:90% And I can't remember. 00:35:45.820 --> 00:35:47.760 align:middle line:84% And that's killing me a little bit. 00:35:47.760 --> 00:35:50.760 align:middle line:84% JUSTIN REICH: How many of you clicked the question mark hint 00:35:50.760 --> 00:35:52.710 align:middle line:84% button that has an explanation and how many 00:35:52.710 --> 00:35:54.480 align:middle line:90% didn't discover the-- 00:35:54.480 --> 00:35:57.390 align:middle line:84% it's kind of an interesting design feature, 00:35:57.390 --> 00:36:02.018 align:middle line:84% to be able to say that you can play 00:36:02.018 --> 00:36:03.810 align:middle line:84% the game without any instructions about how 00:36:03.810 --> 00:36:04.450 align:middle line:90% to play the game. 00:36:04.450 --> 00:36:06.730 align:middle line:84% You're just put in the world, start clicking things around, 00:36:06.730 --> 00:36:07.855 align:middle line:90% start moving things around. 00:36:07.855 --> 00:36:11.610 align:middle line:84% If you had to put The Logical Journey of the Zoombinis 00:36:11.610 --> 00:36:13.750 align:middle line:84% as instructor-guided learning at scale, 00:36:13.750 --> 00:36:15.580 align:middle line:84% algorithm-guided learning at scale, 00:36:15.580 --> 00:36:18.620 align:middle line:84% or peer-guided learning at scale, what would you call it? 00:36:18.620 --> 00:36:22.380 align:middle line:90% 00:36:22.380 --> 00:36:25.020 align:middle line:84% AUDIENCE: I feel like instructor-guided learning. 00:36:25.020 --> 00:36:28.620 align:middle line:84% It's not like if I was able to solve all of the-- 00:36:28.620 --> 00:36:32.800 align:middle line:84% or get all of the Zoombinis through one or two errors, 00:36:32.800 --> 00:36:35.790 align:middle line:84% that it would automatically jump me to level 5. 00:36:35.790 --> 00:36:37.770 align:middle line:84% There was very much a sequence that you 00:36:37.770 --> 00:36:39.690 align:middle line:90% were supposed to follow. 00:36:39.690 --> 00:36:43.200 align:middle line:84% And when I would place one, and then it would try and go up, 00:36:43.200 --> 00:36:45.450 align:middle line:84% it almost felt like an autograder of it's 00:36:45.450 --> 00:36:47.902 align:middle line:84% like this is either pass or fail. 00:36:47.902 --> 00:36:50.110 align:middle line:84% JUSTIN REICH: Yeah, the autograder is a lot more fun, 00:36:50.110 --> 00:36:52.700 align:middle line:84% because a thing crosses a bridge, or gets kicked back, 00:36:52.700 --> 00:36:54.380 align:middle line:84% or somebody eats a pizza, but doesn't eat a pizza. 00:36:54.380 --> 00:36:56.547 align:middle line:84% But it's not really any different from a green check 00:36:56.547 --> 00:36:59.200 align:middle line:84% or a red check or other kinds of things like that. 00:36:59.200 --> 00:37:02.945 align:middle line:84% Now, the way that I argued that Zoombinis fit 00:37:02.945 --> 00:37:04.570 align:middle line:84% is that it's instructor-guided learning 00:37:04.570 --> 00:37:05.870 align:middle line:90% at scale with an autograder. 00:37:05.870 --> 00:37:08.560 align:middle line:84% However, almost every MOOC is inspired 00:37:08.560 --> 00:37:11.600 align:middle line:84% by more instructive kinds of approaches, 00:37:11.600 --> 00:37:14.440 align:middle line:84% and this is very much more inspired by kind of a situated 00:37:14.440 --> 00:37:16.570 align:middle line:84% learning apprenticeship kind of approach, 00:37:16.570 --> 00:37:19.460 align:middle line:84% where it's almost trying to hide the autograder, 00:37:19.460 --> 00:37:21.430 align:middle line:90% trying to hide the instructor. 00:37:21.430 --> 00:37:23.960 align:middle line:84% I mean, it's very much trying to hide the instructions. 00:37:23.960 --> 00:37:25.880 align:middle line:84% It says just like, go into this world, 00:37:25.880 --> 00:37:27.380 align:middle line:84% immerse yourself into this world, 00:37:27.380 --> 00:37:29.300 align:middle line:90% play around, see what happens. 00:37:29.300 --> 00:37:31.160 align:middle line:84% There's sort of winning and losing, 00:37:31.160 --> 00:37:34.160 align:middle line:84% but it's actually fine to fail a level. 00:37:34.160 --> 00:37:36.290 align:middle line:84% It's fine to have all your Zoombinis kicked back. 00:37:36.290 --> 00:37:38.750 align:middle line:84% You can try again and rescue some other Zoombinis, 00:37:38.750 --> 00:37:40.390 align:middle line:90% other sorts of things. 00:37:40.390 --> 00:37:43.210 align:middle line:84% There's no point in the game where 00:37:43.210 --> 00:37:46.240 align:middle line:84% it stops and is like, OK, kids, actually, 00:37:46.240 --> 00:37:49.613 align:middle line:84% we're teaching you combinatorics, or probability, 00:37:49.613 --> 00:37:52.030 align:middle line:84% or whatever else you think is the mathematics of the game. 00:37:52.030 --> 00:37:55.410 align:middle line:84% It just lets that be a kind of natural learning 00:37:55.410 --> 00:37:58.140 align:middle line:90% that happens throughout. 00:37:58.140 --> 00:38:03.540 align:middle line:84% There are other-- so this is an exercise 00:38:03.540 --> 00:38:06.390 align:middle line:84% that I think is the kind of thing that might be helpful 00:38:06.390 --> 00:38:11.250 align:middle line:84% if you were a department head, a teacher trying to figure out 00:38:11.250 --> 00:38:13.290 align:middle line:84% whether or not you should adopt this technology 00:38:13.290 --> 00:38:16.920 align:middle line:84% in your classroom, which is, for any new technology, three 00:38:16.920 --> 00:38:19.020 align:middle line:84% good questions to ask are, who drives and controls 00:38:19.020 --> 00:38:20.498 align:middle line:90% the pace and path of learning? 00:38:20.498 --> 00:38:21.790 align:middle line:90% What are the technologies here? 00:38:21.790 --> 00:38:22.790 align:middle line:90% What are the pedagogies? 00:38:22.790 --> 00:38:24.660 align:middle line:84% And then, what can I predict will 00:38:24.660 --> 00:38:26.850 align:middle line:84% happen in my educational system, given 00:38:26.850 --> 00:38:28.210 align:middle line:90% a consequence of this design? 00:38:28.210 --> 00:38:32.350 align:middle line:84% So Math Blaster is pretty instructor-guided. 00:38:32.350 --> 00:38:33.353 align:middle line:90% It's got an autograder. 00:38:33.353 --> 00:38:35.020 align:middle line:84% I mean, they're just worksheet problems. 00:38:35.020 --> 00:38:38.770 align:middle line:84% There's no difference between that and a worksheet problem, 00:38:38.770 --> 00:38:41.760 align:middle line:84% except maybe I will slightly motivate some of my students 00:38:41.760 --> 00:38:44.868 align:middle line:84% to do more worksheet problems because there's aliens, 00:38:44.868 --> 00:38:46.660 align:middle line:84% and it's kind of fun, and things like that. 00:38:46.660 --> 00:38:47.230 align:middle line:90% But maybe not. 00:38:47.230 --> 00:38:48.640 align:middle line:84% Maybe there's some people who are like, 00:38:48.640 --> 00:38:49.848 align:middle line:90% I really don't want to have-- 00:38:49.848 --> 00:38:54.210 align:middle line:84% just give me the math problems to answer. 00:38:54.210 --> 00:38:57.063 align:middle line:84% I had often thought of Duo-- how many of you 00:38:57.063 --> 00:38:58.980 align:middle line:84% played around with Duolingo for the first time 00:38:58.980 --> 00:39:00.355 align:middle line:84% as you were clicking around here? 00:39:00.355 --> 00:39:03.420 align:middle line:90% Any sort of new Duolingo people? 00:39:03.420 --> 00:39:06.030 align:middle line:84% It certainly is algorithm-guided, 00:39:06.030 --> 00:39:09.040 align:middle line:84% as it's trying to-- there's a lot of good writing. 00:39:09.040 --> 00:39:12.660 align:middle line:84% The developers of Duolingo keep blogs and other kinds of things 00:39:12.660 --> 00:39:16.805 align:middle line:84% where they talk about, for instance, they have-- 00:39:16.805 --> 00:39:18.180 align:middle line:84% anybody ever heard of the concept 00:39:18.180 --> 00:39:20.140 align:middle line:84% of the zone of proximal development? 00:39:20.140 --> 00:39:21.947 align:middle line:84% It's sort of a classic educational concept. 00:39:21.947 --> 00:39:24.280 align:middle line:84% Basically, you want to try to as you're teaching people, 00:39:24.280 --> 00:39:28.020 align:middle line:84% have there be a difficulty level where you feel competent, 00:39:28.020 --> 00:39:29.680 align:middle line:90% but also stretched. 00:39:29.680 --> 00:39:31.390 align:middle line:84% If things are too easy, it's boring. 00:39:31.390 --> 00:39:32.890 align:middle line:84% If things are too hard, it's boring. 00:39:32.890 --> 00:39:36.580 align:middle line:84% There's a zone of proximal ideal development for that. 00:39:36.580 --> 00:39:39.510 align:middle line:84% They measure that zone of proximal development by saying, 00:39:39.510 --> 00:39:41.430 align:middle line:84% we're trying to aim for people to have 00:39:41.430 --> 00:39:42.730 align:middle line:90% a certain kind of error rate. 00:39:42.730 --> 00:39:43.620 align:middle line:84% I don't know exactly what it is, but it 00:39:43.620 --> 00:39:45.995 align:middle line:84% might be like, we want you to get about 20% of the things 00:39:45.995 --> 00:39:46.900 align:middle line:90% you're hitting wrong. 00:39:46.900 --> 00:39:48.710 align:middle line:84% If you're getting more than 20% wrong, 00:39:48.710 --> 00:39:50.360 align:middle line:84% then we're probably making it too hard. 00:39:50.360 --> 00:39:52.420 align:middle line:90% Otherwise, too easy. 00:39:52.420 --> 00:39:53.920 align:middle line:90% But what are the-- 00:39:53.920 --> 00:39:54.462 align:middle line:90% I don't know. 00:39:54.462 --> 00:39:55.962 align:middle line:84% We talked a little bit, but what are 00:39:55.962 --> 00:39:58.190 align:middle line:84% some of the situated learning elements that you saw? 00:39:58.190 --> 00:40:00.398 align:middle line:84% If you were to say that it was not so instructionist, 00:40:00.398 --> 00:40:03.758 align:middle line:84% but it was more inspired by kind of constructionism, 00:40:03.758 --> 00:40:05.800 align:middle line:84% situated learning, what kinds of things like that 00:40:05.800 --> 00:40:09.400 align:middle line:90% are in Duolingo? 00:40:09.400 --> 00:40:13.780 align:middle line:84% AUDIENCE: I found out Duolingo has a music thing now, which 00:40:13.780 --> 00:40:16.400 align:middle line:84% is really cool, because I've always wanted to learn music. 00:40:16.400 --> 00:40:19.370 align:middle line:84% I'd like to play around with that, but I don't understand it. 00:40:19.370 --> 00:40:21.470 align:middle line:90% I can't read notes or anything. 00:40:21.470 --> 00:40:23.990 align:middle line:84% I just think, and then play piano. 00:40:23.990 --> 00:40:28.990 align:middle line:84% But it's more interesting to see they have this stupid simulation 00:40:28.990 --> 00:40:31.660 align:middle line:84% of playing "Hot Cross Buns," and I swear I could play that if I 00:40:31.660 --> 00:40:34.670 align:middle line:84% tried, but doing it with their little app, 00:40:34.670 --> 00:40:37.780 align:middle line:84% it was like you had to pace it, and you had to recognize 00:40:37.780 --> 00:40:38.290 align:middle line:90% the notes. 00:40:38.290 --> 00:40:40.280 align:middle line:84% And it went back to that 10 times. 00:40:40.280 --> 00:40:42.020 align:middle line:84% And it's simulating playing the piano. 00:40:42.020 --> 00:40:44.500 align:middle line:84% It's really simple for a computer game, 00:40:44.500 --> 00:40:46.942 align:middle line:90% but that was pretty engaging. 00:40:46.942 --> 00:40:48.900 align:middle line:84% JUSTIN REICH: So it immersed you in the feeling 00:40:48.900 --> 00:40:51.850 align:middle line:84% of playing a piano, even though you're kind of on your phone. 00:40:51.850 --> 00:40:54.390 align:middle line:84% That seems like a good quality of a situated apprenticeship 00:40:54.390 --> 00:40:56.070 align:middle line:90% kind of thing. 00:40:56.070 --> 00:41:00.060 align:middle line:84% Now, you had played "Hot Cross Buns" before. 00:41:00.060 --> 00:41:03.180 align:middle line:84% AUDIENCE: I feel like you could just play it sitting at a piano 00:41:03.180 --> 00:41:03.990 align:middle line:90% without-- 00:41:03.990 --> 00:41:07.980 align:middle line:84% if I looked for the notes sound-wise, like, 00:41:07.980 --> 00:41:11.670 align:middle line:84% I can't read piano notes, but I can figure out songs 00:41:11.670 --> 00:41:15.082 align:middle line:84% if you tell me what they are, like if I hear them. 00:41:15.082 --> 00:41:16.290 align:middle line:90% JUSTIN REICH: Playing by ear. 00:41:16.290 --> 00:41:17.715 align:middle line:90% Yeah, cool. 00:41:17.715 --> 00:41:20.560 align:middle line:84% And so you feel like this could be a pathway to doing-- 00:41:20.560 --> 00:41:24.390 align:middle line:84% I mean, I'm sort of increasingly convinced now that Duolingo-- 00:41:24.390 --> 00:41:26.520 align:middle line:84% Duolingo came out of Carnegie Mellon University, 00:41:26.520 --> 00:41:28.450 align:middle line:84% and they have some of the best learning scientists, 00:41:28.450 --> 00:41:29.500 align:middle line:90% learning engineers in the world. 00:41:29.500 --> 00:41:31.125 align:middle line:84% And one of the things I like about them 00:41:31.125 --> 00:41:33.530 align:middle line:84% is that they're methodological pluralists. 00:41:33.530 --> 00:41:35.280 align:middle line:84% They're like, what are all the good ideas, 00:41:35.280 --> 00:41:37.350 align:middle line:84% and how do we shove them all in there? 00:41:37.350 --> 00:41:41.092 align:middle line:84% So in some sense, Duolingo is very much constructionist. 00:41:41.092 --> 00:41:43.050 align:middle line:84% It's taking big, complex things and breaking it 00:41:43.050 --> 00:41:44.170 align:middle line:90% down into small pieces. 00:41:44.170 --> 00:41:46.580 align:middle line:84% It'll show you examples, then give you practice 00:41:46.580 --> 00:41:47.830 align:middle line:90% and all those kinds of things. 00:41:47.830 --> 00:41:49.600 align:middle line:84% But it also has this way of, well, 00:41:49.600 --> 00:41:51.558 align:middle line:84% let's give you an identity in this environment. 00:41:51.558 --> 00:41:53.475 align:middle line:84% Let's have you build a parasocial relationship 00:41:53.475 --> 00:41:54.120 align:middle line:90% with an owl. 00:41:54.120 --> 00:41:56.162 align:middle line:84% Let's feel like you're sitting down with a piano. 00:41:56.162 --> 00:41:57.092 align:middle line:90% Let's reward you. 00:41:57.092 --> 00:41:58.800 align:middle line:84% Let's help you go on a journey from being 00:41:58.800 --> 00:42:00.810 align:middle line:84% a newcomer to an old timer by having 00:42:00.810 --> 00:42:03.120 align:middle line:84% badges and streaks and these other sorts of things 00:42:03.120 --> 00:42:07.430 align:middle line:84% with it, kind of a nice blend of those things. 00:42:07.430 --> 00:42:10.973 align:middle line:90% 00:42:10.973 --> 00:42:12.390 align:middle line:84% We've talked a bunch in this class 00:42:12.390 --> 00:42:14.490 align:middle line:84% about how one of the signature challenges 00:42:14.490 --> 00:42:17.430 align:middle line:84% of self-paced online learning is that people are typically 00:42:17.430 --> 00:42:17.950 align:middle line:90% bad at it. 00:42:17.950 --> 00:42:19.830 align:middle line:90% People typically quit. 00:42:19.830 --> 00:42:22.840 align:middle line:84% Duolingo has probably been as successful, 00:42:22.840 --> 00:42:25.590 align:middle line:84% maybe more successful, than any other education technology 00:42:25.590 --> 00:42:28.500 align:middle line:84% company in building a product that's 00:42:28.500 --> 00:42:30.540 align:middle line:90% just super sticky for people. 00:42:30.540 --> 00:42:33.150 align:middle line:84% They really have put a lot of effort 00:42:33.150 --> 00:42:35.790 align:middle line:84% into trying to break down the challenges 00:42:35.790 --> 00:42:40.140 align:middle line:84% of self-paced learning, through these gamification, 00:42:40.140 --> 00:42:42.120 align:middle line:84% through streaks, through making it playful, 00:42:42.120 --> 00:42:44.070 align:middle line:84% through having it feel like it'd be fun. 00:42:44.070 --> 00:42:44.720 align:middle line:90% I don't know. 00:42:44.720 --> 00:42:48.110 align:middle line:84% Have any of you tried to quit Duolingo? 00:42:48.110 --> 00:42:51.235 align:middle line:84% They send you haranguing emails when you're not doing it enough, 00:42:51.235 --> 00:42:52.110 align:middle line:90% and things like that. 00:42:52.110 --> 00:42:55.420 align:middle line:84% They've got all kinds of ways of trying to continuously pull them 00:42:55.420 --> 00:42:55.920 align:middle line:90% in there. 00:42:55.920 --> 00:42:58.045 align:middle line:84% They're super scientific about all this stuff, too. 00:42:58.045 --> 00:43:00.440 align:middle line:84% So I'm sure all those emails that you've ever gotten 00:43:00.440 --> 00:43:04.190 align:middle line:84% have been A-B tested a zillion times, all different kinds 00:43:04.190 --> 00:43:06.890 align:middle line:84% of timing, and language they use, 00:43:06.890 --> 00:43:10.890 align:middle line:84% and other things like that to rope you back in. 00:43:10.890 --> 00:43:11.390 align:middle line:90% Good. 00:43:11.390 --> 00:43:14.670 align:middle line:84% How many of you looked around on the Minecraft website? 00:43:14.670 --> 00:43:16.790 align:middle line:90% What did you find there, Khalil? 00:43:16.790 --> 00:43:21.110 align:middle line:84% AUDIENCE: I played a game where they're trying to teach kids 00:43:21.110 --> 00:43:23.760 align:middle line:90% good game, like, stuff. 00:43:23.760 --> 00:43:28.790 align:middle line:90% And it's still very-- 00:43:28.790 --> 00:43:30.420 align:middle line:90% there's not much to do. 00:43:30.420 --> 00:43:31.890 align:middle line:90% It's just like, go there. 00:43:31.890 --> 00:43:33.000 align:middle line:90% Do that. 00:43:33.000 --> 00:43:35.180 align:middle line:90% Should you feed the dog? 00:43:35.180 --> 00:43:37.566 align:middle line:84% You probably should feed the dog. 00:43:37.566 --> 00:43:38.495 align:middle line:90% [LAUGHTER] 00:43:38.495 --> 00:43:42.380 align:middle line:84% So it felt a little bit forced in terms of the path 00:43:42.380 --> 00:43:43.650 align:middle line:90% that you could take. 00:43:43.650 --> 00:43:47.930 align:middle line:84% But after doing these chores, it tells you, hey, you 00:43:47.930 --> 00:43:51.950 align:middle line:84% have to write this good game guide book. 00:43:51.950 --> 00:43:54.080 align:middle line:90% And, oh, what's the-- 00:43:54.080 --> 00:43:56.430 align:middle line:84% you should write a certain instruction. 00:43:56.430 --> 00:43:57.780 align:middle line:90% Do you want an example? 00:43:57.780 --> 00:44:00.720 align:middle line:84% Oh, what you did today was chores. 00:44:00.720 --> 00:44:03.330 align:middle line:84% You should do your chores before playing games. 00:44:03.330 --> 00:44:05.510 align:middle line:84% And then you get to write whatever 00:44:05.510 --> 00:44:07.970 align:middle line:84% you want, in which case, they don't really 00:44:07.970 --> 00:44:10.110 align:middle line:90% check what you write in there. 00:44:10.110 --> 00:44:13.520 align:middle line:84% But after that, you get to play a little game where 00:44:13.520 --> 00:44:15.540 align:middle line:90% you shoot balloons, I guess. 00:44:15.540 --> 00:44:18.990 align:middle line:84% And then you're back into a new day. 00:44:18.990 --> 00:44:25.290 align:middle line:84% So it felt a lot of let's simulate life in Minecraft. 00:44:25.290 --> 00:44:30.080 align:middle line:84% You have even-- you have parents in the game, which is they 00:44:30.080 --> 00:44:32.160 align:middle line:90% named them Trusted Adult. 00:44:32.160 --> 00:44:34.100 align:middle line:90% [LAUGHTER] 00:44:34.100 --> 00:44:38.870 align:middle line:84% But yeah, it felt a little bit too much constrained 00:44:38.870 --> 00:44:41.640 align:middle line:84% in terms of what activity you could be doing. 00:44:41.640 --> 00:44:49.350 align:middle line:84% And it felt very much clearly instructivist. 00:44:49.350 --> 00:44:51.892 align:middle line:84% I mean, I'm very much an adult. So it's not like-- 00:44:51.892 --> 00:44:53.767 align:middle line:84% JUSTIN REICH: You're not the target audience. 00:44:53.767 --> 00:44:53.905 align:middle line:90% But-- 00:44:53.905 --> 00:44:55.270 align:middle line:90% AUDIENCE: [INAUDIBLE] for this. 00:44:55.270 --> 00:44:58.700 align:middle line:84% But I hypothesized a kid would recognize 00:44:58.700 --> 00:45:01.842 align:middle line:84% that they are telling me to do my chores before playing games. 00:45:01.842 --> 00:45:04.050 align:middle line:84% JUSTIN REICH: But they're just doing it in Minecraft. 00:45:04.050 --> 00:45:04.580 align:middle line:90% Yeah. 00:45:04.580 --> 00:45:04.980 align:middle line:90% AUDIENCE: Yeah. 00:45:04.980 --> 00:45:05.780 align:middle line:84% JUSTIN REICH: Yeah, I think that's-- 00:45:05.780 --> 00:45:08.540 align:middle line:84% I mean, to me-- how many of you are Minecraft players just 00:45:08.540 --> 00:45:11.640 align:middle line:84% in your regular lives or were at some point in you-- to me, 00:45:11.640 --> 00:45:14.240 align:middle line:84% the joy of Minecraft, the thing that they discovered 00:45:14.240 --> 00:45:19.400 align:middle line:84% was the joy of the boundlessness of just people inventing 00:45:19.400 --> 00:45:22.427 align:middle line:84% their own missions, and structure, and things like that. 00:45:22.427 --> 00:45:24.510 align:middle line:84% There's a little bit of the Overworld of the game, 00:45:24.510 --> 00:45:26.400 align:middle line:84% but I mean, for me, it was always like, all right, 00:45:26.400 --> 00:45:27.120 align:middle line:90% I'm in a new world. 00:45:27.120 --> 00:45:28.328 align:middle line:90% I will need a wizard's tower. 00:45:28.328 --> 00:45:32.060 align:middle line:84% I had a construct in my mind of what I should live in. 00:45:32.060 --> 00:45:34.160 align:middle line:90% And it was a wizard's tower. 00:45:34.160 --> 00:45:39.050 align:middle line:90% And that just cuts off-- 00:45:39.050 --> 00:45:42.110 align:middle line:84% it's very difficult to conduct educational activities which 00:45:42.110 --> 00:45:44.370 align:middle line:84% are like, everyone, go do whatever you want. 00:45:44.370 --> 00:45:46.828 align:middle line:84% I mean, usually, when you have some instructional activity, 00:45:46.828 --> 00:45:48.950 align:middle line:84% you're like, let's all do x, you know? 00:45:48.950 --> 00:45:51.410 align:middle line:84% And so, to some extent, the kind of, let's all 00:45:51.410 --> 00:45:55.310 align:middle line:84% do go do x is directly against the fabric of one 00:45:55.310 --> 00:45:58.970 align:middle line:84% of the main things that makes Minecraft so special 00:45:58.970 --> 00:46:01.800 align:middle line:84% is this open world where you can do anything you want. 00:46:01.800 --> 00:46:03.536 align:middle line:90% So there's a sort of-- 00:46:03.536 --> 00:46:07.670 align:middle line:84% those kinds of tensions show up all the time in-- 00:46:07.670 --> 00:46:09.510 align:middle line:90% I think it could-- 00:46:09.510 --> 00:46:13.080 align:middle line:84% I think Minecraft is incredibly sticky to young people. 00:46:13.080 --> 00:46:14.695 align:middle line:90% It is incredibly compelling. 00:46:14.695 --> 00:46:16.070 align:middle line:84% And so you go, oh, maybe we could 00:46:16.070 --> 00:46:17.930 align:middle line:84% take this really sticky, compelling world 00:46:17.930 --> 00:46:20.220 align:middle line:84% and have people do learning in it. 00:46:20.220 --> 00:46:22.850 align:middle line:84% But as soon as you add this dimension of, OK, you all 00:46:22.850 --> 00:46:25.670 align:middle line:84% have to do x inside my Minecraft world, 00:46:25.670 --> 00:46:28.440 align:middle line:84% it's often much, much less compelling for people. 00:46:28.440 --> 00:46:31.820 align:middle line:84% You're taking away one of the key features that made 00:46:31.820 --> 00:46:34.190 align:middle line:90% it sticky in the first place. 00:46:34.190 --> 00:46:35.940 align:middle line:84% But even if you wanted to do that, 00:46:35.940 --> 00:46:38.150 align:middle line:84% I think one of the central questions 00:46:38.150 --> 00:46:45.450 align:middle line:84% that games pose is, if you spend a lot of time in Minecraft, 00:46:45.450 --> 00:46:47.930 align:middle line:84% are there things that you're learning 00:46:47.930 --> 00:46:50.130 align:middle line:84% that are useful for other things, 00:46:50.130 --> 00:46:53.420 align:middle line:84% or are you just learning how to play Minecraft? 00:46:53.420 --> 00:46:56.323 align:middle line:84% And that is a very old educational question. 00:46:56.323 --> 00:46:58.490 align:middle line:84% It is an educational question that actually somebody 00:46:58.490 --> 00:47:00.680 align:middle line:84% we've talked about a bunch, Edward Thorndike, 00:47:00.680 --> 00:47:03.590 align:middle line:84% was one of the very earliest researchers in. 00:47:03.590 --> 00:47:06.860 align:middle line:84% And it relates to a problem that's called transfer. 00:47:06.860 --> 00:47:09.390 align:middle line:84% To what extent, when you learn something, 00:47:09.390 --> 00:47:11.570 align:middle line:84% can you take that learning and apply it 00:47:11.570 --> 00:47:14.008 align:middle line:84% into some other different kind of context? 00:47:14.008 --> 00:47:15.800 align:middle line:84% That is one of the most important questions 00:47:15.800 --> 00:47:17.840 align:middle line:90% we could ask in education. 00:47:17.840 --> 00:47:21.350 align:middle line:84% The more you believe that things learned in one context 00:47:21.350 --> 00:47:25.610 align:middle line:84% will help you do other kinds of things, in some ways, 00:47:25.610 --> 00:47:27.243 align:middle line:84% it simplifies the education system, 00:47:27.243 --> 00:47:29.660 align:middle line:84% like, let's just get you to be really good thinkers in one 00:47:29.660 --> 00:47:31.910 align:middle line:84% domain, and then you'll be good at all kinds of stuff. 00:47:31.910 --> 00:47:34.993 align:middle line:84% The more that you believe that things don't transfer very well, 00:47:34.993 --> 00:47:37.160 align:middle line:84% that people are really only good at doing the things 00:47:37.160 --> 00:47:39.920 align:middle line:84% that they're taught how to do, the more difficult and complex 00:47:39.920 --> 00:47:40.880 align:middle line:90% education becomes. 00:47:40.880 --> 00:47:46.610 align:middle line:84% In Thorndike's time, the thorny challenge that he was faced with 00:47:46.610 --> 00:47:50.340 align:middle line:84% was that people were still big fans of classical curriculum, 00:47:50.340 --> 00:47:52.290 align:middle line:90% teaching people Latin and Greek. 00:47:52.290 --> 00:47:54.710 align:middle line:84% And the argument that was out there 00:47:54.710 --> 00:47:56.840 align:middle line:84% was that if people got really good 00:47:56.840 --> 00:48:00.620 align:middle line:84% at learning Latin and Greek, they would learn, generically, 00:48:00.620 --> 00:48:01.870 align:middle line:90% how to think well. 00:48:01.870 --> 00:48:04.370 align:middle line:84% They would learn how to think well, how to communicate well. 00:48:04.370 --> 00:48:05.460 align:middle line:90% You all are giggling at this. 00:48:05.460 --> 00:48:06.668 align:middle line:90% Why are you giggling at this? 00:48:06.668 --> 00:48:12.950 align:middle line:90% 00:48:12.950 --> 00:48:14.620 align:middle line:84% What seems silly about that argument? 00:48:14.620 --> 00:48:20.900 align:middle line:90% 00:48:20.900 --> 00:48:23.600 align:middle line:84% AUDIENCE: I mean, learning Latin and Greek is-- 00:48:23.600 --> 00:48:25.820 align:middle line:84% first of all, it seems kind of elitist 00:48:25.820 --> 00:48:27.480 align:middle line:90% and hilarious for that reason. 00:48:27.480 --> 00:48:30.830 align:middle line:84% But also, it's like any language, 00:48:30.830 --> 00:48:33.090 align:middle line:84% or I guess, any [INAUDIBLE] task, 00:48:33.090 --> 00:48:35.420 align:middle line:84% it's a lot of pattern matching, I imagine. 00:48:35.420 --> 00:48:36.960 align:middle line:84% I'm not a great language learner, 00:48:36.960 --> 00:48:39.590 align:middle line:84% but I can't imagine how you could 00:48:39.590 --> 00:48:42.830 align:middle line:90% apply the same underlying-- 00:48:42.830 --> 00:48:44.910 align:middle line:84% JUSTIN REICH: If I just learned a lot of Polish, 00:48:44.910 --> 00:48:48.328 align:middle line:84% I would be really good at lots of other kinds of things. 00:48:48.328 --> 00:48:49.620 align:middle line:90% It doesn't stick quite as well. 00:48:49.620 --> 00:48:50.480 align:middle line:90% Good. 00:48:50.480 --> 00:48:54.020 align:middle line:84% What are things in our current educational system that we tell 00:48:54.020 --> 00:48:55.973 align:middle line:84% people, "if you learn this thing, 00:48:55.973 --> 00:48:58.015 align:middle line:84% you'll be good at lots of other kinds of things"? 00:48:58.015 --> 00:49:01.040 align:middle line:90% 00:49:01.040 --> 00:49:03.390 align:middle line:84% AUDIENCE: Math, especially higher level stuff. 00:49:03.390 --> 00:49:04.712 align:middle line:90% JUSTIN REICH: OK, good. 00:49:04.712 --> 00:49:06.170 align:middle line:84% And can people think of other kinds 00:49:06.170 --> 00:49:08.337 align:middle line:84% of things that have claims that are similar to that? 00:49:08.337 --> 00:49:11.780 align:middle line:90% 00:49:11.780 --> 00:49:13.370 align:middle line:84% AUDIENCE: Just reading in general. 00:49:13.370 --> 00:49:15.150 align:middle line:84% People claim like, oh, if you can read, 00:49:15.150 --> 00:49:17.570 align:middle line:84% then you can read anything, and you can learn it. 00:49:17.570 --> 00:49:20.110 align:middle line:84% JUSTIN REICH: So there's domain-independent reading 00:49:20.110 --> 00:49:20.610 align:middle line:90% skills. 00:49:20.610 --> 00:49:22.068 align:middle line:84% That's a thing that comes out there 00:49:22.068 --> 00:49:24.740 align:middle line:90% that we can ask some more about. 00:49:24.740 --> 00:49:28.490 align:middle line:84% AUDIENCE: I could argue that many academics 00:49:28.490 --> 00:49:32.600 align:middle line:84% stuff not relating to academic stuff gets that claim. 00:49:32.600 --> 00:49:36.470 align:middle line:84% If you do this good academic achieve whatever, like, 00:49:36.470 --> 00:49:41.100 align:middle line:84% learning, you will do good at non-academic certain tasks, 00:49:41.100 --> 00:49:44.210 align:middle line:84% whether that be normal day-to-day getting 00:49:44.210 --> 00:49:45.630 align:middle line:90% a job, or something like that. 00:49:45.630 --> 00:49:47.030 align:middle line:84% JUSTIN REICH: Yeah, yep, yeah, that schools help you 00:49:47.030 --> 00:49:47.990 align:middle line:90% with other things. 00:49:47.990 --> 00:49:49.650 align:middle line:90% Chess gets this a lot. 00:49:49.650 --> 00:49:52.490 align:middle line:84% Learn to play chess, you get strategic thinking. 00:49:52.490 --> 00:49:55.310 align:middle line:84% AUDIENCE: If you take classes, you will be good at the job 00:49:55.310 --> 00:49:56.520 align:middle line:90% that they're teaching you. 00:49:56.520 --> 00:49:57.650 align:middle line:90% JUSTIN REICH: Yeah. 00:49:57.650 --> 00:50:03.450 align:middle line:84% My wife works on the faculty here in material science. 00:50:03.450 --> 00:50:06.500 align:middle line:84% And she had somebody who did material science 00:50:06.500 --> 00:50:09.230 align:middle line:84% in an engineering company once come, 00:50:09.230 --> 00:50:11.660 align:middle line:84% and came to a faculty meeting, and basically 00:50:11.660 --> 00:50:14.000 align:middle line:84% was telling the faculty how little she 00:50:14.000 --> 00:50:17.520 align:middle line:84% thought of the training that they got in labs. 00:50:17.520 --> 00:50:19.740 align:middle line:84% She was like, well, I guess it's OK if you do a UROP. 00:50:19.740 --> 00:50:21.290 align:middle line:84% It's better than doing lifeguarding 00:50:21.290 --> 00:50:22.350 align:middle line:84% for a summer, or something like that, 00:50:22.350 --> 00:50:23.733 align:middle line:84% but it's really not very helpful, 00:50:23.733 --> 00:50:24.900 align:middle line:90% which was funny at the time. 00:50:24.900 --> 00:50:27.680 align:middle line:84% But the point being that, yeah, the things 00:50:27.680 --> 00:50:30.110 align:middle line:84% that people do in companies to actually build products 00:50:30.110 --> 00:50:31.400 align:middle line:84% are quite different than the things that 00:50:31.400 --> 00:50:33.230 align:middle line:84% happen in academic research labs to imagine 00:50:33.230 --> 00:50:35.750 align:middle line:90% these kinds of things. 00:50:35.750 --> 00:50:39.070 align:middle line:84% Here are a couple of striking examples. 00:50:39.070 --> 00:50:40.320 align:middle line:90% And I'll point out this later. 00:50:40.320 --> 00:50:41.940 align:middle line:84% I think computer programming gets this. 00:50:41.940 --> 00:50:44.232 align:middle line:84% Have you any of you ever heard the phrase computational 00:50:44.232 --> 00:50:45.260 align:middle line:90% thinking? 00:50:45.260 --> 00:50:47.840 align:middle line:84% So computational thinking is roughly the claim 00:50:47.840 --> 00:50:51.248 align:middle line:84% that if you teach people how to program computers, 00:50:51.248 --> 00:50:52.790 align:middle line:84% they will have a bunch of skills that 00:50:52.790 --> 00:50:55.020 align:middle line:84% are useful in other kinds of domains, 00:50:55.020 --> 00:50:57.020 align:middle line:84% like, you will learn how to break problems down 00:50:57.020 --> 00:50:58.050 align:middle line:90% into smaller problems. 00:50:58.050 --> 00:50:59.900 align:middle line:84% You'll learn how to think through things with algorithms, 00:50:59.900 --> 00:51:00.810 align:middle line:90% and stuff like that. 00:51:00.810 --> 00:51:02.810 align:middle line:84% It is actually that line of argument 00:51:02.810 --> 00:51:06.530 align:middle line:84% is very similar to the argument a century-plus ago 00:51:06.530 --> 00:51:08.888 align:middle line:84% or a couple of centuries ago that if you learn 00:51:08.888 --> 00:51:11.180 align:middle line:84% Latin and Greek, you will have these domain-independent 00:51:11.180 --> 00:51:12.980 align:middle line:84% thinking skills that are good for reasoning 00:51:12.980 --> 00:51:15.830 align:middle line:90% and other kinds of domains. 00:51:15.830 --> 00:51:18.500 align:middle line:84% Here's a remarkable study that a colleague of mine 00:51:18.500 --> 00:51:20.340 align:middle line:84% did recently that illustrates this. 00:51:20.340 --> 00:51:22.460 align:middle line:84% He was trying to figure out, how good 00:51:22.460 --> 00:51:26.240 align:middle line:90% are people at differentiating-- 00:51:26.240 --> 00:51:28.790 align:middle line:84% at sorting truth from fiction online? 00:51:28.790 --> 00:51:30.390 align:middle line:90% So he shows them two websites. 00:51:30.390 --> 00:51:32.640 align:middle line:84% One of them is the American Academy of Pediatrics. 00:51:32.640 --> 00:51:35.145 align:middle line:84% The other is the American College of Pediatricians. 00:51:35.145 --> 00:51:36.270 align:middle line:90% Those sound pretty similar. 00:51:36.270 --> 00:51:38.180 align:middle line:84% The websites actually look pretty similar. 00:51:38.180 --> 00:51:42.620 align:middle line:84% The American Academy of Pediatrics is a 40,000-member 00:51:42.620 --> 00:51:45.380 align:middle line:84% organization that represents pediatricians across the United 00:51:45.380 --> 00:51:46.250 align:middle line:90% States. 00:51:46.250 --> 00:51:49.020 align:middle line:84% The American College of Pediatricians is like a 400, 00:51:49.020 --> 00:51:52.430 align:middle line:84% or 500, 600-person splinter group that split off from 00:51:52.430 --> 00:51:55.880 align:middle line:84% the American Academy of Pediatricians in opposition 00:51:55.880 --> 00:51:57.770 align:middle line:90% to same sex adoption. 00:51:57.770 --> 00:51:59.700 align:middle line:84% So like one is a very mainstream group. 00:51:59.700 --> 00:52:01.670 align:middle line:90% One is a splinter group. 00:52:01.670 --> 00:52:04.730 align:middle line:84% Professional fact checkers who work at newspapers and magazines 00:52:04.730 --> 00:52:07.800 align:middle line:84% and stuff like that could identify this-- 00:52:07.800 --> 00:52:10.880 align:middle line:84% the difference between these websites and organizations 100% 00:52:10.880 --> 00:52:11.810 align:middle line:90% of the time. 00:52:11.810 --> 00:52:14.780 align:middle line:84% How well do you think historians did at this, 00:52:14.780 --> 00:52:17.090 align:middle line:84% professional trained historians did 00:52:17.090 --> 00:52:19.880 align:middle line:84% at this, who spend their entire lives studying how to evaluate 00:52:19.880 --> 00:52:22.980 align:middle line:84% information, and how well do you think Stanford freshmen do 00:52:22.980 --> 00:52:25.380 align:middle line:90% at this task? 00:52:25.380 --> 00:52:27.900 align:middle line:84% AUDIENCE: Like, saying this group is not 00:52:27.900 --> 00:52:29.560 align:middle line:90% the same as this other group? 00:52:29.560 --> 00:52:31.470 align:middle line:84% JUSTIN REICH: Yeah, I mean, it was slightly more detailed 00:52:31.470 --> 00:52:31.970 align:middle line:90% than that. 00:52:31.970 --> 00:52:33.360 align:middle line:84% It was like, which of these would 00:52:33.360 --> 00:52:38.500 align:middle line:84% have better substantiated claims, 00:52:38.500 --> 00:52:41.695 align:middle line:84% but even recognizing the difference between them, 00:52:41.695 --> 00:52:43.320 align:middle line:84% like even identifying that the American 00:52:43.320 --> 00:52:45.720 align:middle line:84% College of Pediatricians is not the same thing 00:52:45.720 --> 00:52:50.100 align:middle line:84% as the American Academy of Pediatricians. 00:52:50.100 --> 00:52:53.490 align:middle line:84% They do shockingly, shockingly poorly. 00:52:53.490 --> 00:52:55.980 align:middle line:84% So 50% of trained, award-winning, 00:52:55.980 --> 00:52:58.540 align:middle line:84% tenured historians either wasn't sure-- 00:52:58.540 --> 00:53:00.540 align:middle line:84% I think the question they ask is, which of these 00:53:00.540 --> 00:53:03.010 align:middle line:84% is a more reliable source of information? 00:53:03.010 --> 00:53:05.265 align:middle line:84% And the fact checkers were like, yeah, the Academy, 00:53:05.265 --> 00:53:07.140 align:middle line:84% with all the people and the mainstream group. 00:53:07.140 --> 00:53:09.160 align:middle line:84% And historians, about half of them did that. 00:53:09.160 --> 00:53:10.830 align:middle line:84% And actually, about 20% of Stanford 00:53:10.830 --> 00:53:14.970 align:middle line:84% freshmen people, who got into a university with a 3% acceptance 00:53:14.970 --> 00:53:17.310 align:middle line:90% rate or something like that. 00:53:17.310 --> 00:53:23.070 align:middle line:84% Are fact checkers smarter than historians, tenured historians, 00:53:23.070 --> 00:53:25.400 align:middle line:90% or Stanford students? 00:53:25.400 --> 00:53:28.190 align:middle line:84% Like, it doesn't stand to reason that they 00:53:28.190 --> 00:53:32.810 align:middle line:84% would have better generic, like domain-independent thinking 00:53:32.810 --> 00:53:35.000 align:middle line:84% skills, or critical thinking skills, or problem 00:53:35.000 --> 00:53:37.290 align:middle line:90% solving, something like that. 00:53:37.290 --> 00:53:39.410 align:middle line:84% A better way to think about it is, 00:53:39.410 --> 00:53:42.350 align:middle line:84% they must be doing something different. 00:53:42.350 --> 00:53:46.010 align:middle line:84% And it turned out that when you observe them, they were. 00:53:46.010 --> 00:53:47.900 align:middle line:84% When historians and Stanford freshmen 00:53:47.900 --> 00:53:50.130 align:middle line:84% go to a website to evaluate the website, 00:53:50.130 --> 00:53:51.680 align:middle line:90% they scroll up and down. 00:53:51.680 --> 00:53:55.580 align:middle line:84% And they look on the website for markers of credibility. 00:53:55.580 --> 00:53:57.838 align:middle line:84% Does this-- is it formatted correctly? 00:53:57.838 --> 00:53:59.880 align:middle line:84% Does it seem to be from a reputable organization? 00:53:59.880 --> 00:54:01.322 align:middle line:90% Is it a .org or a .com? 00:54:01.322 --> 00:54:02.280 align:middle line:90% Does it have citations? 00:54:02.280 --> 00:54:02.820 align:middle line:90% Stuff like that. 00:54:02.820 --> 00:54:04.362 align:middle line:84% The American College of Pediatricians 00:54:04.362 --> 00:54:06.230 align:middle line:84% has all of those markers of credibility. 00:54:06.230 --> 00:54:08.300 align:middle line:84% All the bad guys have lists of how 00:54:08.300 --> 00:54:10.168 align:middle line:84% to make a credible looking website. 00:54:10.168 --> 00:54:12.210 align:middle line:84% The fact checkers do something totally different. 00:54:12.210 --> 00:54:14.833 align:middle line:84% As soon as they go to a website, they leave it. 00:54:14.833 --> 00:54:17.250 align:middle line:84% So they would go to the American College of Pediatricians, 00:54:17.250 --> 00:54:19.292 align:middle line:84% and they immediately just Google American College 00:54:19.292 --> 00:54:21.680 align:middle line:84% of Pediatricians, in which you very quickly learn 00:54:21.680 --> 00:54:23.900 align:middle line:84% it's a 600-person splinter group, 00:54:23.900 --> 00:54:26.030 align:middle line:90% and those sorts of things. 00:54:26.030 --> 00:54:29.750 align:middle line:90% And so people with-- 00:54:29.750 --> 00:54:31.290 align:middle line:84% the thing which is so striking here, 00:54:31.290 --> 00:54:33.320 align:middle line:84% actually, the reason why this research group 00:54:33.320 --> 00:54:35.360 align:middle line:84% picked this tenured group of historians is they 00:54:35.360 --> 00:54:39.083 align:middle line:84% were hoping that they would be examples of experts. 00:54:39.083 --> 00:54:41.750 align:middle line:84% They would be like, these people have trained their entire lives 00:54:41.750 --> 00:54:45.050 align:middle line:84% to study information and to sort truth from fiction 00:54:45.050 --> 00:54:46.140 align:middle line:90% in historical documents. 00:54:46.140 --> 00:54:47.900 align:middle line:84% And it turns out spending your entire life 00:54:47.900 --> 00:54:50.090 align:middle line:84% learning how to truth from fiction in the archives 00:54:50.090 --> 00:54:52.820 align:middle line:84% doesn't help you very much, learning how to sort truth 00:54:52.820 --> 00:54:54.860 align:middle line:90% from fiction on the internet. 00:54:54.860 --> 00:54:57.740 align:middle line:84% So this is one of the key ideas of transfer, 00:54:57.740 --> 00:55:00.950 align:middle line:84% that if you learn how to do something in one domain, 00:55:00.950 --> 00:55:03.840 align:middle line:84% even changing the context of that domain 00:55:03.840 --> 00:55:06.778 align:middle line:84% not that much, going from sorting truth from fiction 00:55:06.778 --> 00:55:09.320 align:middle line:84% in the archives, is sufficiently different from sorting truth 00:55:09.320 --> 00:55:11.600 align:middle line:84% from fiction online, that there's not 00:55:11.600 --> 00:55:13.790 align:middle line:84% a generic set of critical thinking skills 00:55:13.790 --> 00:55:15.812 align:middle line:90% that helps you bridge that. 00:55:15.812 --> 00:55:18.020 align:middle line:84% If you want to get good at sorting truth from fiction 00:55:18.020 --> 00:55:20.103 align:middle line:84% online, you have to learn a specific set of skills 00:55:20.103 --> 00:55:23.540 align:middle line:84% that are related to learning to truth from fiction online. 00:55:23.540 --> 00:55:27.680 align:middle line:84% Here's another one of those examples. 00:55:27.680 --> 00:55:29.810 align:middle line:84% This is a classic experiment that gets replicated 00:55:29.810 --> 00:55:30.930 align:middle line:90% in various kinds of ways. 00:55:30.930 --> 00:55:32.760 align:middle line:84% In one experiment, third graders, 00:55:32.760 --> 00:55:34.342 align:middle line:84% some identified by a reading test 00:55:34.342 --> 00:55:36.050 align:middle line:84% as good readers, some as poor, were asked 00:55:36.050 --> 00:55:37.530 align:middle line:90% to read a passage about soccer. 00:55:37.530 --> 00:55:39.720 align:middle line:84% The poor readers, who knew a lot about soccer, 00:55:39.720 --> 00:55:42.140 align:middle line:84% were three times as likely to make accurate inferences 00:55:42.140 --> 00:55:44.810 align:middle line:84% about the passage as the good readers, who didn't know much 00:55:44.810 --> 00:55:45.690 align:middle line:90% about the game. 00:55:45.690 --> 00:55:48.290 align:middle line:84% Current education practices show that reading comprehension 00:55:48.290 --> 00:55:49.410 align:middle line:90% is misunderstood. 00:55:49.410 --> 00:55:51.110 align:middle line:84% It's treated like a general skill 00:55:51.110 --> 00:55:53.700 align:middle line:84% that can be applied with equal success to all texts. 00:55:53.700 --> 00:55:55.970 align:middle line:84% Rather, comprehension is intimately 00:55:55.970 --> 00:55:57.330 align:middle line:90% intertwined with knowledge. 00:55:57.330 --> 00:56:00.090 align:middle line:84% So we think there's ways of measuring, 00:56:00.090 --> 00:56:02.280 align:middle line:84% are you a good reader or a bad reader? 00:56:02.280 --> 00:56:04.640 align:middle line:84% But it actually sets aside the fact 00:56:04.640 --> 00:56:06.900 align:middle line:84% that if a bunch of stuff about soccer, 00:56:06.900 --> 00:56:08.280 align:middle line:90% you're a good soccer reader. 00:56:08.280 --> 00:56:10.363 align:middle line:84% And if you don't know a lot of stuff about soccer, 00:56:10.363 --> 00:56:11.660 align:middle line:90% you're a bad soccer reader. 00:56:11.660 --> 00:56:14.520 align:middle line:84% This is terrible for reading instruction, right? 00:56:14.520 --> 00:56:16.100 align:middle line:84% It would be much better for the world 00:56:16.100 --> 00:56:17.940 align:middle line:84% if we could just make you a good reader. 00:56:17.940 --> 00:56:20.810 align:middle line:84% But actually, the way to be good at reading in a domain 00:56:20.810 --> 00:56:23.630 align:middle line:84% is to teach you a bunch of facts and knowledge and information 00:56:23.630 --> 00:56:27.930 align:middle line:84% in that specific domain, that, in fact, the skill of reading 00:56:27.930 --> 00:56:31.340 align:middle line:84% doesn't transfer that well from reading about soccer to reading 00:56:31.340 --> 00:56:32.720 align:middle line:90% about softball. 00:56:32.720 --> 00:56:35.240 align:middle line:84% It's not that it doesn't transfer at all. 00:56:35.240 --> 00:56:39.140 align:middle line:84% And also, being able to read well about soccer 00:56:39.140 --> 00:56:42.740 align:middle line:84% is more like being able to read well about softball, 00:56:42.740 --> 00:56:45.000 align:middle line:84% and less likely-- less related to being 00:56:45.000 --> 00:56:47.000 align:middle line:84% able to read well about restaurants or something 00:56:47.000 --> 00:56:47.500 align:middle line:90% like that. 00:56:47.500 --> 00:56:50.120 align:middle line:90% 00:56:50.120 --> 00:56:53.270 align:middle line:84% This is my way of making fun of the claim 00:56:53.270 --> 00:56:55.610 align:middle line:84% that computer science taught me to break down problems 00:56:55.610 --> 00:56:56.160 align:middle line:90% into pieces. 00:56:56.160 --> 00:56:58.850 align:middle line:84% This is a classic kind of computational thinking argument, 00:56:58.850 --> 00:57:01.350 align:middle line:84% that if you teach someone computer science, 00:57:01.350 --> 00:57:05.000 align:middle line:84% they'll be able to break down problems into pieces. 00:57:05.000 --> 00:57:06.860 align:middle line:84% So here's an AP literature question. 00:57:06.860 --> 00:57:08.960 align:middle line:84% Before using computer science, I answered this 00:57:08.960 --> 00:57:10.490 align:middle line:84% with one long sentence, but now I 00:57:10.490 --> 00:57:12.900 align:middle line:84% use paragraphs to break up my arguments into pieces. 00:57:12.900 --> 00:57:15.320 align:middle line:84% No, of course, from the very beginning 00:57:15.320 --> 00:57:17.000 align:middle line:84% of you learning how to write, you 00:57:17.000 --> 00:57:19.100 align:middle line:84% were taught to break down problems into pieces. 00:57:19.100 --> 00:57:20.090 align:middle line:90% That's what writing is. 00:57:20.090 --> 00:57:21.780 align:middle line:84% You break big ideas into paragraphs. 00:57:21.780 --> 00:57:23.280 align:middle line:84% You break paragraphs into sentences. 00:57:23.280 --> 00:57:24.870 align:middle line:90% You make sentences into phrases. 00:57:24.870 --> 00:57:28.140 align:middle line:84% You've been taught forever how to break ideas down into pieces. 00:57:28.140 --> 00:57:32.060 align:middle line:84% But turning written words into breaking them down into pieces 00:57:32.060 --> 00:57:34.880 align:middle line:84% doesn't really help you at all in taking computer programming 00:57:34.880 --> 00:57:37.550 align:middle line:84% problems and breaking them down into pieces. 00:57:37.550 --> 00:57:39.360 align:middle line:84% Before studying computer science, 00:57:39.360 --> 00:57:41.580 align:middle line:84% I wrote one equation to solve all these problems, 00:57:41.580 --> 00:57:44.990 align:middle line:84% but now I break problems down into pieces. 00:57:44.990 --> 00:57:47.270 align:middle line:84% AUDIENCE: Yeah, I could speak like that 00:57:47.270 --> 00:57:49.550 align:middle line:90% from firsthand experience. 00:57:49.550 --> 00:57:53.810 align:middle line:84% I've done computer science problem-solving very much 00:57:53.810 --> 00:57:55.920 align:middle line:90% since like middle school. 00:57:55.920 --> 00:57:58.560 align:middle line:84% And in terms of domain knowledge, 00:57:58.560 --> 00:58:02.340 align:middle line:84% I don't-- when I encounter with a different domain, 00:58:02.340 --> 00:58:06.350 align:middle line:84% I don't know where to even cut the pieces for the problem. 00:58:06.350 --> 00:58:08.190 align:middle line:90% Sure, I want to do that. 00:58:08.190 --> 00:58:09.030 align:middle line:90% But where? 00:58:09.030 --> 00:58:11.760 align:middle line:84% I don't know where to cut it into smaller pieces. 00:58:11.760 --> 00:58:15.017 align:middle line:84% This is something I have to acquire, just like anyone else. 00:58:15.017 --> 00:58:17.600 align:middle line:84% JUSTIN REICH: Yeah, that is the domain-specific knowledge that 00:58:17.600 --> 00:58:20.480 align:middle line:84% you need to acquire in that particular domain to know what-- 00:58:20.480 --> 00:58:25.610 align:middle line:84% I mean, it is probably slightly useful to have the intuition. 00:58:25.610 --> 00:58:27.590 align:middle line:84% Hey, bigger problems can be broken down 00:58:27.590 --> 00:58:28.550 align:middle line:90% into smaller problems. 00:58:28.550 --> 00:58:30.720 align:middle line:84% That's probably a little bit helpful in a lot of cases. 00:58:30.720 --> 00:58:32.360 align:middle line:84% But I like the way that you phrased that. 00:58:32.360 --> 00:58:33.930 align:middle line:84% If you don't know anything about the domain, 00:58:33.930 --> 00:58:36.472 align:middle line:84% you don't know what the relevant division points for breaking 00:58:36.472 --> 00:58:37.615 align:middle line:90% something into pieces are. 00:58:37.615 --> 00:58:41.090 align:middle line:90% 00:58:41.090 --> 00:58:43.835 align:middle line:84% One of the first people to study this was Edward Thorndike. 00:58:43.835 --> 00:58:45.710 align:middle line:84% I was just rereading today one of the studies 00:58:45.710 --> 00:58:49.730 align:middle line:84% that he did, which was to have people 00:58:49.730 --> 00:58:52.440 align:middle line:84% be really good at estimating the size of rectangles. 00:58:52.440 --> 00:58:54.560 align:middle line:84% So he would cut rectangles on pieces of paper 00:58:54.560 --> 00:58:58.620 align:middle line:84% and have people look at rectangles, estimate their area, 00:58:58.620 --> 00:59:00.710 align:middle line:84% give them some feedback, and you can actually 00:59:00.710 --> 00:59:03.600 align:middle line:84% get-- there's 100 different rectangles of different sizes. 00:59:03.600 --> 00:59:05.808 align:middle line:84% And he found pretty quickly you can get people better 00:59:05.808 --> 00:59:07.120 align:middle line:90% at estimating rectangles. 00:59:07.120 --> 00:59:10.840 align:middle line:84% Then he would give them piles of circles and triangles 00:59:10.840 --> 00:59:12.500 align:middle line:84% and rhombuses and things like that. 00:59:12.500 --> 00:59:14.230 align:middle line:84% And they're no better at estimating 00:59:14.230 --> 00:59:15.710 align:middle line:84% the size of circles and triangles, 00:59:15.710 --> 00:59:18.790 align:middle line:84% having studied how to estimate rectangles again. 00:59:18.790 --> 00:59:20.500 align:middle line:84% There are all kinds of studies that 00:59:20.500 --> 00:59:22.720 align:middle line:84% are done like this, a lot-- a wave 00:59:22.720 --> 00:59:26.710 align:middle line:84% recently being studies that disprove that there's 00:59:26.710 --> 00:59:31.990 align:middle line:84% any relationship between chess knowledge and academic outcomes, 00:59:31.990 --> 00:59:33.890 align:middle line:84% or chess proficiency and academic outcomes, 00:59:33.890 --> 00:59:36.520 align:middle line:84% between music knowledge and academic outcomes. 00:59:36.520 --> 00:59:38.410 align:middle line:84% These things are much more separate 00:59:38.410 --> 00:59:40.600 align:middle line:90% than we might have hoped. 00:59:40.600 --> 00:59:42.710 align:middle line:84% AUDIENCE: About the reading comprehension thing. 00:59:42.710 --> 00:59:43.502 align:middle line:90% JUSTIN REICH: Yeah. 00:59:43.502 --> 00:59:46.900 align:middle line:84% AUDIENCE: Do the standardized tests like a SAT or ACT take 00:59:46.900 --> 00:59:49.220 align:middle line:90% that into account or they don't? 00:59:49.220 --> 00:59:51.790 align:middle line:84% JUSTIN REICH: It's a huge problem. 00:59:51.790 --> 00:59:55.060 align:middle line:84% So I mean, in a sense, they-- here's 00:59:55.060 --> 00:59:57.340 align:middle line:84% how they try to take it into account 00:59:57.340 --> 01:00:02.290 align:middle line:84% is they try to indicate to educators what 01:00:02.290 --> 01:00:06.730 align:middle line:84% are the domains of knowledge that will be sampled by the SAT 01:00:06.730 --> 01:00:09.550 align:middle line:84% and have people read things that are related 01:00:09.550 --> 01:00:12.820 align:middle line:90% to those domains of knowledge. 01:00:12.820 --> 01:00:16.227 align:middle line:84% One of the knocks about particularly early 01:00:16.227 --> 01:00:17.060 align:middle line:90% standardized tests-- 01:00:17.060 --> 01:00:19.352 align:middle line:84% I think standardized testing companies have done better 01:00:19.352 --> 01:00:20.650 align:middle line:90% about these kinds of things-- 01:00:20.650 --> 01:00:25.990 align:middle line:84% is there would be all kinds of questions that are about-- 01:00:25.990 --> 01:00:29.020 align:middle line:84% or where the examples take place in a country club 01:00:29.020 --> 01:00:30.100 align:middle line:90% or something like that. 01:00:30.100 --> 01:00:32.802 align:middle line:84% And 75 years ago or whatever, maybe it 01:00:32.802 --> 01:00:35.260 align:middle line:84% would have been pretty likely that just about everybody who 01:00:35.260 --> 01:00:37.433 align:middle line:84% is applying to college knows what a country club is. 01:00:37.433 --> 01:00:39.100 align:middle line:84% But there are tons of Americans who have 01:00:39.100 --> 01:00:40.810 align:middle line:90% no idea what a country club is. 01:00:40.810 --> 01:00:47.308 align:middle line:84% And so if you treat reading as a domain-independent entity, where 01:00:47.308 --> 01:00:49.600 align:middle line:84% everybody should be able to recognize the words country 01:00:49.600 --> 01:00:51.470 align:middle line:84% club, and then process the rest of the sentence, 01:00:51.470 --> 01:00:52.345 align:middle line:90% then that seems fair. 01:00:52.345 --> 01:00:55.540 align:middle line:84% But the more you realize, no, knowing something 01:00:55.540 --> 01:00:58.270 align:middle line:84% about a country club gives you an advantage on questions 01:00:58.270 --> 01:01:00.760 align:middle line:90% that are about country clubs. 01:01:00.760 --> 01:01:03.478 align:middle line:84% And I think we could all agree that knowing about country clubs 01:01:03.478 --> 01:01:05.770 align:middle line:84% is not something that's particularly important to going 01:01:05.770 --> 01:01:07.570 align:middle line:84% to college, or advancing higher education, 01:01:07.570 --> 01:01:08.653 align:middle line:90% or other things like that. 01:01:08.653 --> 01:01:10.750 align:middle line:84% And they're like, oh, that is kind of unfair. 01:01:10.750 --> 01:01:12.760 align:middle line:84% So I think there's more recognition 01:01:12.760 --> 01:01:14.150 align:middle line:90% of those things over time. 01:01:14.150 --> 01:01:15.858 align:middle line:84% Of course, you have the opposite problem, 01:01:15.858 --> 01:01:18.790 align:middle line:84% which is that the more you tell people 01:01:18.790 --> 01:01:22.990 align:middle line:84% exactly what the domains are that might be tested, 01:01:22.990 --> 01:01:26.770 align:middle line:84% the more you have people just studying and practicing 01:01:26.770 --> 01:01:27.710 align:middle line:90% those domains. 01:01:27.710 --> 01:01:32.510 align:middle line:84% And that's not exactly the point of these testing systems either. 01:01:32.510 --> 01:01:34.750 align:middle line:84% Ideally, they would give us some kind 01:01:34.750 --> 01:01:36.850 align:middle line:84% of indication of people's proficiency 01:01:36.850 --> 01:01:38.350 align:middle line:84% in a wide range of things that might 01:01:38.350 --> 01:01:39.558 align:middle line:90% help them be good at college. 01:01:39.558 --> 01:01:41.920 align:middle line:90% But it's a real problem. 01:01:41.920 --> 01:01:45.490 align:middle line:84% AUDIENCE: I've heard that SAT kind of changed their argument 01:01:45.490 --> 01:01:49.570 align:middle line:84% a little bit, or the way they phrase it into it's 01:01:49.570 --> 01:01:53.110 align:middle line:84% not that the test matters at all. 01:01:53.110 --> 01:01:58.870 align:middle line:84% The argument now is more leaning towards their performance 01:01:58.870 --> 01:02:01.960 align:middle line:84% on the test indicates the student's ability 01:02:01.960 --> 01:02:06.070 align:middle line:84% to prepare for the test, even though the test is meaningless, 01:02:06.070 --> 01:02:09.920 align:middle line:84% the ability to prepare for the test is what is being tested, 01:02:09.920 --> 01:02:13.210 align:middle line:84% and that correlated well with other kinds 01:02:13.210 --> 01:02:19.600 align:middle line:84% of things, which is not completely outrageous to say. 01:02:19.600 --> 01:02:23.440 align:middle line:84% I give you a completely stupid subject no one heard of 01:02:23.440 --> 01:02:25.820 align:middle line:90% and tell you to prepare for it. 01:02:25.820 --> 01:02:28.360 align:middle line:84% The people who better prepare for it are probably better 01:02:28.360 --> 01:02:32.350 align:middle line:84% studiers who will perform better at college, where you will also 01:02:32.350 --> 01:02:34.520 align:middle line:90% do that study. 01:02:34.520 --> 01:02:36.430 align:middle line:90% But it's not perfect. 01:02:36.430 --> 01:02:39.400 align:middle line:84% And it will lead to disproportionate outcomes 01:02:39.400 --> 01:02:45.220 align:middle line:84% when these people who do not fit into the neat, nice line where 01:02:45.220 --> 01:02:48.482 align:middle line:90% standards fit go. 01:02:48.482 --> 01:02:49.940 align:middle line:84% JUSTIN REICH: I think that's right. 01:02:49.940 --> 01:02:53.680 align:middle line:84% I think an early rhetoric of testing 01:02:53.680 --> 01:02:57.850 align:middle line:84% would have been something along the lines of intelligence 01:02:57.850 --> 01:03:02.180 align:middle line:90% is a universal human quality. 01:03:02.180 --> 01:03:03.740 align:middle line:90% We can design tests. 01:03:03.740 --> 01:03:05.450 align:middle line:90% That's not culturally specific. 01:03:05.450 --> 01:03:07.520 align:middle line:84% We can design tests that measure intelligence. 01:03:07.520 --> 01:03:10.040 align:middle line:84% And using these accurate tests that measure intelligence, 01:03:10.040 --> 01:03:12.100 align:middle line:84% we can find the most intelligent people, 01:03:12.100 --> 01:03:13.775 align:middle line:84% and then we can send them into college. 01:03:13.775 --> 01:03:16.150 align:middle line:84% As soon as you recognize that some degree of intelligence 01:03:16.150 --> 01:03:19.690 align:middle line:84% tests are culturally biased, because of all these transfer 01:03:19.690 --> 01:03:21.910 align:middle line:84% problems that we've just been talking about, 01:03:21.910 --> 01:03:29.230 align:middle line:84% that if you have a bunch of questions about ranching, then 01:03:29.230 --> 01:03:32.212 align:middle line:84% you benefit young people who are in rural states 01:03:32.212 --> 01:03:33.920 align:middle line:84% where there's a lot of ranching going on. 01:03:33.920 --> 01:03:36.503 align:middle line:84% But what does that have-- that's the equivalent of the country 01:03:36.503 --> 01:03:38.200 align:middle line:90% club kind of thing. 01:03:38.200 --> 01:03:40.210 align:middle line:84% And then the other argument for them 01:03:40.210 --> 01:03:44.050 align:middle line:84% is that they are pretty good-- there are correlations 01:03:44.050 --> 01:03:45.790 align:middle line:84% between people's success on these tests 01:03:45.790 --> 01:03:47.360 align:middle line:84% and their later success in college. 01:03:47.360 --> 01:03:49.210 align:middle line:84% It's not a perfect correlation, but it 01:03:49.210 --> 01:03:52.300 align:middle line:84% is one indicator that if someone is struggling to do well 01:03:52.300 --> 01:03:54.245 align:middle line:84% on these tests, there's reasonable reasons 01:03:54.245 --> 01:03:56.620 align:middle line:84% to believe that they might struggle to do well in college 01:03:56.620 --> 01:03:58.360 align:middle line:90% settings and so forth. 01:03:58.360 --> 01:04:03.220 align:middle line:84% Anyway, you can imagine unraveling this kind of question 01:04:03.220 --> 01:04:07.630 align:middle line:84% has huge implications for how valuable it is for kids to spend 01:04:07.630 --> 01:04:09.500 align:middle line:90% zillions of hours on Minecraft. 01:04:09.500 --> 01:04:12.230 align:middle line:84% So when kids play zillions of hours on Minecraft, 01:04:12.230 --> 01:04:14.290 align:middle line:90% they learn a ton. 01:04:14.290 --> 01:04:19.010 align:middle line:84% They learn how to collaborate with other people. 01:04:19.010 --> 01:04:22.000 align:middle line:84% They learn how to find information on the internet. 01:04:22.000 --> 01:04:25.060 align:middle line:84% Minecraft, a little bit like The Logical Journey of the Zoombinis 01:04:25.060 --> 01:04:27.430 align:middle line:84% has no instructions on the screen. 01:04:27.430 --> 01:04:29.120 align:middle line:90% You have to go to wikis. 01:04:29.120 --> 01:04:30.815 align:middle line:84% You have to watch playthrough videos. 01:04:30.815 --> 01:04:32.440 align:middle line:84% You have to do other kinds of exercises 01:04:32.440 --> 01:04:34.820 align:middle line:84% to figure out how to engage in Minecraft. 01:04:34.820 --> 01:04:38.260 align:middle line:84% And some of those things seem like domain-independent sorts 01:04:38.260 --> 01:04:41.210 align:middle line:84% of skills-- how to find information from other places, 01:04:41.210 --> 01:04:44.860 align:middle line:84% how to set goals, how to pursue those goals, other kinds 01:04:44.860 --> 01:04:45.970 align:middle line:90% of things like that. 01:04:45.970 --> 01:04:49.390 align:middle line:84% But an enormous question about the value of playing games 01:04:49.390 --> 01:04:52.450 align:middle line:84% like that is something like, well, 01:04:52.450 --> 01:04:54.940 align:middle line:84% if you know how to do all that in Minecraft, does it help 01:04:54.940 --> 01:04:56.710 align:middle line:90% you do that somewhere else? 01:04:56.710 --> 01:04:59.980 align:middle line:84% There's a lot of educational psychology research that 01:04:59.980 --> 01:05:04.270 align:middle line:84% suggests that transfer is really hard, that when you learn things 01:05:04.270 --> 01:05:08.770 align:middle line:84% in one domain, people are not that good at passing it 01:05:08.770 --> 01:05:10.450 align:middle line:90% to another domain. 01:05:10.450 --> 01:05:14.590 align:middle line:84% To me, there are two contrary arguments to that. 01:05:14.590 --> 01:05:16.220 align:middle line:84% There's a joking way of putting it. 01:05:16.220 --> 01:05:18.040 align:middle line:84% But I think one of the best arguments 01:05:18.040 --> 01:05:19.642 align:middle line:90% against humans' ability-- 01:05:19.642 --> 01:05:21.850 align:middle line:84% one of the best arguments in favor of humans' ability 01:05:21.850 --> 01:05:24.190 align:middle line:84% to transfer learning is civilization. 01:05:24.190 --> 01:05:27.940 align:middle line:84% It's pretty hard to imagine how you would develop civilization 01:05:27.940 --> 01:05:30.340 align:middle line:84% if people weren't pretty good at learning some things, 01:05:30.340 --> 01:05:33.550 align:middle line:84% and then applying them to new kinds of tasks. 01:05:33.550 --> 01:05:36.880 align:middle line:84% So a theme of counterargument is that, basically, 01:05:36.880 --> 01:05:40.900 align:middle line:84% educational researchers are not good at detecting and observing 01:05:40.900 --> 01:05:41.720 align:middle line:90% transfer. 01:05:41.720 --> 01:05:44.260 align:middle line:84% We're not good at conducting studies 01:05:44.260 --> 01:05:47.440 align:middle line:84% that measure people's abilities to take things from one domain 01:05:47.440 --> 01:05:49.340 align:middle line:90% and put them to another domain. 01:05:49.340 --> 01:05:51.610 align:middle line:84% If you listen to that podcast episode with Constance 01:05:51.610 --> 01:05:54.910 align:middle line:84% Steinkuehler, who's a huge advocate of the benefits 01:05:54.910 --> 01:05:56.380 align:middle line:84% of games for learning, you'll hear 01:05:56.380 --> 01:05:59.630 align:middle line:84% her make an argument along those lines, which is like, come on, 01:05:59.630 --> 01:06:00.340 align:middle line:90% Justin. 01:06:00.340 --> 01:06:03.220 align:middle line:84% If you get really good at doing cool stuff in Minecraft, 01:06:03.220 --> 01:06:04.720 align:middle line:84% that will help you with other things 01:06:04.720 --> 01:06:05.570 align:middle line:90% that you're doing in life. 01:06:05.570 --> 01:06:06.730 align:middle line:84% Not that different from what Dana 01:06:06.730 --> 01:06:08.855 align:middle line:84% said at the beginning of the class about the things 01:06:08.855 --> 01:06:10.852 align:middle line:84% you learn playing The Resistance. 01:06:10.852 --> 01:06:11.560 align:middle line:90% I can learn how-- 01:06:11.560 --> 01:06:13.640 align:middle line:84% I learned how to keep a poker face there, 01:06:13.640 --> 01:06:16.330 align:middle line:84% and now I can keep a poker face in other kinds of places. 01:06:16.330 --> 01:06:19.640 align:middle line:84% So unanswered questions to be sure. 01:06:19.640 --> 01:06:20.920 align:middle line:90% Good. 01:06:20.920 --> 01:06:21.940 align:middle line:90% 2:25. 01:06:21.940 --> 01:06:24.773 align:middle line:84% You all should have a wonderful spring break. 01:06:24.773 --> 01:06:27.190 align:middle line:84% You should come-- there's a little bit of reading that you 01:06:27.190 --> 01:06:28.565 align:middle line:84% have to do, but mostly you should 01:06:28.565 --> 01:06:30.580 align:middle line:84% come with a draft, the strongest draft 01:06:30.580 --> 01:06:33.280 align:middle line:84% that you can put together for paper two on Monday, a week 01:06:33.280 --> 01:06:33.890 align:middle line:90% from Monday. 01:06:33.890 --> 01:06:37.210 align:middle line:84% And we'll have a chance to do some peer editing in that class. 01:06:37.210 --> 01:06:39.520 align:middle line:84% And we're going to shift from genres 01:06:39.520 --> 01:06:41.350 align:middle line:84% of learning at scale to the as yet 01:06:41.350 --> 01:06:43.840 align:middle line:84% intractable dilemmas in the second half of Failure 01:06:43.840 --> 01:06:46.080 align:middle line:90% to Disrupt. 01:06:46.080 --> 01:06:54.000 align:middle line:90%