WEBVTT 00:00:00.090 --> 00:00:02.430 The following content is provided under a Creative 00:00:02.430 --> 00:00:03.850 Commons license. 00:00:03.850 --> 00:00:06.060 Your support will help MIT OpenCourseWare 00:00:06.060 --> 00:00:10.150 continue to offer high quality educational resources for free. 00:00:10.150 --> 00:00:12.690 To make a donation or to view additional materials 00:00:12.690 --> 00:00:16.620 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:16.620 --> 00:00:25.785 at ocw.mit.edu 00:00:25.785 --> 00:00:27.210 JANET RANKIN: All right, great. 00:00:27.210 --> 00:00:30.300 I looked over some of the Mud cards we had from last time. 00:00:30.300 --> 00:00:32.049 And there were some really good questions, 00:00:32.049 --> 00:00:33.750 some really good points, and one lingering one 00:00:33.750 --> 00:00:35.820 from the previous time that I forgot to address. 00:00:35.820 --> 00:00:37.236 Someone said, could we please just 00:00:37.236 --> 00:00:39.030 move the desks into an arrangement where 00:00:39.030 --> 00:00:40.440 we could see each other. 00:00:40.440 --> 00:00:42.420 It was a fabulous suggestion. 00:00:42.420 --> 00:00:44.430 Today we can't do it. 00:00:44.430 --> 00:00:47.040 But I'm going to try to do it in the future whenever 00:00:47.040 --> 00:00:49.740 the logistics allow, because I think you're absolutely right. 00:00:49.740 --> 00:00:51.540 A little bit more of a U, or a circle, 00:00:51.540 --> 00:00:53.560 would be great for this class. 00:00:53.560 --> 00:00:57.060 So thank you for that suggestion. 00:00:57.060 --> 00:01:02.040 Another one was that we didn't get to motivation theory. 00:01:02.040 --> 00:01:04.080 And I think I mentioned this last time, 00:01:04.080 --> 00:01:07.650 the idea that there's a huge amount of stuff that's 00:01:07.650 --> 00:01:09.420 been written and published on motivation 00:01:09.420 --> 00:01:11.430 and theories of motivation. 00:01:11.430 --> 00:01:14.430 And I'd be more than happy to discuss this 00:01:14.430 --> 00:01:18.540 as the semester goes on. 00:01:18.540 --> 00:01:20.400 And it might be a great topic for one 00:01:20.400 --> 00:01:23.181 of those lunches that somebody was going to organize, 00:01:23.181 --> 00:01:24.930 one of those informal lunches that someone 00:01:24.930 --> 00:01:25.946 was going to organize. 00:01:29.910 --> 00:01:31.680 Or maybe we can carve out some space 00:01:31.680 --> 00:01:34.200 at the end of the semester and pop it in, take something out 00:01:34.200 --> 00:01:36.570 and pop it in, because it's really fascinating. 00:01:36.570 --> 00:01:38.730 It's really interesting, but it doesn't quite 00:01:38.730 --> 00:01:40.684 fit anywhere in particular. 00:01:40.684 --> 00:01:42.100 So thank you for bringing that up. 00:01:42.100 --> 00:01:44.050 And keep bugging me about it. 00:01:44.050 --> 00:01:50.070 The other thing is someone asked if all those learning 00:01:50.070 --> 00:01:52.610 theories that we discussed were equally valid. 00:01:52.610 --> 00:01:55.140 And yes, they are equally valid. 00:01:55.140 --> 00:02:00.960 I mean, there are situations where being behaviorist 00:02:00.960 --> 00:02:02.160 could help. 00:02:02.160 --> 00:02:05.910 There are situations where being constructivist could help. 00:02:05.910 --> 00:02:09.210 It depends on what you're trying to have students learn. 00:02:09.210 --> 00:02:13.110 So you want to make sure that you tailor the way that you're 00:02:13.110 --> 00:02:15.980 teaching to support what it is you want students to learn. 00:02:15.980 --> 00:02:18.480 And we'll get into that in a lot more detail 00:02:18.480 --> 00:02:22.270 today when we talk about developing learning outcomes, 00:02:22.270 --> 00:02:25.260 and then also when we talk about tailoring 00:02:25.260 --> 00:02:27.210 your instructional activities to support 00:02:27.210 --> 00:02:30.300 those learning outcomes. 00:02:30.300 --> 00:02:33.720 So this class, this particular class today 00:02:33.720 --> 00:02:39.726 is a balance between developing course design, developing 00:02:39.726 --> 00:02:41.100 intended learning outcomes, which 00:02:41.100 --> 00:02:42.870 for me is at the heart of it. 00:02:42.870 --> 00:02:45.180 If you can come up with some good intended 00:02:45.180 --> 00:02:48.974 learning outcomes, you basically have designed your course. 00:02:48.974 --> 00:02:49.890 And I'll show you how. 00:02:49.890 --> 00:02:54.420 I mean, you can't just ignore it, but that's the hard work, 00:02:54.420 --> 00:02:57.189 and then the act of writing constructing a syllabus. 00:02:57.189 --> 00:02:58.980 So once you've got those learning outcomes, 00:02:58.980 --> 00:03:02.460 you have an incredible anchor for the rest of your course. 00:03:02.460 --> 00:03:04.380 And there was just a very timely haiku. 00:03:04.380 --> 00:03:05.760 I put it on the wiki. 00:03:05.760 --> 00:03:10.860 But there's this Twitter feed, academic haikus, 00:03:10.860 --> 00:03:14.289 which if you want to waste any time and read them, 00:03:14.289 --> 00:03:15.330 it's a way to waste time. 00:03:15.330 --> 00:03:17.850 But there was one here, "that question you have? 00:03:17.850 --> 00:03:20.610 I know where the answer is-- on the syllabus. 00:03:20.610 --> 00:03:23.250 So it was, like I said, it came out yesterday. 00:03:23.250 --> 00:03:25.050 I thought it was pretty timely. 00:03:25.050 --> 00:03:25.950 And it's true. 00:03:25.950 --> 00:03:28.500 If you write a good syllabus, it should really clear up 00:03:28.500 --> 00:03:34.860 a lot of, not the content, not the understanding level issues 00:03:34.860 --> 00:03:37.230 students have, but the issues with respect 00:03:37.230 --> 00:03:40.710 to when's the test, do we have to know this, 00:03:40.710 --> 00:03:44.130 how will this be graded, what's important? 00:03:44.130 --> 00:03:46.990 All those things should be in your syllabus. 00:03:46.990 --> 00:03:49.590 So that's why that's there. 00:03:49.590 --> 00:03:54.570 So here's a little slide, a graph 00:03:54.570 --> 00:03:56.520 that I saw a few years ago, and I 00:03:56.520 --> 00:04:00.180 thought it's a really good way to start this discussion. 00:04:00.180 --> 00:04:05.550 So on the y-axis we have sort of an assessment, a proclamation 00:04:05.550 --> 00:04:07.020 of students understanding. 00:04:07.020 --> 00:04:09.030 They got it, they didn't get it. 00:04:09.030 --> 00:04:11.490 And I think the word in French is [FRENCH]. 00:04:11.490 --> 00:04:15.470 So for some reason I find it very cool to say [FRENCH]. 00:04:15.470 --> 00:04:16.519 Got it. 00:04:16.519 --> 00:04:17.950 I don't know. 00:04:17.950 --> 00:04:20.696 So whether they get it, like whether after the tests, 00:04:20.696 --> 00:04:23.070 you've given a test, and you go, man, that student really 00:04:23.070 --> 00:04:23.730 got it. 00:04:23.730 --> 00:04:27.300 That student really gets this material, 00:04:27.300 --> 00:04:31.440 and whether they passed the exam or failed the exam. 00:04:31.440 --> 00:04:34.260 So the size of the circle is more or less 00:04:34.260 --> 00:04:39.132 the number of people that fall into those quadrants. 00:04:41.910 --> 00:04:47.520 So anybody surprised by this plot? 00:04:51.020 --> 00:04:51.520 Yes, Rachel. 00:04:51.520 --> 00:04:54.500 AUDIENCE: [INAUDIBLE] 00:04:54.500 --> 00:04:55.480 JANET RANKIN: OK. 00:04:55.480 --> 00:04:56.000 All right. 00:04:56.000 --> 00:04:58.670 Well, there's the segue. 00:04:58.670 --> 00:05:02.210 So we want, we don't want the green, 00:05:02.210 --> 00:05:05.090 but we more or less expect the green, right? 00:05:05.090 --> 00:05:07.150 If you get it, you pass the exam. 00:05:07.150 --> 00:05:08.780 If you pass the exam, you get it. 00:05:08.780 --> 00:05:11.360 That's that, and we like to have lots of people 00:05:11.360 --> 00:05:13.220 have that happen too. 00:05:13.220 --> 00:05:15.627 And then we also acknowledge that somebody 00:05:15.627 --> 00:05:17.210 isn't going to get it, and they're not 00:05:17.210 --> 00:05:18.920 and they're not going to pass the exam. 00:05:18.920 --> 00:05:24.080 And that kind of maybe makes us sad, but it's not surprising. 00:05:24.080 --> 00:05:25.850 And then anybody can kind of have 00:05:25.850 --> 00:05:28.680 a bad day or kind of a fluke. 00:05:28.680 --> 00:05:30.830 So sometimes, sure, somebody will fail the exam, 00:05:30.830 --> 00:05:33.590 and actually really understand the material. 00:05:33.590 --> 00:05:35.780 Generally speaking, that doesn't happen, 00:05:35.780 --> 00:05:38.030 and then there's often a reason for it. 00:05:38.030 --> 00:05:40.220 But it's that red circle that Rachel 00:05:40.220 --> 00:05:47.354 mentioned that is troublesome, at the least, troublesome. 00:05:47.354 --> 00:05:48.395 So why might that happen? 00:05:55.170 --> 00:05:56.170 Why so many red circles? 00:05:56.170 --> 00:05:56.670 Yeah, Dave. 00:05:56.670 --> 00:05:59.340 AUDIENCE: Because the test was set up for basic memorization, 00:05:59.340 --> 00:06:01.416 or you could figure out with a lucky guess 00:06:01.416 --> 00:06:02.290 with multiple choice. 00:06:02.290 --> 00:06:05.225 So you could easily pass just with good test-taking skills 00:06:05.225 --> 00:06:06.081 or a good memory. 00:06:06.081 --> 00:06:07.080 JANET RANKIN: Excellent. 00:06:07.080 --> 00:06:09.330 So you pass with sort of good test taking skills. 00:06:09.330 --> 00:06:11.460 Now I want us to hang on to that. 00:06:11.460 --> 00:06:15.060 There's an underlying assumption in Dave's comment. 00:06:15.060 --> 00:06:17.000 So I'd like to hang on to that thought, 00:06:17.000 --> 00:06:19.857 but let's hear a few other conjectures and see. 00:06:19.857 --> 00:06:21.940 Rachel, why do you think you might have passed it? 00:06:21.940 --> 00:06:24.648 AUDIENCE: So often I feel like TAs are either 00:06:24.648 --> 00:06:26.106 really good or really terrible when 00:06:26.106 --> 00:06:28.681 it comes to the exam reviews, and they're mainly good. 00:06:28.681 --> 00:06:31.394 They often told us too much of what 00:06:31.394 --> 00:06:32.560 was going to be on the exam. 00:06:32.560 --> 00:06:34.350 And so then you kind of just knew 00:06:34.350 --> 00:06:36.952 what the test was going to be if you could well on it 00:06:36.952 --> 00:06:38.266 if you [INAUDIBLE]. 00:06:38.266 --> 00:06:39.550 JANET RANKIN: OK. 00:06:39.550 --> 00:06:40.660 That's interesting. 00:06:40.660 --> 00:06:41.484 Yes, Gordon. 00:06:41.484 --> 00:06:46.278 AUDIENCE: Yeah, it's possible that [INAUDIBLE] 00:06:46.278 --> 00:06:50.742 tests [INAUDIBLE]. 00:06:50.742 --> 00:06:54.230 When it comes to true life situations and something 00:06:54.230 --> 00:06:56.730 that [INAUDIBLE] knowledge that was 00:06:56.730 --> 00:07:00.674 gained from that material, then [INAUDIBLE]. 00:07:03.765 --> 00:07:04.860 JANET RANKIN: OK. 00:07:04.860 --> 00:07:13.020 So Gordon is sort of kind of flirting with this idea of you 00:07:13.020 --> 00:07:15.374 as the instructor, this real life knowledge, 00:07:15.374 --> 00:07:16.290 this is what it means. 00:07:16.290 --> 00:07:19.500 They have to have this practical working knowledge for us 00:07:19.500 --> 00:07:24.340 as the instructors to say they got it, right? 00:07:24.340 --> 00:07:26.650 But the exam isn't measuring it. 00:07:26.650 --> 00:07:29.800 And that's exactly what, that's what Dave was saying as well. 00:07:29.800 --> 00:07:32.920 This idea that the exam is measuring one thing, 00:07:32.920 --> 00:07:36.610 but it's not really measuring whether they got it. 00:07:36.610 --> 00:07:38.380 When we go home and think what does 00:07:38.380 --> 00:07:40.900 it mean to be a good material scientist, 00:07:40.900 --> 00:07:42.659 or what does it mean to be a good chemist, 00:07:42.659 --> 00:07:44.200 we don't think, oh, it means that you 00:07:44.200 --> 00:07:46.520 can define what a mitochondria is, 00:07:46.520 --> 00:07:49.790 or you can balance a simple equation. 00:07:52.530 --> 00:07:57.450 And yet, we put that on the exam. 00:07:57.450 --> 00:07:59.430 It's also a bit of the idea that students 00:07:59.430 --> 00:08:00.900 are good at taking tests. 00:08:00.900 --> 00:08:03.000 They're good at gaming systems. 00:08:03.000 --> 00:08:06.180 They're good at listening to TAs or TAs may have not given 00:08:06.180 --> 00:08:07.860 out appropriate information. 00:08:07.860 --> 00:08:09.750 So it comes from both sides. 00:08:09.750 --> 00:08:13.620 It comes from the students and it comes from the professors. 00:08:13.620 --> 00:08:16.920 But you have control over this. 00:08:16.920 --> 00:08:20.430 You shouldn't be writing an exam that people 00:08:20.430 --> 00:08:24.090 can get a good grade on, but then when you go home 00:08:24.090 --> 00:08:26.400 and say, oh, they got a good grade on this test, 00:08:26.400 --> 00:08:28.340 but they didn't get it. 00:08:28.340 --> 00:08:32.600 You need to define what it means to get it in your field, 00:08:32.600 --> 00:08:37.789 to get this subject, whatever it is that the test is about. 00:08:37.789 --> 00:08:42.890 And then you need to write a test that measures that. 00:08:42.890 --> 00:08:45.920 And too many times we don't do it. 00:08:45.920 --> 00:08:48.020 We're kind of rushed, or we don't really know. 00:08:48.020 --> 00:08:49.936 We haven't really thought about how to measure 00:08:49.936 --> 00:08:52.980 whether somebody gets it. 00:08:52.980 --> 00:08:56.540 So the first step in all this is to define what it 00:08:56.540 --> 00:09:00.520 is that means that they got it. 00:09:00.520 --> 00:09:03.680 And that's the first step, and that's the hardest step. 00:09:03.680 --> 00:09:06.590 But once that's done, the rest is easier, 00:09:06.590 --> 00:09:11.042 and it will save you from this. 00:09:11.042 --> 00:09:12.521 OK? 00:09:12.521 --> 00:09:15.490 OK. 00:09:15.490 --> 00:09:19.720 So that's the topic of today's class. 00:09:19.720 --> 00:09:22.860 And if you needed a little more motivation, 00:09:22.860 --> 00:09:26.170 I'd like you to read this paragraph, 00:09:26.170 --> 00:09:30.160 and I'd like you to tell me what it's talking about. 00:09:30.160 --> 00:09:32.800 But I'm just going to give you about 2 and 1/2 minutes 00:09:32.800 --> 00:09:34.390 to read it. 00:09:34.390 --> 00:09:36.924 And then we'll discuss what it means. 00:09:36.924 --> 00:09:38.590 What do you think this is talking about? 00:09:41.520 --> 00:09:42.020 Yes, David. 00:09:42.020 --> 00:09:44.330 AUDIENCE: Yeah, I think it stresses 00:09:44.330 --> 00:09:51.980 about how we can arrange by say, maybe schedules or activities 00:09:51.980 --> 00:09:54.782 that put them into groups. 00:09:54.782 --> 00:10:00.718 Then maybe if we form a group, how we plan [INAUDIBLE]. 00:10:04.566 --> 00:10:05.420 JANET RANKIN: OK. 00:10:05.420 --> 00:10:08.621 So arranging schedules or activities. 00:10:08.621 --> 00:10:09.120 OK. 00:10:24.870 --> 00:10:25.370 OK. 00:10:25.370 --> 00:10:28.350 Something else? 00:10:28.350 --> 00:10:29.940 AUDIENCE: When you're planning, how 00:10:29.940 --> 00:10:34.392 to go about teaching a module, [INAUDIBLE]. 00:10:42.858 --> 00:10:44.990 JANET RANKIN: Planning teaching. 00:10:44.990 --> 00:10:47.361 What else? 00:10:47.361 --> 00:10:47.860 Anything? 00:10:47.860 --> 00:10:49.660 AUDIENCE: It's like a mailroom operation or something. 00:10:49.660 --> 00:10:50.618 JANET RANKIN: Mailroom. 00:10:58.470 --> 00:11:00.674 Something from the back, maybe? 00:11:00.674 --> 00:11:02.488 You know you're smiling. 00:11:02.488 --> 00:11:04.400 AUDIENCE: I couldn't think of anything. 00:11:04.400 --> 00:11:05.400 JANET RANKIN: All right. 00:11:05.400 --> 00:11:08.040 Well, then don't say anything. 00:11:08.040 --> 00:11:13.590 AUDIENCE: The first is the course content, the course 00:11:13.590 --> 00:11:16.910 outline, and what you are going to teach 00:11:16.910 --> 00:11:18.798 the students, [INAUDIBLE]. 00:11:21.594 --> 00:11:22.580 JANET RANKIN: OK. 00:11:22.580 --> 00:11:24.920 So students, grouping students-- 00:11:24.920 --> 00:11:25.610 Adam, right? 00:11:25.610 --> 00:11:26.671 Adam. 00:11:26.671 --> 00:11:27.170 OK. 00:11:27.170 --> 00:11:39.477 So content, arranging content. 00:11:39.477 --> 00:11:40.310 Anybody in the back? 00:11:43.485 --> 00:11:43.985 Flyer? 00:11:47.430 --> 00:11:48.030 Yes, Gordon. 00:11:48.030 --> 00:11:50.652 AUDIENCE: I think you talked about sorting materials. 00:11:50.652 --> 00:11:52.100 JANET RANKIN: Sorting materials. 00:11:52.100 --> 00:11:53.875 So we'll go with maybe sorting. 00:11:59.880 --> 00:12:04.120 So if I tell you-- so these are all great guesses. 00:12:04.120 --> 00:12:06.930 They're as good as what I'm going 00:12:06.930 --> 00:12:09.090 to tell you the answer is. 00:12:09.090 --> 00:12:11.760 So this was written to describe the process 00:12:11.760 --> 00:12:12.765 of doing your laundry. 00:12:16.110 --> 00:12:17.490 Like doing the wash. 00:12:17.490 --> 00:12:19.440 You take the clothes and you put, maybe 00:12:19.440 --> 00:12:23.130 you put the dark clothes in one bin and the white clothes 00:12:23.130 --> 00:12:24.600 in another bin. 00:12:24.600 --> 00:12:28.680 And then you fold them and you put them away. 00:12:28.680 --> 00:12:29.370 It's impossible. 00:12:29.370 --> 00:12:31.650 This happens almost every time I show 00:12:31.650 --> 00:12:34.380 this little piece of writing. 00:12:34.380 --> 00:12:36.870 People don't generally get it. 00:12:36.870 --> 00:12:39.720 They get the fact that it has to do with sorting, 00:12:39.720 --> 00:12:43.380 and some planning, some arranging things, 00:12:43.380 --> 00:12:46.190 but they don't get that it's about the laundry. 00:12:46.190 --> 00:12:48.660 If you skim it-- you don't have to totally reread it-- 00:12:48.660 --> 00:12:51.450 but if you skim it with the idea that it's 00:12:51.450 --> 00:12:55.290 about doing the laundry, you probably 00:12:55.290 --> 00:13:03.860 can understand it better, maybe. 00:13:03.860 --> 00:13:10.860 So now I'm going to ask why did I show you this now, 00:13:10.860 --> 00:13:17.430 today, before the third, the beginning of the third class? 00:13:17.430 --> 00:13:17.930 Yes. 00:13:17.930 --> 00:13:20.878 AUDIENCE: I think it's sort of about arrangement of course 00:13:20.878 --> 00:13:24.091 content [INAUDIBLE]. 00:13:24.091 --> 00:13:26.410 JANET RANKIN: So it could be about how you arrange 00:13:26.410 --> 00:13:28.670 the content in the syllabus. 00:13:28.670 --> 00:13:29.170 [? Hina. ?] 00:13:29.170 --> 00:13:32.670 AUDIENCE: I think it's easy to describe methodology 00:13:32.670 --> 00:13:34.668 as a person more in a different field. 00:13:34.668 --> 00:13:37.905 But if there's no context for the audience about what 00:13:37.905 --> 00:13:42.140 this is, it doesn't matter that you've laid out everything. 00:13:42.140 --> 00:13:43.445 No one knows [INAUDIBLE]. 00:13:43.445 --> 00:13:44.350 JANET RANKIN: Right. 00:13:44.350 --> 00:13:46.780 It's very detailed at a certain level, 00:13:46.780 --> 00:13:52.420 but it's still really hard to understand, right? 00:13:52.420 --> 00:13:53.800 Any other ideas? 00:13:53.800 --> 00:13:54.300 Yes. 00:13:54.300 --> 00:13:56.300 AUDIENCE: I think it's interesting. 00:13:56.300 --> 00:13:58.300 There's a lot of value judgements in it. 00:13:58.300 --> 00:13:59.712 Like, it's really easy. 00:13:59.712 --> 00:14:02.450 And one can never tell with comments like that. 00:14:02.450 --> 00:14:05.052 Because I'm trained to figure out what they're saying, 00:14:05.052 --> 00:14:07.522 it's actually just frustrating me more, 00:14:07.522 --> 00:14:10.230 because I don't know what they're talking about. 00:14:10.230 --> 00:14:12.271 And it's really easy-- 00:14:12.271 --> 00:14:13.270 JANET RANKIN: All right. 00:14:13.270 --> 00:14:15.729 So that's a great point. 00:14:15.729 --> 00:14:17.770 So we're talking about writing learning outcomes, 00:14:17.770 --> 00:14:19.660 about organizing the course around writing 00:14:19.660 --> 00:14:20.570 learning outcomes. 00:14:20.570 --> 00:14:22.450 If you're using phrases, or you're 00:14:22.450 --> 00:14:24.820 using language that's just, there's 00:14:24.820 --> 00:14:28.960 just like throwaway words in there or value judgments, 00:14:28.960 --> 00:14:33.700 it's not doing your students any good, right? 00:14:33.700 --> 00:14:37.690 And also, that if you have this beautiful course that's 00:14:37.690 --> 00:14:41.320 beautifully organized even, wonderfully organized, 00:14:41.320 --> 00:14:44.230 but if you don't tell them what it is you're 00:14:44.230 --> 00:14:45.910 trying to do with it all-- 00:14:45.910 --> 00:14:47.680 what it is they're going to be able to do 00:14:47.680 --> 00:14:52.410 with it all, whatever it is you expect them to do-- 00:14:52.410 --> 00:14:55.600 they're not going to be able to follow you. 00:14:55.600 --> 00:14:58.300 So imagine a procedure as fairly straightforward 00:14:58.300 --> 00:15:00.640 as doing the laundry. 00:15:00.640 --> 00:15:02.650 Now imagine, OK, you're going to be 00:15:02.650 --> 00:15:05.950 able to select the right, the appropriate method 00:15:05.950 --> 00:15:14.440 to integrate equations, so methods of integration, 00:15:14.440 --> 00:15:20.770 by separation, by card or substitution or whatever. 00:15:20.770 --> 00:15:22.990 So if you don't tell them what it 00:15:22.990 --> 00:15:24.580 is you're expecting them to do, even 00:15:24.580 --> 00:15:26.746 if you're very good at explaining what it is they're 00:15:26.746 --> 00:15:30.740 going to do, they won't get it. 00:15:30.740 --> 00:15:34.570 So this is my sell for why when you write a learning outcome, 00:15:34.570 --> 00:15:36.940 you have to write a learning outcome that's clear 00:15:36.940 --> 00:15:40.090 and that's understandable to the students, that doesn't have 00:15:40.090 --> 00:15:42.130 a lot of extra words and that tells them 00:15:42.130 --> 00:15:44.220 exactly what they're supposed to do. 00:15:48.670 --> 00:15:54.520 So I have some learning outcomes for today 00:15:54.520 --> 00:15:56.680 and I hope that they're clear. 00:15:56.680 --> 00:15:59.110 You'll be able to describe the components 00:15:59.110 --> 00:16:03.640 of the constructive alignment and backward design processes. 00:16:03.640 --> 00:16:05.860 And those, constructive alignment anyway, 00:16:05.860 --> 00:16:08.290 was in the pre-class reading. 00:16:08.290 --> 00:16:10.810 You'll be able to classify the content of a course 00:16:10.810 --> 00:16:12.010 that you might teach. 00:16:12.010 --> 00:16:17.170 So what do I expect students to do with this content, 00:16:17.170 --> 00:16:18.770 to be able to do with this content? 00:16:18.770 --> 00:16:22.600 You'll be able to create measurable, specific, and 00:16:22.600 --> 00:16:26.530 realistic learning outcomes for a class you will teach, 00:16:26.530 --> 00:16:30.310 and you'll be able to state the components of a syllabus. 00:16:30.310 --> 00:16:33.280 So I feel that those are pretty understandable. 00:16:33.280 --> 00:16:39.500 You may disagree with me, or maybe you want to wait. 00:16:43.450 --> 00:16:46.720 So constructive alignment, it was in the pre-class reading, 00:16:46.720 --> 00:16:48.430 right? 00:16:48.430 --> 00:16:51.930 Anybody want to just give a brief summary of it? 00:16:54.720 --> 00:16:58.210 I mean it's kind of up here, but in your own-- 00:16:58.210 --> 00:17:00.020 AUDIENCE: I think constructive alignment 00:17:00.020 --> 00:17:09.468 is about aligning assessment, method of assessment, 00:17:09.468 --> 00:17:12.280 with course content. 00:17:12.280 --> 00:17:15.230 [INAUDIBLE] assess [INAUDIBLE] type of assessment 00:17:15.230 --> 00:17:18.670 so that what the teacher has in mind, the course 00:17:18.670 --> 00:17:21.973 content or the content I want them to get, 00:17:21.973 --> 00:17:24.329 to test it with the assessment process. 00:17:24.329 --> 00:17:27.209 Then we would also do the assessment, 00:17:27.209 --> 00:17:32.497 and then come up with the intended outcome. 00:17:32.497 --> 00:17:33.330 JANET RANKIN: Right. 00:17:33.330 --> 00:17:34.913 They'll achieve the learning outcomes. 00:17:34.913 --> 00:17:36.870 And if you do this right, you get rid 00:17:36.870 --> 00:17:40.380 of that red circle on that graph. 00:17:40.380 --> 00:17:44.280 You get rid of students who did well on the assessment 00:17:44.280 --> 00:17:46.680 but don't really know what's going on, 00:17:46.680 --> 00:17:49.320 because you have aligned the assessment 00:17:49.320 --> 00:17:53.310 with what it was you wanted them to know or be able to do. 00:17:53.310 --> 00:17:56.100 So that's important. 00:17:56.100 --> 00:17:59.130 I just think this is a very, very compelling image 00:17:59.130 --> 00:18:03.360 to keep in your head, that these three things have to be linked. 00:18:03.360 --> 00:18:05.964 And as I said before, this, you will 00:18:05.964 --> 00:18:08.130 see that the top part, writing the learning outcomes 00:18:08.130 --> 00:18:09.870 is the tricky part, it's the hard part. 00:18:09.870 --> 00:18:13.080 It's the part that requires a lot of thinking and work. 00:18:13.080 --> 00:18:15.010 When you get a good learning outcome, 00:18:15.010 --> 00:18:21.090 the other two things almost take care of themselves. 00:18:21.090 --> 00:18:22.790 OK? 00:18:22.790 --> 00:18:24.290 Did anybody have any, does anybody 00:18:24.290 --> 00:18:26.379 have any questions about something 00:18:26.379 --> 00:18:28.670 from the reading associated with constructive alignment 00:18:28.670 --> 00:18:33.689 or some other idea from the reading? 00:18:33.689 --> 00:18:34.188 Yes, Gordon. 00:18:34.188 --> 00:18:37.860 AUDIENCE: I just wanted to make a contribution. 00:18:37.860 --> 00:18:42.344 I think that this constructive alignment is one of the things 00:18:42.344 --> 00:18:46.005 that the intended outcome should measure. 00:18:46.005 --> 00:19:03.262 So I think maybe that's confusing [INAUDIBLE] 00:19:03.262 --> 00:19:06.496 and also how we can test and be sure and confident. 00:19:06.496 --> 00:19:07.650 JANET RANKIN: Right. 00:19:07.650 --> 00:19:08.580 Right. 00:19:08.580 --> 00:19:12.620 I think the measurability part for scientists and engineers 00:19:12.620 --> 00:19:15.390 is kind of intuitive. 00:19:15.390 --> 00:19:18.330 Like why would you say something if you couldn't 00:19:18.330 --> 00:19:21.470 tell whether or not it was true, or not 00:19:21.470 --> 00:19:24.060 from a scientific point of view. 00:19:24.060 --> 00:19:27.380 What's the point of saying, making 00:19:27.380 --> 00:19:28.930 a proclamation about what somebody 00:19:28.930 --> 00:19:31.860 should be able to do if you can't measure it in the end? 00:19:31.860 --> 00:19:34.764 So it's comforting to say, OK, I'm 00:19:34.764 --> 00:19:36.930 going to make learning outcomes that are measurable. 00:19:36.930 --> 00:19:40.050 But it also can be a bit challenging. 00:19:40.050 --> 00:19:41.010 But we'll do that. 00:19:41.010 --> 00:19:44.110 We're going to do that today. 00:19:44.110 --> 00:19:45.130 Some other comment? 00:19:45.130 --> 00:19:45.630 Nina. 00:19:45.630 --> 00:19:49.130 AUDIENCE: I guess as far as [INAUDIBLE] 00:19:49.130 --> 00:19:51.078 Do people write learning outcomes that 00:19:51.078 --> 00:19:54.850 have no constructive alignment process, 00:19:54.850 --> 00:19:57.402 like choosing to write learning outcomes, 00:19:57.402 --> 00:20:01.080 are you going through the prcoess? 00:20:01.080 --> 00:20:03.732 It seems decoupled a little bit in the reading, 00:20:03.732 --> 00:20:06.504 but it seems like you can't have one without the other. 00:20:06.504 --> 00:20:09.424 JANET RANKIN: You owe it to yourself-- 00:20:09.424 --> 00:20:11.590 and I don't know what people do, and I guarantee you 00:20:11.590 --> 00:20:13.540 people write learning outcomes without thinking about 00:20:13.540 --> 00:20:15.415 whether they can assess them or measure them, 00:20:15.415 --> 00:20:19.070 I guarantee that happens in real life-- 00:20:19.070 --> 00:20:22.500 you owe it to yourself to try very hard not to do that. 00:20:22.500 --> 00:20:23.170 OK? 00:20:23.170 --> 00:20:27.070 And I think the important thing to do is, often we say, 00:20:27.070 --> 00:20:29.260 oh, my god, this is the best assignment. 00:20:29.260 --> 00:20:31.300 This is the greatest question. 00:20:31.300 --> 00:20:33.400 It's this project, and they're going to do this, 00:20:33.400 --> 00:20:34.649 and they're going to whatever. 00:20:34.649 --> 00:20:36.970 They're going to build a race car. 00:20:36.970 --> 00:20:39.550 And if you sit down and say, well, what 00:20:39.550 --> 00:20:41.970 do I want them to get out of that? 00:20:41.970 --> 00:20:44.850 If you can't articulate it in a way that's consistent 00:20:44.850 --> 00:20:46.920 with your class, consistent with the course 00:20:46.920 --> 00:20:49.350 as it sits in this series of their courses, 00:20:49.350 --> 00:20:51.750 or as it sits in the institution, 00:20:51.750 --> 00:20:55.620 then you either need to rethink the assignment, 00:20:55.620 --> 00:20:59.340 rethink the project, or think a lot harder about what 00:20:59.340 --> 00:21:03.740 learning outcomes that activity advances. 00:21:03.740 --> 00:21:06.840 That's a very good point. 00:21:06.840 --> 00:21:08.640 It's really kind of iterative. 00:21:08.640 --> 00:21:11.970 And I think it might be easier to see with this diagram. 00:21:11.970 --> 00:21:15.660 This is another-- so John Biggs is from the UK. 00:21:15.660 --> 00:21:19.320 And so there's sort of the UK camp of people 00:21:19.320 --> 00:21:21.030 and they like John Biggs. 00:21:21.030 --> 00:21:23.070 And then there's Wiggins and McTighe. 00:21:23.070 --> 00:21:26.550 Grant Wiggins and Jay McTighe are from the US. 00:21:26.550 --> 00:21:29.910 They wrote primarily for the K through 12 audience, 00:21:29.910 --> 00:21:32.460 but it's completely applicable. 00:21:32.460 --> 00:21:35.310 And they're from the US, so they have kind of a US following. 00:21:35.310 --> 00:21:38.620 It's just how it works. 00:21:38.620 --> 00:21:40.810 It's essentially the same thing. 00:21:40.810 --> 00:21:44.520 It's identify the results, so the intended learning outcomes. 00:21:44.520 --> 00:21:47.070 Figure out how you're going to get them there. 00:21:47.070 --> 00:21:51.100 So the black is Wiggins and McTighe and the red is Biggs. 00:21:51.100 --> 00:21:53.220 So plan the learning experiences that 00:21:53.220 --> 00:21:55.410 are going to support them, achieving the learning 00:21:55.410 --> 00:21:59.220 outcomes, and really help you measure whether they got them 00:21:59.220 --> 00:22:00.310 or not. 00:22:00.310 --> 00:22:02.672 And then determine what evidence will 00:22:02.672 --> 00:22:05.130 be that acceptable evidence that they achieved the learning 00:22:05.130 --> 00:22:07.320 outcomes and what are the assessments. 00:22:07.320 --> 00:22:12.750 So real often we do have this project that we want. 00:22:12.750 --> 00:22:14.130 Oh, this is a great problem. 00:22:14.130 --> 00:22:17.490 Often in thermo-- I taught a lot of thermo-- there's like, oh, 00:22:17.490 --> 00:22:21.150 they're going to do this double loop, double reactor 00:22:21.150 --> 00:22:25.184 cycle with the refrigerant and a heating cycle. 00:22:25.184 --> 00:22:26.850 And it's going to be coupled and they're 00:22:26.850 --> 00:22:29.870 going to use the output from one for the input in the other 00:22:29.870 --> 00:22:31.350 and all this crazy stuff. 00:22:31.350 --> 00:22:34.680 Like, this is a great assignment. 00:22:34.680 --> 00:22:36.990 And you kind of, I know this is a great assignment. 00:22:36.990 --> 00:22:40.290 If I can't back up from this assignment-- 00:22:40.290 --> 00:22:42.330 whether or not I put it here or here-- 00:22:42.330 --> 00:22:46.490 if I can't back up and say after a successful completion 00:22:46.490 --> 00:22:50.730 of this assignment, you will be able to describe 00:22:50.730 --> 00:22:55.480 the components of the Brayton cycle, 00:22:55.480 --> 00:22:59.730 you will be able to define the increase in entropy 00:22:59.730 --> 00:23:05.250 for a system with x boundaries, If I can't articulate 00:23:05.250 --> 00:23:08.550 those things, then I need to either change the assignment, 00:23:08.550 --> 00:23:11.430 or, well, then I need to change the assignment. 00:23:11.430 --> 00:23:12.180 OK? 00:23:12.180 --> 00:23:14.640 But you can start with the activity, 00:23:14.640 --> 00:23:16.140 and then back up and say, well, what 00:23:16.140 --> 00:23:18.960 learning outcomes does that support, and then, 00:23:18.960 --> 00:23:19.890 am I OK with that? 00:23:19.890 --> 00:23:23.140 Is that relevant to this class? 00:23:23.140 --> 00:23:26.370 So it's often a very iterative, the arrows kind of go this way, 00:23:26.370 --> 00:23:28.038 but it's often iterative. 00:23:31.320 --> 00:23:37.800 So I had asked that you come to class with some topics 00:23:37.800 --> 00:23:39.570 from a class you're likely to teach. 00:23:39.570 --> 00:23:41.340 That was on the assignment page. 00:23:41.340 --> 00:23:45.300 If not, start thinking now. 00:23:45.300 --> 00:23:47.970 And so you want to have, for most classes, most 00:23:47.970 --> 00:23:50.550 undergraduate classes, there's a bunch of topics. 00:23:50.550 --> 00:23:53.910 You might have 10 topics, 20 topics. 00:23:53.910 --> 00:23:57.796 And I don't care right now about the scale of the topics, 00:23:57.796 --> 00:23:58.920 whether they're really big. 00:23:58.920 --> 00:24:01.140 You probably don't want to go with like first law, 00:24:01.140 --> 00:24:03.000 second law. 00:24:03.000 --> 00:24:07.130 That's probably a little bit too big, 00:24:07.130 --> 00:24:12.360 but on the order of 5 to 15 topics. 00:24:12.360 --> 00:24:14.130 Jot down the topics. 00:24:14.130 --> 00:24:16.710 And then I have a chart here. 00:24:16.710 --> 00:24:21.450 So the chart has two sides, and the columns 00:24:21.450 --> 00:24:23.700 are Bloom's Taxonomy, which you don't 00:24:23.700 --> 00:24:25.770 have to worry about that specifically, 00:24:25.770 --> 00:24:29.800 but you can look at the words that are associated with it. 00:24:29.800 --> 00:24:34.170 And then the second column is what students should 00:24:34.170 --> 00:24:36.960 be able to do with each topic. 00:24:36.960 --> 00:24:45.660 So let's say my topic is first law for closed systems. 00:24:45.660 --> 00:24:48.480 So I might think about, OK, do I want 00:24:48.480 --> 00:24:51.150 them to be able to define, to just state 00:24:51.150 --> 00:24:53.685 the first law for closed systems? 00:24:53.685 --> 00:24:56.550 Do I want them to be able to use the first law for closed 00:24:56.550 --> 00:24:59.520 systems in order to solve certain kinds of equations? 00:24:59.520 --> 00:25:03.270 Do I want them to be able to derive the first law? 00:25:03.270 --> 00:25:06.510 Do I want them to be able, whatever it is. 00:25:06.510 --> 00:25:10.110 So you're going to look at the columns, the categories 00:25:10.110 --> 00:25:12.657 on the left side, and you're going 00:25:12.657 --> 00:25:13.740 to think about your topic. 00:25:13.740 --> 00:25:16.830 And you're just going to decide which box in the middle column 00:25:16.830 --> 00:25:18.802 to put the topics into. 00:25:18.802 --> 00:25:20.760 You don't have to worry about writing anything. 00:25:20.760 --> 00:25:26.430 You're just going to distribute the topics to the level of, 00:25:26.430 --> 00:25:29.534 it's really cognitive processes that are associated 00:25:29.534 --> 00:25:30.700 with the things on the left. 00:25:30.700 --> 00:25:36.990 Remember, understand, apply, analyze, evaluate, and create. 00:25:36.990 --> 00:25:39.460 So we'll take about five minutes. 00:25:39.460 --> 00:25:41.230 And you just do that by yourself. 00:25:41.230 --> 00:25:43.170 And just fill in that middle column 00:25:43.170 --> 00:25:49.050 of the table with the topics from your course. 00:25:49.050 --> 00:25:50.890 Do you want more explanation, Rachel? 00:25:55.280 --> 00:25:59.240 It's starting to make you think about this part 00:25:59.240 --> 00:26:05.020 of the beginning circle. 00:26:05.020 --> 00:26:07.440 But you don't want to write any sentences yet. 00:26:07.440 --> 00:26:10.460 But it's starting to help you think about what are they 00:26:10.460 --> 00:26:13.830 going to do with these topics. 00:26:13.830 --> 00:26:15.130 What do I want them to do? 00:26:15.130 --> 00:26:20.350 Is it remember, or is it create something using these things? 00:26:20.350 --> 00:26:21.290 Where is it? 00:26:21.290 --> 00:26:22.970 Those are two ends of the spectrum. 00:26:22.970 --> 00:26:27.110 And in your chart, the things that at the top 00:26:27.110 --> 00:26:28.940 are the more remember, right? 00:26:28.940 --> 00:26:33.260 It's more about kind of a recall information. 00:26:33.260 --> 00:26:35.360 And then the things in the back of the chart 00:26:35.360 --> 00:26:40.160 tend to be more higher order cognitive processes, 00:26:40.160 --> 00:26:43.040 like synthesize, apply, create. 00:26:50.110 --> 00:26:50.609 Yes. 00:26:50.609 --> 00:26:54.281 AUDIENCE: [INAUDIBLE] 00:26:54.281 --> 00:26:57.050 JANET RANKIN: So don't worry about the third column. 00:26:57.050 --> 00:26:58.940 Don't worry about the last column yet. 00:26:58.940 --> 00:27:00.326 We're going to do that later. 00:27:00.326 --> 00:27:01.700 So all you have to do is just say 00:27:01.700 --> 00:27:05.410 where each topic falls loosely. 00:27:05.410 --> 00:27:05.910 OK. 00:27:05.910 --> 00:27:08.302 Does everybody have a chart? 00:27:08.302 --> 00:27:08.802 OK. 00:27:08.802 --> 00:27:10.730 AUDIENCE: Do you want us to write 00:27:10.730 --> 00:27:12.284 as many topics as possible? 00:27:12.284 --> 00:27:13.700 JANET RANKIN: Write as many topics 00:27:13.700 --> 00:27:17.990 as possible, but distribute them through the chart. 00:27:17.990 --> 00:27:19.460 Right? 00:27:19.460 --> 00:27:22.352 Not everything is going to be remember, right? 00:27:22.352 --> 00:27:24.810 So some things will be remember, some things will be apply, 00:27:24.810 --> 00:27:27.841 some things will be analyze, some things will be create. 00:27:27.841 --> 00:27:28.340 OK? 00:27:31.950 --> 00:27:34.451 If you have any questions, just raise your hand. 00:27:34.451 --> 00:27:34.950 All right. 00:27:34.950 --> 00:27:36.533 So what we're going to do is I'm going 00:27:36.533 --> 00:27:40.650 to ask you to just keep those with you. 00:27:40.650 --> 00:27:42.630 We're going to do another activity later 00:27:42.630 --> 00:27:44.670 on using those as a starting point, 00:27:44.670 --> 00:27:49.860 but I want you to just keep that sorting with you. 00:27:49.860 --> 00:27:52.800 Any comments or questions about the experience, 00:27:52.800 --> 00:27:53.860 about what you just did? 00:27:58.330 --> 00:28:00.820 Was it easy? 00:28:00.820 --> 00:28:02.980 No. 00:28:02.980 --> 00:28:04.349 It can sometimes be a challenge. 00:28:04.349 --> 00:28:05.890 Well, what the heck you were supposed 00:28:05.890 --> 00:28:07.002 to do with this anyway? 00:28:07.002 --> 00:28:08.710 What do I really want them to do with it? 00:28:08.710 --> 00:28:12.460 This is the beginning of that process whereby 00:28:12.460 --> 00:28:15.640 you define exactly what it is you want students to do, 00:28:15.640 --> 00:28:17.532 know or be able to do. 00:28:17.532 --> 00:28:19.240 And then you can start to think about how 00:28:19.240 --> 00:28:22.360 you're going to assess whether or not they can do it or not. 00:28:22.360 --> 00:28:24.460 But it's very different if I were 00:28:24.460 --> 00:28:27.040 to say I want everybody to be able to state 00:28:27.040 --> 00:28:29.994 the first law of thermodynamics versus I 00:28:29.994 --> 00:28:32.410 want you to be able to use the first law of thermodynamics 00:28:32.410 --> 00:28:37.480 to define chemical, mechanical, and thermal equilibrium 00:28:37.480 --> 00:28:40.672 in a simple composite system. 00:28:40.672 --> 00:28:41.380 That's different. 00:28:41.380 --> 00:28:42.606 That's very different. 00:28:46.110 --> 00:28:47.750 So that's what we're starting to do. 00:28:51.170 --> 00:28:53.440 So now we have kind of the seeds. 00:28:53.440 --> 00:28:59.530 We thought about which topics kind of we want to have, 00:28:59.530 --> 00:29:01.690 what we want to do with each of the topics. 00:29:01.690 --> 00:29:04.910 But now we really have to turn them into learning outcomes, 00:29:04.910 --> 00:29:06.274 into real statements. 00:29:06.274 --> 00:29:07.690 By the end of this class, you will 00:29:07.690 --> 00:29:13.600 be able to, by the end of this course, you will be able to. 00:29:13.600 --> 00:29:17.980 And just as an aside, remember that sometimes you'll 00:29:17.980 --> 00:29:20.620 notice that for every class, every session we have, 00:29:20.620 --> 00:29:22.510 I have learning outcomes. 00:29:22.510 --> 00:29:24.940 But there's also overarching learning outcomes 00:29:24.940 --> 00:29:26.400 for the course. 00:29:26.400 --> 00:29:31.350 So I will use them interchangeably, 00:29:31.350 --> 00:29:34.360 but they'll be at a slightly different scale. 00:29:34.360 --> 00:29:39.160 The ones for the class are a little bit more fine grained. 00:29:39.160 --> 00:29:41.710 As was mentioned in the reading, and as Gordon mentioned, 00:29:41.710 --> 00:29:46.660 they have to be specific, measurable, and realistic. 00:29:46.660 --> 00:29:49.180 If they don't have those properties 00:29:49.180 --> 00:29:52.420 they're not going to be particularly useful, or as 00:29:52.420 --> 00:29:54.220 useful. 00:29:54.220 --> 00:29:55.550 All right. 00:29:55.550 --> 00:30:00.550 And we'll see some examples of that in just a sec. 00:30:00.550 --> 00:30:02.800 So here's what they're not. 00:30:02.800 --> 00:30:04.810 They're not topics. 00:30:04.810 --> 00:30:06.760 So you came with topics. 00:30:06.760 --> 00:30:10.420 You will leave today with learning outcomes. 00:30:10.420 --> 00:30:14.470 But topics are not learning outcomes. 00:30:14.470 --> 00:30:18.430 The first law of thermodynamics is not a learning outcome. 00:30:18.430 --> 00:30:20.650 You need a verb with it. 00:30:20.650 --> 00:30:22.369 What do you want me to do, what do you 00:30:22.369 --> 00:30:24.160 want your students to do with the first law 00:30:24.160 --> 00:30:25.980 of thermodynamics? 00:30:25.980 --> 00:30:28.090 What do you want them to know about the first law 00:30:28.090 --> 00:30:30.410 of thermodynamics? 00:30:30.410 --> 00:30:32.980 So they're not topics. 00:30:32.980 --> 00:30:34.765 They're not things that you will do. 00:30:37.540 --> 00:30:40.900 It's easy to say, I could have a list a mile long. 00:30:40.900 --> 00:30:47.950 I will talk about the first law of thermodynamics. 00:30:47.950 --> 00:30:49.660 I can check that off pretty easily. 00:30:49.660 --> 00:30:50.734 I mean I just did. 00:30:50.734 --> 00:30:52.900 I just talked about the first law of thermodynamics. 00:30:52.900 --> 00:30:56.410 It says nothing about what you learned about it, what 00:30:56.410 --> 00:30:58.090 your students learned about it. 00:30:58.090 --> 00:31:00.430 So it's not about you. 00:31:00.430 --> 00:31:02.840 Get over it. 00:31:02.840 --> 00:31:03.506 That was a joke. 00:31:06.447 --> 00:31:07.530 It's not value statements. 00:31:07.530 --> 00:31:11.490 As Adrian pointed out, some of these think this is easy, 00:31:11.490 --> 00:31:16.640 this is good, this is useful. 00:31:16.640 --> 00:31:19.135 Any of those things don't belong in a learning outcome. 00:31:22.760 --> 00:31:25.590 And it's not your hopes and dreams for your students. 00:31:25.590 --> 00:31:28.074 I mean, ultimately, it is. 00:31:28.074 --> 00:31:30.240 You hope that they will be able to use the first law 00:31:30.240 --> 00:31:32.340 of thermodynamics, da, da, da. 00:31:32.340 --> 00:31:34.440 But it's really about what they're doing, 00:31:34.440 --> 00:31:36.240 about what they're going to do. 00:31:36.240 --> 00:31:39.500 So, no. 00:31:39.500 --> 00:31:40.610 Here's a little example. 00:31:43.140 --> 00:31:45.990 This is just from like an earth and planetary course. 00:31:45.990 --> 00:31:49.700 Students will understand plate tectonics. 00:31:49.700 --> 00:31:50.640 How's that? 00:31:54.400 --> 00:31:57.520 Is that OK for a learning outcome? 00:31:57.520 --> 00:31:58.438 Why not? 00:31:58.438 --> 00:32:01.924 AUDIENCE: It's too broad. 00:32:01.924 --> 00:32:04.140 JANET RANKIN: It's not very specific. 00:32:04.140 --> 00:32:07.435 What the heck about plate tectonics? 00:32:07.435 --> 00:32:07.935 What else? 00:32:11.620 --> 00:32:13.580 AUDIENCE: Measurable. 00:32:13.580 --> 00:32:14.760 JANET RANKIN: Thank you. 00:32:14.760 --> 00:32:16.890 OK, the measurable part. 00:32:16.890 --> 00:32:23.310 This word understand, strike it from your brains. 00:32:23.310 --> 00:32:25.150 It's a property that we love. 00:32:25.150 --> 00:32:27.120 We love understand. 00:32:27.120 --> 00:32:30.590 But you can't flippin' measure it. 00:32:30.590 --> 00:32:32.840 You can't go in there and measure it. 00:32:32.840 --> 00:32:35.180 You can measure lots of things that tell you 00:32:35.180 --> 00:32:37.220 whether students understand, but you 00:32:37.220 --> 00:32:39.500 can't measure understanding. 00:32:42.180 --> 00:32:44.307 And I think we'll see that with some more examples. 00:32:44.307 --> 00:32:46.390 I don't think anybody is earth and planetary here. 00:32:49.050 --> 00:32:50.664 So how about this one? 00:32:50.664 --> 00:32:52.080 Students will be able to interpret 00:32:52.080 --> 00:32:54.240 unfamiliar tectonic settings based 00:32:54.240 --> 00:32:59.830 on information on volcanic activity and seismicity. 00:32:59.830 --> 00:33:02.340 Again, I don't think any of us are geologists. 00:33:02.340 --> 00:33:04.980 But the idea that I've told, or the person 00:33:04.980 --> 00:33:08.310 has told us what we're going to use 00:33:08.310 --> 00:33:12.060 to interpret these situations, and then we're 00:33:12.060 --> 00:33:15.417 going to make predictions based on those interpretations, I 00:33:15.417 --> 00:33:17.610 mean, that's an assumption. 00:33:17.610 --> 00:33:21.426 So it's specific and now it's measurable. 00:33:21.426 --> 00:33:22.800 Even though we're not geologists, 00:33:22.800 --> 00:33:24.750 I suspect we can think of how we could 00:33:24.750 --> 00:33:29.220 write a test question that would ask students to do this. 00:33:29.220 --> 00:33:32.910 And then we could see whether they did it or not. 00:33:32.910 --> 00:33:35.370 So that's measurable. 00:33:35.370 --> 00:33:40.715 And probably, by doing this, by having students do this, 00:33:40.715 --> 00:33:42.840 we get a pretty good measurement about whether they 00:33:42.840 --> 00:33:46.900 understand plate tectonics. 00:33:46.900 --> 00:33:50.877 But it's much more specific and it's much more measurable. 00:33:58.944 --> 00:34:01.110 From here, this is from the University of Minnesota, 00:34:01.110 --> 00:34:03.590 which has a nice website on learning outcomes. 00:34:03.590 --> 00:34:06.680 They're not from your point of view, as we mentioned. 00:34:06.680 --> 00:34:08.179 They're not what you're going to do. 00:34:08.179 --> 00:34:10.345 I'm going to introduce students to the major turning 00:34:10.345 --> 00:34:13.747 points and processes in North American history. 00:34:13.747 --> 00:34:15.830 You can say anything you want, but it doesn't mean 00:34:15.830 --> 00:34:17.340 the students learned anything. 00:34:17.340 --> 00:34:18.510 So it's not about you. 00:34:18.510 --> 00:34:19.760 It's what they're going to do. 00:34:19.760 --> 00:34:21.777 They're going to list and describe 00:34:21.777 --> 00:34:24.235 the turning points and processes in North American history. 00:34:26.871 --> 00:34:28.370 I'm going to create an understanding 00:34:28.370 --> 00:34:30.230 of the formal constructs of physical design. 00:34:30.230 --> 00:34:32.600 That one is like, this one is, it's 00:34:32.600 --> 00:34:35.120 like, I don't even know what it means, right? 00:34:35.120 --> 00:34:37.719 Create an understanding of the formal constructs 00:34:37.719 --> 00:34:38.510 of physical design. 00:34:38.510 --> 00:34:40.670 I don't even know what it means. 00:34:40.670 --> 00:34:42.739 And so it's like that laundry example. 00:34:42.739 --> 00:34:46.010 Your students aren't going to know what it means, 00:34:46.010 --> 00:34:48.330 and it's not going to help them. 00:34:48.330 --> 00:34:51.469 It's not going to help them access the material. 00:34:51.469 --> 00:34:56.000 So the ones on the left are all teacher-centered 00:34:56.000 --> 00:34:58.250 and they're not very useful. 00:34:58.250 --> 00:35:00.410 The ones on the right are student-centered 00:35:00.410 --> 00:35:03.020 and they are generally useful. 00:35:03.020 --> 00:35:06.230 But there's some things wrong with the things on the right. 00:35:06.230 --> 00:35:07.700 AUDIENCE: Understand. 00:35:07.700 --> 00:35:12.264 JANET RANKIN: Understand, yes. 00:35:12.264 --> 00:35:13.680 So what would you write instead of 00:35:13.680 --> 00:35:17.370 understand the formal constructs of physical design, whatever 00:35:17.370 --> 00:35:19.440 the heck that means? 00:35:19.440 --> 00:35:20.554 What might you use? 00:35:20.554 --> 00:35:22.250 AUDIENCE: Identify. 00:35:22.250 --> 00:35:24.170 JANET RANKIN: Identify. 00:35:24.170 --> 00:35:24.840 OK. 00:35:24.840 --> 00:35:30.510 That might be like a list, right, or identify or list, 00:35:30.510 --> 00:35:32.275 pick it out of the lineup kind of thing. 00:35:34.830 --> 00:35:36.960 This is a nice one. 00:35:36.960 --> 00:35:40.220 Explain to an intelligent non-expert, an INE. 00:35:40.220 --> 00:35:45.240 And so explain this to somebody who's not in the field. 00:35:45.240 --> 00:35:49.200 If you could do that, it probably means that you get it. 00:35:49.200 --> 00:35:51.300 Somebody I know used to say, explain this 00:35:51.300 --> 00:35:54.810 to my four-year-old brother. 00:35:54.810 --> 00:35:58.050 Explain-- what was it-- angular momentum, 00:35:58.050 --> 00:36:01.290 to my four-year-old brother. 00:36:01.290 --> 00:36:04.380 I would argue if you can explain it to a four-year-old kid, 00:36:04.380 --> 00:36:07.050 you probably understand it. 00:36:07.050 --> 00:36:08.820 So you can use that. 00:36:13.400 --> 00:36:21.690 Use formal constructs to design something. 00:36:21.690 --> 00:36:23.677 I mean, that's kind of an inversion. 00:36:23.677 --> 00:36:25.010 How about the one on the bottom. 00:36:25.010 --> 00:36:29.450 Understand gender, race, ethnicity, socioeconomic class, 00:36:29.450 --> 00:36:32.120 understand how they have shaped Americans' lives. 00:36:32.120 --> 00:36:36.272 Completely unmeasurable, right? 00:36:36.272 --> 00:36:37.605 So what would you write instead? 00:36:41.830 --> 00:36:43.770 AUDIENCE: Describe. 00:36:43.770 --> 00:36:46.810 JANET RANKIN: Describe how, sure. 00:36:49.970 --> 00:36:50.695 Anything else? 00:36:50.695 --> 00:36:52.445 AUDIENCE: [INAUDIBLE] it seems like you're 00:36:52.445 --> 00:36:54.905 trying to get [INAUDIBLE]. 00:36:54.905 --> 00:36:57.030 JANET RANKIN: Right. 00:36:57.030 --> 00:36:58.720 So that's a great point, is that we 00:36:58.720 --> 00:37:05.020 don't know where this faculty member put this on their table. 00:37:05.020 --> 00:37:07.370 We don't know whether it really just states 00:37:07.370 --> 00:37:11.590 some ways that, list some ways that these things have affected 00:37:11.590 --> 00:37:15.790 Americans, North Americans' lives, or Americans' lives-- 00:37:15.790 --> 00:37:18.580 which is kind of a remember thing, just make a list-- 00:37:18.580 --> 00:37:22.780 or whether it's do some analysis on some situation. 00:37:22.780 --> 00:37:25.120 We don't know that, but you as the instructors 00:37:25.120 --> 00:37:28.512 get to decide that for your classes. 00:37:28.512 --> 00:37:29.720 So that's an important thing. 00:37:29.720 --> 00:37:32.480 You get to call all the shots. 00:37:32.480 --> 00:37:33.730 You get to make the decision. 00:37:33.730 --> 00:37:36.410 We can only guess about what this person meant, 00:37:36.410 --> 00:37:38.090 and therefore, his or her students 00:37:38.090 --> 00:37:40.420 can only guess about what he or she means. 00:37:46.010 --> 00:37:50.870 So you have on your table this hierarchy of Bloom's Taxonomy. 00:37:50.870 --> 00:37:55.460 It's often shown in this triangle, this pyramid. 00:37:55.460 --> 00:38:01.310 There's a handout which has exactly the same strata on it. 00:38:01.310 --> 00:38:06.140 And it just has some handy dandy verbs associated with it. 00:38:06.140 --> 00:38:09.530 But Benjamin Bloom came up with this classification 00:38:09.530 --> 00:38:15.390 of cognitive processes way back when, in the 1950s. 00:38:15.390 --> 00:38:17.510 And it is this idea that it is this pyramid, that 00:38:17.510 --> 00:38:20.840 at the base of the pyramid are these ideas like remember. 00:38:20.840 --> 00:38:27.140 And those are basic cognitive processes, remember, list. 00:38:27.140 --> 00:38:33.200 And then it moves up the pyramid to things that require a higher 00:38:33.200 --> 00:38:35.890 level of cognitive processing. 00:38:35.890 --> 00:38:38.860 Rather than just pulling facts out of storage 00:38:38.860 --> 00:38:41.110 and stating them, you're going to do something 00:38:41.110 --> 00:38:42.730 with those facts. 00:38:42.730 --> 00:38:46.820 So as you go higher and higher up the pyramid, 00:38:46.820 --> 00:38:49.090 note that he chose-- 00:38:49.090 --> 00:38:53.860 really unfortunately-- to call the second level understand. 00:38:53.860 --> 00:38:57.460 However, there are words that can help you. 00:38:57.460 --> 00:39:00.880 If it really is understand that you want to measure, 00:39:00.880 --> 00:39:03.400 there's words that describe that a little bit better. 00:39:03.400 --> 00:39:04.990 They're in your table, and they're 00:39:04.990 --> 00:39:09.280 also in that second table that I handed out. 00:39:09.280 --> 00:39:11.650 So arrange, list, label. 00:39:11.650 --> 00:39:16.310 For understand-- describe, relate, recognize, explain, 00:39:16.310 --> 00:39:17.020 those things. 00:39:17.020 --> 00:39:17.520 Gordon. 00:39:17.520 --> 00:39:20.204 AUDIENCE: Just a little [INAUDIBLE]. 00:39:20.204 --> 00:39:22.180 When it comes to understand, maybe 00:39:22.180 --> 00:39:24.128 if you put it like [INAUDIBLE]. 00:39:24.128 --> 00:39:25.503 JANET RANKIN: Understand to what? 00:39:25.503 --> 00:39:27.242 AUDIENCE: Understand to an [INAUDIBLE]. 00:39:27.242 --> 00:39:30.530 Maybe that's what we can measure. 00:39:30.530 --> 00:39:33.925 Understand to this level or to this [INAUDIBLE]. 00:39:33.925 --> 00:39:37.316 JANET RANKIN: But how, can you give me a specific example? 00:39:37.316 --> 00:39:53.536 AUDIENCE: I wish I could go back to the last slide [INAUDIBLE] 00:39:53.536 --> 00:39:56.500 can describe [INAUDIBLE]. 00:39:59.958 --> 00:40:01.140 JANET RANKIN: OK. 00:40:01.140 --> 00:40:02.850 So I see where you're going. 00:40:02.850 --> 00:40:04.500 Does someone have a suggestion? 00:40:04.500 --> 00:40:06.010 So think about that laundry. 00:40:06.010 --> 00:40:08.010 Think about that description of doing a laundry. 00:40:10.530 --> 00:40:12.520 Can somebody take what Gordon said 00:40:12.520 --> 00:40:18.640 and get where he wants to go, but get there in perhaps 00:40:18.640 --> 00:40:19.485 a more direct way. 00:40:25.751 --> 00:40:26.251 Katherine. 00:40:26.251 --> 00:40:29.730 AUDIENCE: [INAUDIBLE] 00:40:29.730 --> 00:40:31.410 JANET RANKIN: Exactly. 00:40:31.410 --> 00:40:32.910 So if you're going to do that, and I 00:40:32.910 --> 00:40:35.422 see where you're coming from, understand 00:40:35.422 --> 00:40:37.380 so that you can describe, in order to describe, 00:40:37.380 --> 00:40:38.580 or to the level of describe. 00:40:38.580 --> 00:40:40.530 Just forget the understand part, and just 00:40:40.530 --> 00:40:45.900 say describe, or define, or whatever it is you want. 00:40:45.900 --> 00:40:48.790 But yes, I mean, that first step is saying, 00:40:48.790 --> 00:40:50.820 what do I mean by understand? 00:40:50.820 --> 00:40:57.570 And when you say that, then it generally rewrites itself. 00:40:57.570 --> 00:41:01.650 And so the more direct you can be, generally, the better. 00:41:01.650 --> 00:41:04.410 And keep that laundry example in your head. 00:41:04.410 --> 00:41:04.910 Yes. 00:41:04.910 --> 00:41:10.360 AUDIENCE: You're asking a student [INAUDIBLE]? 00:41:10.360 --> 00:41:14.729 JANET RANKIN: So I would say, well, let's just open that up. 00:41:14.729 --> 00:41:16.770 What do you think about that, of asking students? 00:41:16.770 --> 00:41:21.720 So do you understand this stuff about learning outcomes? 00:41:21.720 --> 00:41:23.790 AUDIENCE: I think as teachers, when 00:41:23.790 --> 00:41:26.123 you, in the front of the class, you're teaching students 00:41:26.123 --> 00:41:28.740 and you say do you understand, they say yes. 00:41:28.740 --> 00:41:34.290 But you see from psychology you know who doesn't understand. 00:41:34.290 --> 00:41:36.870 JANET RANKIN: So David says as I ask you 00:41:36.870 --> 00:41:38.970 that, do you understand? 00:41:38.970 --> 00:41:41.970 Most people are going to say, hey, I understand, sure. 00:41:41.970 --> 00:41:43.290 Don't think I'm an idiot. 00:41:43.290 --> 00:41:45.930 I understand. 00:41:45.930 --> 00:41:47.670 Right? 00:41:47.670 --> 00:41:48.750 So I completely agree. 00:41:48.750 --> 00:41:51.990 Students are not likely to say they don't understand. 00:41:51.990 --> 00:41:53.310 What else? 00:41:53.310 --> 00:42:18.205 AUDIENCE: That's difficult for the teacher to [INAUDIBLE] 00:42:18.205 --> 00:42:20.782 JANET RANKIN: Well, so do you want 00:42:20.782 --> 00:42:21.948 to say something about that? 00:42:21.948 --> 00:42:23.119 AUDIENCE: Yeah, I agree. 00:42:23.119 --> 00:42:24.702 I think the part that we do understand 00:42:24.702 --> 00:42:30.546 as nonspecific is the learning understanding [INAUDIBLE]. 00:42:30.546 --> 00:42:33.450 As an instructor you have to know, understand 00:42:33.450 --> 00:42:35.670 what you're trying to get at, which 00:42:35.670 --> 00:42:40.010 I think is where the clicker question is beautiful, 00:42:40.010 --> 00:42:41.510 because you know already what you're 00:42:41.510 --> 00:42:44.616 trying to evaluate that they're catching in class in real time, 00:42:44.616 --> 00:42:48.600 if you can see who's getting at the nuance of what 00:42:48.600 --> 00:42:50.710 you're teaching. 00:42:50.710 --> 00:42:51.460 JANET RANKIN: Yes. 00:42:51.460 --> 00:42:54.220 I think that's an excellent, excellent point, 00:42:54.220 --> 00:42:57.430 that many of these activities that we do, 00:42:57.430 --> 00:43:01.810 these small group activities, or the clicker 00:43:01.810 --> 00:43:04.450 questions or other activities that students 00:43:04.450 --> 00:43:08.590 are doing in class are helping them learn. 00:43:08.590 --> 00:43:11.860 But they're also formative assessments of whether or not 00:43:11.860 --> 00:43:14.470 they get it or not, whether or not they understand. 00:43:14.470 --> 00:43:17.470 And you as the instructor can see it. 00:43:17.470 --> 00:43:20.080 If I walk around, if I listen, if I 00:43:20.080 --> 00:43:23.680 look at who answered what question on the clickers, 00:43:23.680 --> 00:43:26.320 or on the raising your hand, multiple choice questions, 00:43:26.320 --> 00:43:30.320 whatever methods I used here, I get a better, 00:43:30.320 --> 00:43:32.290 a much better sense of who understands 00:43:32.290 --> 00:43:33.580 or doesn't understand. 00:43:33.580 --> 00:43:38.710 If all I'm doing is lecturing I can't tell. 00:43:38.710 --> 00:43:41.530 And I can ask do you understand, but they're not 00:43:41.530 --> 00:43:43.480 likely to tell me. 00:43:43.480 --> 00:43:46.240 Which is kind of the other point is that sometimes-- 00:43:46.240 --> 00:43:48.580 and I think you were alluding to this, Gordon-- 00:43:48.580 --> 00:43:52.240 that sometimes students don't know whether they understand 00:43:52.240 --> 00:43:54.040 or not. 00:43:54.040 --> 00:43:57.880 So we need to give them the opportunities to find out 00:43:57.880 --> 00:44:00.730 whether they understand or not. 00:44:00.730 --> 00:44:03.310 And arguably, if you're just lecturing 00:44:03.310 --> 00:44:05.560 and you're a great lecturer-- 00:44:05.560 --> 00:44:07.900 you're eloquent, you don't stumble, 00:44:07.900 --> 00:44:11.710 the writing on the board is awesome, all this stuff-- 00:44:11.710 --> 00:44:14.350 students will think they understand maybe 00:44:14.350 --> 00:44:17.830 when they don't, because they're not 00:44:17.830 --> 00:44:19.840 having to confront any misunderstanding, 00:44:19.840 --> 00:44:22.870 and it all seems very nice. 00:44:22.870 --> 00:44:25.570 So these activities where you actually have students 00:44:25.570 --> 00:44:27.460 do something, say something, where 00:44:27.460 --> 00:44:30.220 you ask them a specific question, that's 00:44:30.220 --> 00:44:34.255 when you can determine whether or not they understand or not. 00:44:36.870 --> 00:44:38.690 And because they are not likely to say 00:44:38.690 --> 00:44:41.270 they don't understand, either because they just 00:44:41.270 --> 00:44:43.940 don't want to admit it, or because they really don't know 00:44:43.940 --> 00:44:45.350 whether they don't understand. 00:44:45.350 --> 00:44:48.586 So you need to give them measurements to find out 00:44:48.586 --> 00:44:49.710 whether they get it or not. 00:44:52.698 --> 00:44:53.198 Yes. 00:44:53.198 --> 00:44:57.707 AUDIENCE: Sometimes, for example lecture and the activity 00:44:57.707 --> 00:44:58.665 has [INAUDIBLE]. 00:44:58.665 --> 00:45:02.480 For example, we have lectures and the teacher just 00:45:02.480 --> 00:45:05.206 [INAUDIBLE] and then we have expectations 00:45:05.206 --> 00:45:07.288 as we do exercises. 00:45:07.288 --> 00:45:12.054 So do you recommend that we also do some activities 00:45:12.054 --> 00:45:15.000 during the lecture or just [INAUDIBLE]? 00:45:15.000 --> 00:45:17.520 JANET RANKIN: No, do them during lecture. 00:45:17.520 --> 00:45:20.040 I mean, the model that we're trying to do here 00:45:20.040 --> 00:45:23.010 is the model that you should do even if you have a hundred 00:45:23.010 --> 00:45:24.390 or 200 people in your class. 00:45:24.390 --> 00:45:25.710 You can use clickers. 00:45:25.710 --> 00:45:28.634 You can use pair share discussions. 00:45:28.634 --> 00:45:30.550 You can do all sorts of things to break it up, 00:45:30.550 --> 00:45:32.580 and it's effective. 00:45:32.580 --> 00:45:35.850 There's a paper coming up for next week, 00:45:35.850 --> 00:45:39.120 a meta-analysis, active learning strategies, 00:45:39.120 --> 00:45:42.900 how they actually, students learn more in a lecture 00:45:42.900 --> 00:45:45.630 if you actually ask them to talk to each other, 00:45:45.630 --> 00:45:47.006 break up, share, et cetera. 00:45:47.006 --> 00:45:49.380 And I think when we talked about misconceptions, remember 00:45:49.380 --> 00:45:51.060 that example with the coin? 00:45:51.060 --> 00:45:54.090 How many students got it, could draw the free body 00:45:54.090 --> 00:45:57.180 diagram before and after? 00:45:57.180 --> 00:46:01.650 I mean, before and after in a straight lecture, 00:46:01.650 --> 00:46:03.450 there's hardly any learning gain. 00:46:03.450 --> 00:46:05.760 But if you incorporate active learning, 00:46:05.760 --> 00:46:07.090 the learning gains go way up. 00:46:25.180 --> 00:46:26.830 So that's Bloom's Taxonomy. 00:46:26.830 --> 00:46:29.830 And an interesting take on Bloom's Taxonomy, 00:46:29.830 --> 00:46:32.320 there's a woman, Kathy Schrock, who-- 00:46:32.320 --> 00:46:34.390 it's a nice website and the slides 00:46:34.390 --> 00:46:36.010 we posted so you have the link-- 00:46:36.010 --> 00:46:39.100 but she decided, she has this theory that it's not 00:46:39.100 --> 00:46:40.720 really so pyramidal. 00:46:40.720 --> 00:46:42.532 That everything really is dependent, 00:46:42.532 --> 00:46:44.740 these cognitive processes are dependent on each other 00:46:44.740 --> 00:46:45.770 at a certain level. 00:46:45.770 --> 00:46:48.370 And so she's drawn it as a gear, where everything 00:46:48.370 --> 00:46:50.740 sort of supports the act of creating, 00:46:50.740 --> 00:46:51.900 which is up at the top. 00:46:51.900 --> 00:46:54.760 But the other ones are sort of around the side, 00:46:54.760 --> 00:47:00.130 but they all contribute to the ultimate cognitive 00:47:00.130 --> 00:47:01.470 process of creating. 00:47:01.470 --> 00:47:03.490 So that might be a nice way to think about it. 00:47:03.490 --> 00:47:07.210 And then she's taken the specific words, like analyzing, 00:47:07.210 --> 00:47:11.680 and said, OK, what's an instantiation 00:47:11.680 --> 00:47:17.230 of analyzing, outlining, deconstructing, organizing, 00:47:17.230 --> 00:47:19.030 structuring, surveying, whatever? 00:47:19.030 --> 00:47:22.150 So it's the same idea, it's just a different graphical 00:47:22.150 --> 00:47:23.446 representation. 00:47:27.420 --> 00:47:29.970 So what you've done is you've taken your topics, 00:47:29.970 --> 00:47:32.220 put them in those cells in the table. 00:47:32.220 --> 00:47:37.380 And I just want to point out kind of how as the instructor 00:47:37.380 --> 00:47:39.510 you have a lot of power about where 00:47:39.510 --> 00:47:41.910 you're putting those, where you've decided to put those. 00:47:41.910 --> 00:47:44.940 And it may be different for a first year undergraduate 00:47:44.940 --> 00:47:48.195 class, or a second or third year graduate course. 00:47:48.195 --> 00:47:51.420 It could be the same topic, but you're 00:47:51.420 --> 00:47:54.540 going to put them in a different category. 00:47:54.540 --> 00:47:56.820 So that's why when I asked you to bring a topic, 00:47:56.820 --> 00:47:59.040 it's for a specific class. 00:47:59.040 --> 00:48:02.014 It's for a specific group of students. 00:48:02.014 --> 00:48:04.180 So let's say we take the idea of interstitial sites. 00:48:04.180 --> 00:48:06.790 So I have a crystalline structure, 00:48:06.790 --> 00:48:09.720 it's got an order of periodicity of atoms. 00:48:09.720 --> 00:48:12.540 And then I have spaces between the atoms, 00:48:12.540 --> 00:48:14.400 those are the interstitial sites. 00:48:14.400 --> 00:48:16.742 And so I could say you'll be able to identify 00:48:16.742 --> 00:48:17.700 the interstitial sites. 00:48:17.700 --> 00:48:19.970 So I could say where's the tetrahedral site 00:48:19.970 --> 00:48:22.950 in a face-centered cubic or something, whatever? 00:48:22.950 --> 00:48:27.510 And students just really have to find it. 00:48:27.510 --> 00:48:30.180 I could say calculate the maximum size 00:48:30.180 --> 00:48:32.490 of an interstitial atom that could occupy 00:48:32.490 --> 00:48:34.540 that space, an ion that could occupy 00:48:34.540 --> 00:48:37.480 that space in a particular crystal structure. 00:48:37.480 --> 00:48:40.560 So those are two very different levels, 00:48:40.560 --> 00:48:43.650 but it's the same topic. 00:48:43.650 --> 00:48:46.830 X-ray diffraction, I could say apply Bragg's Law 00:48:46.830 --> 00:48:49.440 to calculate d-spacing. 00:48:49.440 --> 00:48:52.770 So that's very much a turn the crank kind of thing. 00:48:52.770 --> 00:48:54.540 I know the equation for Bragg's Law, 00:48:54.540 --> 00:48:56.375 I stick things in, I solve for the unknown. 00:49:00.350 --> 00:49:03.280 I could then also ask you to index unknown diffraction 00:49:03.280 --> 00:49:07.670 patterns, meaning identify an unknown crystal structure based 00:49:07.670 --> 00:49:09.470 on the diffraction pattern. 00:49:09.470 --> 00:49:12.500 That's a completely different activity, 00:49:12.500 --> 00:49:14.720 completely different set of cognitive 00:49:14.720 --> 00:49:18.180 processes than just using Bragg's Law. 00:49:18.180 --> 00:49:20.890 But you have to decide what it is. 00:49:20.890 --> 00:49:24.419 And arguably, if all you've had students do during the class is 00:49:24.419 --> 00:49:26.710 sort of problems that they applied Bragg's Law and they 00:49:26.710 --> 00:49:29.210 solved for d-spacing-- so they solved for the incident angle 00:49:29.210 --> 00:49:32.500 or they solved for the wavelength or whatever it is-- 00:49:32.500 --> 00:49:38.761 you really can't throw this at them on the exam, right? 00:49:38.761 --> 00:49:41.260 Or conversely, if you spent the whole time, they've indexed, 00:49:41.260 --> 00:49:44.860 they've been in the lab indexing unknown diffraction patterns, 00:49:44.860 --> 00:49:48.050 and then this is the question you ask them on the exam. 00:49:48.050 --> 00:49:51.580 Well, then you have people that perhaps get it without really, 00:49:51.580 --> 00:49:53.680 perhaps pass the exam without really getting it. 00:49:53.680 --> 00:49:57.530 So that's just something to keep in mind. 00:49:57.530 --> 00:50:00.880 The other thing to note is let's say 00:50:00.880 --> 00:50:03.424 this is my intended learning outcome-- 00:50:03.424 --> 00:50:05.590 calculate the maximum size of the interstitial atoms 00:50:05.590 --> 00:50:09.400 in a variety of crystal structures. 00:50:09.400 --> 00:50:11.530 How do I know whether students get that or not? 00:50:16.641 --> 00:50:17.140 What's that? 00:50:17.140 --> 00:50:17.973 AUDIENCE: It's easy. 00:50:17.973 --> 00:50:19.960 JANET RANKIN: It's very easy. 00:50:19.960 --> 00:50:21.916 I give them a crystal structure and I say 00:50:21.916 --> 00:50:23.665 calculate the maximum size of the ion that 00:50:23.665 --> 00:50:25.270 can fit in the space. 00:50:25.270 --> 00:50:30.270 So I have figured out how, the measurement is easy, it's cake. 00:50:30.270 --> 00:50:34.240 It's totally straightforward. 00:50:34.240 --> 00:50:36.960 Now I haven't told them, OK, I'm going to give you this crystal 00:50:36.960 --> 00:50:38.376 structure and I'm going to ask you 00:50:38.376 --> 00:50:44.250 this spacing, this ion size. 00:50:44.250 --> 00:50:46.552 I haven't given it away, but I have told 00:50:46.552 --> 00:50:47.760 them what's expected of them. 00:50:51.080 --> 00:50:55.870 And my colleague wrote these for VSEPR theory. 00:50:55.870 --> 00:50:59.360 It's the same topic, and it's different. 00:50:59.360 --> 00:51:01.040 Identify the common geometric shapes 00:51:01.040 --> 00:51:02.630 found in simple molecules. 00:51:02.630 --> 00:51:05.360 Explain the assumptions of this theory. 00:51:05.360 --> 00:51:08.960 Apply the theory to predict 3D structures. 00:51:08.960 --> 00:51:12.320 Compare and contrast the geometry of a certain molecule 00:51:12.320 --> 00:51:15.920 as predicted by two different theories. 00:51:15.920 --> 00:51:19.380 Evaluate the accuracy of each theory 00:51:19.380 --> 00:51:21.290 for a particular set of compounds, 00:51:21.290 --> 00:51:23.550 and then create some recommendations. 00:51:23.550 --> 00:51:27.590 So it's the same topic, but you decide 00:51:27.590 --> 00:51:31.220 what it is that students should be able to do. 00:51:31.220 --> 00:51:32.840 OK? 00:51:32.840 --> 00:51:38.210 So I have some, just a little exercise for us. 00:51:38.210 --> 00:51:41.162 So they should be specific, measurable, realistic. 00:51:41.162 --> 00:51:43.370 I'm going to put them up here and we're going to say, 00:51:43.370 --> 00:51:46.890 you're going to tell me what's wrong with them. 00:51:46.890 --> 00:51:51.890 So number one, t-tests are like a statistical analysis. 00:51:51.890 --> 00:51:55.940 So is anybody a t-test kind of person? 00:51:55.940 --> 00:51:58.400 We can skip that one. 00:51:58.400 --> 00:52:00.320 How about gaining appreciation for the use 00:52:00.320 --> 00:52:02.930 of linearization techniques? 00:52:02.930 --> 00:52:05.491 What's wrong with it? 00:52:05.491 --> 00:52:08.330 AUDIENCE: It's not specific. 00:52:08.330 --> 00:52:10.524 JANET RANKIN: It's not specific. 00:52:10.524 --> 00:52:12.112 AUDIENCE: It's not measurable. 00:52:12.112 --> 00:52:14.510 JANET RANKIN: It's totally not measurable, exactly. 00:52:14.510 --> 00:52:18.900 I mean, how do I know that you appreciate it? 00:52:18.900 --> 00:52:21.250 Don't do it. 00:52:21.250 --> 00:52:21.750 All right. 00:52:21.750 --> 00:52:23.520 Great. 00:52:23.520 --> 00:52:26.850 Have an intuition for the most effective method of integration 00:52:26.850 --> 00:52:28.770 for a given problem. 00:52:28.770 --> 00:52:30.770 AUDIENCE: It's vague. 00:52:30.770 --> 00:52:33.996 It's not measurable. 00:52:33.996 --> 00:52:36.720 JANET RANKIN: So I've heard it's not specific, 00:52:36.720 --> 00:52:39.340 it's not measurable, and it's not vague, 00:52:39.340 --> 00:52:40.650 I mean, and it's vague. 00:52:40.650 --> 00:52:46.320 So right, so have an intuition, I cannot measure that. 00:52:46.320 --> 00:52:50.350 I have not figured out a way to measure intuition. 00:52:50.350 --> 00:52:55.290 I can tell whether you can do it, right, 00:52:55.290 --> 00:53:00.060 but I can't tell you whether you have an intuition for it. 00:53:00.060 --> 00:53:01.860 Well, how would you rewrite it? 00:53:01.860 --> 00:53:03.317 Actually, how would you rewrite 2? 00:53:06.796 --> 00:53:14.081 AUDIENCE: Use [INAUDIBLE] 00:53:14.081 --> 00:53:18.430 JANET RANKIN: So it could be use the techniques. 00:53:18.430 --> 00:53:19.542 Another suggestion? 00:53:19.542 --> 00:53:22.175 AUDIENCE: Make something using the evaluation techniques. 00:53:22.175 --> 00:53:23.790 JANET RANKIN: Calculate something. 00:53:23.790 --> 00:53:27.570 Make a first pass estimate using a particular linearization 00:53:27.570 --> 00:53:28.418 technique. 00:53:28.418 --> 00:53:30.626 AUDIENCE: Describe when you would use a linearization 00:53:30.626 --> 00:53:31.092 technique. 00:53:31.092 --> 00:53:32.633 JANET RANKIN: Describe when you would 00:53:32.633 --> 00:53:36.300 use it, describe why you would use it, all of those things. 00:53:36.300 --> 00:53:41.706 AUDIENCE: List those areas where [INAUDIBLE] 00:53:41.706 --> 00:53:46.680 JANET RANKIN: List the areas where it's useful, yes. 00:53:46.680 --> 00:53:47.330 Great. 00:53:47.330 --> 00:53:47.830 All right. 00:53:47.830 --> 00:53:49.770 For number 3, the gain an intuition, 00:53:49.770 --> 00:53:51.780 we decided it's vague, it's not measurable. 00:53:54.540 --> 00:53:55.770 So how would you rewrite it? 00:54:03.658 --> 00:54:05.630 AUDIENCE: Compare. 00:54:05.630 --> 00:54:11.570 Compare the areas integration technique [INAUDIBLE] 00:54:11.570 --> 00:54:14.195 Solve a particular problem using different kind of [INAUDIBLE]. 00:54:14.195 --> 00:54:14.990 JANET RANKIN: OK. 00:54:14.990 --> 00:54:20.660 So solve a problem, and then we need a little more. 00:54:20.660 --> 00:54:21.635 Did you have something? 00:54:21.635 --> 00:54:24.252 AUDIENCE: Identify the most effective method of-- 00:54:30.060 --> 00:54:32.060 JANET RANKIN: Let's hear what Rachel had to say. 00:54:32.060 --> 00:54:32.559 Rachel. 00:54:32.559 --> 00:54:33.326 AUDIENCE: Yeah. 00:54:33.326 --> 00:54:37.977 Learn how to identify the simplest [INAUDIBLE]. 00:54:37.977 --> 00:54:38.810 JANET RANKIN: Right. 00:54:38.810 --> 00:54:41.570 So you probably don't want to say learn how. 00:54:41.570 --> 00:54:44.230 You would say be able to identify. 00:54:44.230 --> 00:54:44.800 Right? 00:54:44.800 --> 00:54:45.300 Gordon. 00:54:45.300 --> 00:54:55.467 AUDIENCE: [INAUDIBLE] 00:54:55.467 --> 00:54:56.300 JANET RANKIN: Right. 00:54:56.300 --> 00:55:01.560 I mean, you could just say integrate-- 00:55:01.560 --> 00:55:02.910 AUDIENCE: [INAUDIBLE] 00:55:02.910 --> 00:55:06.120 JANET RANKIN: Integrate a problem using whatever. 00:55:06.120 --> 00:55:13.280 Select a technique to integrate a particular equation. 00:55:16.860 --> 00:55:20.490 And again, those things end up being more measurable. 00:55:20.490 --> 00:55:24.330 So 4, provide problem solving tools and strategies. 00:55:24.330 --> 00:55:25.483 What's wrong with that one? 00:55:25.483 --> 00:55:26.765 AUDIENCE: It's too wide. 00:55:26.765 --> 00:55:27.956 JANET RANKIN: It's too wide. 00:55:27.956 --> 00:55:29.856 AUDIENCE: It's instructor centered. 00:55:29.856 --> 00:55:32.760 JANET RANKIN: It's instructor centered. 00:55:32.760 --> 00:55:33.376 What else? 00:55:33.376 --> 00:55:36.372 AUDIENCE: But it's measurable. 00:55:36.372 --> 00:55:38.580 JANET RANKIN: Well, I can measure that I provided it. 00:55:38.580 --> 00:55:41.690 Here, I provided these handouts, right? 00:55:41.690 --> 00:55:42.390 OK. 00:55:42.390 --> 00:55:43.590 Check. 00:55:43.590 --> 00:55:44.990 It happened. 00:55:44.990 --> 00:55:47.040 Whether or not you could do anything with them, 00:55:47.040 --> 00:55:50.199 I don't know at this point, right? 00:55:50.199 --> 00:55:50.990 That's the problem. 00:55:50.990 --> 00:55:53.833 AUDIENCE: It doesn't tell us what kind of problem 00:55:53.833 --> 00:55:55.920 and what kind of tools we're going to use. 00:55:55.920 --> 00:55:57.795 Just who gets to choose the strategies. 00:55:57.795 --> 00:55:59.290 JANET RANKIN: Right. 00:55:59.290 --> 00:55:59.790 Right. 00:55:59.790 --> 00:56:02.520 It's very, very general, totally not specific. 00:56:02.520 --> 00:56:04.230 So what could we do? 00:56:04.230 --> 00:56:06.750 I mean, we don't know what this instructor had in mind, 00:56:06.750 --> 00:56:08.700 but what could you do if this was your class? 00:56:08.700 --> 00:56:09.491 What would you say? 00:56:09.491 --> 00:56:11.685 AUDIENCE: What about develop instead of provide? 00:56:11.685 --> 00:56:13.890 JANET RANKIN: Develop. 00:56:13.890 --> 00:56:15.930 And again, it would have to be student centered, 00:56:15.930 --> 00:56:19.470 so the student will be able to develop problem 00:56:19.470 --> 00:56:21.540 solving tools and strategies. 00:56:21.540 --> 00:56:24.900 That's for x, exactly. 00:56:24.900 --> 00:56:28.620 We want to make it a little more specific and a little more 00:56:28.620 --> 00:56:32.580 focused, because that's pretty broad. 00:56:32.580 --> 00:56:34.219 I mean, I have developed a problem 00:56:34.219 --> 00:56:35.760 solving strategy that lets me get out 00:56:35.760 --> 00:56:38.190 the door without tripping. 00:56:38.190 --> 00:56:41.340 Does that mean I satisfied your learning outcomes? 00:56:41.340 --> 00:56:44.070 Probably not. 00:56:44.070 --> 00:56:48.570 So in order to do something-- 00:56:48.570 --> 00:56:51.000 Use thermodynamics to solve engineering problems. 00:56:56.536 --> 00:56:58.675 AUDIENCE: That's not specific as well. 00:56:58.675 --> 00:57:00.725 JANET RANKIN: It's not specific at all. 00:57:00.725 --> 00:57:04.209 AUDIENCE: Engineering is [INAUDIBLE]. 00:57:04.209 --> 00:57:05.000 JANET RANKIN: Yeah. 00:57:05.000 --> 00:57:08.240 I mean, it's crazy broad on a number of fronts, right? 00:57:08.240 --> 00:57:10.280 I mean, it's like engineering is huge. 00:57:10.280 --> 00:57:12.680 Thermodynamics is pretty darn big, too. 00:57:12.680 --> 00:57:18.240 So which part of thermodynamics, which part of engineering? 00:57:21.750 --> 00:57:23.503 Build an SAE race car. 00:57:23.503 --> 00:57:24.489 AUDIENCE: [INAUDIBLE] 00:57:24.489 --> 00:57:25.475 JANET RANKIN: What? 00:57:28.433 --> 00:57:29.907 AUDIENCE: It's not realistic. 00:57:29.907 --> 00:57:30.740 JANET RANKIN: Right. 00:57:30.740 --> 00:57:32.660 It's pretty specific. 00:57:32.660 --> 00:57:34.910 Build a functioning SAE race car. 00:57:34.910 --> 00:57:38.000 It's very specific, and I can totally 00:57:38.000 --> 00:57:40.292 think of the metric of how I would measure 00:57:40.292 --> 00:57:41.500 whether students could do it. 00:57:41.500 --> 00:57:43.730 I would just ask them to build the race car. 00:57:43.730 --> 00:57:45.860 But they're never going to do it in a class, 00:57:45.860 --> 00:57:48.230 in one class period, in one course. 00:57:48.230 --> 00:57:51.920 It's unrealistic, exactly. 00:57:51.920 --> 00:57:54.200 Learn to use Laplace transforms to solve 00:57:54.200 --> 00:57:56.234 differential equations. 00:57:56.234 --> 00:57:59.433 AUDIENCE: [INAUDIBLE] 00:57:59.433 --> 00:58:02.010 JANET RANKIN: It's funny. 00:58:02.010 --> 00:58:04.900 I mean, you can imagine the instructor 00:58:04.900 --> 00:58:07.530 saying you will learn to use that. 00:58:07.530 --> 00:58:13.580 But let's go, Katherine, work your magic with this one. 00:58:16.564 --> 00:58:23.452 AUDIENCE: [INAUDIBLE] 00:58:23.452 --> 00:58:24.475 JANET RANKIN: Right. 00:58:24.475 --> 00:58:26.600 You're my go-to person for when you want to get rid 00:58:26.600 --> 00:58:27.869 of all the extra words. 00:58:27.869 --> 00:58:28.910 I'm going to call on you. 00:58:28.910 --> 00:58:29.410 Right. 00:58:29.410 --> 00:58:31.995 Use Laplace transforms. 00:58:31.995 --> 00:58:32.870 That you can measure. 00:58:32.870 --> 00:58:34.760 You can give them an equation and see 00:58:34.760 --> 00:58:38.459 if they can use Laplace transforms to solve it. 00:58:38.459 --> 00:58:40.250 And yes, you might want to be a little more 00:58:40.250 --> 00:58:42.541 specific about the kinds of differential equations they 00:58:42.541 --> 00:58:48.415 use, and know how to upper diagonalize a matrix. 00:58:51.090 --> 00:58:52.207 Katherine. 00:58:52.207 --> 00:58:52.706 AUDIENCE: . 00:58:52.706 --> 00:58:55.600 Upper diagonalize a matrix. 00:58:55.600 --> 00:58:58.300 JANET RANKIN: Upper diagonalize a matrix. 00:58:58.300 --> 00:58:59.770 That's all you have to say, right? 00:58:59.770 --> 00:59:02.620 Forget the extra words. 00:59:02.620 --> 00:59:06.610 And that is really, I have just written the assessment question 00:59:06.610 --> 00:59:07.510 for that. 00:59:07.510 --> 00:59:10.060 I know it's totally measurable. 00:59:10.060 --> 00:59:13.780 I give you a matrix and I ask you to upper diagonalize it. 00:59:13.780 --> 00:59:16.330 I haven't given it away. 00:59:16.330 --> 00:59:19.860 I haven't told you you will upper diagonalize this matrix, 00:59:19.860 --> 00:59:20.560 right? 00:59:20.560 --> 00:59:22.450 Memorize these equations that you're 00:59:22.450 --> 00:59:25.480 going to need to write down in order to upper diagonalize it. 00:59:25.480 --> 00:59:29.440 But I have told you that that's what's expected of you. 00:59:29.440 --> 00:59:31.238 AUDIENCE: I have a comment on it. 00:59:31.238 --> 00:59:33.733 It's pretty useful when a student 00:59:33.733 --> 00:59:36.730 is trying to evaluate the use of the class. 00:59:36.730 --> 00:59:41.226 For example, the career fair tomorrow [INAUDIBLE]. 00:59:41.226 --> 00:59:43.218 There's a million [INAUDIBLE] to get jobs. 00:59:43.218 --> 00:59:45.210 OK, how are they going to evaluate me? 00:59:45.210 --> 00:59:46.635 And if you could come in and say, 00:59:46.635 --> 00:59:48.297 this happened and this is what we 00:59:48.297 --> 00:59:51.560 did, as opposed to having, or have them 00:59:51.560 --> 00:59:53.269 evaluate me on some random knowledge. 00:59:53.269 --> 00:59:55.060 And then I don't know that I could respond, 00:59:55.060 --> 00:59:58.880 but I do know stuff. 00:59:58.880 --> 01:00:02.526 So this is really helpful in kind of a broader sense. 01:00:02.526 --> 01:00:03.801 JANET RANKIN: Right. 01:00:03.801 --> 01:00:04.300 Great. 01:00:04.300 --> 01:00:04.900 No, I'm glad. 01:00:04.900 --> 01:00:06.530 And I think it's so true. 01:00:06.530 --> 01:00:09.880 And people often think, well, you're giving it away. 01:00:09.880 --> 01:00:11.710 But you're not giving anything away. 01:00:11.710 --> 01:00:14.020 You're just telling them the expectations. 01:00:14.020 --> 01:00:15.880 You're not giving them the problems, 01:00:15.880 --> 01:00:20.350 you're not teaching to the test, but you're just telling them 01:00:20.350 --> 01:00:26.290 what they're going to be able to do at the kind of high level, 01:00:26.290 --> 01:00:28.816 or medium level. 01:00:28.816 --> 01:00:30.940 You're telling what they're going to be able to do. 01:00:30.940 --> 01:00:33.310 I will say-- so ABET, the Accreditation Board 01:00:33.310 --> 01:00:35.050 for Engineering Teaching-- 01:00:35.050 --> 01:00:38.170 every accredited engineering program has to list, 01:00:38.170 --> 01:00:40.352 they have A through K outcomes. 01:00:40.352 --> 01:00:41.560 And if you look it up, ABET-- 01:00:44.230 --> 01:00:48.010 and I'm sure you see us as accredited-- 01:00:48.010 --> 01:00:54.870 if you look up, they have to, they 01:00:54.870 --> 01:00:57.060 used to do a lot more bean counting. 01:00:57.060 --> 01:00:59.100 But what they would do is the outcomes 01:00:59.100 --> 01:01:04.340 would be critical thinking and problem solving skills, 01:01:04.340 --> 01:01:06.420 and they were a little bit broad. 01:01:06.420 --> 01:01:08.910 But every department that got accredited 01:01:08.910 --> 01:01:14.130 had to show how various courses contributed to the outcomes. 01:01:14.130 --> 01:01:19.350 So somewhere in your department should be a pretty detailed 01:01:19.350 --> 01:01:22.770 list of how 6 double 00 whatever, did this. 01:01:22.770 --> 01:01:23.980 Or 6 yada, yada-- 01:01:23.980 --> 01:01:25.705 I know your grad students-- but did this. 01:01:25.705 --> 01:01:27.330 The problem is they don't have to do it 01:01:27.330 --> 01:01:28.440 for grad student courses. 01:01:28.440 --> 01:01:32.430 But it might be helpful if you looked at that, because I would 01:01:32.430 --> 01:01:34.770 imagine that sometimes the recruiters would 01:01:34.770 --> 01:01:38.550 be looking at that information, so that might be a commonality. 01:01:38.550 --> 01:01:42.240 But every program in the US that's accredited has to have, 01:01:42.240 --> 01:01:46.270 show at least how they've addressed those outcomes. 01:01:46.270 --> 01:01:48.441 So you might want to take a look at that. 01:01:48.441 --> 01:01:50.190 But yeah, if it was a little more explicit 01:01:50.190 --> 01:01:51.437 it would certainly be better. 01:01:51.437 --> 01:01:52.520 And I'm glad it's helpful. 01:01:55.290 --> 01:01:57.240 So I think we can see, you can start 01:01:57.240 --> 01:02:00.150 to see how there are better and worse learning outcomes, 01:02:00.150 --> 01:02:03.840 that if they're not measurable they're not so useful. 01:02:03.840 --> 01:02:06.580 If they are measurable, they're quite useful. 01:02:06.580 --> 01:02:12.620 So now what I'd like to do is revisit the worksheet. 01:02:12.620 --> 01:02:16.470 And what we can do is we're going 01:02:16.470 --> 01:02:20.490 to take about five minutes where you just sit quietly 01:02:20.490 --> 01:02:25.720 and write out specific intended learning outcomes. 01:02:25.720 --> 01:02:29.160 So you'll be able to upper diagonalize a matrix, 01:02:29.160 --> 01:02:33.180 you will be able to use Laplace transforms 01:02:33.180 --> 01:02:35.790 to solve this type of differential equation, 01:02:35.790 --> 01:02:39.450 whatever you've written in this column. 01:02:39.450 --> 01:02:41.460 And note that if you put it in this column, 01:02:41.460 --> 01:02:43.470 let's say you put it in the apply column, 01:02:43.470 --> 01:02:46.110 then when you write the learning outcome it's going to be you 01:02:46.110 --> 01:02:51.870 will be able to interpret data using blah, blah, blah. 01:02:51.870 --> 01:03:01.380 You will be able to model the vibration, model a car 01:03:01.380 --> 01:03:04.770 suspension system using linearization techniques. 01:03:04.770 --> 01:03:08.280 They're going to be whatever, wherever you put it in the box. 01:03:08.280 --> 01:03:09.750 These are some of the verbs, these 01:03:09.750 --> 01:03:13.295 are the verbs you're going to use in your learning outcome. 01:03:13.295 --> 01:03:14.170 Does that make sense? 01:03:17.050 --> 01:03:20.740 And I have even more verbs for you if that's helpful. 01:03:20.740 --> 01:03:22.401 Did I pass those out? 01:03:22.401 --> 01:03:23.275 You got those, right? 01:03:23.275 --> 01:03:23.980 AUDIENCE: Yeah. 01:03:23.980 --> 01:03:25.480 JANET RANKIN: So there's more verbs. 01:03:25.480 --> 01:03:26.950 It's the same thing. 01:03:26.950 --> 01:03:28.960 I mean, it's consistent. 01:03:28.960 --> 01:03:31.000 So take five minutes by yourself and do that. 01:03:31.000 --> 01:03:33.220 And then you're going to get into pairs 01:03:33.220 --> 01:03:35.380 and you're going to discuss. 01:03:35.380 --> 01:03:38.320 You're going to trade worksheets and give each other feedback. 01:03:38.320 --> 01:03:38.980 All right? 01:03:38.980 --> 01:03:44.310 So five minutes of quiet and then we'll pair up. 01:04:03.540 --> 01:04:06.655 So if you can pair up, you don't necessarily 01:04:06.655 --> 01:04:09.030 have to pair up with people that are sitting next to you. 01:04:09.030 --> 01:04:12.060 You might want to get a different perspective, 01:04:12.060 --> 01:04:13.785 so feel free to get up and move around. 01:04:13.785 --> 01:04:16.780 You can talk to somebody that wasn't in your group. 01:04:16.780 --> 01:04:19.330 So maybe if you go back and talk with Alex, maybe. 01:04:19.330 --> 01:04:20.890 We need one group of three. 01:04:20.890 --> 01:04:22.810 Oh, are you a group of three? 01:04:22.810 --> 01:04:28.740 So now just trade sheets of paper with your partner, 01:04:28.740 --> 01:04:33.670 or if you're a threesome you can figure out how to do it. 01:04:33.670 --> 01:04:38.010 And then review the learning outcomes critically. 01:04:38.010 --> 01:04:40.710 Guys, just one sec. 01:04:40.710 --> 01:04:43.530 So just review your partner's learning outcomes critically. 01:04:43.530 --> 01:04:45.570 Make sure that they're measurable, realistic, 01:04:45.570 --> 01:04:46.650 and specific. 01:04:46.650 --> 01:04:48.210 If your partner has used the word 01:04:48.210 --> 01:04:50.910 understand, help him or her work through that 01:04:50.910 --> 01:04:53.340 to get a better word. 01:04:53.340 --> 01:04:56.820 So you're going to be a critical friend here. 01:04:56.820 --> 01:04:59.730 And then you guys can discuss how to improve it. 01:04:59.730 --> 01:05:02.640 And then I'm going to pass around some flip chart paper 01:05:02.640 --> 01:05:05.880 and each one of you will write, maybe 01:05:05.880 --> 01:05:08.310 just write your name at the top, and write two or three 01:05:08.310 --> 01:05:10.534 learning outcomes on the flip chart paper. 01:05:10.534 --> 01:05:12.450 And then we can stick them up around the room, 01:05:12.450 --> 01:05:14.580 and then everybody can see everybody else's, 01:05:14.580 --> 01:05:16.652 all learning outcomes. 01:05:16.652 --> 01:05:21.170 [INTERPOSING VOICES] 01:05:22.587 --> 01:05:24.045 JANET RANKIN: Was there a question? 01:05:24.045 --> 01:05:24.753 AUDIENCE: Oh, no. 01:05:24.753 --> 01:05:25.600 I just think-- 01:05:25.600 --> 01:05:27.320 AUDIENCE: I think [INAUDIBLE]. 01:05:32.800 --> 01:05:35.340 JANET RANKIN: How are you guys doing? 01:05:35.340 --> 01:05:37.950 Doing OK? 01:05:37.950 --> 01:05:41.868 [INTERPOSING VOICES] 01:05:58.286 --> 01:06:01.470 JANET RANKIN: So you can use this zone if you need it. 01:06:05.490 --> 01:06:08.160 As you're looking at other people's 01:06:08.160 --> 01:06:11.910 learning outcomes, think about how clear, 01:06:11.910 --> 01:06:14.130 how easy it would be, or not easy, 01:06:14.130 --> 01:06:17.130 to come up with an exam question, or a project, 01:06:17.130 --> 01:06:20.070 or an assessment or measurement of whether 01:06:20.070 --> 01:06:23.580 or not the student reached, attained the learning outcome. 01:06:23.580 --> 01:06:25.630 You can put them up here, Ina and Shau. 01:06:34.841 --> 01:06:35.340 OK. 01:06:35.340 --> 01:06:37.810 And as I said, make sure that you've read everybody else's. 01:06:37.810 --> 01:06:40.750 I'm going to say a few words about them. 01:06:45.550 --> 01:06:47.500 I think we may have a record. 01:06:47.500 --> 01:06:51.400 This is like the first time ever that no one 01:06:51.400 --> 01:06:54.850 has used the word understand, which I think 01:06:54.850 --> 01:06:56.370 you're all to be commended for. 01:07:00.190 --> 01:07:04.900 So David, make sure you don't write understand. 01:07:04.900 --> 01:07:06.736 You build me up just to knock me down. 01:07:12.110 --> 01:07:13.597 AUDIENCE: [INAUDIBLE]. 01:07:13.597 --> 01:07:15.430 JANET RANKIN: You can, or you don't have to. 01:07:15.430 --> 01:07:16.179 It doesn't matter. 01:07:18.680 --> 01:07:24.260 Does anyone have any comments, either about the exercise 01:07:24.260 --> 01:07:29.510 itself, or about somebody's learning outcomes, 01:07:29.510 --> 01:07:30.380 that you see around? 01:07:33.250 --> 01:07:35.260 Again, the measurement, the thing 01:07:35.260 --> 01:07:37.510 you want to look at when you look at your own learning 01:07:37.510 --> 01:07:40.640 outcomes and those of others, is are they specific, measurable, 01:07:40.640 --> 01:07:42.940 and realistic? 01:07:42.940 --> 01:07:46.540 And would you know if a student achieved 01:07:46.540 --> 01:07:48.500 those learning outcomes? 01:07:48.500 --> 01:07:53.740 So I would say about 90% of these are very specific, 01:07:53.740 --> 01:07:54.980 which is great. 01:07:54.980 --> 01:07:59.470 I mean, there's some incredibly specific ones here. 01:07:59.470 --> 01:08:04.060 The more specific, the easier it is to measure. 01:08:04.060 --> 01:08:04.900 So that's one thing. 01:08:04.900 --> 01:08:06.610 That's a useful thing. 01:08:06.610 --> 01:08:08.950 You want to make sure you don't get so specific 01:08:08.950 --> 01:08:11.980 that there's only like one question you could ask. 01:08:11.980 --> 01:08:14.170 So that's the balance. 01:08:14.170 --> 01:08:16.270 Sometimes you can't tell that unless you're 01:08:16.270 --> 01:08:18.355 in the discipline. 01:08:18.355 --> 01:08:24.040 Do we have other comments about any of these learning outcomes, 01:08:24.040 --> 01:08:29.279 something you don't get, something that 01:08:29.279 --> 01:08:30.784 looks particularly interesting? 01:08:33.990 --> 01:08:34.890 Yes, Gordon. 01:08:34.890 --> 01:08:38.355 Perhaps you can sit down if you're done reading. 01:08:38.355 --> 01:08:42.323 AUDIENCE: I'm looking around and I'm seeing [INAUDIBLE]. 01:08:52.739 --> 01:08:54.723 I don't know how it's going to be. 01:08:54.723 --> 01:08:56.211 How are we going to test that? 01:08:56.211 --> 01:09:00.740 JANET RANKIN: Well, I guess you have to come up with a reaction 01:09:00.740 --> 01:09:03.810 that you're confident they haven't seen before. 01:09:03.810 --> 01:09:08.359 So if you can do that, then you can ask them to propose 01:09:08.359 --> 01:09:11.560 a mechanism for it, I suppose. 01:09:11.560 --> 01:09:13.370 If the person that wrote it wants to-- 01:09:13.370 --> 01:09:15.578 I know who wrote it-- but if the person that wrote it 01:09:15.578 --> 01:09:18.600 wants to talk a little bit about it, that would be great. 01:09:18.600 --> 01:09:19.100 If not-- 01:09:19.100 --> 01:09:22.390 AUDIENCE: OK, So a lot of people are 01:09:22.390 --> 01:09:26.150 going to just try to memorize each specific outcome. 01:09:26.150 --> 01:09:29.516 But it's easier to just identify your activity group 01:09:29.516 --> 01:09:31.585 and find patterns within them. 01:09:31.585 --> 01:09:45.445 So if you can try [INAUDIBLE] They can't just [INAUDIBLE]. 01:09:50.890 --> 01:09:52.207 JANET RANKIN: Right. 01:09:52.207 --> 01:09:53.290 And this is a great point. 01:09:53.290 --> 01:09:54.790 I mentioned it to one of the groups. 01:09:54.790 --> 01:09:59.550 But sometimes we'll say explain how blah, blah, blah, works. 01:09:59.550 --> 01:10:01.680 And that's a great thought, that, oh, 01:10:01.680 --> 01:10:04.170 they're going to be able to explain it, right? 01:10:04.170 --> 01:10:06.390 But many times they can fall back 01:10:06.390 --> 01:10:09.855 on a memorized explanation or sort 01:10:09.855 --> 01:10:12.960 of kind of a canned procedure. 01:10:12.960 --> 01:10:16.620 So if you really want them to be at the level of explain, 01:10:16.620 --> 01:10:19.410 you will have to kind of make some things up 01:10:19.410 --> 01:10:21.840 to get them out of, to get them away 01:10:21.840 --> 01:10:24.870 from the ability to just pull something completely 01:10:24.870 --> 01:10:26.040 from memory. 01:10:26.040 --> 01:10:30.505 So this is an example of how you might do that. 01:10:34.780 --> 01:10:37.392 Other observations or questions? 01:10:37.392 --> 01:10:41.180 AUDIENCE: I also [INAUDIBLE] something that all [INAUDIBLE]. 01:10:52.680 --> 01:10:56.410 JANET RANKIN: So perhaps the person 01:10:56.410 --> 01:10:58.191 that wrote it wants to comment on it. 01:10:58.191 --> 01:10:59.574 AUDIENCE: OK. 01:10:59.574 --> 01:11:00.957 So [INAUDIBLE]. 01:11:19.390 --> 01:11:22.220 I think what we have, just [INAUDIBLE]. 01:11:26.550 --> 01:11:27.800 So we write and compile. 01:11:31.380 --> 01:11:37.112 So you write a code and then you compile it. 01:11:37.112 --> 01:11:37.945 JANET RANKIN: Right. 01:11:40.600 --> 01:11:45.640 And this is a great example of how 01:11:45.640 --> 01:11:48.040 having people that aren't in the field 01:11:48.040 --> 01:11:51.580 read them can help clarify things for your students, who 01:11:51.580 --> 01:11:54.400 are by default not experts. 01:11:54.400 --> 01:11:57.430 So perhaps to an expert, it totally 01:11:57.430 --> 01:11:59.650 makes sense, of course, code. 01:11:59.650 --> 01:12:02.470 But to somebody that's not an expert, maybe that's confusing. 01:12:02.470 --> 01:12:06.010 Maybe the use of code twice was confusing. 01:12:06.010 --> 01:12:08.290 To an expert, it's totally readable. 01:12:08.290 --> 01:12:09.542 It's totally understandable. 01:12:09.542 --> 01:12:10.750 Perhaps to a novice it isn't. 01:12:10.750 --> 01:12:11.416 You're laughing. 01:12:11.416 --> 01:12:14.140 AUDIENCE: It's funny, because we're both [INAUDIBLE]. 01:12:14.140 --> 01:12:17.957 JANET RANKIN: Yeah, that was kind of random. 01:12:17.957 --> 01:12:19.540 Well, that's a perfect example, right? 01:12:19.540 --> 01:12:22.300 It's a perfect example of how getting maybe other people that 01:12:22.300 --> 01:12:23.830 aren't in your field to look at them 01:12:23.830 --> 01:12:28.390 can help make them more accessible to your students. 01:12:28.390 --> 01:12:29.327 OK. 01:12:29.327 --> 01:12:29.910 This is great. 01:12:29.910 --> 01:12:30.890 Oh, Hina. 01:12:30.890 --> 01:12:34.260 AUDIENCE: I had, it's on a different topic. 01:12:34.260 --> 01:12:36.890 But really, I'm talking about buzz words 01:12:36.890 --> 01:12:38.270 in learning outcomes. 01:12:38.270 --> 01:12:40.582 I think with engineering, you're really 01:12:40.582 --> 01:12:43.950 excited about telling people about real world problems. 01:12:43.950 --> 01:12:46.880 I guess that's the reason why. 01:12:46.880 --> 01:12:50.370 I guess it's more of an opinion question. 01:12:50.370 --> 01:12:54.290 As you write your learning outcomes I think in some ways 01:12:54.290 --> 01:12:57.230 it's better to be specific about the real world problems 01:12:57.230 --> 01:12:58.700 that they're going to attack. 01:12:58.700 --> 01:13:02.130 Because I've seen that on syllabuses all the time. 01:13:02.130 --> 01:13:04.090 I'm very excited about it. 01:13:04.090 --> 01:13:06.540 I think it's a way to teach people in your class. 01:13:06.540 --> 01:13:10.460 It's going to help them know what tools they're getting. 01:13:10.460 --> 01:13:11.460 JANET RANKIN: Right. 01:13:11.460 --> 01:13:13.650 I mean, real world problems in particular 01:13:13.650 --> 01:13:19.590 really do kind of encapsulate a whole pile of hopes and dreams. 01:13:19.590 --> 01:13:21.600 So it may help to be a little more specific. 01:13:21.600 --> 01:13:24.630 You can use it in your learning outcomes, 01:13:24.630 --> 01:13:27.450 but then maybe describe it in a little more detail 01:13:27.450 --> 01:13:32.280 in your syllabus, the kinds of real world problems, 01:13:32.280 --> 01:13:34.170 or why we care about real world problems. 01:13:34.170 --> 01:13:35.140 They're messy. 01:13:35.140 --> 01:13:36.750 They're ill-defined. 01:13:36.750 --> 01:13:39.310 What about them is so important? 01:13:39.310 --> 01:13:43.530 What about them is so important? 01:13:43.530 --> 01:13:45.420 Try to minimize the buzz words. 01:13:45.420 --> 01:13:47.859 But at some level, they may not know about something 01:13:47.859 --> 01:13:49.900 until they're completely through with the course, 01:13:49.900 --> 01:13:51.316 and you may have to use that word. 01:13:51.316 --> 01:13:53.550 So there's a bit of a balance. 01:13:53.550 --> 01:13:54.990 What I'm going to do before, I'll 01:13:54.990 --> 01:13:56.880 probably bring all these back to my office, 01:13:56.880 --> 01:14:00.570 take pictures of them, and put them up on the wiki 01:14:00.570 --> 01:14:03.810 so you'll have a record of them. 01:14:03.810 --> 01:14:08.230 I wanted to just as we go out here, 01:14:08.230 --> 01:14:14.050 my point is really that the hard part 01:14:14.050 --> 01:14:15.910 is these learning outcomes. 01:14:15.910 --> 01:14:17.830 And that once you write the learning outcomes, 01:14:17.830 --> 01:14:20.110 as we said before, you kind of know 01:14:20.110 --> 01:14:23.920 how you're going to test whether students got them, 01:14:23.920 --> 01:14:28.600 and you know what you're going to do to help them get them. 01:14:28.600 --> 01:14:34.810 The syllabus itself is just an articulation of that. 01:14:34.810 --> 01:14:36.770 It's a description of the course, 01:14:36.770 --> 01:14:38.470 an articulation of the learning outcomes 01:14:38.470 --> 01:14:41.620 stated, like the laundry example, with the title. 01:14:41.620 --> 01:14:43.810 This is what you're going to do. 01:14:43.810 --> 01:14:46.450 By the end of the class this is what you'll be able to do. 01:14:46.450 --> 01:14:49.420 It's kind of a promise that that's what you're going to do. 01:14:49.420 --> 01:14:51.790 It motivates students to take the class, 01:14:51.790 --> 01:14:56.230 to stay in the class, to engage with the class. 01:14:56.230 --> 01:14:58.492 It keeps you on track. 01:14:58.492 --> 01:14:59.950 So at the beginning of the semester 01:14:59.950 --> 01:15:04.120 maybe I'll be very clear, but four weeks in, 01:15:04.120 --> 01:15:06.820 and you're a little bit fuzzy, because you've been dealing 01:15:06.820 --> 01:15:08.440 with all sorts of stuff. 01:15:08.440 --> 01:15:10.690 And you kind of go, what the heck. 01:15:10.690 --> 01:15:12.400 What's this in here for? 01:15:12.400 --> 01:15:15.580 Well, if you've lined everything up at the beginning using 01:15:15.580 --> 01:15:17.320 your learning outcomes as the anchor, 01:15:17.320 --> 01:15:19.940 then you don't have to worry about that. 01:15:19.940 --> 01:15:21.640 You just follow the map. 01:15:21.640 --> 01:15:25.870 So it really, really helps you and the students 01:15:25.870 --> 01:15:30.370 get through the semester in a logical way. 01:15:30.370 --> 01:15:33.550 And it tells the students what they 01:15:33.550 --> 01:15:37.070 can expect of you, what you can expect of them. 01:15:37.070 --> 01:15:40.870 So it's motivational, structural, 01:15:40.870 --> 01:15:45.310 and it's evidence, hopefully of what they did, but certainly 01:15:45.310 --> 01:15:47.330 of what you hope to do with them. 01:15:50.750 --> 01:15:53.060 So for the post-session assignment, 01:15:53.060 --> 01:15:55.862 you'll have to write up more learning outcomes 01:15:55.862 --> 01:15:56.570 from your course. 01:15:56.570 --> 01:16:00.140 You can use these as the basis for sure. 01:16:00.140 --> 01:16:04.190 Hopefully, that table will help you as you write more of them. 01:16:04.190 --> 01:16:05.847 And I will go in and give you feedback. 01:16:05.847 --> 01:16:08.180 I want to make sure that everybody knows that you should 01:16:08.180 --> 01:16:10.550 read each other's postings. 01:16:10.550 --> 01:16:14.577 So a lot of you had great examples from the first class 01:16:14.577 --> 01:16:16.160 about how you would use these learning 01:16:16.160 --> 01:16:20.154 theories in your courses and how you teach. 01:16:20.154 --> 01:16:21.320 They were fabulous examples. 01:16:21.320 --> 01:16:23.153 Make sure you read those postings of others. 01:16:23.153 --> 01:16:26.100 That's one reason why it's up on a wiki and not just 01:16:26.100 --> 01:16:28.640 you handing it in to me. 01:16:28.640 --> 01:16:30.530 So you have the Mud cards. 01:16:30.530 --> 01:16:33.420 Fill them out if you have any more information that you want. 01:16:33.420 --> 01:16:39.730 And otherwise, we'll see next week.