WEBVTT 00:00:08.660 --> 00:00:11.120 So at the end of the class, there are really 00:00:11.120 --> 00:00:13.760 two major forms of assessment. 00:00:13.760 --> 00:00:16.820 One is an online written exam, which 00:00:16.820 --> 00:00:22.040 is an exam where the questions and the way to submit 00:00:22.040 --> 00:00:24.050 the answers is fully online. 00:00:24.050 --> 00:00:26.600 And that link is unveiled, and then I 00:00:26.600 --> 00:00:32.270 give three to four days for the students to answer this exam. 00:00:32.270 --> 00:00:35.330 It is an open book, open internet exam. 00:00:35.330 --> 00:00:39.740 They can use any materials they want, but no collaboration. 00:00:39.740 --> 00:00:43.670 So I don't have a formal way to check for this 00:00:43.670 --> 00:00:46.400 but essentially it's an individual exam. 00:00:46.400 --> 00:00:50.260 And by submitting the answers online, 00:00:50.260 --> 00:00:54.040 again using for example a mechanism like Google Forms, 00:00:54.040 --> 00:00:58.750 you can relatively quickly compare the answers and grade 00:00:58.750 --> 00:01:01.420 these exams very efficiently and also see 00:01:01.420 --> 00:01:04.099 if there's any patterns. 00:01:04.099 --> 00:01:10.120 So this is a way to essentially check individual learning 00:01:10.120 --> 00:01:12.820 with all the materials at hand. 00:01:12.820 --> 00:01:15.610 The way I structure the online exam 00:01:15.610 --> 00:01:21.160 is a mix of multiple-choice questions, essay questions, 00:01:21.160 --> 00:01:23.320 but also some calculations. 00:01:23.320 --> 00:01:28.150 And the exam starts with some very easy questions 00:01:28.150 --> 00:01:33.220 that can be referenced in readings or materials 00:01:33.220 --> 00:01:36.580 in textbooks, interpretive questions. 00:01:36.580 --> 00:01:39.580 And then they get progressively more difficult. 00:01:39.580 --> 00:01:42.080 And the last two questions-- 00:01:42.080 --> 00:01:44.830 in one of the questions, I give them a calculation 00:01:44.830 --> 00:01:50.230 that they have to do that has a very non-obvious answer. 00:01:50.230 --> 00:01:54.580 And it's interesting to see some students that 00:01:54.580 --> 00:01:57.160 force an answer that's not actually the right answer 00:01:57.160 --> 00:02:01.480 because somehow they believe that any question given 00:02:01.480 --> 00:02:05.930 on an exam must have this one correct answer. 00:02:05.930 --> 00:02:09.460 And other students scratch their heads and say, 00:02:09.460 --> 00:02:13.250 I don't think this question has the right answer. 00:02:13.250 --> 00:02:15.790 And so it's a very interesting thing 00:02:15.790 --> 00:02:20.080 because, in the real world, you often get requirements 00:02:20.080 --> 00:02:23.560 from customers that are unrealistic or can't 00:02:23.560 --> 00:02:24.730 be fulfilled. 00:02:24.730 --> 00:02:27.130 And so the purpose of these questions 00:02:27.130 --> 00:02:30.190 is to really get students to think and connect 00:02:30.190 --> 00:02:32.260 the material to real-world situations 00:02:32.260 --> 00:02:34.770 that they will encounter. 00:02:34.770 --> 00:02:39.197 In the last question, I then ask about systems 00:02:39.197 --> 00:02:41.530 engineering-relevant things that have happened that they 00:02:41.530 --> 00:02:43.060 might have seen in the news. 00:02:43.060 --> 00:02:48.430 So for example, one year I asked about the Volkswagen emissions 00:02:48.430 --> 00:02:52.130 scandal, which is very related to system engineering, trade 00:02:52.130 --> 00:02:57.770 offs between fuel efficiency, cost, NOx emissions. 00:02:57.770 --> 00:03:01.000 It's very much current affairs. 00:03:01.000 --> 00:03:04.150 This year, I asked the students about the problems 00:03:04.150 --> 00:03:08.590 that Samsung experienced with their Galaxy Note 7 batteries 00:03:08.590 --> 00:03:10.690 and what is the root cause of those problems. 00:03:10.690 --> 00:03:13.840 And how are those problems related to the material 00:03:13.840 --> 00:03:15.220 that we teach in class? 00:03:15.220 --> 00:03:17.800 So I recommend that one of the last questions 00:03:17.800 --> 00:03:21.010 connect the students with things that they're 00:03:21.010 --> 00:03:22.540 seeing in the news. 00:03:22.540 --> 00:03:26.200 The oral exam is given either in person in the same room 00:03:26.200 --> 00:03:30.490 or through an online platform like Skype or WebEx or Google 00:03:30.490 --> 00:03:31.450 Hangouts. 00:03:31.450 --> 00:03:34.540 The quality of the online experience, 00:03:34.540 --> 00:03:37.810 the image, the sound, is generally so good 00:03:37.810 --> 00:03:39.370 that there's virtually no difference 00:03:39.370 --> 00:03:43.900 between doing the oral exam in person or over the web. 00:03:43.900 --> 00:03:47.740 And the oral exam is anywhere between 15 to 20 minutes 00:03:47.740 --> 00:03:48.910 per student. 00:03:48.910 --> 00:03:50.740 That doesn't sound like a lot of time, 00:03:50.740 --> 00:03:54.190 but with well-structured questions you can learn a lot 00:03:54.190 --> 00:03:56.710 and have a great dialogue with the students. 00:03:56.710 --> 00:03:58.640 I try to put the students at ease. 00:03:58.640 --> 00:04:04.000 It is more a discussion than a classical exam. 00:04:04.000 --> 00:04:09.160 Nevertheless, it's a great way for them 00:04:09.160 --> 00:04:11.320 to engage with the material again, 00:04:11.320 --> 00:04:14.950 to engage with us as instructors and TAs, 00:04:14.950 --> 00:04:20.769 and usually it sort of puts a nice bow on the class, 00:04:20.769 --> 00:04:24.220 and it's a nice finishing touch to the class. 00:04:24.220 --> 00:04:26.440 Logistically, the way we do the oral exam 00:04:26.440 --> 00:04:30.040 is we open up typically one or two full days. 00:04:30.040 --> 00:04:32.380 So I have to reserve a couple of days 00:04:32.380 --> 00:04:36.010 on my calendar just dedicated to the oral exams. 00:04:36.010 --> 00:04:37.690 And then there's sign-up slots where 00:04:37.690 --> 00:04:42.580 students can just sign up online and show up at the right time. 00:04:42.580 --> 00:04:45.780 And it's been working pretty well.