1
00:00:08,660 --> 00:00:11,120
So at the end of the
class, there are really

2
00:00:11,120 --> 00:00:13,760
two major forms of assessment.

3
00:00:13,760 --> 00:00:16,820
One is an online
written exam, which

4
00:00:16,820 --> 00:00:22,040
is an exam where the questions
and the way to submit

5
00:00:22,040 --> 00:00:24,050
the answers is fully online.

6
00:00:24,050 --> 00:00:26,600
And that link is
unveiled, and then I

7
00:00:26,600 --> 00:00:32,270
give three to four days for the
students to answer this exam.

8
00:00:32,270 --> 00:00:35,330
It is an open book,
open internet exam.

9
00:00:35,330 --> 00:00:39,740
They can use any materials they
want, but no collaboration.

10
00:00:39,740 --> 00:00:43,670
So I don't have a formal
way to check for this

11
00:00:43,670 --> 00:00:46,400
but essentially it's
an individual exam.

12
00:00:46,400 --> 00:00:50,260
And by submitting
the answers online,

13
00:00:50,260 --> 00:00:54,040
again using for example a
mechanism like Google Forms,

14
00:00:54,040 --> 00:00:58,750
you can relatively quickly
compare the answers and grade

15
00:00:58,750 --> 00:01:01,420
these exams very
efficiently and also see

16
00:01:01,420 --> 00:01:04,099
if there's any patterns.

17
00:01:04,099 --> 00:01:10,120
So this is a way to essentially
check individual learning

18
00:01:10,120 --> 00:01:12,820
with all the materials at hand.

19
00:01:12,820 --> 00:01:15,610
The way I structure
the online exam

20
00:01:15,610 --> 00:01:21,160
is a mix of multiple-choice
questions, essay questions,

21
00:01:21,160 --> 00:01:23,320
but also some calculations.

22
00:01:23,320 --> 00:01:28,150
And the exam starts with
some very easy questions

23
00:01:28,150 --> 00:01:33,220
that can be referenced
in readings or materials

24
00:01:33,220 --> 00:01:36,580
in textbooks,
interpretive questions.

25
00:01:36,580 --> 00:01:39,580
And then they get
progressively more difficult.

26
00:01:39,580 --> 00:01:42,080
And the last two questions--

27
00:01:42,080 --> 00:01:44,830
in one of the questions,
I give them a calculation

28
00:01:44,830 --> 00:01:50,230
that they have to do that has
a very non-obvious answer.

29
00:01:50,230 --> 00:01:54,580
And it's interesting to
see some students that

30
00:01:54,580 --> 00:01:57,160
force an answer that's not
actually the right answer

31
00:01:57,160 --> 00:02:01,480
because somehow they believe
that any question given

32
00:02:01,480 --> 00:02:05,930
on an exam must have
this one correct answer.

33
00:02:05,930 --> 00:02:09,460
And other students scratch
their heads and say,

34
00:02:09,460 --> 00:02:13,250
I don't think this question
has the right answer.

35
00:02:13,250 --> 00:02:15,790
And so it's a very
interesting thing

36
00:02:15,790 --> 00:02:20,080
because, in the real world,
you often get requirements

37
00:02:20,080 --> 00:02:23,560
from customers that are
unrealistic or can't

38
00:02:23,560 --> 00:02:24,730
be fulfilled.

39
00:02:24,730 --> 00:02:27,130
And so the purpose
of these questions

40
00:02:27,130 --> 00:02:30,190
is to really get students
to think and connect

41
00:02:30,190 --> 00:02:32,260
the material to
real-world situations

42
00:02:32,260 --> 00:02:34,770
that they will encounter.

43
00:02:34,770 --> 00:02:39,197
In the last question, I
then ask about systems

44
00:02:39,197 --> 00:02:41,530
engineering-relevant things
that have happened that they

45
00:02:41,530 --> 00:02:43,060
might have seen in the news.

46
00:02:43,060 --> 00:02:48,430
So for example, one year I asked
about the Volkswagen emissions

47
00:02:48,430 --> 00:02:52,130
scandal, which is very related
to system engineering, trade

48
00:02:52,130 --> 00:02:57,770
offs between fuel efficiency,
cost, NOx emissions.

49
00:02:57,770 --> 00:03:01,000
It's very much current affairs.

50
00:03:01,000 --> 00:03:04,150
This year, I asked the
students about the problems

51
00:03:04,150 --> 00:03:08,590
that Samsung experienced with
their Galaxy Note 7 batteries

52
00:03:08,590 --> 00:03:10,690
and what is the root
cause of those problems.

53
00:03:10,690 --> 00:03:13,840
And how are those problems
related to the material

54
00:03:13,840 --> 00:03:15,220
that we teach in class?

55
00:03:15,220 --> 00:03:17,800
So I recommend that one
of the last questions

56
00:03:17,800 --> 00:03:21,010
connect the students
with things that they're

57
00:03:21,010 --> 00:03:22,540
seeing in the news.

58
00:03:22,540 --> 00:03:26,200
The oral exam is given either
in person in the same room

59
00:03:26,200 --> 00:03:30,490
or through an online platform
like Skype or WebEx or Google

60
00:03:30,490 --> 00:03:31,450
Hangouts.

61
00:03:31,450 --> 00:03:34,540
The quality of the
online experience,

62
00:03:34,540 --> 00:03:37,810
the image, the sound,
is generally so good

63
00:03:37,810 --> 00:03:39,370
that there's virtually
no difference

64
00:03:39,370 --> 00:03:43,900
between doing the oral exam
in person or over the web.

65
00:03:43,900 --> 00:03:47,740
And the oral exam is anywhere
between 15 to 20 minutes

66
00:03:47,740 --> 00:03:48,910
per student.

67
00:03:48,910 --> 00:03:50,740
That doesn't sound
like a lot of time,

68
00:03:50,740 --> 00:03:54,190
but with well-structured
questions you can learn a lot

69
00:03:54,190 --> 00:03:56,710
and have a great dialogue
with the students.

70
00:03:56,710 --> 00:03:58,640
I try to put the
students at ease.

71
00:03:58,640 --> 00:04:04,000
It is more a discussion
than a classical exam.

72
00:04:04,000 --> 00:04:09,160
Nevertheless, it's
a great way for them

73
00:04:09,160 --> 00:04:11,320
to engage with the
material again,

74
00:04:11,320 --> 00:04:14,950
to engage with us as
instructors and TAs,

75
00:04:14,950 --> 00:04:20,769
and usually it sort of puts
a nice bow on the class,

76
00:04:20,769 --> 00:04:24,220
and it's a nice finishing
touch to the class.

77
00:04:24,220 --> 00:04:26,440
Logistically, the way
we do the oral exam

78
00:04:26,440 --> 00:04:30,040
is we open up typically
one or two full days.

79
00:04:30,040 --> 00:04:32,380
So I have to reserve
a couple of days

80
00:04:32,380 --> 00:04:36,010
on my calendar just
dedicated to the oral exams.

81
00:04:36,010 --> 00:04:37,690
And then there's
sign-up slots where

82
00:04:37,690 --> 00:04:42,580
students can just sign up online
and show up at the right time.

83
00:04:42,580 --> 00:04:45,780
And it's been
working pretty well.