WEBVTT

00:00:05.410 --> 00:00:07.380
PROFESSOR 1: So I think
in a course like this,

00:00:07.380 --> 00:00:09.170
assessment is tricky.

00:00:09.170 --> 00:00:14.420
That these are individual
clients and projects.

00:00:14.420 --> 00:00:17.100
And we can try to
standardize maybe

00:00:17.100 --> 00:00:20.550
the design and engineering
challenge to some extent,

00:00:20.550 --> 00:00:23.280
but that's not
necessarily what we took

00:00:23.280 --> 00:00:24.940
as our primary consideration.

00:00:24.940 --> 00:00:27.890
We really started
with the people

00:00:27.890 --> 00:00:31.810
first, and tried to
match people to projects.

00:00:31.810 --> 00:00:34.100
And there's no doubt, there
are some projects that

00:00:34.100 --> 00:00:37.400
are more technically involved.

00:00:37.400 --> 00:00:39.650
There are some projects
where the logistics are

00:00:39.650 --> 00:00:42.560
more difficult--
meeting with the client

00:00:42.560 --> 00:00:48.190
or even working with him or
her, or even within the team.

00:00:48.190 --> 00:00:51.650
So it is a tricky kind of thing.

00:00:51.650 --> 00:00:56.860
But I think having us as
instructors, very involved

00:00:56.860 --> 00:00:59.990
in the class, and getting a
sense of all of the projects,

00:00:59.990 --> 00:01:04.160
having the mentors really
have a strong understanding

00:01:04.160 --> 00:01:06.330
of different
projects' progress--

00:01:06.330 --> 00:01:10.890
so we had one mentor
for every two teams--

00:01:10.890 --> 00:01:17.190
was really key to the
assessment at a high level.

00:01:17.190 --> 00:01:20.620
We had different pieces over
the course of the semester.

00:01:20.620 --> 00:01:25.910
So doing this
mid-term design review

00:01:25.910 --> 00:01:29.030
counts as part of
the assessment.

00:01:29.030 --> 00:01:31.400
Certainly the final
presentations,

00:01:31.400 --> 00:01:34.470
the documentation is a big part.

00:01:34.470 --> 00:01:39.760
But we also rely on mentor
feedback, client feedback--

00:01:39.760 --> 00:01:43.350
we met with all the clients
at the end of the semester.

00:01:43.350 --> 00:01:44.770
As well as peer feedback.

00:01:44.770 --> 00:01:49.240
This is a team-based
project course.

00:01:49.240 --> 00:01:52.540
So taking in all
these inputs is what

00:01:52.540 --> 00:01:55.070
we tried to do when
it came to assessment.

00:01:55.070 --> 00:01:58.140
To really try to understand
what students had learned

00:01:58.140 --> 00:02:01.580
and how they performed
in the course.

00:02:01.580 --> 00:02:04.340
GRACE: So for some of these
we did prepare rubric,

00:02:04.340 --> 00:02:07.790
or we had a vague sense of
what we were looking for.

00:02:07.790 --> 00:02:09.729
So for example,
for the blog posts,

00:02:09.729 --> 00:02:12.350
we would score it
based on not just

00:02:12.350 --> 00:02:14.660
how well did you
write it, and grammar,

00:02:14.660 --> 00:02:16.650
and how much
information is there,

00:02:16.650 --> 00:02:20.420
but we were looking for really
deep, personal reflection.

00:02:20.420 --> 00:02:24.590
And so that's what we gave
distinctiveness points for,

00:02:24.590 --> 00:02:25.090
I believe.

00:02:25.090 --> 00:02:27.060
And so, for example,
you could have

00:02:27.060 --> 00:02:30.610
a post just on assistive
dogs, and what types

00:02:30.610 --> 00:02:32.250
of assistive dogs are there.

00:02:32.250 --> 00:02:35.690
Which is great, it shows
that a student is learning.

00:02:35.690 --> 00:02:40.250
But another post-- one of
my favorite posts, actually,

00:02:40.250 --> 00:02:43.260
over the semester-- was
from one of my students

00:02:43.260 --> 00:02:45.940
who was struggling
with, should I

00:02:45.940 --> 00:02:49.040
go into assistive
technology as a career

00:02:49.040 --> 00:02:50.700
when everyone is
saying you should

00:02:50.700 --> 00:02:52.160
do something that's lucrative.

00:02:52.160 --> 00:02:53.810
But this is beneficial.

00:02:53.810 --> 00:02:58.520
But what if beneficial isn't
profitable, what happens then?

00:02:58.520 --> 00:03:00.230
And she was really
struggling with,

00:03:00.230 --> 00:03:02.440
am I a selfish person,
things like that.

00:03:02.440 --> 00:03:05.260
And so that kind of very
deep, personal growth

00:03:05.260 --> 00:03:10.150
is something that
I valued a lot.

00:03:10.150 --> 00:03:11.310
We all valued a lot.

00:03:14.670 --> 00:03:17.110
There's actually
very little weight

00:03:17.110 --> 00:03:21.050
put on the final project.

00:03:21.050 --> 00:03:24.150
We put a lot of weight
on how well did they

00:03:24.150 --> 00:03:26.390
go about the design process.

00:03:26.390 --> 00:03:29.980
And so, how many prototypes
did they manage to do?

00:03:29.980 --> 00:03:34.280
When they didn't have the
skills to do a certain project,

00:03:34.280 --> 00:03:36.460
were they able to go
look for resources?

00:03:36.460 --> 00:03:39.280
Did they go around
MIT knocking on doors,

00:03:39.280 --> 00:03:42.750
asking professors for help,
asking machine shop technicians

00:03:42.750 --> 00:03:44.470
for help?

00:03:44.470 --> 00:03:47.155
That was also highly valued.

00:03:50.680 --> 00:03:55.440
What else do you
think I'm missing out?

00:03:55.440 --> 00:04:02.520
We also took into account
when the clients were-- so,

00:04:02.520 --> 00:04:07.210
some clients would
change their mind

00:04:07.210 --> 00:04:10.440
about what project they
wanted the students to work on

00:04:10.440 --> 00:04:12.210
over the course of semester.

00:04:12.210 --> 00:04:14.036
Which is really difficult
for the students,

00:04:14.036 --> 00:04:15.910
because they would make
progress on something

00:04:15.910 --> 00:04:22.760
and then have to change the
project direction completely.

00:04:22.760 --> 00:04:25.370
And so we would try and take
that into account as well.

00:04:25.370 --> 00:04:28.150
Like, how were the students
able to manage that?

00:04:28.150 --> 00:04:30.670
And so because of these
kind of little things,

00:04:30.670 --> 00:04:32.410
we really didn't
put a lot of weight

00:04:32.410 --> 00:04:35.070
on how the project turned
out at the end of the day.

00:04:35.070 --> 00:04:36.530
It was more about
how much effort

00:04:36.530 --> 00:04:39.070
did you put into it, how much
did you grow as a person,

00:04:39.070 --> 00:04:42.150
how well did you manage
your clients' expectations,

00:04:42.150 --> 00:04:47.910
and were you able to
satisfy the client given

00:04:47.910 --> 00:04:49.330
all the constraints.

00:04:49.330 --> 00:04:52.300
So very, very organic.

00:04:52.300 --> 00:04:54.030
Which is why we
had to spread out

00:04:54.030 --> 00:04:57.199
the grading rubric across
lots of little components.

00:04:57.199 --> 00:04:58.740
And that's why we
had to get feedback

00:04:58.740 --> 00:05:01.710
from mentors, and
peers, and clients,

00:05:01.710 --> 00:05:05.030
and have mid-semester reviews
and final semester reviews.

00:05:08.210 --> 00:05:09.224
I hope I explained that.

00:05:09.224 --> 00:05:10.640
PROFESSOR 1: Yeah,
I think there's

00:05:10.640 --> 00:05:15.570
a lot of aspects to this.

00:05:15.570 --> 00:05:19.740
One is, we really did try
to structure the evaluation

00:05:19.740 --> 00:05:25.050
as much as possible in
terms of being able to try

00:05:25.050 --> 00:05:26.850
to grade objectively.

00:05:26.850 --> 00:05:30.990
And so we did have a decent
number of structured rubrics,

00:05:30.990 --> 00:05:33.500
that we're happy
to share, in terms

00:05:33.500 --> 00:05:38.780
of how we evaluated different
components of the project.

00:05:38.780 --> 00:05:42.860
So everything from the course,
everything from the blog posts,

00:05:42.860 --> 00:05:46.710
to the videos, to the
mid-semester and final semester

00:05:46.710 --> 00:05:54.660
panels, the documentation,
all aspects of the class.

00:05:54.660 --> 00:06:04.610
I think for the
individual components,

00:06:04.610 --> 00:06:07.540
as Grace says, really
trying to personalize

00:06:07.540 --> 00:06:10.930
the learning a little bit in
terms of having reflections

00:06:10.930 --> 00:06:14.160
or blog posts where people
could write about topics

00:06:14.160 --> 00:06:16.010
that interested
them, as it related

00:06:16.010 --> 00:06:17.751
to assistive technology.

00:06:17.751 --> 00:06:20.250
I think that's something that
we tried this year, as opposed

00:06:20.250 --> 00:06:26.280
to maybe more formed lab reports
on some of the labs that we did

00:06:26.280 --> 00:06:30.520
or something like that.

00:06:30.520 --> 00:06:33.620
The other part I think
we tried this year

00:06:33.620 --> 00:06:36.670
was really to take in,
as Grace was saying,

00:06:36.670 --> 00:06:42.740
all of these human
inputs or feedback.

00:06:42.740 --> 00:06:48.330
So we met weekly as a staff,
and reflected on the projects

00:06:48.330 --> 00:06:51.980
and the students' progress.

00:06:51.980 --> 00:06:57.780
We talked to clients and
had students themselves talk

00:06:57.780 --> 00:07:01.450
a little bit about their peers.

00:07:01.450 --> 00:07:07.240
And so that's, I think, what
we tried to take into account.

00:07:07.240 --> 00:07:09.020
To some extent we tried
to start from what

00:07:09.020 --> 00:07:15.720
would be ideal in terms
of understanding students'

00:07:15.720 --> 00:07:17.960
experiences in the class.

00:07:17.960 --> 00:07:21.110
And that, I think,
makes it a little bit

00:07:21.110 --> 00:07:25.560
tricky to evaluate or grade.

00:07:25.560 --> 00:07:32.960
But hopefully it was
one reasonable way,

00:07:32.960 --> 00:07:38.360
or one valuable way to do it.