WEBVTT

00:00:00.000 --> 00:00:05.368
[SQUEAKING]
[RUSTLING] [CLICKING]

00:00:11.407 --> 00:00:12.490
CHARLENE: Hello, everyone.

00:00:12.490 --> 00:00:13.443
My name is Charlene.

00:00:13.443 --> 00:00:14.610
THERESE: My name is Therese.

00:00:14.610 --> 00:00:16.735
CHARLENE: And we're here
to present INMUSE, a music

00:00:16.735 --> 00:00:18.330
capturing game.

00:00:18.330 --> 00:00:21.360
So you're going for a walk,
and you put on Bose Air glasses

00:00:21.360 --> 00:00:24.235
and turn on your phone,
and open up the INMUSE app.

00:00:24.235 --> 00:00:25.860
As you walk with your
Bose Air glasses,

00:00:25.860 --> 00:00:28.650
you will hear a
melody to your right.

00:00:28.650 --> 00:00:31.110
You turn towards the melody
to get a better hearing of it.

00:00:31.110 --> 00:00:33.630
And if you like it, you
double-tap on your glasses

00:00:33.630 --> 00:00:36.050
to capture the melody.

00:00:36.050 --> 00:00:38.300
If you feel the melody is
not something that you like,

00:00:38.300 --> 00:00:39.900
you can shake your
head and let it go.

00:00:39.900 --> 00:00:41.550
And by the end of
the walk, you will

00:00:41.550 --> 00:00:44.220
have to capture a
series of melodies

00:00:44.220 --> 00:00:47.910
and compose your own little
soundtrack that you experienced

00:00:47.910 --> 00:00:49.470
during your walk.

00:00:49.470 --> 00:00:53.070
So here's a little
video of a [INAUDIBLE]..

00:00:53.070 --> 00:00:54.000
[LAUGHTER]

00:00:54.000 --> 00:00:57.177
AUDIENCE: [INAUDIBLE]

00:00:57.177 --> 00:00:58.010
AUDIENCE: All right.

00:00:58.010 --> 00:00:58.790
[LAUGHTER]

00:00:58.790 --> 00:01:02.276
CHARLENE: [INAUDIBLE]

00:01:02.276 --> 00:01:05.748
[STEADY BEAT PLAYING]

00:01:14.676 --> 00:01:18.148
[MUSIC PLAYING]

00:02:00.308 --> 00:02:01.870
AUDIENCE: Yeah, that was great.

00:02:01.870 --> 00:02:04.640
CHARLENE: Great, so
now we can do the demo.

00:02:04.640 --> 00:02:07.690
So the demo is set up
where if you take 10 steps

00:02:07.690 --> 00:02:09.820
and it's in a very high
sensitivity for the step

00:02:09.820 --> 00:02:14.140
counter because we're doing
this indoor to trigger a sound,

00:02:14.140 --> 00:02:16.480
and it's ready for whoever
want to try it out.

00:02:16.480 --> 00:02:26.020
AUDIENCE: [INAUDIBLE]

00:02:26.020 --> 00:02:27.640
CHARLENE: We have
a very simple UI

00:02:27.640 --> 00:02:30.670
to indicate how many
melodies you encounter,

00:02:30.670 --> 00:02:32.470
how many steps
you took, and then

00:02:32.470 --> 00:02:33.970
how many melodies captured.

00:02:33.970 --> 00:02:36.610
The whole experience is
aimed to be phones down,

00:02:36.610 --> 00:02:38.260
heads up, and just
for you to walk.

00:02:38.260 --> 00:02:40.910
You don't really want to be
distracted, first of all.

00:02:40.910 --> 00:02:43.090
It's really just to enhance
and to kind of capture

00:02:43.090 --> 00:02:44.230
the moment as you walk.

00:02:44.230 --> 00:02:47.611
So let me turn and
get on the app.

00:02:47.611 --> 00:02:48.667
There you go.

00:02:48.667 --> 00:02:51.250
You should connect, and then you
can just slip the phone right

00:02:51.250 --> 00:02:51.875
in your pocket.

00:02:56.550 --> 00:02:58.730
AUDIENCE: So this is not
head direction dependent,

00:02:58.730 --> 00:03:00.510
meaning one direction
or the other--

00:03:00.510 --> 00:03:01.760
CHARLENE: That doesn't matter.

00:03:01.760 --> 00:03:02.240
AUDIENCE: Doesn't matter.

00:03:02.240 --> 00:03:03.282
CHARLENE: Doesn't matter.

00:03:03.282 --> 00:03:05.285
So the audio is spatialized.

00:03:05.285 --> 00:03:07.910
We experiment a couple of times
with spatialized music in order

00:03:07.910 --> 00:03:11.390
for the user can distinguish
between the new sound

00:03:11.390 --> 00:03:13.580
and the sound that
they captured.

00:03:13.580 --> 00:03:16.340
But during our testing,
we did find out it

00:03:16.340 --> 00:03:18.465
does make the walking
experience more uncomfortable

00:03:18.465 --> 00:03:20.673
because they would like to
face the direction they're

00:03:20.673 --> 00:03:23.030
walking instead of keep
turning their head around.

00:03:23.030 --> 00:03:27.380
So spatialization
in the moving--

00:03:27.380 --> 00:03:30.410
when they're moving isn't
as reasonable for the user

00:03:30.410 --> 00:03:31.610
to do as a request.

00:03:31.610 --> 00:03:32.780
So we cut that out.

00:03:35.945 --> 00:03:38.070
THERESE: Yeah, and one
thing that the more and more

00:03:38.070 --> 00:03:41.610
we worked on this game
we wanted to focus on

00:03:41.610 --> 00:03:45.660
is distinguish it from just
playing your normal music

00:03:45.660 --> 00:03:48.593
and have it more of a
meditative and almost reflective

00:03:48.593 --> 00:03:50.760
experience, and really make
sure that you're picking

00:03:50.760 --> 00:03:52.200
the sounds that speak to you.

00:03:52.200 --> 00:03:54.250
So that way at the end,
when you have this song,

00:03:54.250 --> 00:03:56.850
it's kind of connected
to a memory or whatever

00:03:56.850 --> 00:03:59.340
specific feeling that
you're having at the time.

00:03:59.340 --> 00:04:00.780
CHARLENE: One big caveat--

00:04:00.780 --> 00:04:03.900
all the music that's in the app
is something that we composed,

00:04:03.900 --> 00:04:05.610
and we're not composers.

00:04:05.610 --> 00:04:08.530
So the music itself isn't--

00:04:08.530 --> 00:04:10.530
it's like the one we pick
out that's open source

00:04:10.530 --> 00:04:13.230
online and the one that we just
dropped our own little drum

00:04:13.230 --> 00:04:13.950
set drum beat.

00:04:13.950 --> 00:04:16.470
So they're not a wide set of it.

00:04:16.470 --> 00:04:19.560
So it's not super descriptive
of everyone's experience

00:04:19.560 --> 00:04:20.235
at that time.

00:04:20.235 --> 00:04:23.718
So don't hate on
the music too much.

00:04:23.718 --> 00:04:25.212
[LAUGHTER]

00:04:25.212 --> 00:04:26.208
AUDIENCE: Sounded good.

00:04:26.208 --> 00:04:29.250
CHARLENE: Oh, good.

00:04:29.250 --> 00:04:33.180
So implementation evolution--
so throughout our user

00:04:33.180 --> 00:04:36.545
testing, the how to
experience-- is it working well?

00:04:36.545 --> 00:04:37.170
AUDIENCE: Cool.

00:04:37.170 --> 00:04:38.087
CHARLENE: Yeah, great.

00:04:38.087 --> 00:04:40.170
So feel free if you
want to try it later on,

00:04:40.170 --> 00:04:42.270
come on and try it.

00:04:42.270 --> 00:04:44.820
Originally, we had
several implementation

00:04:44.820 --> 00:04:50.780
in terms of having
spatialized music

00:04:50.780 --> 00:04:52.880
where we describe having
the new audio coming

00:04:52.880 --> 00:04:54.350
in different direction.

00:04:54.350 --> 00:04:56.667
The previous iteration when
we first started the idea

00:04:56.667 --> 00:04:58.250
is that in order to
capture the music,

00:04:58.250 --> 00:05:01.220
we wanted to actually face
the direction of the music.

00:05:01.220 --> 00:05:03.987
You'd kind of be looking
at it with the frame

00:05:03.987 --> 00:05:06.320
because you're hearing it in
the direction of the sound.

00:05:06.320 --> 00:05:07.745
AUDIENCE: Should I turn it off?

00:05:07.745 --> 00:05:09.650
CHARLENE: Yeah.

00:05:09.650 --> 00:05:12.680
But then as we user test,
we realized again having--

00:05:12.680 --> 00:05:16.160
while moving forward and dictate
what a user's head direction be

00:05:16.160 --> 00:05:17.930
is not a reasonable
thing to ask.

00:05:17.930 --> 00:05:19.718
So we cut that part down.

00:05:19.718 --> 00:05:21.260
The user just had
the option later on

00:05:21.260 --> 00:05:23.720
to change how many
steps they want to take.

00:05:23.720 --> 00:05:25.512
There's a range between
low amount of step,

00:05:25.512 --> 00:05:27.262
depending on the
environment [INAUDIBLE],,

00:05:27.262 --> 00:05:29.730
to the higher amount of step
for the music to be generated.

00:05:29.730 --> 00:05:31.063
So they've gone for a long walk.

00:05:31.063 --> 00:05:32.897
They don't want to be
bombarded with a bunch

00:05:32.897 --> 00:05:33.980
of new music coming up.

00:05:33.980 --> 00:05:36.920
They select a longer generation.

00:05:36.920 --> 00:05:39.238
If they are indoor, for
example, if you just

00:05:39.238 --> 00:05:41.030
want to take a short
walk around the office

00:05:41.030 --> 00:05:43.100
and it's snowing
outside, then you

00:05:43.100 --> 00:05:49.600
do a short iteration,
short generation.

00:05:49.600 --> 00:05:52.410
THERESE: Yeah, so one of
our main research questions

00:05:52.410 --> 00:05:58.080
during this process was how we
can create a mind space using

00:05:58.080 --> 00:06:04.350
only audio to make the player
more meditative and reflective.

00:06:04.350 --> 00:06:08.310
So at first-- so we drew
a lot of inspiration

00:06:08.310 --> 00:06:09.610
from a lot of different places.

00:06:09.610 --> 00:06:12.750
So I think, obviously,
first is Pokémon GO.

00:06:12.750 --> 00:06:16.627
So originally, we were thinking
have a phone AR component,

00:06:16.627 --> 00:06:17.460
and you walk around.

00:06:17.460 --> 00:06:20.040
And in addition to
hearing the sounds,

00:06:20.040 --> 00:06:22.170
also have a little
sprite character

00:06:22.170 --> 00:06:24.150
where you would put your
phone there and then

00:06:24.150 --> 00:06:25.770
capture it on the phone.

00:06:25.770 --> 00:06:29.040
But we found that really drew
a person out of their own head

00:06:29.040 --> 00:06:32.100
and put them in more of
a virtual world, which

00:06:32.100 --> 00:06:35.640
was kind of the opposite
of what we wanted.

00:06:35.640 --> 00:06:38.820
Yeah, so then another thing that
we started thinking about was

00:06:38.820 --> 00:06:41.490
meditation, and how
when people meditate,

00:06:41.490 --> 00:06:45.970
a very common way to do that
is to count your breathing.

00:06:45.970 --> 00:06:49.590
So one way that we wanted
to implement that with audio

00:06:49.590 --> 00:06:52.110
is have a consistent cadence.

00:06:52.110 --> 00:06:54.420
So you kind of match
your steps to it,

00:06:54.420 --> 00:06:58.410
and it's that rhythmic,
meditative state.

00:06:58.410 --> 00:07:00.930
And we found that it
really helped out the users

00:07:00.930 --> 00:07:03.143
to get into that mindset.

00:07:03.143 --> 00:07:04.560
And then during
some play testing,

00:07:04.560 --> 00:07:06.248
we found in the
beginning, people

00:07:06.248 --> 00:07:07.290
were really stressed out.

00:07:07.290 --> 00:07:09.750
And it's a lot of things
going on, new sounds,

00:07:09.750 --> 00:07:11.500
a lot of decisions
that they have to make.

00:07:11.500 --> 00:07:13.583
And people were saying
that it was really intense.

00:07:13.583 --> 00:07:15.090
But the more that
they played it,

00:07:15.090 --> 00:07:18.360
it became almost second
nature, and players

00:07:18.360 --> 00:07:20.070
were really able to
get into that mindset

00:07:20.070 --> 00:07:21.540
as soon as they
put on the glasses,

00:07:21.540 --> 00:07:23.430
and wanted to make
these decisions

00:07:23.430 --> 00:07:25.390
and make their own creation.

00:07:25.390 --> 00:07:27.380
CHARLENE: So the user
that we've being asking

00:07:27.380 --> 00:07:29.880
around have been people who are
really into composing music,

00:07:29.880 --> 00:07:32.130
and people who are more
interested in just having

00:07:32.130 --> 00:07:33.690
music in their life in general.

00:07:33.690 --> 00:07:36.330
And as we work with
the instructor,

00:07:36.330 --> 00:07:38.850
there was a differentiation
between composing music

00:07:38.850 --> 00:07:41.430
and almost like the
experience of birdwatching,

00:07:41.430 --> 00:07:43.500
where you have spur of
the moment encounter,

00:07:43.500 --> 00:07:46.050
and you can choose to
capture it or let it go.

00:07:46.050 --> 00:07:48.870
And we want to emphasize
the feeling of you

00:07:48.870 --> 00:07:51.150
can save the music,
but you don't want to--

00:07:51.150 --> 00:07:53.250
very spur of the
moment, instead of keep

00:07:53.250 --> 00:07:56.340
tweaking it and premeditating
over the experience,

00:07:56.340 --> 00:07:59.460
and recomposing it
over and over again.

00:07:59.460 --> 00:08:02.750
THERESE: Yeah, so that leads
me to our next question,

00:08:02.750 --> 00:08:03.980
which was about control.

00:08:03.980 --> 00:08:07.690
So how does the level of
control that you give the user

00:08:07.690 --> 00:08:09.700
affect the user experience?

00:08:09.700 --> 00:08:12.670
So as you were
saying, we had a lot

00:08:12.670 --> 00:08:16.240
of different musical
backgrounds of play testers.

00:08:16.240 --> 00:08:18.145
So that can be
frustrating for people who

00:08:18.145 --> 00:08:20.290
don't know a lot about music.

00:08:20.290 --> 00:08:24.810
Or it was just kind of changing
the experience in a way

00:08:24.810 --> 00:08:26.640
that we didn't want it.

00:08:26.640 --> 00:08:30.720
So also, if you give
the user the option

00:08:30.720 --> 00:08:32.400
of as soon as you
get a sound, you

00:08:32.400 --> 00:08:35.190
can change all these
different things about it,

00:08:35.190 --> 00:08:37.950
it made walks much longer,
and drew people out

00:08:37.950 --> 00:08:38.700
of the experience.

00:08:38.700 --> 00:08:40.533
Because you don't want
to wait until the end

00:08:40.533 --> 00:08:43.299
to be like, oh, that thing that
I caught like 10 minutes ago,

00:08:43.299 --> 00:08:44.260
what can I do with it?

00:08:44.260 --> 00:08:47.910
So it was kind of
derailing the experience.

00:08:47.910 --> 00:08:52.500
And presenting it more as a
kind of birdwatching experience

00:08:52.500 --> 00:08:56.850
really helped align everyone's
expectations so that everyone

00:08:56.850 --> 00:09:00.360
would enjoy it, and kind of
dealing with what you get

00:09:00.360 --> 00:09:04.200
and making the most
of what you get also

00:09:04.200 --> 00:09:08.550
supported our goal of having
it as a meditative experience,

00:09:08.550 --> 00:09:10.500
and kind of analogous to life.

00:09:10.500 --> 00:09:14.078
You just make the most
of what you get, and have

00:09:14.078 --> 00:09:15.370
something beautiful in the end.

00:09:15.370 --> 00:09:18.180
So it was really exciting.

00:09:18.180 --> 00:09:20.840
CHARLENE: So the future
iteration of what we want to do

00:09:20.840 --> 00:09:22.760
is more--

00:09:22.760 --> 00:09:25.880
right now, the current set of
music is at a particular BPM.

00:09:25.880 --> 00:09:28.520
But sometimes on a slow day,
you want a slower melody.

00:09:28.520 --> 00:09:31.430
Or if you're on a fast day,
you want a higher, pumpy one.

00:09:31.430 --> 00:09:33.770
We would like to have the
option for the user to choose

00:09:33.770 --> 00:09:35.180
that in the beginning.

00:09:35.180 --> 00:09:38.160
Ideally, it's a input
that's more intuitive.

00:09:38.160 --> 00:09:41.655
So it's to have them
input-- so it's hard to--

00:09:41.655 --> 00:09:44.030
for someone who is not musically
inclined, if I tell you,

00:09:44.030 --> 00:09:47.210
oh, put it into
BPM of your choice,

00:09:47.210 --> 00:09:48.980
they're not going
to be able to have

00:09:48.980 --> 00:09:52.882
an idea of 140 BPM versus
120 BPM and how that feels.

00:09:52.882 --> 00:09:55.340
Instead, we want to be able to
incorporate more the sensors

00:09:55.340 --> 00:09:58.500
to address the gyroscope
or accelerometer to track

00:09:58.500 --> 00:10:00.433
your head nod to the
beat that they want.

00:10:00.433 --> 00:10:01.850
So if they want a
faster beat one,

00:10:01.850 --> 00:10:03.650
they could shake their
head a little bit faster.

00:10:03.650 --> 00:10:06.150
If they want a slow beat one,
it'd be a mellower head shake.

00:10:06.150 --> 00:10:07.790
And from the
tracking of the beat,

00:10:07.790 --> 00:10:10.160
you pre-select
what music you want

00:10:10.160 --> 00:10:12.720
to be able to offer to the user.

00:10:12.720 --> 00:10:19.070
So in the future, we want to
be able to passively track

00:10:19.070 --> 00:10:24.740
users' emotional state with
the physiological sensor that's

00:10:24.740 --> 00:10:30.860
embedded onto the AR frame, and
this way have a more continuous

00:10:30.860 --> 00:10:32.715
experience.

00:10:32.715 --> 00:10:34.340
THERESE: Yeah, and
then right now, it's

00:10:34.340 --> 00:10:35.828
a very personalized experience.

00:10:35.828 --> 00:10:38.120
And one thing that we wanted
to implement in the future

00:10:38.120 --> 00:10:40.070
is start to build a
community around that.

00:10:40.070 --> 00:10:43.340
So right now, at the end of
your walk you have this song,

00:10:43.340 --> 00:10:45.260
and you can just
save it for yourself.

00:10:45.260 --> 00:10:47.810
But in the future,
we'd like to have it

00:10:47.810 --> 00:10:49.640
as an open platform
for a community

00:10:49.640 --> 00:10:51.740
of other users on the app.

00:10:51.740 --> 00:10:55.490
And then maybe someone
has this really nice beat.

00:10:55.490 --> 00:10:57.230
And then someone else could--

00:10:57.230 --> 00:11:01.580
they could collaborate and add
vocals and make songs, or just

00:11:01.580 --> 00:11:03.800
share with your friends.

00:11:03.800 --> 00:11:06.740
CHARLENE: So the whole idea
started out with the feeling

00:11:06.740 --> 00:11:10.100
that music can capture
emotion or a certain moment.

00:11:10.100 --> 00:11:12.620
And we want to be able
to incorporate that

00:11:12.620 --> 00:11:14.120
into everyone's life.

00:11:14.120 --> 00:11:16.780
And that's the goal
of INMUSE, hopefully.

00:11:16.780 --> 00:11:19.250
Yeah, and that's it.

00:11:19.250 --> 00:11:22.600
[APPLAUSE]