WEBVTT

00:00:04.490 --> 00:00:08.680
We will discuss the results of
the classification tree model.

00:00:11.730 --> 00:00:15.080
So we first observe that
the overall accuracy

00:00:15.080 --> 00:00:20.300
of the method regarding the
percentage that it accurately

00:00:20.300 --> 00:00:27.340
predicts is 80%, compared
to 75% of the baseline.

00:00:27.340 --> 00:00:30.380
But notice that this is
done in an interesting way.

00:00:30.380 --> 00:00:35.260
For bucket one patients, the
two models are equivalent.

00:00:35.260 --> 00:00:38.390
But of course this
suggests the idea

00:00:38.390 --> 00:00:40.430
that healthy people
stay healthy,

00:00:40.430 --> 00:00:42.950
which is the idea of
the baseline model.

00:00:42.950 --> 00:00:47.080
The cost repeats is
valid in the data.

00:00:47.080 --> 00:00:50.000
But then for
buckets two to five,

00:00:50.000 --> 00:00:53.200
notice that the accuracy
increases substantially from

00:00:53.200 --> 00:00:58.370
31% to 60%-- it doubles--
from 21% to 53%--

00:00:58.370 --> 00:01:02.360
more than doubles-- and
from 19% to 39%-- doubles.

00:01:02.360 --> 00:01:06.680
There's an improvement from 23%
to 30%, not as big as before,

00:01:06.680 --> 00:01:09.510
but there is indeed an
improvement for bucket five.

00:01:09.510 --> 00:01:15.240
But notice the improvement on
the penalty from 0.56 to 0.52

00:01:15.240 --> 00:01:17.130
overall.

00:01:17.130 --> 00:01:20.640
A small improvement
in bucket one,

00:01:20.640 --> 00:01:26.280
but a significant improvement
as we increase on the buckets.

00:01:26.280 --> 00:01:32.320
For example, here
for bucket five,

00:01:32.320 --> 00:01:37.120
the penalty error decreases
from 1.88 to 1.01,

00:01:37.120 --> 00:01:38.280
a substantial improvement.

00:01:41.770 --> 00:01:44.039
So we observed that there's
a substantial improvement

00:01:44.039 --> 00:01:48.210
over the baseline, especially
as we go down on buckets.

00:01:48.210 --> 00:01:51.539
It doubles the accuracy over
the baseline in some cases.

00:01:54.789 --> 00:01:58.270
And so we have seen
there's a smaller accuracy

00:01:58.270 --> 00:02:03.860
improvement in bucket five, but
there's a much lower penalty

00:02:03.860 --> 00:02:06.980
in the prediction
for bucket five.

00:02:06.980 --> 00:02:09.470
So what is the edge
of the analytics

00:02:09.470 --> 00:02:12.050
provided to D2Hawkeye?

00:02:12.050 --> 00:02:15.190
First and foremost, there
was a substantial improvement

00:02:15.190 --> 00:02:16.950
in the company's
ability to identify

00:02:16.950 --> 00:02:19.050
patients who need
more attention.

00:02:21.900 --> 00:02:24.720
Another advantage was
related to the fact

00:02:24.720 --> 00:02:28.210
that the model was in fact
interpretable by physicians.

00:02:28.210 --> 00:02:30.930
So the physicians were
able to improve the model

00:02:30.930 --> 00:02:36.070
by identifying new variables
and refining existing variables.

00:02:36.070 --> 00:02:39.350
That really led to
further improvements.

00:02:39.350 --> 00:02:42.720
Finally, and quite
importantly, the analytics

00:02:42.720 --> 00:02:48.630
gave the company an edge
over the competition using--

00:02:48.630 --> 00:02:51.970
that the competition used
last century methods.

00:02:51.970 --> 00:02:53.960
And the use of machine
learning methods--

00:02:53.960 --> 00:02:55.890
in this case,
classification trees--

00:02:55.890 --> 00:02:59.850
provided an edge that
also helped Hawkeye

00:02:59.850 --> 00:03:04.100
when it was sold to
Verisk Analytics in 2009.