WEBVTT 00:00:00.000 --> 00:00:01.434 align:middle line:90% [SQUEAKING] 00:00:01.434 --> 00:00:02.390 align:middle line:90% [CLICKING] 00:00:02.390 --> 00:00:04.302 align:middle line:90% [RUSTLING] 00:00:04.302 --> 00:00:07.170 align:middle line:90% [CLICKING] 00:00:07.170 --> 00:00:09.307 align:middle line:90% 00:00:09.307 --> 00:00:11.390 align:middle line:84% MICHAEL SCOTT ASATO CUTHBERT: Hello, computational 00:00:11.390 --> 00:00:12.930 align:middle line:90% music analysts. 00:00:12.930 --> 00:00:18.020 align:middle line:84% I'm going to talk a little bit today about some topics that 00:00:18.020 --> 00:00:20.540 align:middle line:84% are not in themselves essentially 00:00:20.540 --> 00:00:23.960 align:middle line:84% computational although having a computer 00:00:23.960 --> 00:00:26.660 align:middle line:84% helps but are really about the intersection 00:00:26.660 --> 00:00:29.810 align:middle line:84% of mathematical and quantitative thinking 00:00:29.810 --> 00:00:34.740 align:middle line:84% with music analysis, music theory, and musicology 00:00:34.740 --> 00:00:37.220 align:middle line:90% or the study of music history. 00:00:37.220 --> 00:00:39.740 align:middle line:84% And these are some of the things that I've 00:00:39.740 --> 00:00:41.630 align:middle line:84% done in my own research that I hope 00:00:41.630 --> 00:00:47.060 align:middle line:84% will make you think about some ways that data analysis can go 00:00:47.060 --> 00:00:50.450 align:middle line:84% beyond the score and tell us something more 00:00:50.450 --> 00:00:53.540 align:middle line:90% about the history of music. 00:00:53.540 --> 00:00:57.030 align:middle line:84% When I'm not doing computational music analysis, 00:00:57.030 --> 00:01:01.370 align:middle line:84% my other job is as a medieval music researcher, 00:01:01.370 --> 00:01:04.750 align:middle line:84% and one of the certainties I was always 00:01:04.750 --> 00:01:10.970 align:middle line:84% taught about medieval music is that we've lost most of it. 00:01:10.970 --> 00:01:14.770 align:middle line:84% We've lost so much of it, and it's just incredible 00:01:14.770 --> 00:01:15.980 align:middle line:90% how much we have. 00:01:15.980 --> 00:01:22.570 align:middle line:84% In fact, most people always said that the music that survives 00:01:22.570 --> 00:01:24.530 align:middle line:90% is just the tip of the iceberg. 00:01:24.530 --> 00:01:28.480 align:middle line:84% It was first said by a very important medieval historian-- 00:01:28.480 --> 00:01:32.050 align:middle line:84% musicologist Nino Pirrotta, but this term 00:01:32.050 --> 00:01:37.130 align:middle line:84% was picked up from on by dozens of other people. 00:01:37.130 --> 00:01:42.410 align:middle line:84% And so when we think about what's lost, 00:01:42.410 --> 00:01:44.420 align:middle line:84% it could be a number of manuscripts that's lost 00:01:44.420 --> 00:01:48.130 align:middle line:84% or it could be things that were lost because they were never 00:01:48.130 --> 00:01:49.940 align:middle line:84% written down and we've forgotten them, 00:01:49.940 --> 00:01:53.650 align:middle line:84% but if we want to think about even the things that were once 00:01:53.650 --> 00:01:56.510 align:middle line:84% written down on songs that were once written down, 00:01:56.510 --> 00:01:59.800 align:middle line:84% the pieces that were once written down, how many of them 00:01:59.800 --> 00:02:03.520 align:middle line:84% do we no longer have any copy of at all? 00:02:03.520 --> 00:02:05.350 align:middle line:90% What is the tip of the iceberg? 00:02:05.350 --> 00:02:07.790 align:middle line:90% Is it 90%? 00:02:07.790 --> 00:02:11.130 align:middle line:84% That's what a lot of people say the visible part of the iceberg 00:02:11.130 --> 00:02:11.630 align:middle line:90% is. 00:02:11.630 --> 00:02:14.830 align:middle line:90% Is it 99% lost? 00:02:14.830 --> 00:02:17.630 align:middle line:90% 99.9% lost, or as 1%-- 00:02:17.630 --> 00:02:19.370 align:middle line:90% basically everything. 00:02:19.370 --> 00:02:23.710 align:middle line:90% So I'll call that 99.9999%. 00:02:23.710 --> 00:02:27.310 align:middle line:84% My friend and former teacher, Sean Gallagher 00:02:27.310 --> 00:02:31.670 align:middle line:84% once asked me, hey, would there be any way-- 00:02:31.670 --> 00:02:35.930 align:middle line:84% you're a math guy-- to prove this, to show that it's true. 00:02:35.930 --> 00:02:39.760 align:middle line:84% And I immediately said, no, there's no way we could do this. 00:02:39.760 --> 00:02:41.930 align:middle line:84% Then I thought a little bit more and realized 00:02:41.930 --> 00:02:47.530 align:middle line:84% there are people who look at how things get lost and stuff. 00:02:47.530 --> 00:02:50.350 align:middle line:84% Later on I found this great article 00:02:50.350 --> 00:02:54.640 align:middle line:84% on what happens in peer-to-peer song sharing 00:02:54.640 --> 00:02:58.080 align:middle line:84% when you just start disconnecting lots of computers 00:02:58.080 --> 00:03:01.920 align:middle line:84% from the network as happened during the crackdown on Napster 00:03:01.920 --> 00:03:04.020 align:middle line:90% and things like that. 00:03:04.020 --> 00:03:06.060 align:middle line:84% And they showed that as you go from-- 00:03:06.060 --> 00:03:08.830 align:middle line:84% if you have a 100,000 computers on a network, 00:03:08.830 --> 00:03:11.400 align:middle line:90% you might have 5 million songs. 00:03:11.400 --> 00:03:16.380 align:middle line:84% But if the network gets down to maybe 10,000 or 15,000 00:03:16.380 --> 00:03:18.910 align:middle line:84% computers, you only lose about half the songs. 00:03:18.910 --> 00:03:20.830 align:middle line:90% So that was interesting to me. 00:03:20.830 --> 00:03:23.050 align:middle line:84% That was a little bit counterintuitive 00:03:23.050 --> 00:03:27.060 align:middle line:84% that you could really lose a lot of nodes on a network 00:03:27.060 --> 00:03:29.560 align:middle line:84% and not lose a lot of the content there. 00:03:29.560 --> 00:03:32.880 align:middle line:84% And I thought, well, computers might 00:03:32.880 --> 00:03:36.360 align:middle line:84% be something like medieval manuscripts 00:03:36.360 --> 00:03:39.220 align:middle line:84% that we lose a bunch, but maybe we still have the songs. 00:03:39.220 --> 00:03:42.580 align:middle line:84% So I wrote two articles about this topic. 00:03:42.580 --> 00:03:46.210 align:middle line:84% First is what I'm really going to be talking about now. 00:03:46.210 --> 00:03:48.300 align:middle line:84% It's an article called "Tipping the Iceberg" 00:03:48.300 --> 00:03:51.510 align:middle line:84% in a journal called Musica Disciplina and another, 00:03:51.510 --> 00:03:53.170 align:middle line:84% it's a little bit harder to find. 00:03:53.170 --> 00:03:56.190 align:middle line:84% It's called Monks, Manuscripts, and Other Peer-to-Peer Song 00:03:56.190 --> 00:03:58.570 align:middle line:84% Sharing Networks of the Middle Ages. 00:03:58.570 --> 00:04:03.420 align:middle line:84% It's a great book that the University of Pennsylvania 00:04:03.420 --> 00:04:06.030 align:middle line:84% helped publish called Cantus Scriptus-- 00:04:06.030 --> 00:04:07.840 align:middle line:90% Technologies of Medieval Song. 00:04:07.840 --> 00:04:12.870 align:middle line:84% I like the idea of technology existing throughout history. 00:04:12.870 --> 00:04:15.180 align:middle line:84% Anyhow so one of the ways that we 00:04:15.180 --> 00:04:19.560 align:middle line:84% could do this is just look at all the records of missing 00:04:19.560 --> 00:04:22.330 align:middle line:84% pieces and see how many of them we have. 00:04:22.330 --> 00:04:26.940 align:middle line:84% And that's sarcastic, but there are things that 00:04:26.940 --> 00:04:29.220 align:middle line:90% exist in the world like this-- 00:04:29.220 --> 00:04:32.250 align:middle line:84% there's a manuscript, it's actually 00:04:32.250 --> 00:04:37.530 align:middle line:84% only I think two or four pages survive now 00:04:37.530 --> 00:04:41.290 align:middle line:84% that was owned by the Duchess of Trémouille. 00:04:41.290 --> 00:04:46.080 align:middle line:84% So it's often called the Trémouille manuscript. 00:04:46.080 --> 00:04:49.840 align:middle line:84% It's in the National Library of Paris now. 00:04:49.840 --> 00:04:55.250 align:middle line:84% And if you look at this manuscript, what-- 00:04:55.250 --> 00:04:56.880 align:middle line:84% we only have a couple pages of it, 00:04:56.880 --> 00:05:01.190 align:middle line:84% but we have the entire index, the table of contents 00:05:01.190 --> 00:05:04.400 align:middle line:84% that tells you what the names of the pieces are 00:05:04.400 --> 00:05:06.980 align:middle line:84% and what page they would have been 00:05:06.980 --> 00:05:08.520 align:middle line:90% on in the original manuscript. 00:05:08.520 --> 00:05:11.730 align:middle line:84% So we can look at-- well, we don't have this manuscript, 00:05:11.730 --> 00:05:14.300 align:middle line:84% but how many of these pieces do we-- 00:05:14.300 --> 00:05:15.390 align:middle line:90% have we lost? 00:05:15.390 --> 00:05:19.080 align:middle line:84% So we can see that I've marked here, 00:05:19.080 --> 00:05:21.560 align:middle line:84% these are all the pieces that were pretty 00:05:21.560 --> 00:05:24.630 align:middle line:90% sure we don't have copies of. 00:05:24.630 --> 00:05:28.970 align:middle line:84% And here's all the pieces that we're pretty 00:05:28.970 --> 00:05:31.310 align:middle line:90% sure we do have copies of. 00:05:31.310 --> 00:05:33.710 align:middle line:84% And there's some pieces that we can't really 00:05:33.710 --> 00:05:39.230 align:middle line:84% be sure they're parts of medieval music 00:05:39.230 --> 00:05:42.020 align:middle line:84% traditions like the mass that have these somewhat 00:05:42.020 --> 00:05:45.960 align:middle line:84% generic names like credo or something, like if we saw, 00:05:45.960 --> 00:05:47.370 align:middle line:90% oh, we lost symphony. 00:05:47.370 --> 00:05:51.905 align:middle line:84% Well, whose symphony, things like that. 00:05:51.905 --> 00:05:54.380 align:middle line:84% So when we overlay these two together, 00:05:54.380 --> 00:05:59.050 align:middle line:84% we see that actually those that survive really outrank those 00:05:59.050 --> 00:05:59.660 align:middle line:90% that don't. 00:05:59.660 --> 00:06:03.500 align:middle line:84% So in this particular index, we have 114 compositions 00:06:03.500 --> 00:06:10.600 align:middle line:84% that of which 109 are identifiable and 68% of them 00:06:10.600 --> 00:06:12.610 align:middle line:90% seem to survive. 00:06:12.610 --> 00:06:15.370 align:middle line:84% Or another thing that I looked at 00:06:15.370 --> 00:06:21.910 align:middle line:84% was this collection of sonnets by cool but second 00:06:21.910 --> 00:06:24.430 align:middle line:84% tier Italian poet named Prodenzani. 00:06:24.430 --> 00:06:28.330 align:middle line:84% And one of the things that makes a lot of people 00:06:28.330 --> 00:06:32.300 align:middle line:84% think he's second tier is he throws in a lot of his songs 00:06:32.300 --> 00:06:36.010 align:middle line:84% just lists of things-- lists of all the food they eat-- ate, 00:06:36.010 --> 00:06:39.200 align:middle line:84% lists of all the dances they danced that day, 00:06:39.200 --> 00:06:42.490 align:middle line:84% and in this particular case, a list of all the songs 00:06:42.490 --> 00:06:43.570 align:middle line:90% that were sung. 00:06:43.570 --> 00:06:45.730 align:middle line:90% That begins at the top of-- 00:06:45.730 --> 00:06:49.180 align:middle line:84% nice for where we're about to be in the year 00:06:49.180 --> 00:06:52.180 align:middle line:84% with the violin, viola, something. 00:06:52.180 --> 00:06:57.100 align:middle line:84% They made all the may songs like "Rosetta Che Non Cambi Mai 00:06:57.100 --> 00:07:00.230 align:middle line:84% Colore," "The Rose That Never Changes Its Colors," 00:07:00.230 --> 00:07:03.950 align:middle line:84% "Je Sui Nafres Tam Fort, Dolce Sapore," et cetera, et cetera. 00:07:03.950 --> 00:07:07.600 align:middle line:84% We can basically figure out which ones of these things 00:07:07.600 --> 00:07:11.260 align:middle line:84% are titles of songs, and then we can see how many of them 00:07:11.260 --> 00:07:11.840 align:middle line:90% we know. 00:07:11.840 --> 00:07:17.140 align:middle line:84% So "Rosetta Che Non Cambi Mai Colore" is a song that survives. 00:07:17.140 --> 00:07:20.170 align:middle line:90% So is "Je Sui Nafres Tam Fort." 00:07:20.170 --> 00:07:23.930 align:middle line:84% But some of them like toward the bottom, 00:07:23.930 --> 00:07:27.850 align:middle line:84% "Costei Sarebbe Bella in Paradiso," 00:07:27.850 --> 00:07:29.390 align:middle line:90% we don't know that song. 00:07:29.390 --> 00:07:32.530 align:middle line:84% So we can look over, and, again, we can see how many of them 00:07:32.530 --> 00:07:35.600 align:middle line:84% survive, how many of them don't survive. 00:07:35.600 --> 00:07:39.280 align:middle line:84% And then these other four, where-- 00:07:39.280 --> 00:07:43.370 align:middle line:84% we can argue about, but there's songs with similar titles. 00:07:43.370 --> 00:07:46.340 align:middle line:84% Are they just being misquoted or are they same 00:07:46.340 --> 00:07:48.740 align:middle line:84% or is this really a song, things like that. 00:07:48.740 --> 00:07:52.390 align:middle line:84% So these are the two proportions that it could be. 00:07:52.390 --> 00:07:55.230 align:middle line:84% So depending on what we look at, there's somewhere 00:07:55.230 --> 00:08:00.030 align:middle line:84% between 16 and 17 compositions, and three to seven of them 00:08:00.030 --> 00:08:00.780 align:middle line:90% are lost. 00:08:00.780 --> 00:08:03.210 align:middle line:90% And 10 to 12 survive. 00:08:03.210 --> 00:08:09.750 align:middle line:84% So that's about 59% to 75% of the songs here survive. 00:08:09.750 --> 00:08:12.600 align:middle line:84% If we look at all of the sonnets in this collection 00:08:12.600 --> 00:08:16.110 align:middle line:84% by Prodenzani, there's 59 identifiable compositions, 00:08:16.110 --> 00:08:18.730 align:middle line:90% and 40 of them survive. 00:08:18.730 --> 00:08:21.630 align:middle line:90% So that's 71% 00:08:21.630 --> 00:08:23.940 align:middle line:84% So here's two pieces of evidence, 00:08:23.940 --> 00:08:29.820 align:middle line:84% both from textual sources, to try to see how much survives 00:08:29.820 --> 00:08:31.350 align:middle line:90% or how much doesn't. 00:08:31.350 --> 00:08:35.370 align:middle line:90% But can we do better? 00:08:35.370 --> 00:08:37.650 align:middle line:84% One of the things I thought of was 00:08:37.650 --> 00:08:40.590 align:middle line:84% that there are people who spend a lot of time trying 00:08:40.590 --> 00:08:43.140 align:middle line:84% to count things that are difficult to count because you 00:08:43.140 --> 00:08:45.390 align:middle line:90% can't capture them all. 00:08:45.390 --> 00:08:50.080 align:middle line:84% There the animal population in biologists, 00:08:50.080 --> 00:08:55.740 align:middle line:84% they are people who go out, and they can't capture every deer 00:08:55.740 --> 00:08:56.890 align:middle line:90% in a forest. 00:08:56.890 --> 00:09:00.630 align:middle line:90% So what do they do? 00:09:00.630 --> 00:09:03.400 align:middle line:84% They might go out-- here's one way that they do it. 00:09:03.400 --> 00:09:04.920 align:middle line:84% They might go out at a certain point 00:09:04.920 --> 00:09:10.350 align:middle line:84% and go and capture all the deer they 00:09:10.350 --> 00:09:12.550 align:middle line:90% can capture in a couple days. 00:09:12.550 --> 00:09:14.850 align:middle line:84% Maybe it's-- let's say it's 10, 10 deer. 00:09:14.850 --> 00:09:19.690 align:middle line:84% And they go, and they tag the deer in some way or something. 00:09:19.690 --> 00:09:22.570 align:middle line:84% Somehow they-- so they can recognize the same deer again. 00:09:22.570 --> 00:09:25.960 align:middle line:84% Then maybe they let a month go by or something, 00:09:25.960 --> 00:09:28.440 align:middle line:84% not too long but more-- enough time 00:09:28.440 --> 00:09:30.550 align:middle line:84% that animals can move around and stuff. 00:09:30.550 --> 00:09:33.130 align:middle line:84% And then they go out, and they capture deer for a day again. 00:09:33.130 --> 00:09:37.350 align:middle line:84% And let's say they capture 10 deer again. 00:09:37.350 --> 00:09:40.010 align:middle line:84% The point is, the second time they do a capture-- 00:09:40.010 --> 00:09:42.280 align:middle line:84% this is called capture recapture methods-- 00:09:42.280 --> 00:09:47.330 align:middle line:84% the second time they do it, they look to see how many of the deer 00:09:47.330 --> 00:09:49.610 align:middle line:84% that we captured the second time did 00:09:49.610 --> 00:09:51.740 align:middle line:90% we also capture the first time? 00:09:51.740 --> 00:09:53.720 align:middle line:84% If you think about it, if you go into a forest 00:09:53.720 --> 00:09:59.570 align:middle line:84% and you walked around pretty randomly or something and you-- 00:09:59.570 --> 00:10:02.570 align:middle line:84% all of the deer or nine out of the 10 deer 00:10:02.570 --> 00:10:05.180 align:middle line:84% that you capture the first time you 00:10:05.180 --> 00:10:08.510 align:middle line:84% see the same ones, the second time, you might think, 00:10:08.510 --> 00:10:12.790 align:middle line:84% wow, we've seen almost all the deer in the forest. 00:10:12.790 --> 00:10:15.695 align:middle line:90% 00:10:15.695 --> 00:10:20.870 align:middle line:84% On the other hand, let's say you capture 10 deer and none of them 00:10:20.870 --> 00:10:23.580 align:middle line:84% that you captured the second time had the tags. 00:10:23.580 --> 00:10:28.260 align:middle line:84% You'd be like, wow, I know I've seen 20 deer here, 00:10:28.260 --> 00:10:31.800 align:middle line:84% but law of averages suggests there's a lot more than 20 deer. 00:10:31.800 --> 00:10:37.160 align:middle line:84% And, in fact, there's formulas that can-- you can-- 00:10:37.160 --> 00:10:41.570 align:middle line:84% that people do with this type of thing. 00:10:41.570 --> 00:10:44.420 align:middle line:84% So I tried to figure out what would 00:10:44.420 --> 00:10:47.060 align:middle line:90% be the equivalent in music. 00:10:47.060 --> 00:10:54.340 align:middle line:84% And so I started to think, let's go and look at some equations 00:10:54.340 --> 00:10:56.380 align:middle line:90% we might make. 00:10:56.380 --> 00:10:59.330 align:middle line:90% If we had the number of pieces-- 00:10:59.330 --> 00:11:02.120 align:middle line:84% if we knew the number of pieces that originally existed 00:11:02.120 --> 00:11:04.600 align:middle line:84% and we multiplied by the probability 00:11:04.600 --> 00:11:07.750 align:middle line:84% that any given piece would be missing, 00:11:07.750 --> 00:11:11.950 align:middle line:84% we would know the total number of missing pieces. 00:11:11.950 --> 00:11:13.700 align:middle line:90% That's fantastic. 00:11:13.700 --> 00:11:17.840 align:middle line:90% That is a very hard equation. 00:11:17.840 --> 00:11:22.160 align:middle line:84% It's very hard to argue against, but it's completely useless. 00:11:22.160 --> 00:11:24.740 align:middle line:84% We don't know the number of pieces originally. 00:11:24.740 --> 00:11:26.710 align:middle line:84% We don't know the probability that any given 00:11:26.710 --> 00:11:28.550 align:middle line:90% piece would be missing. 00:11:28.550 --> 00:11:31.370 align:middle line:84% And therefore we don't know how many missing pieces are. 00:11:31.370 --> 00:11:37.060 align:middle line:84% So one equation, three unknowns, we don't know any of them. 00:11:37.060 --> 00:11:42.640 align:middle line:84% But we can think through some of the things here. 00:11:42.640 --> 00:11:46.580 align:middle line:84% I'm going to go through a quicker version of this, 00:11:46.580 --> 00:11:51.040 align:middle line:84% but I have elsewhere-- well, I have in my article a little bit 00:11:51.040 --> 00:11:54.460 align:middle line:84% deeper version that talks about all the un-- 00:11:54.460 --> 00:11:56.390 align:middle line:84% the assumptions that could be in play. 00:11:56.390 --> 00:12:00.190 align:middle line:84% But let's start with how can we figure out the probability 00:12:00.190 --> 00:12:02.030 align:middle line:84% that we're missing a given piece. 00:12:02.030 --> 00:12:09.790 align:middle line:84% Well, here's one model, and it's kind of a bad assumption 00:12:09.790 --> 00:12:10.640 align:middle line:90% maybe to start with. 00:12:10.640 --> 00:12:12.820 align:middle line:90% But here's one model. 00:12:12.820 --> 00:12:17.860 align:middle line:84% What if there's just medieval scribes had baskets 00:12:17.860 --> 00:12:19.910 align:middle line:90% of pieces, just all the pieces. 00:12:19.910 --> 00:12:22.368 align:middle line:84% They knew them all their head, there's something like that, 00:12:22.368 --> 00:12:24.160 align:middle line:84% and they randomly chose ones that they 00:12:24.160 --> 00:12:25.460 align:middle line:90% were going to write down. 00:12:25.460 --> 00:12:28.930 align:middle line:84% Well, then the probability that a given piece 00:12:28.930 --> 00:12:31.270 align:middle line:84% would be in a particular manuscript 00:12:31.270 --> 00:12:35.620 align:middle line:84% is approximately the proportion of all 00:12:35.620 --> 00:12:37.130 align:middle line:90% of the pieces in the manuscript. 00:12:37.130 --> 00:12:39.460 align:middle line:84% That is, if R sub I is the number of pieces 00:12:39.460 --> 00:12:42.220 align:middle line:84% in the manuscript and N is the total number of pieces 00:12:42.220 --> 00:12:44.550 align:middle line:84% originally copied, then we would know the probability 00:12:44.550 --> 00:12:48.420 align:middle line:84% that a piece would be in a given manuscript or songbook. 00:12:48.420 --> 00:12:50.880 align:middle line:84% It's not exactly that number because you 00:12:50.880 --> 00:12:55.020 align:middle line:84% don't do duplication, but it's pretty close. 00:12:55.020 --> 00:12:58.942 align:middle line:84% It's actually the people have argued against me that, well, 00:12:58.942 --> 00:13:00.400 align:middle line:84% you have to correct for that error, 00:13:00.400 --> 00:13:05.040 align:middle line:84% but you only have to correct for the probability of duplication 00:13:05.040 --> 00:13:07.740 align:middle line:84% if the number of pieces in that manuscript 00:13:07.740 --> 00:13:12.450 align:middle line:84% is close to some major percentage of the total number 00:13:12.450 --> 00:13:13.660 align:middle line:90% of pieces originally copied. 00:13:13.660 --> 00:13:16.570 align:middle line:90% So that's already getting that. 00:13:16.570 --> 00:13:18.057 align:middle line:84% Anyhow so this is the probability 00:13:18.057 --> 00:13:20.140 align:middle line:84% that a given piece would be in a given manuscript. 00:13:20.140 --> 00:13:22.920 align:middle line:84% From here, we can work out what's 00:13:22.920 --> 00:13:25.570 align:middle line:84% the probability the piece is not in a given manuscript. 00:13:25.570 --> 00:13:28.450 align:middle line:84% And we just subtract that from one. 00:13:28.450 --> 00:13:28.950 align:middle line:90% Great. 00:13:28.950 --> 00:13:31.170 align:middle line:90% So that's pretty good. 00:13:31.170 --> 00:13:34.510 align:middle line:84% Now let's say you have two independent manuscripts. 00:13:34.510 --> 00:13:36.350 align:middle line:84% They're being copied in different places, 00:13:36.350 --> 00:13:38.350 align:middle line:84% different people, whatever, something like that. 00:13:38.350 --> 00:13:40.600 align:middle line:84% What would be the probability that the piece would not 00:13:40.600 --> 00:13:43.600 align:middle line:84% be in any of these two manuscripts. 00:13:43.600 --> 00:13:46.230 align:middle line:84% And so you just multiply the probability 00:13:46.230 --> 00:13:49.660 align:middle line:84% that it wouldn't be in one manuscript and not in the other, 00:13:49.660 --> 00:13:51.960 align:middle line:84% and so R sub 1 and R sub 2, they're 00:13:51.960 --> 00:13:54.935 align:middle line:84% going to depend on how many pieces are in the manuscript. 00:13:54.935 --> 00:13:56.310 align:middle line:84% But this would be the probability 00:13:56.310 --> 00:14:00.420 align:middle line:84% that some piece existing or not would be in-- 00:14:00.420 --> 00:14:03.630 align:middle line:84% would not be in any of two manuscripts. 00:14:03.630 --> 00:14:07.200 align:middle line:84% So there's about 85 surviving medieval manuscripts 00:14:07.200 --> 00:14:11.370 align:middle line:84% in the period I'm most interested in, Italy from 00:14:11.370 --> 00:14:16.770 align:middle line:84% 1370-1420 just after the Black Death and during the time 00:14:16.770 --> 00:14:18.580 align:middle line:90% of the Great Papal Schism. 00:14:18.580 --> 00:14:21.090 align:middle line:84% So there's about 85 of these manuscripts, so what's 00:14:21.090 --> 00:14:23.130 align:middle line:84% the probability that any given piece would not 00:14:23.130 --> 00:14:27.150 align:middle line:84% be in any of the 85 surviving manuscripts? 00:14:27.150 --> 00:14:29.210 align:middle line:84% And it looks something like this. 00:14:29.210 --> 00:14:32.910 align:middle line:90% 00:14:32.910 --> 00:14:40.210 align:middle line:84% So this is something that it's a big long number, 00:14:40.210 --> 00:14:43.760 align:middle line:84% but you can see that right at the very end, this-- 00:14:43.760 --> 00:14:48.550 align:middle line:84% whatever this works out to is the equal to the probability 00:14:48.550 --> 00:14:49.800 align:middle line:90% that a piece would be missing. 00:14:49.800 --> 00:14:53.430 align:middle line:84% Because if a piece is not in any of the surviving manuscripts, 00:14:53.430 --> 00:14:56.490 align:middle line:90% then it's missing to us today. 00:14:56.490 --> 00:15:00.110 align:middle line:84% So this is great, but we know all these r numbers 00:15:00.110 --> 00:15:02.780 align:middle line:84% because they're simply the number of pieces in a manuscript 00:15:02.780 --> 00:15:06.050 align:middle line:90% today, not in originally. 00:15:06.050 --> 00:15:09.200 align:middle line:84% So we can take this, and we can go back 00:15:09.200 --> 00:15:11.810 align:middle line:84% to our original equation with three unknowns 00:15:11.810 --> 00:15:18.600 align:middle line:84% and substitute it into for that P of m place. 00:15:18.600 --> 00:15:22.760 align:middle line:84% So we can see that now we have one equation and two unknowns. 00:15:22.760 --> 00:15:25.700 align:middle line:84% It's still a pretty honking equation, 00:15:25.700 --> 00:15:27.390 align:middle line:90% but, well, that's good. 00:15:27.390 --> 00:15:29.970 align:middle line:84% One equation, two unknowns, still too many unknowns. 00:15:29.970 --> 00:15:31.640 align:middle line:84% We need two equations at least if we're 00:15:31.640 --> 00:15:33.300 align:middle line:90% going to do two unknowns. 00:15:33.300 --> 00:15:36.830 align:middle line:84% But we can look at this last answer, this m. 00:15:36.830 --> 00:15:40.070 align:middle line:84% What are the total number of missing pieces. 00:15:40.070 --> 00:15:42.700 align:middle line:84% Well, the total number of missing pieces 00:15:42.700 --> 00:15:46.100 align:middle line:84% is just the number of pieces that originally existed, 00:15:46.100 --> 00:15:50.420 align:middle line:84% which we don't know, minus the number of pieces that survive, 00:15:50.420 --> 00:15:51.350 align:middle line:90% which we do know. 00:15:51.350 --> 00:15:53.590 align:middle line:84% If we have a pretty good catalog of everything 00:15:53.590 --> 00:15:56.800 align:middle line:84% that survives today, then we can figure that out. 00:15:56.800 --> 00:15:59.950 align:middle line:84% So now we go back, and we see that we have actually just 00:15:59.950 --> 00:16:02.650 align:middle line:90% one equation and one unknown. 00:16:02.650 --> 00:16:05.020 align:middle line:90% It's a pretty big equation. 00:16:05.020 --> 00:16:07.780 align:middle line:84% In fact, a lot of the literature that I was reading from 00:16:07.780 --> 00:16:12.250 align:middle line:84% the '60s and '70s and '80s was about you can't possibly solve 00:16:12.250 --> 00:16:16.210 align:middle line:84% an equation of this dimensionality exactly. 00:16:16.210 --> 00:16:18.550 align:middle line:84% So here's all the sophisticated math 00:16:18.550 --> 00:16:25.010 align:middle line:84% that we can do to try to approximate it. 00:16:25.010 --> 00:16:29.090 align:middle line:84% But, in fact, nowadays it's pretty simple. 00:16:29.090 --> 00:16:31.570 align:middle line:84% What we can do is we can try different numbers 00:16:31.570 --> 00:16:36.550 align:middle line:84% for N, the number of original pieces, starting from one more 00:16:36.550 --> 00:16:39.050 align:middle line:84% than the number of surviving pieces or something. 00:16:39.050 --> 00:16:42.880 align:middle line:84% Try all these numbers into here until the left side 00:16:42.880 --> 00:16:45.850 align:middle line:84% and the right side approximately balance. 00:16:45.850 --> 00:16:47.390 align:middle line:84% And we can do that with computers. 00:16:47.390 --> 00:16:53.400 align:middle line:84% So when I did this, it's-- this is an article I think like 13, 00:16:53.400 --> 00:16:57.830 align:middle line:84% 14 years old, it's before Python was big. 00:16:57.830 --> 00:17:02.140 align:middle line:84% So I used Perl, and I just did find N, find 00:17:02.140 --> 00:17:04.310 align:middle line:84% the hypothetical total number of manuscripts 00:17:04.310 --> 00:17:07.430 align:middle line:84% given all the things that we know, and so on and so on. 00:17:07.430 --> 00:17:15.440 align:middle line:84% And we're able for each of four different genres-- 00:17:15.440 --> 00:17:16.940 align:middle line:84% these are different types of music-- 00:17:16.940 --> 00:17:20.829 align:middle line:84% we're able to figure out what percentage of the pieces 00:17:20.829 --> 00:17:23.150 align:middle line:90% are probably missing. 00:17:23.150 --> 00:17:24.880 align:middle line:90% These are-- I'm-- 00:17:24.880 --> 00:17:26.260 align:middle line:90% there's error bars on this. 00:17:26.260 --> 00:17:28.960 align:middle line:84% It's rough guesses, but we can see 00:17:28.960 --> 00:17:35.140 align:middle line:84% that the percentage of missing pieces may be in only one-- 00:17:35.140 --> 00:17:37.650 align:middle line:84% in only one of the genres is it predicted 00:17:37.650 --> 00:17:42.490 align:middle line:84% to be more than the number of total number of pieces 00:17:42.490 --> 00:17:43.990 align:middle line:84% than the number of surviving pieces. 00:17:43.990 --> 00:17:46.270 align:middle line:84% And, in fact, for some of these genres, 00:17:46.270 --> 00:17:49.025 align:middle line:84% we predict that, hey, we have almost everything there. 00:17:49.025 --> 00:17:52.260 align:middle line:90% 00:17:52.260 --> 00:17:56.020 align:middle line:84% So one of the things we can look at, 00:17:56.020 --> 00:17:58.710 align:middle line:84% instead of saying how many pieces were once copied 00:17:58.710 --> 00:18:04.800 align:middle line:84% but no longer exist, we can take that total number and say, wow, 00:18:04.800 --> 00:18:07.490 align:middle line:84% maybe we have all but 25% of them. 00:18:07.490 --> 00:18:11.940 align:middle line:90% 00:18:11.940 --> 00:18:15.030 align:middle line:84% Now there were a lot of assumptions that went into that. 00:18:15.030 --> 00:18:17.400 align:middle line:84% Scribes are collecting things randomly and so on 00:18:17.400 --> 00:18:19.290 align:middle line:84% and so forth and that there's not 00:18:19.290 --> 00:18:22.210 align:middle line:84% pieces that are more popular than each other and so on. 00:18:22.210 --> 00:18:24.220 align:middle line:84% And so in the article that I wrote, 00:18:24.220 --> 00:18:27.540 align:middle line:84% there's ways that you can compensate for this and ways 00:18:27.540 --> 00:18:28.660 align:middle line:90% that we can test. 00:18:28.660 --> 00:18:33.060 align:middle line:84% For instance, we can hold out 10% of the manuscripts at random 00:18:33.060 --> 00:18:40.450 align:middle line:84% and then say not just how many pieces would we expect to-- 00:18:40.450 --> 00:18:41.980 align:middle line:90% would we expect were lost. 00:18:41.980 --> 00:18:45.660 align:middle line:84% But if we gained 10 more manuscripts, how many more 00:18:45.660 --> 00:18:49.600 align:middle line:84% pieces would we expect to have that we didn't know already 00:18:49.600 --> 00:18:52.060 align:middle line:90% and then we can look at? 00:18:52.060 --> 00:18:54.730 align:middle line:84% Now put those 10 manuscripts back and voila. 00:18:54.730 --> 00:18:55.315 align:middle line:90% I think 00:18:55.315 --> 00:18:56.940 align:middle line:84% I can't remember off the top of my head 00:18:56.940 --> 00:18:58.980 align:middle line:84% right now, one of those things I publish it 00:18:58.980 --> 00:19:01.350 align:middle line:84% so I don't have to remember, but I 00:19:01.350 --> 00:19:05.010 align:middle line:84% think that the percentage of pieces that were lost I 00:19:05.010 --> 00:19:09.550 align:middle line:84% think it was within 10% of something or-- 00:19:09.550 --> 00:19:12.360 align:middle line:84% so these models were pretty, pretty good. 00:19:12.360 --> 00:19:14.790 align:middle line:84% So we can use this mathematical thinking and just 00:19:14.790 --> 00:19:17.550 align:middle line:84% a little bit of programming to answer 00:19:17.550 --> 00:19:20.310 align:middle line:84% questions that were thought to be completely 00:19:20.310 --> 00:19:23.220 align:middle line:90% unanswerable in history. 00:19:23.220 --> 00:19:24.770 align:middle line:90% Thanks. 00:19:24.770 --> 00:19:35.000 align:middle line:90%