15.071 | Spring 2017 | Graduate

The Analytics Edge

5.4 Predictive Coding: Bringing Text Analytics to the Courtroom (Recitation)

5.4 Predictive Coding: Bringing Text Analytics to the Courtroom (Recitation)

Video 3: Pre-Processing

Important Note: In the following video, we ask you to use the “tm” package to perform the pre-processing steps. Due to function changes that occurred after this video was recorded, you will need to run the following command immediately after converting all of the words to lowercase letters (it converts all documents in the corpus to the PlainTextDocument type):

corpus = tm_map(corpus, PlainTextDocument)

Then you can continue with the R commands as they are in the video.

If the code length(stopwords(“english”)) does not return 174 for you, then please run the line of code in stopwords (TXT) file, which will store the standard stop words in a variable called sw. When removing stop words, use tm_map(corpus, removeWords, sw) instead of tm_map(corpus, removeWords, stopwords(“english”)). 

Course Info

As Taught In
Spring 2017
Level
Learning Resource Types
Lecture Videos
Lecture Notes
Problem Sets with Solutions