ARCH 3819/ARCH5819 Text mining intro

Preparation

This page is a companion to a guest lecture prepared for ARCH 3819/ARCH5819. 
Please bring a laptop! If you have one, bring your own. If you do not have one, please feel free to check one out at the Olin circulation desk. Having a laptop will allow you to participate in the explorations and make discussion more meaningful.
No special software will be needed. All explorations will be done through a Web browser, without any special plugins.

Discussion of terms

There will be a short discussion where we define terms (at least loosely) that should give us a common language for our discussion and critique of these tools and their strategies.

Explorations

All analysis tools will be demonstrated; no prior knowledge of any tools will be required. The room is equipped with video input to allow easy sharing of your desktop on the screen, so if you discover something that you would like to share and discuss, we can easily do so.

nGrams

nGrams depict the frequency of a word or word phrase and are most often depicted over publication year.  We have two nGram tools, each leveraging different source data.

Google's nGram Viewer. Note that many modifications can be made to refine the analysis, so please consider the links below as starting points. Syntax for refinement is found on the About page.

Data:

Analysis:

Visualization:

• nGram tool - delete the words and supply your own.
  • Names of New York City
  • Spanish words in English publications
  • Frequency for which Roland Barthes has been cited in various languages
  • Terms related to racial integration
  • "Ebola" 1975-2008 in various languages (compare with data from WHO on Ebola outbreaks and a map of official languages of African countries)
• HathiTrust Research Center (HTRC) Bookworm

Voyant

Data:

Analysis:

Visualization:

Voyant is a low barrier text analysis tool that delivers a rich, interactive interface and a variety of visualizations.  Input format can be plain text, a PDF (with OCR), a MS Word Document or a URL for HTML analysis.  Please feel free to bring your own material for upload to the workshop, understanding that upload of any material will be subject to the Voyant privacy policy.  Sample texts and URLs for analysis are listed below for experimentation, in case you run low on ideas.

• Sample texts for upload are below, courtesy of Project Gutenberg. Download plain text version to your local machine for upload into the Voyant interface
  
  • Crane, Stephen, 1871-1900. The Red Badge of Courage: An Episode of the American Civil War.
  • Montessori, Maria, 1870-1952. (George, Anne E., Translator). The Montessori Method
  • Upham, Charles Wentworth, 1802-1875. Salem Witchcraft, Volumes I and II
• Sample URLS: copy and paste into the Voyant upload browser window to get started.
  • DSPS Press - http://blogs.cornell.edu/dsps/
  • Copyright Law of the United States of America and Related Laws Contained in Title 17 of the United States Code - http://www.copyright.gov/title17/92preface.html
  • Niagara Falls (Wikipedia) - http://en.wikipedia.org/wiki/Niagara_Falls
• Sample Visualization
  
  • Martin Luther King, Jr's "I Have A Dream..."

Network Analysis

Data:

Analysis:

Visualization:

Spatial and Temporal Representation

Data:

Analysis:

Visualization:

If time permits, we might look at Viewshare, a free tool provided by the Library of Congress that allows you to generate and customize interactive maps, timelines, facets, and tag cloud visualizations of digital collections.  The tool presupposes that you have an account and your data ready, including location and/or time related data fields in some basic forms.  A few helpful tutorials are available. Viewshare can be embedded into other web experiences.