Page History

...

Voyant is a low barrier text analysis tool that delivers a rich, interactive interface and a variety of visualizations based on token counts within a single or a few texts.  Input format can be plain text, a PDF (with OCR), an MS Word Document or a URL for HTML analysis. Documentation will help you use this tool and it's many features. Upload of any material will be subject to the Voyant privacy policy. Sample texts and URLs for analysis are listed below for experimentation, but feel free to use other source data that interests you. 

• Sample texts, courtesy of Project Gutenberg. Use the plain text version.
  
  • Harriet Beecher Stowe, 1811-1896. Uncle Tom's Cabin
  • Zora Neale Hurston, 1891-1960. Three Plays
  • Bret Harte, 1836-1902.  Urban Sketches
  • Steven Levy, 1951- . Hackers, Heroes of the Computer Revolution
  • Crane, Stephen, 1871-1900. The Red Badge of Courage: An Episode of the American Civil War.
  • Dickens, Charles, 1812-1870. A Tale of Two Cities.
  • Upham, Charles Wentworth, 1802-1875. Salem Witchcraft, Volumes I and II
• Sample URLS: copy and paste into the Voyant upload browser window to get started.Sample URLS: copy and paste into the Voyant upload browser window to get started.

  • Economics of Crisis - http://www.economicsofcrisis.com/indications.html 

  • Instructions to major John Sullivan. Washington, George, 1732-1799. The writings of George Washington from the original manuscript sources. Internet Archive.

  • Copyright Law of the United States of America and Related Laws Contained in Title 17 of the United States Code - http://www.copyright.gov/title17/92preface.html
  • From Politico - transcripts of the 2016 Presidential debate
    • First Debate (09/27/2016) http://www.politico.com/story/2016/09/full-transcript-first-2016-presidential-debate-228761
    • Second Debate (10/10/2016) http://www.politico.com/story/2016/10/2016-presidential-debate-transcript-229519
    • Third Debate (10/20/2016) http://www.politico.com/story/2016/10/full-transcript-third-2016-presidential-debate-230063
• Sample visualization: Dr. Martin Luther King, Jr. I Have a Dream speech.

(1) Visualization of derived data

...

• Experiment with stopwords - documentation
• Experiment with the slider for word counts
• Consider raw vs relative frequencies

...

Google's nGram Viewer. Use the links below as starting points; dynamic modifications can be made at any point. Rules for syntax can be found on the About page.

• a few racist depictions
• women authors of color
• computer and cybernetics - broken out by language
• enslaved * (This link will yield an error, but simply click on the blue button marked "search lots of books" and the error will resolve.)
• different types of alleys (This link will yield an error, but simply click on the blue button marked "search lots of books" and the error will resolve.)
• Seaports of the Mediterranean - note groupings by country
• graffiti - let's start with this, and make it more meaningful
• "Ralph Ellison" and "Invisible Man
• "lyric/lyrics"
• "Jarndyce and Jarndyce" and "Bleak House"
• vacant lot,urban renewal,gentrification

HathiTrust Research Center (HTRC) Bookworm.  (Tied to a corpus pre-1923). Again, consider these links as starting points.  Rules for faceting ad controls are available on the HTRC wiki. 

• monster in fiction/non-fiction/unknown
• mystery (contrast fiction and nonfiction)
• various words for "creole"
• computer in fiction/non-fiction conference proceedings
• Does truth correlate with beauty?"phonograph" in three languages

Discussion

• How easily can these examples above be refined and improved?
• Compare the two interfaces, especially as to the affordances and the limits of each. 
• What additional elements of control would be useful that aren't available?
• When we see unexpected or entirely expected wave forms, what do we make of these? 
  
  • How much can we read into coorelations?
  • Do these constitute discoveries or represent errors?  How can we distinguish?
  • Would the flaws be due to the data, the metadata, the algorithms?
• Are there lenses that we should be wary of?

More Macroanalysis: HTRC

HathiTrust Research Center (HTRC) is a collaborative research center (jointly managed by Indiana University and the University of Illinois) dedicated to developing cutting-edge software tools and cyberinfrastructure that enable advanced computational access to large amounts of digital text. A basic orientation of HTRC services is available, and features and steps for each are documented on the HTRC community wiki.  We will be spending time in the HTRC Portal looking at the results of a few algorithms as a sampling of possibilities (links below will not render for all, but are parked to make sharing easier).  Algorithms in the portal can

...

• In general, do the results look valid? Do any of these algorithms yield results that might be considered confusing or less than perfect?

• Note that results can be downloaded.  What might be advantages of this portability?
• Are there things that the researcher would want or need to know about these algorithms when making claims about results?

On your own - network analysis and image analysis

Immersion

Image analysis

Ukiyo-e.org is a database and image similarity analysis engine, created by John Resig to aide aid researchers in the study of Japanese woodblock prints.  The data is over 213,000 digital copies of prints from 24 institutions, and their cataloging metadata.  Metadata is indexed and searchable, as you might expect. (Details are noted in the about page.)  But images are also searchable.  Resig'sImage search uses the TinEye matching engine to determine edges in an uploaded sample and compares with analyzed edges in database, returning probable matches (edge analysis).  Use samples below to experiment.  

...