An overview of computational analysis, and exploration of strategies and challenges.

Prep

• bring text to play with in Voyant

Anatomy and Physiology of a Digital Book

• Two books - page images and companion OCR - (sample book from HTDL)
  • What our eyes like to read is images
  • What machines read  needs to be constrained and predictable:"machine actionable"
• Books have metadata -
  
  • bibliographic (catalog view, MARC view)
  • structural
• OCR - where does it come from?
• OCR issues
• • language specific dictionaries
  • font styles
  • image resolution
  • handwriting

We count words - tokenization; why tokenize?

• What is a word? (spaces are like any other character for a machine)
• What isn't a word? 
  
  • Punctuation
  • page numbers
  • grammatical constructs of various languages (ex: negation, especially in French; passive reflexive in Spanish)
• What words do we prefer not to count?
  
  • headers/footers

Introducing Control - "Microanalysis" and Voyant

• Voyant
• Load your text sample
• Playing with control
  • observe counts
  • stopwords
• What does exerting control do to our results?  Does it change the validity of our assertions?
• What is signal?  What is noise?

We calculate frequency

• Why not express our counts simply?  Why calculate frequencies?
• Is this misleading?  If so in what ways?

Biblio

Writing about your results - the structure of scientific papers

Matt Jockers Book