Page History

An overview of computational analysis, and exploration of strategies and challenges.

Prep

• bring text to play with in Voyant

Anatomy and Physiology of a Digital Book

• Two books - page images and companion OCR - (sample book from HTDL)
  • What our eyes like to read is images
  • What machines read  needs to be constrained and predictable.
• Books have metadata -
  
  • bibliographic (catalog view, MARC view)
  • structural
• OCR - where does it come from?
• OCR issues
• • language specific dictionaries
  • font styles
  • image resolution
  • handwriting

We count words - tokenization; why tokenize?

• What is a word? (spaces are like any other character for a machine)
• What isn't a word? 
  
  • Punctuation
  • page numbers
  • headers/footers
  • grammatical constructs of various languages (ex: negation, especially in French; passive reflexive in Spanish)

Introducing Control - "Microanalysis" and Voyant

• Voyant
• Load your text sample
• Playing with control
  • observe counts
  • stopwords
• What does exerting control do to our results. 

We calculate frequency

• Why not express our counts simply?  Why calculate frequencies?
• Is this misleading?  If so in what ways?

Biblio

Matt Jockers Book