Page History

...

How is a digital book made?  How does the structure relate to it's functionalityits function? What opportunities does this afford us in terms of text analysis?

...

Computational analysis of text

We count tokens - why What is tokenization?  Why tokenize?  What are some strategies for tokenizationused to tokenize?

Let's look again at the Arapaho gospel of St. Luke.  Switch to text view.

• What is a word?
• What isn't a word? 
• Can you think of special cases where tokens might contain more than one word?
• Are there words we prefer not to count in this exampleat all?
• Can you think of other types of words you might not want to count?
  
  • headers/footers
  • preface
  • indexes
  • title pages
  • abstracts
• What sets of rules would we need to tokenize?  Would these be ordered in any specific way?
• Is there a "right way" to tokenize?

What are the opportunity points that the structure and arrangement of a book afford?

• How do challenges with OCR intersect with strategies for computational analysis of text?  What might be effective strategies to deal with these challenges?
• What exactly is the "text"?  Can you think of parts of a book that you might not want to include in your analysis? Why or why not?

Introducing Control - "Microanalysis" and Voyant

...

• Why not express our counts simply (as counts)?  Why calculate frequencies?
• Is this misleading?  If so, in what ways?

Moving from Microanalysis to Macroanalysis

...