*** This version of Confluence is for testing only and contains a copy of content from June 29th 2026. No changes will be preserved. ***
...
(1) Let's look again at the Arapaho gospel of St. Luke. Switch to text view.
- Is this OCR accurate to the visually captured page?
- What is a word? How would you define "word" to a computer?
- What isn't a word? How would you tell a computer to exclude these?
- Consider languages with which you are familiar. Can you think of cases where tokens might contain more than one word?
- What sets of rules would we need in order to tokenize effectively? Would these be ordered in any specific way?
- Is there a "right way" to tokenize?
...
In general, do the results look valid? Do any of these algorithms yield results that might be considered a false positiveconfusing or less than perfect?
- Note that results can be downloaded. Can you think of advantages to What might be advantages of this portability?
- Are there things that the researcher would want or need to know about these algorithms when making claims about results?
...