Abstract

Grammatical category ambiguity (distinct from semantic and structural ambiguity) is extremely frequent in natural language. In the Brown Corpus (a million-word grammatically tagged sample of English prose) 11% of all word forms, or 48% of all word instances, occur as members of more than one grammatical category. These figures greatly under-represent actual categorial ambiguity, for instance because uncommon words may seem unambiguous when they are actually not. Such frequent ambiguity poses extreme problems of non-determinism for parsers. Therefore means of resolving such ambiguities are important to the progress of natural language processing systems.

This thesis examines probabilistic strategies for resolving categorial ambiguity. I consider the contextual probability of a given category, given a categorial context, and the relative probability that a given word form represents a particular category. Up to 96% of all words can be assigned the correct category without morphological analysis, special handling of idioms, or other non-probabilistic features. Dynamic programming yields disambiguation time directly proportional to text length. Probabilistic methods are thus both faster and more accurate than previous methods, and overcome the non-determinism which renders many other methods unworkable.

I apply these methods to the Brown Corpus (English) and to the Greek New Testament (140,000 words of Koiné Greek). I discuss the effects of various parameters on the accuracy of category assignment, and analyze the types and frequencies of residual errors. I report control studies which help to predict the algorithm’s effectiveness for unrestricted text, and investigate the amount of normalization text required to obtain reliable probability estimates. Analyses of related information-theoretic properties of natural language corpora are also included, for example, investigations of the effect of sample size on measurement of entropy.

Chapters

• Introduction
• Theoretical Foundations
• Previous Research
• Analysis
• Conclusions
• Collected Footnotes
• Bibliography

Appendices

• Appendix 01: Computer Methods
• Appendix 02: Brown Corpus Tag-set
• Appendix 03: English tag frequencies
• Appendix 04: English Ambiguity Sets
• Appendix 05: English Collocational Probabilities
• Appendix 06: Error Tables
• Appendix 07: Greek tag system
• Appendix 08: Greek Tag Frequencies
• Appendix 09: Greek Contractions
• Appendix 10: Graphs
  • Graph 1: RTP Weight vs. total accuracy (English)
  • Graph 2: RTP limit vs. accuracy (English, weighted .3)
  • Graph 3: RTP Weight vs. accuracy (English, with RAS)
  • Graph 4: Dictionary coverage vs. accuracy (English)
  • Graph 5: Dictionary coverage vs. accuracy (English, open-class)
  • Graph 6: Frequency vs. rank (English tagging errors)
  • Graph 7: Number of tokens vs. rank (English tags)
  • Graph 8: Number of types vs. rank (English tags)
  • Graph 9: Number of tokens vs. rank (English ambiguity sets)
  • Graph 10: Number of types vs. rank (English ambiguity sets)
  • Graph 11: Collocational probability vs. rank (English)
  • Graph 12: Collocational probability vs. rank (English)
  • Graph 13: Vocabulary vs. Zipfian entropy
  • Graph 14: Vocabulary vs. Zipfian relative entropy
  • Graph 15: Relative entropy vs. corpus size (English)
  • Graph 16: % tag pairs attested vs corpus size (English)
  • Graph 17: Number of types vs. rank (Greek tags)
  • Graph 18: Number of tokens vs. rank (Greek tags)
  • Graph 19: RTP limit vs. accuracy (Greek)
• List of Tables
  • Table 1: Winograd’s NP components
  • Table 2: Greek adjective inflections
  • Table 3: Ideal dictionary coverage
  • Table 4: English degrees of ambiguity
  • Table 5: Greek degrees of ambiguity
  • Table 6: Total vs. hard-case accuracy (Brown Corpus)
  • Table 7: Type coverage (after Choueka & Lusignan 1985)
  • Table 8: Syntactic tags, Beale (1985)
  • Table 9: Volsunga tagging accuracy (by genre)
  • Table 10: Accuracy with RTP only (baseline)
  • Table 11: Accuracy using collocations only
  • Table 12: Negative RTP weight vs. accuracy
  • Table 13: Tagging by halves of the Corpus
  • Table 14: Tags with accuracy < 60%
  • Table 15: Most frequent misassignments
  • Table 16: Frequent error pairs
  • Table 17: Some frequent errors
  • Table 18: Ideal entropy (tags)
  • Table 19: Actual entropy (tags)

Edition information

The original published form of the dissertation is available from University Microfilms, order number 9002217.

This electronic edition was converted from the original word-processor files to XHTML by the author. BBEdit did most of the table conversions automatically. I did the rest via an awful lot of regex changes and hand-editing.

I have added a few notes and corrections during the conversion process in 2012-2013. They are marked up as paragraphs or spans, with HTML class="annot", and appear in magenta type, either bracketed inline or in blocks like this:

This is how a recently-added note looks.

Differences of which the author is aware:

1. I have omitted most of the front matter (Preface, CV, Table of Contents).
2. The text has been made into one HTML file for each chapter and appendix (plus a CSS file to control layout.
3. I have added the original page numbers at the end of the titles of larger sections, in the same color as notes and corrections, like0; and added hierarchical section-numbers at the start of all headings. These should correspond exactly to the sequence and levels evident in the original Table of Contents (although no explicit section-numbering appeared there).
4. I have added many links. This starting file has the Abstract and links to all the other files, graphs, and tables. Chapters have a "next" link at the end, to the next file. The Bibliography has section dividers and links at the top to get the the start of each letter in the alphabetical list (by first author).
5. Citations have been linked to the corresponding Bibliography entries. During this process I discovered a few omissions and typos in the Bibliography, whose correction is indicated using the convention described above.
6. I have gathered the footnotes here, and they are linked from their original attachment points in the text. I have renumbered them using their original page, plus a dot, plus their original superscript symbol.
7. Equations have been manually converted to MathML; a few are incomplete.
8. Italics were lost. I've re-tagged some, but probably not all.

Copyright 1989, 2013, Steven J. DeRose.

This work by Steven J. DeRose is licensed under a Creative Commons Attribution - No Derivatives license. This means you are free to reproduce or distribute it, but must cite the source, and cannot make changes. For further information on this license, see http://creativecommons.org/licenses/by-sa/3.0/.

For the most recent version, see http://www.derose.net.

This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License.