Use this URL to cite or link to this record in EThOS:
Title: Adaptive Models of Chinese Text
Author: Wu, Peiliang
ISNI:       0000 0001 3573 3587
Awarding Body: University of Wales, Bangor
Current Institution: Bangor University
Date of Award: 2007
Availability of Full Text:
Access from EThOS:
This thesis is concerned with how to build adaptive language models of Chinese text, which can be used in different Chinese natural language processing (NLP) applications. The Prediction by Partial Matching (PPM) language model has been widely used in many NLP areas. To apply this model for Chinese, many problems arise that originate from the language's large alphabet. In this thesis, the PPM-ch model is introduced first to improve the traditional PPM model, by first using preprocessing techniques, then a frequency sorting technique and a variation of PPM that performs no exclusions. PPMO, a novel variant of the PPM model is then proposed. Unlike traditional PPM models, which output an escape symbol when a novel symbol occurs in a context model, PPMO separates the coding process into two streams, named the orders stream and the symbols stream. This algorithm is the first PPM variant that does not use the escape mechanism and it achieves the best compression results. We have also investigated Chinese Word segmentation by using our PPM-ch and PPMO model. Although our PPMO models havenot been carefully crafted for segmentation usage, we still achieve satisfactory results.
Supervisor: Not available Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available