Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

On document representation and term weights in text classification

Liu, Ying ORCID: 2009. On document representation and term weights in text classification. Song, Min and Wu, Yi-Fang Brook, eds. Handbook of Research on Text and Web Mining Technologies, Hershey, PA, USA: Information Science Reference, pp. 1-22. (10.4018/978-1-59904-990-8.ch001)

Full text not available from this repository.


In the automated text classification, a bag-of-words representation followed by the tfidf weighting is the most popular approach to convert the textual documents into various numeric vectors for the induction of classifiers. In this chapter, we explore the potential of enriching the document representation with the semantic information systematically discovered at the document sentence level. The salient semantic information is searched using a frequent word sequence method. Different from the classic tfidf weighting scheme, a probability based term weighting scheme which directly reflect the term’s strength in representing a specific category has been proposed. The experimental study based on the semantic enriched document representation and the newly proposed probability based term weighting scheme has shown a significant improvement over the classic approach, i.e., bag-of-words plus tfidf, in terms of Fscore. This study encourages us to further investigate the possibility of applying the semantic enriched document representation over a wide range of text based mining tasks.

Item Type: Book Section
Date Type: Publication
Status: Published
Schools: Engineering
Subjects: T Technology > TA Engineering (General). Civil engineering (General)
Publisher: Information Science Reference
ISBN: 9781599049908
Related URLs:
Last Modified: 25 Oct 2022 08:03

Citation Data

Actions (repository staff only)

Edit Item Edit Item