Extracting Semantic Representations From Word Co-Occurrence Statistics: Stop-Lists, Stemming, and Svd

Research output: Contribution to journalArticle

Authors

Colleges, School and Institutes

Abstract

In a previous article, we presented a systematic computational study of the extraction of semantic representations from the word-word co-occurrence statistics of large text corpora. The conclusion was that semantic vectors of pointwise mutual information values from very small co-occurrence windows, together with a cosine distance measure, consistently resulted in the best representations across a range of psychologically relevant semantic tasks. This article extends that study by investigating the use of three further factors-namely, the application of stop-lists, word stemming, and dimensionality reduction using singular value decomposition (SVD)-that have been used to provide improved performance elsewhere. It also introduces an additional semantic task and explores the advantages of using a much larger corpus. This leads to the discovery and analysis of improved SVD-based methods for generating semantic representations (that provide new state-of-the-art performance on a standard TOEFL task) and the identification and discussion of problems and misleading results that can arise without a full systematic study.

Details

Original languageEnglish
Pages (from-to)890-907
JournalBehavior Research Methods
Volume44
Issue number3
Publication statusPublished - 19 Jan 2012