Aproksimacija verovatnoća i optimalna veličina jezičkog uzorka

Translated title of the contribution: Probability estimate and the optimal text size

Aleksandar Kostić, Svetlana Ilić, Petar Milin

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

Reliable language corpus implies a text sample of size n that provides stable probability distributions of linguistic phenomena. The question is what is the minimal (i.e. the optimal) text size at which probabilities of linguistic phenomena become stable. Specifically, we were interested in probabilities of grammatical forms. We started with an a priori assumption that text size of 1.000.000 words is sufficient to provide stable probability distributions. Text of this size we treated as a "quasipopulation". Probability distribution derived from the "quasi-population" was then correlated with probability distribution obtained on a minimal sample size (32 items) for a given linguistic category (e.g. nouns). Correlation coefficient was treated as a measure of similarity between the two probability distributions. The minimal sample was increased by geometrical progression, up to the size where correlation between distribution derived from the quasi-population and the one derived from an increased sample reached its maximum (r=1). Optimal sample size was established for grammatical forms of nouns, adjectives and verbs. General formalism is proposed that allows estimate of an optimal sample size from minimal sample (i.e. 32 items).

Translated title of the contributionProbability estimate and the optimal text size
Original languageBosnian
Pages (from-to)35-51
Number of pages17
JournalPsihologija
Volume41
Issue number1
DOIs
Publication statusPublished - 1 Dec 2008

Keywords

  • Corpus linguistics
  • Reliability of text sample

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint

Dive into the research topics of 'Probability estimate and the optimal text size'. Together they form a unique fingerprint.

Cite this