Word embeddings (Word2Vec, GloVe, LexVec): lang-uk

Ukrainian NLP community
Word embeddings (Word2Vec, GloVe, LexVec)

Word embeddings (Word2Vec, GloVe, LexVec)

From the corpora we collected — news, articles, fiction, laws and legal texts — we computed the most widely used word embeddings: Word2Vec (and its improved version LexVec) and GloVe. We publish these models because computing them takes considerable time and server resources.

We built separate 300d models for every text category, and also computed them for lemmatized versions of the same corpora. If you need different model settings, take the corpora we prepared and compute the models to your needs.

Corpus	300d as is	300d lowercase	300d lemmatized	300d lemmatized lowercase
Fiction	LexVec 105MB Word2Vec 98MB GloVe 99MB	LexVec 100MB Word2Vec 93MB GloVe 94MB	LexVec 53MB Word2Vec 51MB GloVe 51MB	LexVec 53MB Word2Vec 50MB GloVe 50MB
News	LexVec 329MB Word2Vec 328MB GloVe 324MB	LexVec 296MB Word2Vec 296MB GloVe 292MB	LexVec 160MB Word2Vec 161MB GloVe 159MB	LexVec 156MB Word2Vec 157MB GloVe 155MB
Ubercorpus	LexVec 536MB Word2Vec 530MB GloVe 526MB	LexVec 483MB Word2Vec 480MB GloVe 476MB	LexVec 297MB Word2Vec 295MB GloVe 294MB	LexVec 291MB Word2Vec 289MB GloVe 288MB

With the help of Tetiana Kodliuk we created test sets, for evaluating the quality of the vectors, analogous to those that existed for English.
Quality evaluations were also performed; the results are available here.

word2vec parameters for small corpora (fiction):

./word2vec_standalone.py -size 300 -negative 7 -window 4 -threads 6 -min_count 10 -iter 5 -alpha 0.030

lexvec parameters for small corpora (fiction):

./lexvec -dim 300 -verbose 2 -negative 7 -subsample 1e-3 -window 4 -threads 6 -minfreq 10 -iterations 5 -alpha 0.030

GloVe parameters for small corpora (fiction):

MEMORY=4.0
VOCAB_MIN_COUNT=10
VECTOR_SIZE=300
MAX_ITER=15
WINDOW_SIZE=9
BINARY=2
NUM_THREADS=12
X_MAX=10:

For large corpora, minfreq is 25

Links

Word2Vec LexVec GloVe LexVec 105MB Tetiana Kodliuk test sets here Word2Vec 98MB GloVe 99MB LexVec 100MB Word2Vec 93MB GloVe 94MB LexVec 53MB Word2Vec 51MB LexVec 53MB Word2Vec 50MB GloVe 50MB LexVec 329MB Word2Vec 328MB GloVe 324MB LexVec 296MB Word2Vec 296MB GloVe 292MB LexVec 160MB Word2Vec 161MB GloVe 159MB LexVec 156MB Word2Vec 157MB GloVe 155MB LexVec 536MB Word2Vec 530MB GloVe 526MB LexVec 483MB Word2Vec 480MB GloVe 476MB LexVec 297MB Word2Vec 295MB GloVe 294MB LexVec 291MB Word2Vec 289MB GloVe 288MB GloVe 51MB