EstNLTK -- Open source tools for Estonian natural language processing

EstNLTK provides common natural language processing functionality such as paragraph, sentence and word tokenization, morphological analysis, named entity recognition, etc. for the Estonian language.

The project is funded by EKT (Eesti Keeletehnoloogia Riiklik Programm).

This package contains EstNLTK's basic linguistic analysis, system and database tools:

• Text class with the Estonian NLP pipeline;
• tokenization tools: word, sentence and paragraph tokenization; clause segmentation;
• morphology tools: morphological analysis and disambiguation, spelling correction, morphological synthesis and syllabification, HFST based analyser, GT and UD converters;
• information extraction tools: addresses tagger, named entity recognizer, temporal expression tagger; tools for rule based and grammar based fact extraction;
• experimental taggers: verb chain detector, noun phrase chunker, adjective phrase tagger;
• syntactic analysis tools: preprocessing for syntactic analysis, VislCG3 and Maltparser based syntactic parsers;
• Estonian Wordnet and Collocation-Net;
• web taggers -- such as bert embeddings web tagger, neural named entity recognition web tagger, stanza syntax web tagger and stanza ensemble syntax web tagger;
• corpus importing tools -- tools for importing data from large Estonian corpora, such as the Reference Corpus or the National Corpus of Estonia;
• system taggers -- regex tagger, disambiguator, atomizer, merge tagger etc;
• utils for downloading additional resources (e.g. model files required by taggers);
• Postgres database tools;

Version 1.7

Installation

EstNLTK is available for osx, windows-64, and linux-64, and for python versions 3.8 to 3.11. You can install the latest version via PyPI:

pip install estnltk==1.7.2

Alternatively, you can install EstNLTK via Anaconda. Installation steps with conda:

1. create a conda environment with python 3.9, for instance:

conda create -n py39 python=3.9

2. activate the environment, for instance:

conda activate py39

3. install EstNLTK with the command:

conda install -c estnltk -c conda-forge estnltk=1.7.2

Note: for using some of the tools in estnltk, you also need to have Java installed in your system. We recommend using Oracle Java http://www.oracle.com/technetwork/java/javase/downloads/index.html, although alternatives such as OpenJDK (http://openjdk.java.net/) should also work.

Using on Google Colab

You can install EstNLTK on Google Colab environment via command:

!pip install estnltk==1.7.2

Documentation

EstNLTK's tutorials come in the form of jupyter notebooks.

• Starting point of tutorials

Additional educational materials on EstNLTK are available on web pages of an NLP course taught at the University of Tartu:

• https://github.com/d009/EstNLP (in Estonian)

Note: if you have trouble viewing jupyter notebooks in github (you get an error message Sorry, something went wrong. Reload? at loading a notebook), then try to open notebooks with the help of https://nbviewer.jupyter.org

Source

The source of the last release is available at the main branch.

Changelog is available here.

Citation

In case you use EstNLTK in your work, please cite us as follows:

@InProceedings{laur-EtAl:2020:LREC,
  author    = {Laur, Sven  and  Orasmaa, Siim  and  Särg, Dage  and  Tammo, Paul},
  title     = {EstNLTK 1.6: Remastered Estonian NLP Pipeline},
  booktitle = {Proceedings of The 12th Language Resources and Evaluation Conference},
  month     = {May},
  year      = {2020},
  address   = {Marseille, France},
  publisher = {European Language Resources Association},
  pages     = {7154--7162},
  url       = {https://www.aclweb.org/anthology/2020.lrec-1.884}
}

License: GNU General Public License v2.0

(C) University of Tartu