A Python NLP Library for Many Human Languages, by the Stanford NLP Group
Project description
Stanza: A Python NLP Library for Many Human Languages
The Stanford NLP Group's official Python NLP library. It contains support for running various accurate natural language processing tools on 60+ languages and for accessing the Java Stanford CoreNLP software from Python. For detailed information please visit our official website.
References
If you use our neural pipeline including the tokenizer, the multi-word token expansion model, the lemmatizer, the POS/morphological features tagger, the dependency parser, or the named entity recognition tool in your research, please kindly cite our CoNLL 2018 Shared Task system description paper:
@inproceedings{qi2018universal,
address = {Brussels, Belgium},
author = {Qi, Peng and Dozat, Timothy and Zhang, Yuhao and Manning, Christopher D.},
booktitle = {Proceedings of the {CoNLL} 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies},
month = {October},
pages = {160--170},
publisher = {Association for Computational Linguistics},
title = {Universal Dependency Parsing from Scratch},
url = {https://nlp.stanford.edu/pubs/qi2018universal.pdf},
year = {2018}
}
The PyTorch implementation of the neural pipeline in this repository is due to Peng Qi, Yuhao Zhang, and Yuhui Zhang, with help from Jason Bolton and Tim Dozat.
If you use the CoreNLP software through Stanza, please cite the CoreNLP software package and the respective modules as described here ("Citing Stanford CoreNLP in papers"). The CoreNLP client is mostly written by Arun Chaganty, and Jason Bolton spearheaded merging the two projects together.
Issues and Usage Q&A
To ask questions, report issues or request features, please use the GitHub Issue Tracker.
Setup
Stanza supports Python 3.6 or later. We strongly recommend that you install Stanza from PyPI. If you already have pip, the Python package manager, installed on your system, simply run:
pip install stanza
this should also help resolve all of the dependencies of Stanza, for instance PyTorch 1.0.0 or above.
If you currently have a previous version of stanza
installed, use:
pip install stanza -U
Alternatively, you can also install from source of this git repository, which will give you more flexibility in developing on top of Stanza and training your own models. For this option, run
git clone https://github.com/stanfordnlp/stanza.git
cd stanza
pip install -e .
Running Stanza
Getting Started with the neural pipeline
To run your first Stanza pipeline, simply following these steps in your Python interactive interpreter:
>>> import stanza
>>> stanza.download('en') # This downloads the English models for the neural pipeline
# IMPORTANT: The above line prompts you before downloading, which doesn't work well in a Jupyter notebook.
# To avoid a prompt when using notebooks, instead use: >>> stanza.download('en', force=True)
>>> nlp = stanza.Pipeline() # This sets up a default neural pipeline in English
>>> doc = nlp("Barack Obama was born in Hawaii. He was elected president in 2008.")
>>> doc.sentences[0].print_dependencies()
The last command will print out the words in the first sentence in the input string (or Document
, as it is represented in Stanza), as well as the indices for the word that governs it in the Universal Dependencies parse of that sentence (its "head"), along with the dependency relation between the words. The output should look like:
('Barack', '4', 'nsubj:pass')
('Obama', '1', 'flat')
('was', '4', 'aux:pass')
('born', '0', 'root')
('in', '6', 'case')
('Hawaii', '4', 'obl')
('.', '4', 'punct')
See our getting started guide for more details.
Accessing Java Stanford CoreNLP software
Aside from the neural pipeline, this package also includes an official wrapper for acessing the Java Stanford CoreNLP software with Python code.
There are a few initial setup steps.
- Download Stanford CoreNLP and models for the language you wish to use
- Put the model jars in the distribution folder
- Tell the Python code where Stanford CoreNLP is located by setting the
CORENLP_HOME
environment variable (e.g., in *nix):export CORENLP_HOME=/path/to/stanford-corenlp-full-2018-10-05
We provide comprehensive examples in our documentation that show how one can use CoreNLP through Stanza and extract various annotations from it.
Online Colab Notebooks
To get your started, we also provide interactive Jupyter notebooks in the demo
folder. You can also open these notebooks and run them interactively on Google Colab. To view all available notebooks, follow these steps:
- Go to the Google Colab website
- Navigate to
File
->Open notebook
, and chooseGitHub
in the pop-up menu - Note that you do not need to give Colab access permission to your github account
- Type
stanfordnlp/stanza
in the search bar, and click enter
Trained Models for the Neural Pipeline
We currently provide models for all of the Universal Dependencies treebanks v2.5, as well as NER models for a few widely-spoken languages. You can find instructions for downloading and using these models here.
Batching To Maximize Pipeline Speed
To maximize speed performance, it is essential to run the pipeline on batches of documents. Running a for loop on one sentence at a time will be very slow. The best approach at this time is to concatenate documents together, with each document separated by a blank line (i.e., two line breaks \n\n
). The tokenizer will recognize blank lines as sentence breaks. We are actively working on improving multi-document processing.
Training your own neural pipelines
All neural modules in this library can be trained with your own data. The tokenizer, the multi-word token (MWT) expander, the POS/morphological features tagger, the lemmatizer and the dependency parser require CoNLL-U formatted data, while the NER model requires the BIOES format. Currently, we do not support model training via the Pipeline
interface. Therefore, to train your own models, you need to clone this git repository and set up from source.
For detailed step-by-step guidance on how to train and evaluate your own models, please visit our training documentation.
LICENSE
Stanza is released under the Apache License, Version 2.0. See the LICENSE file for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for stanza-1.0.0rc0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a122f83d83891cd733c180536aaf2a0797d46cc1f62b7c7ffe35283b6b51b8b |
|
MD5 | b089144542aebc6b3b2510c40dba4e1d |
|
BLAKE2b-256 | 94bcd95cbdd1ff666b17042356458691272fe9d59c965642f376c33152d7a08b |