Language detection using FastText and Spacy
Project description
spacy_fastlang
Install
Assuming you have a working python environment, you can simply install it using
pip install spacy_fastlang
Usage
The library exports a pipeline component called language_detector
that will set two spacy extensions
- doc._.language = ISO code of the detected language or
xx
as a fallback - doc._.language_score = confidence
import spacy_fastlang # noqa: F401 # pylint: disable=unused-import
nlp = spacy.load("...")
nlp.add_pipe("language_detector")
doc = nlp(en_text)
doc._.language == "..."
doc._.language_score >= ...
Options
Check the tests to see more examples and available options
License
Everythin is under MIT
except the default model which is distributed under Creative Commons Attribution-Share-Alike License 3.0 by facebook here
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spacy_fastlang-2.0.0.tar.gz
(782.4 kB
view hashes)
Built Distribution
spacy_fastlang-2.0.0-py3-none-any.whl
(780.4 kB
view hashes)
Close
Hashes for spacy_fastlang-2.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bed792486c8e8fe3c2c4e7566134f0980173525c200c0a5e285bcc98c9c7bd3c |
|
MD5 | 96c151c2c270741e87f103eb2dd6d0bb |
|
BLAKE2b-256 | 5094576ddc9dcf6841e0b7effbc0bda4f30696053e582b68407e888d6c98bb0e |