LEKCut (เล็ก คัด) is a Thai tokenization library that ports the deep learning model to the onnx model.
Project description
LEKCut
LEKCut (เล็ก คัด) is a Thai tokenization library that ports the deep learning model to the onnx model.
Install
pip install lekcut
How to use
from lekcut import word_tokenize
word_tokenize("ทดสอบการตัดคำ")
# output: ['ทดสอบ', 'การ', 'ตัด', 'คำ']
API
word_tokenize(text: str, model: str="deepcut", path: str="default") -> List[str]
Model
deepcut- We ported deepcut model from tensorflow.keras to ONNX model. The model and code come from Deepcut's Github. The model is here.
Load custom model
If you has trained custom your model from deepcut or other that LEKCut support, You can load the custom model by path in word_tokenize after porting your model.
- How to train custom model ith your dataset by deepcut - Notebook (Needs to update
deepcut/train.pybefore train model)
How to porting model?
See notebooks/
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
LEKCut-0.1.tar.gz
(2.0 MB
view hashes)
Built Distribution
LEKCut-0.1-py3-none-any.whl
(2.0 MB
view hashes)
