This repository contains code to run faster sentence-transformers using tools like quantization, ONNX and pruning.
Project description
Fast Sentence Transformers
This repository contains code to run faster sentence-transformers
using tools like quantization and ONNX
. Just run your model much faster, while reducing memory a lot of memory. There is not much to it!
Quickstart
from fast_sentence_transformers import FastSentenceTransformer as SentenceTransformer
# use any sentence-transformer
encoder = SentenceTransformer("all-MiniLM-L6-v2", device=-1, quantize=True)
encoder.encode("Hello hello, hey, hello hello")
encoder.encode(["Life is too short to eat bad food!"] * 2)
Shout-Out
This package heavily leans on sentence-transformers
and txtai
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for fast-sentence-transformers-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | deb4c54ef8493871af82b841928f80b95ace8387cb37fc9227f7af0ad97c27e4 |
|
MD5 | f0c189331b910b9476b508146fc6a705 |
|
BLAKE2b-256 | 18a9f3a5d31ef54fa7aa023b576f57336bc0ccfa3aee83c007c689d8a20e94af |
Close
Hashes for fast_sentence_transformers-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 460c2de9ffc6b00461d67c605d8ebe204c4365dc206f15b94160f620b6feb5f5 |
|
MD5 | 9831bf204bf22387f9fd98fc2ca90c97 |
|
BLAKE2b-256 | cc9f8631df7782751883c4417401d24a1c2e81ecb78e1e3d77ca429f4db639bd |