This repository contains code to run faster sentence-transformers using tools like quantization, ONNX and pruning.
Project description
Fast Sentence Transformers
This repository contains code to run faster sentence-transformers using tools like quantization and ONNX. Just run your model much faster, while reducing memory a lot of memory. There is not much to it!
Quickstart
from fast_sentence_transformers import FastSentenceTransformer as SentenceTransformer
# use any sentence-transformer
encoder = SentenceTransformer("all-MiniLM-L6-v2", device=-1, quantize=True)
encoder.encode("Hello hello, hey, hello hello")
encoder.encode(["Life is too short to eat bad food!"] * 2)
shoutout
This package heavily leans on sentence-transformers and txtai.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for fast-sentence-transformers-0.1.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 05a4c66ac4d0463ed961916143426e74f4b2db03e75c524155925395c5e49c93 |
|
| MD5 | 316ff0c237f080fa4d09a45e1f4b4a74 |
|
| BLAKE2b-256 | 074e40862e8fc7ae50ca94e4ee954934be1ca931a1f9d3a965de23b3b93568b4 |
Close
Hashes for fast_sentence_transformers-0.1.0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 98bfc505ce384d66515baee267056c44001c0993672dc66c6ac45d2d66807f61 |
|
| MD5 | dd1d6334d7c2f94f2124bfa28d23de20 |
|
| BLAKE2b-256 | 0ea32435da6bc1ad2b06faf9d997d5469f374a93ca64d88d037d1baddf4e922f |
