Emulator for Custom Handlers for Inference Endpoints
Project description
Emulator for Custom Handlers for Inference Endpoints
🤗 Inference Endpoints offers a secure production solution to easily deploy any 🤗 Transformers and Sentence-Transformers models from the Hub on dedicated and autoscaling infrastructure managed by Hugging Face.
🤗 Inference Endpoints support all of the 🤗 Transformers and Sentence-Transformers tasks as well as custom tasks not supported by 🤗 Transformers yet like speaker diarization and diffusion.
The hf-endpoints-emulator
package provides a simple way to test your custom handlers locally before deploying them to Inference Endpoints. It is also useful for debugging your custom handlers.
The package provides a hf-endpoints-emulator
command line tool that can be used to run your custom handlers locally. It also provides a hf_endpoint_emulator
Python package that can be used to run your custom handlers locally from Python.
Installation
pip install hf-endpoints-emulator
Usage
You can check the examples/
directory for examples on how to use the hf-endpoints-emulator
package.
Command Line
hf-endpoints-emulator --handler <handler>
This will start a web server that will run your custom handler. The web server will be accessible at http://localhost:5000
. You can then send requests to the web server to test your custom handler.
curl
curl --request POST \
--url http://localhost/:5000 \
--header 'Content-Type: application/json' \
--data '{
"inputs": "I like you."
}'
python
import requests
url = "http://localhost:5000/"
payload = {"inputs": "test"}
headers = {"Content-Type": "application/json"}
response = requests.request("POST", url, json=payload, headers=headers)
print(response.json())
Python pacakge
from hf_endpoints_emulator.emulator import emulate
emulate(handler_path="examples/my_handler.py", port=5000)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hf-endpoints-emulator-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9e82c5589159d9d7c9eefd0dba8188e57c511d56ac5fbc3d54d24d9e76ab6cc |
|
MD5 | 67ba5229b4177e182ab5899d49eb594b |
|
BLAKE2b-256 | fdf7173c7826a63fbc5837286212a8e50212ac3dbc8a87a81227fc2431266e39 |
Hashes for hf_endpoints_emulator-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ac15e1a0845176856db30818017e82b5697ce6ff83627b534c1ef9c2282b99f3 |
|
MD5 | efcad0453784f47de8cddd0dc6b37eb7 |
|
BLAKE2b-256 | f93cfaf45c590ae19f2404b805a2aca9b754b16d4043604f83365783dda92f30 |