With no prior knowledge of machine learning or device-specific deployment, you can deploy a computer vision model to a range of devices and environments using Roboflow Inference.
Project description
👋 hello
Roboflow Inference is an opinionated tool for running inference on state-of-the-art computer vision models. With no prior knowledge of machine learning or device-specific deployment, you can deploy a computer vision model to a range of devices and environments.
Inference provides a Python library and Docker interface. Using the Docker interface, you can scale up your inference as needed.
📌 Install pip vs Docker:
- pip: Installs
inference
into your Python environment. Lightweight, good for Python-centric projects. - Docker: Packages
inference
with its environment. Ensures consistency across setups; ideal for scalable deployments.
💻 install
With ONNX CPU Runtime:
For CPU powered inference:
pip install inference
or
pip install inference-cpu
With ONNX GPU Runtime:
If you have an NVIDIA GPU, you can accelerate your inference with:
pip install inference-gpu
Without ONNX Runtime:
Roboflow Inference uses Onnxruntime as its core inference engine. Onnxruntime provides an array of different execution providers that can optimize inference on differnt target devices. If you decide to install onnxruntime on your own, install inference with:
pip install inference-core
Alternatively, you can take advantage of some advanced execution providers using one of our published docker images.
Extras:
Some functionality requires extra dependancies. These can be installed by specifying the desired extras during installation of Roboflow Inference.
extra | description |
---|---|
http |
Ability to run the http interface |
Example install with http dependancies:
pip install inference[http]
🐋 docker
You can learn more about Roboflow Inference Docker Image build, pull and run in our documentation.
- Run on x86 CPU:
docker run --net=host roboflow/roboflow-inference-server-cpu:latest
- Run on Nvidia GPU:
docker run --network=host --gpus=all roboflow/roboflow-inference-server-gpu:latest
👉 more docker run options
- Run on arm64 CPU:
docker run -p 9001:9001 roboflow/roboflow-inference-server-arm-cpu:latest
- Run on Nvidia GPU with TensorRT Runtime:
docker run --network=host --gpus=all roboflow/roboflow-inference-server-trt:latest
- Run on Nvidia Jetson with JetPack
4.x
:
docker run --privileged --net=host --runtime=nvidia roboflow/roboflow-inference-server-trt-jetson:latest
- Run on Nvidia Jetson with JetPack
5.x
:
docker run --privileged --net=host --runtime=nvidia roboflow/roboflow-inference-server-trt-jetson-5.1.1:latest
🔥 quickstart
Docker Quickstart:
import requests
dataset_id = "soccer-players-5fuqs"
version_id = "1"
image_url = "https://source.roboflow.com/pwYAXv9BTpqLyFfgQoPZ/u48G0UpWfk8giSw7wrU8/original.jpg"
#Replace ROBOFLOW_API_KEY with your Roboflow API Key
api_key = "ROBOFLOW_API_KEY"
confidence = 0.5
url = f"http://localhost:9001/{dataset_id}/{version_id}"
params = {
"api_key": api_key,
"confidence": confidence,
"image": image_url,
}
res = requests.post(url, params=params)
print(res.json())
Pip Quickstart:
After installing via pip, you can run a simple inference using:
from inference.core.data_models import ObjectDetectionInferenceRequest
from inference.models.yolov5.yolov5_object_detection import (
YOLOv5ObjectDetectionOnnxRoboflowInferenceModel,
)
model = YOLOv5ObjectDetectionOnnxRoboflowInferenceModel(
model_id="soccer-players-5fuqs/1", device_id="my-pc",
#Replace ROBOFLOW_API_KEY with your Roboflow API Key
api_key="ROBOFLOW_API_KEY"
)
request = ObjectDetectionInferenceRequest(
image={
"type": "url",
"value": "https://source.roboflow.com/pwYAXv9BTpqLyFfgQoPZ/u48G0UpWfk8giSw7wrU8/original.jpg",
},
confidence=0.5,
iou_threshold=0.5,
)
results = model.infer(request)
print(results)
📝 license
The Roboflow Inference code is distributed under an Apache 2.0 license. The models supported by Roboflow Inference have their own licenses. View the licenses for supported models below.
model | license |
---|---|
inference/models/clip |
MIT |
inference/models/sam |
Apache 2.0 |
inference/models/vit |
Apache 2.0 |
inference/models/yolact |
MIT |
inference/models/yolov5 |
AGPL-3.0 |
inference/models/yolov7 |
GPL-3.0 |
inference/models/yolov8 |
AGPL-3.0 |
🚀 enterprise
With a Roboflow Inference Enterprise License, you can access additional Inference features, including:
- Server cluster deployment
- Device management
- Active learning
- YOLOv5 and YOLOv8 model sub-license
To learn more, contact the Roboflow team.
📚 documentation
Visit our documentation for usage examples and reference for Roboflow Inference.
🏆 contribution
We would love your input to improve Roboflow Inference! Please see our contributing guide to get started. Thank you to all of our contributors! 🙏
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for inference_core-0.7.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 276a7f54ac36bf1bb649a21d167e881f26c40f324506f31445170cbbc6fc3116 |
|
MD5 | 7485387dafa8c10f17b0a0e8597990b7 |
|
BLAKE2b-256 | c46b8b342280649e1a6446f65f2c4fa1dbc62dac0a0f54204ca88bfd8962debf |