Hectiq console collector

A python package to track your inference API using the Hectiq Console.

This service is for Hectiq's client only.

Installation

The installation is only possible from the repo for now.

pip install hectiq-console

Starlette middleware

Below is an example how to use the middleware for Starlette application.

from starlette.applications import Starlette
from starlette.middleware import Middleware
from hectiq_console import HectiqConsoleStarletteMiddleware
middleware = [
    Middleware(HectiqConsoleStarletteMiddleware, 
                ressource="hectiq-test")
]
app = Starlette(middleware=middleware)

FastAPI

Below is an example how to use the middleware for FastAPI. It shares the same Middleware as Starlette.

import time
import random
from fastapi import FastAPI, Request
from hectiq_console import HectiqConsoleStarletteMiddleware, store_metrics

app = FastAPI(title="Demo application")
app.add_middleware(HectiqConsoleStarletteMiddleware, 
                   ressource="hectiq-demo",
                   custom_metrics={
                    "random-number": "float"  
                   },
                   include_paths=["/predict"])

@app.get("/")
async def root():
    return {"message": "🚨 This route is not monitored by the hectiq console."}

@app.get("/predict")
async def root(request: Request):
    # Store a random number
    return {"message": "✅ This route is monitored by the hectiq console."}

Send metrics

By default, the middleware stores the latency and counts of the monitored requests. You may add other metrics using the store_metrics in a request handler.

@app.get("/predict")
async def root(request: Request):
    store_metrics(request=request, key=metrics_key, value=metrics_value)

You can send as many metrics in the same request as you want. However, if you use the same key in the same request, the previous value is overwritten by the new one.

Do not forget to create the metrics definition in the console beforehand. Otherwise, you'll get an error at handshake.

Send annotations

Annotations are useful to track predictions in your application. For example, you may want to track the result of a model. Use the method store_annotation. You can send as many annotations as you want in the same request.

@app.get("/predict")
async def root(request: Request):
    store_annotation(request=request, 
                        inputs={"y": [0,1,2,3,4], "x": [0,1,2,3,4]}, 
                        outputs={"y_true": [5,6,7,8], "y_pred": [5,6,7,8]}, 
                        label="high-accuracy",
                        metadata={"model": "demo-model", 
                                "accuracy": 0.89})

Send a file

🔨 To do.

Create an incident

🔨 To do.