Jina is the cloud-native neural search framework for any kind of data
Project description
Cloud-Native Neural Search? Framework for Any Kind of Data
Jina is a neural search framework that empowers anyone to build SOTA and scalable neural search applications in minutes.
โฑ๏ธ Save time - The design pattern of neural search systems. Quickly build solutions of indexing, querying, understanding multi-/cross-modal data such as video, image, text, audio, source code, PDF.
๐ฉ๏ธ Local & cloud friendly - Distributed architecture, scalable & cloud-native from day one. Same developer experience on local, Docker compose, Kubernetes.
๐ Serve, scale & share - Serve a local project with HTTP, WebSockets or gRPC endpoints in just minute. Scale your neural search applications to meet your availability and throughput requirements. Share and reuse building blocks from Hub.
๐ฑ Own your stack - Keep end-to-end stack ownership of your solution. Avoid integration pitfalls you get with fragmented, multi-vendor, generic legacy tools. Enjoy the integration with the neural search ecosystem including DocArray, Hub and Finetuner.
Install
pip install jina
For Jina 2.x users, please follow the below and read the migration guide:
pip uninstall jina && pip install jina
More install options including Conda, Docker, and Windows can be found here.
Documentation
Get Started
We promise you can build a scalable ResNet-powered image search service in 20 minutes or less, from scratch to Kubernetes. If not, you can forget about Jina.
Basic Concepts
Document, Executor and Flow are three fundamental concepts in Jina.
- Document is a data structure contains multi-modal data.
- Executor is a self-contained component and performs a group of tasks on Documents.
- Flow ties Executors together into a processing pipeline, provides scalability and facilitates deployments in the cloud.
Leveraging these three concepts, let's build a simple image search with the Totally Looks Like dataset. This is a microservice version of the DocArray example.
Create Executor for embedding and storing images
Preliminaries: download dataset, install PyTorch & Torchvision
We first build an Executor generating embeddings with PyTorch and ResNet50:
from jina import Executor, requests
from docarray import Document, DocumentArray
class ImageEmbeddingExecutor(Executor):
def __init__(self, **kwargs):
super().__init__(**kwargs)
import torchvision
self.model = torchvision.models.resnet50(pretrained=True)
@requests
def embedding(self, docs: DocumentArray, **kwargs):
docs.apply(self.preproc) # preprocess images
docs.embed(self.model, device='cuda') # embed via GPU to speed up
def preproc(self, d: Document):
return (
d.load_uri_to_image_tensor() # load
.set_image_tensor_shape((200, 200)) # resize all to 200x200
.set_image_tensor_normalization() # normalize color
.set_image_tensor_channel_axis(-1, 0)
) # switch color axis for the PyTorch model later
We need to build the second Executor for storing and retrieving images:
class IndexExecutor(Executor):
_docs = DocumentArray()
@requests(on='/index') # set the function to handle the `/index` endpoint
def index(self, docs: DocumentArray, **kwargs):
self._docs.extend(docs)
docs.clear() # save bandwidth as it is not needed
@requests(on='/search') # set the function to handle the `/search` endpoint
def search(self, docs: DocumentArray, **kwargs):
docs.match(self._docs, limit=9) # limit to returning top 9 matches
docs[...].embeddings = None # save bandwidth as it is not needed
docs[...].blobs = None # save bandwidth as it is not needed
Orchestrate Executors in a Flow
Building a Flow to wire up the Executors, we can index some images and start searching by sending requests to the Flow APIs.
Note that we only index the 100 images now for quick reproducible demo:
from jina import Client, Flow, DocumentArray, Document
img_dataset = DocumentArray.from_files('left/*.jpg')[:100]
if __name__ == '__main__':
f = Flow(port_expose=12345).add(uses=ImageEmbeddingExecutor).add(uses=IndexExecutor)
with f:
client = Client(port=12345)
client.post('/index', img_dataset)
query_img = Document(uri='right/000001.jpg')
response = client.post('/search', query_img)
(
response[0]
.docs[0]
.matches.apply(
lambda d: d.set_image_tensor_channel_axis(
0, -1
).set_image_tensor_inv_normalization()
)
.plot_image_sprites(output='matches.png')
) # save the matched images
You will find the query image at query.png and the top 9 matches at matches.png.
This is everything: You just level up your neural search application as an API service! ๐
If you want to expose your application with a REST API so that you can send HTTP requests,
just set the protocol of the Flow to http:
from jina import Flow
if __name__ == '__main__':
f = (
Flow(protocol='http', port_expose=12345)
.add(uses=ImageEmbeddingExecutor)
.add(uses=IndexExecutor)
)
with f:
f.block()
Now you can use cURL to send search requests:
curl -X POST http://127.0.0.1:12345/search -H 'Content-type: application/json' -d '{"data":[{"uri": "<data_set_path>/right/00000.jpg"}]}' > curl_response
Use Docker Compose or Kubernetes
If we want to further upgrade your Flow with Docker Compose or Kubernetes, we will first need to containerize the Executors. The easiest way to do that is by using Jina Hub.
Move each of the two Executors to a separate folder with one Python file in each:
ImageEmbeddingExecutor-> ๐embed_img/exec.pyIndexExecutor-> ๐match_img/exec.py
Create a requirements.txt in embed_img and add torchvision as a requirement.
.
โโโ embed_img
โ โโโ exec.py # copy-paste codes of ImageEmbeddingExecutor
โ โโโ requirements.txt # add the requirement `torchvision`
โโโ match_img
โโโ exec.py # copy-paste codes of IndexExecutor
Push all Executors to Jina Hub. (Important: Write down the string you get for the usage. It looks like this jinahub://1ylut0gf)
jina hub push embed_img # publish at jinahub+docker://1ylut0gf
jina hub push match_img # publish at jinahub+docker://258lzh3c
You will get two Hub Executors that can be used for any container.
Run Flow with Docker Compose
A Flow can generate a Docker Compose configuration file so that you can easily start a Flow via docker-compose up.
Replace the uses arguments in the Flow with the values you have got from Jina Hub from previous steps. This will run the Flow with containerized Executors.
Generate the docker compose configuration from the Flow using one line of Python code.
f = (
Flow(protocol='http', port_expose=12345)
.add(uses='jinahub+docker://1ylut0gf')
.add(uses='jinahub+docker://258lzh3c')
)
f.to_docker_compose_yaml() # By default, stored at `docker-compose.yml`
Flow@62548[I]:Docker compose file has been created under docker-compose.yml. You can use it by running `docker-compose up -f docker-compose.yml`
Now you can start your neural search application with docker compose.
docker-compose up
Deploy Flow with Kubernetes
You can easily deploy a Flow with containerized Executors to a Kubernetes cluster as well.
Create a Kubernetes cluster and get credentials (example in GCP, more K8s providers here):
gcloud container clusters create test --machine-type e2-highmem-2 --num-nodes 1 --zone europe-west3-a
gcloud container clusters get-credentials test --zone europe-west3-a --project jina-showcase
Create a namespace flow-k8s-namespace for demonstration purpose ,
kubectl create namespace flow-k8s-namespace
Generate the kubernetes configuration files using one line of code:
f.to_k8s_yaml('./k8s_config', k8s_namespace='flow-k8s-namespace')
k8s_config
โโโ executor0
โ โโโ executor0-head.yml
โ โโโ executor0.yml
โโโ executor1
โ โโโ executor1-head.yml
โ โโโ executor1.yml
โโโ gateway
โโโ gateway.yml
Use kubectl to deploy your neural search application:
kubectl apply -R -f ./k8s_config
Run port forwarding so that you can send requests to our Kubernetes application from local CLI :
kubectl port-forward svc/gateway -n flow-k8s-namespace 12345:12345
Now we have the Flow up running in Kubernetes and we can use the Client or cURL to send requests.
Note that we are running everything in the cloud and make sure the image URIs are accessible from the Kubernetes cluster.
Run Quick Demo
- ๐ Fashion image search:
jina hello fashion - ๐ค QA chatbot:
pip install "jina[demo]" && jina hello chatbot - ๐ฐ Multimodal search:
pip install "jina[demo]" && jina hello multimodal - ๐ด Fork the source of a demo to your folder:
jina hello fork fashion ../my-proj/ - Create a new Jina project:
jina new hello-jina
Support
- Join our Slack community to chat to our engineers about your use cases, questions, and support queries.
- Join our Engineering All Hands meet-up to
discuss your use case and learn Jina's new features.
- When? The second Tuesday of every month
- Where? Zoom (see our public calendar/.ical/Meetup group) and live stream on YouTube
- Subscribe to the latest video tutorials on our YouTube channel
Join Us
Jina is backed by Jina AI and licensed under Apache-2.0. We are actively hiring AI engineers, solution engineers to build the next neural search ecosystem in open source.
Contributing
We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
