Use natural language interface to create, deploy and update your microservice infrastructure.
Project description
GPT Deploy: One line to create them all 🧙🚀
Turn your natural language descriptions into fully functional, deployed microservices with a single command! Your imagination is the limit!
Quickstart
Requirements
- Create an account at cloud.jina.ai where your microservice will be deployed
Installation
pip install gptdeploy
gptdeploy configure --key <your openai api key>
If you set the environment variable OPENAI_API_KEY, the configuration step can be skipped.
Your api key must have access to gpt-4 to use this tool.
We are working on a way to use gpt-3.5-turbo as well.
Create Microservice
gptdeploy create --description "Given a PDF, return its text" --test "https://www.africau.edu/images/default/sample.pdf"
To create your personal microservice two things are required:
- A
descriptionof the task you want to accomplish. - A
testscenario that ensures the microservice works as expected.
The creation process should take between 5 and 15 minutes. During this time, GPT iteratively builds your microservice until it finds a strategy that make your test scenario pass. Once the microservice is created and deployed, you can test it using the generated Streamlit playground. The deployment is made on the Jina`s infrastructure. When creating a Jina account, you get some free credits, which you can use to deploy your microservice ($0.025/hour). If you run out of credits, you can purchase more.
Delete Microservice
To save credits you can delete your microservice via the following commands:
jc list # get the microservice id
jc delete <microservice id>
Examples
Animal Detector
gptdeploy create --description "Given an image, return the image with bounding boxes of all animals (https://pjreddie.com/media/files/yolov3.weights, https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg)" --test "https://images.unsplash.com/photo-1444212477490-ca407925329e contains animals"
Meme Generator
gptdeploy create --description "Generate a meme from an image and a caption" --test "Surprised Pikachu: https://media.wired.com/photos/5f87340d114b38fa1f8339f9/master/w_1600%2Cc_limit/Ideas_Surprised_Pikachu_HD.jpg, TOP:When you discovered GPTDeploy"
Rhyme Generator
gptdeploy create --description "Given a word, return a list of rhyming words using the datamuse api" --test "hello"
Word Cloud Generator
gptdeploy create --description "Generate a word cloud from a given text" --test "Lorem ipsum dolor sit amet, consectetur adipiscing elit."
3d model info
gptdeploy create --description "Given a 3d object, return vertex count and face count" --test "https://raw.githubusercontent.com/polygonjs/polygonjs-assets/master/models/wolf.obj"
Table extraction
--description "Given a URL, extract all tables as csv" --test "http://www.ins.tn/statistiques/90"
Audio to mel spectrogram
gptdeploy create --description "Create mel spectrograms from audio file" --test "https://cdn.pixabay.com/download/audio/2023/02/28/audio_550d815fa5.mp3"
Text to speech
gptdeploy create --description "Convert text to speech" --test "Hello, welcome to GPT Deploy!"
Heatmap Generator
gptdeploy create --description "Create a heatmap from an image and a list of relative coordinates" --test "https://images.unsplash.com/photo-1574786198875-49f5d09fe2d2, [[0.1, 0.2], [0.3, 0.4], [0.5, 0.6], [0.2, 0.1], [0.7, 0.2], [0.4, 0.2]]"
QR Code Generator
gptdeploy create --description "Generate QR code from URL" --test "https://www.example.com"
Mandelbrot Set Visualizer
gptdeploy create --description "Visualize the Mandelbrot set with custom parameters" --test "center=-0+1i, zoom=1.0, size=800x800, iterations=1000"
Technical Insights
The graphic below illustrates the process of creating a microservice and deploying it to the cloud elaboration two different implementation strategies.
graph TB
description[description: generate QR code from URL] --> make_strat{think a}
test[test: https://www.example.com] --> make_strat[generate strategies]
make_strat --> implement1[implement strategy 1]
implement1 --> build1{build image}
build1 -->|error message| implement1
build1 -->|failed 10 times| implement2[implement strategy 2]
build1 -->|success| registry[push docker image to registry]
implement2 --> build2{build image}
build2 -->|error message| implement2
build2 -->|failed 10 times| all_failed[all strategies failed]
build2 -->|success| registry[push docker image to registry]
registry --> deploy[deploy microservice]
deploy --> streamlit[create streamlit playground]
streamlit --> user_run[user tests microservice]
- GPT Deploy identifies several strategies to implement your task.
- It tests each strategy until it finds one that works.
- For each strategy, it creates the following files:
- executor.py: This is the main implementation of the microservice.
- test_executor.py: These are test cases to ensure the microservice works as expected.
- requirements.txt: This file lists the packages needed by the microservice and its tests.
- Dockerfile: This file is used to run the microservice in a container and also runs the tests when building the image.
- GPT Deploy attempts to build the image. If the build fails, it uses the error message to apply a fix and tries again to build the image.
- Once it finds a successful strategy, it:
- Pushes the Docker image to the registry.
- Deploys the microservice.
- Creates a Streamlit playground where you can test the microservice.
- If it fails 10 times in a row, it moves on to the next approach.
🔮 vision
Use natural language interface to create, deploy and update your microservice infrastructure.
✨ Contributers
If you want to contribute to this project, feel free to open a PR or an issue. In the following, you can find a list of things that need to be done.
next steps:
- check if windows and linux support works
- support gpt3.5-turbo
- add video to README.md
- bug: it can happen that the code generation is hanging forever - in this case aboard and redo the generation
- new user has free credits but should be told to verify account
Nice to have:
- smooth rendering animation of the responses
- if the user runs gptdeploy without any arguments, show the help message
- don't show this message: 🔐 You are logged in to Jina AI as florian.hoenicke (username:auth0-unified-448f11965ce142b6). To log out, use jina auth logout.
- rest endpoint instead of grpc since it is more popular
- put the playground into the custom gateway (without rebuilding the custom gateway)
- hide prompts in normal mode and show them in verbose mode
- tests
- clean up duplicate code
- support popular cloud providers - lambda, cloud run, cloud functions, ...
- support local docker builds
- autoscaling enabled for cost saving
- add more examples to README.md
- support multiple endpoints - example: todolist microservice with endpoints for adding, deleting, and listing todos
- support stateful microservices
- The playground is currently printed twice even if it did not change. Make sure it is only printed twice in case it changed.
- allow to update your microservice by providing feedback
- support for other large language models like Open Assistent
- for cost savings, it should be possible to insert less context during the code generation of the main functionality - no jina knowledge is required
- use gptdeploy list to show all deployments
- gptdeploy delete to delete a deployment
- gptdeploy update to update a deployment
- test param optional - but how would you test the pdf extractor without a pdf?
- section for microservices built by the community
- test feedback for playground generation (could be part of the debugging)
- should we send everything via json in the text attribute for simplicity?
- fix release workflow
Proposal:
- just generate the non-jina related code and insert it into an executor template
- think about strategies after the first approach failed?
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
gptdeploy-0.18.15.tar.gz
(22.1 kB
view hashes)
