Skip to main content

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

Project description

MindSpore Serving

查看中文

Overview

MindSpore Serving is a lightweight and high-performance service module that helps MindSpore developers efficiently deploy online inference services in the production environment. After completing model training on MindSpore, you can export the MindSpore model and use MindSpore Serving to create an inference service for the model.

MindSpore Serving architecture:

MindSpore Architecture

MindSpore Serving includes two parts: Client and Server. On a Client node, you can deliver inference service commands through the gRPC or RESTful API. The Server consists of a Main node and one or more Worker nodes. The Main node manages all Worker nodes and their model information, accepts user requests from Clients, and distributes the requests to Worker nodes. Servable is deployed on a worker node, indicates a single model or a combination of multiple models and can provide different services in various methods. `

On the server side, when MindSpore is used as the inference backend,, MindSpore Serving supports the Ascend 910/310P/310 and Nvidia GPU environments. When MindSpore Lite is used as the inference backend, MindSpore Serving supports Ascend 310, Nvidia GPU and CPU environments. Client` does not depend on specific hardware platforms.

MindSpore Serving provides the following functions:

  • gRPC and RESTful APIs on clients
  • Pre-processing and post-processing of assembled models
  • Batch. Multiple instance requests are split and combined to meet the batch size requirement of the model.
  • Simple Python APIs on clients
  • The multi-model combination is supported. The multi-model combination and single-model scenarios use the same set of interfaces.
  • Distributed model inference

Installation

For details about how to install and configure MindSpore Serving, see the MindSpore Serving installation page.

Quick Start

MindSpore-based Inference Service Deployment is used to demonstrate how to use MindSpore Serving.

Documents

Developer Guide

For more details about the installation guide, tutorials, and APIs, see MindSpore Python API.

Community

Governance

MindSpore Open Governance

Communication

Contributions

Welcome to MindSpore contribution.

Release Notes

RELEASE

License

Apache License 2.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

mindspore_serving-2.0.2-cp39-none-any.whl (7.4 MB view hashes)

Uploaded CPython 3.9

mindspore_serving-2.0.2-cp38-none-any.whl (7.4 MB view hashes)

Uploaded CPython 3.8

mindspore_serving-2.0.2-cp37-none-any.whl (7.4 MB view hashes)

Uploaded CPython 3.7

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page