TensorSpace.js Python tool package
Project description
TensorSpace Converter
English | 中文
TensorSpace-Converter is a TensorSpace preprocess tool for pre-trained model from TensorFlow, Keras, TensorFlow.js. TensorSpace-Converter can extract hidden layer value from a pre-trained model and generate a new multi-output model for TensorSpace 3D visualization. TensorSpace-Converter steeps the learning curve of TensorSpace preprocessing. Using TensorSpace-Converter in development, it is possible to separate the work of model training and model visualization.
Table of Content
Motivation
TensorSpace is a 3D visualization framework for pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js. Before applying TensorSpace, there is an important pipeline —— preprocessing the pre-trained model ( Checkout this article for more information about TensorSpace preprocessing ). TensorSpace-Converter is designed for TensorSpace users to quickly accomplish the TensorSpace Preprocessing.
TensorSpace-Converter out-of-the-box supports preprocessing different formats of pre-trained model from TensorFlow, Keras, TensorFlow.js. It only takes several lines of TensorSpace-Converter codes to preprocess a pre-trained model. Before TensorSpace-Converter, to preprocess a pre-trained model for TensorSpace, developers shall master some frameworks, such as TensorFlow, Keras, tfjs-converter. For example, before TensorSpace-Converter, when preprocessing a Keras model, developers need to prepare a pre-trained Keras model, write some Keras codes to get a new multi-output model, and then use tfjs-converter to convert the multi-output model to a TensorSpace compatible format. By applying TensorSpace-Converter, it just takes 6 lines of TensorSpace-Converter commands to do the same thing.
As a component of TensorSpace ecosystem, TensorSpace-Converter simplify the TensorSpace preprocessing, steeps the learning curve of TensorSpace. As a development tool, TensorSpace-Converter makes it possible to separate the work of backend model training
and frontend model visualization
.
Fig. 1 - TensorSpace LeNet
Fig. 2 - TensorSpace Preprocess Concept
Getting Started
Install
Install the tensorspacejs pip package:
$ pip install tensorspacejs
If successfully downloading tensorspacejs
, you can check the TensorSpace-Converter version in command line:
$ tensorspacejs_converter -v
- Note
TensorSpace-Converter must run under Python 3.6, Node 11.3+. If you have other pre-installed Python version in your local environment, we suggest you to use Conda
to create a pure Python 3.6 virtual environment.
$ conda create -n envname python=3.6
Usage
Then we will introduce the usage and workflow of using TensorSpace and TensorSpace-Converter to convert a pre-trained model and visualizing pre-processed model with TensorSpace. And we will preprocess and visualize a MNIST-digit Keras model.
The sample files used in this tutorial includes pre-trained Keras model, TensorSpace-Converter script and TensorSpace visualization code.
Step 1: Use TensorSpace-Converter preprocess pre-trained model
$ tensorspacejs_converter \
--input_model_from="keras" \
--input_model_format="topology_weights_combined" \
--output_layer_names="Conv2D_1,MaxPooling2D_1,Conv2D_2,MaxPooling2D_2,Dense_1,Dense_2,Softmax" \
./PATH/TO/MODEL/keras_MNIST_model.h5 \
./PATH/TO/SAVE/DIR
Step 2: Load preprocessed into TensorSpace visualization model
model.load({
type: "keras",
url: "/PATH/TO/MODEL/model.json"
});
Converter API
Sample TensorSpace-Converter script:
$ tensorspacejs_converter \
--input_model_from="XXX" \
--input_model_format="YYY" \
--output_layer_names="EEE1,EEE2,EEE3" \
input_path \
output_path
Arguments explanation:
Positional Arguments | Description |
---|---|
input_path |
Full path of the saved model directory, session bundle directory, frozen model file or TensorFlow Hub module handle or path. |
output_path |
Path for all output artifacts. |
Options | Description |
---|---|
--input_model_from |
Configure the training library for pre-trained model, use: tensorflow for TensorFlow, keras for Keras, tfjs for TensorFlow.js |
--input_model_format |
The format of input model, use |
--output_layer_names |
The names of the layer which will be visualized in TensorSpace, separated by comma ",". If not set, converter will export all layers. |
Converter Usage Examples
This section introduces the usage of TensorSpace-Converter for different types of pre-trained model from TensorFlow, Keras, TensorFlow.js.
TensorFlow
A pre-trained model built with TensorFlow, may be saved as saved model, frozen model, checkpoint model, combined HDF5 model or separated HDF5 model. Use different TensorSpace-Converter commands for different kinds of TensorFlow model formats. A TensorFlow graph may not have layer
concept, however, a specific tensor
can be corresponding a layer
for TensorSpace visualization, collect all these tensor names and set the name list to output_layer_names
.
For a TensorFlow saved model. Set input_model_format
to be tf_saved_model
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="tensorflow" \
--input_model_format="tf_saved_model" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
input_path \
./PATH/TO/SAVE/DIR
For a TensorFlow frozen model. Set input_model_format
to be tf_frozen_model
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="tensorflow" \
--input_model_format="tf_frozen_model" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
input_path \
./PATH/TO/SAVE/DIR
For a TensorFlow checkpoint model. Set input_model_format
to be tf_checkpoint_model
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="tensorflow" \
--input_model_format="tf_checkpoint_model" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
input_path \
./PATH/TO/SAVE/DIR
For a combined HDF5 model, topology and weights are saved in a combined HDF5 file xxx.h5
. Set input_model_format
to be tf_hdf5_model
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="tensorflow" \
--input_model_format="tf_hdf5_model" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
./PATH/TO/MODEL/xxx.h5 \
./PATH/TO/SAVE/DIR
For a separated HDF5 model, topology and weights are saved in separate files, topology file xxx.json
and weights file xxx.hdf5
. Set input_model_format
to be tf_hdf5_separated_model
. In this case, the model have two input files, merge two file's paths and separate them with comma (.json first, .hdf5 last), and then set the combined path to positional argument input_path
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="tensorflow" \
--input_model_format="tf_hdf5_separated_model" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
./PATH/TO/MODEL/xxx.json,./PATH/TO/MODEL/eee.hdf5 \
./PATH/TO/SAVE/DIR
Checkout this TensorFlow Tutorial for more practical usage of TensorSpace-Converter for TensorFlow models.
Keras
A pre-trained model built with Keras, may have two formats: topology and weights in a combined HDF5 file or topology and weights in separated files. Use different TensorSpace-Converter commands for these two keras model formats.
For a Keras model, topology and weights are saved in a combined HDF5 file xxx.h5
. Set input_model_format
to be topology_weights_combined
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="keras" \
--input_model_format="topology_weights_combined" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
./PATH/TO/MODEL/xxx.h5 \
./PATH/TO/SAVE/DIR
For a Keras model, topology and weights are saved in separated files, a topology file xxx.json
and a weights file xxx.hdf5
. Set input_model_format
to be topology_weights_separated
, in this case, the model have two input files, merge two file's paths and separate them with comma (.json first, .hdf5 last), and then set the combined path to positional argument input_path
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="keras" \
--input_model_format="topology_weights_separated" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
./PATH/TO/MODEL/xxx.json,./PATH/TO/MODEL/eee.hdf5 \
./PATH/TO/SAVE/DIR
Checkout this Keras Tutorial for more practical usage of TensorSpace-Converter for Keras models.
TensorFlow.js
A pre-trained model built with TensorFlow.js, may have a topology file xxx.json
and a weights file xxx.weight.bin
. To converter the model with TensorSpace-Converter, put two files in the same folder and set topology file's path to input_path
. Demo converter script:
$ tensorspacejs_converter \
--input_model_from="tfjs" \
--output_layer_names="layer1Name,layer2Name,layer3Name" \
./PATH/TO/MODEL/xxx.json \
./PATH/TO/SAVE/DIR
Checkout this TensorFlow.js tutorial for more practical usage of TensorSpace-Converter for TensorFlow.js models.
Contributors
Thanks goes to these wonderful people (emoji key):
Chenhua Zhu 💻 🎨 📖 💡 |
syt123450 💻 🎨 📖 💡 |
---|
Contact
If you have any issue or doubt, feel free to contact us by:
- Email: tensorspaceteam@gmail.com
- GitHub Issues: create issue
- Slack: #questions
- Gitter: #Lobby
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tensorspacejs-0.0.12-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 831b93c8fb5cffa9970bfc93d3c29bb3e5e488fba196fe689e5ed2053aa7ebd6 |
|
MD5 | 681a75229f28a8acf5a0d603c7462c3b |
|
BLAKE2b-256 | e99ea7e5be2b4f255facaa4a58589ac3f6343a7401b4e7a66317519877da1327 |