Skip to main content

An end-to-end machine learning and data mining framework on Hadoop.

Project description

Build Status

pyshifu

pyshifu is a Python module for machine learning build on top of java version shifu, which provided large scalability of build high accuracy models with TB level data set in hours.

More details about shifu, visit shifu’s wiki page: https://github.com/shifuml/shifu/wiki
Shifu Pipeline

pyshifu provided the basic operations in the pipeline above, such as new, init, stats…

Installation

Dependencies

shifu requires:
* Python(>=2.7 or >=3.3)
* Java(>=7.0)
Shifu Optional:
* Hadoop
Platform requirement:
* Mac
* Linux
* Windows(>=10586.1007) As pyshifu currently depended on bash script to set environment, so windows without shell support could not work correctly. In the future, we will remove all shell script.

User installation

The easiest way to install pyshifu is using pip:

pip install pyshifu

or use conda:

conda install pyshifu

Development

We welcome new contributors of all experience levels. The shifu community goals are to be helpful, welcoming, and effective. The Contribute Guide has detailed information about contributing code, documentation, tests, and more. We’ve included some basic information in this README.

Source code

You can check the latest sources with the command:

git clone https://github.com/ShifuML/pyshifu.git

Setting up a development environment

Quick tutorial on how to go about setting up your environment to contribute to pyshifu.

Testing

This project intend to make the python code 100% test coverage. You can test by tox.

pip install -r requirements-build.txt
# run the python tests
tox -r

Submitting a Pull Request

Before opening a Pull Request, have a look at the full Contributing page to make sure your code complies with our Develop guide. ## Project History This project is started for help user using shifu in python environment.

Help and Support

Documentation

Communication

You can leave your message here, Message Board.

Citation

If you use scikit-learn in a scientific publication, we would appreciate Citations.

Thanks

1, Thanks kyhau for python-repo-template project to create an empty python module. https://github.com/kyhau/python-repo-template

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

pyshifu-0.0.2-py2.py3-none-any.whl (14.7 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page