A spark entry point for python
Project description
How to build a Sparpy plugin
On package setup.py a entry point must be configured for Sparpy:
setup(
name='yourpackage',
...
entry_points={
...
'sparpy.cli_plugins': [
'my_command_1=yourpackage.module:command_1',
'my_command_2=yourpackage.module:command_2',
]
}
)
Install
It must be installed on a Spark edge node.
$ pip install sparpy
How to use
Using default Spark submit parameters:
$ sparpy-submit --plugin "mypackage>=0.1" my_command_1 --myparam 1
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
sparpy-0.1.0-py3-none-any.whl
(17.8 kB
view hashes)