Spark Connect adapter for Ocean Spark
Project description
Wrapper to create Spark Connect session for Spark Applications in Ocean
from ocean_spark_connect.ocean_spark_session import OceanSparkSession
spark = OceanSparkSession.Builder().cluster_id("osc-cluster").appid("appid").profile("default").getOrCreate()
spark.sql("select random()").show()
spark.stop()
Options for OceanSparkSession.Builder with and without default values
In addition to the existing SparkSession.Builder option, the following options are available:
- token
- profile
- appid
- account_id
- cluster_id
- host = "api.spotinst.io"
- port = "15002"
- bind_address = "0.0.0.0"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for ocean_spark_connect-0.3.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd0cd4e62723c15f55bdeb637a3b26967641a2aabd02a276f10b9fb3c6af5bde |
|
MD5 | 703c00b5b9b8036fed067405b64bf863 |
|
BLAKE2b-256 | e8cffee964f9881962724a4b5b61ccf69636efab5ce83ce5f07837f03f4aa43f |
Close
Hashes for ocean_spark_connect-0.3.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fedd674f59017d62ba918ecc666866f5157da9c88ac2d4726adf27ff90cdc300 |
|
MD5 | 7fc5bcfec6065227f467f515481a94b8 |
|
BLAKE2b-256 | 5f25493a888bd41b1eb8d44e7808d95a577e83090ef5266e5493ad3cf674066a |