Spark Connect adapter for Ocean Spark
Project description
Wrapper to create Spark Connect session for Spark Applications in Ocean
from ocean_spark_connect.ocean_spark_session import OceanSparkSession
spark = OceanSparkSession.Builder().cluster_id("osc-cluster").appid("appid").profile("default").getOrCreate()
spark.sql("select random()").show()
spark.stop()
Options for OceanSparkSession.Builder with and without default values
In addition to the existing SparkSession.Builder option, the following options are available:
- token
- profile
- appid
- account_id
- cluster_id
- host = "api.spotinst.io"
- port = "15002"
- bind_address = "0.0.0.0"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for ocean_spark_connect-0.3.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d7563802a86b48c1d08d874ce19c2b80a2c9735efa3d1b5c959019e0cd95c224 |
|
MD5 | daeff9dbfda5b6f888d7cdc234ea2b43 |
|
BLAKE2b-256 | 06b1b808a5c69f214a71c3935bf4ad195de52ec02687c6b92ca1970dc982061b |
Close
Hashes for ocean_spark_connect-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | caca84af280ea2a0797b1b3a810ae1fcb73318f30e27f84ac120274742ed466d |
|
MD5 | 0a8a3ed21550e492e79d7a222d18f2cf |
|
BLAKE2b-256 | 5dd39d9a4a48d2a7681e50a33a32967d5dc0207c0b276f8385ead466f4a745f2 |