Spark Connect adapter for Ocean Spark
Project description
Wrapper to create Spark Connect session for Spark Applications in Ocean
from ocean_spark_connect.ocean_spark_session import OceanSparkSession
spark = OceanSparkSession.Builder().cluster_id("osc-cluster").appid("appid").profile("default").getOrCreate()
spark.sql("select random()").show()
spark.stop()
Options for OceanSparkSession.Builder with and without default values
In addition to the existing SparkSession.Builder option, the following options are available:
- token
- profile
- appid
- account_id
- cluster_id
- host = "api.spotinst.io"
- port = "15002"
- bind_address = "0.0.0.0"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for ocean_spark_connect-0.3.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9d560b0103be8a1f511319fc17e18cf7f521f475a6ba0509dfbd88c131de6467 |
|
MD5 | 38cd647ffb02fa5db9d14bf4352aad5a |
|
BLAKE2b-256 | f8f43ff835dcceaa336c0ca5ed83111c3883c554766ab58b5def35a693f3b5f4 |
Close
Hashes for ocean_spark_connect-0.3.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c136e5a88f1e1f0a3a7a181c8ddf7c86b4bb880032d23606f9aa95253a26dbcf |
|
MD5 | 0122333f26aa2873886422b2565dec47 |
|
BLAKE2b-256 | ceb10050c99cc0a1d6ecf84bc2a890e0995c1f1bd7681e189575bc87a47b731b |