Spark Connect adapter for Ocean Spark
Project description
Wrapper to create Spark Connect session for Spark Applications in Ocean
from ocean_spark_connect.ocean_spark_session import OceanSparkSession
spark = OceanSparkSession.Builder().cluster_id("osc-cluster").appid("appid").profile("default").getOrCreate()
spark.sql("select random()").show()
spark.stop()
Options for OceanSparkSession.Builder with and without default values
In addition to the existing SparkSession.Builder option, the following options are available:
- token
- profile
- appid
- account_id
- cluster_id
- host = "api.spotinst.io"
- port = "15002"
- bind_address = "0.0.0.0"
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for ocean_spark_connect-0.3.5.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 97ab5302a0bd0c91f2baed1e62ac65ab216ad1e70ac984ec17b452866ffda6b3 |
|
| MD5 | b25117b6a1d6f3bfc9dbd5342d94bcb4 |
|
| BLAKE2b-256 | 34bef7c3fe16c9243cb573dc8faf880a24aedcf500ae2c721748798d2fc7cd26 |
Close
Hashes for ocean_spark_connect-0.3.5-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 938d7c9f19fd81bc50957a0cda90f23d4d384de0501d2c1ad56cac4869881bb6 |
|
| MD5 | 8a189741849c9c82e7cbaa85e0e9afeb |
|
| BLAKE2b-256 | ae6a3c16ec1e6fac3f971fa777ccfa76f13220ea9a5a3b918e15cabbc3ab0ff7 |
