Skip to main content

a Python Polars interface to kdb+/q

Project description

kola

a Python Polars Interface to kdb+/q

Basic Data Type Map

Deserialization

Atom

k type n size python type note
boolean 1 1 bool
guid 2 16 str
byte 4 1 int
short 5 2 int
int 6 4 int
long 7 8 int
real 8 4 float
float 9 8 float
char 10 1 str
string 10 1 str
symbol 11 * str
timestamp 12 8 datetime
month 13 4 -
date 14 4 date 0001.01.01 - 9999.12.31
datetime 15 8 datetime
timespan 16 8 timedelta
minute 17 4 time 00:00 - 23:59
second 18 4 time 00:00:00 - 23:59:59
time 19 4 time 00:00:00.000 - 23:59:59.999

Composite Data Type

k type n size python type
boolean list 1 1 pl.Boolean
guid list 2 16 pl.List(pl.Binary(16))
byte list 4 1 pl.Uint8
short list 5 2 pl.Int16
int list 6 4 pl.Int32
long list 7 8 pl.Int64
real list 8 4 pl.Float32
float list 9 8 pl.Float64
char list 10 1 pl.Utf8
string list 10 1 pl.Utf8
symbol list 11 * pl.Categorical
timestamp list 12 8 pl.Datetime
month list 13 4 -
date list 14 4 pl.Date
datetime list 15 8 pl.Datetime
timespan list 16 8 pl.Duration
minute list 17 4 pl.Time
second list 18 4 pl.Time
time list 19 4 pl.Time
table 98 * pl.DataFrame
dictionary 99 * -
keyed table 99 * pl.DataFrame

performance is impacted by converting guid to string, deserialize the uuid to 16 fixed binary list, use .hex() to convert binary to string if required

real/float 0n is mapped to Polars null not NaN

short/int/long 0Nh/i/j, 0Wh/i/j and -0Wh/i/j are mapped to null

df.with_columns([
    (pl.col("uuid").apply(lambda u: u.hex()))
    ])

Serialization

Basic Data Type

python type k type note
bool boolean
int long
float float
str symbol
bytes string
datetime timestamp
date date 0001.01.01 - 9999.12.31
datetime datetime
timedelta timespan
time time 00:00:00.000 - 23:59:59.999

Dictionary, Series and DataFrame

python type k type
dict dict
pl.Boolean boolean
pl.List(pl.Binary(16)) guid
pl.Uint8 byte
pl.Int16 short
pl.Int32 int
pl.Int64 long
pl.Float32 real
pl.Float64 float
pl.Utf8 char
pl.Categorical symbol
pl.Datetime timestamp
pl.Date date
pl.Datetime datetime
pl.Duration timespan
pl.Time time
pl.DataFrame table

Limited Support for dictionary as arguments, python string as keys and Python Basic Data Types and pl.Series as values.

Quick Start

Create a Connection

import polars as pl
import kola
q = kola.Q('localhost', 1800)

Connect(Optional)

Automatically connect when querying q process

q.connect()

Disconnect

Automatically disconnect if any IO error

q.disconnect()

String Query

q.sync("select from trade where date=last date")

Functional Query

For functional query, kola supports Python Basic Data Type, pl.Series, pl.DataFrame and Python Dictionary with string keys and Python Basic Data Type and pl.Series values.

from datetime import date, time

q.sync(
    ".gw.query",
    "table",
    {
        "date": date(2023, 11, 21),
        "syms": pl.Series("", ["sym0", "sym1"], pl.Categorical),
        # 09:00
        "startTime": time(9),
        # 11:30
        "endTime": time(11, 30),
    },
)

Send DataFrame

# pl_df is a Polars DataFrame
q.sync("upsert", "table", pl_df)
# pd_df is a Pandas DataFrame, use pl.DateFrame to cast Pandas DataFrame
q.sync("upsert", "table", pl.DataFrame(pd_df))

Async Query

# pl_df is a Polars DataFrame
q.asyn("upsert", "table", pl_df)

Polars Documentations

Refer to

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

kola-0.7.1-cp312-none-win_amd64.whl (3.7 MB view hashes)

Uploaded CPython 3.12 Windows x86-64

kola-0.7.1-cp312-cp312-manylinux_2_34_x86_64.whl (7.6 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.34+ x86-64

kola-0.7.1-cp311-none-win_amd64.whl (3.7 MB view hashes)

Uploaded CPython 3.11 Windows x86-64

kola-0.7.1-cp311-cp311-manylinux_2_34_x86_64.whl (7.6 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.34+ x86-64

kola-0.7.1-cp310-none-win_amd64.whl (3.7 MB view hashes)

Uploaded CPython 3.10 Windows x86-64

kola-0.7.1-cp310-cp310-manylinux_2_31_x86_64.whl (7.1 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.31+ x86-64

kola-0.7.1-cp39-none-win_amd64.whl (3.7 MB view hashes)

Uploaded CPython 3.9 Windows x86-64

kola-0.7.1-cp39-cp39-manylinux_2_31_x86_64.whl (7.1 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.31+ x86-64

kola-0.7.1-cp38-none-win_amd64.whl (3.7 MB view hashes)

Uploaded CPython 3.8 Windows x86-64

kola-0.7.1-cp38-cp38-manylinux_2_31_x86_64.whl (7.1 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.31+ x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page