Skip to main content

Python jsonl query engine

Project description

Build Status Coverage PyPI Documentation Status

JF

JF, aka “jndex fingers” or more commonly “json filter pipeline”, is a jq-clone written in python. It supports evaluation of python one-liners, making it especially appealing for data scientists who are used to working with python.

Installing

pip install jf

Basic usage

Filter selected fields

$ cat samples.jsonl | jf '{id: x.id, subject: x.fields.subject}'
{"id": "87086895", "subject": "Swedish children stories"}
{"id": "87114792", "subject": "New Finnish storybooks"}
...

Features

supported formats:

  • json (uncompressed, gzip, bz2)

  • jsonl (uncompressed, gzip, bz2)

  • yaml (uncompressed, gzip, bz2)

  • csv and xlsx support if pandas and openpyxl is installed

  • markdown table output support

  • xlsx (excel)

  • parquet

transformations:

  • import and use python modules with –import

  • import additional json for merging and joining using –import name=filename.json

  • initialize transformations with –init

  • access json dict as classes with dot-notation for attributes

  • datetime and timedelta comparison

    • age() for timedelta between datetime and current time

  • first(N), last(N), islice(start, stop, step)

    • head and tail alias for last and first

  • firstnlast(N) (or headntail(N))

  • import your own modules for more complex filtering and transformations

    • Support stateful classes for complex interactions between items

  • sklearn toolbox for machine learning

  • running restful service for the transformation pipeline

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jf-1.1.2.tar.gz (17.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page