skip to navigation
skip to content

Not Logged In

exoline 0.7.2

Command line interface for Exosite platform.

Latest Version: 0.7.9

Exoline
=======

Exoline is command line tool for working with the Exosite [One Platform](http://exosite.com/products/onep).

Installation
------------

Install the latest released version of Exoline from PyPI.

```
    # pip install exoline
```

(pip is a package manager for Python. To get pip, try `sudo easy_install pip` in Mac OS X,  `sudo apt-get install python-setuptools;sudo easy_install pip` in Ubuntu. See below for Windows instructions.)

Here's how to install from source:

```
    $ git clone git://github.com/exosite/exoline
    $ cd exoline
    $ python setup.py install
```

[virtualenvwrapper](http://virtualenvwrapper.readthedocs.org/en/latest/) is a great way to manage Python environments and avoid needing to use sudu for package installs.

Exoline supports Python 2.6 and above. (Tests run against 2.6, 2.7, 3.2 and 3.3.)

Installation - Windows
----------------------

First install the prerequisites:

- [Python](http://www.python.org/getit/)

- [pip-win](https://sites.google.com/site/pydatalog/python/pip-for-windows) (Alternatively, you can install [setuptools](https://pypi.python.org/pypi/setuptools) and [pip](https://pypi.python.org/pypi/pip) individually. pip-win just saves a few steps.)

After pip-win is installed, a GUI window will pop up. To install Exoline, type
`pip install exoline` into the command field.


Usage
-----

```
$ exo -h
Exosite RPC API Command Line Interface
   Provides command line access to the Remote Procedure Call API:
   https://github.com/exosite/api/tree/master/rpc

Usage:
  exo [--help] [options] <command> [<args> ...]

Commands:
  read        Read data from a resource.
  write       Write data at the current time.
  record      Write data at a specified time.
  create      Create a resource from a json description passed on stdin (with -),
              or using command line shorthand (other variants).
  listing     List the RIDs of a client's children.
  info        Get metadata for a resource in json format.
  update      Update a resource from a json description passed on stdin.
  map         Add an alias to a resource.
  unmap       Remove an alias from a resource.
  lookup      Look up a resource's RID based on its alias cik.
  drop        Drop (permanently delete) a resource.
  flush       Remove all time series data from a resource.
  usage       Display usage of One Platform resources over a time period.
  tree        Display a resource's descendants.
  script      Upload a Lua script
  spark       Show distribution of intervals between points.
  copy        Make a copy of a client.
  diff        Show differences between two clients.
  ip          Get IP address of the server.
  data        Read or write with the HTTP Data API.
  portals     Invalidate the Portals cache for a CIK by telling Portals
              a particular procedure was taken on client identified by <cik>.
  share       Generate a code that allows non-owners to access resources
  revoke      Revoke a share code or CIK
  activate    Activate a share code or CIK
  deactivate  Deactivate a share code or expire a CIK
  spec        Determine whether a client matches a specification (beta)

Options:
  --host=<host>        OneP host. Default is $EXO_HOST or m2.exosite.com
  --port=<port>        OneP port. Default is $EXO_PORT or 443
  --httptimeout=<sec>  HTTP timeout [default: 60]
  --https              Enable HTTPS (deprecated, HTTPS is default)
  --http               Disable HTTPS
  --debug              Show debug info (stack traces on exceptions)
  -d --debughttp       Turn on debug level logging in pyonep
  --discreet           Obfuscate RIDs in stdout and stderr
  -c --clearcache     Invalidate Portals cache after running command
  --portals=<server>   Portals server [default: https://portals.exosite.com]
  -h --help            Show this screen
  -v --version         Show version

See 'exo <command> --help' for more information on a specific command.
```

Examples
--------

Show a tree view of a client

```
$ exo tree 5de0cfcf7b5bed2ea7a801234567890123456789
Dev client cik: 5de0cfcf7b5bed2ea7a801234567890123456789 (aliases: (see parent))
  ├─device1 client cik: 970346d3391a2d8c703a01234567890123456789 (aliases: [u'device1'])
  └─device2 client cik: e95052ab56f985e6807d01234567890123456789 (aliases: [u'device2'])
      └─json string dataport rid: 82209d5888a3bd1530d201234567890123456789 (aliases: [u'json'])

```

Upload a Lua script

```

    $ exo script translate_gps.lua e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa
    Updated script RID: 6c130838e14903f7e12d39b5e76c8e3aaaaaaaaa
```

Monitor output of a script

```

    $ exo read e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa translate_gps.lua --follow
    2013-07-09 11:57:45,line 2: Running translate_gps.lua...
    2013-07-09 12:00:17,"line 12: New 4458.755987,N,09317.538945,W
    line 23: Writing 4458.755987_-09317.538945"
    2013-07-09 12:15:41,"line 12: New 4458.755987,N,09317.538945,W
    line 23: Writing 4458.755987_-09317.538945"
```

Write raw data

```

    $ exo write e469e336ff9c8ed9176bc05ed7fa40daaaaaaaa gps-raw --value=4458.755987,N,09317.538945,W
```

Record a bunch of data without timestamps

```

    $ cat myrawgps | exo record e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa gps-raw -
```

Dump data from multiple dataports to CSV

```

    $ time ./exo.py read 2ca4f441538c1f2cc8bfaaaaaaaaaaaaaaaaaaaa gas temperature humidity event --start=5/1/2013 --end=8/1/2013 --chunkhours=24 > alldata.csv

    real    1m58.377s
    user    0m10.981s
    sys     0m0.506s

    $ wc -l alldata.csv
      316705 alldata.csv
```

Make a copy of a device

```

    $ exo copy e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa ed6c3facb6a3ac68c4de9a6996a89594aaaaaaaa
    cik: c81e6ae0fbbd7e9635aa74053b3ab6aaaaaaaaaa
```

Create a new client or resource

```

    $ ../exoline/exo.py create ad02824a8c7cb6b98fdfe0a9014b3c0faaaaaaaa --type=dataport --format=string --name=NewString
    rid: 34eaae237988167d90bfc2ffeb666daaaaaaaaaa
```

Show differences between two clients

```

    $ exo copy 3ae52bdd5280d7cb96a2077b0cd5aaaaaaaaaaaa 5de0cfcf7b5bed2ea7a802ebe0679baaaaaaaaaa
    cik: cc080a86b1c9b53d5371e0fa793faaaaaaaaaaa
    $ exo diff 3ae52bdd5280d7cb96a2077b0cd5aaaaaaaaaaaa cc080a86b1c9b53d5371e0fa793f1daaaaaaaaaa
    $ exo create cc080a86b1c9b53d5371e0fa793f1aaaaaaaaaaa --type=dataport --format=float --name=Humidity
    rid: 6a8974d3d7d1f0ffd28385c90a1bebaaaaaaaaaa
    $ ../exoline/exo.py diff 3ae52bdd5280d7cb96a2077b0cd5dbaaaaaaaaaa cc080a86b1c9b53d5371e0fa793f1daaaaaaaaaa
    {
        "<<RID>>": {
        "aliases": {
            "<<RID>>": [
            "temp"
            ]
        },
        "basic": {
            "subscribers": 0,
            "type": "client"
        },
        "children": {
            "<<RID>>": {
    +         "basic": {
    +           "subscribers": 0,
    +           "type": "dataport"
    +         },
    +         "children": {},
    +         "comments": [],
    +         "description": {
    +           "format": "float",
    +           "meta": "",
    +           "name": "Humidity",
    +           "preprocess": [],
    +           "public": false,
    +           "retention": {
    +             "count": "infinity",
    +             "duration": "infinity"
    +           },
    +           "subscribe": null
    +         },
    +         "shares": [],
    +         "subscribers": [],
    +         "tags": []
    +       },
    +       "Temperature.f2a40b81cb677401dffdc2cfad0f8a266d63590b": {
            "basic": {
                "subscribers": 0,
                "type": "dataport"
            },
            "children": {},
            "comments": [],
            "description": {
                "format": "float",
                "meta": "",
                "name": "Temperature",
                "preprocess": [],
                "public": false,
                "retention": {
                "count": "infinity",
                "duration": "infinity"
                },
                "subscribe": null
            },
            "shares": [],
            "subscribers": [],
            "tags": []
            }
        },
        "comments": [],
        "counts": {
            "client": 0,
    -       "dataport": 1,
    ?                   ^
    +       "dataport": 2,
    ?                   ^
            "datarule": 0,
            "dispatch": 0
        },
        "description": {
            "limits": {
            "client": "inherit",
            "dataport": "inherit",
            "datarule": "inherit",
            "disk": "inherit",
            "dispatch": "inherit",
            "email": "inherit",
            "email_bucket": "inherit",
            "http": "inherit",
            "http_bucket": "inherit",
            "share": "inherit",
            "sms": "inherit",
            "sms_bucket": "inherit",
            "xmpp": "inherit",
            "xmpp_bucket": "inherit"
            },
            "locked": false,
            "meta": "",
            "name": "MyDevice",
            "public": false
        },
        "shares": [],
        "subscribers": [],
        "tagged": [],
        "tags": []
        }
    }
```

See the HTTP requests and responses being made by pyonep:

```

$ exo --debughttp --discreet read <cik> temperature
DEBUG:pyonep.onep:POST /api:v1/rpc/process
Host: m2.exosite.com:80
Headers: {'Content-Type': 'application/json; charset=utf-8'}
Body: {"calls": [{"id": 70, "procedure": "read", "arguments": [{"alias": "temperature"}, {"sort": "desc", "selection": "all", "limit": 1, "endtime": 1376943416, "starttime": 1}]}], "auth": {"cik": "2ca4f441538c1f2cc8bf01234567890123456789"}}
DEBUG:pyonep.onep:HTTP/1.1 200 OK
Headers: [('date', 'Mon, 19 Aug 2013 20:16:53 GMT'), ('content-length', '54'), ('content-type', 'application/json; charset=utf-8'), ('connection', 'keep-alive'), ('server', 'nginx')]
Body: [{"id":70,"status":"ok","result":[[1376819736,24.1]]}]
2013-08-18 04:55:36,24.1
```

Share a dataport with another client.

```
# we want to share client1/dataport1 with client2
$ exo tree wb
Dev client cik: 5de0cf0000000000000000000000000000000000 (aliases: (see parent))
  ├─client1 client cik: 0a35320000000000000000000000000000000000 (aliases: [u'client1'])
  │   └─dataport1 string dataport rid: 4775090000000000000000000000000000000000 (aliases: [u'dataport1'])
  └─client2 client cik: c2d4f30000000000000000000000000000000000 (aliases: [u'client2'])

# generate a share code
$ exo share 0a35320000000000000000000000000000000000 dataport1
e9a52a0000000000000000000000000000000000

# activate the share code
$ exo activate c2d4f30000000000000000000000000000000000 --share=e9a52a0000000000000000000000000000000000

# share appears in tree
$ exo tree wb
Dev client cik: 5de0cf0000000000000000000000000000000000 (aliases: (see parent))
  ├─client1 client cik: 0a35320000000000000000000000000000000000 (aliases: [u'client1'])
  │   └─dataport1 string dataport rid: 4775090000000000000000000000000000000000 (aliases: [u'dataport1'])
  └─client2 client cik: c2d4f30000000000000000000000000000000000 (aliases: [u'client2'])
      └─dataport1 string dataport rid: 4775090000000000000000000000000000000000

# listing shows owned children by default (not shares)
$ exo listing c2d4f30000000000000000000000000000000000
{"dataport": [], "datarule": [], "client": [], "dispatch": []}

# ...unless you filter for activated shares
$ exo listing c2d4f30000000000000000000000000000000000 --filter=activated
{"dataport": ["4775090000000000000000000000000000000000"], "datarule": [], "client": [], "dispatch": []}

# write to the shared dataport from its owner
$ exo write 0a35320000000000000000000000000000000000 dataport1 --value="Share me"

# you can read the dataport from the non-owner
$ exo read c2d4f30000000000000000000000000000000000 4775090000000000000000000000000000000000
2013-12-13 11:34:13-06:00,Share me

# ...but you can't write from a non-owner
$ exo write c2d4f30000000000000000000000000000000000 4775090000000000000000000000000000000000 --value="Non-owner can't write"
One Platform error: restricted

# look up RID for a share code
$ exo lookup c2d4f30000000000000000000000000000000000 --share e9a52a0000000000000000000000000000000000
4775090000000000000000000000000000000000

# the non-owner can deactivate a share code
$ exo deactivate c2d4f30000000000000000000000000000000000 --share=e9a52a0000000000000000000000000000000000

# now the share is gone
$ exo tree wb
Dev client cik: 5de0cf0000000000000000000000000000000000 (aliases: (see parent))
  ├─client1 client cik: 0a35320000000000000000000000000000000000 (aliases: [u'client1'])
  │   └─dataport1 string dataport rid: 4775090000000000000000000000000000000000 (aliases: [u'dataport1'])
  └─client2 client cik: c2d4f30000000000000000000000000000000000 (aliases: [u'client2'])

# the owner may also revoke the share code. This makes it unusable.
$ exo revoke 0a35320000000000000000000000000000000000 --share=e9a52a0000000000000000000000000000000000
ok
```


Environment Variables
---------------------

For convenience, several command line options may be replaced by environment variables.

* EXO\_HOST: host, e.g. m2.exosite.com. This supplies --host to exo and --url for exodata.
* EXO\_PORT: port, e.g. 80. Currently this only applies to exo, not exodata.


CIK Shortcuts
-------------

Store your commonly used CIKs in a file:

```

$ printf "keys:\n" > ~/.exoline
$ printf "    foo: 2ca4f441538c1f2cc8bf01234567890123456789\n" >> ~/.exoline
$ exo read foo temperature
2013-08-18 04:55:36,24.1
```

Help
----

For help, run each command with -h from the command line.

Portals
-------

Portals caches One Platform data, so changes made in Exoline may take up to 15 minutes to show up in Portals. You can work around this by passing `--clearcache` (or `-c`). This option tells Exoline to clear the relevent cached information in Portals.

```
$ exo --clearcache create <cik> --type=client
```

If you're using Portals on a different server, pass `--portals` to specify that server.

```
exo --clearcache --portals=https://myportals.com create <cik> --type=dataport --format=string
```

It's also possible to invalidate the cache directly.

```
$ exo portals clearcache <cik>
```

Usage as a Library
------------------

Exoline can be directly imported and used in Python as a library. There are two patterns
for doing this. First, you can call `exo.run` with whatever arguments you would have
passed on the command line, plus an optional string stdin parameter.

```python

from exoline import exo

result = exo.run(['exo',
                  'script',
                  'scripts/myscript.lua',
                  'ad02824a8c7cb6b98fdfe0a9014b3c0faaaaaaaa'])

print(result.exitcode)    # 0
print(result.stdout)      # Updated script RID: c9c6daf83c44e44985aa724fea683f14eda71fac
print(result.stderr)      # <no output>
```

It's also possible to use Exoline's wrapper for the pyonep library, which covers a lot of
Exoline's functionality.

```python

from exoline import exo

rpc = exo.ExoRPC()

rpc.upload_script(ciks=['ad02824a8c7cb6b98fdfe0a9014b3c0faaaaaaaa'],
                  filename='scripts/myscript.lua')
```


Issues/Feature Requests
-----------------------

If you see an issue with exoline or want to suggest an improvement, please log it [here](https://github.com/exosite/exoline/issues).


Test
----

To run the tests the packages in test/requirements.txt, and then type:

```
    $ cd test
    $ pip install -r requirements.txt

    # run tests in current Python environment
    $ ./test.sh

    # test multiple version of Python (requires virtualenv)
    $ ./test.sh full
```

For more on testing (including running individual tests), see [test/README.md](test/README.md).


What's New?
-----------

For information about what features are in what Exoline versions look [here](HISTORY.md).


TODO
----

- copy command should support taking input from stdin (which would be the ouput of info --recursive)
- investigate copying resources with public: True to destination with public: False (One Platform error: restricted)
- clarify what the first and second parameter to copy need to be. Maybe require explicit --device= and --portal= They could be anything, but 95% of the time the first param will be a device CIK, the second a portal CIK.
- --name parameter to copy command so names don't conflict
- --desconly parameter for info command, to show info as json so it can be piped to create
- add raw command, taking full RPC json from stdin
- Make the info command take multiple rids (or stdin)
- delete serial number when dropping device
- add --howmany option to create command
- tab completion for commands and shortcuts
- add create command shorthand: "exo create float foo" to create a dataport of format float with alias and name both set to foo
- add test for --tz option
- add the option of using requests to authenticate with https (see warning here: http://docs.python.org/2/library/httplib.html)
- add windows executable to build and test
- reimplement copy using OneP's create clone command https://github.com/exosite/api/tree/master/rpc#create-clone
- differentiate dataport and client shares in tree command
- handle unicode characters when reading string dataports


History
=======

0.7.2 (2014-01-14)
------------------

- add --generate option for spec command (beta)
- fix regression in tree command on python 3.2
- remove wsgiref to fix nose for python 3.3

0.7.1 (2014-01-13)
------------------

- handle unicode in csv output
- fix error when piping tree output to file
- remove binary and boolean dataport formats

0.7.0 (2013-12-13)
------------------

- add share, activate, deactivate, and lookup --share commands
- listing command now accepts filtering options and clearer JSON
  output (non-backward compatible)
- updates for incorrect timezone setting

0.6.1 (2013-12-10)
------------------

- add owner lookup command (lookup --owner-of)
- change "Options" to "Command Options" in usage

0.6.0 (2013-12-09)
------------------

- make portals server customizable, e.g., for use with sandbox

0.5.2 (2013-12-09)
------------------

- add --portals options and portals command for cache invalidation,
  so Portals and Exoline can stay in sync

0.5.1 (2013-12-02)
------------------

- support Python 3.x

0.5.0 (2013-11-21)
------------------

- remove --counts option to tree command
- remove storage option to the info command

0.4.3 (2013-11-19)
------------------

- make second parameter to exo.cmd optional
- restore std* so stdout is visible after calling exo.cmd()

0.4.2 (2013-11-13)
------------------

- spec command support for units and json format validation
- example spec file

0.4.1 (2013-11-11)
------------------

- add activate command
- fix spec message for dataport format differences
- add documentation of spec command yaml syntax
- fix data write to handle urlencode characters (e.g. %)

0.4.0 (2013-10-30)
------------------

- use https by default, specify --http for http
- fix issue where read --follow could not be piped to other commands due to stdout buffering
- show commands in a consistent order in 'exo --help'
- show command summaries in 'exo --help'

0.3.6 (2013-10-29)
------------------

- read command defaults to reading all dataports/datarules if no RIDs are specified
- listing command outputs valid JSON

0.3.5 (2013-10-28)
------------------

- reuse connection to speed up API calls

0.3.4 (2013-10-10)
------------------

- default to utc if local timezone can't be determined
- fix timezone bug in read output

0.3.3 (2013-10-4)
-----------------

- decode scripts as utf-8 for spec command

0.3.2 (2013-10-4)
-----------------

- remove plugin dependency on script install location

0.3.1 (2013-10-1)
-----------------

- fix install issue

0.3.0 (2013-9-30)
-----------------

- add plugin framework
- update tree output, incl. sort by client name
- add spec command as a plugin (beta)
- make listing default to all resource types
- timezone support for read command

0.2.6 (2013-9-19)
-----------------

- fixed update command

0.2.5 (2013-8-26)
-----------------

- record reads csv on stdin
- fixed read --sort=asc
- fixed --follow order when multiple values come within the polling window

0.2.4 (2013-8-19)
-----------------

- fixed combination of --debughttp and --discreet

0.2.3 (2013-8-19)
-----------------

- --debughttp shows http requests & responses
- --discreet hides ciks/rids
- documented usage as library

0.2.2 (2013-8-16)
-----------------

- --header option for read command

0.2.1 (2013-8-15)
-----------------

- cik lookup in ~/.exoline
- support ISO8601 dates for read
- copy comments

0.2.0 (2013-8-13)
-----------------

- tree is faster for large portals
- --level option for tree
- copy checks limits when possible (when not set to 'inherit')
- improve json format for info --recursive

0.1.3 (2013-8-9)
----------------

- set up for automated testing in jenkins
- --include and --exclude flags for info
- info and listing commands output json when using --pretty
- --recursive flag for script command
- fixed regression in read --follow

0.1.2 (2013-7-31)
-----------------

- added --port option
- added --chunkhours option to break up large reads

0.1.1 (2013-7-30)
-----------------

- fixed --httptimeout
- show model and serial number from metadata in tree output

0.1.0 (2013-7-24)
-----------------

- read from multiple data sources
- copy command (make a copy of a client)
- diff command (compare clients)
- --recursive option for info

0.0.33 (2013-7-19)
------------------

- support python 2.6

0.0.32 (2013-7-19)
-----------------

- lookup command looks up RID of CIK if no alias is passed
- fixed exception


0.0.31 (2013-7-18)
------------------

- updated to use pyonep 0.7.0
- added usage command

0.0.30 (2013-7-16)
------------------

- fixed regression in tree

0.0.29 (2013-7-16)
------------------

- fixed pyonep reference

0.0.28 (2013-7-16)
------------------

- usage command
- Better test coverage

0.0.27 (2013-7-14)
------------------

- Support uploading script to multiple CIKs
- Added code coverage for tests
- read --intervals shows the distribution of
  delays between points

0.0.26 (2013-7-12)
------------------

- Fixed https port

0.0.25 (2013-7-12)
------------------

- Added --https flag

0.0.24 (2013-7-12)
------------------

- Added raw read format

0.0.23 (2013-7-12)
------------------

- Made <rid> optional for all commands
- Added root node detail output to tree

0.0.22 (2013-7-11)
------------------

- Bumped up version requirement for pyonep

0.0.21 (2013-7-11)
------------------

- Fixed tree output for devices with expired status
- Hide KeyboardInterrupt exception except when --debug set

0.0.20 (2013-7-10)
------------------

- Fixed script command

0.0.19 (2013-7-10)
------------------

- Fixed README.md

0.0.18 (2013-7-10)
------------------

- Help for individual commands, git style
- Fixed regression in 0.0.17 affecting all commands taking <rid>
- record-backdate is now record with --interval


0.0.17 (2013-7-09)
------------------

- Handle keyboard interrupt gracefully for read --follow
- Added example usage in README.md
- Fixed read --follow when dataport has no data

0.0.16 (2013-7-08)
------------------

- Support passing alias for <rid>
- Make read return latest value by default

0.0.15 (2013-7-08)
------------------

- script upload

0.0.14 (2013-7-07)
------------------

- tests for create, read, write

0.0.13 (2013-7-03)
------------------

- record, unmap, lookup commands, better/stronger/faster tree

0.0.12 (2013-6-27)
------------------

- Show OnePlatform library exceptions nicely

0.0.11 (2013-6-27)
------------------

- Changed defaults for tree

0.0.10 (2013-6-27)
------------------

- flush command

0.0.9 (2013-6-26)
-----------------

- Added format to tree output


0.0.8 (2013-6-26)
-----------------

- Added units to tree output, support writing negative numeric values

0.0.7 (2013-6-23)
-----------------

- Time series data write and read commands, with --follow option


0.0.6 (2013-6-23)
-----------------

- RID lookup and bulk drop commands


0.0.5 (2013-6-21)
-----------------

- Install two command line scripts: exo, exodata


0.0.4 (2013-6-18)
-----------------

- Complete Exosite Data API
- Subset of Exosite RPC API
 
File Type Py Version Uploaded on Size
exoline-0.7.2.tar.gz (md5) Source 2014-01-15 57KB
  • Downloads (All Versions):
  • 644 downloads in the last day
  • 2577 downloads in the last week
  • 7608 downloads in the last month