cmdlet 0.3.0

pipe.func(generator_function)

The most basic wrapper. In Python, generator function is a function with yield statement in it. The generator_function defined here is a Python generator function with at least one argument. The first argument is a generator object passed by previous Pipe object. generator_function can take it as input or just leave it. It looks like:

# Generator function which use prev as input.
@pipe.func
def my_generator(prev):
    for data in prev:
        # ... Put some code to process data ...
        yield new_data

# Generator function which ignore input.
@pipe.func
def my_generator_ignore_prev(prev):
    while True:
        # ... Generate data and break loop in some conditions. ...
        yield data

For example:

@pipe.func
def randint_generator(prev, num):
    for i in range(num):
        yield random.randint(0, 1000)

@pipe.func
def power(prev, th):
    for n in prev:
        yield n ** th

cmds = randint_generator(10) | power
ans = result(cmds)
# Equals to:
# ans = []
# for i in range(10):
#     ans.append(random.randint(0, 1000)

pipe.map(function)

Wrap function to a mapper. The input is a normal function with at least one argument for data input. The returned value will be passed to next Pipe object. It looks like:

@pipe.map
def my_mapper(data):
    # ... Put some code to process data ...
    return new_data

For example:

@pipe.func
def randint_generator(prev, num):
    for i in range(num):
        yield random.randint(0, 1000)

@pipe.map
def power(n, th):
    return n ** th

cmds = randint_generator(10) | power
ans = result(cmds)
# Equals to:
# ans = []
# for i in range(10):
#     ans.append(random.randint(0, 1000)

The power pipe can also be written in this way:

power = pipe.map(lambda n, th: n ** th)

Anything returned by mapper will be sent to next Pipe object. If mapper return None, next Pipe object will receive None. That is, you can’t use mapper to filter data out. That’s why we have pipe.filter.

pipe.filter(function)

Wrap function to a filter. Filter is a function with at least one argument as data input. Filter should return Boolean value, True or False. If True, data from previous Pipe object is allowed to pass through. If False, data is dropped. It looks like:

@pipe.filter
def my_filter(data):
    # Handle data and check conditions.
    if you_should_not_pass:
        return False
    else:
        return True

For example:

@pipe.filter
def less_than(data, thrd):
    return data < thrd

cmds = range(10) | less_than(3)
ans = result(cmds)
# Equals to:
# ans = []
# thrd = 3
# for n in range(10):
#     if n < thrd:
#          ans.append()

You can write filter pipe in this way:

less_than = pipe.filter(lambda data, thrd: data < thrd)

pipe.reduce(function)

Wrap function as a reducer. A reducer is a function which has at least two arguments. The first one is used as accumulated result, the second one is the data to be processed. A optional keyword argument init can be used to specify initial value to accumulated result. It looks like:

@pipe.reduce
def my_reducer(accum_result, data):
    # Calculate new accum_result according to data.
    return accum_result

For example:

@pipe.reduce
def count_mod(accum_result, data, mod_by):
    if (data % mod_by) == 0:
        return accum_result
    else:
        return accum_result + 1

cmds = range(1000) | count_mod(10, init=0)

pipe.stopper(function)

Wrap function as a stopper. Stopper is used to stop the pipe execution. It returns true to stop the pipe execution. Return false to pass data to next. It looks like:

@pipe.stopper
def my_stopper(data):
    if check_stop_criteria(data):
        return True
    return False

The usage of wrapper

Here is a example to show how to use function wrapper.

from random import randint
from cmdlet.cmds import *

@pipe.func
def random_number(prev, amount):
    for i in range(amount):
        yield randint(0, 100000)

@pipe.filter
def in_range(data, lower_bound, upper_bound):
    return data >= lower_bound and data <= upper_bound

@pipe.reduce
def count(accum_result, data):
    return accum_result + 1

@pipe.map
def format_output(data, format):
    return format % data

# Generate 1000 random number and count how many of them between 100 and 500.
# Then, format the result to 'ans=%d'.
cmds = random_number(1000) | in_range(100, 500) | count(init=0) | format_output('ans=%d')

print cmds.run()
# >>> ans=40

If wrapped code is just a expression, following code shows another way to make them:

in_range = pipe.filter(lambda data: data >= lower_bound and data <= upper_bound)
count = pipe.reduce(lambda accum_result, data: accum_result + 1)
format_output = pipe.reduce(lambda data, format: format % data)

NOTE: > As you might already noticed, the number of argument using in piped commands > is different from the definition of wrapped function. You should know your > function is wrapped to a Pipe object. The function is not invoked when > cascading pipes. It is called when using run(), result() or iteration. The > arguments will be stored in Pipe object and append to the argument list of > wrapped function when it is invoked.

Auto-type conversion

If the operand of | operator is not a Pipe object, cmdlet will call proper creator to convert and wrap it to a Pipe object. The data type of operand must be registered in cmdlet. Otherwise, exception UnregisteredPipeType will be raised.

cmdlet.cmds has registered some basic types by default. You can use them directly.

Pipe commnds for iterable object.

Pipe commands for file

Pipe commands for shell

Pipe commands for strings