skip to navigation
skip to content

Not Logged In

azkaban 0.6.8

AzkabanCLI

Latest Version: 0.9.1

AzkabanCLI

A lightweight Azkaban client providing:

  • A command line interface to run workflows, upload projects, etc.
  • A convenient and extensible way for building projects.

Sample

Below is a simple configuration file for a project containing a workflow with four pig scripts.

from azkaban import PigJob, Project
from getpass import getuser

PROJECT = Project('sample', root=__file__)

# default options for all jobs
DEFAULTS = {
  'user.to.proxy': getuser(),
  'param': {
    'input_root': 'sample_dir/',
    'n_reducers': 20,
  },
  'jvm.args.mapred': {
    'max.split.size': 2684354560,
    'min.split.size': 2684354560,
  },
}

# list of pig job options
OPTIONS = [
  {'pig.script': 'first.pig'},
  {'pig.script': 'second.pig', 'dependencies': 'first.pig'},
  {'pig.script': 'third.pig', 'param': {'foo': 48}},
  {'pig.script': 'fourth.pig', 'dependencies': 'second.pig,third.pig'},
]

for option in OPTIONS:
  PROJECT.add_job(option['pig.script'], PigJob(DEFAULTS, option))

The examples directory contains another sample project that uses Azkaban properties to build a project with two configurations: production and test, without any job duplication.

Documentation

The full documentation can be found here.

Installation

Using pip:

$ pip install azkaban
 
File Type Py Version Uploaded on Size
azkaban-0.6.8.tar.gz (md5) Source 2014-05-09 21KB
  • Downloads (All Versions):
  • 457 downloads in the last day
  • 2534 downloads in the last week
  • 10113 downloads in the last month