csv.toolkit 0.1.0

Built-In Simple CSV Validator

Included with this package is a simple CSV file validation mechanism to use to validate simple CSV structures where fields may contain any values or may be empty. This is also a good example of how to implement a CSV validation schema as an internal tool available to the CLI.

New Implementations

Subclass the BaseFileValidator class to create a new CSV validation tool. The required fields validators, delimeter, default_validator, check_duplicate_headers, and logger attributes must be defined. Creating a new logger for each CSV validating tool is recommended, but not necessary.

An example bare-bones implementation would be:

>>> class YourFirstValidatorLogger(Logger):
>>>     pass
>>>
>>> class YourFirstValidator(BaseFileValidator):
>>>     validators = {
>>>         "Field1": [],
>>>         "Field2": [],
>>>         "Field3": [],
>>>     }
>>>     delimiter = ","
>>>     default_validator = AnyVal
>>>     check_duplicate_headers = True
>>>     logger = YourFirstValidatorLogger
>>>
>>>     def validate(self):
>>>         ... validation mechanism here...
>>>
>>> validator = YourFirstValidator(LocalFileLoader('/path/to/example.csv'))
>>> print validator.validate()
True
>>> result = validator()
>>> print result.validation
True
>>> print result.log
... validation log text...

Obviously, you may call the validate property directly without a logger, but you may also call the validator instance, which returns a named tuple Result with validation and log attributes.

Please note, att this time the BaseFileValidator only supports loggers of the built-in type. Pull requests and contributions to change this are more than welcome.

Validator Attribute Definition

The validators attribute must define the validation schema for your type of CSV. It must be a dictionary with string keys defining the available columns and list values specifying the validator (with any initialization parameters the validator requires).

An example validation schema would look like:

>>> validators = {
>>>     "Foo": [
>>>         UniqueVal(),
>>>     ],
>>>     "Bar": [
>>>         RegexVal(r'^baz$'),
>>>     ],
>>>     "hello world": [
>>>         IntVal(empty_ok=True),
>>>     ],
>>> }

This schema corresponds to a CSV with headers Foo, Bar, and hello world. The Foo column must contain unique values, the Bar column must contain fields matching the regular expression ^baz$, and the hello world column must contain integer values, but allows for empty fields as well.

Built-In Validators

This package comes with built-in validators. For example:

• IntVal: Integer values (allows empty values)

• FloatVal: Float values (allows empty values)

• BoolVal: Boolean values (allows empty values)

• EnumVal: Enumerated values:

  EnumVal(['a', 'list', 'of', 'enumerations',])

• UniqueVal: Unique values only

• RegexVal: Fields must match supplied regex value (or no fields are matched)

• EmptyVal: All fields must be empty

• AnyVal: Any allowed values, but not empty

NOTE: Inclusion of a JSON validator has not been made at this time, but pull requests and contributions of an implementation are welcome.

Arguments

The arguments must be a list of tuples with each touple containing the parameters usually passed to the argparse.add_argument() function. For example, a typical implementation looks like:

>>> self.parser.add_argument(
>>>     "filename",
>>>     type=argparse.FileType('r'),
>>>     help="A file."
>>> )

which, for a tool implementation, should be converted too:

>>> arguments = [
>>>     (
>>>         'filename',
>>>         {'type': argparse.FileType('r')},
>>>         {'help': 'A file.'},
>>>     ),
>>> ]

Please note that the scripts.py file (the entry point for the CLI) will parse known arguments from the command line, and pass the rest to your tooling implementation.