django-bend 0.0.5

# Django Bend

[](https://travis-ci.org/excellalabs/django-bend)
[](https://coveralls.io/github/excellalabs/django-bend?branch=master)

A tool to help translate a legacy mysqldump to fixtures for a similar-but-different Django model.

The use case for this tool is when you want to port data from a legacy system, but your Django models were designed in a way that the data doesn't quite align correctly. Table names and column names could be different, or columns could be completely missing.

See the example below for a Before and After scenario.

### How to install

Note that django-bend requires Django >=1.8.0. This requirement is not enforced via `pip install` to prevent the scenario of unknowingly upgrading your Django version.

Install django-bend

```shell
pip install django-bend
```

Add bend to your Django INSTALLED_APPS

```python
INSTALLED_APPS = (
...
'django_bend',
)
```

### Create your schema

The example for this readme expects a mysqldump file with the following table:

Filename: sample.sql

```sql
CREATE TABLE `ftbl_individuals` (
`ID` int(10) NOT NULL,
`FirstName` varchar(25) DEFAULT NULL,
`LastName` varchar(30) DEFAULT NULL,
`Middle` varchar(30) DEFAULT NULL,
`Gender` varchar(1) DEFAULT NULL,
`DOB` datetime DEFAULT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `Constraint` (`FirstName`,`LastName`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO `ftbl_individuals` (`ID`, `FirstName`, `LastName`, `Middle`, `Gender`, `DOB`) VALUES ('1','Frank','Thomas','L','M','1972-11-28 00:00:00'),('2','Carlos','Baerga','3','M','1966-05-08 00:00:00'),('3','Dolly','Parton','#','F','2000-02-24 00:00:00');
```

Suppose the Django model looks like this:

Filename: myapp/models.py

```python
class Person(models.Model):
first_name = models.TextField()
date_of_birth = models.DateTimeField(blank=True, null=True)
```

Create a mapping json file with a naming convention similar to Django Migrations:

Filename: myapp/bend/0001_person.json

```json
[
{
"from_table": "ftbl_individuals",
"to_table": "foo.person",
"columns": [
{
"from": "FirstName",
"to": "first_name"
},
{
"from": "DOB",
"to": "date_of_birth"
}
]
}
]
```

Your individual columns can have data mapping values for cleaning up bad data:

```json
[
{
"from_table": "ftbl_individuals",
"to_table": "foo.person",
"columns": [
{
"from": "FirstName",
"to": "first_name"
},
{
"from": "DOB",
"to": "date_of_birth"
},
{
"from": "Active",
"to": "is_active",
"mapping":
[
{"to": 1, "from": 2},
{"to": 0, "from": 1}
]
}
]
}
]
```

You can define more tables in the same file, or create new files for new definitions (e.g. `myapp/bend/0002_company.json`). Bend will find all the files that match the `XXXX_name.json` format where X is a digit.

### Create the fixture file

Run `./manage.py bend_data myapp/bend/ sample.sql --indent=2` to dump the fixture output to the terminal.

```json
[
{
"fields": {
"first_name": "Frank",
"date_of_birth": "1972-11-28 00:00:00"
},
"model": "foo.person",
"pk": 1
},
{
"fields": {
"first_name": "Carlos",
"date_of_birth": "1966-05-08 00:00:00"
},
"model": "foo.person",
"pk": 2
},
{
"fields": {
"first_name": "Dolly",
"date_of_birth": "2000-02-24 00:00:00"
},
"model": "foo.person",
"pk": 3
}
]
```

If it looks like what you expect, pipe it to a fixture file:

```shell
./manage.py bend_data myapp/bend/ sample.sql > myapp/fixtures/bend.json
```

The fixture can then be loaded easily:

```shell
./manage.py loaddata myapp/fixtures/bend.json
```

## Development and Testing

To develop on django-bend:

```
git clone https://github.com/excellalabs/django-bend
virtualenv venv
. venv/bin/activate
pip install -r requirements.txt
```

To run tests:

```
pip install -r requirements-test.txt
py.test
```