essential-generators 0.4

Install

Using pip:

pip install essential_generators

Document Templates

Now let’s say we want to generate hundreds of thousands of these records. For making documents, we first need to define the template:

gen = DocumentGenerator()

template = {
     'id': 'guid',
     'status': ['online', 'offline', 'dnd', 'anonymous'],
     'age': 'small_int',
     'homepage': 'url',
     'name': 'name',
     'headline': 'sentence',
     'about': 'paragraph'
 }

 gen.set_template(template)
 documents = gen.gen_docs(1000)

The template gives the structure and type for each field in the document. Note that status has a list and not a single type; when a list is provided as the type, one of the items in the list will be randomly selected for each generated documents using random.choice(list)

Custom Fields

Now we want to implement a new feature where users can rate each other between 1-5 stars and we want to keep track of the average rating (a float between 1 and 5). We can do this by passing in a function as the type, like so:

def gen_rating():
    return random.uniform(1, 5)

template = {
    'id': 'guid',
    'status': ['online', 'offline', 'dnd', 'anonymous'],
    'age': 'small_int',
    'homepage': 'url',
    'name': 'name',
    'headline': 'sentence',
    'about': 'paragraph',
    'rating': gen_rating,
}

In this case, when each document is created, gen_rating is called and the returned value is added to the document.

Nested Documents

Now that users are rating each other, of course they’ll want to get in contact with each other. The schema gets extended to include a nested contact object. Just like any custom field, we can generate nested documents using generator functions as the type:

def gen_contact():
    return {
        'email': gen.email(),
        'phone': gen.phone()
    }

template = {
    'id': 'guid',
    'status': ['online', 'offline', 'dnd', 'anonymous'],
    'age': 'small_int',
    'homepage': 'url',
    'name': 'name',
    'headline': 'sentence',
    'about': 'paragraph',
    'contact': gen_contact
}

Word & Sentence Caching

Creating word and sentence cache’s serves two purposes: it resticts the possible space of generated elements to a discreet size (for instance, the average American’s vocabulary is between 5k and 10k words) and it greatly speeds subsequent document generation. Use them like this:

gen.init_word_cache(5000)
gen.init_word_cache(5000)

In the first line, 5000 words are generated. In the second line, 5000 sentences made up of 5 to 15 words from the word cache will be generated. subsequent call to gen.word() and gen.sentence() will be selected from the caches. If you want to generate a new to a word or sentence not in the cache, call gen.gen_word() and gen.gen_sentence() respectively. If you want finer grain control, gen.word_cache and gen.sentence_cache are arrays of strings that can be directly manipulated.

Finer Grained Control

Now we want the user to be able to set a link to their current favorite post. You could do this by adding a field called ‘favpost’ and settings its type to ‘slug’ (like the ones used to url-encode blog post ids while keeping them human readable). The problem is, this would likely generate a unique favpost for each document, but in the real world there would be a finite set of posts.

You can control this behaviour by using python lists as the type. In this example, we use a list comprehension to generate a list of 1000 slugs that will be randomly seletected from when the documents are generated:

template = {
    'id': 'guid',
    'status': ['online', 'offline', 'dnd', 'anonymous'],
    'age': 'small_int',
    'homepage': 'url',
    'name': 'name',
    'headline': 'sentence',
    'about': 'paragraph',
    'favpost': [gen.slug() for n in range(1000)]
}

So, what did we end up with?

This is one result:

{
    'name': 'Ster Ev',
    'age': 87,
    'status': 'anonymous',
    'favpost': 'anre-regtehcie57',
    'headline': 'ilrendna anr mo inttuonth anuir',
    'homepage': 'http://enar692.com/ten/erst/eresnn.heotiatin-neworwnti54-atnd',
    'id': 'ced10e96-b02c-4292-9be8-22dd8772c64e',
    'rating': 1.9779484996288086,
    'contact': {
                   'email': 'osat@ind.ru',
                   'phone': '695-323-8276'
               }
    'about': 'Yeormftd or an on authar hei po heheat este ler hearain hethe
    hetiarte ti oren. Oncs yemf edhe inhe th bain thfin nanfee st. Thheannd
    chenes hein thin. Edrdth ttind te uearedor heoea hehaeren seonstth tith
    vemoal an rein gel don in. Anao is fecttrr.',

}

Documents are basic Python dictionaries, so you can use the directly in your program or convert them to json or any other serialization format for testing anywhere.