matchtpl 0.1.0.dev3

Keywords & Elements

Here are typical keywords:

• select: select target element(s) from document.

  • selector_string (string): CSS3 Selector to choose target.

• get: get internal text | html of target DOM element.

  • type (string): “text” | “html”.

• eval: locally evaluate via python syntax. (Often used to call jquery-like API.)

  • script_text (string): script using python syntax.

• default: default value if none.

  • value (string): default value.

• as: output format in human-readable way.

  • type (string): str | json | yaml. If not provided, will return python builtin data strucutures.

(Keywords are not limited as above.)

And extensible elements are:

• Strucuture element: <s></s>, <array></array>, <map></map> (see: above).

• Root element: <root></root>. Act as serilization class, and provide multiple formats to output result.

• Customized element: <action></action>, where action here can be other non-conflictive tag. action is a customized action provide by user when calling parser.parse(…, {‘action’: some_function}).

Python Code

In python, typical usage often looks like this:

#!/usr/bin/env python

from matchtpl import MTemplateEnv, MTemplate, MTemplateParser

if __name__ == '__main__':
    # initialize environment
    env = MTemplateEnv(template = 'tpl_amazon.xml')

    # build template
    tpl = MTemplate()
    tpl.build(env)

    # initialize parser and parse
    parser = MTemplateParser(tpl)
    results = parser.parse('amazon.html')

Configurable Template

The pre-defined template is written in xml, which acts as a config file to indicates the meta information of the target (usually another html/xml file or stream). Then, parser will use the template to guide its processing, and output the result:

<!-- serilize result as json. (other format is also supported) -->
<root as="json">
    <!-- the collection of entries are started with 'result_*' in their IDs,
         and each entry is a map -->
    <array select="div[id^='result_']" >
        <map>
            <!-- title: get internal text as result -->
            <s key="title" select="h3 span.lrg" get="text" />
            <s key="info" select="h3 span.med" get="text" />
            <!-- image: get src link in jquery-like way -->
            <s key="image" select="div.image img.productImage" eval="attr('src')" />
            <!-- price: pseudo-class of CSSSelector is used -->
            <s key="price" select="li.newp span:eq(0)" get="text" />
            <!-- review: default value is enabled -->
            <s key="review" select="span.asinReviewsSummary a" eval="attr('alt')" default='0' />
        </map>
    </array>
</root>

After execution, the output is organized as json:

[
    [
        {
            "image": "http://ec4.images-amazon.com/images/I/516Vhic-I9L._AA160_.jpg",
            "info": "刘亚莉 广东省出版集团，广东经济出版社  (2011-05) - Kindle电子书",
            "price": "￥1.99",
            "review": "平均4.4 星",
            "title": "总经理财务一本通"
        },
        // up to 25 results: map
    ]
]

(At present, json, yaml, plaintext or python builtin structures are allowed. More format will be supported later.)