Skip to main content

very powerful and optional parser framework for python

Project description

Build Status GPLv3.0 License PyPI version

EBNFParser

Parse Many, Any, Every

HomePage

Multi-Language-Versions


An Introduce to EBNFParser

EBNFParser seems to be a parser framework for EBNF, however, it’s just what I want to do at the beginning, and so far this framework is much more powerful than what I used to expect it to be.

As a result, I prefer to call it EEBNF(which means Extented(Extented Backus-Naur Form)).

What’s more, tokenizer is automatically generated from EEBNF, as well as the parsers for your DSL.

Here are some grammars for EEBNF, and they’re quite easy to be learnt.

I’m going to write the parsers for Lisp quickly to tell you how to use EBNFParser.

The reason why I choose Lisp is that its EEBNF codes is very very short.

Atom  := R'[^\(\)\s]+' # use Regex
# define a literal parser. `Atom::= R'[^\(\)\s]+'` is ok, but the ast parsed from the two is a little different with each other.
Left  := '('
Right := ')'
SExpr ::= Atom | Left SExpr* Right
Okay, now a parser for Lisp is finished! Let’s save this file as lisp.eebnf. Just - download CPython 3.6(If you’re in China, go to Tsinghua Tuna Mirror
and choose the corresponding installer for you OS. - download EBNFParser pip install EBNFParser. - type this codes parserGenerator.py lisp.eebnf ./lispParser.py -test True -comment True And now there should be two files(testLang.py, lispParser.py) automatically generated near by lisp.eebnf. - feel free to try any lisp codes as <Lisp Codes> with following command.
```shell
python testLang.py SExpr `<Lisp Codes>`
# python testLang.py SExpr "(+ 1 +(2 3))" -o test1
SExpr[Left[(]

  SExpr[Atom[+]

  ]
  SExpr[Atom[1]
    ...
]
# python testLang.py SExpr "(define a x y (+ x y))" -o test2
SExpr[Left[(]

  SExpr[Atom[define]

  ]
  SExpr[Atom[a]

  ]
    ...
]
```
See the results at `test1.json, test1Ast, test2.json, test2Ast`.
A complete EEBNF for Lisp can be found at [Grammar](./tests/Python/Lang/Lisp/grammar).
Here are more examples given at the following sections.
Each example has the same structure like:
- grammar. The only file you have to write. - parser.py. Parser and token generated by EBNFParser. - testLang.py. To do some testing easily. - testn.json. The n-th testing result in JSON format. - testnAst. The n-th testing result in S-Expr format.

Some Examples

  • Lisp

    • Grammar See ./tests/Python/Lang/Lisp/grammar.

      Expr  Throw NEWLINE ::= Atom | Quote | '(' NEWLINE* Expr* NEWLINE* ')'
      Quote   ::= '`' Expr
      Atom    := R'\S+'
      NEWLINE := R'\n'
      Stmt Throw NEWLINE  ::= (NEWLINE* Expr* NEWLINE*)*
    • testCodes See ./testpy.sh.

      export PYTHONPATH="Python"
      python Python/parserGenerator.py tests/Python/Lang/Lisp/grammar tests/Python/Lang/Lisp/parser.py -test True
      python tests/Python/Lang/Lisp/testLang.py Stmt "(set r 1) (define a b (+ a (+ r 1)))"  -o tests/Python/Lang/Lisp/test1
    • Result

      • JSON.
        See ./tests/Python/Lang/Lisp/test1.json.
        ...
        {
        "name": "Stmt",
        "value": [
            {
                "name": "Expr",
                "value": [
                    {
                        "name": "'('",
                        "value": "(",
                        "meta": {
                            "rowIdx": 0,
                            "hasParsed": 1,
                            "fileName": "<input>"
                        }
                    },
                    {
                        "name": "Expr",
                        "value": [
                            {
                                "name": "Atom",
                                "value": "set",
                                "meta": {
                                    "rowIdx": 0,
                                    "hasParsed": 2,
                                    "fileName": "<input>"
                                }
                            }
                        ],
        ...
      • Ast See ./tests/Python/Lang/Lisp/test1Ast.

        Stmt[Expr['('[(]
        
            Expr[Atom[set]
        
            ]
            Expr[Atom[r]
        
            ]
            Expr[Atom[1]
        
            ]
            ')'[)]
        
        ]
        Expr['('[(]
        
            Expr[Atom[define]
        
            ]
            Expr[Atom[a]
        
            ]
            Expr[Atom[b]
        
            ]
            Expr['('[(]
        
                Expr[Atom[+]
        
                ]
                Expr[Atom[a]
        
                ]
                Expr['('[(]
        
                        Expr[Atom[+]
        
                        ]
                        Expr[Atom[r]
        
                        ]
                        Expr[Atom[1]
        
                        ]
                        ')'[)]
        
                ]
                ')'[)]
        
            ]
            ')'[)]
        
        ]
        ]
  • Python(Just Expression)

  • ExtraPy Language

  • EBNF(bootstrap)

  • CmLang

Usage

  • Requirement(for Python version)

    • Python 3.6.x

Feel free to clone this project and make parsers for your own language quickly and easily. - Command 1. move to the root of project directory. 2. cd Python/ 3. write an EBNF file to define the grammars for your own language. 4. see the following codes. Also, you can find out more details in each example listed above. Or you can see testpy.sh. shell python parserGenerator.py <EBNF filename> <outputParser.py filename> -test True python <outputParser.py filename> "<codes of your language>" -o <JSON filename> However, if you download EBNFParser with pip, you can use it more conveniently. - Command shell pip install EBNFParser parserGenerator.py <EBNF filename> <outputParser.py filename> python <outputParser.py filename> "<codes of your language>" -o <JSON filename>

Parser-Generator

It is implemented by using bootstrap EBNF gramamr.

Will support C# sooner.


License

GPL

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

EBNFParser-0.1.1.2.tar.gz (12.3 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page