generic_lexer

The minimun python version is 3.9

Original Author:	Eli Bendersky <eliben@gmail.com> with this gist last modified on 2010/08
Author:	Leandro Benedet Garcia <cerberus1746@gmail.com>
Version:	1.1.2
License:	The Unlicense
Documentation:	The documentation can be found here: https://leandro-benedet-garcia.github.io/generic_lexer/

Example

If we try to execute the following code:

.. testcode:: module_sample

   from generic_lexer import Lexer

   rules = {
      "VARIABLE": r"(?P<var_name>[a-z_]+):(?P<var_type>[A-Z]\w+)",
      "EQUALS": r"=",
      "SPACE": r" ",
      "STRING": r"\".*\"",
   }

   data = "first_word:String = \"Hello\""

   for curr_token in Lexer(rules, skip_whitespace=False, text_buffer=data):
      print(curr_token)

Will give us the following output:

.. testoutput:: module_sample

   VARIABLE({'var_name': 'first_word', 'var_type': 'String'}) at 0
   SPACE(' ') at 17
   EQUALS('=') at 18
   SPACE(' ') at 19
   STRING('"Hello"') at 20