Python Forum
Simple Recursive-Descent Parser - Python 3.7
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Simple Recursive-Descent Parser - Python 3.7
#5
(Sep-13-2021, 05:06 PM)Gribouillis Wrote:
WannaBePythonDev Wrote:As for unittesting - actually, I'm testing my framework using Pytest. Is there some reason unittest would be better than that?
Not especially, many developers prefer pytest because it's easier to parametrize the tests with pytest. Unittest is in the standard library, so that everybody can run the code without installing anything. It is too bad you don't include the tests for your parser, it would allow us to run the code.

WannaBePythonDev Wrote:As for the tokenizer - I think I didn't fully understand the purpose of this tokenization as regards to my code? You recommend it as an addition to my code? Because I don't clearly see how can I take it to my behalf. I'd be glad if you can elaborate on that.
In the field of language parsing, it is customary to separate the lexical analyser from the parser. Of course it is not mandatory. In your case, I'd see two advantages of this
  • A lexer separates the input into tokens which carry additional information: the token type and the exact position of the token in the input string. You can use this position information to generate detailed error messages. This is time saved for the future users of the parser.
  • You could use Python's own lexer and filter the stream of tokens, thus using an extremely robust lexer.

WannaBePythonDev Wrote:As for assertion - how do I know what standard exception should I inherit from for raising an exception? For example, You've inherited RuntimeError in your example - what's posed behind this choice?
RuntimeError is raised for any error that doesn't fall in the other built-in exception categories. Unless your own exceptions fall into these categories, it is a good idea to always make them subclasses of RuntimeError. It is consistent with the documentation.

WannaBePythonDev Wrote:BTW - Can you look at the TODO in line 8?
Unfortunately, I'm not a Windows programmer and I don't know how to use Microsoft tools.

WannaBePythonDev Wrote:Another question - is it fine to make the Parser as a class?
I think it is perfectly sensible as it is what most parsing modules do. One could object that your parser doesn't need to maintain a state during the parsing which makes the object unnecessary, but it is reasonable to think that after a few iterations of the code, you'll want to add some state in the parser, so keep the class.

Great. Thank you so much.

As for the lexer - actually, I tried implementing my Parser using a lexer but I had hard typing issues. Eventually, I gave up and left this way.

######## Start of my trial to implement a firmer hierarchy as befits a RD parser ########
# class TokenType(str, Enum):
#     And: str = "&"
#     Or: str = "|"
#     Imply: str = "->"
#     Neg: str = "~"
#     T: str = "T"
#     F: str = "F"


##### Token types: #####
# BinaryOp = Union[TokenType.And, TokenType.Or, TokenType.Imply]
# UnaryOp = Union[TokenType.Neg]

#### Tokens: ####
# Variable = str

# Constant = Union[TokenType.T, TokenType.F]

# @dataclass
# class UnaryFormula:
#     op: UnaryOp
#     operand: Optional[Formula]  # To follow the typing of Formula class,
#     # though it doesn't make sense for this field to be None

# @dataclass
# class BinaryFormula:
#     left: Optional[Formula]
#     right: Optional[Formula]
#     op: BinaryOp

# ParserFormula = Union[Variable, Constant, UnaryFormula, BinaryFormula]

# def parser_adapter_to_formula(formula_to_adapt: ParserFormula) -> Optional[Formula]:
#     if isinstance(formula_to_adapt, Variable):
#         return Formula(formula_to_adapt, None, None)
#     if isinstance(formula_to_adapt, UnaryFormula):
#         return Formula(formula_to_adapt.op, formula_to_adapt.operand, None)
#     if isinstance(formula_to_adapt, BinaryFormula):
#         return Formula(
#             formula_to_adapt.op, formula_to_adapt.left, formula_to_adapt.right
#         )
#     return None

######## End of trial to implement a firmer hierarchy ########
Of course, I tried using this accordingly in the rest of the code but it caused those issues...
Reply


Messages In This Thread
RE: Simple Recursive-Descent Parser - Python 3.7 - by WannaBePythonDev - Sep-13-2021, 05:29 PM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020