Python Forum

Full Version: how to treat two token as one token(lexer)?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
lets say i have two type of token (the # token and the = token)
when each token is found the position variable will increase by one,but when theres something like joined syntax(not sure what it is called)
example:#=,==,##
the position variable will increase by two ,because it treats=as one token and #as another token
how do i make the the joined syntax increment the position by one instead of two?

i am not on MY computer now ,so i cant show my current python code(the code is in my computer).
so for now,i can only explain.
hope you understand.
sorry for any inconvenient caused.

what i am feeling now: Doh Doh Doh Doh Doh Wall Wall Wall Wall Wall
This isn't as much a Python issue as a compiler/interpreter issue. You should probably find a forum focused more on that domain than something as generic as the language you're implementing things in.