Python Forum
how to treat two token as one token(lexer)?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to treat two token as one token(lexer)?
#1
lets say i have two type of token (the # token and the = token)
when each token is found the position variable will increase by one,but when theres something like joined syntax(not sure what it is called)
example:#=,==,##
the position variable will increase by two ,because it treats=as one token and #as another token
how do i make the the joined syntax increment the position by one instead of two?

i am not on MY computer now ,so i cant show my current python code(the code is in my computer).
so for now,i can only explain.
hope you understand.
sorry for any inconvenient caused.

what i am feeling now: Doh Doh Doh Doh Doh Wall Wall Wall Wall Wall
Reply
#2
This isn't as much a Python issue as a compiler/interpreter issue. You should probably find a forum focused more on that domain than something as generic as the language you're implementing things in.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Refresh token for Wyze SDK duckredbeard 0 1,105 May-16-2022, 04:33 AM
Last Post: duckredbeard
  Need to sign JWT token with JWK key stucoder 1 1,684 Feb-21-2022, 09:04 AM
Last Post: stucoder
  Python requests oauth2 access token herobpv 6 3,895 Sep-27-2021, 06:54 PM
Last Post: herobpv
  unexpected token < in json at position 0 Frodoxzibit 5 2,792 Jul-10-2021, 09:33 AM
Last Post: Larz60+
  Treat error ? JohnnyCoffee 11 4,544 Apr-17-2020, 07:42 AM
Last Post: buran
  / token in function parameterlist tpourjalali 1 1,869 Apr-12-2020, 07:05 PM
Last Post: deanhystad
  [gpxpy] "Error parsing XML: not well-formed (invalid token): line 1, column 1" Winfried 5 6,663 Jan-26-2020, 01:09 AM
Last Post: Winfried
  Getting an "Unexpected Token" Error and don't know why... NotAHackusator 1 1,839 Nov-20-2019, 03:00 PM
Last Post: buran
  get and reuse the token value with huawei modem Reims 1 4,568 Oct-02-2019, 04:29 AM
Last Post: Reims
  Need help in adding xsrf token keerthiprashanth 1 2,029 Aug-08-2019, 02:35 PM
Last Post: keerthiprashanth

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020