Python Forum
how to treat two token as one token(lexer)?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to treat two token as one token(lexer)?
#1
lets say i have two type of token (the # token and the = token)
when each token is found the position variable will increase by one,but when theres something like joined syntax(not sure what it is called)
example:#=,==,##
the position variable will increase by two ,because it treats=as one token and #as another token
how do i make the the joined syntax increment the position by one instead of two?

i am not on MY computer now ,so i cant show my current python code(the code is in my computer).
so for now,i can only explain.
hope you understand.
sorry for any inconvenient caused.

what i am feeling now: Doh Doh Doh Doh Doh Wall Wall Wall Wall Wall
Reply
#2
This isn't as much a Python issue as a compiler/interpreter issue. You should probably find a forum focused more on that domain than something as generic as the language you're implementing things in.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Refresh token for Wyze SDK duckredbeard 0 1,122 May-16-2022, 04:33 AM
Last Post: duckredbeard
  Need to sign JWT token with JWK key stucoder 1 1,706 Feb-21-2022, 09:04 AM
Last Post: stucoder
  Python requests oauth2 access token herobpv 6 3,988 Sep-27-2021, 06:54 PM
Last Post: herobpv
  unexpected token < in json at position 0 Frodoxzibit 5 2,828 Jul-10-2021, 09:33 AM
Last Post: Larz60+
  Treat error ? JohnnyCoffee 11 4,596 Apr-17-2020, 07:42 AM
Last Post: buran
  / token in function parameterlist tpourjalali 1 1,891 Apr-12-2020, 07:05 PM
Last Post: deanhystad
  [gpxpy] "Error parsing XML: not well-formed (invalid token): line 1, column 1" Winfried 5 6,715 Jan-26-2020, 01:09 AM
Last Post: Winfried
  Getting an "Unexpected Token" Error and don't know why... NotAHackusator 1 1,864 Nov-20-2019, 03:00 PM
Last Post: buran
  get and reuse the token value with huawei modem Reims 1 4,600 Oct-02-2019, 04:29 AM
Last Post: Reims
  Need help in adding xsrf token keerthiprashanth 1 2,052 Aug-08-2019, 02:35 PM
Last Post: keerthiprashanth

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020