Python Forum
how to treat two token as one token(lexer)?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to treat two token as one token(lexer)?
#1
lets say i have two type of token (the # token and the = token)
when each token is found the position variable will increase by one,but when theres something like joined syntax(not sure what it is called)
example:#=,==,##
the position variable will increase by two ,because it treats=as one token and #as another token
how do i make the the joined syntax increment the position by one instead of two?

i am not on MY computer now ,so i cant show my current python code(the code is in my computer).
so for now,i can only explain.
hope you understand.
sorry for any inconvenient caused.

what i am feeling now: Doh Doh Doh Doh Doh Wall Wall Wall Wall Wall
Reply
#2
This isn't as much a Python issue as a compiler/interpreter issue. You should probably find a forum focused more on that domain than something as generic as the language you're implementing things in.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Refresh token for Wyze SDK duckredbeard 0 1,111 May-16-2022, 04:33 AM
Last Post: duckredbeard
  Need to sign JWT token with JWK key stucoder 1 1,692 Feb-21-2022, 09:04 AM
Last Post: stucoder
  Python requests oauth2 access token herobpv 6 3,928 Sep-27-2021, 06:54 PM
Last Post: herobpv
  unexpected token < in json at position 0 Frodoxzibit 5 2,807 Jul-10-2021, 09:33 AM
Last Post: Larz60+
  Treat error ? JohnnyCoffee 11 4,571 Apr-17-2020, 07:42 AM
Last Post: buran
  / token in function parameterlist tpourjalali 1 1,875 Apr-12-2020, 07:05 PM
Last Post: deanhystad
  [gpxpy] "Error parsing XML: not well-formed (invalid token): line 1, column 1" Winfried 5 6,689 Jan-26-2020, 01:09 AM
Last Post: Winfried
  Getting an "Unexpected Token" Error and don't know why... NotAHackusator 1 1,853 Nov-20-2019, 03:00 PM
Last Post: buran
  get and reuse the token value with huawei modem Reims 1 4,584 Oct-02-2019, 04:29 AM
Last Post: Reims
  Need help in adding xsrf token keerthiprashanth 1 2,040 Aug-08-2019, 02:35 PM
Last Post: keerthiprashanth

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020