i am looking for a simpler tokenizer

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Thread Modes

i am looking for a simpler tokenizer

Skaperen
Weighs the Same as a Duck

Posts: 4,646

Threads: 1,493

Joined: Sep 2016

Reputation: 15

Jul-29-2019, 05:31 PM

i want quoted strings to be kept whole, including all the spaces. sure, i can implement this, myself (BTDT in C). in Python, i want to use what's there. that's the Python way. i just need to know what is there.

Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.

Find

Messages In This Thread

i am looking for a simpler tokenizer - by Skaperen - Jul-28-2019, 11:06 PM

RE: i am looking for a simpler tokenizer - by scidam - Jul-28-2019, 11:17 PM

RE: i am looking for a simpler tokenizer - by jonnin - Jul-29-2019, 12:21 AM

RE: i am looking for a simpler tokenizer - by Skaperen - Jul-29-2019, 12:33 AM

RE: i am looking for a simpler tokenizer - by jonnin - Jul-29-2019, 12:01 PM

RE: i am looking for a simpler tokenizer - by DeaD_EyE - Jul-29-2019, 02:19 PM

RE: i am looking for a simpler tokenizer - by Gribouillis - Jul-29-2019, 03:26 PM

RE: i am looking for a simpler tokenizer - by Skaperen - Jul-29-2019, 05:31 PM

Possibly Related Threads…
Thread		Author	Replies	Views	Last Post
	How can you make it simpler?	bitcoin10mil	5	3,760	Aug-22-2020, 08:38 PM Last Post: snippsat
	How to make this simpler.	leoahum	7	5,057	Mar-11-2019, 01:57 PM Last Post: leoahum

Users browsing this thread: 1 Guest(s)

View a Printable Version

i am looking for a simpler tokenizer

User Panel Messages

Announcements