It doesn't need to care about indents. I mean the 'untokenize' module cares about them but adddot doesn't. Here is an example (taken from pymotw 3)
import io
# this is PYTHON CODE WITHOUT DOTS
code = r'''
def show_tree(tree, total_width=36, fill=' '):
"""Pretty-print a tree."""
output = StringIO()
last_row = -1
for i, n in enumerate(tree):
if i:
row = int(math floor(math log(i + 1, 2)))
else:
row = 0
if row != last_row:
output write('\n')
columns = 2 ** row
col_width = int(math floor(total_width / columns))
output write(str(n) center(col_width, fill))
last_row = row
print(output getvalue())
print('-' * total_width)
print()
'''
result = adddot(io.StringIO(code))
print(result)
Output:
def show_tree(tree, total_width=36, fill=' '):
"""Pretty-print a tree."""
output = StringIO()
last_row = -1
for i, n in enumerate(tree):
if i:
row = int(math.floor(math.log(i + 1, 2)))
else:
row = 0
if row != last_row:
output.write('\n')
columns = 2 ** row
col_width = int(math.floor(total_width / columns))
output.write(str(n).center(col_width, fill))
last_row = row
print(output.getvalue())
print('-' * total_width)
print()
As you can see, it added the dots at their proper places without modifying the indentation.
I should modernize the code one day or the other because
generate_tokens()
is deprecated in favor of
tokenize()
which takes bytes instead of text... A minor change.