Read Multiples Text Files get specific lines based criteria - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: Read Multiples Text Files get specific lines based criteria (/thread-26851.html) |
Read Multiples Text Files get specific lines based criteria - zinho - May-15-2020 How can I read multiples text files get specific lines by criteria a save into CSV master file I try this import glob import pandas as pd lista = [] try: for f_name in glob.glob(r'C:\Users\zinho\Desktop\Projet_Txt'): if f_name.endswith('.txt'): with open(f_name, 'r') as f: for i in f: if i[:6] == '|D100|': lista.append(i) except UnicodeDecodeError: pass df = pd.DataFrame(lista) df.to_csv('Master.csv', index=False, header=False)Note: I can't use pandas to read, becouse everything that tray fail. RE: Read Multiples Text Files get specific lines based criteria - menator01 - May-15-2020 Look into os.walk and readlines and split RE: Read Multiples Text Files get specific lines based criteria - zinho - May-18-2020 I don't undstand, at home I find way to solve my problem, but when I test this code at work does't work. After run this code (change filepaths to my job computer), don't show D100 lines, showing nothing. import glob import pandas as pd filepaths = glob.glob("/home/zinho/Downloads/*.txt") lista = [] ''' Faz a leitura de vários arquivos txt Copia as linhas baseado no critério |D100| e salva em um csv ''' try: for fp in filepaths: with open(fp, 'r') as f: lin = f.readlines() for cnt in lin: if cnt[:6] == '|D100|': lista.append(cnt) # Essa linha cuida do final do arquivo com caracteres estrnhos except UnicodeDecodeError: pass df = pd.DataFrame(lista) df.to_csv('/home/zinho/Downloads/Master.csv', index=False, header=False) RE: Read Multiples Text Files get specific lines based criteria - menator01 - May-18-2020 This works for me # /usr/bin/env python3 import os for root, dirs, files in os.walk('./', topdown=True): for name in files: if 'txt' in name: with open(name, 'r') as lines: for line in lines: for word in line.split(): if '|D100|' in word: print(line) Directory structure
RE: Read Multiples Text Files get specific lines based criteria - zinho - May-18-2020 I try your code, but the result it's same of mine, I mean nothing is show My text letter file as exemple has 70K rows, I take peace of it After run your code My original file is herehttps://santacruzdistribuidora-my.sharepoint.com/:t:/g/personal/alexandre_goncalves_gruposc_com_br/EW8ZAkBQuRpNvi6BlKUaWHEB6H0Mz4CBoSgngZj-h8wXIg?e=GWdqxr RE: Read Multiples Text Files get specific lines based criteria - zinho - May-19-2020 Hi Finaly a solve this. import glob, os import pandas as pd path = 'C:\\Users\\zinho\\Desktop\\Projet_Txt\\*.txt' f_names = glob.glob(path) lista = [] for file in f_names: try: with open(file, 'r') as f: try: for line in f: if line[:6] == '|D100|': lista.append(line) except UnicodeDecodeError: pass except IOError as exc: if exc.errno != errno.EISDIR: raise df = pd.DataFrame(lista) df.to_csv('C:\\Users\\zinho\\Downloads\\Master.csv', index=False, header=False) |