Loop through values in dictrionary and find the same as in previous row

[Paqqno]

I am new in Python and I am stuck with one problem for a few days now. I made a script that:

-takes data from CSV file
-sort it by same values in first column of data file
-instert sorted data in specifield line in different template text file
-save the file in as many copies as there are different values in first column from data file This picture below show how it works:


But there are two more things I need to do. When in separate files as showed above, there are some of the same values from second column of the data file, then this file should insert value from third column instead of repeating the same value from second column. On the picture below I showed how it should look like



What I also need is to add somewhere separeted value of first column from data file by "_".

There is datafile:

111_0,3005,QWE
111_0,3006,SDE
111_0,3006,LFR
111_1,3005,QWE
111_1,5345,JTR
112_0,3103,JPP
112_0,3343,PDK 
113_0,2137,TRE
113_0,2137,OMG

and there is code i made:

import shutil

with open("data.csv") as f:
    contents = f.read()
    contents = contents.splitlines()

values_per_baseline = dict()

for line in contents:
    key = line.split(',')[0]
    values = line.split(',')[1:]
    if key not in values_per_baseline:
        values_per_baseline[key] = []
    values_per_baseline[key].append(values)

for file in values_per_baseline.keys():
    x = 3
    shutil.copyfile("of.txt", (f"of_%s.txt" % file))
    filename = f"of_%s.txt" % file
    for values in values_per_baseline[file]:
        with open(filename, "r") as f:
            contents = f.readlines()
            contents.insert(x, '      o = ' + values[0] + '\n          ' + 'a = ' + values[1] +'\n')
        with open(filename, "w") as f:
            contents = "".join(contents)
            f.write(contents)
            f.close()

Any suggestions would be appreciated.

[deanhystad]

I think you need to organize the data as a dictionary of dictionaries of lists. Like this:

import csv
from pprint import pprint

data = {}
with open("data.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        a, b, c = row
        data[a] = data.get(a, {})
        data[a][b] = data[a].get(b, []) + [c]

pprint(data)

Output:
{'111_0': {'3005': ['QWE'], '3006': ['SDE', 'LFR']},
 '111_1': {'3005': ['QWE'], '5345': ['JTR']},
 '112_0': {'3103': ['JPP'], '3343': ['PDK ']},
 '113_0': {'2137': ['TRE', 'OMG']}}

Now when you write the file for "111_0" you know there is one "3005" (QWE) and two "3006" (SDE, LFR).

[Paqqno]

It looks very promising but I still struggle to implement your solution to my code because attempt to write it to a file ends up with writing only the last row from data file, even though It creates files of every different value as it should. Do you have any more tips how to deal with it? Sorry if I am asking for dumb things, it's basically my first program made in python.

[deanhystad]

Please post your code that writes the files.

[Paqqno]

import csv
import shutil

data = {}

with open("data.csv", "r") as f:
    reader = csv.reader(f)
    for row in reader:
        a, b, c = row
        data[a] = data.get(a, {})
        data[a][b] = data[a].get(b, []) + [c]

for file in data:
    x=3
    shutil.copyfile("of.txt", (f"of_%s.txt" % file))
    filename = f"of_%s.txt" % file
    with open(filename, "r") as f:
        contents = f.readlines()
        contents.insert(x, '      o = ' + b + '\n          ' + 'a = ' + c + '\n')
    with open(filename, "w") as f:
        contents = "".join(contents)
        f.write(contents)
        f.close()

I have no idea how to loop through values after looping by the first row (where files are created). Basically the idea was to write it like that:
If second column value from CSV file is not the same as in previous row then:
Write in file: "O = <value from second column> A = <value from third column>"
If second column value from CSV file is the same as in previous row then:
Write in file: "A = <value from third column>"

[deanhystad]

I am calling the three columns in the datafile a, b and c. I'd give them better names if I knew what they meant. So when you see "a", "b" or "c" that is what they mean.

You need to loop through the entries in the dictionary. Looping through data ("a") gives you the filenames. Inside that loop you loop through the numbers ("b") and insert a line for each number. The line you insert uses the three letter codes ("c"). You are missing the "b" loop.

import csv

data = {}
with open("data.csv", "r") as f:
    reader = csv.reader(f)
    for row in reader:
        a, b, c = row
        data[a] = data.get(a, {})
        data[a][b] = data[a].get(b, []) + [c]

for a in data:
    with open("of.txt", "r") as src:
        lines = src.readlines()
        for index, b in enumerate(data[a]):
            c = data[a][b]
            lines.insert(3+index, f"\to = {b}\n\t\ta = {', '.join(c)}\n")
    with open(f"of_{a}.txt", "w") as dst:
        dst.writelines(lines)

There is no need for shutil. Opening a file for writing automatically creates the file if it doesn't exist. You also don't have to close a file if it is opened using a context manager (with open()...). I created a dummy of.txt file and ran the code. This is of_111_0.txt:

Output:
line
line
line
	o = 3005
		a = QWE
	o = 3006
		a = SDE, LFR
line
line
line
line
line

---

Something was nagging me about the code I posted and it rewrote itself while I slept.

import csv

data = {}
with open("data.csv", "r") as data_file:
    for a, b, c in  csv.reader(data_file):
        data[a] = data.get(a, {})
        data[a][b] = data[a].get(b, []) + [c]

with open("of.txt", "r") as src:
    lines = src.readlines()

for a in data:
    with open(f"of_{a}.txt", "w") as dst:
        dst.writelines(lines[:3])
        for b in data[a]:
            c = data[a][b]
            dst.write(f"\to = {b}\n\t\ta = {', '.join(c)}\n")
        dst.writelines(lines[3:])