Python Forum
Data manipulation code running but not functioning correctly
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Data manipulation code running but not functioning correctly
#1
Hello! I'm quite new to Python, but I'm learning to use it to manipulate some survey data for my PhD project. The data has been exported as one big .csv file, with a column for every question and a new row for each participant. Every question is number coded as follows: 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.1, 7.2, 7.3...17.18, 17.19, 17.20. I have tried to write some code that will extract all the ".1"s (i.e. 7.1, 8.1, 9.1 etc) from every row and write them to a separate .csv file, then do the same for all the ".2"s and so on. In every file, there should also be each participant's answers to the first six questions at the start of their respective row. The code I have written does run (so there are no syntax errors), and it does write all the individual files to the correct locations, but these files are all empty when opened. I'm not sure why the code I have written isn't populating the files with any data, and I'd really appreciate some help! It's probably not the neatest way to write what I want to do, but it should be easy enough to follow my logic and I'd be really grateful if anyone could point out where I'm going wrong! The code is posted below.

fn = 'File Location'
f = open(fn, 'r')
data = f.read()
outstr1 = ''
outstr2 = ''
outstr3 = ''
outstr4 = ''
outstr5 = ''
outstr6 = ''
outstr7 = ''
outstr8 = ''
outstr9 = ''
outstr10 = ''
outstr11 = ''
outstr12 = ''
outstr13 = ''
outstr14 = ''
outstr15 = ''
outstr16 = ''
outstr17 = ''
outstr18 = ''
outstr19 = ''
outstr20 = ''
data = data.split('\n')
headers = data[0]
data = data[1:]
for h in headers:
    h.split(',')

for d in data:
    row = d.split(',')
    row_dict = dict(zip(headers, row))
    for h in headers:
        if '1.' in h:
            outstr1 += row_dict[h] + ','
            outstr2 += row_dict[h] + ','
            outstr3 += row_dict[h] + ','
            outstr4 += row_dict[h] + ','
            outstr5 += row_dict[h] + ','
            outstr6 += row_dict[h] + ','
            outstr7 += row_dict[h] + ','
            outstr8 += row_dict[h] + ','
            outstr9 += row_dict[h] + ','
            outstr10 += row_dict[h] + ','
            outstr11 += row_dict[h] + ','
            outstr12 += row_dict[h] + ','
            outstr13 += row_dict[h] + ','
            outstr14 += row_dict[h] + ','
            outstr15 += row_dict[h] + ','
            outstr16 += row_dict[h] + ','
            outstr17 += row_dict[h] + ','
            outstr18 += row_dict[h] + ','
            outstr19 += row_dict[h] + ','
            outstr20 += row_dict[h] + ','
        elif '2.' in h:
            outstr1 += row_dict[h] + ','
            outstr2 += row_dict[h] + ','
            outstr3 += row_dict[h] + ','
            outstr4 += row_dict[h] + ','
            outstr5 += row_dict[h] + ','
            outstr6 += row_dict[h] + ','
            outstr7 += row_dict[h] + ','
            outstr8 += row_dict[h] + ','
            outstr9 += row_dict[h] + ','
            outstr10 += row_dict[h] + ','
            outstr11 += row_dict[h] + ','
            outstr12 += row_dict[h] + ','
            outstr13 += row_dict[h] + ','
            outstr14 += row_dict[h] + ','
            outstr15 += row_dict[h] + ','
            outstr16 += row_dict[h] + ','
            outstr17 += row_dict[h] + ','
            outstr18 += row_dict[h] + ','
            outstr19 += row_dict[h] + ','
            outstr20 += row_dict[h] + ','
        elif '3.' in h:
            outstr1 += row_dict[h] + ','
            outstr2 += row_dict[h] + ','
            outstr3 += row_dict[h] + ','
            outstr4 += row_dict[h] + ','
            outstr5 += row_dict[h] + ','
            outstr6 += row_dict[h] + ','
            outstr7 += row_dict[h] + ','
            outstr8 += row_dict[h] + ','
            outstr9 += row_dict[h] + ','
            outstr10 += row_dict[h] + ','
            outstr11 += row_dict[h] + ','
            outstr12 += row_dict[h] + ','
            outstr13 += row_dict[h] + ','
            outstr14 += row_dict[h] + ','
            outstr15 += row_dict[h] + ','
            outstr16 += row_dict[h] + ','
            outstr17 += row_dict[h] + ','
            outstr18 += row_dict[h] + ','
            outstr19 += row_dict[h] + ','
            outstr20 += row_dict[h] + ','
        elif '4.' in h:
            outstr1 += row_dict[h] + ','
            outstr2 += row_dict[h] + ','
            outstr3 += row_dict[h] + ','
            outstr4 += row_dict[h] + ','
            outstr5 += row_dict[h] + ','
            outstr6 += row_dict[h] + ','
            outstr7 += row_dict[h] + ','
            outstr8 += row_dict[h] + ','
            outstr9 += row_dict[h] + ','
            outstr10 += row_dict[h] + ','
            outstr11 += row_dict[h] + ','
            outstr12 += row_dict[h] + ','
            outstr13 += row_dict[h] + ','
            outstr14 += row_dict[h] + ','
            outstr15 += row_dict[h] + ','
            outstr16 += row_dict[h] + ','
            outstr17 += row_dict[h] + ','
            outstr18 += row_dict[h] + ','
            outstr19 += row_dict[h] + ','
            outstr20 += row_dict[h] + ','
        elif '5.' in h:
            outstr1 += row_dict[h] + ','
            outstr2 += row_dict[h] + ','
            outstr3 += row_dict[h] + ','
            outstr4 += row_dict[h] + ','
            outstr5 += row_dict[h] + ','
            outstr6 += row_dict[h] + ','
            outstr7 += row_dict[h] + ','
            outstr8 += row_dict[h] + ','
            outstr9 += row_dict[h] + ','
            outstr10 += row_dict[h] + ','
            outstr11 += row_dict[h] + ','
            outstr12 += row_dict[h] + ','
            outstr13 += row_dict[h] + ','
            outstr14 += row_dict[h] + ','
            outstr15 += row_dict[h] + ','
            outstr16 += row_dict[h] + ','
            outstr17 += row_dict[h] + ','
            outstr18 += row_dict[h] + ','
            outstr19 += row_dict[h] + ','
            outstr20 += row_dict[h] + ','
        elif '6.' in h:
            outstr1 += row_dict[h] + ','
            outstr2 += row_dict[h] + ','
            outstr3 += row_dict[h] + ','
            outstr4 += row_dict[h] + ','
            outstr5 += row_dict[h] + ','
            outstr6 += row_dict[h] + ','
            outstr7 += row_dict[h] + ','
            outstr8 += row_dict[h] + ','
            outstr9 += row_dict[h] + ','
            outstr10 += row_dict[h] + ','
            outstr11 += row_dict[h] + ','
            outstr12 += row_dict[h] + ','
            outstr13 += row_dict[h] + ','
            outstr14 += row_dict[h] + ','
            outstr15 += row_dict[h] + ','
            outstr16 += row_dict[h] + ','
            outstr17 += row_dict[h] + ','
            outstr18 += row_dict[h] + ','
            outstr19 += row_dict[h] + ','
            outstr20 += row_dict[h] + ','
        elif '.1' in h:
            outstr1 += row_dict[h] + ','
        elif '.2' in h:
            outstr2 += row_dict[h] + ','
        elif '.3' in h:
            outstr3 += row_dict[h] + ','
        elif '.4' in h:
            outstr4 += row_dict[h] + ','
        elif '.5' in h:
            outstr5 += row_dict[h] + ','
        elif '.6' in h:
            outstr6 += row_dict[h] + ','
        elif '.7' in h:
            outstr7 += row_dict[h] + ','
        elif '.8' in h:
            outstr8 += row_dict[h] + ','
        elif '.9' in h:
            outstr9 += row_dict[h] + ','
        elif '.10' in h:
            outstr10 += row_dict[h] + ','
        elif '.11' in h:
            outstr11 += row_dict[h] + ','
        elif '.12' in h:
            outstr12 += row_dict[h] + ','
        elif '.13' in h:
            outstr13 += row_dict[h] + ','
        elif '.14' in h:
            outstr14 += row_dict[h] + ','
        elif '.15' in h:
            outstr15 += row_dict[h] + ','
        elif '.16' in h:
            outstr16 += row_dict[h] + ','
        elif '.17' in h:
            outstr17 += row_dict[h] + ','
        elif '.18' in h:
            outstr18 += row_dict[h] + ','
        elif '.19' in h:
            outstr19 += row_dict[h] + ','
        elif '.20' in h:
            outstr20 += row_dict[h] + ','
    outstr1 += '\n'
    outstr2 += '\n'
    outstr3 += '\n'
    outstr4 += '\n'
    outstr5 += '\n'
    outstr6 += '\n'
    outstr7 += '\n'
    outstr8 += '\n'
    outstr9 += '\n'
    outstr10 += '\n'
    outstr11 += '\n'
    outstr12 += '\n'
    outstr13 += '\n'
    outstr14 += '\n'
    outstr15 += '\n'
    outstr16 += '\n'
    outstr17 += '\n'
    outstr18 += '\n'
    outstr19 += '\n'
    outstr20 += '\n'

fn1 = 'New File1 Location'
fn2 = 'New File2 Location'
fn3 = 'New File3 Location'
fn4 = 'New File4 Location'
fn5 = 'New File5 Location'
fn6 = 'New File6 Location'
fn7 = 'New File7 Location'
fn8 = 'New File8 Location'
fn9 = 'New File9 Location'
fn10 = 'New File10 Location'
fn11 = 'New File11 Location'
fn12 = 'New File12 Location'
fn13 = 'New File13 Location'
fn14 = 'New File14 Location'
fn15 = 'New File15 Location'
fn16 = 'New File16 Location'
fn17 = 'New File17 Location'
fn18 = 'New File18 Location'
fn19 = 'New File19 Location'
fn20 = 'New File20 Location'
f1 = open(fn1, 'w')
f1.write(outstr1)
f1.close()
f2 = open(fn2, 'w')
f2.write(outstr2)
f2.close()
f3 = open(fn3, 'w')
f3.write(outstr3)
f3.close()
f4 = open(fn4, 'w')
f4.write(outstr4)
f4.close()
f5 = open(fn5, 'w')
f5.write(outstr5)
f5.close()
f6 = open(fn6, 'w')
f6.write(outstr6)
f6.close()
f7 = open(fn7, 'w')
f7.write(outstr7)
f7.close()
f8 = open(fn8, 'w')
f8.write(outstr8)
f8.close()
f9 = open(fn9, 'w')
f9.write(outstr9)
f9.close()
f10 = open(fn10, 'w')
f10.write(outstr10)
f10.close()
f11 = open(fn11, 'w')
f11.write(outstr11)
f11.close()
f12 = open(fn12, 'w')
f12.write(outstr12)
f12.close()
f13 = open(fn13, 'w')
f13.write(outstr13)
f13.close()
f14 = open(fn14, 'w')
f14.write(outstr14)
f14.close()
f15 = open(fn15, 'w')
f15.write(outstr15)
f15.close()
f16 = open(fn16, 'w')
f16.write(outstr16)
f16.close()
f17 = open(fn17, 'w')
f17.write(outstr17)
f17.close()
f18 = open(fn18, 'w')
f18.write(outstr18)
f18.close()
f19 = open(fn19, 'w')
f19.write(outstr19)
f19.close()
f20 = open(fn20, 'w')
f20.write(outstr20)
f20.close()
Reply
#2
Your code is really unclear and hard to follow. Use more descriptive variable names, and use comments to clarify what you are doing and why you are doing it.

Based on your description, I think you want something more like this:

first_six = []
point_ones = []
point_twos = []
...
for row in data:
    fields = row.split(',')
    first_six.append(fields[:6])
    row_ones = []
    row_twos = []
    ...
    for name, field in zip(headers, fields):
        if name.endswith('.1'):
            row_ones.append(field)
        if name.endswith('.2'):
            row_twos.append(field)
        ....
    point_ones.append(row_ones)
    point_twos.append(row_twos)
That will give you the first six fields for every row, and split out the .1, .2, and so on fields into point_foo lists. Each file would then be made with '\n'.join([','.join(six + extra) for six, extra in zip(first_six, point_one)])
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Why can I not see the code correctly in Python IDLE.? Trump 7 439 Mar-14-2024, 09:05 PM
Last Post: deanhystad
  problem in running a code akbarza 7 547 Feb-14-2024, 02:57 PM
Last Post: snippsat
  writing and running code in vscode without saving it akbarza 1 346 Jan-11-2024, 02:59 PM
Last Post: deanhystad
  hi need help to make this code work correctly atulkul1985 5 702 Nov-20-2023, 04:38 PM
Last Post: deanhystad
  the order of running code in a decorator function akbarza 2 480 Nov-10-2023, 08:09 AM
Last Post: akbarza
  googletrans library to translate text language for using data frame is not running gcozba2023 0 1,164 Mar-06-2023, 09:50 AM
Last Post: gcozba2023
  Correctly read a malformed CSV file data klllmmm 2 1,815 Jan-25-2023, 04:12 PM
Last Post: klllmmm
  Code running many times nad not just one? korenron 4 1,326 Jul-24-2022, 08:12 AM
Last Post: korenron
  Error while running code on VSC maiya 4 3,550 Jul-01-2022, 02:51 PM
Last Post: maiya
  code running for more than an hour now, yet didn't get any result, what should I do? aiden 2 1,424 Apr-06-2022, 03:41 PM
Last Post: Gribouillis

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020