Appending list Trouble Big Time

[Milfredo]

I am trying to append a list of elements to a larger list as I iterate through the data. Something crazy is happening.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

def extract_horse_data():     horses_name.append( xx[horse_count][18])       horses_info.append( xx[horse_count][0]),horses_info.append( xx[horse_count][1]), horses_info.append(xx[horse_count][2]),     horses_info.append( xx[horse_count][3]),horses_info.append( xx[horse_count][4]), horses_info.append(xx[horse_count][5]),     horses_info.append( xx[horse_count][6]),horses_info.append( xx[horse_count][7]), horses_info.append(xx[horse_count][8]),     horses_info.append( xx[horse_count][9]),horses_info.append( xx[horse_count][10]), horses_info.append(xx[horse_count][11]),     horses_info.append( xx[horse_count][12]),horses_info.append( xx[horse_count][13]), horses_info.append(xx[horse_count][14]),     horses_info.append( xx[horse_count][15]),horses_info.append( xx[horse_count][16]), horses_info.append(xx[horse_count][17]),     horses_info.append( xx[horse_count][18])     print(horses_info[18])             #READ FILE AND PULL OUT HORSES AND HORSE INFO FOR SELECTED RACE AND TRACK         for horse_count in range(0,len(xx)):                                     if int(xx[horse_count][1])== int(race_number):                 horses_info.clear()                          extract_horse_data()                                    #load_horse_data()                                   hcount+=1   print(horse_single_race_info)

As you can see the names print out just fine. But when I print out the horse_single_race_info all I get is just the last horse and none of the first 5 horses. I don't understand what I'm doing wrong.

$ python pyfuncs.py
EYE ON THE FINISH
GODSGIFT
LILLET
I DON'T WANT TO GO
RELEASE THE HEAT
FOGGY FLIGHT
This is what I get when printing the appended list.
[0.0, 15.62, 14.29, 7.58, 14.04, 8.8, 1.6, 'FOGGY FLIGHT', 16, 0.0, 15.62, 14.29, 7.58, 14.04, 8.8, 1.6, 'FOGGY FLIGHT', 16, 0.0, 15.62, 14.29, 7.58, 14.04, 8.8, 1.6, 'FOGGY FLIGHT', 16, 0.0, 15.62, 14.29, 7.58, 14.04, 8.8, 1.6, 'FOGGY FLIGHT', 16, 0.0, 15.62, 14.29, 7.58, 14.04, 8.8, 1.6, 'FOGGY FLIGHT', 16, 0.0, 15.62, 14.29, 7.58, 14.04, 8.8, 1.6, 'FOGGY FLIGHT', 16]

[bowlofred]

It depends on how the data gets here. But there's not enough information to tell. Probably the append is just fine, but the information you're getting to append is wrong.

[Milfredo]

I printed out the the data in the form it is in before trying to append it and it prints out just fine. I know I am doing something wrong. I just don't understand why only the last horse gets appended and not the first five. The name list comes from the same data that the race info comes from. Names are fine, but the other is not.

[deanhystad]

The problem is not the function extract_horse_data, though I don't know why you didn't write it as:

1 2 3 4

def extract_horse_data():     horses_name.append(xx[horse_count][18])     for i in range(19)         horses_info.append(xx[horse_count][i]) # or use horses_info.extend()

My guess is that horse_count is not what you think it is. You probably started with something like this:

1 2 3 4 5 6 7 8 9 10 11 12

all_data = [[str(i)+letter for letter in 'abcdefg'] for i in range(1, 11)] data = []   def extract_data():     for i in [1, 3, 5]:         data.append(all_data[index][i])   for index in range(len(all_data)):     if int(all_data[index][1][0]) in [3, 6, 8]:         extract_data()   print(data)

Output:
['3b', '3d', '3f', '6b', '6d', '6f', '8b', '8d', '8f']

The index used by the for loop and the index used by extract_data() are the same variable. But then you changed the code to something like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

all_data = [[str(i)+letter for letter in 'abcdefg'] for i in range(1, 11)]   index = 0   def extract_data():     for i in [1, 3, 5]:         data.append(all_data[index][i])   def get_data(indices):     for index in range(len(all_data)):         if int(all_data[index][1][0]) in indices:             extract_data()   data = [] get_data([3, 6, 8]) print(data)

Output:
['1b', '1d', '1f', '1b', '1d', '1f', '1b', '1d', '1f']

Global variables are bad. The "index" used in extract_data() is the global variable which is zero (or whatever it was last set). The "index" used in get_data() is a local variable. Looping in get_data() is not changing "index" in "extract_data()".

Instead of using global variables you should use function arguments.

1 2 3 4 5 6 7 8 9 10 11 12 13 14

all_data = [[str(i)+letter for letter in 'abcdefg'] for i in range(1, 11)]   def extract_data(index):     for i in [1, 3, 5]:         data.append(all_data[index][i])   def get_data(indices):     for index in range(len(all_data)):         if int(all_data[index][1][0]) in indices:             extract_data(index)   data = [] get_data([3, 6, 8]) print(data)

This is only a guess. You did not include any code about how xx is made and what clear_data does. You got a bit too overzealous with your redacting and the indentation in your post makes me wonder how all the pieces are related. Also I am left wondering why extract_data appends 19 values, but the repeat in your printout is 9. Are you showing a real printout generated using the extract_data() provided in the post? Is the data messed up in xx?

You mention you can print out the data in the form and it looks ok. What do you mean by that, and where did you put your print statement? Can you amend your example to include the print statement?

[Milfredo]

I struggle with finding the correct logic a lot of the time. So I program with a sledgehammer instead of a scalpel. I am trying to get better. I just changed the extract data to your code.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116

def extract_horse_data():           horses_name.append( xx[horse_count][18])     for i in range(105)         horses_info.append(xx[horse_count][i])       print(horses_info[18])   # just print here to make sure the horse names were being accessed correctly. Won't be in final program     horse_single_race_info.append(horses_info)       def load_track_to_Handi():     my_tracklabel= Label(root, text ="                                                                               ", fg= "#F50808", font = ("sans_serif" , 16)).place(x=380, y=430)     read_csvfile_into_array()                     global track_abrev     global race_number     global xx     global horses_name     global horse_saddle     global hcount     global horse_count     global horse_array     global horse_single_race_info     global horses_info     global horse_data         horse_array = np.arange(2100).reshape(20,105)           #if fileFound ==True:     horse_data = {}     horses_info =  []     horse_single_race_info = []      horse_saddle = []     horses_name = []     hcount = 0                 with open( 'C://guis/f_racefiles/'+ track_abrev + '.csv' , 'r') as race_file:                     df = pd.read_csv(race_file, header =None)                   xx = df.iloc[ :, 0: 105 ].values                     #READ FILE AND PULL OUT HORSES AND HORSE INFO FOR SELECTED RACE AND TRACK         for horse_count in range(0,len(xx)):                                     if int(xx[horse_count][1])== int(race_number):                 horses_info.clear()                          extract_horse_data()                                    #load_horse_data()                                   hcount+=1                                                                                                                                           get_scratches()       #the following is where I start creating factors and where I printed out the 9 data elements checking my work as I go along.   def trnr_jockey(counter):     trainer_win_curmeet =  (horse_single_race_info[counter][6]) / (horse_single_race_info[counter][5])     race_processed.append(trainer_win_curmeet)       jockey_win_curmeet = round(((horse_single_race_info[counter][8])    / (horse_single_race_info[counter][7])) * 100, 2)     race_processed.append(jockey_win_curmeet)     trainer_Win_Pvyr =  round(((horse_single_race_info[counter][10])    / (horse_single_race_info[counter][9])) * 100, 2)     race_processed.append(trainer_Win_Pvyr)     jockey_Win_Pvyr =  round(((horse_single_race_info[counter][12]) / (horse_single_race_info[counter][11])) * 100, 2)     race_processed.append(jockey_Win_Pvyr)     trainer_Win_Cyr =  round(((horse_single_race_info[counter][14]) / (horse_single_race_info[counter][13])) * 100, 2)     race_processed.append(trainer_Win_Cyr )     jockey_Win_Cyr =  round(((horse_single_race_info[counter][16])  / (horse_single_race_info[counter][15 ])) * 100, 2)     race_processed.append(jockey_Win_Cyr)     #   set horses name, morning line,  fold year def name_mornline_foldyr(counter):     morning_line = horse_single_race_info[counter][17]     race_processed.append(morning_line)     horse_name = horse_single_race_info[counter][18]     race_processed.append(horse_name)     fold_yr = horse_single_race_info[counter][19]     race_processed.append(fold_yr)       #print(horse_single_race_info)       #PROCESS HORSES INFOR TO GET FACTORS def process_race(scratched_list):     global race_processed     global counter       race_processed = []     individual_horse_data = {}     counter = 0       for i in range(hcount): # need to change this so can cycle through                   trnr_jockey(counter)         name_mornline_foldyr(counter)                   counter+=1               print(race_processed)

at tyhe get_scratches a windo opens, horses are scratched if they need be and then program is sent to the def process_race(scratched_list):
At the bottom of the program I put a print statement in to check the data in xx to make sure the data structure and actual data is correct. It's not part of the program just a way of looking at xx.

print(xx) and check the printout in the ide gitbash.

[deanhystad]

105??? There are at least 105 pieces of information associated with a horse? There are better ways to organize data than stuffing it in a list and referencing it using indices. Your data must have some structure, the way your data is stored/referenced should reflect that structure.

[Milfredo]

Yes there is. In fact I have a friend who wrote a program much like mine, but in his he has 4,000 factors. The individual track and horse data I read through to get info for just one race is in a data frame. But I do not know how to deal with them yet. As I am just learning Python. Some of the 105 data points need to be combined and at times selected based on conditional statements, which in the end will amount to more than 105 factors.

[Milfredo]

Found out how to create a new data frame with just the rows I need. Now I will massage the data to get what I want.

Thanks again Guys.