May-22-2018, 03:28 AM
I am not expecting anyone to write a code for me.
I need a suggestion.
I had a script for inserting data into hbase table.
But table.put statement is inserting only the last output value of the print (store) statement.
Print store output:
Please suggest.
Thanks
I need a suggestion.
I had a script for inserting data into hbase table.
import glob import connect_to_hbase import os import csv import testingsun1 #Changing to the directory path to search for the directory name containing "GSK" table = connect_to_hbase.conn.table(connect_to_hbase.table_name_Target) base_path = '/ai2/data/dev/admin/inf/*{}*' patterns = testingsun1.z #x = table.row(row.serialnum) for pattern in patterns: search_path = base_path.format(pattern) for f in glob.glob(search_path): print("-----------------------") print ("The directory path is:") print f print("List of files in the directory are:") os.chdir('/ai2/data/dev/admin/inf/') os.chdir(f) cwd = os.getcwd() for subdir, dirs, files in os.walk(cwd, topdown=True): for file23 in glob.glob('*.err'): print file23 with open(file23) as f: strings = f.read() search_word = 'checksum' search_word1 = 'Threshold' if(search_word1 in strings): with open('/ai2/data/dev/serial/sys/sys_ath_pvt/main/qv_thresholdhistory.csv','r') as csvfile: readCSV = csv.reader(csvfile, delimiter=',') next(readCSV, None) for row in readCSV: #print row[2] #print os.path.splitext(file23)[0] if(os.path.splitext(file23)[0] in row[2]): if row[5] and row[9]: if(int(row[5]) - int(row[9]) >= 0): a = row[3].split("|") b = row[4].split("|") recom=b[1] app_id=row[0] entity=row[2] existing=a[1] data_control=search_word1 store=recom,app_id,entity,existing,data_control print(store) table.put('row-key', {'APP_VALUES:SEGMENT':'','APP_VALUES:API_KEY':'','APP_VALUES:RECOMMENDED_VALUE':store[0], 'APP_VALUES:APPLICATION_NAME':store[1],'APP_VALUES:ENTITY_NAME':store[2],'APP_VALUES:EXISTING_VALUE':store[3],'APP_VALUES:DATA_CONTROL':store[4],'APP_VALUES:APPLICATION_ID':store[1]}) else: print ("word not found")In the above code print (store) is printing all the values which are to be inserted into the hbase table.
But table.put statement is inserting only the last output value of the print (store) statement.
Print store output:
Output:The directory path is:
/ai2/data/dev/admin/inf/inf_DRM_pvt
List of files in the directory are:
call_report_y9clookup.err
('0', 'drm', 'call_report_y9clookup', '300', 'Threshold')
-----------------------
The directory path is:
/ai2/data/dev/admin/inf/inf_GSK_pvt
List of files in the directory are:
gs_attribute.err
word not found
-----------------------
The directory path is:
/ai2/data/dev/admin/inf/inf_IPT_pvt
List of files in the directory are:
run_ingest_IPT_daily_1246.err
word not found
-----------------------
The directory path is:
/ai2/data/dev/admin/inf/inf_XU0_pvt
List of files in the directory are:
t_itm.err
('12', 'xu0', 't_itm', '500', 'Threshold')
Data inserted in the table:Output:1.8.7-p357 :030 > scan 'TMP_ATLAS_DSCVRY_APPS'
ROW COLUMN+CELL
row-key column=APP_VALUES:API_KEY, timestamp=1526959583897, value=
row-key column=APP_VALUES:APPLICATION_ID, timestamp=1526959583897, value=xu0
row-key column=APP_VALUES:APPLICATION_NAME, timestamp=1526959583897, value=xu0
row-key column=APP_VALUES:DATA_CONTROL, timestamp=1526959583897, value=Threshold
row-key column=APP_VALUES:ENTITY_NAME, timestamp=1526959583897, value=t_itm
row-key column=APP_VALUES:EXISTING_VALUE, timestamp=1526959583897, value=500
row-key column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526959583897, value=12
row-key column=APP_VALUES:SEGMENT, timestamp=1526959583897, value=
1 row(s) in 0.3440 seconds
Where I am doing wrong?Please suggest.
Thanks