Python Forum
Python Forum

python script for inserting rows into hbase table
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Modes
python script for inserting rows into hbase table
lravikumarvsp
Programmer named Tim
Posts: 19
Threads: 10
Joined: May 2018
Reputation: 0
#7
May-22-2018, 06:33 AM (This post was last modified: May-22-2018, 06:36 AM by lravikumarvsp.)
Hi Buran,

How can I insert unique row key in this code
Please help

Thanks
Hi Buran,

Please find the code and output below

import glob
import connect_to_hbase
import os
import csv
import testingsun1
#Changing to the directory path to search for the directory name containing "GSK"

table = connect_to_hbase.conn.table(connect_to_hbase.table_name_Target)
base_path = '/ai2/data/dev/admin/inf/*{}*'
patterns = testingsun1.z

#x = table.row(row.serialnum) 
for pattern in patterns:
	
	search_path =  base_path.format(pattern)
	for f in glob.glob(search_path):
		print("-----------------------")
		print ("The directory path is:")
		print f
		print("List of files in the directory are:")
		os.chdir('/ai2/data/dev/admin/inf/')
    		os.chdir(f)
    		cwd = os.getcwd()
		for subdir, dirs, files in os.walk(cwd, topdown=True):
				for file23 in glob.glob('*.err'):
					print file23
					with open(file23) as f:
						strings = f.read()
						search_word = 'checksum'
						search_word1 = 'Threshold'				
						if(search_word1 in strings):
							with open('/ai2/data/dev/serial/sys/sys_ath_pvt/main/qv_thresholdhistory.csv','r') as csvfile:
								readCSV = csv.reader(csvfile, delimiter=',')
								next(readCSV, None)
								for row in readCSV:
									#print row[2]
									#print os.path.splitext(file23)[0]
									if(os.path.splitext(file23)[0] in row[2]):
										if row[5] and row[9]:
											if(int(row[5]) - int(row[9]) >= 0):
												a = row[3].split("|")
												b = row[4].split("|")
												recom=b[1]
												app_id=row[0]
												entity=row[2]
												existing=a[1]
												data_control=search_word1
												store=recom,app_id,entity,existing,data_control
												print(store)
												for i in range(20):
													table.put('row-%04d' % i, {'APP_VALUES:SEGMENT':'','APP_VALUES:API_KEY':'','APP_VALUES:RECOMMENDED_VALUE':store[0], 'APP_VALUES:APPLICATION_NAME':store[1],'APP_VALUES:ENTITY_NAME':store[2],'APP_VALUES:EXISTING_VALUE':store[3],'APP_VALUES:DATA_CONTROL':store[4],'APP_VALUES:APPLICATION_ID':store[1]})
											
									
						else:
							print ("word not found")
									
Output:
 row-0000                                                    column=APP_VALUES:API_KEY, timestamp=1526970471014, value=
 row-0000                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471014, value=xu0
 row-0000                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471014, value=xu0
 row-0000                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471014, value=Threshold
 row-0000                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471014, value=t_itm
 row-0000                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471014, value=500
 row-0000                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471014, value=12
 row-0000                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471014, value=
 row-0001                                                    column=APP_VALUES:API_KEY, timestamp=1526970471016, value=
 row-0001                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471016, value=xu0
 row-0001                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471016, value=xu0
 row-0001                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471016, value=Threshold
 row-0001                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471016, value=t_itm
 row-0001                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471016, value=500
 row-0001                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471016, value=12
 row-0001                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471016, value=
 row-0002                                                    column=APP_VALUES:API_KEY, timestamp=1526970471020, value=
 row-0002                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471020, value=xu0
 row-0002                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471020, value=xu0
 row-0002                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471020, value=Threshold
 row-0002                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471020, value=t_itm
 row-0002                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471020, value=500
 row-0002                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471020, value=12
 row-0002                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471020, value=
 row-0003                                                    column=APP_VALUES:API_KEY, timestamp=1526970471027, value=
 row-0003                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471027, value=xu0
 row-0003                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471027, value=xu0
 row-0003                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471027, value=Threshold
 row-0003                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471027, value=t_itm
 row-0003                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471027, value=500
 row-0003                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471027, value=12
 row-0003                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471027, value=
 row-0004                                                    column=APP_VALUES:API_KEY, timestamp=1526970471029, value=
 row-0004                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471029, value=xu0
 row-0004                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471029, value=xu0
 row-0004                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471029, value=Threshold
 row-0004                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471029, value=t_itm
 row-0004                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471029, value=500
 row-0004                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471029, value=12
 row-0004                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471029, value=
 row-0005                                                    column=APP_VALUES:API_KEY, timestamp=1526970471031, value=
 row-0005                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471031, value=xu0
 row-0005                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471031, value=xu0
 row-0005                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471031, value=Threshold
 row-0005                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471031, value=t_itm
 row-0005                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471031, value=500
 row-0005                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471031, value=12
 row-0005                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471031, value=
 row-0006                                                    column=APP_VALUES:API_KEY, timestamp=1526970471043, value=
 row-0006                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471043, value=xu0
 row-0006                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471043, value=xu0
 row-0006                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471043, value=Threshold
 row-0006                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471043, value=t_itm
 row-0006                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471043, value=500
 row-0006                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471043, value=12
 row-0006                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471043, value=
 row-0007                                                    column=APP_VALUES:API_KEY, timestamp=1526970471052, value=
 row-0007                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471052, value=xu0
 row-0007                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471052, value=xu0
 row-0007                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471052, value=Threshold
 row-0007                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471052, value=t_itm
 row-0007                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471052, value=500
 row-0007                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471052, value=12
 row-0007                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471052, value=
 row-0008                                                    column=APP_VALUES:API_KEY, timestamp=1526970471054, value=
 row-0008                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471054, value=xu0
 row-0008                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471054, value=xu0
 row-0008                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471054, value=Threshold
 row-0008                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471054, value=t_itm
 row-0008                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471054, value=500
 row-0008                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471054, value=12
 row-0008                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471054, value=
 row-0009                                                    column=APP_VALUES:API_KEY, timestamp=1526970471055, value=
 row-0009                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471055, value=xu0
 row-0009                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471055, value=xu0
 row-0009                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471055, value=Threshold
 row-0009                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471055, value=t_itm
 row-0009                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471055, value=500
 row-0009                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471055, value=12
 row-0009                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471055, value=
 row-0010                                                    column=APP_VALUES:API_KEY, timestamp=1526970471057, value=
 row-0010                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471057, value=xu0
 row-0010                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471057, value=xu0
 row-0010                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471057, value=Threshold
 row-0010                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471057, value=t_itm
 row-0010                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471057, value=500
 row-0010                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471057, value=12
 row-0010                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471057, value=
 row-0011                                                    column=APP_VALUES:API_KEY, timestamp=1526970471064, value=
 row-0011                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471064, value=xu0
 row-0011                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471064, value=xu0
 row-0011                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471064, value=Threshold
 row-0011                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471064, value=t_itm
 row-0011                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471064, value=500
 row-0011                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471064, value=12
 row-0011                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471064, value=
 row-0012                                                    column=APP_VALUES:API_KEY, timestamp=1526970471066, value=
 row-0012                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471066, value=xu0
 row-0012                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471066, value=xu0
 row-0012                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471066, value=Threshold
 row-0012                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471066, value=t_itm
 row-0012                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471066, value=500
 row-0012                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471066, value=12
 row-0012                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471066, value=
 row-0013                                                    column=APP_VALUES:API_KEY, timestamp=1526970471068, value=
 row-0013                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471068, value=xu0
 row-0013                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471068, value=xu0
 row-0013                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471068, value=Threshold
 row-0013                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471068, value=t_itm
 row-0013                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471068, value=500
 row-0013                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471068, value=12
 row-0013                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471068, value=
 row-0014                                                    column=APP_VALUES:API_KEY, timestamp=1526970471069, value=
 row-0014                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471069, value=xu0
 row-0014                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471069, value=xu0
 row-0014                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471069, value=Threshold
 row-0014                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471069, value=t_itm
 row-0014                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471069, value=500
 row-0014                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471069, value=12
 row-0014                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471069, value=
 row-0015                                                    column=APP_VALUES:API_KEY, timestamp=1526970471071, value=
 row-0015                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471071, value=xu0
 row-0015                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471071, value=xu0
 row-0015                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471071, value=Threshold
 row-0015                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471071, value=t_itm
 row-0015                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471071, value=500
 row-0015                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471071, value=12
 row-0015                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471071, value=
 row-0016                                                    column=APP_VALUES:API_KEY, timestamp=1526970471073, value=
 row-0016                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471073, value=xu0
 row-0016                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471073, value=xu0
 row-0016                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471073, value=Threshold
 row-0016                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471073, value=t_itm
 row-0016                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471073, value=500
 row-0016                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471073, value=12
 row-0016                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471073, value=
 row-0017                                                    column=APP_VALUES:API_KEY, timestamp=1526970471074, value=
 row-0017                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471074, value=xu0
 row-0017                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471074, value=xu0
 row-0017                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471074, value=Threshold
 row-0017                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471074, value=t_itm
 row-0017                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471074, value=500
 row-0017                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471074, value=12
 row-0017                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471074, value=
 row-0018                                                    column=APP_VALUES:API_KEY, timestamp=1526970471076, value=
 row-0018                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471076, value=xu0
 row-0018                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471076, value=xu0
 row-0018                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471076, value=Threshold
 row-0018                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471076, value=t_itm
 row-0018                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471076, value=500
 row-0018                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471076, value=12
 row-0018                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471076, value=
 row-0019                                                    column=APP_VALUES:API_KEY, timestamp=1526970471078, value=
 row-0019                                                    column=APP_VALUES:APPLICATION_ID, timestamp=1526970471078, value=xu0
 row-0019                                                    column=APP_VALUES:APPLICATION_NAME, timestamp=1526970471078, value=xu0
 row-0019                                                    column=APP_VALUES:DATA_CONTROL, timestamp=1526970471078, value=Threshold
 row-0019                                                    column=APP_VALUES:ENTITY_NAME, timestamp=1526970471078, value=t_itm
 row-0019                                                    column=APP_VALUES:EXISTING_VALUE, timestamp=1526970471078, value=500
 row-0019                                                    column=APP_VALUES:RECOMMENDED_VALUE, timestamp=1526970471078, value=12
 row-0019                                                    column=APP_VALUES:SEGMENT, timestamp=1526970471078, value=
Find
Reply


Messages In This Thread
RE: python script for inserting rows into hbase table - by lravikumarvsp - May-22-2018, 06:33 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Is there a *.bat DOS batch script to *.py Python Script converter? pstein 3 3,423 Jun-29-2023, 11:57 AM
Last Post: gologica
  The code I have written removes the desired number of rows, but wrong rows Jdesi1983 0 1,652 Dec-08-2021, 04:42 AM
Last Post: Jdesi1983
  Dynamically Add rows to table TommyAutomagically 1 2,135 Nov-04-2021, 10:59 PM
Last Post: TommyAutomagically
  Slittping table into Multiple tables by rows drunkenneo 1 2,100 Oct-06-2021, 03:17 PM
Last Post: snippsat
  Pandas DataFrame combine rows by column value, where Date Rows are NULL rhat398 0 2,151 May-04-2021, 10:51 PM
Last Post: rhat398
  Indexing [::-1] to Reverse ALL 2D Array Rows, ALL 3D, 4D Array Columns & Rows Python Jeremy7 8 7,255 Mar-02-2021, 01:54 AM
Last Post: Jeremy7
  Python script to summarize excel tables, then output a composite table? i'm a total n surfer349 1 2,415 Feb-05-2021, 04:37 PM
Last Post: nilamo
  write mariadb table rows query to each file? shams 1 1,940 Feb-02-2021, 04:10 PM
Last Post: buran
  Inserting multiple rows in a single request. swaroop 2 2,969 Jan-11-2021, 01:34 PM
Last Post: swaroop
  Fetch Oracle DB rows & print it in HTML file with table's col headers in table format tssr_2001 1 3,040 Sep-04-2020, 01:39 PM
Last Post: ibreeden

Forum Jump:

User Panel Messages

Log Out
My Profile
Pay your profile a visit
User Control Panel
Do some changes on your profile
My Messages
View private messages unread
Avatar
Change avatar
Signature
Change signature
Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020