Python Forum
Python Forum

spliting similar but multiple lines
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Modes
spliting similar but multiple lines
anna
Spam, spam, eggs, and spam
Posts: 221
Threads: 71
Joined: Dec 2017
Reputation: -3
#1
Apr-17-2019, 12:45 PM (This post was last modified: Apr-17-2019, 12:45 PM by anna.)
Hi All

I have similar data, 147 columns, need to split each with = and take require data. Instead of spiting line by line.

is there alternative for minimum code ?
sample code
for line in data.split('\n'):
    if "FirstOccurrence" in line:
        Status_NodeAlias = line.split('=')[1].strip()
Quote:Status_NodeAlias = ""
Status_c_FMSONTime_Text = ""
Status_c_GIS = 0
Status_ProcessReq = 0
Status_NmosEntityId = 0
Status_c_CNMSName = ""
Status_FirstOccurrence = 1527503101
Status_c_NEOnTime_Text = ""
Status_c_EnrichmentFlag = 0
Status_RemoteSecObj = ""
Status_ProbeSubSecondId = 24
Status_PhysicalPort = 0
Status_c_SMSNotify = ""
Status_c_HighestRecordedSeverity = 0
Status_c_State = ""
Status_c_AutoCreateTT = ""
Status_SentToAgg = 0
Status_c_Technology = ""
Status_SuppressEscl = 0
Status_c_IPAddress = ""
Status_RemoteRootObj = ""
Status_c_Summary1 = ""
Status_ExtendedAttr = ""
Status_c_CorelationFlag = 0
Status_c_Reporting = 0
Status_c_LowestRecordedSeverity = 0
Status_PhysicalSlot = 0
Status_c_TT_Delay_Time = 0
Status_LocalNodeAlias = ""
Status_NmosManagedStatus = 0
Status_c_AlarmId = 0
Status_c_DelayTimer = 0
Status_AlertGroup = ""
Status_c_Threshold = 0
Status_Class = 0
Status_Flash = 0
Status_URL = ""
Status_Summary = ""
Status_c_NEOffTime_Text = ""
Status_c_EquipmentType = ""
Status_c_PENumber = 0
Status_c_EmailNotify = ""
Status_c_AssignedPortLabel = ""
Status_c_NEOffTime = 0
Status_NmosCauseType = 0
Status_c_AlarmType = ""
Status_Tally = 0
Status_c_FMSONTime = 0
Status_Poll = 0
Status_Agent = ""
Status_c_Operator = ""
Status_c_Interface = ""
Status_c_LowestRecordedSeverityTimestamp = 0
Status_c_HighestRecordedSeverityTimestamp = 0
Status_c_ServiceImpact = ""
Status_StateChange = 0
Status_AlertKey = ""
Status_LocalSecObj = ""
Status_CollectionFirst = 0
Status_c_AMO = ""
Status_c_MEName = ""
Status_TaskList = 0
Status_SourceStateChange = 0
Status_LocalRootObj = ""
Status_OwnerGID = 0
Status_PhysicalCard = ""
Status_Serial = 0
Status_Manager = "eci_lightsoft_tmf814"
Status_X733EventType = 0
Status_c_UnknownFlag = 0
Status_NmosEventMap = ""
Status_SourceServerSerial = 0
Status_RemotePriObj = ""
Status_Customer = ""
Status_ExpireTime = 0
Status_Identifier = ""
Status_CollectionExpireTime = 0
Status_c_EscalateStartTime = 0
Status_NmosObjInst = 0
Status_X733CorrNotif = ""
Status_X733SpecificProb = ""
Status_c_FriendlyName = ""
Status_OldRow = 0
Status_c_DelayClear = 0
Status_AggregationFirst = 0
Status_OwnerUID = 65534
Status_Location = ""
Status_c_Region = ""
Status_c_PE_Id = 0
Status_c_LogDetailFlag = 0
Status_ServerName = "DEL_OBC"
Status_c_PEFlag = 0
Status_Service = ""
Status_c_SuppressFlag = 0
Status_InternalLast = 0
Status_c_HostName = ""
Status_Node = ""
Status_c_PEONFlag = 0
Status_X733ProbableCause = 0
Status_c_FMSOFFTime_Text = ""
Status_RemoteNodeAlias = ""
Status_Severity = 1
Status_SourceServerName = ""
Status_LastOccurrence = 1527503101
Status_Grade = 0
Status_c_Escalate = 0
Status_ServerSerial = 0
Status_c_ProblemType = ""
Status_c_Circle = ""
Status_c_NEOnTime = 0
Status_LocalPriObj = ""
Status_DisplayFirst = 0
Status_c_FMSOFFTime = 0
Status_NmosDomainName = ""
Status_Type = 0
Status_c_City = ""
Status_BSM_Identity = ""
Status_EventId = ""
Status_c_FirstRecordedSeverity = 0
Status_NmosSerial = ""
Status_Acknowledged = 0
notificationId = "53950165"
LSNExt_GroupName = "GZB NEs"
LSNExt_MEName = "408"
LSNExt_AckStatus = "ALM_UNACKNOWLEDGED"
additionalText = "No Info"
ManagedElement = "LSN/EMS_BG-40_80/408"
nativeEMSName = "L12:MPS_6F EoS Port 2-EoS"
perceivedSeverity = "5"
nativeProbableCause = "LOFD"
layerRate = "98"
EMS = "ECI/LightSoft_1"
emsTime = "20180528154750.6"
PTP = "/shelf=1/slot=9/ebtype=MPS_6F/port=2"
domain_name = "tmf_mtnm"
isClearable = "true"
event_name = ""
LSNExt_ALCV = "Unknown"
LSNExt_AssignedPortLabel = "L12:MPS_6F EoS Port 2"
event_type = "NT_ALARM"
objectType = "5"
rcaiIndicator = "false"
serviceAffecting = "1"
probableCause = "LOF"
neTime = "20180528154933.0"
probableCauseQualifier = "408@@8@@-1@@-1@@-1@@-1@@-1@@-1@@-1@@16@@1@@6@@1@@335"
RawCaptureTimeStamp = 1527503101
Find
Reply
Yoriz

Super Moderators
Posts: 2,165
Threads: 35
Joined: Sep 2016
Reputation: 192
#2
Apr-17-2019, 05:53 PM
You could make a dictionary, for each line, split on the '=' use the first split as the key, the second as the value.
Something like below (not tested)
my_dict = {}
for line in data.split('\n'):
    key, value = line.split('=')
    my_dict[key.strip()] = value.strip()
using BBCode | How to ask Smart Questions | What to include | What NOT to include | Homework/No Effort posts
Find
Reply
anna
Spam, spam, eggs, and spam
Posts: 221
Threads: 71
Joined: Dec 2017
Reputation: -3
#3
Apr-17-2019, 06:14 PM 
Error:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-297-371c51b2fefa> in <module>
      1 my_dict = {}
      2 for line in data.split('\n'):
----> 3     key, value = line.split('=')
      4     #my_dict[key.strip()] = value.strip()
      5     print(key,value)

ValueError: too many values to unpack (expected 2)
thanks, but its creating problem on below line

Quote:PTP = "/shelf=1/slot=9/ebtype=MPS_6F/port=2"

having multiple =
Find
Reply
ichabod801
Bunny Rabbit

 Team Member
Posts: 4,229
Threads: 97
Joined: Sep 2016
Reputation: 273
#4
Apr-17-2019, 06:18 PM
Try this:

key, value = line.split('=', maxsplit = 1)
Craig "Ichabod" O'Brien - xenomind.com
I wish you happiness.
Recommended Tutorials: BBCode, functions, classes, text adventures
Website Find
Reply
anna
Spam, spam, eggs, and spam
Posts: 221
Threads: 71
Joined: Dec 2017
Reputation: -3
#5
Apr-18-2019, 09:04 AM
Thanks, its working for me.

Smile
Find
Reply
anna
Spam, spam, eggs, and spam
Posts: 221
Threads: 71
Joined: Dec 2017
Reputation: -3
#6
Apr-19-2019, 01:19 PM
something going wrong, not able to figure out. I am appending dict into list and printing. However its repeating same value again and again.

import codecs
import re
import datetime as ne_dt
import time
import datetime
from datetime import datetime
#from datetime import datetime
from datetime import datetime as dt

def string2date(date):
    readable_date = dt.fromtimestamp(int(date)).strftime("%d/%m/%Y %H:%M:%S")
    return readable_date

def string2neTime(neTime):
    dt_object = datetime.strptime(neTime,'%Y%m%d%H%M%S').strftime("%d/%m/%Y %H:%M:%S")
    return dt_object


netcool_data = {}
netcool_data_list = []
netcool_csv = open('netcool.csv','w')

with open('ECI_LightSoft_DL_DLHI_ECIEMS_FM_DEL.cap.DEL_OBC_1527508037','r',encoding='utf-8',errors='ignore') as logfile:
    #lines = [line for line in logfile.readlines() if line.strip()]
    lines = logfile.read().splitlines()
    #lines = [line for line in logfile.readlines() if line.strip()]
    i = 0
    for line in lines:
        #skipping blank line
        if len(line.strip())== 0:
            continue
        else:
            key, value = line.split('=', maxsplit = 1)
            netcool_data[key.strip()] = value.strip().strip('"')
            #storing dict in list
            netcool_data_list.append(netcool_data)
    for alarm in netcool_data_list:
            #converting date format
            FirstOccurrence = string2date(alarm['Status_FirstOccurrence'])
            LastOccurrence = string2date(alarm['Status_LastOccurrence'])
            print(FirstOccurrence,alarm['notificationId'],alarm['LSNExt_GroupName'],alarm['LSNExt_MEName'],alarm['LSNExt_AckStatus'],alarm['ManagedElement'],alarm['nativeEMSName'],LastOccurrence)
    netcool_csv.close()
Output:
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
28/05/2018 15:55:06 53950171 GGN NEs AM_MARKETPLACES_GGN ALM_UNACKNOWLEDGED LSN/EMS_BG-40_81/7708 A1:SAM_4 oPort 1-VC4#3 28/05/2018 15:55:06
Find
Reply
Yoriz

Super Moderators
Posts: 2,165
Threads: 35
Joined: Sep 2016
Reputation: 192
#7
Apr-19-2019, 02:29 PM
The dict netcool_data is being created outside of the loop.
Inside of the loop the same dict is being overwritten.
The list netcool_data_list has pointers to the same dict object that will all be the last values it was set to.
See following example
netcool_data = {}
netcool_data_list = []

for number in range(10):
    netcool_data['one'] = f'one: {number}'
    netcool_data['two'] = f'two: {number}'
    netcool_data_list.append(netcool_data)

for item in netcool_data_list:
    print(item)
Output:
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
{'one': 'one: 9', 'two': 'two: 9'}
If you move the creation of dict netcool_data to inside the loop you will get independent dict objects
see next example
netcool_data_list = []

for number in range(10):
    netcool_data = {}
    netcool_data['one'] = f'one: {number}'
    netcool_data['two'] = f'two: {number}'
    netcool_data_list.append(netcool_data)

for item in netcool_data_list:
    print(item)
Output:
{'one': 'one: 0', 'two': 'two: 0'}
{'one': 'one: 1', 'two': 'two: 1'}
{'one': 'one: 2', 'two': 'two: 2'}
{'one': 'one: 3', 'two': 'two: 3'}
{'one': 'one: 4', 'two': 'two: 4'}
{'one': 'one: 5', 'two': 'two: 5'}
{'one': 'one: 6', 'two': 'two: 6'}
{'one': 'one: 7', 'two': 'two: 7'}
{'one': 'one: 8', 'two': 'two: 8'}
{'one': 'one: 9', 'two': 'two: 9'}
using BBCode | How to ask Smart Questions | What to include | What NOT to include | Homework/No Effort posts
Find
Reply
anna
Spam, spam, eggs, and spam
Posts: 221
Threads: 71
Joined: Dec 2017
Reputation: -3
#8
Apr-20-2019, 08:53 AM
there are 147 dict items, and i required on few, required alarm names are taken in list and checking against dict items and printing. however its now working, is this correct way?

import codecs
import re
import datetime as ne_dt
import time
import datetime
from datetime import datetime
#from datetime import datetime
from datetime import datetime as dt

def string2date(date):
    readable_date = dt.fromtimestamp(int(date)).strftime("%d/%m/%Y %H:%M:%S")
    return readable_date

def string2neTime(neTime):
    dt_object = datetime.strptime(neTime,'%Y%m%d%H%M%S').strftime("%d/%m/%Y %H:%M:%S")
    return dt_object
def alarmName(alarm_Name,alarmvalue):
    alarmNameList = ['Status_FirstOccurrence','Status_Manager','Status_ServerName','Status_LastOccurrence',\
                     'notificationId','neTime','RawCaptureTimeStamp']
    for alarm_Name in alarmNameList:
        if alarmvalue == '':
           continue
        if alarm_Name == 'Status_FirstOccurrence':
            print(alarmvalue)
            FirstOccurrence = string2date(alarmvalue)
            return FirstOccurrence
        if alarm_Name == 'Status_Manager':
            Status_Manager = alarmvalue
            return Status_Manager
        if alarm_Name == 'Status_ServerName':
            ServerName = alarmvalue
            return ServerName
        if alarm_Name == 'Status_LastOccurrence':
            LastOccurrence = string2date(alarmvalue)
            return LastOccurrence
        if alarm_Name == 'notificationId' :
            notificationId = alarmvalue
            return notificationId
        if alarm_Name == 'neTime':
            neTime = string2neTime(alarmvalue.strip('.')[0])
            return neTime
        if alarm_Name == 'RawCaptureTimeStamp':
            CaptureTimeStamp = string2date(alarmvalue)
	    return CaptureTimeStamp
netcool_data_list = []

with open('ECI_LightSoft_DL_DLHI_ECIEMS_FM_DEL.cap.DEL_OBC_1527508037','r',encoding='utf-8',errors='ignore') as logfile:
    lines = logfile.read().splitlines()
    i = 0
    for line in lines:
        netcool_data = {}
        if len(line.strip())== 0:
            continue
        else:
            key, value = line.split('=', maxsplit = 1)
            netcool_data[key.strip()] = value.strip().strip('"')
            netcool_data_list.append(netcool_data)
    new_dict = {}
    for alarm in netcool_data_list:
        new_dict.update(alarm)
    for key,value in new_dict.items():
            FirstOccurrence = alarmName(key,value)
            LastOccurrence =  alarmName(key,value)
            notificationId = alarmName(key,value)
            CaptureTimeStamp = alarmName(key,value)
            print(FirstOccurrence,LastOccurrence,notificationId,CaptureTimeStamp)
below output is not correct

Output:
01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00
0
0
0
0
01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00
None None None None
None None None None
None None None None
None None None None
0
0
0
0
01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00
None None None None
None None None None
0
0
0
0
01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00 01/01/1970 05:30:00
Find
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  problem with spliting line in print akbarza 3 372 Jan-23-2024, 04:11 PM
Last Post: deanhystad
  How to write the condition for deleting multiple lines? Lky 3 1,126 Jul-10-2022, 02:28 PM
Last Post: Lky
  Delete multiple lines from txt file Lky 6 2,269 Jul-10-2022, 12:09 PM
Last Post: jefsummers
  Display table field on multiple lines, 'wordwrap' 3python 0 1,763 Aug-06-2021, 08:17 PM
Last Post: 3python
  Sum similar items tester_V 3 1,943 Jun-29-2021, 06:58 AM
Last Post: tester_V
  pulling multiple lines from a txt IceJJFish69 3 2,564 Apr-26-2021, 05:56 PM
Last Post: snippsat
  Iterate 2 large text files across lines and replace lines in second file medatib531 13 5,768 Aug-10-2020, 11:01 PM
Last Post: medatib531
  print python json dump onto multiple lines lhailey 2 19,786 Mar-02-2020, 12:47 PM
Last Post: vishalhule
  Random nr. no repetition & printing multiple lines Joey 7 2,783 Feb-05-2020, 04:23 PM
Last Post: Larz60+
  how to insert # to multiple lines? hkfatasy 1 2,886 Dec-22-2019, 01:51 AM
Last Post: ichabod801

Forum Jump:

User Panel Messages

Log Out
My Profile
Pay your profile a visit
User Control Panel
Do some changes on your profile
My Messages
View private messages unread
Avatar
Change avatar
Signature
Change signature
Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020