print only last matched line

[tester_V]

Greetings!
I'm having a hard time with this problem.
I need to print only the last match of "second line" for each "first line".

The output should look like this:

First line, second line
First line, second line

Here is a kind of example of a file:

some lines here
some lines here and more
first line to find --InitializeRequest more stuff
some lines here
some lines here and more
second line to find --LotComplete some stuff here
second line to find --LotComplete some stuff here
some lines here
some lines here and more
first line to find --InitializeRequest more stuff
some lines here
some line here and more
second line to find --LotComplete some stuff here
second line to find --LotComplete some stuff here
second line to find --LotComplete some stuff here
second line to find --LotComplete some stuff here
second line to find --LotComplete some stuff here
second line to find --LotComplete some stuff here
some lines here
some lines here and more
first line to find --InitializeRequest more stuff
some lines here
some line here and more
second lines to find --LotComplete some stuff here
second lines to find --LotComplete some stuff here
second lines to find --LotComplete some stuff here
some lines here
some lines here and more
first line to find --InitializeRequest more stuff
some lines here
some lines here and more
second line to find --LotComplete some stuff here

Thank you.

[deanhystad]

A good start is finding which lines match your pattern. Write a program that prints all the lines that match your start pattern.

[tester_V]

Here is the code.
It prints all lines (the ones I'm looking for) but I do not need all lines.
I need to print only the last match of "the second line that is LotComplete" for each "first line that is -- InitializeRequest ".
Do you know how to do that?

Thank you.

import os
import re

lstr = '-- InitializeRequest'
lt_nd = '-- LotComplete'
ltend = ''
hand_f = '\\somelog.txt'

with open(hand_f) as apse:
    for rn_l in apse:
        rn_l= rn_l.rstrip()
        #print (rn_l)
        if lstr in rn_l :
            #print (rn_l)
            rn_l = rn_l.replace('[','').replace(']','')  
            st1,*extraWords, = rn_l.split('-')  
            print ("Lot Start-->> ", st1,'\n')       
          
            
        if lt_nd in rn_l :
            rn_l = rn_l.replace('[','').replace(']','')
            st2,*extraWords, = rn_l.split('-') 
            print("LotComplete -->> ", st2, '\n')

[deanhystad]

Time to pull out pencil and paper. Instead of writing a bunch of text, create a simplified map consisting of first lines, indicated by F, second lines indicated by S and other lines indicated by O.

S O F S S O F O S O S  F  S  O  F  O  S  S  F  O
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Draw a line under each of the first lines and circle all the LAST second lines. How did you do that?

Repeat the exercise again, but this time record your first and second lines using their index. The first/second pairs in my example are (F2,S4), (F6,S10), (F11,S10), (F14,S17). How did you find your pairs? Can you write down the steps used? For my example, what would you do with 0S? This is a second line, but it was not preceded by a First line. What would you do for 18F? This is a first line, but we run out of lines before encountering a second line.

[tester_V]

To deanhystad,
I appreciate your answer but the problem is I do not know when in my log file I'll get the first line,
Or how many lines will be between the first line and the block of second lines,
Or how many second lines I'll have in that block.
Also, the log file could have up to 100,000 lines.

Thank you for trying!

[deanhystad]

I solved the problem with pencil and paper by recording when I encounter a first line or a last line. "Ahh! Here is a first line. When was the last time I saw a first line? When was the last time I saw a second line? What should I do now?"

[tester_V]

To deanhystad,
Thanks again buddy!
I understand, in your head, it is all absolutely clear explanation but not for me.
I have no idea what that means, and how to write code following your direction.
In my mind, I'd push all of the "second lines" into some kind of an array and would read just a last element. I tried to push the strings to the dictionary but it same problem again, when I read the last "Val -[-1]" it is not really "the last element" because elements updated after each iteration of the IF loop.

with open(hand_f) as apse:
    for rn_l in apse:
        rn_l= rn_l.rstrip()
        #print (rn_l)
        if lstr in rn_l :
            #print (rn_l)
            rn_l = rn_l.replace('[','').replace(']','')  
            st1,*extraWords, = rn_l.split('-') # lot start time # 
            print ("Lot Start Time-->> ", st1,'\n')                    # Lot Start Time #
            
        if lt_nd in rn_l :
            rn_l = rn_l.replace('[','').replace(']','')
            st2,*extraWords, = rn_l.split('-') 
            ltend = st2+','
            
            #print("Lot End -->> ", ltend,'\n')
            dict = {"st1": [ltend] }
            print ("Last one -"+dict["st1"][-1])

[deanhystad]

You only need to remember 2 things. Where was the last "first line" and where was the last "second line". Stop thinking about writing it in python and write instructions in natural language. Once you are comfortable that your instructions are good enough that a person could follow them, make them a little more "code-like" (pseudo-code). The final step is to translate to python. I think you are stuck trying to think two steps ahead. This is not a difficult problem if patiently think it out.

My instructions to myself to solve this problem is look at a line.

Write down First Line not found and Second line not found.

For every line in the log file
    Read the line.

    If the line is not a first line or a second line.  Ignore it.

    If this is a second line, write down where it is: Second line = 1ine number

    If this is a first line, check if we have a first/second pair.
        If first line is found and second line is found we have a pair.  Print out (first line, second line)
        write down the current line number as where we found a first line: first line = line number
        Write down second line not found.  We need to restart our search for a second line

Are these adequate instructions for solving the problem with pencil and paper?

[tester_V]

To deanhystad,
Thanks again man but I do not know what "Are these adequate instructions for solving the problem with pencil and paper?" means!

[tester_V]

I'm moving somewhere but I stuck on a first loop.
How can I move to the next loop?

f_toread = 'C:\\02\mytext.txt'
lcomp = '-- LotComplete'
lstr  = '-- InitializeRequest'

lstart = "LotComplete"

with open(f_toread) as fp:  
   line = fp.readline()
   print (line)
   cnt = 1
   flag = False
   while line:
       if lstart in line:
            print(lstart+"string found in line {}".format(cnt))
            break
       line = fp.readline()
       cnt += 1
       if lstr in line :
            print(lstr+"string found in line {}".format(cnt))  
       line = fp.readline()
       cnt += 1