Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 How to Find & Count String Patterns Between two Markers in a HTML file
#1
Hello,

I have a HTML file from which I am trying to extract strings so that I can count them and then hopefully export the analysis of those results in the form of a table.

My HTML file has many "<div></div>" tags that house a repetitive unique pattern. The pattern is:

" to [Name of Player], "

The only thing that is standard in the above are the blank spaces before and after the 'to' along with the comma and space at the end. I need a way to get the full player name from the above pattern based on my HTML file.


The HTML File is as below

</div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">1.6</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Archer to Fakhar Zaman, no run, wide ball prompts a rash swing from Fakhar. Bat nowhere near ball. Better over from England</div>
                                                </div><!-- react-text: 3750 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">1.5</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Archer to Fakhar Zaman, no run, beats the inside edge this time. Bit of swing for Archer. This has been a challenging over</div>
                                                </div><!-- react-text: 3758 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item pre ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">1.4</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Archer to Fakhar Zaman, no run, beautiful bowling. Good length, at pace and swinging away from Imam's outside edge</div>
                                                </div>
                                                <div>
                                                    <p class="comment">"Pakistan could learn a lot from the fearless approach of the Bangladeshi batsmen", Nathan Barnes says. "Will they have the courage to take a risk? "</p>
                                                </div>
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">1.3</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Archer to Fakhar Zaman, no run, shorter and wider, left alone this time</div>
                                                </div><!-- react-text: 3775 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">1.2</div>
                                                        <div class="over-circle low-score"><span class="over-score">4lb</span></div>
                                                    </div>
                                                    <div class="description">Archer to Fakhar Zaman, 4 leg byes, too straight, almost legsid-ish, and Fakhar flicks it off the hips to the fine leg boundary. This is a <strong>good start for Pakistan</strong></div>
                                                </div><!-- react-text: 3783 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">1.1</div>
                                                        <div class="over-circle low-score"><span class="over-score">1</span></div>
                                                    </div>
                                                    <div class="description">Archer to Imam-ul-Haq, 1 run, immediate threat from Archer. Straighter, squared Imam up, an outside edge trickles through to third man</div>
                                                </div><!-- react-text: 3791 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item end-of-over">
                                                <h4>
                                                    <!-- react-text: 3794 -->END OF OVER:
                                                    <!-- /react-text -->
                                                    <div>
                                                        <!-- react-text: 3796 -->1 | 9 Runs | PAK: 9/0
                                                        <!-- /react-text -->
                                                        <!-- react-text: 3797 -->| RR: 9.00
                                                        <!-- /react-text -->
                                                    </div>
                                                </h4>
                                                <ul class="two-col-table">
                                                    <li>Fakhar Zaman<span>8 (4b)</span></li>
                                                    <li>Imam-ul-Haq<span>1 (2b)</span></li>
                                                </ul>
                                                <ul class="two-col-table">
                                                    <li>Chris Woakes<span>1-0-9-0</span></li>
                                                </ul>
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">0.6</div>
                                                        <div class="over-circle high-score"><span class="over-score">4</span></div>
                                                    </div>
                                                    <div class="description">Woakes to Fakhar Zaman, <b>FOUR</b> runs, <strong>slashed away</strong> by Zaman. Got just a bit more room, fuller delivery angling away from the left-hander, and a slice gets it just wide of the man at point.</div>
                                                </div><!-- react-text: 3810 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">0.5</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Woakes to Fakhar Zaman, no run, driven to mid-off</div>
                                                </div><!-- react-text: 3818 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">0.4</div>
                                                        <div class="over-circle high-score"><span class="over-score">4</span></div>
                                                    </div>
                                                    <div class="description">Woakes to Fakhar Zaman, <b>FOUR</b> runs, sliced away to the offside. Looked too close to cut, and there was no timing, but surprising misfield in the covers lets the ball squirt through to the boundary. Morgan the man who misfielded</div>
                                                </div><!-- react-text: 3826 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">0.3</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Woakes to Fakhar Zaman, no run, less room this time. Short and straighter, but Fakhar shoulders arms</div>
                                                </div><!-- react-text: 3834 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item  ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">0.2</div>
                                                        <div class="over-circle low-score"><span class="over-score">1</span></div>
                                                    </div>
                                                    <div class="description">Woakes to Imam-ul-Haq, 1 run, similar line and length, cut away to third man, and Imam is up and running</div>
                                                </div><!-- react-text: 3842 -->
                                                <!-- /react-text -->
                                            </div>
                                            <div class="commentary-item pre ">
                                                <div class="item-wrapper">
                                                    <div class="over">
                                                        <div class="time-stamp">0.1</div>
                                                        <div class="over-circle low-score"><span class="over-score">0</span></div>
                                                    </div>
                                                    <div class="description">Woakes to Imam-ul-Haq, no run, wide outside off stump, Imam lets it go safely by</div>
                                                </div>
                                                <div>
Quote
#2
There's no question in your post, nor have you shown us what you've tried, but you can at least use Beautiful Soup to parse the HTML and extract the data you want from it.
Quote
#3
Can also take a look at Web-Scraping part-1.
Quote
#4
(Aug-18-2019, 01:35 PM)ndc85430 Wrote: There's no question in your post, nor have you shown us what you've tried, but you can at least use Beautiful Soup to parse the HTML and extract the data you want from it.

below is the code I've tried until now (sorry I forgot to attach in the previous post), however the result includes additional strings as well which I didn't exactly want, I just need the player name from this string => ' to [Player Name], '

import numpy as mp
import matplotlib.pyplot as plt
from matplotlib import rc
import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
import re

fname = 'PakVEng.html'
HtmlFile = open(fname, 'r', encoding='utf-8')
source_code = HtmlFile.read()
HtmlFile.close()

testVariable = str(source_code)
vanillaData = testVariable.split('window.__INITIAL_STATE__ = {', 1)[0] + 'window.__INITIAL_STATE__ = {'

totalCount = re.search(' to (.*), ', vanillaData)
print(totalCount)
This code gives me the following result

ahmedwaqas92@ideapad:~/Downloads$ python3 ImportHtml.py 
<_sre.SRE_Match object; span=(268692, 268742), match=' to Shadab Khan, <b>FOUR</b> runs, not quite 350,>
ahmedwaqas92@ideapad:~/Downloads$
The html file is on my computer for testing purposes only, if you want to take a look then go here is the link. All I need from this HTML file is all the player names which occur between ' to [player name], ' and the runs scored after it. This process repeats itself multiple times in the html file so I would want to count all of such instances and then put them in the form of a table.

(Aug-18-2019, 04:49 PM)snippsat Wrote: Can also take a look at Web-Scraping part-1.

I am successfully able to scrape the data onto a string variable, its just that I am having trouble in determining on how to extract sub strings that I want from the html file that I have obtained using Firefox driver selenium scroll and using page source.

This question was part of a larger project that I am working on independently. Below is the link of another question I asked on this thread.

https://python-forum.io/Thread-How-to-Ca...1#pid89131
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Count & Sort occurrences of text in a file oradba4u 7 169 Sep-06-2020, 03:23 PM
Last Post: oradba4u
  Extracting data based on specific patterns in a text file K11 1 190 Aug-28-2020, 09:00 AM
Last Post: Gribouillis
  Find specific subdir, open files and find specific lines that are missing from a file tester_V 8 508 Aug-25-2020, 01:52 AM
Last Post: tester_V
  String Count Kristenl2784 1 233 Jul-23-2020, 02:10 AM
Last Post: millpond
  find a string in a field in MongoDB Leon79 2 242 Jul-19-2020, 09:20 PM
Last Post: menator01
  Find, delete and add text into pdf file a_shvechkov 2 319 Jul-08-2020, 10:50 AM
Last Post: a_shvechkov
  find file not knowing the extension Leon79 6 395 Jul-07-2020, 04:44 PM
Last Post: Leon79
  how to count a retransmission packets from pcap file? salwa17 0 163 Jul-04-2020, 11:22 PM
Last Post: salwa17
  Find the realpath for a symlinked .pth file in itself with python code. hongyi 7 444 Apr-27-2020, 05:30 AM
Last Post: Larz60+
  Making .exe file that requires access to text and html files ClassicalSoul 0 216 Apr-23-2020, 05:03 PM
Last Post: ClassicalSoul

Forum Jump:


Users browsing this thread: 1 Guest(s)