Securing King James Bible (KJV) + King James Bible 1611 (KJV1611) in MariaDB/MySQL

[BrandonKastning]

Securing King James Bible (KJV) + King James Bible 1611 (KJV1611) in MariaDB/MySQL

I have the need to have King James Bible (KJV) in .SQL (which I have secured from bibleprotector.com); I had to use wayback from archive.org, and I was able to download the .SQL

The only King James Bible 1611 I have found online that is as authoritative as you can find (at least I felt this way until Ads appeared all over it one day).

Domain: kingjamesbibleonline.org

They have both KJV & KJV1611 (Very nice website [extremely comprehensive] and the Scripture is Accurate as far as I have tested).

I used WGET to download an offline copy. Now I need to parse the html using bs4 and learning python loops / scripts to parse and payload insert into MariaDB 10.3.x / 10.4.x.

I used the following WGET command:
(wget.start.txt)

wget \
     --random-wait \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --no-check-certificate \
     --output-file=logfile \
     --user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20160101 Firefox/66.0 | SHARPEN YOUR SWORD MINISTRIES - WGET - US1, USIV, USXIV, USARTVI, THIS CONSTITUTION 1787 | Wash 1,1 | Wash 1,2 | Wash 1,7 | Wash 1,11 | Wash 1,12 | Wash 1,29 | Wash 1,30 | Wash 1,32 Nov. 11 1889" "www.kingjamesbibleonline.org"
     --domains www.kingjamesbibleonline.org \
     --no-parent \
         https://www.kingjamesbibleonline.org

The nice thing about doing WGET's this way; I learned that if you open up a terminal; you can divide horizontally and on the top you can paste the WGET (and modify the variables for different offline download projects) and on the bottom you can use the following to watch the download using WGET in real-time from it's log file specified.

tail -f logfile

I found it incredibly helpful when learning to use WGET over the last several years.

Now that I am ready to start parsing this to MariaDB. I used the following information & source:

Source/Tutorials:
https://unix.stackexchange.com/questions...ut-to-file

script.sh:

#!/bin/sh

ls -lrt >> files.log

This an example of the my output from the working HTML directory / file structure (files.log) [My Offline Copy of kingjamesbibleonline.org]:

rwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 1611_1-Kings-7-15
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Discussion-Thread-49691
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Isaiah-23-1_23-7
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Jeremiah-42-22
-rw-r--r--  1 brandon brandon    58571 Nov  4 22:37 search.php?q=Fathers+And+Children&page=22&order=0&bsec=&quote=.html
-rw-r--r--  1 brandon brandon    70626 Nov  4 22:37 search.php?q=Women+For+Pastors&page=76&order=0&bsec=&quote=.html
-rw-r--r--  1 brandon brandon    52271 Nov  4 22:37 search.php?word=Animals+In+Captivity&order=1&bsec=&quote=Animals In Captivity.html
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Discern_Gods_Will
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 1611_2-Kings-14-3
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 1611_Ecclesiastes-Chapter-9
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Jude-1-8_1-10
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Bible-Verses-About-Theft_KJV
-rw-r--r--  1 brandon brandon    44385 Nov  4 22:37 search.php?q=Knowledge&bsec=O&order=0.html
-rw-r--r--  1 brandon brandon    54752 Nov  4 22:37 search.php?q=Your+Neighbor&page=27&order=0&bsec=&quote=.html
-rw-r--r--  1 brandon brandon    83136 Nov  4 22:37 search.php?q=Ask+And+You+Shall+Receive&page=153&order=0&bsec=Z&quote=.html
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Discussion-Thread-149262
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Psalms-Chapter-93_Original-1611-KJV
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 bible-verses-like_Genesis-50-26
-rw-r--r--  1 brandon brandon    52456 Nov  4 22:37 search.php?q=Willful+Sin&page=13&order=0&bsec=&quote=.html
-rw-r--r--  1 brandon brandon    63610 Nov  4 22:37 search.php?q=Obeying+God&page=125&order=0&bsec=&quote=.html
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Discussion-Thread-150082
-rw-r--r--  1 brandon brandon    57947 Nov  4 22:37 search.php?q=The+Second+Coming+Of+Jesus&page=33&order=0&bsec=&quote=.html
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Exodus-21-9
-rw-r--r--  1 brandon brandon    57152 Nov  4 22:37 search.php?q=Trusting+Other+People&page=23&order=0&bsec=&quote=.html
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 1611_Job-30-22
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Bible-Verses-About-Church-Planting
-rw-r--r--  1 brandon brandon    49734 Nov  4 22:37 search.php?q=No+One+Is+Perfect&bsec=O&order=0.html
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Isaiah-59-9_59-11
drwxr-xr-x  2 brandon brandon     4096 Nov  4 22:37 Judges-6-20

I would like to somehow take files.log and output all the "1611" Folders and Filenames to a different .log or .txt.

How do I go about doing this?


Thank you everyone for this forum!

Best Regards,

Brandon Kastning