Docx Convert Word Header to Body

[CaptainCsaba]

I am trying to scrape Pdf information. I need a to collect a lot of information about each paragraph. Right now it seems that the way I can extract all the information I want is by converting the pdf into a word document and then going over each paragraph with python-docx. (other methods do not provide as many information).

Everything seems to work as intended, however, an annoying part of the pdf to docx conversion is that some parts of the text get randomly converted to headers and footers in Word if they are too close to the edge of the page. You can't read these parts while iterating over the paragraphs of the body. Python-Docx has a header-footer reader, however it does not work on all documents like on the types I read because of this issue: https://github.com/python-openxml/python...issues/868. So I can't just iterate over the header objects in each section object.

The other method to solve this would be to somehow convert the headers/footers into parts of the body text. There is probably a way in Word to do this, which I would be able to call with win32com, but I can't find it.

So, is there a way to convert a word header/footer into part of the body text, with or without Python?

[Larz60+]

PDF is by nature a strange file structure. There are many forms that a file can take.
some text may actually be just that text, other may be an image. There may or may not be headers.
Same with data, it might be stored in tables (this is what we want), or it might be an image of someone's handwriting.

In my opinion, it's one of the worst methods available for storing data, yet you run across it again and again and must be able, as best as possible, to extract the data into a useful format.

There are many commercial and free packages available for achieving the desired conversion.
each does better on one type of data than others.
pdfminer.six (many other packages are built as wrappers around pdfminer.six), PyPDF2, PyPDF4, which more commonly used from amongst 279 (as of today) PDF packages available here.

I've done a lot of work with U.S. Government, national, state, and local jurisdiction data, and have had to find a different solution for many. Some instances can only be done manually, these are usually the handwritten ones that cannot be interpreted by OCR methods.

Start with the ones (like pdfminer.six and PyPDF2 and/or PyPDF4). There are tutorials for these available. If they don't work, start looking for alternatives.

Good Luck. PDF's are easy, only when the format corporates.

[CaptainCsaba]

I am fairly used the pdfminer and have also worked with PyPDF2 and pdfrw.

I have to extract each paragraph in the pdf, with as many data as possible: font, size, color, bold, italic etc, as much as possible. This is so later I can filter out the ones I want. Let's say I want to select all the text in this pdf that is size 16 with the Times new Roman Font, that has an rgb color of (255,255,255) and is bold etc. I have already created this part, I just have to include the headers somehow.

Pdfminer is almost good, but you can't extract the color and many other properties. Would love for it to work but there is just not enough data that you can extract from the LTTextLine and LTChar objects. The same is true with PyPDF2 and pdfrw.

Converting it to XML and HTML causes data loss thus those are not good alternatives either.

This is why docx is what I am looking for, it is easy to navigate and contains the most data in an easy to get format.

I have tried 3 methods of conversion:

1. pdf2docx pip package is slow and unreliable.
2. Calling Word in win32com is good but really slow.
3. Calling Adobe in win32com is good and also much faster as the others. This is the one I am using.

The problem is that all of them randomly create headers from the top of each page. This is what I am trying to solve. Is there no way to include the headers as part of the body text? Even if it is a way in Word for example, then it can be called and automated. Does nobody have any idea?

EDIT: I am only using non-scanned pdfs. So these things should be extractable.

[Larz60+]

I'm not familiar with any of the MS windows software as I'm on Linux 99% of the time.
someone else will probably know more.