Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Scraping a dynamic data-table in python through AJAX request
#1
Ref to the stackoverflow thread:
I would like to implement a demo backend script in python in order to scrape the list of document paths through the URL. If I try to handle through the basic request with urllib.request, it does not return the content of the datatable. It is just returned an empty content as below. Hence, the content of the data table might be captured with the AJAX request in this scenario. The example code which is taken from homepage of the actual website (Actual Page). I think main issue here might be token side. How can I handle the python side in order to capture the content of the dynamic datatable?

The code I wrote on python side (Current python code) can not handle this request and returns "The page has expired due to inactivity."

Any suggestions, directions ?

Current python code:

from requests import Session
session = Session()
session.head("mainpage")
response = session.post(
    url= 'apiurl/getviewdatatable',
    data= {
        'name' : 'name',
        'documentPath' : 'documentPath',
    }           
)

print(response.text)
Actual Page: mainpage

<script>
        $(document).ready(function () {
            $('#posts').DataTable({
                "processing": true,
                "serverSide": true,
                "ajax": {
                    "url": "apiurl/getviewdatatable",
                    "dataType": "json",
                    "type": "POST",
                    "data": {
                        _token: "5uC8cmrJRYnn2gB4TCU9EFj0wjgA8BtmqqO64Uk1",
                    }
                },
                "columns": [
                                            {
                        "data": "name"
                    },
                                        {
                        "data": "documentPath"
                    },

                ],
                'createdRow': function (row, data, index) {
                    //
                    // if the second column cell is blank apply special formatting
                    //
                    $(row).addClass('table-row');

                },
                "language": {
                    "url": "//cdn.datatables.net/plug-ins/9dcbecd42ad/i18n/Turkish.json"
                },
                "rowReorder":{
                    selector:'td:nth-child(2)'
                },
            });
        });
</script>
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Extract data from a table Bob_M 0 67 Aug-05-2020, 12:49 PM
Last Post: Bob_M
  Web Scraping and data collection. LordDeanUK 6 1,616 Jun-29-2020, 06:04 PM
Last Post: HarleyQuin
  scraping multiple pages from table bandar 1 169 Jun-27-2020, 10:43 PM
Last Post: Larz60+
  error zomato scraping data syxzetenz 3 1,046 Jun-23-2020, 08:53 PM
Last Post: Gribouillis
  Scraping Data issues TrexKikBut 0 740 Jun-02-2020, 05:49 PM
Last Post: TrexKikBut
  Python Request's Proxies not working. Fudster 1 598 May-01-2020, 06:42 AM
Last Post: buran
  POST request with form data issue web scraping hoff1022 0 382 Apr-12-2020, 03:22 PM
Last Post: hoff1022
  Flask SelectField using Ajax noah_f 0 201 Mar-23-2020, 04:03 PM
Last Post: noah_f
  Web Scraping, Merging two lists and getting data from various dates? AgileAVS 0 272 Feb-07-2020, 04:05 PM
Last Post: AgileAVS
  Table data with BeatifulSoup gerry84 11 1,045 Oct-23-2019, 10:09 AM
Last Post: Larz60+

Forum Jump:


Users browsing this thread: 1 Guest(s)