scraping from a website that hides source code

[PIWI_Protein]

Hello Pythoners,

I want to make python get some data from this website: https://tritrypdb.org/


The idea is to then implement that into excel, so I can have it search for the same piece of information for hundreds of genes, and put it in my excel sheet.


It is now searching for a gene on the website (I just have it add the name to the URL), and it reads the source code.

import urllib.request
import re
#this is just an example for the gene ID. It will be a list of gene IDs.
geneID="Tb927.7.2390"
#open the webpage, directly going to the right gene ID
page=urllib.request.urlopen("https://tritrypdb.org/tritrypdb/app/record/gene/"+geneID)
#read entire source code
scode=page.read()

The plan was now to search the source code for the information needed, and return that. But It seems that the source code just doesn't contain any of the actual text which is there with the normal graphic view of the browser. Instead there are huge blank spaces.


Is this webpage somehow hiding that information? and is there a way to still get the information out of there?

Thank you for your help people!

[Larz60+]

You need to learn a bit more about scraping.
There's a good two part tutorial on this forum.
see:
web scraping part 1
https://python-forum.io/Thread-Web-scraping-part-2