Any errors that you get with the code I supplied should only be for uninstalled packages which you can easily get with pip.
If you just don't want to work with selenium, there is perhaps an alternative.
If you are logged into Linkedin, this should work.
What I suggest is to alter the URL to proceed to the next page.
If you examine the URL that you provide, you will notice that it ends with: &pageNum=0
You may be able to modify the page number (again if logged in to Linkedin) and fetch the next page with requests.
untested code
Again this may possibly work, but the only sure way is to render the JavaScript which beautifulsoup is incapable of doing.
If you just don't want to work with selenium, there is perhaps an alternative.
If you are logged into Linkedin, this should work.
What I suggest is to alter the URL to proceed to the next page.
If you examine the URL that you provide, you will notice that it ends with: &pageNum=0
You may be able to modify the page number (again if logged in to Linkedin) and fetch the next page with requests.
untested code
def get_url(pageno):
return f"https://www.linkedin.com/jobs/search?keywords=Data%20Science&location=United%20Kingdom&redirect=false&position=1&pageNum={pageno}"You may also have to change position number as I don't know if that refers to first item to show, or item number from entire list if the latter, then this would change by incrementing by 25.Again this may possibly work, but the only sure way is to render the JavaScript which beautifulsoup is incapable of doing.