web scraping HTML - :(

[snippsat]

And also move with open into the loop.

import requests

domains = ['https://webdesigner.dk', 'https://www.dk4.dk/item/4128-persondatapolitik']
for domain in domains:
    print(domain)
    response = requests.get(domain)
    print(f"Response data length: {len(response.text)}")
    # Remove # from line below to see source
    # print(response.text)

    with open('source.html', 'a', encoding='utf-8') as f:
        f.write(response.text.strip())

[Kingoman]

Thank you for keep trying to help me out here, but it is not working.

I have posted a picture of my screen.

https://imgur.com/hg0P9fq

The 10696 lines is still only from the last mentioned website.

[ndc85430]

Your writing isn't in the loop, is it?

[Kingoman]

Your writing isn't in the loop, is it?

I'm sorry but I don't know what you mean by this.


Please see the link with the image I posted in my previous post

[supuflounder]

Here's a link to some scraping software I did a couple weeks ago:
https://python-forum.io/newreply.php?tid...yto=139401

[ndc85430]

The call to write is done once, after the loop. Is that really what you intended?

[Kingoman]

Thus is the First thing I try to setup as code, so I dont know

The intention is:

Put in my list of addresses, have it combine all the public HTML source code into one long text That I can view.

- thats it, it is just instead of me going through each website, right click and view source

[Kingoman]

I don’t have any idea How to get it to loop, it was an idea for me to try this out, but yeah, doesn’t seem to work out

[ndc85430]

Do you understand that indentation tells you what's part of a block, like a for loop?

[Kingoman]

No.

I just wanted to try this out instead of manually doing 100 websites, but It doesn’t
Seem so simple to do, after all :)

T