For Beautifulsoup you have only the dummy package
lxml is ok.
You most remember to always active environment
To test use first code in Web-Scraping part-1 this code will test if these 3 libraries work.
I use cmder but command are the same in cmd.
lxml is ok.
You most remember to always active environment
base-dup
.To test use first code in Web-Scraping part-1 this code will test if these 3 libraries work.
import requests from bs4 import BeautifulSoup url = 'http://CNN.com' response = requests.get(url) soup = BeautifulSoup(response.content, 'lxml') print(soup.find('title').text)
Output:CNN International - Breaking News, US News, World News and Video
A quick run i activate environment install and run code using miniconda3.I use cmder but command are the same in cmd.
# Activate base G:\miniconda3\Scripts λ activate.bat # Cd down (base) G:\miniconda3\Scripts λ cd .. # Activate my_env for you base_dup (base) G:\miniconda3 λ conda activate my_env # See that (my_env) look like this # Test pip see that it point to my_env folder (my_env) G:\miniconda3 λ pip -V pip 21.1.2 from G:\miniconda3\envs\my_env\lib\site-packages\pip (python 3.9) # Install (my_env) G:\miniconda3 λ pip install beautifulsoup4 requests lxml Collecting beautifulsoup4 Using cached beautifulsoup4-4.9.3-py3-none-any.whl (115 kB) Requirement already satisfied: requests in g:\miniconda3\envs\my_env\lib\site-packages (2.25.1) Collecting lxml Using cached lxml-4.6.3-cp39-cp39-win_amd64.whl (3.5 MB) Collecting soupsieve>1.2 Using cached soupsieve-2.2.1-py3-none-any.whl (33 kB) Requirement already satisfied: urllib3<1.27,>=1.21.1 in g:\miniconda3\envs\my_env\lib\site-packages (from requests) (1.26.5) Requirement already satisfied: certifi>=2017.4.17 in g:\miniconda3\envs\my_env\lib\site-packages (from requests) (2021.5.30) Requirement already satisfied: chardet<5,>=3.0.2 in g:\miniconda3\envs\my_env\lib\site-packages (from requests) (4.0.0) Requirement already satisfied: idna<3,>=2.5 in g:\miniconda3\envs\my_env\lib\site-packages (from requests) (2.10) Installing collected packages: soupsieve, lxml, beautifulsoup4 Successfully installed beautifulsoup4-4.9.3 lxml-4.6.3 soupsieve-2.2.1 # See code that i run now (my_env) G:\miniconda3 λ type test_parse.py import requests from bs4 import BeautifulSoup url = 'http://CNN.com' response = requests.get(url) soup = BeautifulSoup(response.content, 'lxml') print(soup.find('title').text) # Run code (my_env) G:\miniconda3 λ python test_parse.py CNN International - Breaking News, US News, World News and Video