Python Forum

Python Forum > Python Coding > Web Scraping & Web Development > Here we go again another nightmare
Full Version: Here we go again another nightmare
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2

Blue Dog

Nov-10-2016, 12:18 AM
I am trying to in stall  scapy in python 2.7.12,I am running windows 7. I downloaded it and put it in the scrips folder. cd to C:\Python27\Scripts then I used
pip install scapy.

No luck the error are many see:
Error:
Exception:
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "c:\python27\lib\site-packages\pip\commands\install.py", line 335, in run

    wb.build(autobuilding=True)
  File "c:\python27\lib\site-packages\pip\wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "c:\python27\lib\site-packages\pip\req\req_set.py", line 380, in prepare_
files
    ignore_dependencies=self.ignore_dependencies))
  File "c:\python27\lib\site-packages\pip\req\req_set.py", line 554, in _prepare
_file
    require_hashes
  File "c:\python27\lib\site-packages\pip\req\req_install.py", line 276, in popu
late_link
    self.link = finder.find_requirement(self, upgrade)
  File "c:\python27\lib\site-packages\pip\index.py", line 465, in find_requireme
nt
    all_candidates = self.find_all_candidates(req.name)
  File "c:\python27\lib\site-packages\pip\index.py", line 423, in find_all_candi
dates
    for page in self._get_pages(url_locations, project_name):
  File "c:\python27\lib\site-packages\pip\index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "c:\python27\lib\site-packages\pip\index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "c:\python27\lib\site-packages\pip\index.py", line 811, in get_page
    inst = cls(resp.content, resp.url, resp.headers)
  File "c:\python27\lib\site-packages\pip\index.py", line 731, in __init__
    namespaceHTMLElements=False,
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\html5parser.py", line
 33, in parse
    tb = treebuilders.getTreeBuilder(treebuilder)
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\treebuilders\__init__
.py", line 73, in getTreeBuilder
    return etree.getETreeModule(implementation, **kwargs).TreeBuilder
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\_utils.py", line 104,
 in moduleFactory
    objs = factory(baseModule, *args, **kwargs)
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\treebuilders\etree.py
", line 19, in getETreeBuilder
    ElementTreeCommentType = ElementTree.Comment("asd").tag
AttributeError: 'module' object has no attribute 'Comment'

C:\Python27\Scripts>
So I am right back to where I was over a week ago. what is the problem with trying to install things into python, every thing is a fight.
I get the praw thing when I try to install praw same error.

If anyone can help that would be nice.
thank you Wall
Never mind I cot it installed. if any one need help with installing scapy, let me know i will show how to do it.

metulburr

Nov-10-2016, 12:34 AM
I think for you i would suggest teamviewer
It allows temporary access to another user's computer for X time via remote control. You can view everything the person is doing in real time. All you would have to do is PM the mod/admin the login ID and password and then get on the same page in chat for the same time.

metulburr

Nov-10-2016, 12:35 AM
(Nov-10-2016, 12:18 AM)Blue Dog Wrote: [ -> ]Never mind I cot it installed. if any one need help with installing scapy, let me know i will show how to do it.
care to elaborate the process or what you had incorrect?

snippsat

Nov-10-2016, 12:57 AM
scapy is a more difficult case for Windows,
this info do you get if you search for it.

You need dcap,pcap,scapy and wincap.
This give you binaries and also install WinPcap.

For Python 3 there is a fork that dos not require dnet nor pypcap, no C compilation required.
scapy for python3 (aka scapy3k)

Blue Dog

Nov-10-2016, 04:47 AM
Your are right, when I try to run any scrapy script I get all kinds of error. I have tryto install about 10 thing to day and none of them would install.

snippsat

Nov-10-2016, 09:14 AM
(Nov-10-2016, 04:47 AM)Blue Dog Wrote: [ -> ]I have tryto install about 10 thing to day and none of them would install.
Did you follow what i post?
That do work,i have done that installation on several pc.

I tested this on VirtualBox,so from a clean win10-install with Python 2.7.12.
So you run:
Output:
dnet-1.12.win32-py2.7.exe, pcap-1.1.win32-py2.7.exe, scapy-2.2.0.win32.exe, WinPcap_4_1_3.exe
Then test in interactive shell.
*** Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)] on win32. ***
>>> from scapy.all import *

>>> a = IP(dst="python-forum.io")
>>> a
<IP  dst=Net('python-forum.io') |>
>>> [p for p in a]
[<IP  dst=198.199.80.192 |>]
>>> b = IP(ttl=[1,2,(5,9)])
>>> b
<IP  ttl=[1, 2, (5, 9)] |>
>>> [p for p in b]
[<IP  ttl=1 |>,
 <IP  ttl=2 |>,
 <IP  ttl=5 |>,
 <IP  ttl=6 |>,
 <IP  ttl=7 |>,
 <IP  ttl=8 |>,
 <IP  ttl=9 |>]

>>> c  = TCP(dport=[80,443])
>>> [p for p in a/c]
[<IP  frag=0 proto=tcp dst=198.199.80.192 |<TCP  dport=http |>>,
 <IP  frag=0 proto=tcp dst=198.199.80.192 |<TCP  dport=https |>>]

Blue Dog

Nov-12-2016, 11:34 PM
yes , I tryed that, did not work, this is what I got:


Error:
Type "copyright", "credits" or "license()" for more information.
>>> from scapy.all import *

Traceback (most recent call last):
 File "<pyshell#0>", line 1, in <module>
   from scapy.all import *
 File "C:\Python27\lib\site-packages\scapy\all.py", line 16, in <module>
   from arch import *
 File "C:\Python27\lib\site-packages\scapy\arch\__init__.py", line 79, in <module>
   from windows import *
 File "C:\Python27\lib\site-packages\scapy\arch\windows\__init__.py", line 23, in <module>
   from scapy.arch import pcapdnet
 File "C:\Python27\lib\site-packages\scapy\arch\pcapdnet.py", line 30, in <module>
   import pcapy as pcap
ImportError: No module named pcapy
>>>
I can't get scapy to work at even know it should be installed. I am going to wait on that for now.

I am trying to install urllib
and I get this error:


Error:
c:\Python27\Scripts>pip install urllib
['attrs==16.2.0', 'beautifulsoup4==4.5.0', 'cffi==1.8.3', 'constantly==15.1.0',
'cryptography==1.5.3', 'cssselect==1.0.0', 'dnet==1.12', 'enum34==1.1.6', 'get==
0.0.0', 'idna==2.1', 'incremental==16.10.1', 'ipaddress==1.0.17', 'lxml==3.6.4',
 'parsel==1.0.3', 'pip==9.0.0', 'post==0.0.0', 'public==0.0.0', 'pyasn1-modules=
=0.0.8', 'pyasn1==0.1.9', 'pycparser==2.17', 'pydispatcher==2.0.5', 'pyopenssl==
16.2.0', 'query-string==0.0.0', 'queuelib==1.4.2', 'request==0.0.0', 'requests==
2.11.1', 'scapy==2.3.1', 'scrapy==1.2.1', 'service-identity==16.0.0', 'setupfile
s==0.0.13', 'setuptools==20.10.1', 'six==1.10.0', 'twisted==16.5.0', 'w3lib==1.1
5.0', 'wheel==0.29.0', 'zope.interface==4.3.2']
Collecting urllib
  Could not find a version that satisfies the requirement urllib (from versions:
 )
No matching distribution found for urllib
You are using pip version 9.0.0, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' comm
and.
c:\Python27\Scripts>


Why is it so hard to install anything into python. Not one thing I have tryed in the past week would install.
I hope someone know what is going on and can help.
thank you.

I also need to insatll mechanize but I get the same error, see:



Error:
c:\Python27\Scripts>pip install mechanize
['attrs==16.2.0', 'beautifulsoup4==4.5.0', 'cffi==1.8.3', 'constantly==15.1.0',
'cryptography==1.5.3', 'cssselect==1.0.0', 'dnet==1.12', 'enum34==1.1.6', 'get==
0.0.0', 'idna==2.1', 'incremental==16.10.1', 'ipaddress==1.0.17', 'lxml==3.6.4',
 'parsel==1.0.3', 'pip==9.0.0', 'post==0.0.0', 'public==0.0.0', 'pyasn1-modules=
=0.0.8', 'pyasn1==0.1.9', 'pycparser==2.17', 'pydispatcher==2.0.5', 'pyopenssl==
16.2.0', 'query-string==0.0.0', 'queuelib==1.4.2', 'request==0.0.0', 'requests==
2.11.1', 'scapy==2.3.1', 'scrapy==1.2.1', 'service-identity==16.0.0', 'setupfile
s==0.0.13', 'setuptools==20.10.1', 'six==1.10.0', 'twisted==16.5.0', 'w3lib==1.1
5.0', 'wheel==0.29.0', 'zope.interface==4.3.2']
Collecting mechanize
Exception:
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "c:\python27\lib\site-packages\pip\commands\install.py", line 335, in run

    wb.build(autobuilding=True)
  File "c:\python27\lib\site-packages\pip\wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "c:\python27\lib\site-packages\pip\req\req_set.py", line 380, in prepare_
files
    ignore_dependencies=self.ignore_dependencies))
  File "c:\python27\lib\site-packages\pip\req\req_set.py", line 554, in _prepare
_file
    require_hashes
  File "c:\python27\lib\site-packages\pip\req\req_install.py", line 276, in popu
late_link
    self.link = finder.find_requirement(self, upgrade)
  File "c:\python27\lib\site-packages\pip\index.py", line 465, in find_requireme
nt
    all_candidates = self.find_all_candidates(req.name)
  File "c:\python27\lib\site-packages\pip\index.py", line 423, in find_all_candi
dates
    for page in self._get_pages(url_locations, project_name):
  File "c:\python27\lib\site-packages\pip\index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "c:\python27\lib\site-packages\pip\index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "c:\python27\lib\site-packages\pip\index.py", line 811, in get_page
    inst = cls(resp.content, resp.url, resp.headers)
  File "c:\python27\lib\site-packages\pip\index.py", line 731, in __init__
    namespaceHTMLElements=False,
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\html5parser.py", line
 33, in parse
    tb = treebuilders.getTreeBuilder(treebuilder)
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\treebuilders\__init__
.py", line 73, in getTreeBuilder
    return etree.getETreeModule(implementation, **kwargs).TreeBuilder
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\_utils.py", line 104,
 in moduleFactory
    objs = factory(baseModule, *args, **kwargs)
  File "c:\python27\lib\site-packages\pip\_vendor\html5lib\treebuilders\etree.py
", line 19, in getETreeBuilder
    ElementTreeCommentType = ElementTree.Comment("asd").tag
AttributeError: 'module' object has no attribute 'Comment'
You are using pip version 9.0.0, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' comm
and.
c:\Python27\Scripts>


Can anybody tell me why, Nobody else is having this problem, very thing I have try to install will not. Wall

sparkz_alot

Nov-13-2016, 12:20 AM
If I'm not mistaken, urllib (as well as urllib2) is one of Python's standard librarys and should already be installed

snippsat

Nov-13-2016, 12:23 AM
Quote:yes , I tryed that, did not work, this is what I got:

You have not follow what i post.
pcapy is not installed:
Output:
pcap-1.1.win32-py2.7.exe
Quote:I am trying to install urllib
urllib is part of standard library,so no need to install.

Quote:Why is it so hard to install anything into python. Not one thing I have tryed in the past week would install. 
When you get a little practice it will work well.
scapy is an old low level network tool designed for Linux,
so it's harder to install than most stuff.

Blue Dog

Nov-13-2016, 01:09 AM
That may be so but I can run it, i get error not found
Pages: 1 2
Python Forum > Python Coding > Web Scraping & Web Development > Here we go again another nightmare