Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping a CRM
#1
Hi Forum,

I'm trying to scrape some data regularly from the company CRM (Customer Relationship Management) tool that contains all the data entered & produces reports.
I'm using some adapted code & I'm fairly new to the process.
I'm also using MS Excel with VBA to do this.
Firstly, the code below will open the CRM so I thought it must then be able to scrape some content?
I've gone to the source & found the correct class to extract some data, it's where I need to split this into One Item at a time & paste into Excel is where I'm stuck.
Because I've reused some code that maybe won't fit my purpose, I've included the HTML where I've got my class from so someone could maybe help here.



Run Time Error '462'

The remote server machine does not exist or is unavailable


My code is below:

Quote:Sub scrape_crm_open()
' declare the variables
Dim ieObj As InternetExplorer
Dim htmlEle As IHTMLElement
Dim i As Integer

' initialize i to one
i = 1

' create and get access to an instance of IE
Set ieObj = New InternetExplorer
ieObj.Visible = True
ieObj.navigate "http://our_co_crm/main.aspx#"

' give the webpage some time to load all content
Application.Wait Now + TimeValue("00:00:05")

' loop through all the rows in the table
For Each htmlEle In ieObj.Document.getElementsByClassName("ms-crm-List_Data")(0).getElementsByTagName("tr") 'Error is here
With ActiveSheet
.Range("A" & i).Value = htmlEle.Children(0).textContent
.Range("B" & i).Value = htmlEle.Children(1).textContent
.Range("C" & i).Value = htmlEle.Children(2).textContent
.Range("D" & i).Value = htmlEle.Children(3).textContent
.Range("E" & i).Value = htmlEle.Children(4).textContent
End With

i = i + 1
Next htmlEle

End Sub
Reply
#2
This is a Python forum, not VBA forum
Reply
#3
Thanks but I didn't think that a VBA forum would be able to help with this either considering I use Python usually for scraping.
But of course, you're right.

Could anyone answer this question though, Using Python, can I scrape data from an application such as a CRM database, in this case Microsoft Dynamics 365?

Thanks again
Reply
#4
I only have a vague idea of what a CRM database looks like.
If you can give me a url for a site I can experiment on, I'll give it a go.
Reply
#5
I now see that a 'CRM database' is not a database at all, but rather CRM uses a database to offer sales, marketing and support services. So it uses a database and adds tools on the top.

Being inviolved in DBMS design back in the 70's and 80's, I don't like the misuse of the name 'database', even though it will most likely stick.

Initially, I thought there was a new DBMS on the block.
Reply
#6
CRM requires a database, but CRM is not a database itself.
It's just a nice wrapper around a database for customer relationship management.

The problem with CRM is, there are a zillion of them and all are working different.
They way he tries to acquire Information is via http. Mostly all CRMs are Web applications and
many of them do have an API. This should be the first approach, to use the API.

The second approach is the use of web scraping, which has its issues.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020