|
Search Results
|
| Post |
Author |
Forum |
Replies |
Views |
Posted
[asc]
|
| |
|
Thread: Retrieve images base64 encoded MongoDB and Flask
Post: RE: Retrieve images base64 encoded MongoDB and Fla...
| Actually I want if anyone access file system, he can't be find an Image easily and so stored images as binary in db collections. |
|
Nuwan16 |
Web Scraping & Web Development |
2 |
3,308 |
Oct-13-2020, 06:25 PM |
| |
|
Thread: Retrieve images base64 encoded MongoDB and Flask
Post: Retrieve images base64 encoded MongoDB and Flask
| I'm developing simple web application using flask , scrapy and mongodb. In my case crawl webpages and store some data in mongodb collection and represent using flask. I stored some images in mongodb a... |
|
Nuwan16 |
Web Scraping & Web Development |
2 |
3,308 |
Oct-13-2020, 08:36 AM |
| |
|
Thread: Selenium Screenshots store in Database
Post: RE: Selenium Screenshots store in Database
| Using mongodb with mongoengine. |
|
Nuwan16 |
Web Scraping & Web Development |
3 |
2,014 |
Oct-05-2020, 02:55 PM |
| |
|
Thread: Selenium Screenshots store in Database
Post: Selenium Screenshots store in Database
| When taking web screenshots using selenium web driver, it saved PNG screenshots correctly. So, is there a way to stored these PNG screenshots in database directly? currently I used below code
options... |
|
Nuwan16 |
Web Scraping & Web Development |
3 |
2,014 |
Oct-04-2020, 07:58 PM |
| |
|
Thread: Binary data to Image convert
Post: Binary data to Image convert
| In my case I stored binary data inside user collection as user_pic using mongoengine. It looks like this
"user_pic": {
"$binary": {
"base64": "/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDABQO... |
|
Nuwan16 |
General Coding Help |
1 |
5,683 |
Aug-23-2020, 08:38 PM |
| |
|
Thread: Store Screenshot Selenium + MongoDB
Post: RE: Store Screenshot Selenium + MongoDB
| Is it possible to store and retrieve large size images in mongodb? If we can, which is the data type should give for image file? |
|
Nuwan16 |
Web Scraping & Web Development |
9 |
3,641 |
Aug-17-2020, 09:25 PM |
| |
|
Thread: Store Screenshot Selenium + MongoDB
Post: RE: Store Screenshot Selenium + MongoDB
| actually I'm developing a scrapy spider and I need take screenshot of each crawled url with other data. Then It should retrieve all scraped data with screenshot when search by title. something like li... |
|
Nuwan16 |
Web Scraping & Web Development |
9 |
3,641 |
Aug-16-2020, 09:33 AM |
| |
|
Thread: Store Screenshot Selenium + MongoDB
Post: RE: Store Screenshot Selenium + MongoDB
| actually I need do that with scrapy. In my program when scrapy spider crawl webpage and store some data like title,description,url... of webpage in mongodb. So I need to when spider crawl a web page, ... |
|
Nuwan16 |
Web Scraping & Web Development |
9 |
3,641 |
Aug-16-2020, 06:54 AM |
| |
|
Thread: Store Screenshot Selenium + MongoDB
Post: Store Screenshot Selenium + MongoDB
| I just need to take screenshot of a webpage and store / retrieve that image in mongodb. Below code I'm using to take screenshot of webpage and How I store it in a mongodb collection? **huh**
from sel... |
|
Nuwan16 |
Web Scraping & Web Development |
9 |
3,641 |
Aug-15-2020, 09:44 PM |
| |
|
Thread: Extract json-ld schema markup data and store in MongoDB
Post: Extract json-ld schema markup data and store in Mo...
| I'am creating a spider to crawl webpage' json-ld schema markup and store data in mongodb. actually I want to scrape json-ld schema markup and extract the data type("@type" : "_____") from schema marku... |
|
Nuwan16 |
Web Scraping & Web Development |
0 |
2,463 |
Apr-05-2020, 04:06 PM |
| |
|
Thread: TypeError list indices must be integers or slices not str
Post: RE: TypeError list indices must be integers or sli...
| Thank you Buran **smile** **dance** now It's working |
|
Nuwan16 |
Web Scraping & Web Development |
4 |
3,504 |
Apr-04-2020, 09:15 AM |
| |
|
Thread: TypeError list indices must be integers or slices not str
Post: RE: TypeError list indices must be integers or sli...
| Error:2020-04-04 11:38:16 [scrapy.utils.log] INFO: Scrapy 2.0.1 started (bot: applespider)
2020-04-04 11:38:16 [scrapy.utils.log] INFO: Versions: lxml 4.5.0.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1... |
|
Nuwan16 |
Web Scraping & Web Development |
4 |
3,504 |
Apr-04-2020, 06:08 AM |
| |
|
Thread: TypeError list indices must be integers or slices not str
Post: TypeError list indices must be integers or slices ...
| when i try this
import scrapy
from extruct.w3cmicrodata import MicrodataExtractor
class AppleSpider(scrapy.Spider):
name = "apple"
allowed_domains = ["apple.com"]
start_urls = (
... |
|
Nuwan16 |
Web Scraping & Web Development |
4 |
3,504 |
Apr-03-2020, 10:10 PM |
| |
|
Thread: How to crawl schema markup data type using scrapy?
Post: How to crawl schema markup data type using scrapy?
| I tried to crawl and extract a data type of schema markup using python using this
response.xpath("//script[@type='application/ld+json']/text()").extract()but this extract a whole schema markup code. ... |
|
Nuwan16 |
Web Scraping & Web Development |
1 |
3,086 |
Mar-31-2020, 02:08 PM |
