PHP to Scrape large amount of data but database question....
Extracting a huge amount of data from the likes of rightmove.co.uk and zoopla.co.uk. We will need to use this information to show changes on a membership website as well as inside each of the websites (Rightmove, zoopla etc)in questions (shown in attached screenshots)
What would the best way to store this information so it can retrieved quickly and easily please?
I do not think a csv file will do the trick as for example, if we start tracking the number of images, having the URLS will not help and we will need to store the actual images too in our database - right?
Any guidance and tips you can give would be deeply appreciated :)
http://forums.codeguru.com/images/ie.../2020/06/1.jpg
http://forums.codeguru.com/images/ie.../2020/06/1.jpg
Re: PHP to Scrape large amount of data but database question....
Do these sites offer an api to help you get the data directly?
I ask because scraping is very fragile - one tiny ui change by the site developers and your scraping can break. It can turn into a maintenance nightmare for sites that get updated frequently.
Re: PHP to Scrape large amount of data but database question....
The easiest way to scrap data from webpages is to use regular expressions. For new users best&quick way - use some program services like a https://mydataprovider.com/solutions/web-scraping/.