CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 2 of 2

Thread: PHP to Scrape large amount of data but database question....

  1. #1
    Join Date
    Jun 2020
    Posts
    1

    Question PHP to Scrape large amount of data but database question....

    Extracting a huge amount of data from the likes of rightmove.co.uk and zoopla.co.uk. We will need to use this information to show changes on a membership website as well as inside each of the websites (Rightmove, zoopla etc)in questions (shown in attached screenshots)

    What would the best way to store this information so it can retrieved quickly and easily please?

    I do not think a csv file will do the trick as for example, if we start tracking the number of images, having the URLS will not help and we will need to store the actual images too in our database - right?

    Any guidance and tips you can give would be deeply appreciated



    Last edited by 2kaud; June 7th, 2020 at 08:07 AM.

  2. #2
    Arjay's Avatar
    Arjay is offline Moderator / EX MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    13,360

    Re: PHP to Scrape large amount of data but database question....

    Do these sites offer an api to help you get the data directly?

    I ask because scraping is very fragile - one tiny ui change by the site developers and your scraping can break. It can turn into a maintenance nightmare for sites that get updated frequently.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width




On-Demand Webinars (sponsored)