CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 2 of 2
  1. #1
    Join Date
    Mar 2012

    Need Webcrawler without RSS

    I need information pulled from some websites, but the websites do not have RSS. I am looking for a method which can "crawl" and retrieve the information which is updated in the websites when they are updated. I have been told that this can be done by HTML parsing. However, I was also told that if the HTML structures of the websites change that the parsers would have to be updated or rewritten. Are there any other methods of retrieving updated information from websites? In case anyone was wondering, the information I need pulled is from daily deal sites to go into a daily deal aggregator.

    Admin - if this is in the wrong section please remove the posting, I apologize; I'm new to the forum.

  2. #2
    Join Date
    Jan 2006

    Re: Need Webcrawler without RSS

    Have you contacted the site owners to ask if they are willing to provide a suitable RSS feed? After all, this could be to their advantage, so if it makes life simpler for you, why not?
    C + C++ Compiler: MinGW port of GCC
    Build + Version Control System: SCons + Bazaar

    Look up a C/C++ Reference and learn How To Ask Questions The Smart Way
    Kindly rate my posts if you found them useful

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Windows Mobile Development Center

Click Here to Expand Forum to Full Width

On-Demand Webinars (sponsored)

We have made updates to our Privacy Policy to reflect the implementation of the General Data Protection Regulation.