Need Webcrawler without RSS
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 2 of 2

Thread: Need Webcrawler without RSS

  1. #1
    Join Date
    Mar 2012
    Posts
    1

    Need Webcrawler without RSS

    I need information pulled from some websites, but the websites do not have RSS. I am looking for a method which can "crawl" and retrieve the information which is updated in the websites when they are updated. I have been told that this can be done by HTML parsing. However, I was also told that if the HTML structures of the websites change that the parsers would have to be updated or rewritten. Are there any other methods of retrieving updated information from websites? In case anyone was wondering, the information I need pulled is from daily deal sites to go into a daily deal aggregator.

    Admin - if this is in the wrong section please remove the posting, I apologize; I'm new to the forum.

  2. #2
    Join Date
    Jan 2006
    Location
    Singapore
    Posts
    6,250

    Re: Need Webcrawler without RSS

    Have you contacted the site owners to ask if they are willing to provide a suitable RSS feed? After all, this could be to their advantage, so if it makes life simpler for you, why not?
    C + C++ Compiler: MinGW port of GCC
    Build + Version Control System: SCons + Bazaar

    Look up a C/C++ Reference and learn How To Ask Questions The Smart Way
    Kindly rate my posts if you found them useful

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Azure Activities Information Page

Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center