|
-
January 29th, 2006, 08:24 PM
#1
Scraping a website
I wrote a program about a year ago that scrapes a webpage and dumps the data into a database. The website made some major changes, so I have had to rewrite some of my program. All the search options used to be passed as parameters in the url; now all of the parameters have been removed and the only thing passed is: "?Page=SEARCHRESULTS". I have been able to overcome all the changes except one. One of the parameters I used to pass was one that told how many entries to show. Now it just defaults to 25. This is ok as long as I can figure out a way to navigate through the pages. "javascript:JumpToResultsPage(3);" is how they do it on the website. I am currently using CInternetSession to pull data off the internet, I thought I could do this with internetSession.OpenURL("javascript:JumpToResultsPage(3);");. Apparently you can only pass http, ftp, etc to it.
How can I go about navigating through the pages?
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|