dcsimg
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 27

Thread: How to grab data off a website in VB

  1. #1
    Join Date
    Jul 2002
    Posts
    62

    How to grab data off a website in VB

    I would like to know if it is possible to have a visual basic application that can open a web site in the background and find specific text on the page and store it as a variable. Thanks for any help.

  2. #2
    Join Date
    Mar 2002
    Location
    NY
    Posts
    236

    Re: How to grab data off a website in VB

    You can use MS Internet Transfer Control.
    Place Command Button, RichTextBox and Inet Controls on your form.
    And add the following code (just change "Your_URL" to a valid URL)
    Code:
    Private Sub Command1_Click()
    
        Dim strHTML As String
        
        strHTML = Inet1.OpenURL("Your_URL")
        RichTextBox1.Text = strHTML
    End Sub

  3. #3
    Join Date
    Jul 2002
    Posts
    62
    is there any other way other than using the source code? I've heard some talk of a webbrowser control. would that also work?

  4. #4
    Join Date
    Apr 2002
    Location
    Melbourne, Victoria, Australia
    Posts
    1,792
    You have to make your own webbrowser control from the Inet control.

    If you want to have a VB application to do this, then you have to write some source code....unless you want to pay for one of us to write the thing for you!!!!

  5. #5
    Join Date
    Mar 2002
    Location
    NY
    Posts
    236
    Well, both (WEB-browser and Inet) will work. But Inet Was easier to work for me.
    Tomorrow I'll post the code with WEB-browser control - almost the same.

  6. #6
    Join Date
    Jul 2002
    Posts
    62
    Using the Inet control, the open url command only retrieves part of the web page source. Is there a limitation on this control that would cause this?

  7. #7
    Join Date
    Dec 2001
    Posts
    6,332
    I like using winsock. I tried the inet control, and did not like it.

    As far as retrieving the entire page, if the site uses frames, you will need to determine which frame has the data you want, and get only that frame. A site can be scripted however, to disallow this, so you will have to find a way around it if that's the case. I have posted the code to get a Web page with winsock recently, so a search should turn it up (if you're interested).
    Please remember to rate the posts and threads that you find useful.
    How can something be both new and improved at the same time?

  8. #8
    Join Date
    Jul 2002
    Posts
    62
    the page didn't have frames, it just cut off the source code in the middle of a line. very wierd.

  9. #9
    Join Date
    Jul 2002
    Posts
    62
    the site i'm having trouble with is www.***.com and scores.***.com/scores/. It works fine with my web site and other sites that i have tried. anyone have any explanation for this?

  10. #10
    Join Date
    Dec 2001
    Posts
    6,332
    If you are using a textbox to catch the data, you will run out of room. The page contains more than 64k.
    Please remember to rate the posts and threads that you find useful.
    How can something be both new and improved at the same time?

  11. #11
    Join Date
    Jul 2002
    Posts
    62
    i was just storing it in a variable. how much can one string hold and is it possible just to store the source text?

  12. #12
    Join Date
    Dec 2001
    Posts
    6,332
    A string holds a bit over 2 Gigs of characters. When you get a page using winsock or inet, the source html is all you are getting, without images or anything else.

    When I tried the url you gave, it worked fine for me.
    Please remember to rate the posts and threads that you find useful.
    How can something be both new and improved at the same time?

  13. #13
    Join Date
    Jul 2002
    Posts
    62
    any idea why it is not getting the entire source when I try it?

    Here is my syntax

    Private Sub Command2_Click()
    Dim strHTML As String
    strHTML = Inet1.OpenURL("http://scores.***.com/scores/")
    RichTextBox1.Text = strHTML
    End Sub

    The text displayed in the rich text box is cut off in the middle of a line. It probably only displays about 1/10 of the source code, if that much. I've tried it on pages with much longer source code and it worked fine.

  14. #14
    Join Date
    Dec 2001
    Posts
    6,332
    Only two things come to mind:

    1) maybe the inet control has a timeout or something.

    2) maybe try this:

    Private Sub Command2_Click()
    Dim strHTML As String
    RichTextBox1.Text = ""
    strHTML = Inet1.OpenURL("http://scores.***.com/scores/")
    RichTextBox1.SelText = strHTML
    End Sub
    Please remember to rate the posts and threads that you find useful.
    How can something be both new and improved at the same time?

  15. #15
    Join Date
    Jul 2002
    Posts
    62
    the inet control does have a timeout, but it is set at 60 seconds. it returns the partial string within 1-2 seconds. Setting it as the seltext didn't work either. I doesn't really make any sense why this website should be the only one that does this. does anyone have any ideas. I becoming very curious.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width




On-Demand Webinars (sponsored)