socket recv problems

[kakalake]

Hello!
I am writing an application that should be able to receive html-pages. But i have some problems to receive the sides. IT seems if i could successfully connect to the host and open a stream socket that is listening on port 80. when i send the command "GET /" to the server it seems if it arrives successfully but when i try to receive the incomming data i get the errorcode WSAECONNRESET.
I don´t know how to find the error, because i don´t know if the preceding connection was successfully. How can i figured it out. Is there a possibility. I started my firewall that tells me that my application wants to connect to the server. But why i am not abel to recv the data. Here is my piece of code....

int Socket::recv_all( std::string& s )
{
std::string::iterator at = s.begin();
int received; //Bytes send

fd_set master;

FD_ZERO( &master ); //Set fd_set to NULL
FD_SET( fdscp, &master ); //Adds discriptor to fd_set

char buffer[255];

int res = select( fdscp+1, &master, NULL, NULL, NULL );

if ( res == SOCKET_ERROR )
throw SocketException( "Select: SOCKET_ERROR" );

if ( FD_ISSET( fdscp, &master ) )
{
received = recv( fdscp, buffer, 255/*at, DATASIZE*/, 0 );

if ( !received )
throw SocketException( "Connection was closed" );

if ( received == SOCKET_ERROR )
throw SocketException( "SocketError in recv_all" );

if ( received == WSAEMSGSIZE )
throw SocketException( "Message larger as buffer" );

std::cout << buffer ;
}

return received;
}


Thanks in advance

[kakalake]

I tried something but it still doesn´t work. When i send the string "GET / HTTP/1.0" to the server and i try to read the data then my programm stops without any error-msg. My programm wait at the command select or at the command recv. It seems so if my programm tries to receive data but it will not get it.

[j0nas]

I don't know what the error is with the short description and code you posted... But I wrote something like that a few months ago. I posted it here at CodeGuru for another guy. Here is the thread:
http://www.codeguru.com/forum/showthread.php?s=&threadid=200185&highlight=wwwgrab
Look a bit down in the thread for wwwgrab.zip. It's plain C and blocking sockets. Good luck.

[Mick]

The error means:


The virtual circuit was reset by the remote side executing a hard or abortive close. The application should close the socket as it is no longer usable. On a UPD-datagram socket this error would indicate that a previous send operation resulted in an ICMP "Port Unreachable" message.

[kakalake]

Thanks fpr the answers but they don´t lead to solve my problem. Mick_2002 i also read this in the documentation but what can be the reason for this?
Thanks for the www-grabber but i look at this and i couldn´t find any differences in your and my code concering the allocation of the socket.
Here is my code where i try to get the socket.

//allocation of the socket
if ( (hten = gethostbyname( host )) == NULL )
throw SocketException( "Client: gethostbyname" );

sock_struct.sin_addr = *((in_addr *)hten->h_addr);//Set to my IP
sock_struct.sin_family = PROTOCOL_FAMILY; //Set protocol familiy ( must be AF_INET )
sock_struct.sin_port = htons(port); //Set port
memset( &(sock_struct.sin_zero), '\0', 8 );

if ( (fdscp = socket( PROTOCOL_FAMILY, socketTyp, 0 )) == INVALID_SOCKET )
throw SocketException( "socket" );

//Send method
while ( total )
{
int n = send( fdscp, "GET / HTTP/1.0\n\n", 16, 0 );

if ( n == SOCKET_ERROR )
throw SocketException( "SocketError in send_all" );

if ( n == WSAEMSGSIZE )
throw SocketException( "Message to long to send" );

asend += n;
total -= n;
}


My problem is that my programm stops while receiving data. why ?????

[kakalake]

I don´t know why but my programm works fine for another URLs. I tried to receive data from www.heise.de but the server will not return any data after i sent GET / HTTP/1.0\n\n. But then i tried receive data from www.yahoo.de and it works fine. Could it be that www.heise.de doesn´t know the command GET i sent?
I tried to connect to www.heise.de with telnet and receive data with the GET command. What is wrong with my code?

j0nas i tried your application an it doesn´t work, too!

Thanks...

[Surgeonde]

There is a demo project
with tear.c
in the MSDN.
I does not have any problem with the
www.heise.de.

download it and have a look at the code
http://search.microsoft.com/gomsuri.asp?n=1&c=rp_Results&siteid=us/dev&target=http://msdn.microsoft.com/library/en-us/vcsample/html/_sample_mfc_tear.asp

[kakalake]

Haha!
You are using MFC and i don´t want to use it.

[Surgeonde]

It did not want to urge zou to use MFC, I just wanted to encourage you to learn from working things.

This MFC sample has a
QueryInfo(HTTP_QUERY_RAW_HEADERS_CRLF, strNewLocation) and this is the content
of the string for both sites
Maybe this rings you a bell.

This is www.yahoo.de
with one redirection.
{"HTTP/1.1 302 Found
Date: Sat, 12 Oct 2002 17:34:38 GMT
Location: http://de.yahoo.com/
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
"}
{"HTTP/1.1 302 Found
Date: Sat, 12 Oct 2002 17:38:48 GMT
Location: http://de.yahoo.com/
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1

"}

This is the www.heise.de
{"HTTP/1.1 200 OK
Date: Sat, 12 Oct 2002 17:36:32 GMT
Server: Apache/1.3.26
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
"}

[j0nas]

I got it to work now... Both of us use HTTP 1.0. They (www.heise.de) must have configured their HTTP server to only accept HTTP 1.1.

I think the "HOST:" header tag in HTTP1.1 made it work. I monitored a Internet Explorer 6.0 session against the site (with a free network analyzing tool) and took almost every thing from the HTTP1.1 header IE sent and put it into my wwwgrab utility.

This is what IE6 sent:

GET / HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword, */*
Accept-Language: en-us,sv;q=0.5
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; T312461)
Host: www.heise.de
Connection: Keep-Alive


I removed all three "Accept" lines (it will of course work with them too). I also change the "User-Agent" value to "wwwgrab".

Let me know if you need my updated version of wwwgrab. I highly recommend you to use some network monitor tool. It's very handy in cases like this.

[kakalake]

can you recommend me some of your network tools

[kakalake]

Could you send me the new version of your grabber..

[j0nas]

Originally posted by kakalake
can you recommend me some of your network tools

Yes. I installed WinPcap (a free packet capture driver) from http://windump.polito.it. This is just the driver. You then need a display and analyzing tool. Windump.exe is a console program (the Unix version is called TcpDump). You can download it from the same place as WinPcap.

I didn't use windump. I instead found a free GUI app that is based on WinPcap. Here is link: http://www.ethereal.com/distribution/win32

[j0nas]

Originally posted by kakalake
Could you send me the new version of your grabber..

Sure, here it is:

[kakalake]

Thanks a lot j0nas. How can i rate you??

[j0nas]

kakalake, you can rate the thread if you like to. It doesn't matter for me, I am just glad to help people out.

Good luck with your program.

PS. I just noticed that since my updated version of wwwgrab use "Connection: Keep-Alive", recv() won't terminate with zero until it times out... Maybe you need to remove the keep-alive feature...