FtpWebRequest - downloading big files

[Ka-lolek]

Hello,
I'm using FtpWebRequest to download files from FTP server.
For small files <= 20MB everything is going well, but if the file is 100-200MB, it fails..
it downloads like 50-70MB and then application hangs... after some time it shows popup window with this information:

Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

the setting sor FtpWebRequest look like this:

                    partRequest = FtpWebRequest.Create(ftpUrl) as FtpWebRequest;
                    partRequest.Method = WebRequestMethods.Ftp.DownloadFile;
                    partRequest.Credentials = new NetworkCredential(FtpUsername, FtpPassword);
                    partRequest.UsePassive = true;
                    partRequest.UseBinary = true;
                    partRequest.KeepAlive = false; // close the connection when done
                    partRequest.Timeout = Int32.MaxValue; //this line didn't help :(

                    FtpWebResponse partResponse = partRequest.GetResponse() as FtpWebResponse;
                    Stream partReader = partResponse.GetResponseStream();
                    byte[] buffer = new byte[1024];
                    FileInfo fi = new FileInfo(path);
                    FileStream memStream = fi.Create();
                    while (true)
                    {
                        Application.DoEvents(); 

                        // Try to read the data
                        int bytesRead = partReader.Read(buffer, 0, buffer.Length);
                        if (bytesRead == 0)
                              break;

                        memStream.Write(buffer, 0, bytesRead);
                    }

Do you have any suggestions? What can be a reason why it crashes with bigger files..

Maybe you know some library which is tested and works with large files (~2GB)

Hope somebody know the solution.. i spent 2 days on looking for it in the internet with no result.

Cheers
Ka-

[eclipsed4utoo]

you could try using the WebClient class instead...

WebClient client = new WebClient();
// Starts the download
client.DownloadFile(new Uri("SomeURLToFile"), "SomePlaceOnLocalHardDrive");

[monalin]

You need to use the Async method described in this MSDN document.

http://msdn.microsoft.com/en-us/libr...ebrequest.aspx

You're problem is that it takes too long to download that file, not the actual size of the file. Changing it to the async method with provide much better results and since it should be in another thread your application wont be locking up and you wont need to use Application.DoEvents()

[MadHatter]

ftpwebrequest is the worst ftp class I've ever seen. first, ftp is its own protocol, it has nothing to do with the www. webrequest works well for http since its a stateless protocol, but ftp is another beast all together.

[Ka-lolek]

i was considering using webclient instead of ftpwebrequest, but the problem is that my server doesn't accept active mode so that i can't display progress.. is there any other way to do it using webclient + passive mode?

[MadHatter]

webclient is an http download (not ftp). I guess it all depends on what your requirements are (whether ftp must be used, or whether http is allowed).

I wrote an FTP client library a while back, feel free to use it (or not). http://sanity-free.org/dist/NullFX.Net-binary.zip

I've used it in production to download files around 3 gig in size. it does passive and active (I think, there was a version that didn't do active correct and I can't remember if this one has that fix or not) as well as secure ftp (ftp over ssl/tls). plus the entire source code is there if you feel like adding something (like events for upload / downloading to display progress).

the api for it is pretty intuitive (at least to me the author

FtpClient client = new FtpClient( new IPEndPoint( IPAddress.Loopback, 21 ), new NetworkCredential( "test", "testing@localdomain" ) );
client.Connect();
client.Download("testfile.zip", @"C:\downloads\testfile.zip");

[Ka-lolek]

Thanks Mad for your interest! (and API)
regarding the WebClient.. certainly, it was designed for http, but works fine with FTP too
i used it instead of ftpwebrequest and works fine.. despite the fact that it doesn't show the progress while using passive mode..

And about your API.. which class are you using there to process FTP tasks like downloading etc. Is it also FtpWebRequest?

Cheers!

[MadHatter]

I wrote the FTP client myself from scratch. It was first written for .net 1.1 where no ftp component existed.

[eclipsed4utoo]

here is an MSDN article on the WebClient class and overriding the GetWebRequest method to set the UsePassive property.

http://msdn.microsoft.com/en-us/libr...sschanged.aspx

[Ka-lolek]

i read this article earlier, but as i said previously for me the problem is server which doesn't support active mode..
anyway, thx

[eclipsed4utoo]

i read this article earlier, but as i said previously for me the problem is server which doesn't support active mode..
anyway, thx

then set the WebClient to use passive mode. Will that not work? or is it that you want to show progress of the file download while using passive mode?

[Ka-lolek]

i used it instead of ftpwebrequest and works fine.. despite the fact that it doesn't show the progress while using passive mode..

so.. yes, i would like to show the progress

[MadHatter]

I updated my library to include uploaded and downloaded events that fire when parts of a file are downloaded or uploaded (the event args have a property denoting the number of bytes transferred). you can get the total file size using the client.GetFileSize(file) and use the events to increment/decrement how much has been downloaded/uploaded.

[rliq]

Ha Mad... reacting to user requests? Keep quiet or I think you'll be giving the rest of us a bad name

[MadHatter]

lol, it was 4 lines of code.