Listing files at a URL

Does anyone know how (or if it is possible) to get a directory listing for a given URL?

I thought I had it working using HttpWebRequest and specifying the URL and the parsing the HTML directory listing (yuck) that it returned.

However, if the directory contains a file called say "default.htm", then the HTML contents of that file are returned instead of the HTML directory listing.

Is there a smarter way to aceive my goal?

Note: It's read-only, so I don't want to use FTP with a User Id and Password.

If I look over HTTP protocol, I see no way how to do it. It is possible only if server is configured to do this.

boudino, I'm tending to agree with you. I suspect it would be a security issue if you could do this by default.

Then how do search engines work?

By following links and parsing HMTL documents returned from server.

thus creating a directory listing