Accessing Web Page's Source Code; Problem
Hey all! So I'm trying to access a few pages from a website. Staples.ca to be more precise. sending information into the search property of the site and trying to grab the source code of the page of results. It appears, however that the webpage that is being returned is some SSL page, and I can't seem to autoRedirect around it. The following is my code
Code:
private static bool ValidateRemoteCertificate(
object sender,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors policyErrors
)
{
if (Convert.ToBoolean(ConfigurationManager.AppSettings["IgnoreSslErrors"]))
{
// allow any old dodgy certificate...
return true;
}
else
{
return policyErrors == SslPolicyErrors.None;
}
}
private string getPageSource(string URL)
{
try
{
HttpWebRequest myWebRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
myWebRequest.AllowAutoRedirect = true;
ServicePointManager.ServerCertificateValidationCallback += new RemoteCertificateValidationCallback(ValidateRemoteCertificate);
myWebRequest.CookieContainer = new CookieContainer();
myWebRequest.Method = "GET";
HttpWebResponse myWebResponse = (HttpWebResponse)myWebRequest.GetResponse();
StreamReader myWebSource = new StreamReader(myWebResponse.GetResponseStream());
string myPageSource = string.Empty;
myPageSource = myWebSource.ReadToEnd();
//myWebResponse.Close();
return myPageSource;
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
return null;
}
}
Instead of returning the source code of the result page, I get this Source:
Code:
<html>
<head><title>Staples</title><meta http-equiv="Refresh" content="0; url=http://www.staples.ca/ENG/Catalog/cat_i_results.asp?txtSearch=dr210cl%2Dbk"></head>
<body bgcolor="#FFFFFF"></body>
</html>
Does anyone know something I could do to get passed this SSL page?
PS. I know its an SSL page because this is the AbsoluteUri in myWebResponse:
https://ssl.staples.ca/ENG/Libraries...dr210cl%252Dbk
