However, you can gain access to the underlying details by subclassing WebClient.
This allows you to override GetWebResponse, GetWebRequest, handle cookies, and
to get a final url where autoredirection lands the WebClient on a different target
page than it originally requested, along with other features. Decompiled, the
WebClient class consists of 3,237 lines of very complex and carefully engineered
code. Most of this is hidden from the user.
This is a sample WebClientEx class derived from WebClient that implements most of
the extra features developers need.
The easiest way to explain this is to have a look at the code for the class, as the
comments tell all:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Net;
namespace WebUtil
{
/// <summary>
/// A class to extend WebClient with additional features
/// </summary>
public class WebClientEx : WebClient
{
private CookieContainer _cookieContainer;
private string _userAgent;
private int _timeout;
Uri _responseUri;
/// <summary>
/// Gets or sets the timeout.
/// </summary>
/// <value>
/// The timeout.
/// </value>
public int Timeout
{
get { return _timeout; }
set { _timeout = value; }
}
public WebClientEx()
{
this._cookieContainer = new CookieContainer();
this.SetTimeout(60 * 1000);
}
/// <summary>
/// Gets or sets custom user agent.
/// </summary>
/// <value>
/// The user agent.
/// </value>
public string UserAgent
{
get { return _userAgent; }
set { _userAgent = value; }
}
public WebClientEx SetTimeout(int timeout)
{
this.Timeout = timeout;
return this;
}
/// <summary>
/// Gets the final target response URI.
/// </summary>
public Uri ResponseUri
{
get { return _responseUri; }
}
/// <summary>
/// Overrides the <see cref="T:System.Net.WebResponse"/> for the
specified <see cref="T:System.Net.WebRequest"/>.
/// Sets the ResponseUri.
/// </summary>
/// <param name="request">A <see cref="T:System.Net.WebRequest"/>
that is used to obtain the response.</param>
/// <returns>
/// A <see cref="T:System.Net.WebResponse"/> containing the response
for the specified <see cref="T:System.Net.WebRequest"/>.
/// </returns>
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = null;
try
{
response = base.GetWebResponse(request);
_responseUri = response.ResponseUri;
}
catch
{
}
return response;
}
/// <summary>
/// Overrides the <see cref="T:System.Net.WebRequest"/> object for
the specified resource.
/// Adds Gzip/deflat Request Headers.
/// Sets Automatic decompression of compressed responses.
/// Sets PreAuthenticate header.
/// Adds CookieContainer to transmit cookies.
/// Sets custom User-Agent string.
/// Sets Timeout for the Request.
/// </summary>
/// <param name="address">A <see cref="T:System.Uri"/>
that identifies the resource to request.</param>
/// <returns>
/// A new <see cref="T:System.Net.WebRequest"/> object for the specified
resource.
/// </returns>
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");
// add auto- Gzip / deflate handling
((HttpWebRequest)request).AutomaticDecompression = DecompressionMethods.GZip
| DecompressionMethods.Deflate;
// next line avoids double hits to a WWW-Authenticate / Basic source
request.PreAuthenticate = true;
if (this.UserAgent !=null)
(request as HttpWebRequest).UserAgent=this.UserAgent;
if (request.GetType() == typeof(HttpWebRequest))
{
((HttpWebRequest)request).CookieContainer = this._cookieContainer;
((HttpWebRequest)request).UserAgent = this._userAgent;
((HttpWebRequest)request).Timeout = this._timeout;
}
return request;
}
/// <summary>
/// Enables certificate validation.
/// </summary>
/// <param name="validate">if set to <c>true</c> [validate].</param>
/// <returns>instance of the WebClientEx class to use</returns>
public WebClientEx ServerCertificateValidation(bool validate)
{
if (!validate) ServicePointManager.ServerCertificateValidationCallback += delegate(object
sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{ return true; };
return this;
}
}
}
I've constructed a sample ASP.NET project that illustates the use of some of
these features. In this project there are three pages - Default.aspx, Receiver.aspx,
and CheckCookie.aspx.
Default.aspx is where we start:
protected void Button1_Click(object sender, EventArgs e)
{
WebClientEx wc = new WebClientEx();
wc= wc.ServerCertificateValidation(true);
wc.SetTimeout(60000);
wc.UserAgent = "dorky";
wc.Headers.Add("Content-Type","application/x-www-form-urlencoded");
// Apply ASCII Encoding to obtain the string as a byte array.
byte[] byteArray = Encoding.ASCII.GetBytes("test=hello");
var res = wc.UploadData("http://localhost:10000/Receiver.aspx", byteArray);
var cook = wc.ResponseHeaders["Set-cookie"];
System.Diagnostics.Debug.WriteLine("Cookie: " + cook);
Session["info"] = System.Text.Encoding.ASCII.GetString(res);
wc.Dispose();
Response.Redirect("CheckCookie.aspx");
}
The above code creates a WebClientEx instance, sets Server Certificate Validation,
sets a timeout, sets the Content-Type for a form post, encodes a sample name=value
pair to post, and performs the POST via the UploadData method. It then gets the cookie that was set by the Receiver.aspx page,
stores the result that was sent back in Session, and finally redirects to the
CheckCookie.aspx page which does the following:
protected void Page_Load(object sender, EventArgs e)
{
WebClientEx wc = new WebClientEx();
var res = wc.DownloadString("http://localhost:10000/Receiver.aspx");
Label1.Text = (string) Session["info"] + "<BR>" + res + "<br/>Cookie: " + Request.Cookies["test"].Value;
wc.Dispose();
}
CheckCookie creates a new WebClientEx instance and downloads the info from Receiver.aspx
and displays it.
Receiver.aspx has the following logic:
protected void Page_Load(object sender, EventArgs e)
{
if (Request.Cookies["test"] == null)
{
var cook = new HttpCookie("test", "bork! Bork!");
cook.Expires = DateTime.Now.AddDays(1);
Response.Cookies.Add(cook);
var x = Request.Form["test"];
Response.Write("you posted: " + "test=" + x);
}
else
{
Label1.Text = "Your Cookie 'test': " + Request.Cookies["test"] + "<br/>" + (string) Session["info"];
}
}
If the "test" cookie is not present, Receiver.aspx sets a new "test"
cookie with value "bork! Bork!". It also echoes back what the default.aspx
page POSTED in its form post.
The final display on Receiver.aspx after pressing the POST button on default.aspx
looks like the following, all as a "proof of concept".
you posted: test=hello
you posted: test=
Cookie: bork! Bork!
There are other customizations that can be made to WebClient using this technique.
I leave that exercise to the reader. Feel free to modify the code as you see
fit.
You can download the Visual Studio 2010 demo solution here.