ASP.NET: Extending the WebClient Class

System.Net.WebClient is a convenience "wrapper" class around WebRequest and WebResponse that greatly simplifies the basic HTTP operations of GET and POST. However, as a wrapper, it does not expose the underlying details of the Request and Response, which can cause some frustration as developers begin to realize they may need to use HttpWebrequest and HttpWebResponse and write a lot more code than they wanted to.

However, you can gain access to the underlying details by subclassing WebClient. This allows you to override GetWebResponse, GetWebRequest, handle cookies, and to get a final url where autoredirection lands the WebClient on a different target page than it originally requested, along with other features.  Decompiled, the WebClient class consists of 3,237 lines of very complex and carefully engineered code. Most of this is hidden from the user.

This is a sample WebClientEx class derived from WebClient that implements most of the extra features developers need.

The easiest way to explain this is to have a look at the code for the class, as the comments tell all:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Net;


namespace WebUtil
{

    /// <summary>
    /// A class to extend WebClient with additional features
    /// </summary>
    public class WebClientEx : WebClient
    {
         private CookieContainer _cookieContainer;
         private string _userAgent;
        private int _timeout;
    Uri _responseUri;


        /// <summary>
        /// Gets or sets the timeout.
        /// </summary>
        /// <value>
        /// The timeout.
        /// </value>
        public int Timeout
        {
            get { return _timeout; }
            set { _timeout = value; }
        }

        public WebClientEx()
        {
             this._cookieContainer = new CookieContainer();
             this.SetTimeout(60 * 1000);
        }

        /// <summary>
        /// Gets or sets custom user agent.
        /// </summary>
        /// <value>
        /// The user agent.
        /// </value>
        public string UserAgent
        {
            get { return _userAgent; }
            set { _userAgent = value; }
             
        }

        public WebClientEx SetTimeout(int timeout)
        {                
            this.Timeout = timeout;
             return this;
        }

         /// <summary>
        /// Gets the final target response URI.
        /// </summary>
    public Uri ResponseUri
    {
       get { return _responseUri; }
    }

         /// <summary>
        /// Overrides the <see cref="T:System.Net.WebResponse"/> for the specified <see cref="T:System.Net.WebRequest"/>.
        /// Sets the ResponseUri.
        /// </summary>
        /// <param name="request">A <see cref="T:System.Net.WebRequest"/> that is used to obtain the response.</param>
        /// <returns>
        /// A <see cref="T:System.Net.WebResponse"/> containing the response for the specified <see cref="T:System.Net.WebRequest"/>.
        /// </returns>
    protected override WebResponse GetWebResponse(WebRequest request)
      {
    WebResponse response = null;
    try
    {
    response = base.GetWebResponse(request);
    _responseUri = response.ResponseUri;
    }
    catch
     {
    }
    return response;
    }


     /// <summary>
    /// Overrides the  <see cref="T:System.Net.WebRequest"/> object for the specified resource.
    /// Adds Gzip/deflat Request Headers.
    /// Sets Automatic decompression of compressed responses.
    /// Sets PreAuthenticate header.
    /// Adds CookieContainer to transmit cookies.
    /// Sets custom User-Agent string.
    /// Sets Timeout for the Request.
    /// </summary>
    /// <param name="address">A <see cref="T:System.Uri"/> that identifies the resource to request.</param>
    /// <returns>
    /// A new <see cref="T:System.Net.WebRequest"/> object for the specified resource.
    /// </returns>
        protected override WebRequest GetWebRequest(Uri address)
        {
            WebRequest request = base.GetWebRequest(address);
    request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");
    // add auto- Gzip / deflate handling
            ((HttpWebRequest)request).AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
     // next line avoids double hits to a WWW-Authenticate / Basic source
    request.PreAuthenticate = true;
    if (this.UserAgent !=null)
     (request as HttpWebRequest).UserAgent=this.UserAgent;

             if (request.GetType() == typeof(HttpWebRequest))
            {
                ((HttpWebRequest)request).CookieContainer = this._cookieContainer;
                ((HttpWebRequest)request).UserAgent = this._userAgent;
                ((HttpWebRequest)request).Timeout = this._timeout;
             }
             return request;
        }


        /// <summary>
        /// Enables certificate validation.
        /// </summary>
        /// <param name="validate">if set to <c>true</c> [validate].</param>
        /// <returns>instance of the WebClientEx class to use</returns>
        public WebClientEx ServerCertificateValidation(bool validate)
         {
             if (!validate) ServicePointManager.ServerCertificateValidationCallback += delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) { return true; };
            return this;
        }
    }
}
  
I've constructed a sample ASP.NET project that illustates the use of some of these features. In this project there are three pages - Default.aspx, Receiver.aspx, and CheckCookie.aspx.

Default.aspx is where we start:

protected void Button1_Click(object sender, EventArgs e)
        {
         WebClientEx  wc = new WebClientEx();
         wc= wc.ServerCertificateValidation(true);
         wc.SetTimeout(60000);
          wc.UserAgent = "dorky";
          wc.Headers.Add("Content-Type","application/x-www-form-urlencoded");
           // Apply ASCII Encoding to obtain the string as a byte array.
            byte[] byteArray = Encoding.ASCII.GetBytes("test=hello");

            var res = wc.UploadData("http://localhost:10000/Receiver.aspx", byteArray);
            var cook = wc.ResponseHeaders["Set-cookie"];
            System.Diagnostics.Debug.WriteLine("Cookie: " + cook);
             Session["info"] = System.Text.Encoding.ASCII.GetString(res);
             wc.Dispose();
             Response.Redirect("CheckCookie.aspx");
        }

The above code creates a WebClientEx instance, sets Server Certificate Validation, sets a timeout, sets the Content-Type for a form post, encodes a sample name=value pair to post, and performs the POST via the UploadData method. It then gets the cookie that was set by the Receiver.aspx page, stores the result that was sent back in Session, and finally redirects to the CheckCookie.aspx page which does the following:

protected void Page_Load(object sender, EventArgs e)
        {
            WebClientEx wc = new WebClientEx();
            var res = wc.DownloadString("http://localhost:10000/Receiver.aspx");
            Label1.Text = (string) Session["info"] + "<BR>" + res + "<br/>Cookie: " + Request.Cookies["test"].Value;
            wc.Dispose();

        }

CheckCookie creates a new WebClientEx instance and downloads the info from Receiver.aspx and displays it.

Receiver.aspx has the following logic:

protected void Page_Load(object sender, EventArgs e)
         {
             if (Request.Cookies["test"] == null)
            {

                var cook = new HttpCookie("test", "bork! Bork!");
                cook.Expires = DateTime.Now.AddDays(1);
                 Response.Cookies.Add(cook);
                var x = Request.Form["test"];
                 Response.Write("you posted: " + "test=" + x);
            }
            else
            {
                Label1.Text = "Your Cookie 'test': " + Request.Cookies["test"] + "<br/>" + (string) Session["info"];
            }

        }

If the "test" cookie is not present, Receiver.aspx sets a new "test" cookie with value "bork! Bork!". It also echoes back what the default.aspx page POSTED in its form post.

The final display on Receiver.aspx after pressing the POST button on default.aspx looks like the following, all as a "proof of concept".

you posted: test=hello

you posted: test=

Cookie: bork! Bork!

There are other customizations that can be made to WebClient using this technique. I leave that exercise to the reader. Feel free to modify the code as you see fit.

You can download the Visual Studio 2010 demo solution here.

By Peter Bromberg   Popularity  (3957 Views)