Synchronizing the ASP.NET Cache across AppDomains and Web Farms
By Peter A. Bromberg, Ph.D.
Printer - Friendly Version
Peter Bromberg

Recently I published an article here about preventing multiple logins with the same credentials from different machines in a network or web farm scenario. I showed a simple solution that makes use of the Cache slidingExpiration property to act as a surrogate for the Session_End event handler while using StateServer or SQL Server session modes, which, running out of process, do not cause this event handler to fire.

Now I want to extend that concept along the lines of the thinking exposed near the end of the article - namely, how can you synchronize Cache or Application variables across different AppDomains or different servers on a web farm.

I postulated several possible solutions in the last article, and present what I believe to be one of the easiest and most extensible here. with real - world working code you can easily adapt to your own situation.

Since a unique instance of Cache runs in Application scope on each AppDomain (e.g., different Web Applications on the same machine, or different instances of the "same" application on different machines on a web farm) , our main problem in providing synchronization is how to handle two events -first, having a Cache item added or changed in one of the domains, and second, when a Cache item is either removed or expires (same as "removed") in one of the AppDomains.

One could also make the case the a new machine "joining the chorus" on a web farm might also need to entirely load its Cache collection from that of the other machines when it first comes up, but once you have the infrastructure I propose, that is easy enough to provide for.



The biggest obstacles to providing this updating functionality across App Domains are twofold:

First, the Cache class is sealed, and cannot be derived from as a base class. This means that any functionality specific to the updating we described must either come at the same time an item is added, updated, or removed from the cache, or else we could provide a kind of "wrapper class" that performs not only the requisite Cache action but also handles the updating functionality as well. I opted for the second solution here.

Second is the obstacle of "how" - how to get the information across AppDomains and the network in an efficient and elegant way so that each machine can automatically update its Cache. There are several options here, and no one is particularly better suited to the problem than any other.

First, we could use a file Dependency where changes to a file on the network would cause each machine to perform an update. This is a "pull" model.

Second we could use a SQL Server database with a Table dependency that causes each machine to perform its update action. This is also a "pull" update model, and Jit Ghosh of Microsoft has posted some marvelous code on GotDotNet that provides the infrastructure for this. However, although there are significant advantages to this method, not the least of which is better data persistence, I passed this option up as well.

Finally, there is the "push" model. By providing a wrapper class as I mentioned above and simply having a CacheControl.aspx receiver page in each of the Applications, it is possible to send a WebRequest to each of the machines (maintained in an easy-to-configure web.config AppSettings element) and have each enabled with code to do its own update "on demand".

In order for this concept to work, we need to be able to take the "meat" of a live Cache Item and serialize it using the BinaryFormatter so that it can be sent over the wire and de serialized into a new instance of our CacheControlItem class. This is then used to create, modify or remove items from the Cache on the receiving machine.

What I did was to create a Serializable CacheControlItem class as follows:


	  using System;
using System.Web.Caching;
namespace CacheControl1
{
    public enum CacheControlAction
    {
        AddItem =1,
        RemoveItem=2
        //UpdateItem=3,
        //GetAllItems=4 --etc
    }

    /// <summary>
    /// Represents a serializable wrapper class to hold a Cache item
    /// </summary>    
    [Serializable]
    public class CacheControlItem
    {
        public string Key;
        public object Item;
        public DateTime AbsoluteExpiration;
        public TimeSpan SlidingExpiration;
        public System.Web.Caching.CacheItemPriority Priority;
        // sorry, "Dependency" not marked as Serializable...class is 
		sealed. Too bad!
        //public System.Web.Caching.CacheDependency Dependency;
        public int Action;

        public CacheControlItem(string Key, object Item, DateTime AbsoluteExpiration,
                TimeSpan SlidingExpiration,CacheItemPriority Priority,
CacheControlAction Action) { this.Key=Key; this.Item=Item; this.AbsoluteExpiration=AbsoluteExpiration; this.SlidingExpiration=SlidingExpiration; this.Priority=Priority; this.Action=(int)Action; } } }

So now, whenever we use our wrapper method to add a "real" item to the Cache, it will also create a new populated instance of the same class, serialize it into a compact byte stream, and iterate through our server list sending it over the wire via the WebRequest so that each app in the farm, WebGarden, etc can receive and deserialize it, and update its own Cache. Simple, elegant, and fast! Even if you have a complex object such as a class that you have added to your Cache, provided that it is serializable, it will work.

Now let's take a look at the code in Global, where I've made these all static for ease of use:


using System;
using System.Collections;
using System.ComponentModel;
using System.Web;
using System.Web.SessionState;
using System.Web.Caching ;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;
using System.Diagnostics;
using System.Net;

namespace CacheControl1 
{
    /// <summary>
    /// Summary description for Global.
    /// </summary>
    public
	 class Global : System.Web.HttpApplication
    {
        public static string[] sServers;
        private System.ComponentModel.IContainer components = null;
        public Global()
        {
            InitializeComponent();
        }    

        public static MemoryStream Serialize(CacheControlItem item)    
        {   
            try
            {
                MemoryStream ms = new MemoryStream();                                  
                BinaryFormatter b=new BinaryFormatter();       
                b.Serialize(ms,item); 
                return ms;                 
            }
            catch(Exception ex)
            {
                throw new ApplicationException("Formatter error",ex);
            }
        }
        public static CacheControlItem Deserialize(MemoryStream msInput)    
        { 
            try
            {
                BinaryFormatter b=new BinaryFormatter(); 
                CacheControlItem c=(CacheControlItem)b.Deserialize(msInput);   
                return c;
            }
            catch(Exception ex)
            {
                throw new ApplicationException("Deserialize error",ex);
            }             
        }            


public static void AddCacheControlItem(string key,object value,
    System.Web.Caching.CacheDependency dependsOn, DateTime absoluteExpiration,
    TimeSpan slidingExpiration, CacheItemPriority priority)
        {
CacheItemRemovedCallback CacheOut = new CacheItemRemovedCallback( OnCacheOut);            
      HttpContext.Current.Cache.Add(key,value,dependsOn,absoluteExpiration,
slidingExpiration,priority,CacheOut); CacheControlItem cci = new CacheControlItem(key,value,absoluteExpiration,
slidingExpiration,priority,CacheControlAction.AddItem); MemoryStream ms=Serialize(cci); byte[] dataToSend = ms.ToArray(); for (int i = 0;i<sServers.Length;i++) { // webrequest here to send update to server list if(!sendUpdate(dataToSend,"http://" + sServers[i]+"/CacheControl.aspx")) throw new ApplicationException("Update Error"); } } public static bool sendUpdate( byte[] dataToSend, string sFullUri) { // here we send the serialized data to CacheControl.aspx receiver page--
try { HttpWebRequest req = (System.Net.HttpWebRequest)WebRequest.Create(sFullUri); req.Method = "POST"; Stream stm = req.GetRequestStream(); stm.Write(dataToSend,0,dataToSend.Length); stm.Close(); System.Net.WebResponse resp; resp = req.GetResponse(); stm = resp.GetResponseStream(); StreamReader r = new StreamReader(stm); byte[] bytRes=System.Text.Encoding.UTF8.GetBytes(r.ReadToEnd()); string res=System.Text.Encoding.ASCII.GetString(bytRes); Debug.Write(res); if (res.StartsWith("OK")) { return true; } else { return false; } } catch(Exception ex) { throw new HttpException("Update Error",ex); } } public static void OnCacheOut(string key, object val, CacheItemRemovedReason r) { // a cache item was removed - expired, etc. // do WebRequest to other servers from servers[] array to remove item CacheControlItem cci = new CacheControlItem(key,val,System.DateTime.MaxValue
,new TimeSpan(0,0,0,0,0),CacheItemPriority.Low,CacheControlAction.RemoveItem); MemoryStream ms=Serialize(cci); byte[] dataToSend = ms.ToArray(); for (int i = 0;i<sServers.Length;i++) { // webrequest here to send update to server list
if(!sendUpdate(dataToSend,"http://" + sServers[i]+"/CacheControl.aspx")) throw new ApplicationException("Update Error"); } } protected void Application_Start(Object sender, EventArgs e) { //retrieve list of servers to notify, store in string array sServers =
System.Configuration.ConfigurationSettings.AppSettings["servers"].ToString().Split(';'); } #region Web Form Designer generated code /// <summary> /// Required method for Designer support - do not modify /// the contents of this method with the code editor. /// </summary> private void InitializeComponent() { this.components = new System.ComponentModel.Container(); } #endregion } }

And finally the code in the CacheControl.aspx page the does the receiving and updating:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Web;
using System.Web.SessionState;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.HtmlControls;
using System.IO;
using System.Diagnostics;
namespace CacheControl1
{
/// <summary>
/// Summary description for CacheControl.
/// </summary>
public class CacheControl : System.Web.UI.Page
{
    private void Page_Load(object sender, System.EventArgs e)
    {
    try
    {
        byte[] receivedData = Request.BinaryRead(Request.TotalBytes);
        MemoryStream msInput=new MemoryStream(receivedData);
        CacheControlItem cci = Global.Deserialize(msInput);

        string stuff=cci.Key.ToString()+" : " +cci.Item.ToString()+": "
+cci.SlidingExpiration.ToString(); Debug.WriteLine(stuff); if(cci.Action == (int)CacheControlAction.AddItem) { Cache.Add(cci.Key,cci.Item,null,cci.AbsoluteExpiration,cci.SlidingExpiration
,cci.Priority,null); } if(cci.Action ==(int) CacheControlAction.RemoveItem) Cache.Remove(cci.Key); Response.Write("OK"); } catch(Exception ex) { Response.Write("ERROR"); } string stuff2 =String.Empty; string key=String.Empty; string val=String.Empty; IDictionaryEnumerator CacheEnum = Cache.GetEnumerator(); while (CacheEnum.MoveNext()) { key=CacheEnum.Key.ToString(); val=CacheEnum.Value.ToString(); stuff2+= key+": " + val + "\n\r"; } System.Diagnostics.EventLog.WriteEntry(stuff2,stuff2); } #region Web Form Designer generated code override protected void OnInit(EventArgs e) { // // CODEGEN: This call is required by the ASP.NET Web Form Designer. // InitializeComponent(); base.OnInit(e); } /// <summary> /// Required method for Designer support - do not modify /// the contents of this method with the code editor. /// </summary> private void InitializeComponent() { this.Load += new System.EventHandler(this.Page_Load); } #endregion } }

You can see in the code above that there is left some Event log writing and Debug statements, as it was intended to be more of a "Proof of Concept" than an out-of-the box solution. For example, I didn't bother to handle the situation where an existing Cache Item was being replaced with a new value. (Typically if you are using one of the overloads in the Cache.Add method, the existing Cache item is overwritten if the key is identical) However, with only minor modification, most developers who have a need for this type of solution will find it easy enough to improve and extend. If you are concerned about response time when doing these WebRequests, you may prefer (I didn't need to, however) to perform these in an asynchronous background thread with code like the following:


  public static ManualResetEvent allDone= 
  new ManualResetEvent(false);
  const int BUFFER_SIZE = 1024;
 RequestState myRequestState = new RequestState();     
      myRequestState.request = myWebRequest;
      IAsyncResult asyncResult=(IAsyncResult) myWebRequest.BeginGetResponse(new 
AsyncCallback(RespCallback),myRequestState); allDone.WaitOne(); myRequestState.response.Close(); private static void RespCallback(IAsyncResult asynchronousResult) { try { RequestState myRequestState=(RequestState) asynchronousResult.AsyncState; WebRequest myWebRequest1=myRequestState.request; myRequestState.response = myWebRequest1.EndGetResponse(asynchronousResult); Stream responseStream = myRequestState.response.GetResponseStream(); myRequestState.responseStream=responseStream; IAsyncResult asynchronousResultRead =
responseStream.BeginRead(myRequestState.bufferRead, 0, BUFFER_SIZE,
new
AsyncCallback(ReadCallBack), myRequestState); } catch(Exception e) { throw new Exception(e); } } private static string ReadCallBack(IAsyncResult asyncResult) { string sringContent = String.Empty; try { RequestState myRequestState = (RequestState)asyncResult.AsyncState; Stream responseStream = myRequestState.responseStream; int read = responseStream.EndRead( asyncResult ); if (read > 0) { myRequestState.requestData.Append(Encoding.ASCII.GetString(myRequestState.bufferRead,
0, read)); IAsyncResult asynchronousResult =
responseStream.BeginRead( myRequestState.bufferRead, 0, BUFFER_SIZE,
new
AsyncCallback(ReadCallBack), myRequestState); } else { if(myRequestState.requestData.Length>1) { sringContent = myRequestState.requestData.ToString(); return stringContent; } responseStream.Close(); allDone.Set(); } } catch(Exception ex) { throw new Exception(ex); } }

This approach should work for different applications on the same machine or AppDomains in a WebGarden, different machines on a server farm, and even for a web farm with Sticky IP and a separate copy of StateServer running on each machine. All you need to do is make sure that each Web.config has an AppSettings element containing the semicolon-delimited list of machines/application folders to find the CacheControl.aspx files (except for the current machine, or you could simply use code to eliminate that item so all the web.config files can be identical). You could also add a CacheItemCollection class which should also be serializable. This would be used for instances where a machine requests a full update of the Cache. Instead of deserializing to a CachControlItem class, you could have an "All=yes" on the querystring in the WebRequests that tells your code in the CacheControl.aspx pages to look for an entire Collection of Cache items to refresh its Cache instance with.

[NOTE:] The downloadable Solution is Visual Studio.Net 2003. If you don't have it, just create your own solution and project in VS.Net 2002 and add all the files, and it should work fine.

Download the code that accompanies this article


Peter Bromberg is a C# MVP, MCP, and .NET consultant who has worked in the banking and financial industry for 20 years. He has architected and developed web - based corporate distributed application solutions since 1995, and focuses exclusively on the .NET Platform.