A .NET Memory-Mapped Cache TcpListener Service
By Peter A. Bromberg, Ph.D.

Peter Bromberg

File Mapping

File mapping is the association of a file's contents with a portion of the virtual address space of a process. The system creates a file mapping object to maintain this association. A file view is the portion of virtual address space that the process uses to access the file's contents. Processes read from and write to the file view using pointers, just as they would with dynamically allocated memory. Processes can also manipulate the file view with the VirtualProtect function. File mapping provides two major advantages:

•  Faster and easier file access
•  Shared memory between two or more applications



File mapping allows a process to access files more quickly and easily by using a pointer to a file view. Using a pointer improves efficiency because the file resides on disk, but the file view resides in memory. File mapping allows the process to use both random input and output (I/O) and sequential I/O. It also allows the process to efficiently work with a large data file, such as a database, without having to map the whole file into memory. When the process needs data from a portion of the file other than what is in the current file view, it can unmap the current file view, then create a new file view.

The file mapping functions allow a process to create file mapping objects and file views to easily access and share data.

The file on disk can be any file that you want to map into memory, or it can be the system page file.

The file mapping object can consist of all or only part of the file. It is backed by the file on disk. This means that when the system swaps out pages of the file mapping object, any changes made to the file mapping object are written to the file. When the pages of the file mapping object are swapped back in, they are restored from the file.

A file view can consist of all or only part of the file mapping object. A process manipulates the file through the file views. A process can create multiple views for a file mapping object. The file views created by each process reside in the virtual address space of that process.

When multiple processes use the same file mapping object to create views for a local file, the data is coherent. That is, the views contain identical copies of the file on disk. The file cannot reside on a remote computer if you want to share memory between multiple processes (e.g., you cannot use a UNC path).

In short, memory-mapped files provide a way to look at a file as a chunk of memory. You map the file and get back a pointer to the mapped memory. You can simply read or write to memory from any location in the file mapping, just as you would from an array. When you've processed the file and closed the file mapping, the file is automatically updated. The operating system takes care of all the details of file I/O.

Many developers are not aware that Memory Mapped files are used widely by the operating system itself, and the technique has been available since the first versions of Windows.

Memory Mapped Files and .NET

On the .NET Platform, sharing information across AppDomain boundaries typically only provides two choices: Remoting, and WebServices. Both are slow in comparison to File Mapping or "Memory Mapped Files".

As one might guess, the .NET platform does not provide support for Memory Mapped Files, and as far as I know, none is expected. You must call into the native Windows API methods. There are not many resources available for Memory Mapped File management through .NET. Once excellent article by Natty Gur on codeproject.com provides an unusually sophisticated "global cache" approach, but it also has security considerations that would have to be solved to make it more usable.

At one point, some people called "Metal Wrench" put out a .NET class library which, among other stream-related utility methods, had a Memory Mapped File Stream class.

The Microsoft Caching Application Block also has a Memory Mapped File option, but there is so much additional code and infrastructure that goes along with it that, unless you are a glutton for punishment, I'd recommend against that route for all except the most patient of developers.

Finally, Michael VanHoutte has an article (also on codeproject.com) in which he "tore apart" the MS Caching Application Block and pared it down to the most basic elements to create a kind of simplified "Cache Service". His code is written in VB.NET, which I generally try to avoid whenever possible, but it is certainly one of the best implementations I've seen.

And, not to forget, MVP Thomas Restropo "winterdom" has publish his own implementation of Memory Mapped File API PInvoke methods.

Initially, using Michael's example as a basis, I rewrote and enhanced his code into C# and added some additional features of my own to produce a new version of a C# "Memory Mapped Cache" service. My service adds one very important new element --besides the fact that different applications (including ASP.NET web applications) on the same machine can share .NET data in a common Cache, my creation also sports an asynchronous TCP Listener which enables applications on remote machines on the network to also make use of the cache located on a single central "cache machine". It operates very much the same way the ASP.NET StateServer service works, except that it is for global, enterprise-wide, inter-application data. My cache is configurable through standard AppSettings in the configuration file, both on the server and the clients, and offers "CacheProxy" and "CacheHelper" classes to make it relatively easy for remote applications, including Web Applications,to add or retrieve data from a remote cache. Best of all, as described above, it's fast. You can also have more than one named cache in operation simultaneously.

However, after a series of stress tests I ran on my own using each developer's code, I found that only the original MetalWrench Toolbox assembly performed without fail under heavy load. This is most likely because it has a MemoryMappedFileStream class that is completely written in unsafe C# code with pointers. Consequently, after several revisions to my code and more testing, that is what I decided to use. You will also find a slightly modified version of the XYSocketLib class library which performs remarkably well under heavy multiple - client loads with zero memory leaks and a "dead man's EKG" in the Task Manager CPU meter.

Let's have a look at the architecture of the Memory-Mapped Cache TCPListener Service:

 

As can be seen above, Client applications can talk directly to the Memory Mapped Caches hosted by the Cache Service, or if the Cache Service is on another machine, they can use the CacheHelper API along with a special CacheProxy class that marshals the TCP Socket calls, to talk to the Memory Mapped Caches on the remote Cache Service via TCP sockets.

My Cache itself has a Cache.Items(string key) method that returns an object containing your cached type, and it has Add and Remove methods. You can also refer to a specific Cache instance with Cache(cacheName), so multiple instances of Memory Mapped File caches can be run. For instance, you might want to have two Caches - one for small, lightweight objects (for speed) and another named Cache for fewer, but larger objects. My CacheHelper class provides several overloaded methods that make interacting with the Cache class easy:

public static object Items(string key, ActionType action,object value)

public static object Items(string cacheName, ActionType action, string key,  
string ipAddress,int port, object payload) public object Add(string key) public object Remove(string key)

The cache ipAddress and port parameters are configured in the appSettings section of the CacheService config file.

The ActionType enum includes Get, Add, Remove and Result Action Types which should be self-explanatory.

In order to talk to the Cache on a remote machine, the CacheHelper API wraps your object and request in a CacheProxy class:

using System;
namespace MemoryMappedCache
{
 public enum ActionType
 {
  Get,
  Add,
  Remove,
  Result
 } 
 [Serializable]
 public class CacheProxy
 {
  public ActionType Action;
  public Object Payload;
  public string Key;
  public string CacheName;
  public CacheProxy(string cacheName, ActionType action, string key, object payload)
  {
   this.CacheName =cacheName;
   this.Action =action;
   this.Key =key;
   this.Payload =payload;    
  }
 }
}

The CacheProxy class is essentially a "basket" that holds whatever information is needed to tell the receiving Cache what to do. It is serialized and deserialized to a byte array, and this is what is sent back and forth over the sockets, making for a very compact "packet" each way. Of course, if your Payload is a very large DataSet, for example, don't expect stellar performance from this- or from any other caching mechanism. I only mention this because I remember how incredulous I was when one reader reported that he had been using my CompressedDataSet infrastructure to send a 22MB dataset back and forth over the wire...

We receive a lot of forum posts here about how to create an asynchronous TCP Socket listener which uses the .NET ThreadPool under the hood, and so I am posting some sample code for same below (this is not the final code I decided to use for my socket server):

using System;
using System.Net.Sockets;
using System.Net;
using System.Text;
using System.Threading ;
using System.IO;
using System.Diagnostics ;
using System.Runtime.Serialization.Formatters.Binary ;
using MemoryMappedCache;

namespace MemoryMappedCache {
public class AsyncListener {
public Socket s=null;
public bool isLogging=Convert.ToBoolean(System.Configuration.ConfigurationSettings.AppSettings["isLogging"]);
public  void StartListening(int port) 
  {     
   try
   {
    // Resolve local name to get IP address
    IPHostEntry entry = Dns.Resolve(Dns.GetHostName());
    IPAddress ip = entry.AddressList[0];     
    // Create an end-point for local IP and port
    IPEndPoint ep = new IPEndPoint(ip, port);  
   
   if(isLogging)TraceLog.myWriter.WriteLine ("Address: " + ep.Address.ToString() +" : "
+ ep.Port.ToString(),"StartListening"); EventLog.WriteEntry("MMFCache Async Listener","Listener started on IP: " +
ip.ToString() + " and Port: " +port.ToString()+ "."); // Create our socket for listening s = new Socket(ep.AddressFamily, SocketType.Stream, ProtocolType.Tcp); // Bind and listen with a queue of 100 s.Bind(ep); s.Listen(100); // Setup our delegates for performing callbacks acceptCallback = new AsyncCallback(AcceptCallback); receiveCallback = new AsyncCallback(ReceiveCallback); sendCallback = new AsyncCallback(SendCallback); // Set the "Accept" process in motion s.BeginAccept(acceptCallback, s); } catch(SocketException e) { Console.Write("SocketException: "+ e.Message); } } AsyncCallback acceptCallback; AsyncCallback sendCallback ; void AcceptCallback(IAsyncResult ar) { try { // Cast the user data back to a socket object Socket s = ar.AsyncState as Socket; // End the accept and get the resulting client socket Socket s2 = s.EndAccept(ar); // Keep the "Accept" process in motion s.BeginAccept(acceptCallback, s); // Create a state object for client (real apps may cache these) StateObject state = new StateObject(); state.workerSocket = s2; // Start an async receive state.workerSocket.BeginReceive(state.buffer, 0, state.buffer.Length, 0, receiveCallback, state); } catch(SocketException e) { Debug.WriteLine(e.Message); if(isLogging)TraceLog.myWriter.WriteLine( "SocketException:"+ e.Message+e.StackTrace,"AcceptCallback"); } return; // Return the thread to the pool } // Async receive method + matching delegate variable AsyncCallback receiveCallback; void ReceiveCallback(IAsyncResult ar) { int i=0; string data=String.Empty; try { StateObject state = ar.AsyncState as StateObject; i = state.workerSocket.EndReceive(ar); if(i==0) { if(isLogging)TraceLog.myWriter.WriteLine("Shutting down socket.","ReceiveCallback"); state.workerSocket.Shutdown(SocketShutdown.Both); state.workerSocket.Close(); } else { state.ms.Write(state.buffer ,0 ,i); state.workerSocket.BeginReceive(state.buffer, 0, state.buffer.Length, 0, receiveCallback, state); if(i <state.buffer.Length) { byte[] result=HandleMessage(state); state.workerSocket.BeginSend(result, 0, result.Length, 0, sendCallback, state); } } } catch(SocketException e) { if(isLogging)TraceLog.myWriter.WriteLine("SocketException: "+ e.Message,"ReceiveCallback"); } return; // Return the thread to the pool } // Async send method + matching delegate variable void SendCallback(IAsyncResult ar) { int i=0; try { // Cast the state to an object StateObject state = ar.AsyncState as StateObject; i = state.workerSocket.EndSend(ar); // Begin another receive on the thread state.workerSocket.BeginReceive(state.buffer, 0, state.buffer.Length, 0, receiveCallback, state); } catch(SocketException e) { Debug.WriteLine(e.Message); if(isLogging)TraceLog.myWriter.WriteLine("SocketException: "+ e.Message,"SendCallback"); } return; // Return the thread to the pool } private static byte[] HandleMessage(StateObject state) { byte[] bytResponse=null; BinaryFormatter b= new BinaryFormatter(); state.ms.Position =0; CacheProxy proxy = (CacheProxy) b.Deserialize(state.ms); if(proxy.Action ==ActionType.Get) { string key=proxy.Key ; string cacheName=proxy.CacheName ; MemoryMappedCache.Cache c= new MemoryMappedCache.Cache(cacheName); object payload =c[key]; // get the cache item from the key, package into a new Cache proxy, // serialize and send out CacheProxy proxyResult = new CacheProxy(cacheName,ActionType.Result ,key,payload);
MemoryStream ms = new MemoryStream(); b.Serialize(ms, proxyResult); bytResponse=ms.ToArray(); } else if(proxy.Action ==ActionType.Add) { string key=proxy.Key ; string cacheName=proxy.CacheName ; MemoryMappedCache.Cache c= new MemoryMappedCache.Cache(cacheName); c[key]=proxy.Payload ; byte[] bytTemp=System.Text.Encoding.UTF8.GetBytes("true"); CacheProxy returnProxy = new CacheProxy(cacheName,ActionType.Result,key,bytTemp); BinaryFormatter bResp=new BinaryFormatter(); MemoryStream memResp=new MemoryStream(); bResp.Serialize(memResp,returnProxy); bytResponse=memResp.ToArray(); } else if (proxy.Action ==ActionType.Remove) { string key=proxy.Key ; string cacheName=proxy.CacheName ; MemoryMappedCache.Cache c= new MemoryMappedCache.Cache(cacheName); c.Remove(key); byte[] bytTemp=System.Text.Encoding.UTF8.GetBytes("true"); CacheProxy returnProxy = new CacheProxy(cacheName,ActionType.Result,key,bytTemp); BinaryFormatter bResp=new BinaryFormatter(); MemoryStream memResp=new MemoryStream(); bResp.Serialize(memResp,returnProxy); bytResponse=memResp.ToArray(); } return bytResponse; } } // end class } // end namespace

And here is the StateObject class I've concocted to hold the Async state items. I stuff the Buffer contents into the ms MemoryStream each time around:


  public class StateObject
   {
    // Client socket.
   public Socket workSocket = null;
    // Size of receive buffer.
    public const int BufferSize = 8192;
   // Receive buffer.
     public byte[] buffer = new byte[BufferSize];
    public MemoryStream ms = new MemoryStream();
   }

With the CacheHelper class, talking to the Memory Mapped File Cache on a remote machine is as simple as:

CacheHelper ch= new CacheHelper();

ch["ds"]=DataSet1;

DataSet dataset2=(DataSet) ch["ds"];

In the downloadable solution below, you can run the TestServer Console app to test out the server without having to install the Windows Service. Then, choose Debug/start new instance on the TestClient Console app by right-clicking the TestClient project in Solution Explorer. You will see a series of test cases repeated via the CacheHelper API talking to an instance of the Memory Mapped Cache via TCP sockets, including the serialization of a complete DataSet containing the Northwind Employees table.

NOTE:This is a work in progress and there are more enhancements, bug fixes, and a lot more testing to come, as time permits. For these reasons, I caution readers that I do not yet consider this code ready for production. However, if you are interested in using this as a basis for further exploration of Memory Mapped File Caching, you can download the full solution below. Be sure to change any instances of the Cache IpAddress and port to match what you need for your own testing, and make sure that the "isLogging" appSettings item is set to "false" in order to suppress debugging log actions and Debugger.Launch() statements. And, please be kind enough to post either below or at our forums whatever discoveries you make! As of 12/20/05, I have completely reworked to use the MetalWrench API (fastest) and added a completely new multithreaded Socket server and client.

Download the Visual Studio.NET Solution that accompanies this article

 



Peter Bromberg is a C# MVP, MCP, and .NET consultant who has worked in the banking and financial industry for 20 years. He has architected and developed web - based corporate distributed application solutions since 1995, and focuses exclusively on the .NET Platform.
Article Discussion: