After working a bit with MongoDb and getting comfortable with the NoRM C# driver
with this previous article , it occurred to me that, because of it’s speed and the fact that it is TCP – addressable
from any machine on a network, it could be used as a System.Web.Caching.Cache
replacement on a web site or even a web farm.
I had already created my FileCache class here, and I reasoned that with only some minor changes, I could use this as the basis
for a MongoDb Cache class. It turned out that within an hour, I had a fully-functional
MongoDb – based Cache.
Obviously, the first order of details would be to test the thing and see what kind
of performance it gave. On a local machine, storing 1000 datasets containing
2 columns and 1000 rows each took 2134 ms, or about 2 milliseconds per operation.
Getting 1000 such items from the Cache took about 3267 ms, or about 3 ms per
operation. Updates, which are faster, took 1846ms, and Deletes took 463ms.
After I added an index on the CacheKey field, Get performance on the local machine
dropped to just 1,000 ms or about 1 millisecond per operation:
using (Mongo mongo = Mongo.Create(Helper.ConnectionString()))
{
MongoCollection<CacheItem> coll = mongo.GetCollection<CacheItem>();
coll.CreateIndex(u => u.CacheKey, "CacheKey", true, IndexOption.Descending);
}
From a remote machine, the numbers were 7458ms for 1000 inserts, 16,727ms for 1000
Gets, and 4360 ms for 1000 Deletes. But -- after adding the index as described
above, the Gets dropped down to just 3154 ms. This is pretty decent performance
considering the size of the payload, and the fact that none of this cost me a
dime!
With some of the commercial .NET distributed cache products running from $1,200 to
$2,500 and up, I think this kind of arrangement bears some serious consideration.
The advantages of using such a MongoDb - based Cache are several-fold:
1) It can be accessed from any machine on a network
2) The cost is reasonable (as in “Free Beer”!)
3) The cache survives IIS AppPool recycles and even machine reboots.
4) Since it is designed to be able to implement the base OutputCacheProvider class
(by just adding the required override keywords), it can be easily made to be
a complete replacement for System.Web.Caching.Cache as described in the second
article linked above.
5) MongoDB supports an automated sharding architecture, enabling horizontal scaling
across multiple nodes. For applications that outgrow the resources of a single
database server, MongoDB can convert to a sharded cluster, automatically managing
failover and balancing of nodes, with few or no changes to the original application
code.
6) The NoRM MongoDb C# Driver supports LINQ, making it extremely easy to perform
queries, inserts and updates with a very quick learning curve.
My MongoCache class looks like this:
using System;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
using Norm;
using Norm.Collections;
namespace MongoCache
{
public class Cache
{
public object this[string cacheKey]
{
get { return Get(cacheKey); }
set { Add(cacheKey, value, DateTime.Now.AddDays(365)); }
}
public object Add(string key, object entry, DateTime utcExpiry)
{
Set(key, entry, utcExpiry);
return entry;
}
public object Get(string key)
{
CacheItem item = null;
using (Mongo mongo = Mongo.Create(Helper.ConnectionString()))
{
MongoCollection<CacheItem> coll = mongo.GetCollection<CacheItem>();
item = coll.FindOne(new {_id = key});
}
if (item == null || item.Expires <= DateTime.Now.ToUniversalTime())
{
Remove(key);
return null;
}
var f = new BinaryFormatter();
var ms = new MemoryStream(item.Item);
object o = f.Deserialize(ms);
return o;
}
public void Remove(string key)
{
using (Mongo mongo = Mongo.Create(Helper.ConnectionString()))
{
MongoCollection<CacheItem> coll = mongo.GetCollection<CacheItem>();
var q = new CacheItem
{
CacheKey = key
};
coll.Delete(q);
}
}
public void Set(string key, object entry, DateTime utcExpiry)
{
using (Mongo mongo = Mongo.Create(Helper.ConnectionString()))
{
MongoCollection<CacheItem> coll = mongo.GetCollection<CacheItem>();
var f = new BinaryFormatter();
var ms = new MemoryStream();
f.Serialize(ms, entry);
var q = new CacheItem
{
CacheKey = key,
Expires = utcExpiry,
Item = ms.ToArray()
};
coll.Save(q);
}
}
The CacheItem class, which is used to hold the payload, looks like this:
using System;
using Norm;
namespace MongoCache
{
[Serializable]
public class CacheItem
{
[MongoIdentifier]
public string CacheKey { get; set; }
public DateTime Expires { get; set; }
public byte[] Item { get; set; }
}
}
Note that the MongoIdentifier attribute tells MongoDb that CacheKey is the field
we want it to use for the primary key or _id in the database.
I also have a Helper class that is basically “lifted” from the NoRM driver test suite,
simplified, and used to handle connections and basic operations via a LINQ –
type interface. You may wish to check the Github sources as there is now an active
developer community and occasional updates to the source: http://github.com/atheken/NoRM
When creating this, I debated whether to store the Cache Item property as type Object
(without binary serialization to a byte array) but for conformity’s sake I abandoned
that idea, at least for now. You could certainly try this if you want, since
MongoDb lets you put your junk directly in the trunk, no matter what type it
may be (It performs BSON serialization internally).
In the downloadable Visual Studio 2010 Solution, there is a copy of the NoRM driver
which I have modified very slightly to permit GetCollectionName, which was marked
internal, to be called from outside the NoRM assembly. Otherwise the driver is
left untouched.
There is the MongoCache project, and then there are two test projects, a Web Application,
and a Windows Forms application that I used to do the speed tests.
If you want to play with this, you’ll need to first install MongoDb; complete instructions
may be found in the first article linked above. You will also need to modify
the AppConfig or WebConfig appSettings entry:
<appSettings>
<add key="connectionStringHost" value="192.168.1.100"/>
</appSettings>
-- that is, unless your machine happens to have the same Class C local network IP
Address as mine! Of course if you are only testing locally, you can set this
to either localhost or 127.0.0.1, and if you intend to expose MongoDb over the
internet, it would need to be a public resolvable IP address on the machine.
MongoDb does support authentication; I don't get into that here. The easiest
thing to do is simply restrict via either firewall or IP Restrictions who has
access to the default port of 27017 because typically it would only be your own
machines that would ever access it.
Incidentally, the MongoDb service uses memory-mapped files as the backing store,
so don't look in Task Manager and expect to see an accurate measure of memory
usage. The best way to get a feel for how much memory the running service uses
is to stop and then restart the service, and have a look at the working set then.
It's pretty lean!
And thanks to Andrew Theken, Karl Seguin and the rest of the crew who've been working on the NoRM driver!
You can download the Visual Studio 2010 Solution for MongoCache here.