MongoDb Database Cache with 10Gen C# Driver and FluentMongo

I've been an aficionado of MongoDb for quite some time now; the NoSQL movement continues to grow and more and more developers are discovering it. Up until recently I've used the NoRM C# driver with MongoDb, which implements a very nice LINQ interface over database queries. However, the 10Gen people http://www.10gen.com/ (who put out MongoDb), have focused on an official C# driver, and they did a fantastic job - it fully supports all the features of MongoDb 1.8.

Missing from this driver until recently has been the LINQ support I've come to know and love. However, Craig Wilson has created FluentMongo which implements a fully LINQ - compliant set of extension methods around the AsQueryable extension of the native C# driver's MongoCollection.

This makes it possible to execute virtually any MongoDb  query via familiar LINQ syntax:

  _server = MongoServer.Create("mongodb://localhost");
  _database = _server.GetDatabase("Cache");
  MongoCollection<CacheItem> coll =  _database.GetCollection<CacheItem>("Cache");
string key = "testItem";
  var item = (CacheItem) coll.AsQueryable().Where(x => x.Id == key).FirstOrDefault();

  That's all it takes to find a single item from the "Cache" database by key. Of course, in my demo a lot of the above is commoditized via utility and static methods.

  What I did here was to take a previous "MongoCache" project that I did with the NoRM C# driver, and convert it over to the 10Gen official C# driver plus Craig's FluentMongo. It wasn't difficult at all. For details on the original concept including test times, etc. please review the original article.

I start out with a CacheItem class, which is a container class for a cache item payload:

using System;
using MongoDB.Driver;

namespace MongoCache
{
    [Serializable]
     public class CacheItem : IMongoQuery
    {
         public string Id { get; set; }
        public DateTime Expires { get; set; }
         public byte[] Item { get; set; }
    }
}


The Id property is the Cache key (which MongoDb automatically recognizes as a PK via its name); the Expires property is a DateTime expiry date for the cache item, and the Item property is a byte array that represents the serialized object that was stored.

Note also that this implements the IMongoQuery interface which provides us with all the LINQ queryable goodness and access to methods like Save and Remove on a collection. A very simple class indeed.

Next, I have a Helper class that provides some plumbing:

using System;
using System.Configuration;
using System.Linq;
using MongoDB.Driver;
using MongoDB.Driver.Builders;
using FluentMongo;
using FluentMongo.Linq;

namespace MongoCache
{
    public static class Helper
    {
         private static readonly string _connectionStringHost = ConfigurationManager.AppSettings["connectionStringHost"];

        private static  MongoServer _server;
         private static MongoDatabase _database;
        public static string ConnectionString()
        {
            return ConnectionString(null);
        }

       static Helper()
        {
      
            _server = MongoServer.Create("mongodb://localhost");
            _database = _server.GetDatabase("Cache");
        }

         public static MongoCollection<T> GetCollection<T>(string name)
         {
             return _database.GetCollection<T>(name);
         }

        public static IQueryable<CacheItem> CacheItems
        {
            get { return _database.GetCollection<CacheItem>("Cache").AsQueryable(); }
        }
      
        public static void CreateIndex ()
        {
            var coll = GetCollection<CacheItem>("Cache");
            coll.CreateIndex(new string[] {"Id"});
        }

         public static string ConnectionString(string query)
         {
             return ConnectionString(query, null, null, null);
        }

         public static string ConnectionString(string userName, string password)
         {
             return ConnectionString(null, null, userName, password);
        }

        public static string ConnectionString(string query, string userName, string password)
         {
             return ConnectionString(query, null, userName, password);
        }

        public static string ConnectionString(string query, string database, string userName, string password)
        {
            string authentication = string.Empty;
            if (userName != null)
            {
                authentication = string.Concat(userName, ':', password, '@');
            }
            if (!string.IsNullOrEmpty(query) && !query.StartsWith("?"))
            {
                query = string.Concat('?', query);
            }
            string host = string.IsNullOrEmpty(_connectionStringHost) ? "localhost" : _connectionStringHost;
            database = database ?? "Cache";
            return string.Format("mongodb://{0}{1}/{2}{3}", authentication, host, database, query);
        }
    }
}


Finally, I have my MongoCache class:

using System;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
using FluentMongo;
using FluentMongo.Linq;
using MongoDB.Bson;
using MongoDB.Driver;
using System.Linq;

namespace MongoCache
{
    public class Cache
    {
         public object this[string cacheKey]
        {
            get { return Get(cacheKey); }
            set { Add(cacheKey, value, DateTime.Now.AddDays(365)); }
        }

        public object Add(string key, object entry, DateTime utcExpiry)
        {
            Set(key, entry, utcExpiry);
             return entry;
        }

         public object Get(string key)
        {
            var coll = Helper.GetCollection<CacheItem>("Cache");
            var item = (CacheItem) coll.AsQueryable().Where(x => x.Id == key).FirstOrDefault();

             if (item == null || item.Expires <= DateTime.Now.ToUniversalTime())
             {
                 Remove(key);
                 return null;
            }
            var f = new BinaryFormatter();
            var ms = new MemoryStream(item.Item);
            object o = f.Deserialize(ms);
             return o;
        }

         public void Remove(string key)
        {
            var coll = Helper.GetCollection<CacheItem>("Cache");
            CacheItem itm = new CacheItem() {Id = key};
             coll.Remove(itm);
         }


        public void Set(string key, object entry, DateTime utcExpiry)
        {
            MongoCollection<CacheItem> coll = Helper.GetCollection<CacheItem>("Cache");
            var f = new BinaryFormatter();
            var ms = new MemoryStream();
            f.Serialize(ms, entry);
            var q = new CacheItem
                        {
                           Id = key,
                            Expires = utcExpiry,
                            Item = ms.ToArray()
                          };
             coll.Save(q);
        }
    }
}

The Cache class offers the familiar Get, Set, and Remove methods, as well as an Add method and an indexer so we can use familiar

Cache["key"] = myDataSet and
DataSet myDataSet = (DataSet)Cache["key"] semantics.

That's all there is to the whole thing - simple and elegant.

These are some of the methods that will appear on a MongoCollection via the C# Driver:

.AsQueryable (from FluentMongo)
.Count
.CreateIndex
.Database (property)
.Distinct
.Drop
.DropAllIndexes
.DropIndex
.DropIndexByName
.EnsureIndexx
.Equals
.Find
.FindAll
.FindAllAs
.FindAndModify
.FindAndRemove
.FindAs
.FindOne
.FindOneAs
.FindOneById
.FindOneByIdAs
.FullName
.GeoNear
.GeoNearAs
.GetHashCode
.GetIndexes
.GetStats
.GetTotalDataSize
.GetTotalStorageSize
.GetType
.Group
.IndexExists
.IndexExistsByName
.Insert
.InsertBatch
.IsCapped
.MapReduce
.Name (property)
.Reindex
.Remove
.RemoveAll
.ResetIndexCache
.Save
.Settings (property)
.ToBson
.ToBsonDocument
.ToJson
.ToString
.Update
.Validate


And Here are the LINQ extension methods off AsQueryable:

.Aggregate
.All
.Any
.AsEnumerable
.AsQueryable
.. etc - virtually all the standard LINQ operators.

In the solution there is a Windows Forms app that exercises the Cache:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Diagnostics;

namespace NonWebCacheTest
{
    public partial class Form1 : Form
    {
         public static MongoCache.Cache  FCache = new MongoCache.Cache();
        public Form1()
        {
             InitializeComponent();
        }

         private void Form1_Load(object sender, EventArgs e)
         {

        }

        private DataSet MakeTestDataSet()
        {
            label1.Text = "Was not in Cache.";

            var ds = new DataSet();
             // Don't forget this next line: it will reduce DataSet file size by 90 percent!
        ds.RemotingFormat = SerializationFormat.Binary;
            var tbl = new DataTable();

             tbl.Columns.Add("Test");
            tbl.Columns.Add("Test2");
            ds.Tables.Add(tbl);
            DataRow row = null;
            for (int i = 0; i < 1000; i++)
            {
                row = tbl.NewRow();
                row.ItemArray = new object[] { "hello", "there" };
                 tbl.Rows.Add(row);
            }
            ds.RemotingFormat = SerializationFormat.Binary;
             return ds;
        }

         // Gets an object from the cache based on its key
        private bool GetDsFromCache(string cacheKey, out DataSet ds)
        {
             // Can use either indexer or Get method.
            var myDs = (DataSet)FCache[cacheKey];
            
            if (myDs == null)
            {
                ds = null;
                 return false;
            }
            else
            {
                ds = myDs;
                 return true;
            }
        }

         //Stores 1,000 objects in the MongoCache database by unique key
        private void button2_Click(object sender, EventArgs e)
         {
             // store
            DataSet ds = this.MakeTestDataSet();
            System.Diagnostics.Stopwatch sw = new Stopwatch();
             sw.Start();
            for(int i =0;i<1000;i++)
             {
                 FCache["test" + i.ToString()] = ds;
            }
            sw.Stop();  
            this.label1.Text = sw.ElapsedMilliseconds.ToString() + " ms";
            sw.Reset();
        }

         // Gets out 1,000 objects from the MongoCache database by key
        private void button3_Click(object sender, EventArgs e)
        {
            DataSet ds = this.MakeTestDataSet();
            System.Diagnostics.Stopwatch sw = new Stopwatch();
             sw.Start();
            for (int i = 0; i < 1000; i++)
            {
                 ds = (DataSet)FCache["test" + i.ToString()];
            }
            sw.Stop();
            dataGridView1.DataSource = ds.Tables[0];
             this.label1.Text = sw.ElapsedMilliseconds.ToString() + " ms";
            sw.Reset();


        }

         // Removes all 1,000 objects from the cache
        private void button1_Click_1(object sender, EventArgs e)
        {
            System.Diagnostics.Stopwatch sw = new Stopwatch();
             sw.Start();
            for (int i=0;i<10000;i++)
             {
                 FCache.Remove("test" + i.ToString());
             }
             sw.Stop();
             this.label1.Text = sw.ElapsedMilliseconds.ToString() + " ms";
            sw.Reset();
        }

      
      // creates an index on the Cache key
        private void button5_Click(object sender, EventArgs e)
        {
            MongoCache.Helper.CreateIndex();
        }

       // finds one specific Cache item by key
        private void btnFindOne_Click(object sender, EventArgs e)
        {
            System.Diagnostics.Stopwatch sw = new Stopwatch();
             sw.Start();
            var itm = FCache["test11"];
            sw.Stop();
            this.label1.Text = sw.ElapsedMilliseconds.ToString() + " ms";
            sw.Reset();
            dataGridView1.DataSource = ((DataSet) itm).Tables[0];
        }
    }
}

the above code, with the comments, should be self-explanatory.  Both the Mongo.db.driver assemblies and the Mongo.db.Fluent assembly should be available either through NuGet packages or by downloading the respective packages from the respective sites. There is also an ASP.NET web project in the solution that shows the use of my MongoDb Cache in a web app.


If you do not have MongoDb installed as a service and you would like to do so (I recommend it) you can install it by executing the following command from the C:\MongoDb\bin folder via a command prompt:

mongod --bind_ip 127.0.0.1 --logpath c:\mongodb\logs.txt --logappend  --install

This should result in a command line in Service Control Manager that looks like this:

"C:\MongoDb\bin\mongod" --bind_ip  127.0.0.1  --logpath   "c:\mongodb\logs.txt"  --logappend  --service

Note above that the "--install" directive which is needed only for the initial install has been replaced with "--service".

Other than that no special configuration is needed except to ensure that the default C:\data\db  folders exist.

You can download the demo Visual Studio 2010 solution here.

By Peter Bromberg   Popularity  (5687 Views)