Silverlight 2 RC0 Doing Data Part VIII : Using the Threadpool

Silverlight 2 implements the ThreadPool, which can be useful for parallel operations on background threads. Here we aggregate a number of RSS Search feeds and display the combined results in a DataGrid.

The Threadpool performs operations on background threads via its QueueUserWorkItem method.  Instead of having to make blocking calls for each search operation, and wait for each to complete before we can start the next one, we can kick them all off at once on ThreadPool threads, and they will all be executed in parallel (subject to available ThreadPool Threads and the limitations of the browser HttpRequest API).

If we know in advance (as in this case) how many enqueued WorkItems we've kicked off, then it becomes a simple matter to detect that we're finished with all our background thread operations and then continue on with the process of aggregating all the results.

In order to illustrate how this can be useful, I picked four RSS - formatted search urls (Digg, Google Blog Search, MSDN and ittyurl.net), each of which exposes the required crossdomain.xml or clientaccesspolicy.xml file at the root of the site, enabling a Silverlight client to make cross-domain calls to get data.

My UI has a DataGrid, a TextBox for search term, and a "Search" button that kicks off the operation. First, let's look at the XAML for the UI:

 

<UserControl xmlns:my="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data"  x:Class="SLThreadPool.Page"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    Width="700" Height="500">
    <Grid x:Name="LayoutRoot" Background="White">
        <TextBlock Text="Silverlight Threadpool Sample" VerticalAlignment="Top" Width="500" HorizontalAlignment="Center" FontSize="20" />
        <my:DataGrid x:Name="dgFeeds" AutoGenerateColumns="False"   IsReadOnly="True" Height="400" Width="700" RowHeight="30"  HorizontalAlignment="Left" Margin="0 10 0 0" ColumnWidth="600" >
            <my:DataGrid.Columns>
                <my:DataGridTemplateColumn>
                    <my:DataGridTemplateColumn.CellTemplate>
                        <DataTemplate>
                            <StackPanel Margin="5 5 5 5" Width="600">                             
                                <HyperlinkButton Content="{Binding Title}" NavigateUri="{Binding Link}" TargetName="_blank"></HyperlinkButton>
                            </StackPanel>
                        </DataTemplate>
                    </my:DataGridTemplateColumn.CellTemplate>
                </my:DataGridTemplateColumn>
            </my:DataGrid.Columns>
            <my:DataGrid.RowDetailsTemplate>
                <DataTemplate>
                    <StackPanel Margin="5 5 5 5" Width="600" Height="75" >
                        <TextBlock Text="{Binding Description}" TextWrapping="Wrap" Width="600" FontSize="10"/>
                    </StackPanel>
                </DataTemplate>
            </my:DataGrid.RowDetailsTemplate>
        </my:DataGrid>

        <TextBox x:Name="Text1" HorizontalAlignment="Left" VerticalAlignment="Bottom" Padding="5"  Width="200" Text="Silverlight"></TextBox> 
        <Button Width="150" Height="25" HorizontalAlignment="Right" VerticalAlignment="Bottom" Content="Search!" Click="Button_Click"></Button>
    </Grid>
</UserControl>
Now, let's look at how I implemented the ThreadPool operation in the codebehind class for the Page:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net;
using System.ServiceModel.Syndication;
using System.Text;
using System.Threading;
using System.Windows;
using System.Windows.Controls;
using System.Xml;

namespace SLThreadPool
{
    public class FeedItem
    {
        public FeedItem(string description, string link, string title)
        {
            Description = description;
            Link = link;
            Title = title;
        }

        public String Description { get; set; }
        public string Link { get; set; }
        public string Title { get; set; }
    }

    public partial class Page : UserControl
    {
        private readonly string[] feedUrls = 
        {
              "http://digg.com/rss_search?area=all&type=both&age=7&search=",
             "http://66.102.1.103/blogsearch_feeds?hl=en&spell=1&oi=spell&ie=utf-8&num=100&output=rss&q=",
             "http://lab.msdn.microsoft.com/search/data.ashx?query=",
             "http://ittyurl.net/RSS2.aspx?t="
         };

        public SyndicationFeed bigFeed;
        public int ctr;
        public int feedsCount;
        public List<SyndicationItem> itemList;

        public Page()
        {
            InitializeComponent();
            bigFeed = new SyndicationFeed();
            itemList = new List<SyndicationItem>();
        }

        private void Button_Click(object sender, RoutedEventArgs e)
        {
            string srch = Text1.Text;
            feedsCount = feedUrls.Length;
            foreach (string url in feedUrls)
            {
                ThreadPool.QueueUserWorkItem(ProcessItem, url + srch);
            }
        }

        private void ProcessItem(object stateInfo)
        {
            var wc = new WebClient();
            wc.DownloadStringCompleted += wc_DownloadStringCompleted;
            wc.DownloadStringAsync(new Uri(stateInfo.ToString()));
        }

        private void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
        {
            try
            {
                string res = e.Result;
                if (res == "")
                {
                    feedsCount--;
                    return;
                }

                var ms = new MemoryStream(Encoding.UTF8.GetBytes(res));
                var s = new XmlReaderSettings();
                s.DtdProcessing = DtdProcessing.Parse;
                XmlReader reader = XmlReader.Create(ms, s);
                SyndicationFeed feed = SyndicationFeed.Load(reader);
                foreach (SyndicationItem itm in feed.Items)
                    itemList.Add(itm);
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
            }

            ctr++;
            //Display some progress
            Dispatcher.BeginInvoke(() => Text1.Text = "Feed: "+ctr.ToString() +" Items=" +itemList.Count.ToString());
            Debug.WriteLine(itemList.Count.ToString());

            if (ctr >= feedsCount)
            {
                bigFeed.Items = itemList;
                Dispatcher.BeginInvoke(() => BindGrid());
            }
        }

        private void BindGrid()
        {
            dgFeeds.RowDetailsVisibilityMode = DataGridRowDetailsVisibilityMode.VisibleWhenSelected;        
            Text1.Text = bigFeed.Items.Count() + " Items.";
            // Create custom simple Items collection to bind to grid
            var items = new List<FeedItem>();
            foreach (SyndicationItem itom in bigFeed.Items)
            {
                items.Add(new FeedItem(itom.Summary.Text, itom.Links[0].Uri.ToString(), itom.Title.Text));
            }
            dgFeeds.ItemsSource = items;
        }
    }
}
What's happening here? First, I created a simple container class, "FeedItem" to hold simple properties that can be easily databound. I did this because the SyndicationFeedItem class has properties such as "Link" that hold an array of link Uri's that can be more difficult to handle with simple databinding. When I have all my results, all I need to do is pour them into a List of FeedItems, and I'm ready to go.

In my Page class I start out with a string array of urls, each formatted so that the search term can be appended to the end.  I also declare a List of type SyndicationItem (to handle aggregating each item from multiple ThreadPool operations) and a "bigFeed" of type SyndicationFeed to hold all of the aggregated SyndicationItem(s).

When the Search button is clicked, we simply iterate over our URL list and enqueue a WorkItem for each search. The ThreadPool takes over, and  the ProcessItem callback handles the creation of a WebClient instance to download the results of each query.

In the  wc_DownloadStringCompleted callback for each WebClient call, we get our FeedItems and add each of them to the itemList collection.

When the "ctr" variable is equal to the count of the feeds, we know we are done and  we bind our grid, first simplifying each SyndicationItem into my simple FeedItem List, from which we can easily databind our XAML DataGrid:


private
void BindGrid()

{

dgFeeds.RowDetailsVisibilityMode = DataGridRowDetailsVisibilityMode.VisibleWhenSelected;

Text1.Text = bigFeed.Items.Count() + " Items.";

// Create custom simple Items collection to bind to grid

var items = new List<FeedItem>();

foreach (SyndicationItem itom in bigFeed.Items)

{

items.Add(new FeedItem(itom.Summary.Text, itom.Links[0].Uri.ToString(), itom.Title.Text));

}

dgFeeds.ItemsSource = items;

}

In general, with the proper use of  the ThreadPool we can greatly speed up the process of making multiple WebRequests, allowing us to aggregate the results quite easily.  While there are other ways to use the Silverlight ThreadPool class, you will probably find in your travels that this is the most common one. In order to keep the sample as readable as possible, I've left out most exception handling that one would normally put into a page like this. 

Additional "exercises for the reader" might include using LINQ extension methods to sort the results, as well as to remove duplicate results having the same Link URL.  By the way, can you spot the place in my code where I should be doing something extra? You can download the complete Visual Studio 2008 Silverlight 2 RC0 solution here.

Think big, code like a madman, and fear nothing.
By Peter Bromberg   Popularity  (1956 Views)