The first thing we need is a list of the twitter usernames who our entered user follows.
It turns out that this is so easy, I didn't even use one of the Twitter Client
APIs such as Twitterizer, TweetSharp or Linq To Twitter. There is no oAuth required.
Here is the Format:
string url ="http://api.twitter.com/1/statuses/friends.xml?screen_name=<twitterUserName>&cursor=-1"
For JSON, just replace the .xml? with .json?
The cursor=-1 parameter returns the first set of friends, newest first. By capturing
the next_cursor element in the returned XML (or JSON) you can append this to
the cursor parameter and get the next "set" of results for older friends.
Parsing the XML that comes back is simple:
WebClient wc = new WebClient();
string stuff = wc.DownloadString(url);
wc.Dispose();
XmlDocument doc = new XmlDocument();
doc.LoadXml(stuff);
// you need the next line in order to get more, set the &cursor= to this new
value.
next_cursor = doc.DocumentElement.SelectSingleNode("//next_cursor").InnerText;
XmlNodeList nodes = doc.DocumentElement.SelectNodes("//screen_name");
var users = new List<string>();
foreach (XmlNode nod in nodes)
users.Add(nod.InnerText);
numusers = users.Count;
Now we've got a list of our user's friends, and we can use the Topsy Otter API to get
a list of "LinkPost" objects for each of the users. I'm using the server-side
Otter API wrapper that I originally created for this older article. I won't go into the code for that here, if you are interested, have a look at the
OtterApi project in the downloadable solution below.
So the next step is that I must create a List of type "list" (the object
that my Otter API C# wrapper returns for a GetLinkPosts query), and keep filling this up with posts that have
links in them for each of the list of users. The fastest and most efficient
way to do this is to use the .NET Threadpool:
private ManualResetEvent mre = new ManualResetEvent(false);
foreach (string s in users)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessItem), s);
}
mre.WaitOne();
List<list> finalPosts = new List<list>();
linkposts = linkposts.OrderByDescending(x => x.date).ToList();
finalPosts = linkposts.Distinct(new MyComparer()).ToList();
finalPosts= FixLinks(finalPosts);
this.GridView1.DataSource = finalPosts;
Session["data"] = finalPosts;
GridView1.DataBind();
} // end of Button click handler
private int ctr = 0;
void ProcessItem(object state)
{
var linkpost = Otter.GetLinkPosts((string)state, 0, "http://");
if (linkpost == null || linkpost.list == null)
{
ctr++;
return;
}
foreach (var l in linkpost.list)
linkposts.Add(l);
ctr++;
// you can set this to a higher number, but it may blow up
if (ctr == 20)
mre.Set();
}
With the above code, you can set an arbitrary number (here i use 20 users' lists
of posts) and call the Set method on the ManualResetEvent which releases the
WaitOne call in the first block of code and proceeds to fix up the total results
and bind it to a GridView. For 20 users, I found this takes anywhere from 8 to
15 seconds which is acceptable. If you use a higher number than 20, it can either
take too long or it may blow up and there isn't really anything you can do about
that - The Otter API does have some rate limits, although no user key is required.
The rest of the code just performs some REGEX to create an <a href around each
link, and ensure there are no duplicates. The de-dupe code uses a custom Comparer:
public class MyComparer : IEqualityComparer<OtterApi.list>
{
public bool Equals(OtterApi.list x, OtterApi.list y)
{
return x.content == y.content;
}
public int GetHashCode(OtterApi.list obj)
{
return obj.content.GetHashCode();
}
}
The FixLinks code is as follows:
protected List<list> FixLinks(List<list> items)
{
List<list> newPosts = new List<list>();
Regex urlregex = new Regex(@"(http:\/\/([\w.]+\/?)\S*)",
RegexOptions.IgnoreCase | RegexOptions.Compiled);
foreach (var l in items)
{
if (l.content.Contains("http://"))
{
l.content = urlregex.Replace(l.content,
"<a href=\"$1\" target=\"_blank\">$1</a>");
newPosts.Add(l);
}
}
return newPosts;
}
Finally, I order the items by date descending (most recent first), store the
datasource in Session and set up Paging for the GridView so you can look at 10
rows at a time.
The result should look something like this:
By the way, "Rickasaurus" is Rick Minerich, a friend, fellow MVP, and real
F# genius with a new book out that I highly recommend: Professional F# 2.0
You can download the full Visual Studio 2010 Solution, which includes my OtterAPi wrapper project.