Some cumulative development experiences coupled with an interesting discussion forum thread in one of our forums here at Eggheadcafe.com have given me pause to consider some of the fallacious thinking surrounding the use of XML and sister technologies in the development marketplace today. In particular, we received a post from one of our members - who is obviously quite an experienced programmer in his own right - honestly asking "what are some of the practical uses of XML?" - a perfectly legitimate question that I am sure many of us often ask ourselves (or rather, more of us should be asking ourselves).

In the last 18 months or so, after having been involved with or responsible for the use of XML / XPATH / XSLT / XSD in no less than three separate development organizations or projects, I have seen some intelligent, well - organized and innovative uses of this new medium. I have also had some personal exposure to what I consider the ill-advised misuse of XML, resulting, in some cases, in both unnecessarily high development costs as well as significant loss of time-to-market.

I guess the simplest way to approach the question is first to decide what "XML" represents to you, in it's most basic form. You can then build your working definitions from there. To me, XML is nothing more than a "structured way to represent data". You can have almost any type of "XMLified" data - it could be a Shakespearean Play, records going into or out of a database, or some type of remote method invocation where a set of parameters is passed to some function somewhere, expecting back some sort of result. And of course, that's just for "starters".

The main advantage to XML, as I see it, is that it has the potential to be self-describing (Note I say "potential", not that it "is" self-describing). Not only can you give me your data, but you can - in the simplest form - surround this data with enclosing tags and attributes that tell me exactly what each piece of data is, what it means, and what it's relationship is to the overall dataset that it is a part of. And, you can give me this information in a single "Document" - that is, both the data itself and the descriptive information about this data.

What this should begin to mean is that if you and I do business together, and you use system "A" but I use system "B", as long as we can still agree on the schema of the data, our two systems should be able to talk to each other seamlessly. Now that can have real benefits for anybody who has a business and a thing called "customers". To "take it up a notch", if I'm in a business where there are a number of suppliers like me who have similar business data exchanging needs, and a number of similar or related customers, then if we can all agree on the same schema (or "data dictionary") and all agree to abide by it, we can make the schema available to all our members or interested parties and, provided all our data conforms to it (is "validated"), we can freely share our information without having to worry about who's using what program or operating system or database. If I send you my data - for whatever purpose, then as long as you've got a copy of our "rule book", you'll be able to program your system to efficiently receive and use the data wherever and whenever you see fit. Kind of like a data "Rosetta Stone", if you will, but even more useful. This concept has already been formalized and has seen a dramatic increase in use in many industries over the last few years.

Now that we've all agreed on a lingua franca for our data exchange, we can begin to get a lot more sophisticated. And so we have XSLT, XPATH, XSCHEMA, XQUERY, SOAP, UDDI, WSDL and so on. Obviously this must be working; I doubt the W3C has been working so hard to put out these recommendations for fun! Now the above description may seem simplistic and almost proletarian to many. However, I do this for a reason: let's take a look at a few of the fallacies.

Sometimes organizations decide they want to get on the "XML bandwagon", but they haven't taken the time to research and "think things through". You could have a development project that has a lot of users and a couple of databases and a lot of client - side "front ends", and you may possess perfectly adequate technology to engineer your project with existing proven programming techniques that your developers already understand, and you may have little or no need for disparate customers or partners to be able to hook into your system's internal data. Why then, should you use XML? I say, in this type of situation, that if you do use XML, then you better have some pretty good reasons to do so. Otherwise, your total development costs could be increased, and your time to market could be extended(especially if it's a first effort with XML) -- and the resultant product may end up being no better. In fact, there's a significant chance it could be worse - at least, performance and maintenance- wise.

Another fallacy I've picked up is developers thinking they need to use an XML Parser (like MSXML Core Services) in order to work with sending XML data over the wire to another party. Nothing could be further from the truth. Most modern RDBMS's can stream out XML in the exact schema of your predefined choosing, and this data can be sent out directly without ever having to incur the overhead of "loading" it into some parser or component. I have seen entire extra layers of business component complexity added to applications because people got the idea that they needed to either "XMLify", parse or otherwise manipulate the data coming out of the database separately, when with a little engineering, they might well have had the data XML talking directly to the databases. This is true of all the major database products today - DB2, Oracle, SQL Server and others. And let's not forget that we now have "next generation" databases that are "pure XML" - not relational. So I guess what I'm saying is that we need to have the facts and think - before we start doing the "XML disco dance".

I'm a big proponent of XML and related technologies. We use XML here at eggheadcafe.com. But, we don't "overuse" it. For example, our scrolling newsfeed that you see at the top of our home page is 100% XML / XSLT based. We found this particular need to be a very extensible way to combine different kinds of content from offsite along with our own content, and dynamically create and refresh its display in a useful and compact manner for our visitors. However, much of our site content does not use XML, and we have neither any particular advantage nor any plans to do so. Certainly, not just to be able to trumpet to the world "Look-- its XML!".

Another thing your organization may want to consider is that even though you may not currently have an overpowering need to use XML and related technologies for some or all of a particular project, in some cases it can still be productive to incur the extra cost and time to have your development organization "get up to speed" with using XML so that you can have a broader base of expertise for future projects. Also, as organizations begin to adopt the use of the .NET platform, you will find orders of magnitude more XML interoperability and tools in it to do some of the things that currently require a lot of expert, time-consuming programming code. For example, if I use .NET to expose my book catalog to you as a Webservice, virtually all the SOAP - compliant plumbing and XML infrastructure code I need to expose that service to you is done for me under the hood by the .NET Framework, freeing me to simply concentrate on writing the normal functions I would need to write anyway to expose my catalog database to you. So instead of three programmers, I might only need one. A recent real - world example is where the Microsoft .NET team took SUN's J2EE reference implementation project of an online Pet Store, and replicated it on the .NET platform with 1/3 the amount of code. They also claim it ran up to 28 times faster.

I guess the bottom line of what I am saying here is let's not "do XML" just so we can shout to the world that "It's XML" or it's "XYZ compliant" unless there is a distinct advantage or efficiency to be gained by doing so. Let's have a plan, get the facts figures and arithmetic, and have a sound basis for it.

Peter Bromberg is an independent consultant specializing in distributed .NET solutionsa Senior Programmer /Analyst at in Orlando and a co-developer of the NullSkull.com developer website. He can be reached at info@eggheadcafe.com