Cornelia Davis’ Weblog

WOW - Photojournalism at its best

I got there via a rather convoluted path (that I’ll spare you), and I cannot believe that I hadn’t seen this before but Chris Jordan is truly a talent and one with a strong social conscience to boot. Wow. His current work, called Running the Numbers: an American Self-Portrait, (not yet complete) is to produce art, not from a single photograph (though he does that brilliantly as well - see his work on Katrina), but by building a collage. What is unique about his work is that the number of objects he is depicting in an image, sometimes in the millions, is tied to some statistic - i.e. the number of paper grocery bags used in the US every hour (1.14 million). Some of the pictures are fairly uniform, depicting sheer volume with little form, in others he creates form from arrangements of said objects, often with a degree of irony (i.e. paper bags depicting trees).

While the pictures really do tell a thousand words (or sometimes, as Mark Twain would espouse, far fewer), I found this review of his work, with a few extra words from him quite interesting - in particular, view the interactive feature.

Have a look - you won’t be disappointed!

The JSON and XML debate and what worries me most about JSON…

…my concern is not one of features of JSON - please read on.

Okay, so I’ve spent a fair part of the afternoon today surfing through dozens of articles that address JSON vs. XML. And like any other “vs.” debate, these articles express capabilities of each and there are obviously “things” that one addresses differently than the other. I completely agree with what many have said, that it isn’t a matter of one being better than the other, rather they serve potentially different needs. The choice then on which to use when you are designing a system is dependent on what the usage scenarios are. Specifically in the JSON vs. XML discussion, how is the response message (that will come in JSON or XML or perhaps some other format) to be processed by the client?

There are a few people in the blogosphere who have commented this this affect, pointing out for example that the XML toolset allows for processing beyond deserialization, but I’m afraid that this point is getting lost in the noise. Mind you I loved Dare Obasanjo’s
Updated: XML Has Too Many Architecture Astronauts and while I confess to going too high in the atmosphere at times, I hear the point loud and clear that we cannot argue away something as popular as JSON with an architectural discussion - it’s popular because it enables something that users want (and no, I am not implying that JSON is fundamentally flawed and needs to be argued away). Absolutely right - JSON is seeing tremendous uptake because it makes things easy for a lot of developers. But that does not change the fact that there might be things we want to do with a received message beyond what JSON can support.

I don’t think that there is any argument that JSON, with supporting tools in Javascript and other languages, makes it really easy to take a serialized data structure and parse it, loading the data into a data structure accessible by the client application. What scares me is precisely the fact that this is what a large percentage of the population most often or even always wants to do. JSON is so popular because it allows developers to stay in their comfort zone.* Again, before you throw flames, I don’t mean to imply that the preferred programming paradigm, which remains largely procedural, is always bad, but I am saying that in certain cases there is something better. (*As an aside, this also scares me in the world of SOAP-based Web Services - WSDL is used to generate client side structures and XML is just used to transport the data over to the client structure.)

I love XSLT. I was telling a colleague recently about some of the things I’ve built with XSLT and he suggested that XSLT was a relatively rare skill set. Boy, I hope that isn’t so. When I first encountered XSLT I was sooo psyched - you see, I am a functional programmer at heart, having studied programming languages at IU under Dan Friedman - we did EVERYTHING in Scheme. I REALLY like the notion of declarative programming. I much prefer to express what I want done over precise details on how to do it. I am more than happy to have some framework (i.e. the Scheme interpreter) just “make it so.” As to the prevalence of XSLT use, I do think it is used a fair bit, AND, coming full circle back to the JSON/XML topic, there are a lot of programs where what I want to do on the client side (I don’t necessarily mean in the browser) is simply, or first, transform content, maybe into HTML for rendering, maybe into some normalized data format, maybe something else. Why do so many developers still insist on procedural programming when there is an alternative? A failing of XML/XSLT? Maybe. In any case, this is what scares me the most about JSON.

Admittedly I am an XML bigot and therefore probably fit the profile of someone who considers it “my precious XML” (as Dare puts it). I will, however, reiterate that I am also pragmatic enough to see the popularity and the value of JSON and when appropriate it will have its place in my designs. While David Megginson posted In praise of architecture astronauts (an excellent post in many respects) primarily to support his argument that JSON and XML are really not that different, they are both tree markup languages, he also shares my viewpoint stating

“In various situations, one syntax may have an advantage due to software support — for example, web browsers have built-in support for parsing XML or styling it using CSS, and they can convert JSON directly to JavaScript data structures using the eval() function…”

Exactly the point!

Upgraded Wordpress

I finally got around to upgrading my Wordpress version - if I had known that it would really be as easy as they said it would be I wouldn’t have waited this long. The upgrade was really not more than the 5 steps described here. It almost took me longer to find my DB username and password than it did to do the upgrade. First class organization there at Wordpress!! And with the upgrade the RSS feeds are now working. I’ll get to Atom as soon as I can.

Web 2.0 and SOA World

A couple of weeks ago I attended the SOA World Conference and Expo in New York City. I gave a presentation on interoperability for content management; I’ll post on that topic again very soon, starting with my SOA World presentation but in short, it’s about standards and they need to be service oriented. More on that later.

What I wanted to talk about today was the dominance of the Web 2.0 theme at that show. Admittedly, I did attend a lot of talks in the Web 2.0/Ajax and SOA track so my perspective may be a bit skewed, but it was also present in many of the general sessions and keynotes. Is a Web 2.0 track at SOA World a cheap ploy on behalf of the conference organizers to get on a “hot” topic? Absolutely not! Let me take one step back in order to take several more forward.

SOA is fundamentally about loose coupling, heterogeneity and composition. It also involves governance, messaging infrastructures and sometime registries. It shouldn’t be lost on too many folks who have been working in the software industry for more than a couple of years that many of the themes of SOA are not totally new; we’ve been talking about abstractions, reusability and development methodologies for as long as I have been in computing (and I’m sure much longer than that).

When I started my career 20 years ago I was programming embedded systems – image processing, we were processing 30 frames per second. Everything I coded ran on a single processor. Sometimes there were a couple of processors that ran things in parallel but I remember timing functional components on the target processor and then manually arranging what would run on which processor and it what order. We were, of course, distributing the processing load across CPUs so that it was relatively even and and aligning processing so that a “module” depending on output from some other process wasn’t left waiting on results for too long. We did have a bit of shared memory but managing access was a pretty tractable problem in that only two or three processors were accessing it and we pretty much knew when they would be relative to one another. If I remember correctly we implemented some simple semaphores and they worked just fine.

My code development was governed, I modeled and abstracted plenty of things – things were decoupled where they needed to be, we had a way of multiple processes communicating with one another. Heck we even had virtualization – we called our virtualizations “simulations”. My point is this. What changes all the time, and often very radically, is the environment in which our applications run and that environment continues to become increasingly complex. Where I once was sure that all of my processes mapped their code abstractions to memory the same way, I am now concerned with how my data models are represented across processes because those processes are implemented in different languages and are running on different platforms.

Okay, so SOA is an architectural approach that operates in an environment where we have a distributed network of computers, running a variety of operating systems, hosting a variety of applications, implemented in a variety of languages, oh and they all have to talk to one another. Whew – this is not your grandfather’s computing environment. But then here’s the thing. For the most part, SOA has been applied in the enterprise. The enterprise, I would argue, is still a constrained environment. Not as constrained as the embedded environment I coded to 20 years ago but it still has some constraints. Compared to what? Compared to the world wide web.

Ah so now I finally get to the point on what it is about Web 2.0 makes it an appropriate topic for SOA World 2007. The “enterprise” IT systems aren’t just in the corporation anymore (haven’t been for a while) – partners, vendors, customers all need to connect with the enterprise. I won’t make this already very long post any longer by arguing that utilizing the infrastructure of the web to make those connections is compelling – I’ll assume we all agree on that one. So taken in this context, where our operating environment is the web, the parts of Web 2.0 that are relevant to an SOA discussion are the architectural ones. (There are other Web 2.0 concerns like the emergence of the “procumer” that I won’t cover in this post, though I’m sure to in the future.)

Ajax and RIA Sure we want to improve the user experience while running applications on the web but there is more. Just because the point of integration is not necessarily deep within the enterprise, or even happening on servers doesn’t mean that governance goes away. Is there governance in the Web 2.0? Maybe. Yes, in places. Part of it comes through standards – Atom, for example. But there are also a lot of “accidental architectures out there”. We’ve got some work to do here.

Mashups Yes, I specifically address this separate from Ajax because mashups to me have more to do with the notion of composition than they do with where the composition is happening. A colleague of mine recently created a mashup and reported (no surprise to me) that the process was painful in places – specifically the data models of the two “modules” he was composing were either poorly or entirely undocumented. Rob High had a slide where he listed some core tenets of SOA – right after the line that said “loosely coupled” was a line that said “strongly coherent”. In the Web 2.0 world we aren’t quite there yet.

Agile development and deployment Frequent releases or the perpetual beta. How do we govern in an agile environment?

Heterogeneity Yes, okay, been addressing this with SOA for some time but what is new in the Web 2.0 is that my runtime is the web. What type of web services runtime can I depend on in the wild, wild web? Am I all about REST? What about SOAP and WS-*? As an industry we are working on these answers.

These topics and then some kept the conference very interesting.

Atom 1.0 Overview

I’ve been looking at Atom a bit and just as it seems to be with many all standards, the first thing is to figure out where things are in the standardization process. I thought I had it figured out a couple of weeks ago, and started a blog post on the subject (which I never quite finished) but now I stumble upon more. Something pretty significant as it relates to Atom, surprising that I didn’t find it before (read, “how can I find related content if someone hasn’t explicitly placed the relationship in the content that I do know about?”)

Anyway, here is the overview that I’ve managed to build:

• While the text version of the document is the only normative one, I have found this HTML version to be much friendlier on the eye

“Originally, I was searching for an acronym. One that perhaps combined words like Proposal, Atom, Change, Enhancement. In the process I found a word that connoted forward motion[1], consensus[2], and peace[3].” - Sam Ruby

Web Services enabling the non-programmer

We are headed in the right direction! After years, actually decades, of talking about “high-level” programming languages we have achieved the first major increment since the time where programmers were given a means to program in something other than assembly language.

I’m a computer scientist. I love programming languages. The more sophisticated (complicated) the better. Okay, so I am (and many of you our there are) a bit odd. People I meet outside of work, however, don’t usually peg me for a developer, which I like to think is a sign that I’m still at least a bit grounded in reality. I see the value in enabling a broader audience to build the tools that they need.

Twenty plus years ago when I was a freshman in computer science at Cal State University Northridge the high level language was something like Pascal. I did take a course in assembly language (”in what?” say you 20-somethings ;)) and Pascal was certainly much “higher” than that. Less than a decade later I was at Indiana University studying programming languages and we were looking at things like Smalltalk. While Smalltalk goes a bit higher level, arguably we are still targeting someone who is a programmer.

Did we get there with web services? I say not quite. For the first couple of years that I worked with web services, in the early 2000s, I used to refer to the “mythical” business analyst who would be composing web services into the higher level services, processes and applications they needed. My calling them “mythical” wasn’t a slight on the analyst, rather it was a slight on those of us that were providing them with the tools they needed. For example, somewhere around 2002, eRoom (later acquired by Documentum, and then by EMC) released it’s first web services interface. We talked about composite applications then and the tremendous advantage that these components offered. To leverage the interface you first used the WSDL to generate (say Java) proxy classes and now that you had a web services framework in play that offered run time support your life was “easy”. Yeah, right. Okay, the software developers were happy but the business analyst who could now build up higher value components was still mythical.

Enter Web 2.0 and things have finally changed. You want the evidence? Check this out.

I am working on a presentation for our upcoming EMC Software Developers Conference and on one of my slides I am talking about the soon to be released Documentum Foundation Services - a web services based interface to our content management system. Right now the slide only has text and I just want to put some type of graphic on it in an attempt to keep the neurons of my audience firing. Just a little icon that represents web services - what I want is simple, the equivalent of the COM lollypop diagram for web services. When I google “web services icons”, the first hit sounds promising “Web Services Icons +” and I have to tell you I’m thrilled. Not because I found what I am looking for but because I found something better! The icons I found are to things like Flickr, RSS, del.icio.us and digg and it dawns on me that this is the right view of web services for a far greater number of people than the numbers of traditional programmers out there. The icons for these web services don’t say a thing about how the services are implemented, no WSDL or REST, no SOAP, etc. but these things mean something to a great many people who are doing some very cools things with these services - truly creating composite applications and higher value!

So my search for the right icon for the programmer continues… but what is the right icon? Hmm….

Time as a service

The United States is changing its daylight savings time (DST) dates this year to begin three weeks earlier and end one week later than they have in previous years. I’m sure that everyone agrees (actually, I’m more sure that someone disagrees, preferring to control this entirely themselves) that it’s nice not to have to remember to change our time on numerous devices such as personal computers, mobile devices or even watches when DST changes occur. Each of these systems has a calendar, that has the DST (and other events), coded in enabling the system to automatically update clocks for us. Now those windows have changed and this is causing all sorts of problems in applications that were constructed and deployed before this decision was made. In a few weeks my calendar will think that it is an hour earlier than I (and my many colleagues) think it is. Coordinating meetings for this three week period is already proving challenging.

This is an ideal example of something that should be packaged as a service. Sure, there still exist devices that are not connected to the cloud (my watch, for example), but for those that are connected, establishing the correct time, relative to location/timezone is something that should be provided as a service. This should be the last time that changes such as this should cause such a major headache.

The power of participation (Collaboration 2.0, Reed’s Law and Metcalfe’s Law)

I’ve just been reading the Web 2.0 Principles and Best Practices report from O’Reilly (this is a for fee publication - for an abridged and somewhat older piece see What is Web 2.0?). While the number of typos within it is somewhat disappointing, overall there are some interesting conclusions and insights - not sure what my company paid (I’m sure the payment info can be found with the original link), but I’ve found value in it. There are lots of things that had me scribbling in the margins, the first one that I’ve taken a bit of time to think about is the on a section titled Harnessing Collective Intelligence.

Without getting into a discussion on how to actually measure the “value” of applications I have some thoughts on the contained discussion around Metcalfe’s Law and Reed’s Law. The former says the value of information is proportional to the square of the number of nodes in a network, the latter says its proportional to 2^^n, where n is the number of nodes. In this discussion the nodes are people, people who are using the application.

There are lots of examples in today’s web where whatever the services offered by the application, it’s inarguably valuable because lots of people are using it. Where would Wikipedia be if there weren’t a lot of people contributing content? Of course, there are different types of useage, contributors, consumers, etc. and this report cites some interesting statistics around, for example, what percentage of the registered Wikipedia users are contributors - around 7% of one million. Okay, so here’s what I am thinking about…

Wiki’s are popular - really popular. And, again using Wikipedia as an example, very successful, very valuable. There a numerous features that characterize Wiki’s and while it depends on the specific deployment, most of the time self-registration is one of the simple but most powerful features available. In fact, the whole point of this section of the report is that increasing the numbers participating is absolutely related to the value. What I am thinking about in particular is how this phenomenon changes when we move from Web 2.0 to Enterprise 2.0. Going back to the Wikipedia statistics above, even at 7% the number of people contributing is 70,000 - that is more than twice the number of people working for my employer (EMC has around 31,000 employees). Sure, my colleagues are getting paid to produce content, in contrast to the predominantly volunteer “workforce” of Wikipedia, so I would expect the percentage of active participants to be significantly higher. And sure, we are not trying to capture information about everything like Wikipedia is. The number of nodes, however, is orders of magnitude smaller (and I would argue that the problems we are trying to solve are not orders of magnitude simpler ;-)).

My whole point is that the effect of the number of participants is definitely different in the enterprise than it is out on the world wide web. I’m not implying that the enterprise needs dictate different applications, but calculating value will definitely involve other variables and adjusted formulas. There absolutely is value of social networks in the enterprise, yet we have to be thoughtful about how we harness it.

JSR-170 (JSR-283) as an in the application server interface

I was recently listening to Jon Udell’s podcast conversation with Roy Fielding about HTTP, REST, WebDAV, JSR 170 and Waka. Lots of interesting tidbits in there but one in particular that I wanted to post on is with respect to JSR-170 and JSR-283; I’ve not heard these charactizations put quite as clearly before. Roy, being seen by many as the father of the REST architectural style (or at the very least, the one who captured and documented this in his PhD dissertation) stated that the JSR-170 and subsequent JSR-283 efforts are an attempt to take the engineering and architectural principles that had been applied to the web and apply them to communication inside the application server. The environment within the application server is clearly different from that of the web. Roy points out, for example that within the application server we don’t have the concerns around latency that we have when passing messages around the web, and JCR is optimized for the within the app server environment. He conceeds that as soon as JCR version 1.0 was complete then folks wanted to remote it - his answer put WebDAV over JCR. I’m in violent agreement on all points with the exception of the last - that is the subject of a future post.

The punch line for ontologies

I’m just getting to the point where I understand enough about ontologies to see the challenges that lie ahead for me as an evangalist for them. So when I receive a gift such as this, an absolutely compelling and easy to understand example of the value of ontologies, it makes me speak out loud in spite of the fact that I am sitting alone in my home office. I’m viewing a video taped presentation by Cycorp CEO, Doug Lenat entitled “Computers versus Common Sense.” It is really an outstanding talk, I encourage you to view it, but here’s a quote:

A lot of times people ask me things like how many predicates , and concepts, and assertions ar there all together in the knowlege base? And so I have a slide like this to forestall those questions. But, really, this is a red herring, and you shouldn’t really care about these numbers. To give you an example of why you shouldn’t, a small number of what we call sibling disjoint assertions in the knowledge base take the place of billions of class level disjointness assertinons and really hundreds of trillions of instance level non-set membership assertions.

That’s it - that’s the punchline!! If we model the relationships right (the “sibling disjoint assertions” in this case) then we save ourselves a whole lot of work. Can simple OO class hierarchies do this? I don’t think so but I promise to have a more concrete answer in the future - a proof if you will.

Anyway, the quote goes on, getting more concrete.

So like, if we have a question like, “Is any seagull also a moose?” [snip] And if Cyc knows, let’s say, 10,000 kinds of animals, that means there are about 100 million questions like this it ought to be able to answer. So option one is we could add 100 million assertions to the system [snip]. So option two is we could basically add 50 million disjoint with assertions and one single assertion that says disjoint with is symmetric. A better option is to add 10,000 Lanayan bilogical taxonomy assertions and one single sibling disjointness assertion, which basically says if you’ve got any two taxons, and you don’t know that one is a specialization of teh other, assume they’re disjoint. [snip] And with these 10,001 rules you can answer the same 100 million queries.

(Okay, so some of this was still a little abstract but you get the point )