Cornelia Davis' Weblog

Upgrading Wordpress

… or Why I Love this Software

For the handful of you who have actually been reading my blog on occasion (hi Dad) you might have noticed that not only have the posts been scarce, but the site itself was sorely neglected. So recently, where I thought I had more to share, I became more and more bothered by the state of my blogging software. So a few weeks ago I sat down to upgrade.

For a short time I considered using this as a good opportunity to write my own blogging software. I reasoned that since I was building it for only my consumption (the administrative functions, that is) that I wouldn’t have to build all of the bells and whistles into it and might be able to build it fairly quickly. Of course, my plan was to leverage the XMLREST framework that I’ve talked about here and here. But then I started thinking about categories, and tags, and spam filters, and… and I decided that I’d likely get bored long before I created something that was really functional, so I came back to the task of upgrading Wordpress.

The Wordpress docs point out that the auto upgrade, “which will work for most people”, is the easiest – and I couldn’t agree more, having used it on other occasions – but, well, I’ve often been accused of not being like most people. The auto upgrade option is available from version 2.7 on but, eegahds!!, I was on 2.2.something (I’ve really neglected things!). So I had to do the manual.

The long story involves:

figuring that an upgrade would be very problematic because of how large of a gap there was between what I had installed and the latest version (have you ever heard of enterprise software skipping releases and have a decent upgrade experience?!), I set up another blog side-by side, using the Wordpress export and import functions. This worked fabulously, but would break any permalinks that folks (hi again Dad) might have to the old entries, because my old permalinks had post IDs in them (and the post IDs changed on import).
and then when I sat down to implement the redirects I thought, “what the heck, let’s just try upgrading the old installation, if it fails, I’ve already got the new site up and running.”

I did as the Wordpress docs advised, downloaded and unzipped the latest version, used my ftp client to upload that on top of my existing deployment (overwriting files) and then accessed the admin page. Indeed, it asked to upgrade the DB, which I did, and voila – my blog is upgraded. I had spent some time setting up a new theme which I brought over from the parallel, temporary blog, only had to tweak a few things by hand and I was golden.

All of this is to say – this is software that is easy to use, has ample documentation and works as advertised. This is something that all technology vendors should strive for! I love Wordpress.

A Software Dude(tte) Plays with Hardware

… or Dealing with a Loud Computer Fan.

I’ve been known to pop a card in and out of a desktop, or swap out a hard drive or add memory in my laptop but let’s face it, these things are pretty darn brain-dead so I don’t get a lot of points for that. So what I learned this week about some hardware is noteworthy – at least for me.

The whole things starts with me eating the EMC, or rather VMWare dogfood. Over the last 6 months or so I’ve moved more and more of my environment over to VMs. My goals are slightly different than VMWare usage in most data centers; in data centers (one of) the goal is often consolidation – running multiple VMs on the same hardware – hardware that otherwise would be underutilized. My use of VMware is quite the opposite; I don’t have enough hardware to run all of the VMs that I have, though I rarely have to run more than a couple at a time. I am heavily leveraging VMs to help me with dev and test; I have one image that is my official EMC image that includes the office suite, including email, and then any number of dev images that have different technology stacks on them. I’ve got my XML REST Framework dev image and another for test, I’ve got an image for working with a bunch of latest/greatest Apache products like CXF, ServiceMix and Camel, etc. But that’s just it, my laptop doesn’t have enough horsepower to run two images at once.

So under my desk I had an old machine acting as a footrest. I had replaced that machine with a more full featured desktop a bit more than a year ago – one that could handle some basic home video processing. The old one was a very basic model – absolutely no bells and whistles. But I thought maybe it would have enough horsepower to run one of my images. Over the weekend I loaded up Ubutnu 10.04 and then VMPlayer, transfered over my VM files and fired it up. At this point it all looks good, but it doesn’t sound good. Every few seconds the machine fans kick into overdrive and the resultant sound makes this a non-option. So this self-proclaimed software dude starts googling. I’ll spare you all of the details and get right to the summary.

The fan was kicking in exactly as it should – notching up when the CPU temp was passing a certain threshold
The CPU temp, when idle was running around 57C and was kicking up to the 61-63C range when under load. (Some digging and I found that the max temp recommendation is in the upper 70s for my processor)
I thought briefly about trying to change the trigger points for the different levels of fan speed but found that the current configuration was pretty standard
I took out a graphics card that I had previously put in (don’t need it to read email) and that lowered the temp by a couple of degrees

At this point it was a little better but still not where I wanted it to be. I started looking at heatsinks for a possible replacement. I had also seen a lot of recommendations about cleaning out the dust. Yeah, there was a fair bit of dust in and around the fans and the heatsink so I went down and bought a can of compressed air, blew everything out and hooked it all back up.

And OMG! The idling temp went from 57C to 31C and the under load temp went to the 37-42C range!!! I was thinking “what’s a little dust?” but a little dust makes a big difference! (remember, I’m a software geek) So if your fan is kicking into overdrive all of the time, FIRST, try cleaning out the dust – that may be all you need to do.

Second Version of the XML REST Framework Released

To coincide with EMC World and the presentation I gave on RESTful SOA, I’m delighted to report that we have released the second version of the XML REST Framework. As I described in a previous post, the April 18 release introduced a framework that allowed a services developer to produce RESTful services over EMC Documentum xDB by “declaring” the resource model, including definition of the uniform interface and media type handling, with Jax-RS annotated POJOs and by using XQuery to implement the operations of the service. This was a good start, yet didn’t really go far enough to even call the resulting services RESTful because the resource representations that flowed into and out of the services where completely devoid of hyperlinks.

The new version of the framework now provides the developer a mechanism for insertion of hyperlinks, again leveraging XML standards to do so – this time I’m talking XSLT. Just as with the first version of the framework, the code that goes into your Java POJOs is very thin, essentially calling into the framework that in turn invokes a series of XML-centric steps. The job of the developer then is to inject (using Spring dependency injection) the XML processing artifacts that achieve the required functionality. With the first version of the framework that was an XQuery per operation, and now in version two it is an XQuery plus one or more XSLTs. In version one of the framework you there was only one option available – to execute a single XQuery; in this new version you have the choice of three different “pipelines” – XQuery alone, XQuery followed by an XSLT and XQuery wrapped by XSLTs before and after. Included in the sample application that is part of the framework release are some XSLTs that make it very easy for the developer to simply declare the links they want inserted – no great XSLT expertise required.

The EDN postings go into a good bit of detail on how you can leverage the framework. Each posting has two main parts – code and documentation. If you haven’t already downloaded and installed the code from the first release then go directly to the code in the second release; it is a superset of the former. For documentation, please start with the first release – it describes the base framework – and then move on to the docs for the second release; the latter adds to the former rather than replacing it.

Finally, yes, there is more to come. We have a very early implementation of the next version of the framework running in the lab (thanks to my very able colleague Xiaotao Liu). Stay tuned, I’ll give you some hints as to what will be included over subsequent posts. I promise it will be fun.

RESTful SOA at EMC World 2010 and …

It’s been a really good week for me. I didn’t attend EMC World last year because, well, I didn’t really have much to talk about. But in the last year the work that we have been doing in the CTO Office around an integration architecture that has emerged as Service-oriented and RESTful has reached a point where there is not only plenty to talk about but, wow, there is a stream of additional work that follows on from this. I’ll be busy for a while. Aside from having the opportunity to attend some great sessions and connect with a whole slew of really talented people, I also had the opportunity to present a session in the Developer Track of the conference.

The title of the presentation was RESTful SOA, Developing RESTful Web Services for Distributed Applications (Like Cloud). (Hmm, as I type that title now I realize, of course, that “Cloud” is not an application, but I think you know that I mean “like those running in the cloud.”) I was scheduled in the last slot on the last day of the conference – OUCH! All week I spoke with people who said “I’d love to see your presentation but I’ll already be on my way home.” So I expected around 10 people, half of which would be colleagues, and low and behold, I think there were more than 50 in the room!! That really speaks to the interest that people have in this still emerging architectural style.

After spending just a moment to introduce what we do in the Architecture Group of the CTO Office (work on an integration architecture) and why we are embracing a RESTful approach, the presentation broke out into two almost equal halves.

In the first chunk I talked about REST in general. I still feel that most folks understand only a part of the four fundamental cornerstones of a RESTful architecture. Those cornerstones are:

A resource centricity and addressability
Abiding by a (the?) uniform interface
The use of media types for resource representation
And hyperlinking

For each of these cornerstones I covered what it is and why it’s important. In the discussion on the uniform interface we drilled down the the characteristics of safe, idempotent and cacheable (Joe Gregorio talks about “zippable” as well – I don’t have any experience with that one yet so have little to say at this point). Rather than writing a whole chapter detailing these things let me give you a brief analogy – the web as we know it today. It works as well as it does, it scales as well as it does because it is RESTful. You can access web resources (web pages) via their URLs, new web sites can be added with no performance degradation in part because of the layered web and cacheable operation results, the network can be unreliable yet applications still work because of safe and idempotent operations, a range of different devices and application types all work against the web, and you can navigate to related resources, without going through some central point of control, by simply invoking embedded hyperlinks. Way cool.

Following this discussion on RESTful architectures in general, which took us through the first half of the hour, I moved on into pragmatics and talked about an approach for implementing RESTful services. Using the steps outlined in Richardson and Ruby we quickly covered how to implement services that achieve resource orientation and addressability, implement a uniform interface and support various media types. We use Java and Jax-RS annotated POJOs to declare our services and for implementation I showed how you can use XML technologies such as an XML database like EMC Documentum xDB and an XQuery engine (a part of xDB) to avoid a lot of grungy Java code. The steps for the services developer are very prescriptive; assuming the resource model and physical model are designed she will:

Declare the resource model and uniform interface; a Java class per resource, a method per uniform interface operation supported.
Craft an xQuery to mediate from the resource model to the physical model and to implement the service operations.
And glue it all together using spring.

The first version of the XML REST Framework we have created can be found on the EMC Developer Network and the article that delivers the release goes into much more detail on how all this works.

BUT, we’re not done. The resource representations we produce and consume using the first version of the framework do not include hyperlinks. Can you imagine a Web where you have to go back to some index each time you want to access another web page or complete your Amazon purchase? Or one where you have to know the exact URL for EVERY page you want to access? Well, without links in resource representations that is exactly what we are forcing the client programs to do. So the final part of the presentation covered the extensions we made to the first version of the framework to enable the services developer to easily insert hyperlinks to related resources within their resource representations, using XSLT. For details on how this all works please see the second version of the XML REST Framework, also found on the EMC Developer Network.

Unfortunately, with so much material to cover I had little time left for questions during the session but fielded quite a few after we had broken up. If you were in the session, or even if you weren’t, and have any questions, fire away! Thanks to everyone who attended EMC World this year – and especially those of you who stuck around until the end.

Building Domain Specific RESTful Services with XML Technologies

Some colleagues and I have been very busy in the last months putting together a framework to do just that. Let me briefly back up a bit…

One of the things that I’ve spent some time on in the last couple of years is the Content Management Interoperability Services (CMIS) standards development effort. One of the styles for exposing this set of services is through a RESTful interface. Through incremental improvements happening through the community development process the result is a pretty nice, and quite RESTful, interface. That said, Roy Fielding levied some pretty harsh criticisms toward it when it was first made public – the vast majority of which are now addressed. One issue from Roy’s list remains but is, frankly, something that CMIS didn’t intend to address, and that is the concern that the services exposed over a CMIS compliant repository do NOT expose the domain specific semantics in the resources. Roy is, of course, dead on on this, what CMIS exposes is the repository model, not your domain specific model. Yes, CMIS does provide a service where object types can be obtained by a client, but those details are not a part of the protocol, rather they are tunneled through the protocol. And, while CMIS resource representations do include hyperlinks between related resources, again the related resources are repository resources (i.e. this document is contained in this folder) not domain specific resources.

This is an all too common pattern – we build interfaces that expose the repository models – and, of course, this is not at all without use as it does help consumers more rapidly build applications. Unfortunately, it has the unintended consequence of encouraging architectures where all of the domain specific details are all too often included far too far from the server. My boss calls this the “semantic gorp”, and in the best case it lands on an app server somewhere embedded in some services that in turn call the repository-model-centric services, but in the worst case it ends up far further away. This causes all sorts of headaches, which I’ll go into details on in subsequent posts… for now I’ll just call on your intuition to buy that point.

(Okay, so that wasn’t entirely brief, but important context nonetheless.)

In the deployment of ECM systems you go through a process where you design your domain specific artifacts, content types, lifecycles, etc. You then (ideally) configure the ECM to provide services for those domain specific entities. Wouldn’t it be cool if you got domain specific services as a result of that process? Now, don’t misunderstand, we haven’t built that (yet), but what we just published to the EMC Developer Network is a step in that direction.

We have built a framework that allows for the development of domain specific RESTful services over a repository that supports xQuery. In particular, the implementation available for download operates over EMC Documentum xDB. Using the framework, all of the details of connecting to xDB are abstracted away from the developer, and he or she produces the RESTful services by

“declaring” the resource model in Jax-RS annotated Java classes
defining the uniform interface with more Jax-RS annotations (i.e. @GET, @PUT, @POST, @DELETE, etc.)
producing an XQuery to interface each RESTful operation for each resource to the physical manifestation of that resource
and binds it all together in a spring configuration file

In this first release of the framework we are mostly leveraging Jax-RS for REST features, which means that one of the most important features of RESTful interfaces, hyperlinking, is lacking. This will follow in version 2 of the framework in a few weeks.

One of the primary drivers for this work was to simplify the development process for these services – if they are easy to build then we should see more domain specific services where otherwise, generic services might have been. This is already proving true in that we are leveraging the framework on several projects internally at EMC. The developers were able to concentrate on their resource models, representations and interactions rather than getting bogged down in the details on how to implement them. Sure, they still need to touch java code and spring configuration files, but it’s not a huge leap to see that tooling could shelter the developer from those details – we’re just focusing on the developer model, not the tooling(, for now?).

The framework, including source code, plus a sample application leveraging the framework is available here. Also, for any of you that will be at EMC World in a few weeks, I am giving a talk on this framework – in the highly coveted last slot of the conference in the developer track (Thursday afternoon at 1PM). Hope to see you there.

All the little references when you copy an eclipse project

Okay, maybe I’m not supposed to copy eclipse projects this way, but I simply wanted to make a copy of an eclipse project that had already been appropriately configured to include a number of libraries, etc., so that I didn’t have to go through all that config again. Seems like it should work, intuitively, and I guess we’re close, but there were a few tweaks I had to make by hand to get everything up and running the way I wanted.

Brief overview of my eclipse environment and project:

I am running eclipse V3.4.2 with WTP (and a couple other plugins that are inconsequential)
I’ve configured a Tomcat server so that I can run from within eclipse – i.e. Run As…->Run on Server
My project is a Dynamic Web Project

First thing I do after copying the project from, say, “MyProject” to “MyProject2″ is right click on the project, select Properties, go to Web Project Settings and update the Context Root; I want my new service to run at the URL …/services/MyProject2 instead of …/services/MyProject so I set the context root to MyProject2. I do a Project->Clean and then build the project and all looks fine. When I right-click Run As->Run on Server, select my Tomcat Server, click Next, here is the rub: Instead of showing MyProject2 in the list of configured projects I see MyProject listed twice. Turns out that I can select one of them and click the Remove All button, continue with the deployment (even with the old project name there) and everything runs on Tomcat as you would expect – the server responds to requests at url …/services/MyProject2.

So the question is, why isn’t the new project name picked up in the right places? I believe it is because a key settings file is written at project creation time via the wtp plugins, and writes the project name into that file, but when I created the project via a copy of the old, eclipse does not update that file. I found the file, but try as I might I cannot find a config page anywhere that allows me to update it – I guess this is not too surprising as, in theory, this is just bound to the eclipse project name and it would be confusing for users to diverge from that naming convention (certainly is confusing me).

The solution:

The file is the org.eclipse.wst.common.component file that is in the / /.settings directory. Close eclipse, open that file (it’s an XML file) and update the line that reads:

<wb-module deploy-name="MyProject">

<wb-module deploy-name="MyProject2">

and start eclipse. At this point I had to do another Project->Clean and build (else the context root for the web project was a mess) and then the last thing you need to do is clean the Tomcat work directory by right clicking on the server in the “Servers” tab and selecting Clean Tomcat Work Directory… Now, try running the app again via Run As->Run on Server and when you’ve selected the server and clicked Next you will see both MyProject and MyProject2 in the Configured Projects list – not to worry, the MyProject is just a left over reference from the last time you ran the old project and can be removed by selecting and clicking Remove All.

And now you’ve managed to update what I believe is the last reference to the old project name from your new project.

Basic Tutorial in Open eHealth IPF V2.0

I’ve just upgraded to V2.0 and to verify my install (really just checking my eclipse config for IPF) wanted to do a quick run of the basic tutorial. The first part of it (before extending it to set up an HTTP endpoint) was fine but there is a documentation bug for the latter part of the tutorial. The code shown in the tutorial that needs to be added to the SampleModelExtension.groovy file is missing an import; the file should be as follows:

package org.openehealth.tutorial import org.apache.camel.model.ProcessorDefinition import org.apache.camel.Exchange class SampleModelExtension { static extensions = { ProcessorDefinition.metaClass.reverse = { delegate.transmogrify { it.reverse() } } ProcessorDefinition.metaClass.setFileHeaderFrom = { String sourceHeader -> delegate.setHeader(Exchange.FILE_NAME) { exchange -> def destination = exchange.in.headers."$sourceHeader" destination ? "${destination}.txt" : 'default.txt' } } } }

Threw me for a bit of a loop because the camel javadocs are still describing the pre-camel-v2.0 Exchange interface which doesn’t include a FILE_NAME field.

Now I just have to find the time to figure out how to file doc bugs for IPF and Camel…

Open eHealth IPF – Getting the HL7 Tutorial Running (in version 1.7 of the IPF)

The Open eHealth Integration Platform is pretty darn cool. In addition to enhancements that serve the healthcare domain, it also extends Camel with a great number of features that are of general value, including a Groovy based DSL that supports the likes of closures (!!) and more. I’m liking it.

I’ve been looking at the available tutorials and didn’t get too, too far in before I hit a snag – in the third tutorial on the page, the HL7 Processing tutorial. The solution is very, very simple, and is given at the very end of this post, though finding the solution took me on a very instructive tour through the platform and a bunch of embedded packages. Fortunately, I had the benefit of time (the boys went to my son’s homecoming game last night) and absorbed a lot on the scenic route. Come along…

Following the tutorial instructions to the letter resulted in an error when I tried to run it (unit testing run either from within eclipse or by running mvn test at the command line):

SEVERE: Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.DependencyInjectionTestExecutionListener@3bfc47] to prepare test instance [org.openehealth.tutorial.SampleRouteTest@2f996f] java.lang.IllegalStateException: Failed to load ApplicationContext at org.springframework.test.context.TestContext.getApplicationContext(TestContext.java:201) ... [snip - a big stack trace with some cascading "Caused by:..." messages the last of which was the root cause] ... Caused by: ca.uhn.hl7v2.HL7Exception: No map found for version 2.2. Only the following are available: [2.5, 2.3.1, 2.4, 2.6, 2.1, 2.5.1, 2.0D, 2.0, 2.3, 2.2] at ca.uhn.hl7v2.parser.Parser.getMessageStructureForEvent(Parser.java:352) ... [snip - a bunch more stack trace]

This error was getting thrown at bean instantiation time – the init method for the validationContext was failing. Of course, this error message is puzzling in two respects. First, most obviously, an error message that says “nothing available for x, only x, y and z are available” is rather contradictory. Second, when we look into the code for initialization of the validationContext we see that what we are doing is starting with a DefaultValidationContext and then we fill it in with the rules that are defined in the beans declared in the context.xml file. The DefaultValidationContext has a map, initially empty, from the HL7v2 versions (i.e. 2.1, 2.2, etc.) to the set of rules that apply to that version. Put that together with the error message above and we are easily mislead into thinking the system is complaining about some problem with the ruleMap in the DefaultValidationContext. Yes, mislead. This is NOT the map that is causing the problem.

What happens at bean initialization time starts out exactly as I describe in the previous paragraph – the validationContext bean is instantiated and during initialization a DefaultValidationContext is created at the rules specified in the context.xml file are added. As the rules are added the forContext Groovy method is called, a RuleBuilder is instantiated and the forVersion method is called on it – that is, the rules are basically parsed (more on this in a moment). The rules in the defaultTypeRules are not a problem and are loaded without issue. The difference between these and myCustomRules, which are not loading, is that the custom rules we’ve defined are using the abstractSyntax method (those in the defaultTypeRules are specifying rules using different approaches).

And here is the key…

What happens when you define a rule using abstractSyntax is that the rule is validated against the abstractsyntax of the designated HL7 message version. That’s right, the IPF is doing a bunch of work here to make sure you’ve specified rules that are relevant against the version of HL7v2 that you are targeting. So, for example, if you tried to define a rule against an ADT_A25 message and you targeted it forVersion(‘2.2′), you’re cool. If however you target it forVersion(‘2.1′) then the rule itself won’t validate because version 2.1 only has ADT messages up through ADT_A24. And it even does validation into the message formats. This is a huge value add, it’s a point of model governance that I wasn’t necessarily expecting to have in IPF.

So, back to the error message. The exception is thrown in ca.uhn.hl7v2.parser.Parser.getMessageStructureForEvent, the Parser does, indeed, contain a map from version numbers to messageStructures and the exception occurs because, indeed, there is no entry for version 2.2 (or any other version number for that matter). The question then is from where and when are they supposed to have been loaded. We don’t have to look too far, the Parser class includes a method called loadMessageStructures() and looking at the code in there we see:

for (int i = 0; i < versions.length; i++) { String resource = "ca/uhn/hl7v2/parser/eventmap/" + versions[i] + ".properties"; InputStream in = Parser.class.getClassLoader().getResourceAsStream(resource); ...

That is, for each version (the list is hardcoded in a class variable) load up the structure from a properties file with the right name. I knew then that this had to be a packaging issue, and, assuming the tutorial once worked as written, a result of some recent change. Digging through jars, this specifically had to do with HAPI libraries, and vaguely remembering seeing something about HAPI versions, here:

As of IPF 1.7-m3, the underlying HAPI library has been updated. The HL7 version-dependent classes have to be included separately.

was the solution.

All that is to say that the only thing you need to do beyond what is described in the tutorial is add the following lines in the dependencies section of the pom.xml:

<dependency> <groupid>ca.uhn.hapi</groupid> <artifactid>hapi-structures-v22</artifactid> <version>0.6</version> </dependency>

Increasing VMware disk capacity

There are several helpful articles out there, but I had to use a combination of two of them to get it done. There are two steps:

You need to increase the size of the HD from VMware’s viewpoint.
And (MAYBE) you will need to use tools of the guest OS (the OS of the image) to repartition things.

I found two resources on line that helped me with this:

An article posted on the VMware forums: http://communities.vmware.com/docs/DOC-7471
A blog post from Novell: http://www.novell.com/coolsolutions/tip/15344.html

I won’t rewrite what is in these two articles, rather just give you a bit of guidance…

The article on the vmware community addresses both of the two required steps. The issue I ran into is that what they described for step 1 wasn’t available for me. Instead, I used the method described on the Novell site to change the size from a VMware perspective and then I used the EASEUS Partition Manager (my guest OS is Windows XP) to repartition in the OS.

One more note, depending on the type of hard disk, you may not need to do step 2. After you have run step 1, boot up the image and check the properties of the harddisk in the running image – that’ll tell you if you need to do any partitioning. Also, if I remember correctly, step 1 reported that I needed to partition at the conclusion of it’s work.

Of course, MAKE SURE TO BACKUP your image before doing any of this.

BYOTP?

Seriously?

I was on a 4 hour flight today and was the first person in the lav. No TP. A search through the entire cabin turned up only the rolls already hanging in the other lavs. You must be joking! Isn’t TP as important to the flight as the snacks for purchase? I dare say even more?

And for those who will inevitably ask which airline… United. Not sure one is any better than the other these days. Thank goodness for travel restrictions and far fewer plane rides.