Eric Newcomer's Weblog

As of this week you can download an early release draft document containing 11 design documents we've been working on for the past year or so as the result of the OSGi enterprise initiative.

This is important because according to OSGi Alliance rules, only members are allowed access to working drafts of documents. This is the first time we've released any of these drafts publicly. As a board member and EEG co-chair, I'm very pleased to see this happen because (a) I often get asked about what's going on and why we can't (in this age of open source) release details of what we're doing and (b) I'm very interested in feedback from the broader OSGi community.

The enterprise edition activity started in January, 2007 with a review of requirements gathered at the enterprise workshop event. Following the OSGi Alliance process, the newly formed enterprise expert group members began writing Request for Proposal (RFP) documents. After an RFP on a particular topic is formally accepted, members can begin writing the Request for Comments (RFC) documents to design solutions to one or more of the RFPs. The early draft contains some of the RFCs that are far enough along to be released - in fact this represents a majority of current work items.

One major work item that is not far enough along, BTW, is the Java EE mapping to OSGi. (Not that I'm trying to put any pressure on any of the members of the group to hurry up and finish their work or anything ;-)

The draft is divided into two major sections, like the work: Core and Enterprise (reflecting requirements taken on by the CPEG and EEG, respectively), pretty much (but not strictly) depending on whether the RFC affects the core, or something mapped to the core.

From here, the RFPs (which I call the design docs) are fed into the formal specification drafting process. So it is a great time to submit your comments (please be sure review the feedback form before you do - this basically defines the IP protections and rights involved).

Of course I am particularly interested in any feedback you may have on the distibuted OSGi document (RFC 119), but you may also be interested in:

• the Spring-DM inspired component model design (RFC 124)
• or some of the proposed security enhancements (RFC 120)
• the new command line capability (RFC 132)
• the new service registry hooks (RFC 126)
• the bundle tracker (RFCs 121 and 125)
• transaction support (RFC 98)
• or the DS updates (RFC 134)

It's almost 300 pages, but there's some good stuff there, and 18 months into the task, I think we have a pretty good handle on some things. But then again, maybe not...

The big new areas of course are the distributed OSGi and the "Spring-inspired" component model. And of course, as always, I am very interested in any comments or feedback about the suitability of the OSGi programming model for the enterprise, especially given some of these proposed enhancements.

One of the biggest debates in the software industry is about getting the level of abstraction right. By this I mean a level of interaction with computers higher than binary code or machine language - in other words, anything that presents humans with a more natural or intuitive abstraction of a CPU's instruction set and binary data storage format.

Computers are after all fundamentally stupid electrical systems that have to be told exactly what to do. At the end of the day, everything is just 1s and 0s - the bit is either on or off. But it is really hard for us humans to work with computers at that level, so we keep trying to make it easier for people to tell computers what to do. Getting the abstraction right is key to developer productivity, but it's a constant struggle. Abstraction layers typically remove flexibility and control from one level in order to simplify things at the next.

Years ago you could hear developers saying they would never use COBOL because it was much too slow and verbose compared to assembler. Yet COBOL remains a very widely used language.

It was not too long ago you heard the same complaint about relational databases - they were just too slow compared to hierarchical and network databases. And don't forget the Web - I remember when we called it the "world wide wait" since it took so long to render a basic page.

In other words, it is sometimes more important to make it easier for people to tell computers what to do than it is to give them complete control. When we introduced high-level "English-like" language abstractions for the the Digital business software product set on VMS in the late 80s (which included an RPC in the ACMS product) we endured constant complaints from developers who wanted more flexibility and control. In those days distributed computing programmers were used to the fine-grained control over remote conversations available in the dominant LU6.2 peer-to-peer protocol.

In fact at the time most transaction processing people thought we were nuts to base a TP monitor on RPC. They thought the only way to do distributed computing was to explicitly program it. We thought it was more important to make the developer's life easier by abstracting the distributed programming steps into what we called the Task Definition Language (TDL). You still knew you were calling a program in another process, but you didn't have to open the channel, establish the session, format the data, check that every send that needed one had a reply, etc. (ACMS is still in production in some pretty demanding environments.)

This afternoon I finally caught up up on Steve Vinoski's recent article and blog entries about the "evils" of RPC. If you aren't already among those who have read them thoroughly, I'd encourage you to. Including the comments, it's one of the best discussions of the merits and demerits of RPC and REST that I've ever seen. The core of his argument is that the RPC abstraction is not helpful - in fact the opposite. Explicit programming is preferable when creating distributed applications.

As someone in the middle of designing another RPC based system (Distributed OSGi), though, I'd like to weigh in with a few thoughts. ;-)

As I've said before, the distributed OSGi design does not really represent a new distributed computing system. The goal of distributed OSGi is to standardize how existing distributed software systems work with OSGi, extending the OSGi programming model to include remote services (today the standard only describes single JVM service invocations).

Because the design center for OSGi services is the Java interface, RPC or request/response systems are a more natural fit than asynchronous messaging. In fact because we are trying to finish up our work this year ;-) we have postponed tackling the requirement for asynchronous communication systems to the next version.

Anyway, after carefully reading the article and blog entries, I believe Steve is not against RPC per se. He wants people to think before just automatically using it because it's convenient. He wants to be sure anyone involved in building a highly scalable, highly distributed system considers the superior approaches of REST and Erlang, which were designed specifically for those kinds of applications. Absolutely no argument there. I am a big proponent of using the right tool for the right job, and RPC is not always the right tool.

In the OSGi work, we acknowledged early on in the process that transparent distribution is a myth. We recognize that constraints may be imposed upon an OSGi service when it is changed from a local service to a remote service, including data type fencing, argument passing semantics, latency characteristics, and exception handling. All of this has been the subject of lively debate, but the benefits of introducing distributed computing through configuration changes to an OSGi framework far outweigh the liabilities.

In our case, therefore, making it easier for OSGi developers to create and deploy distributed services is more important than the loss of flexibility and control available when using local services only. The biggest cost of software development is still human labor, and providing helpful abstractions, incluiding RPC, continues to be an important goal.

This isn't to say anyone should blindly use RPC, or use RPC in place of REST where REST is more appropriate. It is simply saying that ease of use - making it easier for humans to do something like distributed computing - can sometimes be more important than technical concerns such as being too verbose or too slow.

It definitely seems like a long time since the process started, but this is about as good an outcome as we could have hoped for.

The number of employees who have both IONA and Progress on their LinkedIn page is probably a couple dozen or even more. We have been neighbors in the Boston area for a long time and there has been a lot of cross-pollination.

Although we have been competing in the ESB and SOA space, we have had a common vision and very similar positioning in the market. It's a bit like two former rivals of the basketball court, each with different strengths and skills, finally getting put on the same team. And it's actually this aspect that's the most interesting - putting together two strong teams with complementary expertise.

I started thinking about this because one of the questions we keep getting on the analyst briefings is how we plan to combing the Artix suite and the Sonic ESB family.

First, it's interesting to note that both companies have been moving away from a "pure" ESB positioning toward a "suite" or "family" of products for SOA. So the question is actually a bit broader: how can we sensibly combine multiple product components and create the best independent and comprehensive "anti-stack" SOA offering?

Some of the specific details are yet to be worked out, but we have always known that even while we were promoting Artix as a unique, configurable microkernel aimed at endpoint integration requirements, the Sonic family's approach, based on leading JMS technology, is something that meets a lot of different and equally important SOA requirements. The Artix suite's focus on distributed service enablement actually adds a lot to the Sonic family, and even as we positioned ourselves competitively in the past I think we each always knew this somehow.

One of the more interesting aspects is the future of the FUSE product line and the view of the combined company toward the open source projects with which we've been involved. We have already seen this commment on the Server Side. As one of the folks who champoined getting involved in open source I am glad to say the Progress folks I've spoken with are very interested and enthusiastic supporters, and see a lot of value in the announced acquisition in helping to get more involved in open source.

We have also worked together as partners. A few years ago we were resellers of the Progress Sonic MQ product, and Artix still offers native integration with Sonic MQ, as does the recently released WCF Connect product. And more recently we had begun integrating the Actional SOA Management product line with the Artix suite and selling them jointly.

A couple of years ago I had the unusual situation of being asked to share a half-day SOA tutorial with someone from Progress. For the first couple of hours we took turns saying exactly the same things about SOA, application architecture, and the unnecessary complexity of Java EE application servers. Then we each took a turn describing how our respective products met the same requirements, and served exactly the same segment of the industry (i.e. SOA infrastructure) with different approaches. Instead of arguing over that, we can now agree 100% and are part of the same team.