Yesterday, I blathered on about how great DataPower would be for usage in WebSphere Process Server. Your developer would be able to stay completely inside the Generic Business Object (GBO) data view while leveraging DataPower’s awesome ability to transform efficiently. Very neat indeed.

An interesting is scenario where DataPower IS the ESB. No WebSphere Process Server, No WebSphere Enterprise Service Bus. Do it all inside the appliance. How can this be done? Well DataPower possesses all the major services that are required in order to implement an integration: Routing, Transformation/Normalization, Validation, Authentication, Logging and protocol bridging.

• Routing: A single request can cause multiple endpoints invocations
  • DataPower provides variables that span the context of a Request/Response
  • Currently datapower does them in serial but I hear that parallel invocations may be supported.
• Transformation/Normalization: As we all know, the XSLT abilities of the product
• Validation: Again, a built-in feature of Datapower to validate SOAP envelops and also against XSD schema
• Authentication: DataPower can integrate with Tivoli Access Manager (TAM) and Tivoli Federated Identity Manager (TFIM)
• Logging: Datapower can write events to it’s own logs, or even emit Common BaseEvents (CBE)
• Protocol Bridging: Datapower supports MQ, SIBus (WebSphere Default Messaging Provider) and Web Services (along with some pure TCP protocols)

If you gave me a system that can just do this, I can provide you an Enterprise Service Bus that’s pretty damn good. 99% of the time in your system will be spent waiting on web service responses, rather than bottlenecked inside of your application service. With the appliance, I get both the ‘server hardware’ and the ‘application server’ in a single box for a cost thats substantially less than a license for ESB, server hardware to support it and administrators to maintain it.

Now, if I look at the functionality of WebSphere Enterprise Service Bus, I notice a lot of the same constructs.

• Routing: Mediation Flow Component*, Imports/Exports
• Transformation/Normalization: Also done via XSLT (in v6.0.2) + various primitives
• Validation: Built into some primitives.
• Authentication: Leverages all the capabilities of WebSphere
• Logging: WebSphere tracing, CBE Emittal
• Protocol Bridging: MQ, MQ/JMS, SOAP/HTTP, SOAP/JMS, HTTP (v6.1)

* Mediation Flow Components are great for mapping a single request to a single response, but they fall apart when you want to invoke multiple endpoints and compose a new service from them.

I’m hard pressed to find a ton of scenarios where I can recommend WESB to a customer with no plans to ‘move up’ to Process Server. Scenarios exist, but I’m trying to take the view of the most common constructs in an integration. Process Server is relatively safe, you’ll still need it for Human Tasks, Long running BPEL, State machines etc. So I see the market for that product alive and well.

Of course, there are trade offs to using DataPower as your ESB. It doesn’t integrate with WebSphere Adapters. It doesn’t participate in a dual phase (XA) transaction. You have to be an XSLT guru. You don’t have a nice candy-coated UI like WebSphere Integration Developer. If you are a mid-size company looking to get your bang for a buck, you may not even miss those features.

In summary: I still like what I see with DataPower

Related Posts

• WB552: Random DataPower Thoughts Part 12
• DataPower – Proxy & Co-Processing
• Make SOA real with IBM WebSphere Enterprise Service Bus and IBM WebSphere DataPower SOA Appliances
• What’s new in IBM WebSphere Enterprise Service Bus V6.1
• WB552: Random DataPower Thoughts Part 10

In my random browsing of the internet, I came across Wikipedia’s comparison of Business Integration Software. I’m so WebSphere focused that I didn’t even know there were this many companies out there with “ESB” products (34). It sparked an idea in my head, I wonder what it would be like to bake-off all these solutions against a common scenario. I like to think I’m a typical Business Integration specialist. I have a large java development background and understand all the core SOA concepts. I also tend to pick up new software pretty easily. I’m thinking that I would come up with one or two standard integration scenarios and blog about the pain and misery that was required to install, develop and run the solutions.

This is more or less a brainstorm as I don’t think I really have the time to go through installing 34 product, plus the issues of getting licenses for the closed source products etc. I actually don’t even know how baked the open source stuff is, so it’d be interesting to see if it performs better or worse that good ol’ closed source WebSphere.

As for the scenario, I think the first would be creating a composite service out of multiple preexisting web service end points. Bonus points for integration directly with a database (as I find this to be the most common). I also realize that some ‘esb’ products actually just behave like mediations and only take one request and transform it into another request, so I think the second scenario would cover this type of usage.

I think each product would be rated on a scale of 1-10 on how easy it is to install (runtime and tool). If I have to read three books to get things configured out of the box, that’s no good. How long it took to implement the scenario as someone with zero previous product experience and only using product documentation, Google would be allowed if error and exceptions occured (with point deductions). How elegant the solution is, a self-documenting system is preferable to a mishmash of hacks and configuration files. I wouldnt’ want to include performance of the resultant system (transactions/min) as this is usually a process that all servers required. They can’t know out of the box what the load of a system would be.

I welcome all readers to drop their two cents as to what they would want to see as a scoring system, or for that matter, any input at all

Related Posts

• Automatic Build Solution for WebSphere Integration Developer
• Does Commerical Enterprise Software Suck?
• Enabling SCA-MQ integration via MQ bindings
• WebSphere Business Services Fabric
• BPEL or ESB: Which should you use?

At my client, we had a unique situation occur when integrating a database and an export for the WebSphere Adapter for JDBC. In a ‘typical’ scenario, a single row is written into a staging table, an SQL trigger writes this single row into the adapter’s event table, the adapter polls, converts the data into a business object and starts the module.

In this migration scenario at my client, a single transaction was actually comprised of multiple rows written into the staging table in a single transaction. The previous implementation had a built-in concept of table scanning and aggregation which we need to reproduce in process server. Each row of data was a ‘delta’ and logic is then used to ‘flatten’ these rows into a single transaction. We need a way of telling the adapter that “these n rows are not unique transactions, but actually a single transaction”.

Out of the box, the adapter doesn’t have any feature to combine multiple rows like this. A proposal was to use a long-running BPEL process using correlation to aggregate the data, but that would induce a ton of overhead and complexity onto our servers. There’s a slightly more elegant solution.

The WebSphere Adapter can represent foreign key relationships between tables as containment business objects (Parent-Child). It also supports lists of children. So what we did at a high level was create a table representation of a wrapper type that defined a primary key. Then, all the children that relate to a single WPS transaction are given foreign key relationships to the wrapper row primary key. When the wrapper row is written to the event table, the adapter picks up all the related children and we get our desired behavior: A single WPS transaction containing multiple rows of data.

The drawback is that we had to add a little bit more logic to our trigger, but the benefits are that we aren’t overloading our server trying to do the event correlation ourselves. We’ve pawned it off on a system (database) that was designed for that kind of work.

Related Posts

• WebSphere Adapter for JDBC Export and SQL Server 2000: Table Locking while Polling
• SCA, BPEL, Websphere Adapter for JDBC and Transactionality
• WebSphere Adapter for JDBC: Superfluous RecordNotFound Exceptions
• WebSphere Adapter for JDBC Export – ‘InvalidVerbException’
• WebSphere Adapter for JDBC: SetObjectKeys NullPointer Exception