Well I sure hope not but the Global WebSphere Community is holding a one hour chat on the subject:
begin a

Session dates: Wednesday, December 3, 2008
Starting time: 3:00 pm, GMT -05:00, Eastern Standard Time (New York)

Many organizations have started down the path of SOA, only to find that reality is more challenging than the hype. Businesses need to see ROI, and IT professionals have a hard time staying on the theoretical course. So, how are we going forward?

In this web-event, we will focus on the current problems around SOA, suggest successful strategies towards attaining SOA benefits and discuss how you can be the catalyst for improving IT and Business-IT alignment: essentially, making SOA work in the here and now.

Our main topic will be SOA governance: coordination, ownership, right-to-decide and versioning of services are all critical factors in SOA. Other topics will include: service lifecycles, portfolio management and an SOA maturity model.

The content of this event is based in part on the book “SOA for Profit, a Manager’s Guide to Success with Service Oriented Architecture”. After the event, we will send attendees an electronic copy of this book.

I don’t think SOA is dead by any means, but I do think that it can expose communication gaps that exist in companies which lead to the delays and corner cutting that hurt SOA implementation. If your company doesn’t really work together well between Business Analysts, Systems Analysts, Developers and Administrators, you are going to have serious issues with any methodology you choose anyway. I think to be successful you have to breed a corporate attitude of the first impulse being “How can I help?” as opposed to “I’d like to help, but talk to my management chain first. Then we can schedule a pre-planning meeting about how we’re going to plan to……yadda…..”.

Related Posts

• Impact 2008: SOA Jam – “Foster an External Community for Practitioners”
• SOA Jam: Foster an External Practitioners Community
• Impact 2008: SOA Jam Day 3
• WebSphere User Group: Ask the Experts Session RIGHT NOW
• Impact 2008: SOA Jam – “ESB: Too Many Products, Skill Spread too thin”

From DeveloperWorks, DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains

Draft Redbook, last updated: Tue, 26 Aug 2008 - Introduction to DataPower Services - Integration Services - Security Services IBM® WebSphere® DataPower® SOA Appliances are purpose-built network devices that offer a wide variety of functionality such as the securing and management of SOA Applications, Enterprise Service Bus Integration, and high speed XSL execution.

I'm happy to see the emergence of design patterns for datapower.

Related Posts

• DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains
• Make SOA real with IBM WebSphere Enterprise Service Bus and IBM WebSphere DataPower SOA Appliances
• IBM WebSphere DataPower SOA Appliances Part III: XML Security Guide
• IBM WebSphere DataPower SOA Appliances Part II: Authentication and Authorization
• IBM WebSphere DataPower SOA Appliances Part IV: Management and Governance

One of the restrictions when dealing with a short running process is that it must always run within a single transaction. The implication is that the process must complete within the default transaction timeout window. On an application server, this is 120s.

We had an issue where our Process Server was creating a large number of locks to rows on the SQL Server table. Eventually, SQL Server escalation would kick in and the entire table would lock. This would preclude other users of the database from being able to read/insert data.

Upon inspection of the offending process, we found that a short running process contained three nested loops which performed multiple invocations against the SQL server database. On the original test data, there weren’t enough nested objects created to determine if a large object could complete within 120s.

In addition, by default in WebSphere Application Server there are 10 activation specs that can process messages from a queue in parallel. This means that you need to ensure thete are enough database connections to service the requests.

Our solution to the problem was to break the transaction down into smaller parts and transmit the parts via JMS queues to new modules. A one-way invocation does not require the source bpel to block for a response, so it is able to now complete within the transaction timeout boundary.

Related Posts

• Event Sequencing Message Locking – Resolution!
• SCA, BPEL, Websphere Adapter for JDBC and Transactionality
• WebSphere Process Server Endurance
• JDBC Adapter Export with events that comprise multiple rows
• WebSphere Process Server v6.1 API

From DeveloperWorks, WebSphere Business Integration V6.1 Performance Tuning

Draft Redpaper, last updated: Mon, 9 Jun 2008 - Learn valuable tips for tuning - Get the latest best practices - Try the example settings This IBM® Redpaper was produced by the IBM WebSphere® Process Server, WebSphere Enterprise Service Bus, WebSphere Adapters, and WebSphere Business Monitor performance teams in Austin Texas, Böblingen Germany, and Hursley England.

This book attacks performance on three levels, Architecture, Developerment and Performance Tuning. All key issues that need to be considered to create systems that can meet Service Level Agreements (SLA). I'm happy to see this kind of information get published so close to the public release of a version.

Related Posts

• WebSphere Process Server – Business Process Choreographer Explorer Customization and Tuning Options
• IBM Redbooks: WebSphere Process Server and WebSphere Integration Developer
• Pre-review: WebSphere Business Integration Primer
• DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains
• Migrating WebSphere Business Integration Server Foundation to WebSphere Process Server V6.2 & Best Practices

In the opening panel, the John Fitzgerald Director of Product Marketing from Software AG mentioned a client’s SOA Lessons Learned. The three points are:

1. Strong Business Sponsorship
2. Start Slow with a Low-Risk Project
3. Integration Competency Center

I agree with all three of these points and I’ll provide my thoughts below.

Strong Business Sponsorship

You aren’t going to be able to get off the ground unless you can prove to the business that there is value in the SOA approach. This usually involves finding someone in the business who understands what this value is, and is willing to champion it to everyone else.

Start Slow With a Low Risk Project

When you are introducing a new architectural style into a company, you will have a few challenges to overcome. The first being that your current developers and architects need to skill-up on creating services. They need to skill up on the tools and patterns that will be used in the future. Another major challenge is that you will have to win over the pessimists in the organization. There will be some subset of employees that do not believe change is necessary and they will need to see immediate results.

In order to complete these two points, picking a small low risk project is the best. The SOA should grow as your organizations capabilities and understandings grow. A key to this project is that it needs to be low risk while still providing business impact, in order to continue to receive that precious business sponsorship.

Integration Competency Center

You need some kind of oversight committee that keeps an eye on the entire developing ecosystem and ensures that the services being developed meet the needs of the business. You don’t them to degenerate into tightly coupled throwaways. The competency center spans the entire organization, providing its expertise in the SOA space as required.

I’ve seen the Integration Competency Center implemented as a multi-person multi-region team and I’ve also see it (smaller companies) as a single knowledgeable and passionate person. It doesn’t necessarily have to be a “Center” but it has to be able to guide service development.

Conclusion

I feel these are three of the higher priority lessons that anyone who wants to implement an SOA should follow. You need strong business buy-in, low risk initial projects and passionate respected people in your Competency Center to succeed building an SOA.

Related Posts

• Does Commerical Enterprise Software Suck?
• Impact 2008: SOA Jam – “Foster an External Community for Practitioners”
• Tip: Agree To Concrete WSDL Definitions before Development Begins
• Business Integration Software Bakeoff Idea
• Impact 2008: SOA Jam – “ESB: Too Many Products, Skill Spread too thin”

The cousin to yesterday’s WSDL Anti-Pattern: The ‘Single XML String’ Service is the use of the XSD specification’s ‘any’.

‘Any’ literally means “any well-formed XML contained in this section is valid”. ‘anyType’ means “any valid XSD Type”. We can start to see the correlation between the Single XML String service and the use of these generic types. They are used to dilute the definition contained in the WSDL to attempt to account for unforeseen changes in the future.

The benefit is that the WSDL doesn’t need to change as the service evolves, even though the services themselves will have to be modified to support the new arbitrary types.

The drawbacks are numerous. Tooling support for xsd:any’s is also low. You lose message validation. You lose strong definitions of your service. All the points that were made in the previous post apply to this style as well.

This is not to say that there aren’t valid reasons to use ‘any’ and ‘anyType’. Say, for a service that dynamically routes messages based on header contents. It’s just that most of the times they are used, it’s for the wrong reasons. Yes, I’m talking about you Industry Standard Schema Definitions (easily the worst offenders).

Just in case you didn’t like any of my arguments, here is a very well written DeveloperWorks Tip on using generic types with WSDL and the pitfalls that can occur:

Summary

Although it may seem like a good idea to use xsd:any in your Web service’s XML schema, there are pitfalls to doing so: you are deviating from the intent of WSDL, and the language mapping of xsd:any may not be easy to use. Although I won’t go so far as to say using xsd:any is a bad practice, I do hope this tip has given you a better awareness of problems you may encounter before you encounter them.

Related Posts

• xsd:any, xsd:anyType, xsd:anySimpleType, xsd:anyAttribute in WebSphere Process Server
• WSDL Anti-Pattern: The ‘Single XML String’ Service
• Tip: Agree To Concrete WSDL Definitions before Development Begins
• MQ Import Anti-Pattern: Multiple MQ Imports connecting to single pair of physical request/response queues
• Handling unmodeled faults within WebSphere Process Server V6.1

WSDL is a very useful technology. It allows service providers and consumers to agree on namespaces, operation names, the data to be transmitted in a request and the data to expect in a response. All very good things to know, all in a platform neutral way.

Now, WSDL tells us what all the elements and attributes are for a service invocation. What can we do at the WSDL level to account for future changes to the service? This is where some implementations get it wrong.

When you are creating a service, you can only account for the changes that are expected. Maybe a new product will be added or a field may become depreciated. These are easy changes to account for via the way you structure your XSD definitions. In this case, I would say that the service takes a List of Products thats unbounded.

If the change that is being requested created a structurally different message (via additional fields), then you just have to suck it up and modify the WSDL (either marking the old fields as optional if you want compatibility with current clients) or just making all your clients upgrade. To make my point, I’m going to ignore multiple version service offerings or use of WebSphere Service Registry and Repository.

What I’ve seen though, are services where the providers attempt to reduce ANY potential modification to the WSDL interface. This usually results in a service where the operation takes a single string parameter called ‘xml’. The client is expected to serialize the message itself and include it as a string payload in the request.

What this anti-pattern does is remove the entire point of WSDL and XSD. The ability to know what data to send to a service call and what data to expect back in a response. This is no longer captured at the level it should be. If I am a consumer of this service, how do I know what the XML to be transmitted should look like? I have to go through alternative social channels to get this information. That’s the very problem that WSDL was created to solve!

Other problems with”services as strings” deal with the added complexity that the service implementor and consumer need to perform themselves. Today, you can generate a SOAP/HTTP client for a wsdl automatically. The client will serialize your platform specific objects into SOAP and transmit them. In the anti-pattern, the developers themselves need to call the code to serialize the real content and then pass it on to the generate proxy which serializes it again.

What do you gain through this anti-pattern? Well you are no longer bound to making WSDL changes as the service matures. If a new product is created then the service provider and service consumer’s implementations must be updated, but the WSDL is left alone. This pattern is normally used to allow for ‘dynamicity, to account for changes in the data sent between the two services. The gotcha is that you are probably going to have to modify the implementation code anyway to handle the new data. You didn’t really gain anything, but you gave up strict typing and validation.

I’ve found that when the anti-pattern is used, it’s with good intentions but you are far better off just specifying the data explicitly in the WSDL and dealing with changes through your regular change management process for both WSDL and implementation.

Related Posts

• WSDL Anti-Pattern: xsd:Any and xsd:AnyType To Encapsulate Future Changes
• MQ Import Anti-Pattern: Multiple MQ Imports connecting to single pair of physical request/response queues
• Tip: Agree To Concrete WSDL Definitions before Development Begins
• Business Object Maps and ‘Required’ Attributes
• Visual Snippet Editor: Forgets Java Syntax

If you are a provider or a consumer about to enter development on a web service, you should have an understanding with your counterpart of the WSDL that defines the service. It should be very well ‘baked’ and changes once development begins should be minimial.

Major things like name spaces and object names should rarely change once development begins. Name spaces and object names are treated as a key by mapping tools. If either of them change, expect to waste time refactoring your development artifacts to use the new definitions.

Your WSDL file is a contract. A contract that describes what data I can expect from an invocation. How can you expect to perform a development exercise without a strongly defined contract?

Also, the runtime that you are going to use to implement your service is irrelevant. I received three WSDLS that were completely different because the developer was switching between runtimes and I assume just autogenerating the WSDL from the tooling.

Good anti-pattern and a solid reason why these kinds of projects miss deadlines.

Related Posts

• WSDL Anti-Pattern: The ‘Single XML String’ Service
• WSDL Anti-Pattern: xsd:Any and xsd:AnyType To Encapsulate Future Changes
• Avoid the WebSphere Integration Developer Debugger
• BOFactory: create(..) or createByElement(..) ?
• WebSphere Enterprise Service Bus – Mediation Flow Component: No Refactoring for WSDL Faults

I’m going to use this post to record the discussion that generated during the Impact 2008 SOA Jam for my idea about “How IBM can foster an external community”. Now that Impact is over, I’d like to keep a copy of it’s point in time, and also allow any future readers to add comments.

Foster an external community for non-IBM practitioners
Dan Zrobok 10 Apr 2008

Abstract

As a non-IBMer who works within the product stack, I find it next to impossible to get the information that I need to make the products a success. The goal is to have IBM create a vibrant external community that is equivalent to the internal one.

Read the rest of this entry »

Related Posts

• SOA Jam: Foster an External Practitioners Community
• Impact 2008: Impressions of Day 2 – SOA Jam
• Impact 2008: SOA Jam Day 3
• My Contributions To The IBM External Community – The 24/7 Answer Man
• Part 2: IBM and the self-sufficient WebSphere Community – An External Conference Call

I’m going to use this post to record the discussion that generated during the Impact 2008 SOA Jam for my idea about “Too ManyESBs, Skill Spread too thin”. Now that Impact is over, I’d like to keep a copy of it’s point in time, and also allow any future readers to add comments.

ESB: Too many products, Skill spread too thin
Dan Zrobok 10 Apr 2008

Abstract

If you want an ESB you can use: WAS, WPS, WESB, DataPower, Message Broker. Based on my selection, I have to find deeply skilled resources on that specific product. Is IBM spreading the community too thin?

Read the rest of this entry »

Related Posts

• Impact 2008: My impressions of Day One – SOA Jam
• SOA Jam: Unify The Numerous Enterprise Service Bus Products
• BPEL or ESB: Which should you use?
• Impact 2008: Impressions of Day 2 – SOA Jam
• Impact 2008: SOA Jam Day 3

Day two is in the bag, the B52’s are B-50-done. Perficient and MTS Allstream gave a session about SOA Patterns that I thought was today but was actually yesterday. Oops. Here’s the abstract if you have a time machine:

TSP-2518 – Effectively selecting integration patterns
Brent Legris, Perficient, Inc., Senior Technical Architect, Olivera Zatezalo, MTS Allstream Inc., Sr Manager, Integration, BT&IT

The foundation of knowledge that rests beneath service oriented architecture (SOA) encompasses a myriad of integration patterns not only messaging, enterprise service bus and orchestration, but shared databases, ETL, operational data stores, and others. Effectively selecting between integration patterns is a critical capability to develop as your SOA evolves, in order to avoid running around with a hammer thinking everything is a nail. This session interactively develops various integration scenarios, discussing pattern alternatives and their effective implementation. We will discuss the tradeoffs of different approaches, but close with guidelines that can be consistently followed to exploit the capabilities of an enterprise infrastructure.

Level: Intermediate

What I like about this presentation is that it’s about patterns and decisions and trade-offs, but primarily focuses on WebSphere Integration Developer and WebSphere Process Server. I like it when patterns come ‘out from the mist’ and you actually see them implemented in a system. I’m a developer at heart. Hopefully the presentation slides will be made available on the Impact site.

The other thing I heard a lot about was WebSphere sMash the IBM commercialized, supported version of Project Zero.

As for other impressions of Day 2, I don’t really have any. It seemed like a normal day at a conference in Vegas to me.

Related Posts

• Impact 2008: My impressions of Day One – SOA Jam
• Impact 2008: Impressions of Day 2 – SOA Jam
• Impact 2008: My impressions of Day One – Blogs & Twitter
• Impact 2008: SOA Jam Day 3
• Impact 2008: Final Day – Blogs

I’ve never been much of a patterns guy. I kind of always have the ability to apply common sense to a system and turn up something that is straight forward and makes sense. Periodically I get asked about what set of patterns I used to decide how a system should behave. To answer this question I tend to laugh maniacally, throw a flash-bang on the floor and run out of the room.  So I was glad to see that the pattern I’ve been using recently is called the VETO pattern.

Allow me to describe the requirements of the system that led me to the VETO pattern.

I have a source system message obtained from a staging table. This message is incomplete and needs to run additional queries against the target database in order to complete it. Once these additional queries are complete, the data needs to be inserted into the target database.

The original system performed all these tasks in a single java proxy, mangling the logic for enriching the source message with the SQL for inserting into the target database. I was tasked to migrate this to WebSphere Process Server with the knowledge that additional clients would need to use this service.

My solution was contained in four modules:

The first module received the message from the source system, validated it and determined what fields needed to be enriched. After enrichment, it transforms it into a target database message.  A second module provided the methods for enrichment from the Target Database. The third module processes a fully enriched source object for insertion into the target database. The fourth module just held the WebSphere Adapter for JDBC. I like keeping it in on it’s own incase independent adapter fixes are required.

Anyway, turns out that this is the VETO pattern. A pattern that I won’t be forgetting anytime soon.

Related Posts

• SCA, BPEL, Websphere Adapter for JDBC and Transactionality
• BPEL: Beware the use of nested loops in a short running process
• Event Sequencing Message Locking – Resolution!
• WebSphere Adapter for JDBC: Superfluous RecordNotFound Exceptions
• WSDL Anti-Pattern: The ‘Single XML String’ Service

Just wanted to point out to my readers that Johannes Brodwall has provided a ‘response to my response‘ on the topic of his post about the state of WebSphere. This blog doesn’t get too many comments, so I felt the need to point it out as he provides his opinions on WebSphere vs an OSS solution.

Related Posts

• Does Commerical Enterprise Software Suck?
• MQ Import Anti-Pattern: Multiple MQ Imports connecting to single pair of physical request/response queues
• Business Integration Software Bakeoff Idea
• Exploring IBM SOA Technology and Practice reviewed by Don Ferguson
• Datapower and Netegrity Siteminder Integration

As I’m sure you can tell, I’m very WebSphere focused. I have a hard enough time keeping up with the zillion WebSphere branded products that get released throughout the year. On my random surfing of the internet (tyvm Technorati) , I turned up Oracle’s entry into the world of SOA: Oracle SOA Suite 11g.

The interesting bit about this product is that it runs in direct competition with WebSphere Integration Developer. It uses the SCA runtime (I assume it’s Apache Tuscany), supports BPEL, Business Rules and even AJAX based Human Task forms. Pretty much the key features that WebSphere Integration Developer lists.

I don’t really know much about the product other than the marketing information found on the oracle site, but I’m all for competition in the marketplace.

The Oracle Assembly diagram:

Pretty much what I would expect to see. Exports on one side, business components in the middle and Imports on the other side. I wonder if Oracle resolved the weirdness in terminology of the “Export” starting your process while the “Import” calls other processes. Most people get confused with that right off the bat.

I also have to laugh at Oracles name for the WPS Failed Event Manager: “Error Hospital”. It’s in quotes in the presentation, so it’s likely it’ll get marketized into something better (and non-infringing on IBM’s FEM).

Oracle also has a “Large Document Handler” which is interesting. Today, most app servers are pretty bad at handling lage industry standard schema’s so it’s good to see Oracle address this up-front rather than trying to tack it on later.

Of course, everything always works without issue in PowerPoint so I’m interested to see how it works in the real world.

Related Posts

• Integrating Oracle BPEL with an HTTP (REST) Service
• Configuring IBM WebSphere Process Server V6.1 with an Oracle Database
• IBM Software Support Lifecycle – General Availability, End Of Marketing, End of Support Dates
• Oracle DBMS_CRYPTO Encryption
• Impact 2008: SOA Jam – “ESB: Too Many Products, Skill Spread too thin”

In Johannes Brodwall’s Blog, he asks “Why does so much enterprise commercial software suck?”. In it, he mentions that his project migrated off of WebSphere and onto Jetty and regrets that his project didn’t migrate soon. The money quote:

But WebSphere is just the most blatant example of software that gives you nothing, gets in the way of a lean process stream, yet costs a lot of dough.

I can hardly agree with this statement. Is WebSphere Application Server expensive? Sure. Does it have a steep learning curve? Yup.

But,

Does it do the job competently and reliably? Yes. Does it actually do everything listed in its marketing slides? Yup. Does it perform? In 6.1 IBM seems to think so. Can you automate the entire deployment process? Yes.

Bad software hurts you in an insidious way: It eats up the time of every developer a little at a time

Agreed. And I believe that the general clunkiness of the Integrated Development Environments that IBM provides is likely what Johannes is referring to. WID/RAD builds still take far too much time than they should, random errors appear and disappear and the UI does freeze needlessly, leading to death by ‘…a thousand pinpricks’. What I don’t quite understand is why his project didn’t evaluate using a better IDE for J2EE development. The WAS server is a reliable workhorse, why throw the baby out with the bath water?

Bad software hurts you in an insidious way: It eats up the time of every developer a little at a time. Instead of testing your changes with a one second unit test, you test your changes with a five minute deployment cycle. Instead of rolling out a new version to a controlled environment with a command in 30 seconds, you schedule a deploy task that the product expert will perform in a few hours.

This experience with a long deployment cycle isn’t really valid. You can script all of this stuff through ant tasks and JUnit cases. There’s no reason (other than the assumed process of the company in question) why an admin would be required to spend hours installing an EAR.

So when managers ask why a project is late, developers feel personally guilty. And it’s a poor craftsman who blames his tool, no?

I’d have to say its the smart craftsman who understands his tool and the time it takes to actually accomplish the task. It’s also the smart developer who comes clean and lets his project manager know when the plan may slip. This seems like more of a process/estimation problem that something caused explicitly by the software. If this was the first project on the WebSphere platform, I would assume a good project manager would build in contingencies for skilling up and learning.

And from the comments:

Hi, Space Monkey.

We either discontinued or replaced all features not included in a web server. Briefly: EJBs we stopped using (thank god for that!); for connection pooling we first used the pool that came with Oracle, and then switched to c3p0; for failover and loadbalancing, we use our expensive switches which already has this build in; for transaction monitoring, we use Spring; we also rolled our own message service implementation (much simpler than JMS, but a drop-in replacement for our use).

Yowza, it looks to me like a ton of work went into reproducing the capabilities that WebSphere provided ‘for free’. Rolling your own message service implementation? Thats not a path I’d want to go down any time soon. Also, what happens to this system two three years down the road? All these multiple components, multiple vendors. They are all going to point the finger at each other when production goes down and Johannes is left holding the ball. I can appreciate the technical marvel that went into replacing WebSphere, but there’s also large questions of support, documentation and skillset that have to be answered.

So. Does Enterprise software suck? I believe like anything in life, its about costs versus benefits. What is most important to the project? Reliability that the vendor will be in business 5 years down the road? Exploiting the latest in Web 2.0 functionality? Fast development cycles? Documentation? Throw-away or 25+ year expected life? Companies need to try to look down the pipeline and find something that will support what they want to do.

Enterprise software tries to be all things to everyone, so it can come up short. Does it necessarily suck? No.

Related Posts

• Response to ‘Does Commercial Enterprise Software Suck’
• Automatic Build Solution for WebSphere Integration Developer
• Getting Started with WebSphere Process Server and WebSphere Enterprise Service Bus Parts 1 & 2
• WebSphere Enterprise Service Bus does not support Event Sequencing
• Securing JMS connections to WebSphere Enterprise Service Bus V6.1