If you are using DataPower to pull messages from MQ which contains persistent messages, do yourself a favor and set the ‘Units of Work’ on the QM in DP to 1.

This will enable DataPower to use SyncPoints from MQ and allow you transactionality on a per-message basis.

Without this setting, you can get very strange behavior when you want to discard a message. You could see the entire batch of messages re-appear on your input queue.

My theory is that MQ implicitly creates a syncpoint when DataPower connects and one end it disconnects. If it doesn’t get confirmation that all the messages were successfully processed, then the only recourse is for the QM to rollback the transaction, causing all the messages datapower successfully processed to return to the queue.

Related Posts

• Issues that I encountered while creating my DataPower to MQ Demo
• “What’s New in WebSphere DataPower v3.6.1″
• DataPower Extension Functions
• WB552: Random DataPower Thoughts Part 4
• IBM WebSphere DataPower SOA Appliances Part I: Overview and Getting Started

As I mentioned before, I wanted to create a demo of DataPower vs the WBI Suite for the conference that I attended today. The goal was to try and show the ‘wow’ factor of DataPower and turn a product that most only know from marketing slides into something real that could insprise the attendees. I came up short of that goal mostly due to time restraints and having some MQ issues. I thought that I would document the parts that cost me a lot of time:

1. Installing MQ in SuSe 10 Fixpack 2 is easy. The part that I forgot is that there is configuration that needs to occur after that. Learning of commands like runmqlsr or  strmqm took awhile.
2. Creating a bridged network using VirtualBox is not as straightforward as it is with VMWare. I had to track down a script that would automatically configure the bridge before I started the VM.
3. Know what IP address your datapower box is setup for. I didn’t have this information when I first got the box in the mail. If you have no idea what IP address it’s using, be sure that you have a serial cable and more importantly a computer with a serial port. My T60 doesn’t have one and there was no docking station handy.
4. Industry Standard Schemas aren’t easy to get your hands on. I thought I would use the HL7 schema but you have to be a member to download it. I found a draft but at that point, I just wanted a large schema and didn’t care about the actual payload.
5. I wasted an insane amount of time with amqsput sample application that can put a message onto a queue. It only accepts input via stdin, so I thought I would be smart and pipe my 500k XML as input. I was puzzled when I saw 5000 messages enter my queue. They were created because my XML contained CF/LFs and that sample app interpreted them as separate messages.
6. I wasted an insane amount of time with amqsput after I filtered out the CF/LFs. I then saw 9 messages created on my queue. It took me awhile to learn which mq command I could use to see the contents of a message on the queue and I realized that each message was 64k, the console limit for standard-in. I then had to recreate my XML/XSD validation to use a smaller XML file.
7. My laptop can’t push enough data into MQ to actually get the box at 100% utilization. I was maxed out in the CPU of my virtualbox VM. I think if this scenario were to be a little more fair, the WebSphere server would have to be pushed off the box as well as the application that drops messages on the queue. I was trying to do everything on one laptop.

Funny part is, the actual datapower work was pretty straightforward. I created an MQ Queue Manager and once I actually got the TCP listener port up everything worked fine. Creating a Multi-Protocol Gateway and rules to transform and encrypt were equally easy.

Anyway, it’s still cool to have my hands on an actual datapower XI50. It’s heavier than you think

Related Posts

• DataPower 3.8.1 and WebSphere MQ
• WB552: Local DataPower XI50 (Blue) Initial Configuration Issues
• IBM WebSphere DataPower SOA Appliances Part I: Overview and Getting Started
• Perficient Booth at the IBM WebSphere SOA Connectivity Briefing in Toronto
• WB552: Random DataPower Thoughts Part 12

IBM is hosting a WebSphere SOA Connectivity Briefing in Toronto: Strategies for recovering your IT budget with IBM WebSphere MQ & SOA Connectivity

Details:

Sheraton Center Toronto Hotel
November 25, 2008
8:00am – 12:00pm

123 Queen Street West
Toronto Ontario M5H2M9
Phone: 416-947-4848

I’ll be there representing Perficient. My current plans are to get a DataPower box and hook up a little demo with DataPower pulling large industry-standard schema messages off MQ and transforming them and comparing that to the time it takes WebSphere Application server to do the same amount of work.

So if you happen to be in Toronto and would like to see a Datapower box in action, let me know and I can add you to the invite list.

Related Posts

• Issues that I encountered while creating my DataPower to MQ Demo
• DataPower 3.8.1 and WebSphere MQ
• IBM WebSphere DataPower SOA Appliances Part I: Overview and Getting Started
• Make SOA real with IBM WebSphere Enterprise Service Bus and IBM WebSphere DataPower SOA Appliances
• “What’s New in WebSphere DataPower v3.6.1″

From IBM Redbooks, DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains

Redbook, published: Mon, 13 Oct 2008

• - Introduction to DataPower Services
• - Integration Services
• - Security Services

IBM® WebSphere® DataPower® SOA Appliances are purpose-built network devices that offer a wide variety of functionality such as the securing and management of SOA Applications, enterprise service bus integration, and high speed XSL execution.

Related Posts

• DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains
• IBM WebSphere DataPower SOA Appliances Part III: XML Security Guide
• Make SOA real with IBM WebSphere Enterprise Service Bus and IBM WebSphere DataPower SOA Appliances
• IBM WebSphere DataPower SOA Appliances Part II: Authentication and Authorization
• IBM WebSphere DataPower SOA Appliances Part IV: Management and Governance

From DeveloperWorks, DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains

Draft Redbook, last updated: Tue, 26 Aug 2008 - Introduction to DataPower Services - Integration Services - Security Services IBM® WebSphere® DataPower® SOA Appliances are purpose-built network devices that offer a wide variety of functionality such as the securing and management of SOA Applications, Enterprise Service Bus Integration, and high speed XSL execution.

I'm happy to see the emergence of design patterns for datapower.

Related Posts

• DataPower Architectural Design Patterns: Integrating and Securing Services Across Domains
• Make SOA real with IBM WebSphere Enterprise Service Bus and IBM WebSphere DataPower SOA Appliances
• IBM WebSphere DataPower SOA Appliances Part III: XML Security Guide
• IBM WebSphere DataPower SOA Appliances Part II: Authentication and Authorization
• IBM WebSphere DataPower SOA Appliances Part IV: Management and Governance

From Developerworks, WebSphere DataPower SOA Appliance: The XML Management Interface

Draft Redpaper, last updated: Thu, 7 Aug 2008 - Appliance Management Protocol (AMP) - SOAP Configuration Management (SOMA) - Debugging The XML Management Interface is the third way to configure and administer the WebSphere DataPower SOA Appliance, besides the WebGUI and the CLI.

Kudos to the DataPower people for putting out more information about this interface that can be useful for administrators that like to run scripts to configure their environments.

Related Posts

• IBM WebSphere DataPower SOA Appliances Part IV: Management and Governance
• IBM WebSphere DataPower SOA Appliances Part III: XML Security Guide
• IBM WebSphere DataPower SOA Appliances Part II: Authentication and Authorization
• IBM WebSphere DataPower SOA Appliances Part I: Overview and Getting Started
• DataPower Problem Determination Techniques

I re-wrote IBM DataPower Certification test 284 over the weekend and passed with a 75% when requiring a 60%. Taking the course made my life a lot easier. I still spent two hours writing and reviewing the answers but I was a lot more confident about passing when I ended the test.

I did notice that there is a bug in the Prometrics Windows based client (non web-browser) that you need to be aware of. One of the questions deals with “Which of the XMLs is invalid”, and the answers use xmlns to define namespaces. The problem is that the URLs follow the http:// format and the windows client is turning them into blue-underlined hyperlinks while removing the surrounding quotes. This will make three of the answers appear as invalid XML.

I have emailed the only person in IBM who I know is deeply involved in the certification organization and I hope IBM will either update the question or fix the windows client bug.

The other weird thing about the windows client was that I didn’t see an option to provide feedback directly when ending the test but not yet getting the score. I took a notes about questions that I really disliked and wanted to provide feedback on but I didn’t have the opportunity. The ‘paper’ I was given to record my notes on when writing the test was an erasable surface that had to be returned at the end.

Anyway,  the test was pretty much the same one that I had written previously except that the hard questions were first which freaked me out a bit

Related Posts

• IBM Certification Tests 253, 142 & 664
• IBM Certification Tests
• Impact 2008: Free Certification Testing
• IBM Certification : Take it again, Free!
• WB552: Random DataPower Thoughts Part 5

From DeveloperWorks, DataPower Problem Determination Techniques

Draft Redpaper, last updated: Fri, 13 Jun 2008 This IBM redpaper provides a guide to many aspects of problem determination on a DataPower appliance, with an emphasis on powerful troubleshooting utilities.

This redpaper illustrates how to debug issues in DataPower using a scenario driven model.

Related Posts

• WebSphere DataPower SOA Appliance: The XML Management Interface
• The Support Authority: A systematic approach to problem solving
• IBM WebSphere DataPower SOA Appliances Part I: Overview and Getting Started
• DataPower Course WB552 Overall Impressions
• DataPower Extension Functions

I have to admit that I was very impressed with DataPower Course WB552. The content of the course covered everything that on would need to know about how to use the box along with security. It also provided a lot of opportunity to go off the beaten path and try modifications on your own. I felt like I got a good feel for debugging on the box as you would almost always miss a step (by default a rule is bi-directional which will throw an error when you only have security on one side).

When you are in this debugging mode, the box provides a pretty good view of exactly what was going on with the ability to see the complete trace log for a single transaction, even though sometimes the errors themselves are cryptic. Luckily, I have more than enough experience translating developer exceptions into English from WebSphere Process Server.

The time allocated for the labs is quite large and you get the opportunity to leverage the skill of your teacher to explain certain dialog boxes and determine the datapower programming model. An example of this was when I was trying to set two URL’s in a match rule, thinking that the operation between multiple rows in the list was an ‘OR’. It’s actually an ‘AND’ meaning that my match would have to satisfy both URL rules.

Given that I had previous datapower experience both writing the certification test and reading the course beforehand, my view may be a little skewed but I noticed that the entire class was able to keep up and usually worked ahead of the lecture::lab relationship.

I was told that there is a follow-on course for WB552, I’m not exactly sure which one it is, or what they start to get into.

I totally recommend this course to anyone who wants a better understanding on the hands-on use of the datapower box.

Curiously, I was also informed that the IBM Business Partner Virtual Innovation Center does not currently provide a DataPower box. This makes it difficult for partners to get the hands-on skill or be able to rerun the labs on their own time. Hopefully this will be addressed soon by IBM.

Related Posts

• DataPower Course WB552: Accelerate and Secure XML and Web Services with IBM DataPower SOA Appliances
• WB552: Random DataPower Thoughts Part 6
• WB552: Random DataPower Thoughts Part 4
• WB552: “Is there an IDE for DataPower?”
• DataPower Extension Functions

Message set (traffic Pattern)

Count monitors increment based on condition.

Duration monitors occur when a configured amount of time passes during processing of messages for the condition.

Message Count Monitors would be used to limit requests to a certain rate (100/second)

Message duration monitors are clock-based. Measure things like average server response.

Traffic Definitions are grouped into a Message Type Definitions, Filter Action Definition,  Monitor definitions, monitor-service association.

Tree: Service -> Message Monitor -> ((Message Types -> Message Match)  | Message Filter Action)

OBJECTS -> Monitoring.

Traffic Definitions identify raw traffic streams.

Durations:

Reqeust: Time required to service requests by the appliance (inside DataPower)
Response: Time required to process responses from the server (inside DataPower)

Server: Time required by the back0end server to process requests (Server Request + Server Reponse)

Message: The round-trip for a message.  (DataPower Entry to DataPower Exit)

Message Filter Action -> What to do when the filter is true.

SLM: Service Level Monitoring. Web Service level montioring.

SLM Policy consists of 1 to n statements which consist of restrictions of traffic.

Statements -> Counts messages or duration. Executed in order.

Can be created top-down or bottom up from the Objects -> Monitoring menu.

SLM policies must be added as actions in the policy editor.

Ensure that the SLM action is placed before the results action or else your SLM will not run. Also, on the WS Proxy page, the SLM page will not run unless there is an SLM action.

SLM Action chooses an SLM policy to run.

SLM Action consists of Credential Class, Resoruce Class, Thresholds and Actions to take.

Token-Bucket: Total number of concurrent requests ?

Related Posts

• WB552: Random DataPower Thoughts Part 5
• WB552: Random DataPower Thoughts Part 4
• WB552: Random DataPower Thoughts Part 9
• WB552: Random DataPower Thoughts Part 1
• WB552: Random DataPower Thoughts Part 6

Tivoli Directory Integration can integrate with multiple LDAP.

LDAP Search Attribute is the attribute in the LDAP that you want to look up.

Load Balancing Algorithms:

1. First-Alive: Secondary servers are only called when the primary server is down. So the first alive entry is always used.
2. Hash: Hashes the IP address of the client to provide affinity between clients and servers.
3. least-connections: Choses the server with the least number of connections
4. Round-Robin:Forwards request to the next server on the list.
5. Weighted-Round-Robin: Forwards request to the next server on the list based on the weighted averages as entered.

LDAP Load Balancing Damp Time speficies how long a server should be marked as unavailable when a heath check fails.

If all servers go down, the default behaviour is to wait for damp time expiration or for the health check to find an active server.

Related Posts

• WB552: Random DataPower Thoughts Part 8
• WB552: Random DataPower Thoughts Part 7
• WB552: Random DataPower Thoughts Part 12
• WB552: Random DataPower Thoughts Part 10
• WB552: Random DataPower Thoughts Part 5

AAA: Authentication, Authorization and Auditing

Authentication can occur without the use of external servers: AAA file, LTPA and a Validation credential object, SAML token (?).  Can use the external Systems: LDAP, SAML, Tivoli, RADIUS

When hitting a webpage, a browser artifact on a third party server can be pointed to embedded in the URL of the request.

Mapping Credentials: Map a credential from one system format to another. Same with Mapping Requested Resource.

Post-Processing steps: to inject the credentials into a message from a different format than the input.

Related Posts

• WB552: Random DataPower Thoughts Part 8
• WB552: Random DataPower Thoughts Part 12
• WB552: Random DataPower Thoughts Part 7
• WB552: Random DataPower Thoughts Part 6
• WB552: Random DataPower Thoughts Part 9

XML Virus scanning uses a filter action sheet: store://Virus-ScanAttachment.xsl. This transform needs to be modified to include the URL of your ICAP server

Dictionary Attacks Protection uses count monitoring.

SQL injection Protection uses the store://SQL-Injection-Filter.xsl transformation.

Related Posts

• WB552: Random DataPower Thoughts Part 5
• WB552: Random DataPower Thoughts Part 12
• WB552: Random DataPower Thoughts Part 6
• WB552: Random DataPower Thoughts Part 1
• WB552: Random DataPower Thoughts Part 2

I have a match action which should accept /encrypt and /encrypt_fl . Could I have both of these specified in a single match action? When I tried it, I got a datapower error with the box unable to match. Not sure if this is user-error or what, but I thought I’d give a heads up to the world.

Related Posts

• DataPower Extension Functions
• WB552: Random DataPower Thoughts Part 6
• DataPower Course WB552 Overall Impressions
• WB552: Random DataPower Thoughts Part 4
• WB552: Random DataPower Thoughts Part 12

SSL: Message Confidentiality, Message Integrity and Non-Repudiation

Server always authenticates to the client. Client optionally authenticates to the server.

During an SSL handshake: Negotiate the level of SSL, decide on cipher suite, authenticate the server, build a secret key to be used for the session.

SSL hello contains list of Cipher Suites. Server responds with hello and has selected a cipher suite from the list. Server also sends the certificate. Client validates certificate. Client encrypts message about the symmetric key with server public key.  Connection is secured. Symmetic key exists for a specified time (2 min) and is then re-negotiated.

Server Identifies, Client Validates.

In the case of mutual authentication, your Crypto Profile would contain both  Crypto Identification Credential and a Crypto Validation Credential.

forward ssl proxy -> Client. reverse ssl proxy -> server. Stupid.

SSL Proxy Profile refers to a Crypto Profile which then refers to the key/certs.

Related Posts

• WB552: Random DataPower Thoughts Part 7
• WB552: Random DataPower Thoughts Part 10
• WB552: Random DataPower Thoughts Part 4
• WB552: Random DataPower Thoughts Part 12
• WB552: Random DataPower Thoughts Part 5