…get lost in the realm of WebSphere

Another really interesting session with Ian Vanstone from the MQ lab in Hursley. The agenda for this session was this:

• Clusters
• Queue Sharing Groups
• Backup Queue Managers
• Restart/Recovery Performance
• Channels

Clusters

In this section we went through all the different aspects of MQ clustering which also provides some sort of HA but mainly load balancing capabilities. The main advantages of MQ clustering are easier administration and that it provides advanced workload balancing functionality.

Queue Sharing Groups (QSG)

Unfortunately only available for MQ on z/OS they provide the functionality that multiple queue managers can access the same shared groups and the same MQ objects. QSG gives you the ability to use “Distributed Queuing” which means:

• Inbound: Workload balancing via support for generic network ports
• Outbound: Shared XMITQ
• High front-end availability to support high backend availability
• Very high availability through peer restart

As well as it provides IntragroupQueueing which means:

• High performance transfer of messages between queue managers in the same QSG

Backup Queue Managers

If you are looking for a solution in case of a unrecoverable queue manager in case of hardware failure this might be the right solution. Backup queue managers work in the following way:

• Inactive backup QM replays the logs of active queue manager
• Logs must be copied to backup location
• Linear logging – copy inactive logs
• Circular logging – stop QM to copy logs
• Can force log (MQSC: ARCHIVE LOG) and script the process
• More frequent the backup, the less data is lost
• Available on Windows/Unix systems

Restart/Recovery Performance

• Restart performance is important for availablility
• The longer the restart, the longer the downtime
• After abend, recovery time is dependent on
• Length of outstanding UoW
• Time since last checkpoint
• WMQ z/OS has info about long-running UoW and lets you auto-commit
• Distributed WMQ reports on logfiles needed for recovery
• An option on rcdmqimg, which requires linear logging
• If this gets too far behind the ‘current’ logfile then you’ve probably got a long UoW that needs to be investigated
• Might be due to indoubt channel
• Log files are key recovery resource
• z/OS supports dual logging
• Other platforms recommend RAID/mirroring designs

Channels

• WMQ Clients
• Clients can connect to one of several queue managers
• CCDT
• Use a generic address for queue-sharing groups
• MQI verbs will return a failure if partner (or network) fails
• Clients ought to be coded to reconnect
• Message availability – clients have no local queues
• IP Load Balancers
• Can resolve a hostname to a selection of addresses or a virtual address to one of a number of real addresses
• Useful for Client channels, but not for Qmgr channels

Recovery

• Qmgr channels will automatically recover from partner failure
• Channel config parameters to assist in noticing partner failures
• Heartbeat, Keepalive, AdoptNewMCA (for Qmgr channels)

Further on we discussed the different kind of HA enablement’s on different operating system platforms. As there are:

z/OS

• Automatic Restart Manager (ARM) and GDPS

Windows

• MSCS support part of WMQ

Unix

• SupportPac MC91
• HACMP
• Veritas Cluster Server (VCS)
• MC/ServiceGuard
• Others
• Compaq Tru64 TruCluster - SupportPac MC68 (doc only)
• iSeries - SupportPac MC41
• Sun cluster 2x – SupportPac MC69
• …other platforms/HA technologies possible

One of the topics was the ARM (Automatic Restart Manager) on the z/OS platform. See a more detailed description of the ARM below:

z/OS recovery function used for jobs or tasks that have high availability requirements or that run in “lights-out” environments. If a job or task fails, it is restarted from the beginning, without operator intervention, either on same z/OS image or on another image within a sysplex.

A Queue Manager can be restarted on the same image if z/OS has not failed or on another image if the orignal fails (a cross-system restart).

We compared different kind of cluster technologies and got information about where each products strength or weakness applies.

Unfortunately the HA clustering of WMB was just mentioned besides this session and not so “deep” as we would like to have it.

Magnus took the “DB2 UDB V8.1 Family Fundamentals” certificate today, so now we have 7 certificates in our pockets when we go home tomorrow.

Examples of Using WBI Statistics and Accounting

This session was held by Colin Paice and had the following agenda:

Introduction

- SupportPac IS11
- Why use Statistics and accounting?
- Audiences for data
- Capacity planner
- Performance analyst
- Systems programmer
- Change management
- Flow developer
- Charge back

- Be aware
- Other problems
- Demo of facilities – Spread sheet, Tivoli Omegamon

IBM provides a supportpac called IS11 which provides you with some tooling to get access to statistics and accounting information. Colin presented a detailed overview of a message flow and explained the different parts of it and how different traces can be applied. An explanation of the trace destinations and collection scope was given as well as a presentation of the contents of IS11. How do you subscribe to the published statistics/accounting data, the supportpac IH03 will perfectly fit your needs to do this. One of the foils listed all the data which will be collected:

Message Flow
• Broker name
• ExecutionGroup name
• Flow name
• Sample time, start, end
• ”Archive” or “Snapshot”
• Msg: Size(Total,Min,Max), number processed,bytes processed
• Times(Total,Min,Max): Elapsed,CPU
• Time: Waiting for input
• Threads: Total, high water count
• Commit/backout count

Threads
• Arbitrary thread number
• Msg: Total processed
• Total Times: Elapsed, CPU, Wait,
• Input Wait
• Msg: Min, Max input size

Nodes
• Label name
• Type
• Times(Total,Min,Max): Elapsed,CPU,
• Invocation count
• Msg: Size(Total,Min,Max),
• number input messages processed
• Bytes processed

Terminals
•Labels and Counts

Information for different roles has been given as follows:

Capacity planner
The capacity planner will mainly be interested in the flow statistics per week. Understand response time requirements and ensuring that there will be sufficient resources.

Performance analyst
The performance analyst wants to look for inside information from the messageflows, response times and fine grained time/cost analysis from all the steps inside the message flow.

Systems programmer
As a systems programmer you want to know how the whole system performs, queue problems, database problems and so on.

Change management
Change management persons want to act as a gatekeeper for production environments and make sure what impacts changes will have to the actual system.

Flow developer
Consider using accounting to user trace and use nodes report, check that correct nodes are used and check the error paths.

Charge back
If you want to charge the users of your system you want to record accounting information to be able to base your cost on that information.

After this introduction of the setup and supportpacs needed Colin demonstrated the functions and features in a demo which went through all the different steps of data collection including the presentation of the collected data within Tivoli Omegamon for Messaging V6.

A session with Peter Toghill from the lab in Hursley. He talked about all different kind of considerations to have in mind about performance related things which can affect your MQ infrastructure.

The agenda for this presentation:

• Application Architecture
• Recovery Requirement
• Asynchronous/Synchronous/Thin clients
• Short or long term WMQ Connections
• Heavyweight WMQ resources
• Persistence, LUW, Log, Queues, Channels, QM
• Security
• Monitoring
• Applications
• Trusted, Isolated, or normal bindings
• Language (C, C++, COM,, … JMS)

The following picture shows an overview of the performance bottlenecks:

In his session he showed us various graphs which compared different operating systems and MQ setups which each other to make the differences visual to the audience. It was very interesting to see how similar setups perform very differently depending on the operating system platform and/or hardware setup. Even application design can have a major effect on how your MQ infrastructure performs.

See Peters summary from the presentation below:

• Many aspects to consider. To get the best:

• Select the right application model

• Use persistent messages only where necessary

• Use the fastest reliable disk for logs

• Set up channels and the network for best throughput

• Use the WebSphere MQ performance reports to understand processing characteristics
• WebSphere MQ is used in high throughput, business critical applications

How does my message flow through an MQ network? WebSphere MQ Version 6 introduces Activity reports and trace route messages to help answer this question. In this hands on exercise we will use these new tools to analyze how messages travel around a simple network containing a number of queue managers.

That was the title of the session and that was what we did. Still a bit confused about this works but it will surely clear up a bit when playing around more with this functionality.

Pete Siddall from the MQ lab in Hursley held this session.

In this hands-on session we have set up and configured an MQ Cluster. Stephen Rowles, a developer from the lab in Hursley has held this one. It was good to be able to test almost all the cluster techniques we have learned us on Ian’s session about advanced WMQ Cluster workload management.

Today Mats took the “IBM WebSphere Business Integration Message Broker V5.0 System Administration” and the “DB2 UDB V8.1 Family Fundamentals” cetificate. Anders and me took the “IBM WebSphere MQ V6.0, System Administrator” both of us.

Just another session with Morag, this time about securing your channels by using SSL. We first discussed the security problems you might encounter and how to find the appropriate solution for each of them. Mainly you may encounter one or more of the following security issues:

• Eavesdropping

• The eavesdropping problem can be solved by encrypting your information.

• Tampering

• To be able to detect if someone has tampered with your message whilst in transit you can make use of a one-way hash function. The message is passed through this function which results in a short fixed length number which is unique to the contents of this message.

• Impersonation

• The CA calculate the hash value of the plaintext (our certificate) and then signs that hash value with the CA private key to generate a CA digital signature. To check that the certificate is valid, the CA’s digital signature can be decrypted using the CA public key (well known CA public keys are installed in many of the security products that use SSL) to check that the hash values match.

Later on we went through the concepts of SSL on a higher level just to refresh the minds with what SSL is all about.

Secure Sockets Layer (SSL) is an industry-standard protocol that provides a data security layer between application protocols and the communications layer, usually TCP/IP. The SSL protocol was designed by the Netscape Development Corporation, and is widely deployed in both Internet applications and intranet applications. SSL defines methods for data encryption, server authentication, message integrity, and client authentication for a TCP/IP connection. SSL uses public key and symmetric techniques to provide the following security services:

Message privacy

SSL uses a combination of public-key and symmetric key encryption to ensure message privacy. Before exchanging messages, an SSL server and SSL client perform an electronic handshake during which they agree to use a session key and an encryption algorithm. All messages between the clientand the server are then encrypted. Encryption ensures that the message remains private even if eavesdroppers intercept it.

Message integrity

SSL uses the combination of a shared secret key and message hash functions. This ensures that nothing changes the content of a message as it travels between client and server.

Mutual authentication

During the initial SSL handshake, the server uses a public-key certificate to convince the client of the server’s identity. Optionally, the client may also exchange a public-key certificate with the server to ensure the authenticity of the client.

The last part of this session was about how to use SSL together with WebSphere MQ; MQ configuration steps and security administration tasks.

We can look at the security of MQ at two levels which I will describe as application level security and link level security.

SSL is used in WebSphere MQ to do queue manager to queue manager authentication and link-level encryption. This means that your messages are encrypted when they are moved across the network but they are not encrypted when they reside on your queues. If your security set-up is such that your queues are secure, then link-level security may be all that you require. If your system already authenticates applications and encrypts the messages at MQPUT time and decrypts them at MQGET time, then the messages are encrypted as they reside on the queues. They should not require further encryption as they are transported across the network. In this scenario, link-level security may not be required, or may only be required for queue manager to queue manager authentication at a minimum.

Very useful and interesting indeed. (All text in italic is taken from the actual presentation.

Today Anders took the “IBM WebSphere MQ V6.0 Solution Design” certificate. I have tried to take a DB2 rookie certificate yesterday but without success, so i took the “IBM WebSphere Business Integration Message Broker V5.0 System Administration” certificate today, instead. The man with the longest name of his WebSphere ceritificate at point of death wins…

Ian Vanstone held this session about the newly added features of cluster workload management in MQ V6. He went through the following aspects:

• Clustering
• Workload balancing
• New features and demos
• The new algorithm
• Workload exit

First we went through what clustering means, what it is used for and how you set it up. Quite a “runmqsc QM_STSE < refresh brain > http://aixpi.com/Atlanta2006” on MQ clustering which was very useful to understand the forthcoming topics of workload balancing. We learned that on a bind_on_open workload balancing never will occur, you have to issue a bind_not_fixed to be able to gain workload balancing in WMQ. Then he explained all the new cluster workload management features as there are:

• Rank
• Priority
• Most recently used
• Channel weighting

I just will give a very short description of all these techniques as they are quite complex and would not really fit into a blog. If you have any further questions feel free to comment this post and I will try to answer your question.

Use-queue

Allows remote queues to be chosen, when a local queue exists.

Rank and overlapping clusters

Rank allows greater control of the routing of messages through overlapping clusters. Rank can be applied on channels or queues but keep in mind that the channel rank will always override the queue rank. It is also important to note that destinations with the higher rank will be chosen regardless of the channel status to that destination. This could lead to messages building up on the SYSTEM.CLUSTER.TRANSMIT.QUEUE.

Priority

Channels and queue with the highest priority are chosen preferentially over those with lover priorities.

Most Recently Used (MRU)

If more than one destination is valid, limit the roubd robin to the n most recently used channel destinations.

Channel Weighting

If more than one destination is valid, the round robin algorithm sends messages in numbers proportional to their channel weights.

Over all it was a really useful session about MQ clustering.