Msg Server (Broker) Details

Introduction

EGEE uses ActiveMQ as the broker implementation. This document outlines the architecture and configuration details for the brokers within the federated broker network. See here on how to install a broker using YAIM.

Architecture

Where possible we wish to run a high-availability broker pair, in order to protect clients against downtimes that require restart of a machine - e.g. kernel upgrades. To do this, we use the Master-Slave capabilities of ActiveMQ. In particular, we use the shared-filesystem master-slave configuration. This gives the best features in terms of recovery after failure and performance. It requires a shared filesystem to be shared between the nodes in the ActiveMQ cluster. This can be done using e.g. a SAN, or NFS.

This implies that two machines are required for the master-slave pair, and that either a SAN storage array or NAS storage box is required for the shared storage.

Site Specific Deployment details

CERN

At CERN, we use 2 x86_64 servers as the broker nodes (egi-1.msg.cern.ch + egi-2.msg.cern.ch). These are currently separate brokers in the network and not failover due to lack of availability of a production SAN @ CERN.

We also run a development instance (dev.msg.cern.ch) and a pilot instance (pilot.msg.cern.ch).

SRCE

ActiveMQ is installed on RedHat Cluster Suite platform. Data directory is stored on SAN volume. Nodes are Sun Fire X4240 with 2 Quad Core AMD Opteron 2356 processors and 16 GB RAM.

Architecture of the whole HA cluster is shown on the picture below. Besides for ActiveMQ HA cluster currently hosts top level BDII and regional Nagios instance. RedHat Cluster resource manager handles virtual IP, mounting of data storage and individual services. In case when node or service failure is detected, resource manager automatically moves everything to one of the healthy node.

AUTH