TWiki> LCG Web>MessagingSystemforGrid (revision 4)EditAttachPDF

WLCG Messaging System for Grids (MSG)

Overview

This page contains an overview of the WLCG MSG - Messaging System from Grids. This is a solution based around Apache ActiveMQ which aims to help the integration and consolidation of the various grid monitoring systems used in WLCG. It arose out of work done within the context of the WLCG Monitoring Working Groups

Background Presentations

We have presented a series of different presentations at the WLCG GDB on different aspects of WLCG monitoring in general, and the messaging layer, MSG, in particular:
  • Architectural principles (ppt) - A more general presentation giving the architectural principles of the entire WLCG monitoring system, out of which comes the requirement for some sort of loose coupling and distribution via a reliable messaging system
  • Strategy (ppt)
  • Some worked examples (ppt) - Examples of integrating some actual components, including OSG RSV tests into SAM

Infrastructure

MSG uses Apache ActiveMQ as the message bus. This is a fully featured JMS implementation that also offers non-Java solutions for publishing and consuming messages. It also integrates Apache Camel, which implements the enterprise integration patterns from the book by Gregor Hohpe and Bobby Woolf. We use much of the terminology of these patterns, such as Message Store, Control Bus, Message Broker, ...

The aim is to use MSG as an integration bus for all the grid monitoring systems we currently have within WLCG, and allow new systems to be easily built using a scalable infrastructure. It is currently a non-goal to provide a general messaging system for general usage by grid middleware, but we hope our experience could be useful if such a system is ever deployed!

Testing

We have carried out (ands are in the process of completing) a series of evaluations of various aspects of ActiveMQ as a message broker:

Message Protocols

AS a standard message format, we use the message format defined in the GridMonitoringProbeSpecification. Of course, depending on the actual usage (see below) the required and optional fields may change - but what says constant is the basic syntactic format.

  • TODO - Split out underlying message format from description of probe interactions.

We believe over time other formats will also become standard for interchange, probably some form of JSON notation and a simple XML format (Atom ?)

Tools provided

msg-publish-simple

Current Status

consume2oracle

Current Status

Specific Channel definitions

Proposed usage

-- JamesCasey - 02 Apr 2008

Edit | Attach | Watch | Print version | History: r8 | r6 < r5 < r4 < r3 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r4 - 2008-05-15 - DanielRodrigues
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback