GangaMSGPeek

Based on MSG Architecture the plugin MSGPeek provides a way to examine job output in real-time on the Grid, looking into the stdout/stderr of a job.

The client is the consumer and the job is the producer. The flowchart used to develop this functionality was:

                                                  client                                                           job

Each client connects to only a single job, the time behaviour is:

• The job start running.
  
• The jobs is waiting for a message from the client to start streaming the stdout/stderr.
• The streaming is collected to the client.
• When the job finishes the client finishes too.

Possible scenarios

1. Several clients could want to obtain the stdout/sterr  from the same client (this scenario should never happen)
   
   • The data is not received in order, client will ask for the data and the broker (Message Broker) will not give it the correct data and time out of the client will expire and the client finish
2. The client dies and the job is still sending data.
   • There will not confirmation of data from the client so the job will stop sending data
     
3. The client dies and other client takes over it.
   • No client will confirm the data so the data will be kept in the broker until a client ask for the rest of the data that has not be transmited.
4. Job dies without an error
   • The client will not receive the finish message from the job, therefore the client will wait until the time out associate expires.

How to use the client

 The client is written in python and accepts several options:

    '-I'  or '--id'      The id of the job to be collected the output. Defaults to number 0 if not specified
    '-H' or '--host'     Hostname or IP to connect to. Defaults to gridmsg001.cern.ch if not specified
    '-P' or '--port'     Port providing stomp protocol connections. Defaults to 6163 if not specified
    '-U' or '--user'     Username for the connection. Only if needed
    '-W' or '--password' Password for the connection. Only if needed
    '-F' or '--file'     File containing the output and of the jobs
    '-T' or '--timeout'  Time out: Time that the client will wait until receive response from the job running

To do

•  Now as identifier of the job is used the number that Ganga provides, it will be change to give a unique identifier for each job
•  The acknowledgement mechanism for the messages is not completed
• Extensive testbeds on the Grid
  

-- ManuelChamberGonzalez - 27 Jul 2009