GangaMSGPeek

Based on MSG Architecture the plugin MSGPeek provides a way to examine job output in real-time on the Grid, looking into the stdout/stderr of a job.

The client is the consumer and the job is the producer. The flowchart used to develop this functionality was:

state_diagram
client job


Each client connects to only a single job, the time behaviour is:

  • The job start running.
  • The jobs is waiting for a message from the client to start streaming the stdout/stderr.
  • The streaming is collected to the client.
  • When the job finishes the client finishes too.

Possible scenarios

  1. Several clients could want to obtain the stdout/sterr from the same client (this scenario should never happen)
    • The data is not received in order, client will ask for the data and the broker (Message Broker) will not give it the correct data and time out of the client will expire and the client finish
  2. The client dies and the job is still sending data.
    • There will not confirmation of data from the client so the job will stop sending data
  3. The client dies and other client takes over it.
    • No client will confirm the data so the data will be kept in the broker until a client ask for the rest of the data that has not be transmited.
  4. Job dies without an error
    • The client will not receive the finish message from the job, therefore the client will wait until the time out associate expires.

How to use the client

The client is written in python and accepts several options:

 '-I' or '--id'   The id of the job to be collected the output. Defaults to number 0 if not specified
 '-H' or '--host'   Hostname or IP to connect to. Defaults to gridmsg001.cern.ch if not specified
 '-P' or '--port'   Port providing stomp protocol connections. Defaults to 6163 if not specified
 '-U' or '--user'  Username for the connection. Only if needed
 '-W' or '--password' Password for the connection. Only if needed
 '-F' or '--file'  File containing the output and of the jobs
 '-T' or '--timeout'  Time out: Time that the client will wait until receive response from the job running


To do

  • Now as identifier of the job is used the number that Ganga provides, it will be change to give a unique identifier for each job
  • The acknowledgement mechanism for the messages is not completed
  • Extensive testbeds on the Grid


-- ManuelChamberGonzalez - 27 Jul 2009

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng state_diagrampng.png r1 manage 77.5 K 2009-07-27 - 12:41 ManuelChamberGonzalez Flowchart jobs
Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2009-07-31 - ManuelChamberGonzalez
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback