CRAB3 architecture and task workflow

Complete: 3 Go to SWGuideCrab

CRAB3 architecture

The following diagram shows (a very simplified version of) the current architecture of the CRAB3 system.

Typical workflow sequence

The arrows in the previous diagram represent a typical workflow, starting from the submission and ending with the publication in DBS. Below we enumerate the steps involved in such a typical workflow.

  1. The CRAB client submits the request to the CRAB server (and uploads the user's input sandbox to the CRAB Cache -also in the CRAB server-).
  2. The CRAB Server inserts the request as an entry in the Task (Oracle) DB.
  3. A subcomponent of the CRAB Server called the Task Worker constantly monitors the Task DB for new requests.
  4. The Task Worker grabs the new request and sends it to the submission infrastructure (GlideInWMS).
  5. The submission infrastructure looks for available Worker Nodes and submits the jobs to them.
  6. The jobs run at the Worker Nodes.
  7. When a job finishes, the Worker Node copies the output files to the temporary storage of the site.
  8. The AsyncStageOut service transfers the output files from the temporary storage to the permanent storage.
  9. Once the transfer is done, the AsyncStageOut service publishes the output dataset in DBS.

Flow diagram of task statuses in the CRAB server database

  • task_statuses_diagram-1.png:

-- AndresTanasijczuk - 23 Oct 2014

Topic attachments
I Attachment History Action Size Date WhoSorted ascending Comment
PNGpng CRAB3_architecture_simple.png r1 manage 75.1 K 2014-10-23 - 09:00 AndresTanasijczuk Simplified diagram of the CRAB3 architecture
PDFpdf CRAB3_task_statuses_diagram.pdf r1 manage 51.0 K 2015-12-06 - 13:33 AndresTanasijczuk Flow diagram of task statuses in the database
PNGpng task_statuses_diagram-1.png r1 manage 324.8 K 2015-12-10 - 13:43 StefanoBelforte  
Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r5 - 2015-12-10 - StefanoBelforte
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback