CRAB3 Deployment Notes

CRAB3 can be deployed by anyone. This page documents the known deploys by the CRAB3 development team and the CMS computing integration team.

Known CRAB3 Frontends

  • cmsweb.cern.ch: Users should use this frontend
This is operated and deployed by the HTTP group. This frontend deploys only RPMs from tagged releases, and can only be updated once a month.

  • cmsweb-testbed.cern.ch: Testers should use this frontend
This is operated and deployed by the HTTP group; help requests should go to cms-service-webtools. This frontend deploys only RPMs from tagged releases, and can only be updated about once a week.

Known CRAB3 Backends

Access to (pre) production machines

Access to centrally maintained servers (i.e. other than personal VM's) is regulated via puppet.

Here [1] is the puppet config for the machines in production. So basically for the access the membership in these e-groups should be enough. If it does not work for some machine it means that the access to the machine is not configured by-egroup (which is the case for vocms059)

[1]
sudo:
   egroups:
     - cms-service-crab3htcondor-admins
sssd::interactiveallowgroups:
   - cms-service-crab
   - cms-service-crab2
   - cms-service-glideinwms
   - cms-service-crab3htcondor
   - cms-voc

TaskWorkers

Deployment notes and operator docs are available in a separate twiki.

  • Associated with cmsweb frontend.
    • crab-prod-tw01.cern.ch: TM operated by CMS integration (contact point is Todor Ivanov).

  • Associated with cmsweb frontend. (We have the two instances running in parallel)
    • crab-preprod-tw01.cern.ch: TM operated by CMS integration (contact point is Todor Ivanov).

  • Associated with cmsweb-dev frontend. (We have the two instances running in parallel)
    • crab-dev-tw01.cern.ch: TM operated by CMS integration (contact point is Todor Ivanov).

  • Private instances
    • c3p1.cern.ch: Operated by Marco Mascheroni. This frontend was used by Igor Sfiligoi for development and testing of GSISSH-based integration with HTCondor. Keeping it as a backup machine
    • mmascher-poc.cern.ch: Operated by Marco Mascheroni. This is used as personal development instance (run against the github repository and used for testing code). Plan to be retired in favour of: Operated by Marco Mascheroni. This is used as personal development instance. It will replace mmascher-poc.cern.ch which is SLC5
    • mmascher-dev6.cern.ch:
    • hammer-crab3.cern.ch: Operated by Marco Mascheroni. This is used as a personal development instance (mainly used to check RPMs or to run stable code. Sometimes used for early tests of new versions).
    • krabben-diener.cern.ch: Operated by Matthias Wolf. This is used as a personal development instance
    • stefanovm2.cern.ch: Operated by Stefano Belforte. This is used as a personal development instance
    • balcas-crab.cern.ch: Operated by Justas Balcas. This is used as a personal development instance
    • markv.cern.ch: Operated by Anna Woodard. This is used as a personal development instance

AsyncStageOut

Deployment notes and operator docs are available in a separate twiki.

Validated HTCondor schedds

All schedds are run by analysis operations; any questions or change requests should go to hn-cms-crabDevelopment@cernNOSPAMPLEASE.ch

Global Pool
HA-HTCondor collectors : cmssrv221.fnal.gov, vocms032.cern.ch
ITB collector: cmsgwms-collector-itb.cern.ch
Schedds : vocms0106.cern.ch vocms0107.cern.ch vocms0119.cern.ch vocms0120.cern.ch vocms0121.cern.ch vocms0122.cern.ch vocms0137.cern.ch vocms0144.cern.ch vocms0155.cern.ch vocms0194.cern.ch vocms0195.cern.ch vocms0196.cern.ch vocms0197.cern.ch vocms0198.cern.ch vocms0199.cern.ch
Schedds for developers: vocms059.cern.ch
Schedds for Site Support and HammerCloud : vocms059.cern.ch
Schedds for ITB : vocms068.cern.ch, vocms069.cern.ch vocms0115.cern.ch

HA-HTCondor collectors:

  • cmssrv221.fnal.gov - backup collector
  • vocms032.cern.ch - cmsgwms-collector-global.cern.ch

ITB collector:

  • vocms0807.cern.ch - cmsgwms-collector-itb.cern.ch

CRAB Schedds - global pool:

  • vocms0106.cern.ch
    • Status: IN PRODUCTION
  • vocms0107.cern.ch
    • Status: IN PRODUCTION
  • vocms0119.cern.ch
    • Status: IN PODUCTION
  • vocms0120.cern.ch
    • Status: IN PRODUCTION
  • vocms0121.cern.ch
    • Status: IN PRODUCTION
  • vocms0122.cern.ch
    • Status: IN PRODUCTION
  • vocms0137.cern.ch
    • Status: IN PRODUCTION
  • vocms0144.cern.ch
    • Status: IN PRODUCTION
  • vocms0155.cern.ch
    • Status: IN PRODUCTION
  • vocms0194.cern.ch
    • Status: IN PRODUCTION
  • vocms0195.cern.ch
    • Status: IN PRODUCTION
  • vocms0196.cern.ch
    • Status: IN PRODUCTION
  • vocms0197.cern.ch
    • Status: IN PRODUCTION
  • vocms0198.cern.ch
    • Status: IN PRODUCTION
  • vocms0199.cern.ch
    • Status: IN PRODUCTION

CRAB Schedds for developers: - global pool

  • vocms059.cern.ch - CRAB-DEV

CRAB Schedds - ITB pool:

  • vocms068.cern.ch
    • Status: IN PRE-PRODUCTION
  • vocms069.cern.ch
    • Status: IN PRE-PRODUCTION

Aliases for the machines in the CRAB3 project:

  • Naming convention:

crab-prod-{ scd0[0-9][0-9]*, tw0[0-9][0-9]*, rest, aso0[0-9][0-9]* }

crab-preprod-{ scd0[0-9][0-9]*, tw0[0-9][0-9]*, rest, aso0[0-9][0-9]* }

crab-dev-{ scd0[0-9][0-9]*, tw0[0-9][0-9]*, rest, aso0[0-9][0-9]* }

  • The currently existing aliases:
    • crab-prod-scd01.cern.ch -> vocms0106.cern.ch
    • crab-prod-scd02.cern.ch -> vocms0107.cern.ch
    • crab-prod-scd03.cern.ch -> vocms0119.cern.ch
    • crab-prod-scd04.cern.ch -> vocms0120.cern.ch
    • crab-prod-scd05.cern.ch -> vocms0121.cern.ch
    • crab-prod-scd06.cern.ch -> vocms0122.cern.ch
    • crab-prod-scd07.cern.ch -> vocms0137.cern.ch
    • crab-prod-scd08.cern.ch -> vocms0144.cern.ch
    • crab-prod-scd09.cern.ch -> vocms0155.cern.ch
    • crab-prod-scd010.cern.ch -> vocms0194.cern.ch
    • crab-prod-scd011.cern.ch -> vocms0195.cern.ch
    • crab-prod-scd012.cern.ch -> vocms0196.cern.ch
    • crab-prod-scd013.cern.ch -> vocms0197.cern.ch
    • crab-prod-scd014.cern.ch -> vocms0198.cern.ch
    • crab-prod-scd015.cern.ch -> vocms0199.cern.ch

    • crab-prod-tw01.cern.ch -> vocms052.cern.ch
    • crab-prod-aso01.cern.ch -> vocms0105.cern.ch
    • crab-prod-aso03.cern.ch -> vocms0109.cern.ch

    • crab-preprod-scd01.cern.ch -> vocms068.cern.ch
    • crab-preprod-scd02.cern.ch -> vocms069.cern.ch
    • crab-preprod-tw01.cern.ch -> vocms0118.cern.ch
    • crab-preprod-aso01.cern.ch -> vocms030.cern.ch

    • crab-dev-scd01.cern.ch -> vocms059.cern.ch
    • crab-dev-scd02.cern.ch -> vocms0115.cern.ch
    • crab-dev-tw01.cern.ch -> vocms058.cern.ch
    • crab-dev-rest01.cern.ch -> vocms035.cern.ch

Other Useful Links

Official twikis for all the AI nodes (Ivan), glideinWMS deployment nodes (Farrukh) and cmsweb nodes (Diego):

https://twiki.cern.ch/twiki/bin/viewauth/CMS/CMSAiNodes

https://twiki.cern.ch/twiki/bin/view/CMSPublic/CompOpsWMSDeploy

https://cms-http-group.web.cern.ch/cms-http-group/activity.html

Edit | Attach | Watch | Print version | History: r42 < r41 < r40 < r39 < r38 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r42 - 2019-09-20 - LeonardoCristella
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback