TWiki> LHCb Web>LHCbComputing>Vcycle (revision 1)EditAttachPDF

LHCb Vcycle

LHCb uses Vcycle to manage resources on the CERN and Yandex OpenStack sites. Currently lbvobox200 hosts the Vcycle agent. Vcycle is not part of DIRAC and so not currently managed through the DIRAC system administration framework.

Vcycle's configuration files are in /etc/vcycle.d/ with one file per `space' where a space is an OpenStack project in this case. Vcycle's log files are in /var/log/vcycle . The rest of Vcycle's files are in /var/lib/vcycle which is in reality a symbolic link to /opt/dirac/vcycle for space reasons.

Configuration

There is a file in /etc/vcycle.d for each of CERN and Yandex. cern-prod.conf and yandex.conf . Each file has a [space ...] section with general configuration options. This includes the username and password to contact the OpenStack API. The password is base64 encoded to prevent casual 'over the shoulder' disclosure of passwords and it can be encoded or decoded from the command line using the base64 command (Vcycle strips off extra newlines if they are included in the encoding by mistake.)

Then there is a [vacuum_pipe ...] section which gives the URL of the LHCb vacuum pipe JSON file which describes how to create LHCb VMs. The pipe is at https://lhcb-portal-dirac.cern.ch/pilot/lhcb.pipe The same pipe is also used for the Vac sites, which run the same LHCb VMs. Yandex has some additional configuration options, due to the Squid caches we run there.

It is not necessary tor restart Vcycle when the configuration changes: a new configuration is assembled and printed to the Vcycle log file at the start of each cycle - once every few minutes.

Machinetypes

Vcycle keeps a cached copy of the pipe file in /var/lib/vcycle/spaces/vcycle-cern.lhcb.cern/machinetypes/lhcb/vacuum.pipe and processes options from it as if they were included in configuration files in /etc/vcycle.d (some of the options are disallowed for security reasons.) The resulting configuration is included in the overall configuration written to the Vcycle log file. The options can be compared with the vcycle.conf man page to see their implications. The pipe file defines several machinetypes, each of which gets its own section in the configuration Vcycle works from. lhcb-vm-prod is the one used to run DIRAC jobs.

The configuration refers to an X.509 host certificate and key, which are given to the VMs to allow them to authenticate to DIRAC. These files are stored in /var/lib/vcycle/certs

Yandex has lhcb-vm-prod VMs and also lhcb-squid VMs which provide the squid caching proxies used by CernVM -FS inside the prod VMs. Unlike the prod VMs, the squid VMs using static CentOS 6 images. Vcycle maintains a list of active squid VMs (based on whether they are sending heartbeats) and gives this list to the prod VMs when they are created as their list of squid URLs for CernVM -FS to use.

Machines

When Vcycle decides to creates a VM, it first creates a directory to shadow it in /var/lib/vcycle/shared/current/SPACE/MACHINE where SPACE is the space name (eg vcycle-cern.lhcb.cern) and the hostname of the VM. Files are created and updated in the directory during the lifetime of the VM, until finally the VM is no longer visible in OpenStack and the directory is then moved to /var/lib/vcycle/shared/deleted/SPACE/MACHINE where it can be examined for debugging.

In particular, the joboutputs subdirectory of the machine directory receives heartbeat files every few minutes and eventually log files (include DIRAC pilot and jobagent log files) just before the VM shuts down.

httpd

There is an Apache web server running on lbvobox200 which receives the joboutputs files and publishes the machinefeatures and jobfeatures directories specific to each machine. The host certificate and key for the web server are maintained automatically in the usual place: /etc/pki/tls/ and there are symbolic links to them from /etc/grid-security where the web server configuration expects them. The Apache httpd log files are in /var/log/httpd as usual.

-- AndrewMcNab - 2019-06-18

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2019-06-18 - AndrewMcNab
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback