Difference: VacConfiguration (1 vs. 21)

Revision 212019-07-21 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 01.00.00 or above. This is needed for CernVM 3 and user_data template support in Vac.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 03.00.00 or above. This is needed for Vacuum Pipe support in Vac.

These instructions also provide most of the information needed to configure Vcycle for LHCb.

 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
Changed:
<
<
  • One or more CE's are grouped together to form a site, which will take the form VAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
>
>
  • One or more CE's are grouped together to form a site, which will take the form VAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using WLCG middleware like the ARC CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
 
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
Changed:
<
<
  • Place the hostcert.pem and hostkey.pem of the certificate in the lhcb (or similar) subdirectory of /var/lib/vac/machinetypes
>
>
  • Place the hostcert.pem and hostkey.pem of the certificate in the files subdirectory of the lhcb (or similar) subdirectory of /var/lib/vac/machinetypes . For example, /var/lib/vac/machinetypes/lhcb/files/hostcert.pem
 
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
  • Create a volume group vac_volume_group which is big enough to hold one 40GB logical volume for each VM the factory machine will run at the same time.
Changed:
<
<
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.
>
>
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on WLCG worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely. In Vac 3.0 you can set a global proxy used for all vacuum pipes and machine types in the [settings] section.
 

Adding lhcb to vac.conf

Changed:
<
<
The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb (or similar) section should look like this, :
[machinetype lhcb]
>
>
The details of the vac.conf options are given in the vac.conf(5) man page. For LHCb the configuration should include lines like this, :

[settings]
...
 user_data_option_cvmfs_proxy = http://squid-cache.example.cc:3128
Changed:
<
<
user_data_file_hostcert = hostcert.pem user_data_file_hostkey = hostkey.pem user_data = https://lhcb-portal-dirac.cern.ch/pilot/user_data machine_model = cernvm3 root_image = https://lhcbproject.web.cern.ch/lhcbproject/Operations/VM/cernvm3.iso rootpublickey = /root/.ssh/id_rsa.pub backoff_seconds = 600 fizzle_seconds = 600 max_wallclock_seconds = 172800 heartbeat_file = heartbeat heartbeat_seconds = 600 accounting_fqan=/lhcb/Role=NULL/Capability=NULL
>
>
... [vacuum_pipe lhcb] vacuum_pipe_url = https://lhcb-portal-dirac.cern.ch/pilot/lhcb.pipe target_share = CHANGEME

Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in the log files stored in subdirectories of /var/lib/vac/machines . These matches will shutdown quickly until either there are some test jobs directed to your site (eg LHCb Test jobs) or your site is put in the production mask and can receive Monte Carlo jobs.

Most of the VMs created will have a machinetype like lhcb-vm-prod but some will be created as lhcb-vm-mcprod. These are elastic VMs that only run Monte Carlo simulation and are designed to backfill unused slots where there is a mixture of single and multiprocessor VMs.

 
Changed:
<
<
Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)
>
>

Multiprocessor LHCb VMs

 
Changed:
<
<
If no work is available from the central LHCb task queue and a VM stops with 'Nothing to do', backoff_seconds determines how long Vac waits before trying to run an LHCb VM again. This waiting is co-ordinated between all factory machines in a space using Vac's UDP protocol.
>
>
If you have Vac's SuperSlots mechanism enabled (ie you have something like processors_per_superslot = 8 in [settings]), then you can enable 8-processor LHCb VMs which can run LHCb multiprocessor payloads. Do this by creating a short machinetype definition like this:
 
Changed:
<
<
You can omit the rootpublickey option, but it is extremely useful for debugging. See the Vac Admin Guide for more about how to set it up.
>
>
[machinetype lhcb-vm8-prod]
target_share = CHANGEME
 
Changed:
<
<
Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in the log files stored in subdirectories of /var/lib/vac/machines . These matches will fail until either there are some test jobs directed to your site (eg LHCb Test jobs) or your site is put in the production mask and can receive Monte Carlo jobs.

>
>
The target share of the 8 processor VMs will count as part of the total share of the LHCb vacuum_pipe section. So if you have target_share = 25 for the LHCb vacuum pipe, then if you set target_share = 5 for the 8 processor VMs then they will get a fifth of the total LHCb share, leaving 20 for the main lhcb-vm-prod single-processor VMs (assuming both types of VM have plenty of payload jobs to keep them as busy as they are allowed by your configuration.)

Revision 202016-11-19 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.17.0 or above. This is needed for CernVM 3 and user_data template support in Vac.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 01.00.00 or above. This is needed for CernVM 3 and user_data template support in Vac.
 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

Line: 10 to 11
 
  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
  • One or more CE's are grouped together to form a site, which will take the form VAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
Changed:
<
<
  • Place the hostcert.pem and hostkey.pem of the certificate in the lhcb (or similar) subdirectory of /var/lib/vac/vmtypes
>
>
  • Place the hostcert.pem and hostkey.pem of the certificate in the lhcb (or similar) subdirectory of /var/lib/vac/machinetypes
 
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
  • Create a volume group vac_volume_group which is big enough to hold one 40GB logical volume for each VM the factory machine will run at the same time.
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.

Adding lhcb to vac.conf

Changed:
<
<
With Vac 0.17.0, it is no longer necessary for sites to create custom user_data files, as Vac can create them on the fly using a template obtained from the LHCb project webserver. The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb (or similar) section should look like this, :
[vmtype lhcb]
user_data_option_dirac_site = VAC.Example.cc
>
>
The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb (or similar) section should look like this, :
[machinetype lhcb]
 user_data_option_cvmfs_proxy = http://squid-cache.example.cc:3128 user_data_file_hostcert = hostcert.pem user_data_file_hostkey = hostkey.pem
Changed:
<
<
user_data = https://lhcbproject.web.cern.ch/lhcbproject/Operations/VM/user_data vm_model = cernvm3
>
>
user_data = https://lhcb-portal-dirac.cern.ch/pilot/user_data machine_model = cernvm3
 root_image = https://lhcbproject.web.cern.ch/lhcbproject/Operations/VM/cernvm3.iso rootpublickey = /root/.ssh/id_rsa.pub backoff_seconds = 600 fizzle_seconds = 600 max_wallclock_seconds = 172800
Changed:
<
<
heartbeat_file = vm-heartbeat
>
>
heartbeat_file = heartbeat
 heartbeat_seconds = 600
Deleted:
<
<
log_machineoutputs = True
 accounting_fqan=/lhcb/Role=NULL/Capability=NULL

Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)

Line: 41 to 40
  You can omit the rootpublickey option, but it is extremely useful for debugging. See the Vac Admin Guide for more about how to set it up.
Changed:
<
<
With log_machineoutputs set to True, log files from within the VM will be copied to subdirectories of /var/lib/vac/machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for it if you run into problems.

Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in the log files stored in subdirectories of /var/lib/vac/machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.

>
>
Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in the log files stored in subdirectories of /var/lib/vac/machines . These matches will fail until either there are some test jobs directed to your site (eg LHCb Test jobs) or your site is put in the production mask and can receive Monte Carlo jobs.
 

\ No newline at end of file

Revision 192014-12-11 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 36 to 35
 log_machineoutputs = True accounting_fqan=/lhcb/Role=NULL/Capability=NULL
Deleted:
<
<
If you are using a different version of the CernVM 3 image, you will need to change the exact file name in the root_image option.
 Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)

If no work is available from the central LHCb task queue and a VM stops with 'Nothing to do', backoff_seconds determines how long Vac waits before trying to run an LHCb VM again. This waiting is co-ordinated between all factory machines in a space using Vac's UDP protocol.

Revision 182014-12-09 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.16.0 or above. This is needed for CernVM 3 support in Vac.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.17.0 or above. This is needed for CernVM 3 and user_data template support in Vac.
 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

Line: 10 to 10
 
  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
  • One or more CE's are grouped together to form a site, which will take the form VAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
Added:
>
>
  • Place the hostcert.pem and hostkey.pem of the certificate in the lhcb (or similar) subdirectory of /var/lib/vac/vmtypes
 
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
Changed:
<
<
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name vdb, as the root disk images aren't big enough for LHCb jobs.
>
>
  • Create a volume group vac_volume_group which is big enough to hold one 40GB logical volume for each VM the factory machine will run at the same time.
 
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.
Deleted:
<
<

Creating the user_data file

The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is very simple as cvmfs within the VM is used for most of the configuration. Just adapt the following steps to use your own site and host specific values:

cd /tmp
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/vacprod/siteconfig
cd siteconfig
./make_user_data --hostcert ~/lhcb-vm.example.cc.cert.pem \
  --hostkey ~/lhcb-vm.example.cc.key.pem --site VAC.Example.cc \
  --cvmfsproxy http://squid-cache.example.cc:3128
mkdir -p /var/lib/vac/vmtypes/lhcb
cp /tmp/user_data.XXXXXXXX /var/lib/vac/vmtypes/lhcb/user_data

The exact value of the /tmp/user_data.XXXXXXXX path is output by the make_user_data script each time you run it.

The options use the values set out in the Requirements section above. --cvmfsproxy can be omitted if you are not using a caching proxy for cvmfs, but the other options are required.

It is no longer necessary to copy vac-shutdown-vm to the shared directory.

 

Adding lhcb to vac.conf

Changed:
<
<
The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb section should look like this:
[vmtype lhcb]
>
>
With Vac 0.17.0, it is no longer necessary for sites to create custom user_data files, as Vac can create them on the fly using a template obtained from the LHCb project webserver. The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb (or similar) section should look like this, :

[vmtype lhcb]
user_data_option_dirac_site = VAC.Example.cc
user_data_option_cvmfs_proxy = http://squid-cache.example.cc:3128
user_data_file_hostcert = hostcert.pem
user_data_file_hostkey = hostkey.pem 
user_data = https://lhcbproject.web.cern.ch/lhcbproject/Operations/VM/user_data
 vm_model = cernvm3
Changed:
<
<
root_image = /usr/share/images/ucernvm-prod.1.16-3.cernvm.x86_64.iso
>
>
root_image = https://lhcbproject.web.cern.ch/lhcbproject/Operations/VM/cernvm3.iso
 rootpublickey = /root/.ssh/id_rsa.pub
Deleted:
<
<
user_data = user_data
 backoff_seconds = 600 fizzle_seconds = 600 max_wallclock_seconds = 172800
Line: 48 to 35
 heartbeat_seconds = 600 log_machineoutputs = True accounting_fqan=/lhcb/Role=NULL/Capability=NULL
Deleted:
<
<
  If you are using a different version of the CernVM 3 image, you will need to change the exact file name in the root_image option.

Revision 172014-04-27 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.15.1 or above. This is needed for CernVM 3 support in Vac.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.16.0 or above. This is needed for CernVM 3 support in Vac.
 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

Line: 11 to 11
 
  • One or more CE's are grouped together to form a site, which will take the form VAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
Changed:
<
<
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.
>
>
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name vdb, as the root disk images aren't big enough for LHCb jobs.
 
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.

Creating the user_data file

Line: 50 to 50
 accounting_fqan=/lhcb/Role=NULL/Capability=NULL
Changed:
<
<
If you are using a different version of the CernVM image, you will need to change the exact file name in the root_image option.
>
>
If you are using a different version of the CernVM 3 image, you will need to change the exact file name in the root_image option.
  Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)
Line: 60 to 60
  With log_machineoutputs set to True, log files from within the VM will be copied to subdirectories of /var/lib/vac/machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for it if you run into problems.
Changed:
<
<
Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in /var/log/vacd-machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.
>
>
Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in the log files stored in subdirectories of /var/lib/vac/machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.
 

Revision 162014-04-27 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.13.0 or above.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.15.1 or above. This is needed for CernVM 3 support in Vac.
 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

Line: 17 to 17
  The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is very simple as cvmfs within the VM is used for most of the configuration. Just adapt the following steps to use your own site and host specific values:
cd /tmp
Changed:
<
<
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/vacprod/scripts/ cd scripts ./user_data_generator --mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem
>
>
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/vacprod/siteconfig cd siteconfig ./make_user_data --hostcert ~/lhcb-vm.example.cc.cert.pem
  --hostkey ~/lhcb-vm.example.cc.key.pem --site VAC.Example.cc
Changed:
<
<
--gridce vac01.example.cc --cvmfsproxy http://squid-cache.example.cc:3128
>
>
--cvmfsproxy http://squid-cache.example.cc:3128
 mkdir -p /var/lib/vac/vmtypes/lhcb
Changed:
<
<
cp /tmp/tmp.XXXXXXXX/user_data /var/lib/vac/vmtypes/lhcb/user_data
>
>
cp /tmp/user_data.XXXXXXXX /var/lib/vac/vmtypes/lhcb/user_data
 
Changed:
<
<
The exact value of the /tmp/tmp.XXXXXXXX/user_data path is output by the user_data_generator each time you run it.
>
>
The exact value of the /tmp/user_data.XXXXXXXX path is output by the make_user_data script each time you run it.
  The options use the values set out in the Requirements section above. --cvmfsproxy can be omitted if you are not using a caching proxy for cvmfs, but the other options are required.
Deleted:
<
<

Copying vac-shutdown-vm to the shared directory

 
Changed:
<
<
Next copy vac-shutdown-vm to /var/lib/vac/vmtypes/lhcb/shared and ensure it is executable:
cp /var/lib/vac/bin/vac-shutdown-vm /var/lib/vac/vmtypes/lhcb/shared/vac-shutdown-vm
chmod +x  /var/lib/vac/vmtypes/lhcb/shared/vac-shutdown-vm
>
>
It is no longer necessary to copy vac-shutdown-vm to the shared directory.
 
Deleted:
<
<
This script is visible inside the VM at /etc/vmtypefiles/vac-shutdown-vm and is used to shut the VM down when the job is finished.
 

Adding lhcb to vac.conf

The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb section should look like this:

[vmtype lhcb]
Changed:
<
<
root_image = /var/lib/vac/images/cernvm-batch-node-2.6.0-4-1-x86_64.hdd
>
>
vm_model = cernvm3 root_image = /usr/share/images/ucernvm-prod.1.16-3.cernvm.x86_64.iso
 rootpublickey = /root/.ssh/id_rsa.pub user_data = user_data
Deleted:
<
<
shutdown_command = /etc/vmtypefiles/vac-shutdown-vm
 backoff_seconds = 600 fizzle_seconds = 600 max_wallclock_seconds = 172800
Added:
>
>
heartbeat_file = vm-heartbeat heartbeat_seconds = 600
 log_machineoutputs = True accounting_fqan=/lhcb/Role=NULL/Capability=NULL
Line: 61 to 58
  You can omit the rootpublickey option, but it is extremely useful for debugging. See the Vac Admin Guide for more about how to set it up.
Changed:
<
<
With log_machineoutputs set to True, the stdout of the jobs will be appended to /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for it if you run into problems.
>
>
With log_machineoutputs set to True, log files from within the VM will be copied to subdirectories of /var/lib/vac/machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for it if you run into problems.
  Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in /var/log/vacd-machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.

\ No newline at end of file

Revision 152014-03-12 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 49 to 49
 backoff_seconds = 600 fizzle_seconds = 600 max_wallclock_seconds = 172800
Changed:
<
<
log_machineoutputs = True
>
>
log_machineoutputs = True accounting_fqan=/lhcb/Role=NULL/Capability=NULL
 

If you are using a different version of the CernVM image, you will need to change the exact file name in the root_image option.

Revision 142014-03-06 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 60 to 60
  You can omit the rootpublickey option, but it is extremely useful for debugging. See the Vac Admin Guide for more about how to set it up.
Changed:
<
<
With log_machineoutputs set to True, the stdout of the jobs will be appended to /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for if you run into problems.
>
>
With log_machineoutputs set to True, the stdout of the jobs will be appended to /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for it if you run into problems.
  Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in /var/log/vacd-machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.

Deleted:
<
<

Problems

For a few hours after a change to the global LHCbDIRAC version, you may see VMs failing to find work with the shutdown message: "300 Cannot match jobs with this pilot version". This situation goes away once the updated version has propagated through cvmfs properly.

 \ No newline at end of file

Revision 132014-02-14 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 66 to 66
 

Problems

Deleted:
<
<
For a few hours after a change to the global LHCbDIRAC version, you may see VMs failing to find work with the shutdown message: "600 Failed, probably JobAgent or Application problem". The detailed errors in vacd-machineoutputs refer to dirac-agent traced down to "undefined symbol: PyUnicodeUCS2 _Decode". This situation goes away once the updated version has propagated properly.
 \ No newline at end of file
Added:
>
>
For a few hours after a change to the global LHCbDIRAC version, you may see VMs failing to find work with the shutdown message: "300 Cannot match jobs with this pilot version". This situation goes away once the updated version has propagated through cvmfs properly.
 \ No newline at end of file

Revision 122014-02-14 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 17 to 17
  The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is very simple as cvmfs within the VM is used for most of the configuration. Just adapt the following steps to use your own site and host specific values:
cd /tmp
Changed:
<
<
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/vacprod/LHCbVMDIRAC/VM/scripts/
>
>
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/vacprod/scripts/
 cd scripts ./user_data_generator --mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem --hostkey ~/lhcb-vm.example.cc.key.pem --site VAC.Example.cc

Revision 112014-02-13 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.11.0 or above.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.13.0 or above.
 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

Line: 30 to 30
 The exact value of the /tmp/tmp.XXXXXXXX/user_data path is output by the user_data_generator each time you run it.

The options use the values set out in the Requirements section above. --cvmfsproxy can be omitted if you are not using a caching proxy for cvmfs, but the other options are required.

Added:
>
>

Copying vac-shutdown-vm to the shared directory

Next copy vac-shutdown-vm to /var/lib/vac/vmtypes/lhcb/shared and ensure it is executable:

cp /var/lib/vac/bin/vac-shutdown-vm /var/lib/vac/vmtypes/lhcb/shared/vac-shutdown-vm
chmod +x  /var/lib/vac/vmtypes/lhcb/shared/vac-shutdown-vm

This script is visible inside the VM at /etc/vmtypefiles/vac-shutdown-vm and is used to shut the VM down when the job is finished.

 

Adding lhcb to vac.conf

The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb section should look like this:

Line: 37 to 45
 root_image = /var/lib/vac/images/cernvm-batch-node-2.6.0-4-1-x86_64.hdd rootpublickey = /root/.ssh/id_rsa.pub user_data = user_data
Changed:
<
<
shutdown_command = /etc/machinefeatures/vac-shutdown-vm
>
>
shutdown_command = /etc/vmtypefiles/vac-shutdown-vm
 backoff_seconds = 600 fizzle_seconds = 600 max_wallclock_seconds = 172800

Revision 102014-02-08 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 8 to 8
 Before configuring Vac for LHCb, you need to follow these steps:

  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
Changed:
<
<
  • One or more CE's are grouped together to form a site, which will take the form DIRAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
>
>
  • One or more CE's are grouped together to form a site, which will take the form VAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
 
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.

Revision 92013-12-06 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 15 to 15
 
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.

Creating the user_data file

Changed:
<
<
The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is currently being simplified to rely more on cvmfs, but for now you should adapt the following steps to use your own values:
>
>
The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is very simple as cvmfs within the VM is used for most of the configuration. Just adapt the following steps to use your own site and host specific values:
 
cd /tmp
Changed:
<
<
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130624/LHCbVMDIRAC/VM/scripts/
>
>
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/vacprod/LHCbVMDIRAC/VM/scripts/
 cd scripts ./user_data_generator --mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem --hostkey ~/lhcb-vm.example.cc.key.pem --site VAC.Example.cc --gridce vac01.example.cc --cvmfsproxy http://squid-cache.example.cc:3128 mkdir -p /var/lib/vac/vmtypes/lhcb
Changed:
<
<
cp user_data /var/lib/vac/vmtypes/lhcb/user_data
>
>
cp /tmp/tmp.XXXXXXXX/user_data /var/lib/vac/vmtypes/lhcb/user_data
 
Added:
>
>
The exact value of the /tmp/tmp.XXXXXXXX/user_data path is output by the user_data_generator each time you run it.
 The options use the values set out in the Requirements section above. --cvmfsproxy can be omitted if you are not using a caching proxy for cvmfs, but the other options are required.

Adding lhcb to vac.conf

Revision 82013-08-06 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 8 to 8
 Before configuring Vac for LHCb, you need to follow these steps:

  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
Changed:
<
<
  • One or more CE's are grouped together to form a site, which will take the form DIRAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg DIRAC.CERN.ch or DIRAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated DIRAC.Example.cc for use with Vac.
>
>
  • One or more CE's are grouped together to form a site, which will take the form DIRAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg VAC.CERN.ch or VAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated VAC.Example.cc for use with Vac.
 
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.
Line: 20 to 20
 svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130624/LHCbVMDIRAC/VM/scripts/ cd scripts ./user_data_generator --mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem
Changed:
<
<
--hostkey ~/lhcb-vm.example.cc.key.pem --site DIRAC.Example.cc
>
>
--hostkey ~/lhcb-vm.example.cc.key.pem --site VAC.Example.cc
  --gridce vac01.example.cc --cvmfsproxy http://squid-cache.example.cc:3128 mkdir -p /var/lib/vac/vmtypes/lhcb cp user_data /var/lib/vac/vmtypes/lhcb/user_data

Revision 72013-07-08 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 53 to 53
 With log_machineoutputs set to True, the stdout of the jobs will be appended to /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for if you run into problems.

Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in /var/log/vacd-machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs. \ No newline at end of file

Added:
>
>

Problems

For a few hours after a change to the global LHCbDIRAC version, you may see VMs failing to find work with the shutdown message: "600 Failed, probably JobAgent or Application problem". The detailed errors in vacd-machineoutputs refer to dirac-agent traced down to "undefined symbol: PyUnicodeUCS2_Decode". This situation goes away once the updated version has propagated properly.

 \ No newline at end of file

Revision 62013-07-04 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="Vac"

Vac Configuration for LHCb

Line: 19 to 19
 
cd /tmp
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130624/LHCbVMDIRAC/VM/scripts/
cd scripts
Changed:
<
<
./user_data_generator -mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem
>
>
./user_data_generator --mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem
  --hostkey ~/lhcb-vm.example.cc.key.pem --site DIRAC.Example.cc --gridce vac01.example.cc --cvmfsproxy http://squid-cache.example.cc:3128 mkdir -p /var/lib/vac/vmtypes/lhcb

Revision 52013-07-04 - AndrewMcNab

Line: 1 to 1
Changed:
<
<
META TOPICPARENT name="LHCbComputing"
>
>
META TOPICPARENT name="Vac"
 

Vac Configuration for LHCb

This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.11.0 or above.

Revision 42013-06-25 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.11.0 or above.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.11.0 or above.
 

Requirements

Before configuring Vac for LHCb, you need to follow these steps:

  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
Changed:
<
<
  • One or more CE's are grouped together to form a site, which will take the form DIRAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg DIRAC.CERN.ch or DIRAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site already has a site using gLite/EMI middleware like the CREAM CE, it will probably have a site name like LCG.Example.cc and you would normally be allocated DIRAC.Example.cc for use with Vac.
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should probably use a certificate which is specific to LHCb but part of your site's DNS space. It doesn't need to correspond to a real host or really exist on your DNS servers: just that you are entitled to use register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( Andrew . McNab @ cern . ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
>
>
  • One or more CE's are grouped together to form a site, which will take the form DIRAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg DIRAC.CERN.ch or DIRAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site is already using gLite/EMI middleware like the CREAM CE for LHCb, it will probably have a site name like LCG.Example.cc and you would normally be allocated DIRAC.Example.cc for use with Vac.
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should normally use a name which is specific to LHCb but is part of your site's DNS space. It doesn't need to correspond to a real host or really exist as an entry on your DNS servers: just that you are entitled to register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( andrew.mcnab AT cern.ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
 
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.

Creating the user_data file

Line: 46 to 46
  Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)
Changed:
<
<
If no work is available from the central LHCb task queue and a VM stops with 'Nothing to do', backoff_seconds determines how long Vac waits before trying to run an LHCb VM again. This information is shared between all factory machines in a space using Vac's UDP protocol.
>
>
If no work is available from the central LHCb task queue and a VM stops with 'Nothing to do', backoff_seconds determines how long Vac waits before trying to run an LHCb VM again. This waiting is co-ordinated between all factory machines in a space using Vac's UDP protocol.
  You can omit the rootpublickey option, but it is extremely useful for debugging. See the Vac Admin Guide for more about how to set it up.
Changed:
<
<
With log_machineoutputs set to True, the stdout of the jobs will appear in /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for if you run into problems.
>
>
With log_machineoutputs set to True, the stdout of the jobs will be appended to /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for if you run into problems.
  Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in /var/log/vacd-machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.

Revision 32013-06-25 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Vac Configuration for LHCb

Line: 12 to 12
 
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should probably use a certificate which is specific to LHCb but part of your site's DNS space. It doesn't need to correspond to a real host or really exist on your DNS servers: just that you are entitled to use register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( Andrew . McNab @ cern . ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.
Changed:
<
<
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on glite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution more likely.
>
>
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on gLite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution will be more likely.
 

Creating the user_data file

Changed:
<
<
The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is currently being simplified to rely more on cvmfs, but for you should use the following steps:
>
>
The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is currently being simplified to rely more on cvmfs, but for now you should adapt the following steps to use your own values:
 
cd /tmp
Changed:
<
<
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130602/LHCbVMDIRAC/VM/scripts/
>
>
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130624/LHCbVMDIRAC/VM/scripts/
 cd scripts ./user_data_generator -mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem --hostkey ~/lhcb-vm.example.cc.key.pem --site DIRAC.Example.cc

Revision 22013-06-24 - AndrewMcNab

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Vac Configuration for LHCb

Changed:
<
<
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the [http://www.gridpp.ac.uk/vac/][Vac website] for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory.
>
>
This page explains how to run LHCb virtual machines on Vac factory machines. Please see the Vac website for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory. These instructions are based on Vac 0.11.0 or above.
 

Requirements

Changed:
<
<
When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
>
>
Before configuring Vac for LHCb, you need to follow these steps:
 
Changed:
<
<
One or more CE's are grouped together to form a site, which will take the form DIRAC.Site.cc where Site is derived from your institutional name and cc is the country code. eg DIRAC.CERN.ch or DIRAC.Manchester.uk

Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.

>
>
  • When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.
  • One or more CE's are grouped together to form a site, which will take the form DIRAC.Example.cc where Example is derived from your institutional name and cc is the country code. eg DIRAC.CERN.ch or DIRAC.Manchester.uk. Site names are allocated and registered in the Dirac configuration service by the LHCb ops team. If your site already has a site using gLite/EMI middleware like the CREAM CE, it will probably have a site name like LCG.Example.cc and you would normally be allocated DIRAC.Example.cc for use with Vac.
  • Obtain a host certificate which the VMs can use as a client certificate to fetch work from the central LHCb task queue. One certificate can be used for all LHCb VMs at a site. You should probably use a certificate which is specific to LHCb but part of your site's DNS space. It doesn't need to correspond to a real host or really exist on your DNS servers: just that you are entitled to use register it. So if your site's domain name is example.cc then a certificate for lhcb-vm.example.cc with a DN like /C=CC/O=XYZ/CN=lhcb-vm.example.cc would be a good choice.
  • Contact someone in the ops team ( Andrew . McNab @ cern . ch at the moment) to agree a site name and to register your CE, Site, and certificate DN in the central LHCb DIRAC configuration.
  • Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.
  • Identify a squid HTTP caching proxy to use with cvmfs. If you already have a proxy set up for cvmfs on glite/EMI worker nodes at your site then you can use that too. You may be able to run without a proxy, but SetupProject failures during LHCb job execution more likely.

Creating the user_data file

 
Changed:
<
<
Obtain a host certificate which the VMs can use as client certificates to fetch work from the central LHCb task queue. One certificate can be used for all VMs at a site.
>
>
The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is currently being simplified to rely more on cvmfs, but for you should use the following steps:
cd /tmp
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130602/LHCbVMDIRAC/VM/scripts/
cd scripts
./user_data_generator -mode vac --hostcert ~/lhcb-vm.example.cc.cert.pem \
  --hostkey ~/lhcb-vm.example.cc.key.pem --site DIRAC.Example.cc \
  --gridce vac01.example.cc --cvmfsproxy http://squid-cache.example.cc:3128
mkdir -p /var/lib/vac/vmtypes/lhcb
cp user_data /var/lib/vac/vmtypes/lhcb/user_data
 
Added:
>
>
The options use the values set out in the Requirements section above. --cvmfsproxy can be omitted if you are not using a caching proxy for cvmfs, but the other options are required.
 

Adding lhcb to vac.conf

The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb section should look like this:

Line: 32 to 47
 Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)

If no work is available from the central LHCb task queue and a VM stops with 'Nothing to do', backoff_seconds determines how long Vac waits before trying to run an LHCb VM again. This information is shared between all factory machines in a space using Vac's UDP protocol.

Deleted:
<
<

Creating the user_data file

 
Changed:
<
<
The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is currently being simplified to rely more on cvmfs, but for you should use the following steps:
cd /tmp
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130602/LHCbVMDIRAC/VM/scripts/
cd scripts
./user_data_generator -mode vac --hostcert ~/lhcb-vm.example.com.cert.pem \
  --hostkey ~/lhcb-vm.example.com.key.pem --site DIRAC.Example.co \
  --gridce vac01.example.com --cvmfsproxy http://squid-cache.example.com:3128
mkdir -p /var/lib/vac/vmtypes/lhcb
cp user_data /var/lib/vac/vmtypes/lhcb
>
>
You can omit the rootpublickey option, but it is extremely useful for debugging. See the Vac Admin Guide for more about how to set it up.

With log_machineoutputs set to True, the stdout of the jobs will appear in /var/log/vacd-machineoutputs once the VM has finished. Again, this is very useful for debugging and something the ops team may ask you for if you run into problems.

 
Changed:
<
<
If you
>
>
Vac re-reads its configuration files at every cycle (once a minute or so) and so the changes to vac.conf will take effect almost immediately. You should see Vac creating lhcb VMs in /var/log/vacd-factory and the VMs themselves attempting to contact the LHCb matcher to fetch work in /var/log/vacd-machineoutputs . These matches will fail until either there are some test jobs directed to your site (eg LHCb SAM tests) or your site is put in the production mask and can receive Monte Carlo jobs.

Revision 12013-06-24 - AndrewMcNab

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbComputing"

Vac Configuration for LHCb

This page explains how to run LHCb virtual machines on Vac factory machines. Please see the [http://www.gridpp.ac.uk/vac/][Vac website] for Vac's Admin Guide and man pages, which explain how to install and configure Vac itself and get a working Vac factory.

Requirements

When you configure Vac, you need to choose a Vac space name. This will be used as the Computing Element (CE) name in LHCb DIRAC.

One or more CE's are grouped together to form a site, which will take the form DIRAC.Site.cc where Site is derived from your institutional name and cc is the country code. eg DIRAC.CERN.ch or DIRAC.Manchester.uk

Make a scratch logical volume of about 25GB available to the VM, with Vac's default device name hdb, as the root disk images aren't big enough for LHCb jobs.

Obtain a host certificate which the VMs can use as client certificates to fetch work from the central LHCb task queue. One certificate can be used for all VMs at a site.

Adding lhcb to vac.conf

The details of the vac.conf options are given in the vac.conf(5) man page. However, the lhcb section should look like this:

[vmtype lhcb]
root_image = /var/lib/vac/images/cernvm-batch-node-2.6.0-4-1-x86_64.hdd
rootpublickey = /root/.ssh/id_rsa.pub
user_data = user_data
shutdown_command = /etc/machinefeatures/vac-shutdown-vm
backoff_seconds = 600 
fizzle_seconds = 600
max_wallclock_seconds = 172800
log_machineoutputs = True

If you are using a different version of the CernVM image, you will need to change the exact file name in the root_image option.

Vac will destroy the VM if it runs for more than max_wallclock_seconds and you may want to experiment with shorter values. Most modern machines should be able to run jobs comfortably within 24 hours (86400 seconds.)

If no work is available from the central LHCb task queue and a VM stops with 'Nothing to do', backoff_seconds determines how long Vac waits before trying to run an LHCb VM again. This information is shared between all factory machines in a space using Vac's UDP protocol.

Creating the user_data file

The user_data file is passed to the VM by Vac and configures it for LHCb ("contextualisation"). The procedure for creating user_data is currently being simplified to rely more on cvmfs, but for you should use the following steps:

cd /tmp
svn co http://svn.cern.ch/guest/dirac/LHCbVMDIRAC/tags/mcnab-20130602/LHCbVMDIRAC/VM/scripts/
cd scripts
./user_data_generator -mode vac --hostcert ~/lhcb-vm.example.com.cert.pem \
  --hostkey ~/lhcb-vm.example.com.key.pem --site DIRAC.Example.co \
  --gridce vac01.example.com --cvmfsproxy http://squid-cache.example.com:3128
mkdir -p /var/lib/vac/vmtypes/lhcb
cp user_data /var/lib/vac/vmtypes/lhcb

If you

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback