TWiki> LCG Web>LCGGridDeployment>LcgDocs>YaimGuide311 (revision 20)EditAttachPDF

YAIM 3.1.1 guide for sysadmins

NOTE: YAIM 4.0.0 is now available. You may want to upgrade to the lastest version and visit the YAIM 4 guide.

Introduction

This document provides a description of YAIM version 3.1.1. This YAIM release is a merge of YAIM 3.0.1-22 and YAIM 3.1.0. It allows to configure all gLite 3.0 and gLite 3.1 services. It contains a modular structure consisting of a main yaim core module, yaim clients and yaim services. Not all the gLite services have a yaim module in this relase. This will change in future releases. Please check the Known Issues section for more details.

For the general installation process check:

Basics

What is YAIM

The aim of YAIM (Yet Another Installation Manager) is to implement a configuration method for the LCG and gLite software. YAIM is a set of bash scripts and functions. YAIM is distributed in rpm form and it usually resides in /opt/glite/yaim.

In order to configure a site, one or more configuration files are edited and the YAIM script is then executed. Since YAIM is mainly bash, all the configuration files have to follow the bash syntax. For example, no space between the equal sign and the key-value variables are allowed.

WRONG :

SITE_NAME = NorthPole
CORRECT:
SITE_NAME=NorthPole
A good syntax test for the site-info.def is to source it:
source ./site-info.def
and looks for errors. The configuration procedure is described in the following sections.

Modular structure

YAIM is distributed in several rpms. The glite-yaim-core contains common functions and definitions, while the other packages like glite-yaim-clients implement the functionality to configure specific node types. The appropriate yaim package will be installed with the service metapackage. The available yaim modules are:

  • glite-yaim-core (it contains the remaining components that do not appear in the list)
  • glite-yaim-clients (UI, WN and VOBOX)
  • glite-yaim-wms (new WMS)
  • glite-yaim-lb (new LB)
  • glite-yaim-fts
  • glite-yaim-dpm
  • glite-yaim-lfc
  • glite-yaim-dcache
  • glite-yaim-myproxy

The configuration variables

Configuration files should be stored in a directory structure. All the involved files should be under the same folder, in a safer place which is not world readable. This folder should contain:

  • site-info.def: It contains a list of configuration variables in the format of key-value pairs. It's a mandatory file and it's a parameter passed to the yaim command.

Optionally, the configuration folder can contain:

  • services: it contains a file per node type with the format glite-node-type. The file contains a list of configuration variables specific to that node type. In the future, each yaim module distributes an example file in /opt/glite/yaim/examples/siteinfo/services/glite-node-type. It's not a mandatory folder.
  • vo.d: it contains a file per VO with the format vo_name. It's not a mandatory folder.
  • nodes: it contains a file per host with the format hostname.domain_name. The file contains host specific variables that are different from one host to another in a certain site. It's not a mandatory folder.

The optional folders are created to allow system administrators to organise their configurations in a more structured way. However, it's still possible to use only the site-info.def file and place all the necessary variables for one site there. In case the optional folders are used, this is the sourcing flow (where siteinfo refers to the path to the configuration folder):

  • siteinfo/site-info.def
  • siteinfo/services/*
  • siteinfo/nodes/*
  • siteinfo/vo.d/*

YAIM distributes an example of site-info.def and services/ under /opt/glite/yaim/examples/. In case the system administrator is interested in using these files, it should move them to a safer location.

YAIM also distributes an example of users.conf and groups.conf files under /opt/glite/yaim/examples/. These files can be placed at any location since their path is specified in the variables USERS_CONF and GROUPS_CONF in site-info.def.

site-info.def

IMPORTANT: YAIM 3.1.1 is backwards compatible with previous YAIM releases because it can use the same site-info.def file

This is the main configuration file of YAIM. It is installed by the glite-yaim-core rpm and it's located under /opt/glite/yaim/examples/siteinfo/site-info.def. In future YAIM releases, each yaim module will distribute a file containing the variables relevant to the associated node type. This file will be under /opt/glite/yaim/examples/siteinfo/services/glite-node-type. In that case, site-info.def can be kept to a minimum containing only the common variables to all node types. In the current YAIM version, the example site-info.def file in /opt/glite/yaim/examples/siteinfo/site-info.def contains a full list of all the variables used to configure any service.

site-info.def contains different configuration variables to be defined. Some of them are compulsory and no default value is assigned to them by the functions. Some of them can be left undefined and the functions will use the default values. In the following table all the variables used by this version of YAIM are listed: ( C = compulsory (if you going to configure that type of node) , O = optional )

(Click on the column name to sort on a column basis)

Variable name Type Description YAIM version >=
APEL_DB_PASSWORD C Database password for APEL. 3.0.1-0
BATCH_BIN_DIR C The path of the lrms commands, e.g. /usr/pbs/bin. 3.0.1-0
BATCH_SPOOL_DIR C The batch system log directory. It will replaced by BATCH_LOG_DIR in 4.0.1-x release. 3.0.1-0
BATCH_VERSION C The version of the Local Resource Managment System, e.g. OpenPBS_2.3. 3.0.1-0
BATCH_SERVER C Set this to specify your torque server host. It obsoletes TORQUE_SERVER variable. 3.1.1-1
BDII_FCR O Set the URL of the Freedom of Choice for Rescources URL. 3.0.1-0
BDII_HOST C BDII hostname. 3.0.1-0
BDII_HTTP_URL C URL pointing to the BDII configuration file (bdii-update.conf). 3.0.1-0
BDII_REGIONS C List of node types publishing information to the BDII. For each item listed in the BDII_REGIONS variable you need to create a set of new variables as follows: 3.0.1-0
BDII_RESOURCE_TIMEOUT C The timeout value to be used with a resource BDII. It is the time the BDII will wait when running the GIP. 3.0.1-0
BDII_SITE_TIMEOUT C The time outvalue to be used with a site BDII. It is the time the BDII will wait when querying resource BDIIs or GRISs. 3.0.1-0
BDII_<NODE-TYPE>_URL C URL of the information producer (e.g. BDII_CE_URL="URL of the CE information producer", BDII_SE_URL="URL of the SE information producer"). 3.0.1-0
CA_REPOSITORY C The repository with Certification Authorities (use the one in the example). 3.0.1-0
CE_BATCH_SYS C Implementation of site batch system. Available values are ``torque'', ``lsf'', ``pbs'', ``condor'' etc. 3.0.1-0
CE_CPU_MODEL C Model of the CPU used by the WN (WN specification). This parameter is a string whose domain is not defined yet in the GLUE EGEE.Schema. The value used for Pentium III is "PIII". 3.0.1-0
CE_CPU_SPEED C Clock frequency in Mhz (WN specification). 3.0.1-0
CE_CPU_VENDOR C Vendor of the CPU. used by the WN (WN specification). This parameter is a string whose domain is not defined yet in the GLUE EGEE.Schema. The value used for Intel is "intel". 3.0.1-0
CE_HOST C Computing Element Hostname. 3.0.1-0
CE_INBOUNDIP C TRUE if inbound connectivity is enabled at your site, FALSE otherwise (WN specification). 3.0.1-0
CE_MINPHYSMEM C RAM size (Mbytes) (per CPU) 3.0.1-0
CE_MINVIRTMEM C Virtual Memory size in (Mbytes) (per CPU) 3.0.1-0
CE_OS C Operating System name (WN specification) - see http://goc.grid.sinica.edu.tw/gocwiki/How_to_publish_the_OS_name. 3.0.1-0
CE_OS_RELEASE C CE Operating System release (WN specification) - see http://goc.grid.sinica.edu.tw/gocwiki/How_to_publish_the_OS_name. 3.0.1-0
CE_OS_VERSION C CE Operating System Version (WN specification) - see http://goc.grid.sinica.edu.tw/gocwiki/How_to_publish_the_OS_name. 3.0.1-0
CE_OS_ARCH C CE Operating System Architecture. Value of 'uname -m' (usually i386, i586, i686, or x86_64) executed on a typical WN (from the SubCluster represented by that CE) (3.0.1-0)
CE_OUTBOUNDIP C TRUE if outbound connectivity is enabled at your site, FALSE otherwise (WN specification). 3.0.1-0
CE_RUNTIMEENV C List of software tags supported by the site. The list can include VO-specific software tags. In order to assure backward compatibility it should include the entry 'LCG-2', the current middleware version and the list of previous middleware tags. 3.0.1-0
CE_SF00 C Performance index of your fabric in SpecFloat 2000 (WN specification). For some examples of Spec values see http://www.specbench.org/osg/cpu2000/results/cint2000.html. 3.0.1-0
CE_SI00 C Performance index of your fabric in SpecInt 2000 (WN specification). For some examples of Spec values see http://www.specbench.org/osg/cpu2000/results/cint2000.html. 3.0.1-0
CE_SMPSIZE C Number of cpus in an SMP box (WN specification). 3.0.1-0
CLASSIC_HOST C The name of your SE_classic host. 3.0.1-0
CLASSIC_STORAGE_DIR C The root storage directory on CLASSIC_HOST. 3.0.1-0
CRON_DIR C Yaim writes all cron jobs to this directory. Change it if you want to turn off Yaim's management of cron. 3.0.1-0
DCACHE_ADMIN C Host name of the server node which manages the pool of nodes. 3.0.1-0
DCACHE_POOLS C List of pool nodes managed by the DCACHE_ADMIN server node. 3.0.1-0
DCACHE_PORT_RANGE C dCache Port Range. This variable is optional and the default value is "20000,25000". 3.0.1-0
DCACHE_DOOR_SRM O Set up srm server on dCache pool nodes (ex door_node1[:port] door_node2[:port]) 3.0.1-0
DCACHE_DOOR_GSIFTP O Set up srm gsiftp on dCache pool nodes (ex door_node1[:port] door_node2[:port]) 3.0.1-0
DCACHE_DOOR_GSIDCAP O Set up gsidcap server on dCache pool nodes (ex door_node1[:port] door_node2[:port]) 3.0.1-0
DCACHE_DOOR_DCAP O Set up dcap server on dCache pool nodes (ex door_node1[:port] door_node2[:port]) 3.0.1-0
DCACHE_DOOR_XROOTD O Set up xrootd server on dCache pool nodes (ex door_node1[:port] door_node2[:port]) 3.0.1-0
DCACHE_DOOR_LDAP O Set up ldap server on dCache admin_node (ex door_node1[:port] door_node2[:port]) 3.0.1-0
DCACHE_PNFS_SERVER O For medium size sites where dCache can be overloaded with the burden of running name servers and the other admin services. If a reasonable sized dCache cluster is set up, it is recommended to dedicate a single host for PNFS, a single host for all central services and a single server for SRM. If this varuiable is not set, the PNFS services will default to running on the admin node, just as te SRM defaults to running on the admin node. 3.0.1-0
DPMFSIZE C The default disk space allocated per file (ex. 200 MB) on a DPM node. 3.0.1-0
DPM_DB_USER C The db user account for the DPM. 3.0.1-0
DPMPOOL C Name of the Pool (per default Permanent). 3.0.1-0
DPM_FILESYSTEMS C Space separated list of DPM pool hostname:/path entries". 3.0.1-0
DPM_DB_PASSWORD C Password of the db user account. 3.0.1-0
DPM_DB_HOST C Set this if your DPM server uses a db on a separate machine. Defaults to localhost. 3.0.1-0
DPM_HOST C Host name of the DPM host, used also as a default DPM for the lcg-stdout-mon. 3.0.1-0
DPM_DB C The dpm database name (default is dpm_db) 3.0.1-15
DPNS_DB C The cns database name (default is cns_db) 3.0.1-15
DPM_INFO_USER C The DPM database info user. 3.0.1-16
DPM_INFO_PASS C The DPM database info user's password. 3.0.1-16
E2EMONIT_LOCATION O e2emonit config download site. 3.0.1-0
E2EMONIT_SITEID O Used when generating e2emonit cron scripts. 3.0.1-0
EDG_WL_SCRATCH O Optional scratch directory for jobs. 3.0.1-0
FTS_HOST C The hostname of your FTS server - use this only if installinf an FTS server. 3.0.1-0
FTS_SERVER_URL C The URL of the File Transfer EGEE.Service server. 3.0.1-0
FUNCTIONS_DIR C The directory where YAIM will find its functions. 3.0.1-0
GIP_CACHE_TTL C How long information in the cache is valid. 3.0.1-0
GIP_FRESHNESS C If the information from the plug-ins is within this timelimit, the dynamics plug-ins will not be executed. 3.0.1-0
GIP_RESPONSE C How long the GIP will wait for dynamic plug-ins to run before reading the information from the cache. 3.0.1-0
GIP_TIMEOUT C The timeout value to be used with dynamic plug-ins. 3.0.1-0
GLOBUS_TCP_PORT_RANGE C Port range for Globus IO. It should be specified as "num1,num2". YAIM automatically handles the syntax of this variable depending on the version of VDT. If it's VDT 1.6 it leaves "num1,num2". If it's a version < VDT 1.6 it changes to "num1 num2". 3.0.1-0
GRIDICE_SERVER_HOST O GridIce server host name (usually run on the MON node). 3.0.1-0
GRID_TRUSTED_BROKERS C List of the DNs of the Resource Brokers host certificates which are trusted by the Proxy node. (ex: /O=Grid/O=CERN/OU=cern.ch/CN=host/testbed013.cern.ch) 3.0.1-0
GROUPS_CONF C Path to the groups.conf file which contains information on mapping VOMS groups and roles to local groups. An example of this configuration file is given in /opt/glite/yaim/examples/groups.conf. 3.0.1-0
GSSKLOG C yes or no, indicating whether the site provides an AFS authentication server which maps gsi credentials into Kerberos tokens. 3.0.1-0
GSSKLOG_SERVER C If GSSKLOG is yes, the name of the AFS authentication server host. 3.0.1-0
INSTALL_ROOT C Installation root - change if using the re-locatable distribution. 3.0.1-0
JAVA_LOCATION C Path to Java VM installation. It can be used in order to run a different version of java installed locally. 3.0.1-0
JOB_MANAGER C The name of the job manager used by the gatekeeper. 3.0.1-0
LCG_REPOSITORY C APT repository for the EGEE middleware. This is only for gLite 3.0 that uses apt. For gLite 3.1, please check this link. 3.0.1-0
LFC_CENTRAL C A list of VOs for which the LFC should be configured as a central catalogue. 3.0.1-0
LFC_DB C db name for LFC server. 3.0.1-0
LFC_DB_HOST C db hostname for the LFC server. Set to LFC server by default 3.0.1-0
LFC_DB_PASSWORD C db password for LFC user. 3.0.1-0
LFC_HOST C Set this if you are building an LFC_HOST, not if you're just using clients. 3.0.1-0
LFC_HOST_ALIAS C Alias of the LFC host 3.0.1-0
LFC_LOCAL C Normally the LFC will support all VOs in the VOS variable. If you want to limit this list, add the ones you need to LFC_LOCAL. 3.0.1-0
    --- For each item listed in the VOS variable you need to create a set of new variables as follows: 3.0.1-0
VO_<VO-NAME>_DEFAULT_SE C Default SE used by the VO. WARNING: VO-NAME must be in capital cases. 3.0.1-0
VO_<VO-NAME>_STORAGE_DIR C Path to the storage area for the VO on an SE_classic. WARNING: VO-NAME must be in capital cases. 3.0.1-0
VO_<VO-NAME>_SW_DIR C Area on the WN for the installation of the experiment software. If on the WNs a predefined shared area has been mounted where VO managers can pre-install software, then these variable should point to this area. If instead there is not a shared area and each job must install the software, then this variables should contain a dot ( . ). Anyway the mounting of shared areas, as well as the local installation of VO software is not managed by yaim and should be handled locally by Site Administrators. WARNING: VO_NAME must be in capital cases. 3.0.1-0
VO_<VO-NAME>_VOMSES C List of entries for the vomses files for this VO. Multiple values can be given if enclosed in single quotes. 3.0.1-0
VO_<VO-NAME>_VOMS_EXTRA_MAPS C To add any further arbitrary maps you need in edg-mkgridmap.conf. 3.0.1-0
VO_<VO-NAME>_VOMS_POOL_PATH C If necessary, append this to the VOMS_SERVER URL for the pool account list. 3.0.1-0
VO_<VO-NAME>_VOMS_SERVERS C A list of VOMS servers for the VO. 3.0.1-0
VO_<VO-NAME>_RBS C A list of RBs which support this VO. 3.0.1-0
    --- End of VOs variable listing.
MON_HOST C MON Box hostname. 3.0.1-0
MYSQL_PASSWORD C The mysql root password. 3.0.1-0
MY_DOMAIN C The site's domain name. 3.0.1-0
OUTPUT_STORAGE C Default Output directory for the jobs. 3.0.1-0
PX_HOST C PX hostname. 3.0.1-0
QUEUES C The name of the queues for the CE. These are by default set as the VO names. 3.0.1-0
<QUEUE-NAME>_GROUP_ENABLE C Space separated list of VO names and VOMS FQANs which are allowed to access the queue. It will be translated to pbs's group_enable parameter. 3.0.1-0
RB_HOST C Resource Broker Hostname. 3.0.1-0
REG_HOST C RGMA Registry hostname. 3.0.1-0
REPOSITORY_TYPE C It can only be apt. Moreover, this is only valid for gLite 3.0. For gLite 3.1 please visit this link 3.0.1-0
RESET_DCACHE_CONFIGURATION O Set this to yes if you want YAIM to configure dCache for you - if unset (or 'no') yaim will only configure the grid front-end to dCache. 3.0.1-0
RESET_DCACHE_PNFS O yes or no DO NOT set these values to yes on existing production services, dCache internal databases will be deleted. 3.0.1-0
RESET_DCACHE_RDBMS O yes or no DO NOT set these values to yes on existing production services, dCache internal databases will be deleted. 3.0.1-0
RFIO_PORT_RANGE O Optional variable for the port range with default value "20000,25000". 3.0.1-0
SE_ARCH C =defaults to multidisk - "disk, tape, multidisk, other" - populates GlueSEArchitecture. 3.0.1-0
SE_LIST C A list of hostnames of the SEs available at your site. 3.0.1-0
SITE_EMAIL C The site contact e-mail address as published by the information system. 3.0.1-0
SITE_SUPPORT_EMAIL C The site's user support e-mail address as published by the information system. 3.0.1-0
SITE_HTTP_PROXY O If you have a http proxy, set this variable (syntax is as that of the http_proxy environment variable) and it will be used in config_crl and used by the cron jobs (http_proxy) in order to reduce to load on the CA host. 3.0.1-0
SITE_LAT C Site latitude. 3.0.1-0
SITE_LOC C "City, Country". 3.0.1-0
SITE_LONG C Site longitude. 3.0.1-0
SITE_NAME C Your GIIS. 3.0.1-0
SITE_SUPPORT_SITE C Support entry point ; Unique Id for the site in the GOC DB and information system. 3.0.1-0
SITE_TIER C Site tier. 3.0.1-0
SITE_WEB C Site site. 3.0.1-0
USERS_CONF C Path to the file containing a list of Linux users (pool accounts) to be created. This file should be created by the Site Administrator, which contains a plain list of the users and IDs. An example of this configuration file is given in /opt/glite/yaim/examples/users.conf. 3.0.1-0
VOBOX_HOST C VOBOX hostname. 3.0.1-0
VOBOX_PORT C The port the VOBOX gsisshd listens on. 3.0.1-0
VOS C List of supported VOs. 3.0.1-0
VO_SW_DIR C Directory for installation of experiment software. 3.0.1-0
WMS_HOST C Hostname of the gLite WMS/LB server. 3.0.1-0
WN_LIST C Path to the list of Worker Nodes. The list of Worker Nodes is a file to be created by the Site Administrator, which contains a plain list of the batch nodes. An example of this configuration file is given in /opt/glite/yaim/examples/wn-list.conf. 3.0.1-0
YAIM_LOGGING_LEVEL C The logging level to print debugging information. Possible values are NONE, ABORT, ERROR, WARNING, INFO, DEBUG. 3.0.1-0

vo.d directory

The vo.d directory was created to make the configuration of the DNS-like VOs easier. Each file name in this directory has to be the lower-cased version of the VO name defined in site-info.def. The matching file should contain the definitions for that VO and will overwrite the ones which are defined in site-info.def. Again, bash syntax should be followed. The syntaxt is different compared to that of site-info.def. In vo.d files, the VO_(VONAME) prefix can be omitted. For example while in site-info.def:
VO_BIOMED_SW_DIR=$VO_SW_DIR/biomed
VO_BIOMED_DEFAULT_SE=$CLASSIC_HOST
VO_BIOMED_STORAGE_DIR=$CLASSIC_STORAGE_DIR/biomed
in vo.d/biomed file:
SW_DIR=$VO_SW_DIR/biomed
DEFAULT_SE=$CLASSIC_HOST
STORAGE_DIR=$CLASSIC_STORAGE_DIR/biomed

After the experience of AEGIS01-PHY-SCL, EGEE SEE ROC who have been deploying DNS-like VO names, the following recipe has been prepared with their contribution:

In you are deploying VO with DNS-like name (example: vo.test.domain.org), due to the limitations of the shell and other languages used (e.g. "." is often not a valid character for a variable name) and length limitations, you would first need to choose shorter name for such a VO (example: test). Now the new VO can be added in all conf files - the only issue is where to use full VO DNS-like name, and where to use short name.

  • site-info.def contains several relevant variables:
    • VOS should contain full VO names; example:
         VOS="atlas cms alice lhcb dteam ops vo.test.domain.org"
    • QUEUES should contain short VO names, in the same order as entered in $VOS (note that the default is QUEUES=${VOS}, which now cannot be used); example:
         QUEUES="atlas cms alice lhcb dteam ops test"
    • If you are configuring LFC and setting LFC_LOCAL and LFC_CENTRAL variables to select for which VOs your LFC will be local and for which central, those variables should contain full VO names; example:
         LFC_LOCAL="atlas cms alice lhcb dteam ops"
         LFC_CENTRAL="vo.test.domain.org"
    • <QUEUE-NAME>_GROUP_ENABLE variable should be named after short VO name (in capital letters), but should contain full VO name, as well as all roles and groups for that VO defined in groups.conf (except for the root); example
         TEST_GROUP_ENABLE="vo.test.domain.org
         /VO=vo.test.domain.org/GROUP=/vo.test.domain.org/ROLE=lcgadmin
         /VO=vo.test.domain.org/GROUP=/vo.test.domain.org/ROLE=production"
  • The preferred way to define other VO-related variables is to put them in vo.d directory in a file named after a full VO name (example: vo.d/vo.test.domain.org); even if your VOs don't have DNS-like names. All such files should contain at least the following variables: SW_DIR, DEFAULT_SE, STORAGE_DIR, VOMS_SERVERS, VOMSES.
  • If these variables are instead put in site-info.def directly, each variable name must be prepended with VO_<VO-NAME>_, where <VO-NAME> is full VO name in capital letters. In case of DNS-like VO names dots in VO name must be additionally changed to e.g. underscores. That can be quite tricky and is not suggested (nor YAIM is full-proofed to work correctly with this). For your reference, here is an example based on the VO name vo.test.domain.org (and its non-existing VOMS server voms.domain.org with the certificate DN /DC=ORG/O=DOMAIN/O=Hosts/CN=host/voms.domain.org):
VO_VO_TEST_DOMAIN_ORG_SW_DIR=$VO_SW_DIR/test
VO_VO_TEST_DOMAIN_ORG_DEFAULT_SE=$CLASSIC_HOST
VO_VO_TEST_DOMAIN_ORG_STORAGE_DIR=$CLASSIC_STORAGE_DIR/test
VO_VO_TEST_DOMAIN_ORG_VOMS_SERVERS="vomss://voms.domain.org:8443/voms/vo.test.domain.org"
VO_VO_TEST_DOMAIN_ORG_VOMSES="vo.test.domain.org voms.domain.org 15001 /DC=ORG/O=DOMAIN/O=Hosts/CN=host/voms.domain.org vo.test.domain.org"
  • If you choose to put all other VO-specific variables in the file in vo.d named after the full VO name (example: vo.d/vo.test.domain.org), it should contain at least the following variables: SW_DIR, DEFAULT_SE, STORAGE_DIR, VOMS_SERVERS, VOMSES. The example is similar as the one above (just without the prepended VO_<VO-NAME>_ part):
      SW_DIR=$VO_SW_DIR/test
      DEFAULT_SE=$CLASSIC_HOST
      STORAGE_DIR=$CLASSIC_STORAGE_DIR/test
      VOMS_SERVERS="vomss://voms.domain.org:8443/voms/vo.test.domain.org"
      VOMSES="vo.test.domain.org voms.domain.org 15001 /DC=ORG/O=DOMAIN/O=Hosts/CN=host/voms.domain.org vo.test.domain.org"
  • Such setup is tested to work; however, note that still some bugs exist and that some manual steps may be needed:Bug #27817.
  • Other VO-related variables (RBS, VOMS_POOL_PATH) can be also defined if needed.
  • users.conf file entries should be based on short VO name (example: test)
  • groups.conf should contain full VO name; example:
   "/VO=vo.test.domain.org/GROUP=/vo.test.domain.org/ROLE=lcgadmin":::sgm:
   "/VO=vo.test.domain.org/GROUP=/vo.test.domain.org/ROLE=production":::prd:
   "/VO=vo.test.domain.org/GROUP=/vo.test.domain.org"::::

services directory

The services directory is created to make easier the configuration of different node types present in one site. In future releases of yaim, each yaim module will distribute a file containing node type specific variables. The file name is glite-node-type and like site-info.def, it contains a list of key-value pairs. The site-info.def example file distributed by yaim core will then contain only the variables that are common to all the node types. It's up to the system administrator to decide whether to keep a modular configuration or keep using a single site-info.def file.

nodes directory

The node directory is created to make easier the configuration of variables that have different values depending on the host in the site. The file name is hostname.domain_name and like site-info.def, it contains a list of key-value pairs.

Example of two hosts which support different VOs:

lxb1430.cern.ch specific parameters
VOS=dteam 

# lxb1431.cern.ch specific parameters
VOS=&#8221;atlas alice&#8221;

users.conf

This file defines the unix users to be created on different service nodes (mainly on CE and WNs). The format is the following:
UID:LOGIN:GID1,GID2,...:GROUP1,GROUP2,...:VO:FLAG:
Each line defines one user with uid, login the user has primary group gid1 and additional secondary, terciary,... groups gid2, gid3,... which corresponds to the groups group1, group2, group3, is member of the VO vo , and has a special role flag, which will be associated with the corresponding line of groups.conf. Whitespaces and blank lines are not allowed. This file will be processed by one of the YAIM functions. The function will create the unix users if they don't exist already in the service node.

YAIM distributes an example users.conf file in /opt/glite/yaim/examples. Below you can find some examples as well:

25001:dteam001:2500:dteam:dteam::
25002:dteam002:2500:dteam:dteam::
25003:dteam003:2500:dteam:dteam::
25004:dteam004:2500:dteam:dteam::
...
25801:prddteam001:2580,2500:prddteam,dteam:dteam:prd:
25802:prddteam002:2580,2500:prddteam,dteam:dteam:prd:
...
25901:sgmdteam001:2590,2500:sgmdteam,dteam:dteam:sgm:
25902:sgmdteam002:2590,2500:sgmdteam,dteam:dteam:sgm:
Please, visit the How to switch to pool accounts for sgm/prd users for more details on sgm/prd pool accounts.

groups.conf

The groups.conf file has the following format:
FQAN:group name:gid:users.conf flag:vo
Thus users with VOMS credential fqan will be mapped to group name, gid, and associated with the users having the same flag in users.conf. That means, if flag is given, the group name and gid are taken from there and do not need to be specified. If the last - optional - field is defined, then the group will be treated as the member of the vo instead of the one which is determined from the fqan.

YAIM distributes an example groups.conf file in /opt/glite/yaim/examples. Below you can find some examples as well:

"/VO=alice/GROUP=/alice/ROLE=lcgadmin":::sgm:
"/VO=alice/GROUP=/alice/ROLE=production":::prd:
"/VO=alice/GROUP=/alice"::::
"/VO=atlas/GROUP=/atlas/ROLE=lcgadmin":::sgm:
"/VO=atlas/GROUP=/atlas/ROLE=production":::prd:
"/VO=atlas/GROUP=/atlas"::::
"/VO=cms/GROUP=/cms/ROLE=lcgadmin":::sgm:
"/VO=cms/GROUP=/cms/ROLE=production":::prd:

Running the configuration

The interface

YAIM comes with a script in /opt/glite/yaim/bin/yaim. This script should be used to perform the different configuration steps. For help just run:
Usage: ./yaim <action>  <parameters>
Actions:
       -i | --install     : Install one or several meta package.
                            Compulsory parameters: -s, -m

       -c | --configure   : Configure already installed services.
                            Compulsory parameters: -s, -n

       -r | --runfunction : Execute a configuration function.
                            Compulsory parameters: -s, -f
                            Optional parameters  : -n

       -v | --verify      : Go through on all the function and checks that
                            the necessary variables required for a given
                            configuration target are all defined in site-info.def.
                            Compulsory parameters: -s -n

       -h | --help        : This help

Specify only one action at a time !

Parameters:
       -s | --siteinfo:   : Location of the site-info.def file
       -m | --metapackage : Name of the metapackage(s) to install
       -n | --nodetype    : Name of the node type(s) to configure
       -f | --function    : Name of the functions(s) to execute

Examples:
       Installation:
         ./yaim -i -s /root/siteinfo/site-info.def -m glite-SE_dpm_mysql

       Configuration:
         ./yaim -c -s /root/siteinfo/site-info.def -n SE_dpm_mysql

       Running a function:
         ./yaim -r -s /root/siteinfo/site-info.def -n SE_dpm_mysql -f config_mkgridmap

       Verify your site-info.def:
         ./yaim -v -s /root/siteinfo/site-info.def -n SE_dpm_mysql
When configuring multiple node types, they have to be defined on the command line, for example:
         ./yaim -i -s /root/siteinfo/site-info.def -m glite-SE_dpm_mysql -m glite-BDII

Installing a node

IMPORTANT NOTE: YAIM doesn't support installation of gLite 3.1 nodes but just its configuration

For installation of gLite-3.1, please have a look at the rpm installation with YUM.

YAIM can be used to install your gLite 3.0 nodes. After defining the appropriate variables in your site-info.def, run the following command:

         ./yaim -i -s <location of site-info.def> -m <meta-package-name>

The table below lists the available meta-packages for SL3 operating system:

Node Type meta-package name meta-package description
WMS glite-WMS WMS node
LB glite-LB LB node
glite CE gliteCE The gLite Computing Element
FTS glite-FTS gLite File Transfer Server
FTA glite-FTA gLite File Transfer Agent
BDII glite-BDII BDII
LCG Computing Element (middleware only) lcg-CE It does not include any LRMS
LCG Computing Element (with Torque) lcg-CE_torque It includes the 'Torque' LRMS
LCG File Catalog (mysql) glite-LFC_mysql LCG File Catalog
LCG File Catalog (oracle) glite-LFC_oracle LCG File Catalog
MON-Box glite-MON RGMA-based monitoring system collector server
MON-Box glite-MON_e2emonit MON plus e2emonit
Proxy glite-PX Proxy Server
Resource Broker lcg-RB Resource Broker
Classic Storage Element glite-SE_classic Storage Element on local disk
dCache Storage Element glite-SE_dcache_pool Storage Element interfaced to dCache for a pool node
dCache Storage Element glite-SE_dcache_admin_postgres Storage Element interfaced to dCache for an admin node pnfs
DPM Storage Element (mysql) glite-SE_dpm_mysql Storage Element with SRM interface
DPM Storage Element (Oracle) glite-SE_dpm_oracle Storage Element with SRM interface
DPM disk glite-SE_dpm_disk Disk server for a DPM SE
Dependencies for the re-locatable distribution glite-TAR This package can be used to satisfy the dependencies of the relocatable distro
User Interface glite-UI User Interface
VO agent box glite-VOBOX Agents and Daemons
Worker Node (middleware only) glite-WN It does not include any LRMS
Worker Node (with Torque) glite-WN glite-torque-client-config The gLite Computing Element with torque client

Configuring a node

If the installation was successful one should run the configuration:
./yaim -c -s <location of site-info.def> -n <node-type-1> -n <node-type-2> ... 
Each node type is a configuration target and *if there is more than one installed and to be configure on a phisical node then the configuration should be run together and not separately*.

The available configuration targets are listed below:

Node Type Configuration target (node type) Description
gLite WMS glite-WMS WMS node
gLite LB glite-LB LB node
glite CE glite-CE The gLite Computing Element
FTS FTS gLite File Transfer Server
FTA FTA gLite File Transfer Agent
top level BDII BDII A top level BDII
site BDII BDII_site A site BDII
Computing Element (middleware only) CE It does not configure any LRMS
LCG File Catalog server LFC_mysql Set up a mysql based LFC server
MON-Box MON RGMA-based monitoring system collector server
e2emonit E2EMONIT RGMA-based monitoring system collector server
Proxy PX Proxy Server
Resource Broker RB Resource Broker
Classic Storage Element SE_classic Storage Element on local disk
Disk Pool Manager (mysql) SE_dpm_mysql Storage Element with SRM interface and mysql backend
Disk Pool Manager disk SE_dpm_disk Disk server for SE_dpm
dCache Storage Element SE_dcache Storage Element interfaced with dCache
Re-locatable distribution glite 3.0: TAR_UI or TAR_WN It can be used to set up a Worker Node or a UI
glite 3.1: WN_TAR or UI_TAR  
User Interface UI User Interface
VO agent box VOBOX Machine to run VO agents
Worker Node (middleware only) WN It does not configure any LRMS

Configuring a Batch system

YAIM only provides configuration for the following batch systems:

  • Torque
  • SGE (under development)

In order to configure the batch system, the following configuration targets can be chosen:

Node Type Configuration target (node type) Description
Computing Element with Torque CE_torque It configures also the 'Torque' LRMS client and server
Worker Node with Torque WN_torque It configures the Torque client in the WN
Torque server TORQUE_server It configures the 'Torque' LRMS server
Torque client TORQUE_client It configures the 'Torque' LRMS client

Please, note that YAIM doesn't configure LSF batch system.

In case you want to install the TORQUE server together with the lcg CE, the following command must be used:

 ./yaim -c -s site-info.def -n CE_torque
Note that
-n CE -n TORQUE_server
is not correct since some TORQUE utils that are needed won't be installed.

On the other hand, in the case of TORQUE clients in the WN, both possibilities are in theory possible for 3.0 WN:

yaim -c -s site-info.def -n WN_torque
yaim -c -s site-info.def -n WN -n TORQUE_client

However, please check the Known issues section because there is a bug in this particular case.

For 3.1 WN, the configuration command is the following:

yaim -c -s site-info.def -n WN -n TORQUE_client

Partial configuration

It is possible to run only a configuration function. See the following example:
         ./yaim -r -s /root/siteinfo/site-info.def -n SE_dpm_mysql -f config_mkgridmap

YAIM log file

The output of a configuration is stored in /opt/glite/yaim/log/yaimlog. The amount of information that appears during the configuration can be selected defining the YAIM_LOGGING_LEVEL variable in site-info.def file. Possible values are:

  • NONE
  • ABORT
  • ERROR
  • WARNING
  • INFO
  • DEBUG

The default value is INFO. The logfile contains the timestamp, the functions that are executed and the output of the functions.

YAIM tool

  • There is a tool helping sysadmins to automatically create VO related YAIM configuration parameters. This is the YAIM tool. Site administrators can use this utility to maintain configuration information for the VOs their site supports.

What is different from yaim 3.0.1 and yaim 3.1.0?

YAIM 3.1.1 is different from YAIM 3.0.1 because YAIM 3.1.1 is now structured in a modular way and can be packaged and released independently for each different node type (not all the node types have a module yet). This doesn't affect the system administrators who can still use the same site-info.def file they used for YAIM 3.0.1.

YAIM 3.1.0 and YAIM 3.1.1 are the same. The only difference is that YAIM 3.1.1 also allows to configure glite 3.0 node types. Moreover, the same site-info.def file that was used with YAIM 3.1.0 can still be used with YAIM 3.1.1.

TO TAKE INTO ACCOUNT: Since now 3.1 and 3.0 UI and WN configuration is put together, in order to differenciate which functions are only used in 3.0, we have used the suffix '_30'. In functions/ you may find config_function_name_30 and in node-info.d/ you may find glite-node-type_30. This means there are differences in the configuration between 3.0 and 3.1. In case you are using 3.0 local functions overriding the existing onces, check whether you may have or not to add the '_30' suffix.

Known issues

  • #28789: Directory /tmp is world readable/writable
  • #27845 Due to this bug, the WN with Torque should be installed:
    • First calling the target name TORQUE_client and them calling glite-WN
    • or if they are configured in one go with the same yaim command, in the following order: -n glite-WN -n TORQUE_client
  • #27761 In all Computing Elements, the configuration does not release the shell at the end. Making ctrl+c to release it also kills the gatekeeper (bug #27761). The script has to be restarted, easy workarounds are "service gLite restart" for the gLite-CE and /etc/init.d/globus-gatekeeper restart on the lcg-CE.
  • #25567: In some of the nodes, the configuration does not release the shell either at the end but ctrl+c can be used without any problem.
  • Remember to stop using TORQUE_SERVER variable and use instead BATCH_SERVER variable.

The following list of known issues is inherited from YAIM 3.01 and YAIM 3.1.0:

  • The site BDII is not configured on the lcg-CE automatically any more. You have to set the configuration target explicitely, i.e. for example:
       configure_node site-info.def CE_torque BDII_site
       
  • There is a known installation conflict between the 'torque-clients' rpm and the 'postfix' mail server (Savannah. bug #5509). In order to workaround the problem you can either uninstall postfix or remove the file /usr/share/man/man8/qmgr.8.gz from the target node.
  • To configure correctly the glite-CE the following parameters MUST be set as follows:
JOB_MANAGER=pbs
CE_BATCH_SYS=pbs

For furher reading

-- MariaALANDESPRADILLO - 27 Jun 2007

Edit | Attach | Watch | Print version | History: r24 | r22 < r21 < r20 < r19 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r20 - 2007-09-14 - MariaALANDESPRADILLO
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback