OSG CE Configurator

%COMPLETE3%

Goals of this page:

Help admins to create Configurator scripts to set their site OSG-CE. The configurator is useful for reproducibility in case of server OS upgrades.

Contents

Customize this configurator script

Use Rocks (OSG Roll)
CE Server Name CE type
Batch
Use GUMS
Use SQUID
Use Glexec
Use SE
Show Storage
Show Network
Show Gip
Show Site Info

A fresh install script should look like this:

#!/bin/bash

touch /var/log/ce-install.log
yum install osg-ca-certs  >> /var/log/ce-install.log 2>&1
yum install osg-ca-certs-updater >> /var/log/ce-install.log 2>&1
yum install osg-ce-condor >> /var/log/ce-install.log 2>&1


#Make sure config templates exists
[ ! -f /etc/osg/config.d/01-squid.ini.template ]&&cp -p /etc/osg/config.d/01-squid.ini /etc/osg/config.d/01-squid.ini.template
[ -f /etc/osg/config.d/10-gateway.ini ]&&[ ! -f /etc/osg/config.d/10-gateway.ini.template ]&&cp -p /etc/osg/config.d/10-gateway.ini /etc/osg/config.d/10-gateway.ini.template
[ ! -f /etc/osg/config.d/10-misc.ini.template ]&&cp -p /etc/osg/config.d/10-misc.ini /etc/osg/config.d/10-misc.ini.template
[ ! -f /etc/osg/config.d/10-storage.ini.template ]&&cp -p /etc/osg/config.d/10-storage.ini /etc/osg/config.d/10-storage.ini.template
[ -f /etc/osg/config.d/20-condor.ini ]&&[ ! -f /etc/osg/config.d/20-condor.ini.template ]&&cp -p /etc/osg/config.d/20-condor.ini /etc/osg/config.d/20-condor.ini.template
[ ! -f /etc/osg/config.d/30-gip.ini.template ]&&cp -p /etc/osg/config.d/30-gip.ini /etc/osg/config.d/30-gip.ini.template
[ ! -f /etc/osg/config.d/40-network.ini.template ]&&cp -p /etc/osg/config.d/40-network.ini /etc/osg/config.d/40-network.ini.template
[ ! -f /etc/osg/config.d/40-siteinfo.ini.template ]&&cp -p /etc/osg/config.d/40-siteinfo.ini /etc/osg/config.d/40-siteinfo.ini.template

#In case OSG repo is disabled make sure we still update osg-ca-certs
[ ! -f /root/osg-ca-certs-updater.template ]&&cp -p /etc/cron.d/osg-ca-certs-updater /root/osg-ca-certs-updater.template
sed -i -e "s#osg-ca-certs-updater -a#osg-ca-certs-updater --enablerepo=osg -a#" /etc/cron.d/osg-ca-certs-updater

#Set Gridmap File Authentication
sed -i -e "s@^gumsclient -> good | bad@#gumsclient -> good | bad@" /etc/lcmaps.db
sed -i -e "s@^#gridmapfile -> good | bad@gridmapfile -> good | bad@" /etc/lcmaps.db

The CE-configurator script (rocks) is like this:

#!/bin/bash

#global attributes
rocks add attr OSG_SquidServer value='squid.server.name:3128'
rocks add attr OSG_GumsServer value='gums.server.name'
rocks add attr OSG_SEServer value='se.server.name'
rocks add attr OSG_GlobusTcpSourceRange value='20000,25000'
rocks add attr OSG_GlobusTcpPortRange value='20000,25000'


#host attributes
rocks add host attr login-0-0 OSG_CVMFS value=true
rocks add host attr login-0-0 OSG_CE value=condor

rocks add host attr login-0-0 OSG_CE_Gateway value='htcondor-ce'

#not yet implemented (next roll release)
#rocks add host attr login-0-0 OSG_CE_AppDir value='/cvmfs/oasis.opensciencegrid.org'
#instead AppDir is set to /sharesoft/osg/app by setting OSG_CE_Mount_ShareDir
rocks add host attr login-0-0 OSG_CE_Mount_ShareDir value='/sharesoft/osg'
rocks add host attr login-0-0 OSG_CE_DataDir value='/data/se/osg'
rocks add host attr login-0-0 OSG_WN_TmpDir value='/tmp'

rocks add host attr login-0-0 OSG_CE_gip_SubCluster1 value='my-subcluster-id-1'
rocks add host attr login-0-0 OSG_CE_gip_ClusterName1 value='my-unique-subcluster-name-1'
rocks add host attr login-0-0 OSG_CE_gip_NumberOfNodes1 value='10'
rocks add host attr login-0-0 OSG_CE_gip_mb_of_Ram1 value='48000'
rocks add host attr login-0-0 OSG_CE_gip_cpu_model1 value='Intel(R) Xeon(R) CPU   xxxx @ 3.66GHz'
rocks add host attr login-0-0 OSG_CE_gip_cpu_vendor1 value='GenuineIntel'
rocks add host attr login-0-0 OSG_CE_gip_cpu_speed1 value='3666'
rocks add host attr login-0-0 OSG_CE_gip_arch1 value='x86_64'
rocks add host attr login-0-0 OSG_CE_gip_CpusPerNode1 value='2'
rocks add host attr login-0-0 OSG_CE_gip_CoresPerNode1 value='24'
rocks add host attr login-0-0 OSG_CE_gip_inbound1 value='FALSE'
rocks add host attr login-0-0 OSG_CE_gip_outbound1 value='TRUE'
rocks add host attr login-0-0 OSG_CE_gip_allowed_vos1 value='cms, osg'
rocks add host attr login-0-0 OSG_CE_gip_max_wall_time1 value='2880'

rocks add host attr login-0-0 OSG_CE_siteinfo_OIM_name value='registered-oim-resource-name'
rocks add host attr login-0-0 OSG_CE_siteinfo_OIM_group value='registered-oim-resource-group'
rocks add host attr login-0-0 OSG_CE_siteinfo_sponsor value='http://institution.site/Policy.html'
rocks add host attr login-0-0 OSG_CE_siteinfo_contact value='Site admin Name'
rocks add host attr login-0-0 OSG_CE_siteinfo_email value='myemail@my.institution'
rocks add host attr login-0-0 OSG_CE_siteinfo_policy value='http://institution.site/Policy.html'

#rocks sync host osg CE  login-0-0

The CE-configurator script (non-rocks) is like this:

#!/bin/bash

#begin config /etc/osg/config.d/01-squid.ini
/bin/cp -f /etc/osg/config.d/01-squid.ini.template  /etc/osg/config.d/01-squid.ini
sed -i -e "s@enabled = True@enabled = False@"  /etc/osg/config.d/01-squid.ini
sed -i -e "s@location = @location = UNAVAILABLE@" /etc/osg/config.d/01-squid.ini
#end config /etc/osg/config.d/01-squid.ini

#begin config /etc/osg/config.d/10-gateway.ini
/bin/cp -f /etc/osg/config.d/10-gateway.ini.template /etc/osg/config.d/10-gateway.ini
#end config /etc/osg/config.d/10-gateway.ini

#begin config /etc/osg/config.d/10-misc.ini
/bin/cp -f /etc/osg/config.d/10-misc.ini.template /etc/osg/config.d/10-misc.ini
sed -i -e "s@authorization_method = DEFAULT@authorization_method = gridmap@" /etc/osg/config.d/10-misc.ini
#end config /etc/osg/config.d/10-misc.ini

#begin config /etc/osg/config.d/10-storage.ini
/bin/cp -f /etc/osg/config.d/10-storage.ini.template /etc/osg/config.d/10-storage.ini
#end config /etc/osg/config.d/10-storage.ini

#begin config /etc/osg/config.d/20-condor.ini
/bin/cp -f /etc/osg/config.d/20-condor.ini.template /etc/osg/config.d/20-condor.ini
sed -i -e "s@enabled = FALSE@enabled = TRUE@" /etc/osg/config.d/20-condor.ini
sed -i -e "s@job_contact = host.name/jobmanager-condor@job_contact = myce.server.name/jobmanager-condor@"  /etc/osg/config.d/20-condor.ini
sed -i -e "s@util_contact = host.name/jobmanager@util_contact = myce.server.name/jobmanager@"  /etc/osg/config.d/20-condor.ini
#end config /etc/osg/config.d/20-condor.ini

#begin config /etc/osg/config.d/30-gip.ini
/bin/cp -f /etc/osg/config.d/30-gip.ini.template /etc/osg/config.d/30-gip.ini
sed -i -e "s@batch = DEFAULT@batch = condor@" /etc/osg/config.d/30-gip.ini
echo "[Subcluster my-subcluster-id-1]" >> /etc/osg/config.d/30-gip.ini
echo "name = my-unique-subcluster-name-1" >> /etc/osg/config.d/30-gip.ini
echo "node_count = 10" >> /etc/osg/config.d/30-gip.ini
echo "ram_mb = 48000" >> /etc/osg/config.d/30-gip.ini
echo "cpu_model = Intel(R) Xeon(R) CPU   xxxx @ 3.66GHz" >> /etc/osg/config.d/30-gip.ini
echo "cpu_vendor = GenuineIntel" >> /etc/osg/config.d/30-gip.ini
echo "cpu_speed_mhz = 3666" >> /etc/osg/config.d/30-gip.ini
echo "cpu_platform = x86_64" >> /etc/osg/config.d/30-gip.ini
echo "cpus_per_node = 2" >> /etc/osg/config.d/30-gip.ini
echo "cores_per_node = 24" >> /etc/osg/config.d/30-gip.ini
echo "inbound_network = FALSE" >> /etc/osg/config.d/30-gip.ini
echo "outbound_network = TRUE" >> /etc/osg/config.d/30-gip.ini
echo "allowed_vos = cms, osg" >> /etc/osg/config.d/30-gip.ini
echo "max_wall_time = 2880" >> /etc/osg/config.d/30-gip.ini
echo " " >> /etc/osg/config.d/30-gip.ini
#end config /etc/osg/config.d/30

#begin config /etc/osg/config.d/40-network.ini
/bin/cp -f /etc/osg/config.d/40-network.ini.template /etc/osg/config.d/40-network.ini
sed -i -e "s@source_range = UNAVAILABLE@source_range = 20000,25000@" /etc/osg/config.d/40-network.ini
sed -i -e "s@port_range = UNAVAILABLE@port_range = 20000,25000@" /etc/osg/config.d/40-network.ini
sed -i -e "s@port_state_file = UNAVAILABLE@port_state_file = /var/tmp/globus-port-state.log@" /etc/osg/config.d/40-network.ini
#end config /etc/osg/config.d/40-network.ini

#begin config /etc/osg/config.d/40-siteinfo.ini
/bin/cp -f /etc/osg/config.d/40-siteinfo.ini.template /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@host_name = UNAVAILABLE@host_name = myce.server.name@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@resource = UNAVAILABLE@resource = registered-oim-resource-name@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@resource_group = UNAVAILABLE@resource_group = registered-oim-resource-group@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@sponsor = UNAVAILABLE@sponsor = uscms@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@site_policy = UNAVAILABLE@site_policy = http://institution.site/Policy.html@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@contact = UNAVAILABLE@contact = Site admin Name@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s/email = UNAVAILABLE/email = myemail@my.institution/" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@city = UNAVAILABLE@city = mycity@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@country = UNAVAILABLE@country = US@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@longitude = UNAVAILABLE@longitude = -45@" /etc/osg/config.d/40-siteinfo.ini
sed -i -e "s@latitude = UNAVAILABLE@latitude = 45@" /etc/osg/config.d/40-siteinfo.ini
#end config /etc/osg/config.d/40-siteinfo.ini

osg-configure -v
#if validation succeed
#osg-configure -c

What is next after Configuration

  • If you want your site participate to the global pool to receive cms jobs. Open a OSG ticket to add your site to the glideins factory. Or to modify if your are still using GRAM and want to move to HTCondor-CE.

References

-- JuanEduardoRamirezVargas - 2016-05-27

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2016-08-11 - JuanEduardoRamirezVargas
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback