SITECONF Repository

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

The JobConfig part of SITECONF needs to be accessible to jobs running at the site. CMS sites have two options for this:

1. they can maintain a "local" copy of the relevant SITECONF section for the site/subsite served on the worker node (or a network filesystem mounted on the worker nodes) at /etc/cvmfs/SITECONF so Singularity binds it in. (The directory names between "SITECONF" and "JobConfig" will be used to determine the site and subsite names, i.e. do not extend the path with additional subdirectories.) If the SITECONF area of the site/subsite is not at ${CMS_PATH}/SITECONF/local then the environmental variable SITECONFIG_PATH should be set to point to the SITECONF area of the site/subsite, i.e. so that ${SITECONFIG_PATH}/JobConfig/site-local-config.xml is correct. (Most installations use ${CMS_PATH}/SITECONF/<sitename>/... and make ${CMS_PATH}/SITECONF/local a link to it.) Relevant files may involve storage.json files from other sites in case one of the site/subsite data-access or stage-out entries point to another site! Those files need to be at .../SITECONF/<other site>/storage.json otherwise fallback data access and stage out will fail.
2. they can use the CERN Virtual Machine FileSystem, CVMFS, of CMS. The CMS CVMFS provides an up-to-date copy of the SITECONF areas of all sites. Site administrators configure the CVMFS mount at their site such that /cvmfs/cms.cern.ch/SITECONF/local points to their site entry. This is done by setting CMS_LOCAL_SITE in /etc/cvmfs/default.local . The CMS CVMFS support team (Bockjoo Kim) keeps the CVMFS copy synchronized with the repository via a cron job that runs every few hours.

In case multiple sites or subsites are served on the same worker node, the environmental variable SITECONFIG_PATH can be used to point to the SITECONF area for each job. The variable should be set in the local batch system, for instance differently for tier-1--tier-2 jobs, and will take take precedence over ${CMS_PATH}/SITECONF/local.

GitLab Organization

The SITECONF repository is currently maintained in CERN's GitLab. GitLab is an enhanced Git-based repository service. The repository is web accessible to all CMS members:

• SITECONF GitLab
• GitLab Help
• Git to GitLab migration information of CERN-IT

The SITECONF GitLab is setup as a group (SITECONF) with a project inside this group for each site (CMS sitename, i.e. TN_CO_Name). This is done to allow direct write access for site administrators to their and only their SITECONF section. Permissions of SITECONF are:

• owner of the GitLab group is cms-comp-ops-site-support-team e-group. This allows the site support team to create new/remove inactive sites and create/modify files to assist sites.
• developer of each GitLab project are the site administrators and executives in the corresponding cms-<facility>-admin and cms-<facility>-exec e-groups. For more information check the SITECONF permissions managment section.
• visibility of the GitLab projects is restricted to cms-authorized-users e-group. This allows anyone in CMS to check out the files of a site and "propose" a change (= create a merge request).

Instructions for site admins

GitLab provides access via SSH key. To get started, please to the CERN GitLab web page https://gitlab.cern.ch/ and use the standard CERN single sign on, SSO, mechanism to logon.

• click on "Profile Settings" in the toolbar on the left
• this changes the toolbar and provides an "SSH Keys" entry, click it
  • add your SSH key if you have one
    • your SSH key is normally stored in ~/.ssh/id_rsa.pub. If you have such a file, take the key from there via
    • cat ~/.ssh/id_rsa.pub
  • otherwise, create an SSH key with the ssh-keygen command and add it
  • detailed SSH key information is available at https://gitlab.cern.ch/help/ssh/README

Changes to the site admin or site executive list in SiteDB as well as registrations with GitLab take about 12 hours to propagate, i.e. for the account to get access priviledges!
After your SSH key is registered you can check out the config files of your site, (= project), update some files, and apply the changes:

1. clone the project of your site
   • $ git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<name_of_site>.git
2. update files using your prefered editor, for instance
   • $ vi <name_of_site>/JobConfig/site-local-config.xml
3. remember that any new files need to be added explicitly, i.e.
   • $ git add <name_of_site>/JobConfig/a_new_file.txt
4. commit the change, i.e.
   • $ git commit -m "<comment describing the update>"
5. and push the update to the master branch of SITECONF
   • $ git push

To check the update made it to the SITECONF repository, go to the project in GitLab, i.e. https://gitlab.cern.ch/siteconf/<site_name&gt;, or go to the SITECONF GitLab, click on "Projects" in the main window, and select your project. The project page lists the last commits.

• for a full list click on "Commits" in the tolbar on the left

To discard a previous commit click on the identifier of the commit, 8-digit hex number. This brings you to a page with details about the update. In the upper right there is a button "Revert" that undoes the update in the commit. Please be carefull and check your config files are as desired afterwards!

Instructions to add a project/site

Run the createSiteConfProject.py script. If you want to add the project manually, follow these instructions:

Go to - https://gitlab.cern.ch/projects/new:

• • Project owner: Groups-->siteconf
  • Project name: <sitename>
  • Import project from: <leave as is>
  • Description: <sitename> site configuration
  • Visibility Level: Private (Visibility is set at the group level to xxx for members of the egroup yyyy so CMS members have all read access.)
  • [Create Project]
• select "edit project" in the config pulldown hidden in the upper right corner of the web page:
  • Path: https://gitlab.cern.ch/SITECONF/<sitename&gt; (use proper upper/lower case string!)
  • [Rename Project]
• insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
  • git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<sitename>.git
  • cd <sitename>
  • mkdir JobConfig
  • touch JobConfig/site-local-config.xml
  • git add JobConfig/site-local-config.xml
  • git commit -a
  • git push -u origin master
• select "protected branches" in the config pulldown hidden in the upper right corner of the web page (https://gitlab.cern.ch/SITECONF/<SITE_NAME>/-/protected_branches) (current GiLab release "Settings" on the left menu bar, then "Repository", expand "Protected Branches", choose Branch "master", Allowed to merge "Developers+Maintainers", Allowed to push "Developers+Maintainers" push "Protect", the old "Master" access is now called "Maintainer" access):
  • check the "Developers can push" checkbox for the master branch
• select "services" in the config pulldown hidden in the upper right corner of the web page (current GitLab release "Settings" on the left menu bar, then "Integration"):
  • select "Emails on push"
    • check the "Active" checkbox
    • Recipients: cms-comp-ops-site-support-team@cernNOSPAMPLEASE.ch
    • [Save changes]
    • Multiple recipients can be specified, separated by whitespace. More info
    • uncheck "Jira Jira issue tracker".
• Add CMSSST user as Master to a new project:
  • Go to https://gitlab.cern.ch/SITECONF/<SITE_NAME>/-/project_members or find in "Project Information->Members"
  • Select members to invite: CMSSST
  • Choose a role permission: Master/Maintainer
  • Press button "Add to project"
  • In the current GitLab version cmssst inherits "Owner" privilege from SITECONF, i.e. doesn't allow (lower) Maintainer access to be granted

Instructions to delete a project/site

Go to - https://gitlab.cern.ch/SITECONF/<cms_site_name>/edit, scroll down and press button "Remove project"

SITECONF permissions managment

The access to each of the site's SITECONF project is done via LDAP using the facility mailing lists for executives and administrators in CERN e-groups. This means that users in the cms-<facility>-admin and cms-<facility>-exec can modify the https://gitlab.cern.ch/SITECONF/<site>/ project. It is not recommended to give users access manually using gitlab.

When a new site is created, the updateSiteConfMembers.py will fetch the new site's information from CRIC, create a new gitlab group for the members of each of e-group mailing lists, and give those groups access to the SITECONF project. This script runs automatically every 8 hours. In order for this to work, the site must be registered in CRIC, the SITECONF project must exist and the mailing lists must have the corresponding facility name in CERN e-groups.